2024-04-26T01:26:32.188 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T01:26:32.188 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_leonidus@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} --archive /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674252 --name leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi --verbose -- /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674252/orig.config.yaml 2024-04-26T01:26:32.257 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T01:26:32.357 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674252 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} email: null first_in_suite: false job_id: '7674252' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: afa1933b0fdbb6c99c947d1eda34d661d23cd327 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3448 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_afa1933b0fdbb6c99c947d1eda34d661d23cd327/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: afa1933b0fdbb6c99c947d1eda34d661d23cd327 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi001.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL/XujWyLEJYS5J7X/Ahu1+R7HnLZNdi66PqvjGrb/CAwXEveoYIH7+bUZ0tXfM1nuvas3hOteXcweIjHqHaF0o= smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHob0Q1kJ1EnXwQX7c0KByKXZ3Xp3JVnjW4xfoCD8dMl+E6hqfdSVb+3z19PG1N8Pnpd5gYugDeC9LI2gIh23yw= smithi173.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJrqVlp3tqSb7KpAqp/LmSgiVWz6fD0D4Q4fIE2p9KMXyg/HRx5o7ThyM0zPr1dJt24p5tLW9esQizIfb+ZPAJU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_01:02:28 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T01:26:32.357 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_afa1933b0fdbb6c99c947d1eda34d661d23cd327/qa; will attempt to use it 2024-04-26T01:26:32.358 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_afa1933b0fdbb6c99c947d1eda34d661d23cd327/qa/tasks 2024-04-26T01:26:32.358 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T01:26:32.359 INFO:teuthology.task.internal:Checking packages... 2024-04-26T01:26:32.384 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-26T01:26:32.384 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T01:26:32.384 INFO:teuthology.packaging:ref: None 2024-04-26T01:26:32.384 INFO:teuthology.packaging:tag: None 2024-04-26T01:26:32.384 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-26T01:26:32.384 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:26:32.384 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-26T01:26:32.630 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-26T01:26:32.632 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T01:26:32.643 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T01:26:32.643 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T01:26:32.669 INFO:teuthology.task.internal:Saving configuration 2024-04-26T01:26:32.691 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T01:26:32.699 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T01:26:32.723 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi001.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674252', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 01:21:23.509331', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL/XujWyLEJYS5J7X/Ahu1+R7HnLZNdi66PqvjGrb/CAwXEveoYIH7+bUZ0tXfM1nuvas3hOteXcweIjHqHaF0o='} 2024-04-26T01:26:32.741 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674252', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 01:21:23.511434', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHob0Q1kJ1EnXwQX7c0KByKXZ3Xp3JVnjW4xfoCD8dMl+E6hqfdSVb+3z19PG1N8Pnpd5gYugDeC9LI2gIh23yw='} 2024-04-26T01:26:32.762 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi173.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674252', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 01:21:23.510460', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJrqVlp3tqSb7KpAqp/LmSgiVWz6fD0D4Q4fIE2p9KMXyg/HRx5o7ThyM0zPr1dJt24p5tLW9esQizIfb+ZPAJU='} 2024-04-26T01:26:32.762 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T01:26:32.773 INFO:teuthology.task.internal:roles: ubuntu@smithi001.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-26T01:26:32.773 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-26T01:26:32.773 INFO:teuthology.task.internal:roles: ubuntu@smithi173.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-26T01:26:32.774 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T01:26:32.878 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff08d5c50d0>, signals=[15]) 2024-04-26T01:26:32.879 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T01:26:32.887 INFO:teuthology.task.internal:Opening connections... 2024-04-26T01:26:32.887 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi001.front.sepia.ceph.com 2024-04-26T01:26:32.889 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:26:32.968 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2024-04-26T01:26:32.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:26:33.041 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi173.front.sepia.ceph.com 2024-04-26T01:26:33.042 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:26:33.115 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T01:26:33.123 DEBUG:teuthology.orchestra.run.smithi001:> uname -m 2024-04-26T01:26:33.141 INFO:teuthology.orchestra.run.smithi001.stdout:x86_64 2024-04-26T01:26:33.142 DEBUG:teuthology.orchestra.run.smithi001:> cat /etc/os-release 2024-04-26T01:26:33.200 INFO:teuthology.orchestra.run.smithi001.stdout:NAME="CentOS Stream" 2024-04-26T01:26:33.200 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION="9" 2024-04-26T01:26:33.200 INFO:teuthology.orchestra.run.smithi001.stdout:ID="centos" 2024-04-26T01:26:33.200 INFO:teuthology.orchestra.run.smithi001.stdout:ID_LIKE="rhel fedora" 2024-04-26T01:26:33.200 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_ID="9" 2024-04-26T01:26:33.200 INFO:teuthology.orchestra.run.smithi001.stdout:PLATFORM_ID="platform:el9" 2024-04-26T01:26:33.200 INFO:teuthology.orchestra.run.smithi001.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T01:26:33.201 INFO:teuthology.orchestra.run.smithi001.stdout:ANSI_COLOR="0;31" 2024-04-26T01:26:33.201 INFO:teuthology.orchestra.run.smithi001.stdout:LOGO="fedora-logo-icon" 2024-04-26T01:26:33.201 INFO:teuthology.orchestra.run.smithi001.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T01:26:33.201 INFO:teuthology.orchestra.run.smithi001.stdout:HOME_URL="https://centos.org/" 2024-04-26T01:26:33.201 INFO:teuthology.orchestra.run.smithi001.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T01:26:33.201 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T01:26:33.201 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T01:26:33.202 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server 2024-04-26T01:26:33.222 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-04-26T01:26:33.240 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-04-26T01:26:33.241 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-04-26T01:26:33.298 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="CentOS Stream" 2024-04-26T01:26:33.299 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="9" 2024-04-26T01:26:33.299 INFO:teuthology.orchestra.run.smithi097.stdout:ID="centos" 2024-04-26T01:26:33.299 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="rhel fedora" 2024-04-26T01:26:33.299 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="9" 2024-04-26T01:26:33.299 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el9" 2024-04-26T01:26:33.299 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T01:26:33.299 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2024-04-26T01:26:33.299 INFO:teuthology.orchestra.run.smithi097.stdout:LOGO="fedora-logo-icon" 2024-04-26T01:26:33.299 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T01:26:33.300 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://centos.org/" 2024-04-26T01:26:33.300 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T01:26:33.300 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T01:26:33.300 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T01:26:33.300 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-04-26T01:26:33.322 DEBUG:teuthology.orchestra.run.smithi173:> uname -m 2024-04-26T01:26:33.340 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2024-04-26T01:26:33.340 DEBUG:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2024-04-26T01:26:33.398 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Stream" 2024-04-26T01:26:33.398 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="9" 2024-04-26T01:26:33.398 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos" 2024-04-26T01:26:33.399 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora" 2024-04-26T01:26:33.399 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="9" 2024-04-26T01:26:33.399 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el9" 2024-04-26T01:26:33.399 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T01:26:33.399 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2024-04-26T01:26:33.399 INFO:teuthology.orchestra.run.smithi173.stdout:LOGO="fedora-logo-icon" 2024-04-26T01:26:33.399 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T01:26:33.399 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://centos.org/" 2024-04-26T01:26:33.399 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T01:26:33.399 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T01:26:33.399 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T01:26:33.400 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2024-04-26T01:26:33.422 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T01:26:33.435 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T01:26:33.462 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T01:26:33.463 DEBUG:teuthology.orchestra.run.smithi001:> test '!' -e /home/ubuntu/cephtest 2024-04-26T01:26:33.465 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2024-04-26T01:26:33.467 DEBUG:teuthology.orchestra.run.smithi173:> test '!' -e /home/ubuntu/cephtest 2024-04-26T01:26:33.484 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T01:26:33.494 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T01:26:33.495 DEBUG:teuthology.orchestra.run.smithi001:> test -z $(ls -A /var/lib/ceph) 2024-04-26T01:26:33.522 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2024-04-26T01:26:33.525 DEBUG:teuthology.orchestra.run.smithi173:> test -z $(ls -A /var/lib/ceph) 2024-04-26T01:26:33.615 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T01:26:33.699 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T01:26:33.718 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-26T01:26:33.718 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-26T01:26:33.718 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-26T01:26:33.718 DEBUG:teuthology.orchestra.run.smithi001:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T01:26:33.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T01:26:33.736 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-26T01:26:34.167 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T01:26:34.167 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T01:26:34.167 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2024-04-26T01:26:34.185 INFO:teuthology.orchestra.run.smithi001.stdout:5.14.0-437.el9.x86_64 2024-04-26T01:26:34.185 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T01:26:34.186 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T01:26:34.186 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum upgrade -y linux-firmware 2024-04-26T01:26:36.456 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-26T01:26:38.540 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-04-26T01:26:42.833 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-26T01:26:44.281 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 16 kB 00:00 2024-04-26T01:26:46.191 INFO:teuthology.orchestra.run.smithi001.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-26T01:26:51.365 INFO:teuthology.orchestra.run.smithi001.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-04-26T01:26:53.413 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2024-04-26T01:26:53.414 INFO:teuthology.orchestra.run.smithi001.stdout:Nothing to do. 2024-04-26T01:26:53.415 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2024-04-26T01:26:53.519 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-26T01:26:53.520 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T01:26:53.668 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T01:26:53.669 DEBUG:teuthology.orchestra.run.smithi001:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T01:26:53.671 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi001.front.sepia.ceph.com to complete... 2024-04-26T01:27:07.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26 01:27:07 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T01:27:07.087 INFO:teuthology.orchestra.run.smithi001.stderr:FINISHED --2024-04-26 01:27:07-- 2024-04-26T01:27:07.087 INFO:teuthology.orchestra.run.smithi001.stderr:Total wall clock time: 13s 2024-04-26T01:27:07.087 INFO:teuthology.orchestra.run.smithi001.stderr:Downloaded: 1 files, 341M in 13s (26.3 MB/s) 2024-04-26T01:27:07.091 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-26T01:27:07.092 DEBUG:teuthology.orchestra.run.smithi001:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T01:27:09.274 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying... ######################################## 2024-04-26T01:27:09.382 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing... ######################################## 2024-04-26T01:27:09.499 INFO:teuthology.orchestra.run.smithi001.stdout:Updating / installing... 2024-04-26T01:27:17.847 INFO:teuthology.orchestra.run.smithi001.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T01:27:57.812 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi001.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T01:27:57.812 DEBUG:teuthology.orchestra.run.smithi001:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T01:27:57.891 INFO:teuthology.orchestra.run.smithi001.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T01:27:57.891 INFO:teuthology.orchestra.run.smithi001.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T01:27:57.891 INFO:teuthology.orchestra.run.smithi001.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T01:27:57.891 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T01:27:57.891 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T01:27:57.891 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T01:27:57.891 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T01:27:57.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T01:27:57.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T01:27:57.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T01:27:57.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T01:27:57.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T01:27:57.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T01:27:57.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T01:27:57.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T01:27:57.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T01:27:57.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T01:27:57.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T01:27:57.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T01:27:57.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T01:27:57.892 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T01:27:57.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T01:27:57.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T01:27:57.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T01:27:57.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T01:27:57.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T01:27:57.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T01:27:57.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T01:27:57.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T01:27:57.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T01:27:57.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T01:27:57.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T01:27:57.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T01:27:57.893 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T01:27:57.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T01:27:57.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T01:27:57.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T01:27:57.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T01:27:57.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T01:27:57.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T01:27:57.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T01:27:57.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T01:27:57.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T01:27:57.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T01:27:57.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T01:27:57.894 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T01:27:57.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T01:27:57.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T01:27:57.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T01:27:57.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T01:27:57.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T01:27:57.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T01:27:57.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T01:27:57.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T01:27:57.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T01:27:57.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T01:27:57.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T01:27:57.895 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T01:27:57.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T01:27:57.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T01:27:57.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T01:27:57.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T01:27:57.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T01:27:57.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T01:27:57.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T01:27:57.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T01:27:57.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T01:27:57.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T01:27:57.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T01:27:57.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T01:27:57.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T01:27:57.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T01:27:57.896 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T01:27:57.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T01:27:57.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T01:27:57.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T01:27:57.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T01:27:57.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T01:27:57.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T01:27:57.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T01:27:57.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T01:27:57.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T01:27:57.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T01:27:57.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T01:27:57.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T01:27:57.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T01:27:57.897 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T01:27:57.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T01:27:57.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T01:27:57.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T01:27:57.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T01:27:57.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T01:27:57.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T01:27:57.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T01:27:57.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T01:27:57.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T01:27:57.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T01:27:57.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T01:27:57.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T01:27:57.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T01:27:57.898 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T01:27:57.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T01:27:57.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T01:27:57.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T01:27:57.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T01:27:57.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T01:27:57.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T01:27:57.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T01:27:57.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T01:27:57.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T01:27:57.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T01:27:57.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T01:27:57.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T01:27:57.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T01:27:57.899 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T01:27:57.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T01:27:57.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T01:27:57.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T01:27:57.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T01:27:57.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T01:27:57.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T01:27:57.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T01:27:57.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T01:27:57.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T01:27:57.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T01:27:57.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T01:27:57.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T01:27:57.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T01:27:57.900 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T01:27:57.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T01:27:57.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T01:27:57.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T01:27:57.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T01:27:57.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T01:27:57.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T01:27:57.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T01:27:57.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T01:27:57.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T01:27:57.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T01:27:57.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T01:27:57.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T01:27:57.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T01:27:57.901 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T01:27:57.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T01:27:57.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T01:27:57.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T01:27:57.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T01:27:57.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T01:27:57.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T01:27:57.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T01:27:57.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T01:27:57.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T01:27:57.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T01:27:57.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T01:27:57.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T01:27:57.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T01:27:57.902 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T01:27:57.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T01:27:57.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T01:27:57.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T01:27:57.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T01:27:57.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T01:27:57.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T01:27:57.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T01:27:57.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T01:27:57.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T01:27:57.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T01:27:57.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T01:27:57.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T01:27:57.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T01:27:57.903 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T01:27:57.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T01:27:57.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T01:27:57.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T01:27:57.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T01:27:57.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T01:27:57.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T01:27:57.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T01:27:57.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T01:27:57.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T01:27:57.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T01:27:57.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T01:27:57.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T01:27:57.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T01:27:57.904 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T01:27:57.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T01:27:57.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T01:27:57.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T01:27:57.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T01:27:57.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T01:27:57.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T01:27:57.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T01:27:57.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T01:27:57.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T01:27:57.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T01:27:57.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T01:27:57.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T01:27:57.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T01:27:57.905 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T01:27:57.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T01:27:57.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T01:27:57.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T01:27:57.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T01:27:57.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T01:27:57.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T01:27:57.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T01:27:57.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T01:27:57.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T01:27:57.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T01:27:57.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T01:27:57.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T01:27:57.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T01:27:57.906 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T01:27:57.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T01:27:57.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T01:27:57.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T01:27:57.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T01:27:57.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T01:27:57.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T01:27:57.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T01:27:57.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T01:27:57.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T01:27:57.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T01:27:57.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T01:27:57.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T01:27:57.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T01:27:57.907 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T01:27:57.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T01:27:57.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T01:27:57.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T01:27:57.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T01:27:57.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T01:27:57.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T01:27:57.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T01:27:57.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T01:27:57.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T01:27:57.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T01:27:57.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T01:27:57.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T01:27:57.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T01:27:57.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T01:27:57.908 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T01:27:57.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T01:27:57.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T01:27:57.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T01:27:57.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T01:27:57.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T01:27:57.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T01:27:57.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T01:27:57.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T01:27:57.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T01:27:57.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T01:27:57.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T01:27:57.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T01:27:57.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T01:27:57.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T01:27:57.909 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T01:27:57.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T01:27:57.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T01:27:57.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T01:27:57.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T01:27:57.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T01:27:57.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T01:27:57.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T01:27:57.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T01:27:57.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T01:27:57.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T01:27:57.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T01:27:57.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T01:27:57.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T01:27:57.910 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T01:27:57.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T01:27:57.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T01:27:57.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T01:27:57.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T01:27:57.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T01:27:57.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T01:27:57.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T01:27:57.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T01:27:57.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T01:27:57.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T01:27:57.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T01:27:57.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T01:27:57.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T01:27:57.911 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T01:27:57.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T01:27:57.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T01:27:57.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T01:27:57.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T01:27:57.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T01:27:57.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T01:27:57.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T01:27:57.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T01:27:57.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T01:27:57.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T01:27:57.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T01:27:57.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T01:27:57.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T01:27:57.912 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T01:27:57.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T01:27:57.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T01:27:57.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T01:27:57.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T01:27:57.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T01:27:57.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T01:27:57.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T01:27:57.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T01:27:57.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T01:27:57.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T01:27:57.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T01:27:57.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T01:27:57.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T01:27:57.913 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T01:27:57.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T01:27:57.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T01:27:57.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T01:27:57.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T01:27:57.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T01:27:57.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T01:27:57.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T01:27:57.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T01:27:57.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T01:27:57.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T01:27:57.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T01:27:57.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T01:27:57.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T01:27:57.914 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T01:27:57.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T01:27:57.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T01:27:57.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T01:27:57.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T01:27:57.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T01:27:57.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T01:27:57.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T01:27:57.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T01:27:57.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T01:27:57.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T01:27:57.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T01:27:57.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T01:27:57.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T01:27:57.915 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T01:27:57.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T01:27:57.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T01:27:57.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T01:27:57.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T01:27:57.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T01:27:57.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T01:27:57.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T01:27:57.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T01:27:57.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T01:27:57.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T01:27:57.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T01:27:57.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T01:27:57.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T01:27:57.916 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T01:27:57.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T01:27:57.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T01:27:57.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T01:27:57.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T01:27:57.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T01:27:57.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T01:27:57.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T01:27:57.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T01:27:57.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T01:27:57.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T01:27:57.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T01:27:57.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T01:27:57.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T01:27:57.917 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T01:27:57.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T01:27:57.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T01:27:57.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T01:27:57.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T01:27:57.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T01:27:57.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T01:27:57.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T01:27:57.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T01:27:57.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T01:27:57.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T01:27:57.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T01:27:57.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T01:27:57.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T01:27:57.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T01:27:57.918 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T01:27:57.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T01:27:57.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T01:27:57.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T01:27:57.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T01:27:57.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T01:27:57.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T01:27:57.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T01:27:57.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T01:27:57.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T01:27:57.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T01:27:57.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T01:27:57.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T01:27:57.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T01:27:57.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T01:27:57.919 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T01:27:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T01:27:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T01:27:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T01:27:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T01:27:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T01:27:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T01:27:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T01:27:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T01:27:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T01:27:57.921 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T01:27:57.921 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T01:27:57.921 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T01:27:57.921 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T01:27:57.921 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T01:27:57.921 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T01:27:57.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T01:27:57.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T01:27:57.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T01:27:57.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T01:27:57.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T01:27:57.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T01:27:57.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T01:27:57.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T01:27:57.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T01:27:57.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T01:27:57.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T01:27:57.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T01:27:57.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T01:27:57.922 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T01:27:57.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T01:27:57.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T01:27:57.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T01:27:57.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T01:27:57.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T01:27:57.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T01:27:57.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T01:27:57.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T01:27:57.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T01:27:57.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T01:27:57.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T01:27:57.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T01:27:57.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T01:27:57.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T01:27:57.923 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T01:27:57.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T01:27:57.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T01:27:57.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T01:27:57.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T01:27:57.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T01:27:57.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T01:27:57.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T01:27:57.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T01:27:57.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T01:27:57.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T01:27:57.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T01:27:57.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T01:27:57.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T01:27:57.924 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T01:27:57.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T01:27:57.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T01:27:57.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T01:27:57.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T01:27:57.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T01:27:57.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T01:27:57.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T01:27:57.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T01:27:57.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T01:27:57.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T01:27:57.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T01:27:57.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T01:27:57.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T01:27:57.925 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T01:27:57.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T01:27:57.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T01:27:57.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T01:27:57.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T01:27:57.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T01:27:57.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T01:27:57.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T01:27:57.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T01:27:57.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T01:27:57.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T01:27:57.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T01:27:57.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T01:27:57.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T01:27:57.926 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T01:27:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T01:27:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T01:27:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T01:27:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T01:27:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T01:27:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T01:27:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T01:27:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T01:27:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T01:27:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T01:27:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T01:27:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T01:27:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T01:27:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T01:27:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T01:27:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T01:27:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T01:27:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T01:27:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T01:27:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T01:27:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T01:27:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T01:27:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T01:27:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T01:27:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T01:27:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T01:27:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T01:27:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T01:27:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T01:27:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T01:27:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T01:27:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T01:27:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T01:27:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T01:27:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T01:27:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T01:27:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T01:27:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T01:27:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T01:27:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T01:27:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T01:27:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T01:27:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T01:27:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T01:27:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T01:27:57.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T01:27:57.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T01:27:57.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T01:27:57.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T01:27:57.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T01:27:57.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T01:27:57.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T01:27:57.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T01:27:57.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T01:27:57.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T01:27:57.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T01:27:57.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T01:27:57.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T01:27:57.930 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T01:27:57.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T01:27:57.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T01:27:57.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T01:27:57.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T01:27:57.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T01:27:57.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T01:27:57.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T01:27:57.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T01:27:57.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T01:27:57.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T01:27:57.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T01:27:57.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T01:27:57.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T01:27:57.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T01:27:57.931 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T01:27:57.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T01:27:57.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T01:27:57.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T01:27:57.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T01:27:57.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T01:27:57.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T01:27:57.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T01:27:57.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T01:27:57.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T01:27:57.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T01:27:57.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T01:27:57.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T01:27:57.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T01:27:57.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T01:27:57.932 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T01:27:57.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T01:27:57.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T01:27:57.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T01:27:57.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T01:27:57.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T01:27:57.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T01:27:57.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T01:27:57.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T01:27:57.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T01:27:57.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T01:27:57.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T01:27:57.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T01:27:57.933 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T01:27:57.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T01:27:57.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T01:27:57.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T01:27:57.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T01:27:57.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T01:27:57.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T01:27:57.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T01:27:57.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T01:27:57.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T01:27:57.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T01:27:57.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T01:27:57.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T01:27:57.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T01:27:57.934 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T01:27:57.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T01:27:57.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T01:27:57.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T01:27:57.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T01:27:57.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T01:27:57.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T01:27:57.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T01:27:57.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T01:27:57.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T01:27:57.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T01:27:57.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T01:27:57.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T01:27:57.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T01:27:57.935 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T01:27:57.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T01:27:57.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T01:27:57.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T01:27:57.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T01:27:57.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T01:27:57.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T01:27:57.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T01:27:57.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T01:27:57.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T01:27:57.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T01:27:57.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T01:27:57.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T01:27:57.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T01:27:57.936 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T01:27:57.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T01:27:57.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T01:27:57.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T01:27:57.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T01:27:57.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T01:27:57.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T01:27:57.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T01:27:57.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T01:27:57.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T01:27:57.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T01:27:57.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T01:27:57.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T01:27:57.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T01:27:57.937 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T01:27:57.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T01:27:57.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T01:27:57.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T01:27:57.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T01:27:57.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T01:27:57.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T01:27:57.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T01:27:57.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T01:27:57.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T01:27:57.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T01:27:57.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T01:27:57.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T01:27:57.938 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T01:27:57.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T01:27:57.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T01:27:57.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T01:27:57.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T01:27:57.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T01:27:57.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T01:27:57.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T01:27:57.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T01:27:57.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T01:27:57.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T01:27:57.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T01:27:57.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T01:27:57.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T01:27:57.939 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T01:27:57.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T01:27:57.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T01:27:57.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T01:27:57.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T01:27:57.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T01:27:57.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T01:27:57.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T01:27:57.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T01:27:57.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T01:27:57.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T01:27:57.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T01:27:57.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T01:27:57.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T01:27:57.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T01:27:57.940 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T01:27:57.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T01:27:57.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T01:27:57.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T01:27:57.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T01:27:57.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T01:27:57.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T01:27:57.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T01:27:57.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T01:27:57.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T01:27:57.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T01:27:57.941 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T01:27:57.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T01:27:57.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T01:27:57.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T01:27:57.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T01:27:57.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T01:27:57.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T01:27:57.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T01:27:57.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T01:27:57.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T01:27:57.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T01:27:57.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T01:27:57.942 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T01:27:57.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T01:27:57.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T01:27:57.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T01:27:57.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T01:27:57.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T01:27:57.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T01:27:57.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T01:27:57.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T01:27:57.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T01:27:57.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T01:27:57.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T01:27:57.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T01:27:57.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T01:27:57.943 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T01:27:57.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T01:27:57.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T01:27:57.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T01:27:57.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T01:27:57.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T01:27:57.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T01:27:57.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T01:27:57.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T01:27:57.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T01:27:57.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T01:27:57.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T01:27:57.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T01:27:57.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T01:27:57.944 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T01:27:57.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T01:27:57.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T01:27:57.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T01:27:57.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T01:27:57.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T01:27:57.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T01:27:57.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T01:27:57.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T01:27:57.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T01:27:57.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T01:27:57.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T01:27:57.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T01:27:57.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T01:27:57.945 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T01:27:57.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T01:27:57.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T01:27:57.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T01:27:57.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T01:27:57.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T01:27:57.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T01:27:57.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T01:27:57.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T01:27:57.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T01:27:57.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T01:27:57.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T01:27:57.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T01:27:57.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T01:27:57.946 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T01:27:57.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T01:27:57.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T01:27:57.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T01:27:57.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T01:27:57.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T01:27:57.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T01:27:57.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T01:27:57.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T01:27:57.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T01:27:57.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T01:27:57.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T01:27:57.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T01:27:57.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T01:27:57.947 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T01:27:57.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T01:27:57.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T01:27:57.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T01:27:57.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T01:27:57.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T01:27:57.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T01:27:57.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T01:27:57.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T01:27:57.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T01:27:57.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T01:27:57.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T01:27:57.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T01:27:57.948 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T01:27:57.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T01:27:57.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T01:27:57.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T01:27:57.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T01:27:57.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T01:27:57.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T01:27:57.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T01:27:57.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T01:27:57.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T01:27:57.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T01:27:57.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T01:27:57.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T01:27:57.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T01:27:57.949 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T01:27:57.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T01:27:57.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T01:27:57.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T01:27:57.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T01:27:57.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T01:27:57.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T01:27:57.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T01:27:57.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T01:27:57.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T01:27:57.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T01:27:57.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T01:27:57.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T01:27:57.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T01:27:57.950 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T01:27:57.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T01:27:57.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T01:27:57.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T01:27:57.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T01:27:57.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T01:27:57.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T01:27:57.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T01:27:57.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T01:27:57.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T01:27:57.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T01:27:57.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T01:27:57.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T01:27:57.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T01:27:57.951 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T01:27:57.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T01:27:57.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T01:27:57.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T01:27:57.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T01:27:57.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T01:27:57.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T01:27:57.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T01:27:57.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T01:27:57.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T01:27:57.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T01:27:57.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T01:27:57.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T01:27:57.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T01:27:57.952 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T01:27:57.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T01:27:57.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T01:27:57.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T01:27:57.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T01:27:57.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T01:27:57.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T01:27:57.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T01:27:57.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T01:27:57.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T01:27:57.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T01:27:57.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T01:27:57.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T01:27:57.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T01:27:57.953 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T01:27:57.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T01:27:57.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T01:27:57.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T01:27:57.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T01:27:57.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T01:27:57.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T01:27:57.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T01:27:57.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T01:27:57.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T01:27:57.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T01:27:57.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T01:27:57.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T01:27:57.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T01:27:57.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T01:27:57.954 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T01:27:57.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T01:27:57.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T01:27:57.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T01:27:57.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T01:27:57.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T01:27:57.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T01:27:57.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T01:27:57.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T01:27:57.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T01:27:57.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T01:27:57.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T01:27:57.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T01:27:57.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T01:27:57.955 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T01:27:57.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T01:27:57.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T01:27:57.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T01:27:57.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T01:27:57.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T01:27:57.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T01:27:57.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T01:27:57.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T01:27:57.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T01:27:57.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T01:27:57.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T01:27:57.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T01:27:57.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T01:27:57.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T01:27:57.956 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T01:27:57.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T01:27:57.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T01:27:57.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T01:27:57.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T01:27:57.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T01:27:57.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T01:27:57.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T01:27:57.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T01:27:57.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T01:27:57.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T01:27:57.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T01:27:57.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T01:27:57.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T01:27:57.957 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T01:27:57.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T01:27:57.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T01:27:57.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T01:27:57.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T01:27:57.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T01:27:57.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T01:27:57.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T01:27:57.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T01:27:57.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T01:27:57.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T01:27:57.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T01:27:57.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T01:27:57.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T01:27:57.958 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T01:27:57.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T01:27:57.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T01:27:57.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T01:27:57.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T01:27:57.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T01:27:57.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T01:27:57.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T01:27:57.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T01:27:57.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T01:27:57.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T01:27:57.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T01:27:57.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T01:27:57.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T01:27:57.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T01:27:57.959 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T01:27:57.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T01:27:57.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T01:27:57.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T01:27:57.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T01:27:57.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T01:27:57.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T01:27:57.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T01:27:57.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T01:27:57.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T01:27:57.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T01:27:57.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T01:27:57.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T01:27:57.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T01:27:57.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T01:27:57.960 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T01:27:57.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T01:27:57.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T01:27:57.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T01:27:57.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T01:27:57.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T01:27:57.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T01:27:57.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T01:27:57.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T01:27:57.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T01:27:57.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T01:27:57.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T01:27:57.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T01:27:57.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T01:27:57.961 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T01:27:57.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T01:27:57.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T01:27:57.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T01:27:57.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T01:27:57.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T01:27:57.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T01:27:57.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T01:27:57.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T01:27:57.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T01:27:57.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T01:27:57.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T01:27:57.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T01:27:57.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T01:27:57.962 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T01:27:57.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T01:27:57.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T01:27:57.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T01:27:57.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T01:27:57.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T01:27:57.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T01:27:57.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T01:27:57.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T01:27:57.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T01:27:57.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T01:27:57.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T01:27:57.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T01:27:57.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T01:27:57.963 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T01:27:57.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T01:27:57.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T01:27:57.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T01:27:57.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T01:27:57.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T01:27:57.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T01:27:57.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T01:27:57.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T01:27:57.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T01:27:57.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T01:27:57.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T01:27:57.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T01:27:57.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T01:27:57.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T01:27:57.964 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T01:27:57.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T01:27:57.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T01:27:57.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T01:27:57.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T01:27:57.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T01:27:57.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T01:27:57.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T01:27:57.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T01:27:57.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T01:27:57.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T01:27:57.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T01:27:57.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T01:27:57.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T01:27:57.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T01:27:57.965 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T01:27:57.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T01:27:57.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T01:27:57.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T01:27:57.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T01:27:57.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T01:27:57.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T01:27:57.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T01:27:57.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T01:27:57.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T01:27:57.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T01:27:57.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T01:27:57.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T01:27:57.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T01:27:57.966 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T01:27:57.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T01:27:57.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T01:27:57.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T01:27:57.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T01:27:57.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T01:27:57.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T01:27:57.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T01:27:57.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T01:27:57.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T01:27:57.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T01:27:57.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T01:27:57.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T01:27:57.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T01:27:57.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T01:27:57.967 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T01:27:57.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T01:27:57.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T01:27:57.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T01:27:57.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T01:27:57.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T01:27:57.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T01:27:57.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T01:27:57.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T01:27:57.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T01:27:57.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T01:27:57.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T01:27:57.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T01:27:57.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T01:27:57.968 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T01:27:57.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T01:27:57.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T01:27:57.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T01:27:57.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T01:27:57.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T01:27:57.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T01:27:57.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T01:27:57.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T01:27:57.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T01:27:57.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T01:27:57.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T01:27:57.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T01:27:57.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T01:27:57.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T01:27:57.969 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T01:27:57.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T01:27:57.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T01:27:57.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T01:27:57.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T01:27:57.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T01:27:57.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T01:27:57.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T01:27:57.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T01:27:57.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T01:27:57.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T01:27:57.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T01:27:57.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T01:27:57.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T01:27:57.970 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T01:27:57.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T01:27:57.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T01:27:57.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T01:27:57.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T01:27:57.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T01:27:57.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T01:27:57.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T01:27:57.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T01:27:57.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T01:27:57.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T01:27:57.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T01:27:57.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T01:27:57.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T01:27:57.971 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T01:27:57.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T01:27:57.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T01:27:57.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T01:27:57.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T01:27:57.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T01:27:57.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T01:27:57.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T01:27:57.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T01:27:57.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T01:27:57.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T01:27:57.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T01:27:57.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T01:27:57.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T01:27:57.972 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T01:27:57.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T01:27:57.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T01:27:57.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T01:27:57.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T01:27:57.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T01:27:57.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T01:27:57.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T01:27:57.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T01:27:57.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T01:27:57.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T01:27:57.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T01:27:57.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T01:27:57.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T01:27:57.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T01:27:57.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T01:27:57.973 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T01:27:57.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T01:27:57.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T01:27:57.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T01:27:57.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T01:27:57.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T01:27:57.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T01:27:57.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T01:27:57.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T01:27:57.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T01:27:57.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T01:27:57.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T01:27:57.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T01:27:57.974 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T01:27:57.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T01:27:57.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T01:27:57.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T01:27:57.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T01:27:57.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T01:27:57.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T01:27:57.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T01:27:57.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T01:27:57.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T01:27:57.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T01:27:57.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T01:27:57.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T01:27:57.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T01:27:57.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T01:27:57.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T01:27:57.975 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T01:27:57.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T01:27:57.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T01:27:57.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T01:27:57.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T01:27:57.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T01:27:57.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T01:27:57.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T01:27:57.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T01:27:57.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T01:27:57.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T01:27:57.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T01:27:57.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T01:27:57.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T01:27:57.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T01:27:57.976 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T01:27:57.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T01:27:57.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T01:27:57.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T01:27:57.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T01:27:57.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T01:27:57.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T01:27:57.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T01:27:57.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T01:27:57.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T01:27:57.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T01:27:57.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T01:27:57.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T01:27:57.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T01:27:57.977 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T01:27:57.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T01:27:57.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T01:27:57.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T01:27:57.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T01:27:57.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T01:27:57.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T01:27:57.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T01:27:57.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T01:27:57.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T01:27:57.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T01:27:57.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T01:27:57.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T01:27:57.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T01:27:57.978 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T01:27:57.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T01:27:57.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T01:27:57.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T01:27:57.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T01:27:57.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T01:27:57.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T01:27:57.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T01:27:57.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T01:27:57.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T01:27:57.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T01:27:57.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T01:27:57.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T01:27:57.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T01:27:57.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T01:27:57.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T01:27:57.979 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T01:27:57.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T01:27:57.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T01:27:57.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T01:27:57.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T01:27:57.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T01:27:57.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T01:27:57.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T01:27:57.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T01:27:57.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T01:27:57.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T01:27:57.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T01:27:57.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T01:27:57.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T01:27:57.980 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T01:27:57.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T01:27:57.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T01:27:57.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T01:27:57.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T01:27:57.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T01:27:57.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T01:27:57.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T01:27:57.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T01:27:57.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T01:27:57.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T01:27:57.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T01:27:57.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T01:27:57.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T01:27:57.981 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T01:27:57.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T01:27:57.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T01:27:57.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T01:27:57.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T01:27:57.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T01:27:57.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T01:27:57.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T01:27:57.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T01:27:57.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T01:27:57.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T01:27:57.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T01:27:57.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T01:27:57.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T01:27:57.982 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T01:27:57.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T01:27:57.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T01:27:57.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T01:27:57.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T01:27:57.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T01:27:57.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T01:27:57.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T01:27:57.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T01:27:57.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T01:27:57.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T01:27:57.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T01:27:57.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T01:27:57.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T01:27:57.983 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T01:27:57.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T01:27:57.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T01:27:57.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T01:27:57.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T01:27:57.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T01:27:57.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T01:27:57.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T01:27:57.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T01:27:57.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T01:27:57.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T01:27:57.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T01:27:57.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T01:27:57.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T01:27:57.984 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T01:27:57.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T01:27:57.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T01:27:57.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T01:27:57.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T01:27:57.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T01:27:57.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T01:27:57.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T01:27:57.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T01:27:57.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T01:27:57.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T01:27:57.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T01:27:57.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T01:27:57.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T01:27:57.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T01:27:57.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T01:27:57.985 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T01:27:57.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T01:27:57.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T01:27:57.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T01:27:57.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T01:27:57.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T01:27:57.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T01:27:57.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T01:27:57.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T01:27:57.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T01:27:57.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T01:27:57.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T01:27:57.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T01:27:57.986 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T01:27:57.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T01:27:57.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T01:27:57.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T01:27:57.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T01:27:57.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T01:27:57.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T01:27:57.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T01:27:57.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T01:27:57.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T01:27:57.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T01:27:57.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T01:27:57.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T01:27:57.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T01:27:57.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T01:27:57.987 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T01:27:57.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T01:27:57.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T01:27:57.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T01:27:57.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T01:27:57.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T01:27:57.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T01:27:57.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T01:27:57.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T01:27:57.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T01:27:57.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T01:27:57.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T01:27:57.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T01:27:57.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T01:27:57.988 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T01:27:57.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T01:27:57.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T01:27:57.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T01:27:57.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T01:27:57.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T01:27:57.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T01:27:57.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T01:27:57.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T01:27:57.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T01:27:57.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T01:27:57.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T01:27:57.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T01:27:57.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T01:27:57.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T01:27:57.989 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T01:27:57.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T01:27:57.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T01:27:57.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T01:27:57.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T01:27:57.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T01:27:57.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T01:27:57.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T01:27:57.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T01:27:57.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T01:27:57.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T01:27:57.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T01:27:57.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T01:27:57.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T01:27:57.990 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T01:27:57.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T01:27:57.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T01:27:57.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T01:27:57.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T01:27:57.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T01:27:57.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T01:27:57.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T01:27:57.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T01:27:57.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T01:27:57.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T01:27:57.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T01:27:57.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T01:27:57.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T01:27:57.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T01:27:57.991 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T01:27:57.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T01:27:57.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T01:27:57.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T01:27:57.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T01:27:57.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T01:27:57.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T01:27:57.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T01:27:57.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T01:27:57.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T01:27:57.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T01:27:57.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T01:27:57.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T01:27:57.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T01:27:57.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T01:27:57.992 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T01:27:57.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T01:27:57.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T01:27:57.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T01:27:57.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T01:27:57.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T01:27:57.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T01:27:57.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T01:27:57.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T01:27:57.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T01:27:57.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T01:27:57.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T01:27:57.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T01:27:57.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T01:27:57.993 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T01:27:57.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T01:27:57.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T01:27:57.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T01:27:57.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T01:27:57.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T01:27:57.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T01:27:57.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T01:27:57.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T01:27:57.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T01:27:57.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T01:27:57.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T01:27:57.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T01:27:57.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T01:27:57.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T01:27:57.994 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T01:27:57.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T01:27:57.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T01:27:57.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T01:27:57.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T01:27:57.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T01:27:57.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T01:27:57.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T01:27:57.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T01:27:57.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T01:27:57.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T01:27:57.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T01:27:57.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T01:27:57.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T01:27:57.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T01:27:57.995 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T01:27:57.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T01:27:57.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T01:27:57.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T01:27:57.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T01:27:57.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T01:27:57.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T01:27:57.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T01:27:57.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T01:27:57.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T01:27:57.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T01:27:57.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T01:27:57.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T01:27:57.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T01:27:57.996 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T01:27:57.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T01:27:57.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T01:27:57.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T01:27:57.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T01:27:57.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T01:27:57.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T01:27:57.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T01:27:57.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T01:27:57.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T01:27:57.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T01:27:57.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T01:27:57.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T01:27:57.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T01:27:57.997 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T01:27:57.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T01:27:57.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T01:27:57.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T01:27:57.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T01:27:57.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T01:27:57.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T01:27:57.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T01:27:57.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T01:27:57.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T01:27:57.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T01:27:57.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T01:27:57.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T01:27:57.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T01:27:57.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T01:27:57.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T01:27:57.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T01:27:57.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T01:27:57.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T01:27:57.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T01:27:57.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T01:27:57.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T01:27:57.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T01:27:57.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T01:27:57.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T01:27:57.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T01:27:57.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T01:27:57.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T01:27:57.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T01:27:57.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T01:27:58.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T01:27:58.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T01:27:58.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T01:27:58.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T01:27:58.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T01:27:58.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T01:27:58.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T01:27:58.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T01:27:58.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T01:27:58.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T01:27:58.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T01:27:58.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T01:27:58.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T01:27:58.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T01:27:58.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T01:27:58.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T01:27:58.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T01:27:58.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T01:27:58.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T01:27:58.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T01:27:58.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T01:27:58.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T01:27:58.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T01:27:58.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T01:27:58.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T01:27:58.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T01:27:58.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T01:27:58.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T01:27:58.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T01:27:58.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T01:27:58.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T01:27:58.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T01:27:58.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T01:27:58.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T01:27:58.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T01:27:58.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T01:27:58.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T01:27:58.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T01:27:58.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T01:27:58.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T01:27:58.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T01:27:58.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T01:27:58.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T01:27:58.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T01:27:58.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T01:27:58.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T01:27:58.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T01:27:58.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T01:27:58.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T01:27:58.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T01:27:58.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T01:27:58.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T01:27:58.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T01:27:58.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T01:27:58.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T01:27:58.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T01:27:58.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T01:27:58.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T01:27:58.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T01:27:58.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T01:27:58.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T01:27:58.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T01:27:58.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T01:27:58.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T01:27:58.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T01:27:58.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T01:27:58.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T01:27:58.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T01:27:58.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T01:27:58.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T01:27:58.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T01:27:58.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T01:27:58.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T01:27:58.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T01:27:58.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T01:27:58.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T01:27:58.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T01:27:58.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T01:27:58.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T01:27:58.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T01:27:58.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T01:27:58.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T01:27:58.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T01:27:58.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T01:27:58.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T01:27:58.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T01:27:58.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T01:27:58.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T01:27:58.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T01:27:58.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T01:27:58.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T01:27:58.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T01:27:58.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T01:27:58.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T01:27:58.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T01:27:58.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T01:27:58.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T01:27:58.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T01:27:58.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T01:27:58.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T01:27:58.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T01:27:58.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T01:27:58.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T01:27:58.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T01:27:58.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T01:27:58.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T01:27:58.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T01:27:58.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T01:27:58.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T01:27:58.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T01:27:58.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T01:27:58.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T01:27:58.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T01:27:58.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T01:27:58.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T01:27:58.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T01:27:58.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T01:27:58.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T01:27:58.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T01:27:58.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T01:27:58.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T01:27:58.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T01:27:58.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T01:27:58.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T01:27:58.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T01:27:58.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T01:27:58.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T01:27:58.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T01:27:58.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T01:27:58.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T01:27:58.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T01:27:58.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T01:27:58.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T01:27:58.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T01:27:58.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T01:27:58.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T01:27:58.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T01:27:58.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T01:27:58.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T01:27:58.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T01:27:58.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T01:27:58.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T01:27:58.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T01:27:58.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T01:27:58.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T01:27:58.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T01:27:58.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T01:27:58.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T01:27:58.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T01:27:58.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T01:27:58.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T01:27:58.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T01:27:58.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T01:27:58.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T01:27:58.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T01:27:58.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T01:27:58.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T01:27:58.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T01:27:58.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T01:27:58.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T01:27:58.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T01:27:58.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T01:27:58.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T01:27:58.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T01:27:58.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T01:27:58.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T01:27:58.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T01:27:58.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T01:27:58.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T01:27:58.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T01:27:58.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T01:27:58.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T01:27:58.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T01:27:58.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T01:27:58.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T01:27:58.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T01:27:58.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T01:27:58.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T01:27:58.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T01:27:58.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T01:27:58.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T01:27:58.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T01:27:58.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T01:27:58.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T01:27:58.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T01:27:58.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T01:27:58.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T01:27:58.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T01:27:58.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T01:27:58.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T01:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T01:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T01:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T01:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T01:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T01:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T01:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T01:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T01:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T01:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T01:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T01:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T01:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T01:27:58.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T01:27:58.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T01:27:58.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T01:27:58.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T01:27:58.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T01:27:58.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T01:27:58.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T01:27:58.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T01:27:58.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T01:27:58.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T01:27:58.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T01:27:58.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T01:27:58.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T01:27:58.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T01:27:58.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T01:27:58.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T01:27:58.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T01:27:58.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T01:27:58.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T01:27:58.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T01:27:58.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T01:27:58.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T01:27:58.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T01:27:58.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T01:27:58.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T01:27:58.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T01:27:58.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T01:27:58.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T01:27:58.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T01:27:58.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T01:27:58.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T01:27:58.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T01:27:58.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T01:27:58.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T01:27:58.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T01:27:58.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T01:27:58.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T01:27:58.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T01:27:58.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T01:27:58.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T01:27:58.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T01:27:58.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T01:27:58.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T01:27:58.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T01:27:58.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T01:27:58.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T01:27:58.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T01:27:58.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T01:27:58.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T01:27:58.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T01:27:58.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T01:27:58.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T01:27:58.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T01:27:58.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T01:27:58.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T01:27:58.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T01:27:58.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T01:27:58.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T01:27:58.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T01:27:58.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T01:27:58.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T01:27:58.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T01:27:58.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T01:27:58.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T01:27:58.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T01:27:58.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T01:27:58.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T01:27:58.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T01:27:58.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T01:27:58.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T01:27:58.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T01:27:58.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T01:27:58.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T01:27:58.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T01:27:58.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T01:27:58.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T01:27:58.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T01:27:58.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T01:27:58.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T01:27:58.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T01:27:58.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T01:27:58.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T01:27:58.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T01:27:58.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T01:27:58.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T01:27:58.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T01:27:58.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T01:27:58.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T01:27:58.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T01:27:58.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T01:27:58.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T01:27:58.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T01:27:58.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T01:27:58.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T01:27:58.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T01:27:58.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T01:27:58.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T01:27:58.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T01:27:58.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T01:27:58.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T01:27:58.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T01:27:58.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T01:27:58.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T01:27:58.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T01:27:58.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T01:27:58.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T01:27:58.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T01:27:58.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T01:27:58.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T01:27:58.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T01:27:58.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T01:27:58.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T01:27:58.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T01:27:58.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T01:27:58.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T01:27:58.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T01:27:58.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T01:27:58.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T01:27:58.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T01:27:58.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T01:27:58.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T01:27:58.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T01:27:58.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T01:27:58.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T01:27:58.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T01:27:58.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T01:27:58.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T01:27:58.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T01:27:58.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T01:27:58.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T01:27:58.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T01:27:58.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T01:27:58.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T01:27:58.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T01:27:58.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T01:27:58.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T01:27:58.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T01:27:58.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T01:27:58.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T01:27:58.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T01:27:58.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T01:27:58.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T01:27:58.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T01:27:58.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T01:27:58.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T01:27:58.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T01:27:58.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T01:27:58.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T01:27:58.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T01:27:58.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T01:27:58.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T01:27:58.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T01:27:58.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T01:27:58.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T01:27:58.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T01:27:58.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T01:27:58.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T01:27:58.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T01:27:58.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T01:27:58.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T01:27:58.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T01:27:58.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T01:27:58.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T01:27:58.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T01:27:58.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T01:27:58.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T01:27:58.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T01:27:58.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T01:27:58.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T01:27:58.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T01:27:58.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T01:27:58.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T01:27:58.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T01:27:58.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T01:27:58.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T01:27:58.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T01:27:58.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T01:27:58.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T01:27:58.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T01:27:58.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T01:27:58.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T01:27:58.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T01:27:58.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T01:27:58.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T01:27:58.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T01:27:58.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T01:27:58.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T01:27:58.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T01:27:58.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T01:27:58.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T01:27:58.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T01:27:58.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T01:27:58.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T01:27:58.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T01:27:58.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T01:27:58.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T01:27:58.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T01:27:58.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T01:27:58.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T01:27:58.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T01:27:58.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T01:27:58.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T01:27:58.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T01:27:58.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T01:27:58.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T01:27:58.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T01:27:58.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T01:27:58.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T01:27:58.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T01:27:58.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T01:27:58.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T01:27:58.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T01:27:58.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T01:27:58.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T01:27:58.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T01:27:58.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T01:27:58.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T01:27:58.032 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T01:27:58.033 DEBUG:teuthology.orchestra.run.smithi001:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T01:27:58.058 INFO:teuthology.orchestra.run.smithi001.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T01:27:58.058 INFO:teuthology.orchestra.run.smithi001.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T01:27:58.058 INFO:teuthology.orchestra.run.smithi001.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T01:27:58.058 INFO:teuthology.orchestra.run.smithi001.stdout:fi 2024-04-26T01:27:58.058 INFO:teuthology.orchestra.run.smithi001.stdout:for file in vmlinuz System.map config; do 2024-04-26T01:27:58.058 INFO:teuthology.orchestra.run.smithi001.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T01:27:58.058 INFO:teuthology.orchestra.run.smithi001.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T01:27:58.058 INFO:teuthology.orchestra.run.smithi001.stdout: fi 2024-04-26T01:27:58.058 INFO:teuthology.orchestra.run.smithi001.stdout:done 2024-04-26T01:27:58.058 INFO:teuthology.orchestra.run.smithi001.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T01:27:58.059 INFO:teuthology.orchestra.run.smithi001.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T01:27:58.059 INFO:teuthology.orchestra.run.smithi001.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T01:27:58.059 INFO:teuthology.orchestra.run.smithi001.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T01:27:58.059 INFO:teuthology.orchestra.run.smithi001.stdout:fi 2024-04-26T01:27:58.059 INFO:teuthology.orchestra.run.smithi001.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T01:27:58.059 INFO:teuthology.orchestra.run.smithi001.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T01:27:58.059 INFO:teuthology.orchestra.run.smithi001.stdout:fi 2024-04-26T01:27:58.059 INFO:teuthology.orchestra.run.smithi001.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T01:27:58.059 INFO:teuthology.orchestra.run.smithi001.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T01:27:58.059 INFO:teuthology.orchestra.run.smithi001.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T01:27:58.059 INFO:teuthology.orchestra.run.smithi001.stdout:fi 2024-04-26T01:27:58.060 DEBUG:teuthology.orchestra.run.smithi001:> sudo rpm -qi grub2-tools 2024-04-26T01:27:58.133 INFO:teuthology.orchestra.run.smithi001.stdout:Name : grub2-tools 2024-04-26T01:27:58.133 INFO:teuthology.orchestra.run.smithi001.stdout:Epoch : 1 2024-04-26T01:27:58.133 INFO:teuthology.orchestra.run.smithi001.stdout:Version : 2.06 2024-04-26T01:27:58.133 INFO:teuthology.orchestra.run.smithi001.stdout:Release : 68.el9 2024-04-26T01:27:58.133 INFO:teuthology.orchestra.run.smithi001.stdout:Architecture: x86_64 2024-04-26T01:27:58.133 INFO:teuthology.orchestra.run.smithi001.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T01:27:58.133 INFO:teuthology.orchestra.run.smithi001.stdout:Group : Unspecified 2024-04-26T01:27:58.134 INFO:teuthology.orchestra.run.smithi001.stdout:Size : 8264254 2024-04-26T01:27:58.134 INFO:teuthology.orchestra.run.smithi001.stdout:License : GPLv3+ 2024-04-26T01:27:58.134 INFO:teuthology.orchestra.run.smithi001.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T01:27:58.134 INFO:teuthology.orchestra.run.smithi001.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T01:27:58.134 INFO:teuthology.orchestra.run.smithi001.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T01:27:58.134 INFO:teuthology.orchestra.run.smithi001.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T01:27:58.134 INFO:teuthology.orchestra.run.smithi001.stdout:Packager : builder@centos.org 2024-04-26T01:27:58.134 INFO:teuthology.orchestra.run.smithi001.stdout:Vendor : CentOS 2024-04-26T01:27:58.134 INFO:teuthology.orchestra.run.smithi001.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T01:27:58.134 INFO:teuthology.orchestra.run.smithi001.stdout:Summary : Support tools for GRUB. 2024-04-26T01:27:58.134 INFO:teuthology.orchestra.run.smithi001.stdout:Description : 2024-04-26T01:27:58.134 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:27:58.134 INFO:teuthology.orchestra.run.smithi001.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T01:27:58.135 INFO:teuthology.orchestra.run.smithi001.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T01:27:58.135 INFO:teuthology.orchestra.run.smithi001.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T01:27:58.135 INFO:teuthology.orchestra.run.smithi001.stdout:hardware devices. 2024-04-26T01:27:58.135 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:27:58.135 INFO:teuthology.orchestra.run.smithi001.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T01:27:58.136 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T01:27:58.136 INFO:teuthology.task.kernel:Updating grub on smithi001 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T01:27:58.136 DEBUG:teuthology.orchestra.run.smithi001:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T01:27:58.669 INFO:teuthology.orchestra.run.smithi001.stderr:Generating grub configuration file ... 2024-04-26T01:28:00.073 INFO:teuthology.orchestra.run.smithi001.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T01:28:00.106 INFO:teuthology.orchestra.run.smithi001.stderr:done 2024-04-26T01:28:00.108 DEBUG:teuthology.orchestra.run.smithi001:> mktemp 2024-04-26T01:28:00.151 INFO:teuthology.orchestra.run.smithi001.stdout:/tmp/tmp.pSmVFGRgWM 2024-04-26T01:28:00.151 DEBUG:teuthology.orchestra.run.smithi001:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.pSmVFGRgWM 2024-04-26T01:28:00.217 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0666 /tmp/tmp.pSmVFGRgWM 2024-04-26T01:28:00.362 DEBUG:teuthology.orchestra.remote:smithi001:/tmp/tmp.pSmVFGRgWM is 6KB 2024-04-26T01:28:00.412 DEBUG:teuthology.orchestra.run.smithi001:> rm -fr /tmp/tmp.pSmVFGRgWM 2024-04-26T01:28:00.427 DEBUG:teuthology.orchestra.run.smithi001:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T01:28:00.488 INFO:teuthology.orchestra.run.smithi001.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T01:28:00.489 INFO:teuthology.orchestra.run.smithi001.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T01:28:00.489 INFO:teuthology.orchestra.run.smithi001.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T01:28:00.489 INFO:teuthology.orchestra.run.smithi001.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T01:28:00.490 DEBUG:teuthology.orchestra.run.smithi001:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T01:28:00.669 DEBUG:teuthology.orchestra.run.smithi001:> sudo shutdown -r now 2024-04-26T01:28:30.672 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-26T01:28:30.673 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-04-26T01:28:30.674 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:29:30.677 DEBUG:teuthology.orchestra.remote:timed out 2024-04-26T01:29:39.678 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-04-26T01:29:39.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:29:42.972 DEBUG:teuthology.orchestra.run.smithi001:> true 2024-04-26T01:29:43.666 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-04-26T01:29:43.666 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T01:29:43.667 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2024-04-26T01:29:43.707 INFO:teuthology.orchestra.run.smithi001.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T01:29:43.707 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T01:29:43.707 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T01:29:43.707 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T01:29:44.711 DEBUG:teuthology.parallel:result is None 2024-04-26T01:29:44.711 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T01:29:44.721 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T01:29:44.721 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T01:29:44.730 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T01:29:44.732 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T01:29:44.781 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T01:29:44.792 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T01:29:44.815 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T01:29:44.815 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T01:29:44.821 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T01:29:44.823 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T01:29:44.927 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T01:29:44.938 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T01:29:44.938 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T01:29:44.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T01:29:44.955 DEBUG:teuthology.orchestra.run.smithi173:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T01:29:44.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T01:29:44.971 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T01:29:44.977 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T01:29:44.998 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T01:29:45.035 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T01:29:45.050 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T01:29:45.054 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T01:29:45.085 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T01:29:45.106 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T01:29:45.154 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T01:29:45.158 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T01:29:45.168 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T01:29:45.168 DEBUG:teuthology.orchestra.run.smithi001:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T01:29:45.172 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T01:29:45.174 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T01:29:45.230 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T01:29:45.241 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T01:29:45.241 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T01:29:45.245 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T01:29:45.247 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T01:29:45.279 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T01:29:45.404 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T01:29:45.482 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T01:29:45.554 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T01:29:45.639 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-04-26T01:29:45.640 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T01:29:45.726 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T01:29:45.771 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T01:29:45.823 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T01:29:45.910 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T01:29:45.952 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T01:29:45.953 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T01:29:46.019 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T01:29:46.064 DEBUG:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T01:29:46.118 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T01:29:46.200 DEBUG:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T01:29:46.244 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-26T01:29:46.244 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T01:29:46.309 DEBUG:teuthology.orchestra.run.smithi001:> sudo service rsyslog restart 2024-04-26T01:29:46.319 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2024-04-26T01:29:46.322 DEBUG:teuthology.orchestra.run.smithi173:> sudo service rsyslog restart 2024-04-26T01:29:46.359 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T01:29:46.397 INFO:teuthology.orchestra.run.smithi173.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T01:29:46.446 INFO:teuthology.orchestra.run.smithi001.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T01:29:46.851 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T01:29:46.861 INFO:teuthology.task.internal:Starting timer... 2024-04-26T01:29:46.861 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T01:29:46.874 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T01:29:46.884 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-26T01:29:46.885 DEBUG:teuthology.orchestra.run.smithi001:> sudo service auditd rotate 2024-04-26T01:29:47.001 INFO:teuthology.orchestra.run.smithi001.stdout:Rotating logs: 2024-04-26T01:29:47.005 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2024-04-26T01:29:47.074 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2024-04-26T01:29:47.076 DEBUG:teuthology.orchestra.run.smithi173:> sudo service auditd rotate 2024-04-26T01:29:47.154 INFO:teuthology.orchestra.run.smithi173.stdout:Rotating logs: 2024-04-26T01:29:47.156 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T01:29:47.156 DEBUG:teuthology.orchestra.run.smithi001:> /usr/sbin/getenforce 2024-04-26T01:29:47.218 INFO:teuthology.orchestra.run.smithi001.stdout:Permissive 2024-04-26T01:29:47.218 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2024-04-26T01:29:47.241 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2024-04-26T01:29:47.241 DEBUG:teuthology.orchestra.run.smithi173:> /usr/sbin/getenforce 2024-04-26T01:29:47.262 INFO:teuthology.orchestra.run.smithi173.stdout:Permissive 2024-04-26T01:29:47.262 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi001.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi097.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi173.front.sepia.ceph.com': 'permissive'} 2024-04-26T01:29:47.263 DEBUG:teuthology.orchestra.run.smithi001:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T01:29:47.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T01:29:47.338 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T01:29:47.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T01:29:47.365 DEBUG:teuthology.orchestra.run.smithi173:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T01:29:47.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T01:29:47.390 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T01:29:47.391 DEBUG:teuthology.orchestra.run.smithi001:> sudo /usr/sbin/setenforce permissive 2024-04-26T01:29:47.473 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2024-04-26T01:29:47.498 DEBUG:teuthology.orchestra.run.smithi173:> sudo /usr/sbin/setenforce permissive 2024-04-26T01:29:47.523 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T01:29:47.533 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T01:29:47.549 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T01:29:47.550 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi001.front.sepia.ceph.com,smithi097.front.sepia.ceph.com,smithi173.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T01:35:01.869 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi001.front.sepia.ceph.com'), Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi173.front.sepia.ceph.com')] 2024-04-26T01:35:01.870 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-04-26T01:35:01.871 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:35:01.984 DEBUG:teuthology.orchestra.run.smithi001:> true 2024-04-26T01:35:02.141 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-04-26T01:35:02.142 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-26T01:35:02.142 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:35:02.218 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-04-26T01:35:02.293 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-26T01:35:02.294 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-04-26T01:35:02.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:35:02.369 DEBUG:teuthology.orchestra.run.smithi173:> true 2024-04-26T01:35:02.449 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-04-26T01:35:02.449 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T01:35:02.461 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T01:35:02.461 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T01:35:02.462 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T01:35:02.470 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T01:35:02.470 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T01:35:02.472 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T01:35:02.473 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T01:35:02.501 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T01:35:02.516 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T01:35:02.519 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T01:35:02.533 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T01:35:02.540 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2024-04-26T01:35:02.551 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2024-04-26T01:35:02.557 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: ntpd: command not found 2024-04-26T01:35:02.563 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T01:35:02.564 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T01:35:02.567 INFO:teuthology.orchestra.run.smithi173.stdout:506 Cannot talk to daemon 2024-04-26T01:35:02.577 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T01:35:02.581 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T01:35:02.594 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T01:35:02.610 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T01:35:02.640 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-04-26T01:35:02.643 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T01:35:02.643 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-04-26T01:35:02.643 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:35:02.643 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:35:02.643 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:35:02.643 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:35:02.651 INFO:teuthology.orchestra.run.smithi173.stderr:bash: line 1: ntpq: command not found 2024-04-26T01:35:02.654 INFO:teuthology.orchestra.run.smithi173.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T01:35:02.654 INFO:teuthology.orchestra.run.smithi173.stdout:=============================================================================== 2024-04-26T01:35:02.655 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:35:02.655 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:35:02.655 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:35:02.655 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:35:02.686 INFO:teuthology.orchestra.run.smithi001.stderr:sudo: ntpd: command not found 2024-04-26T01:35:02.720 INFO:teuthology.orchestra.run.smithi001.stdout:506 Cannot talk to daemon 2024-04-26T01:35:02.766 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T01:35:02.809 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T01:35:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:bash: line 1: ntpq: command not found 2024-04-26T01:35:02.962 INFO:teuthology.orchestra.run.smithi001.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T01:35:02.963 INFO:teuthology.orchestra.run.smithi001.stdout:=============================================================================== 2024-04-26T01:35:02.963 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:35:02.963 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:35:02.963 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:35:02.963 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:35:02.964 INFO:teuthology.run_tasks:Running task install... 2024-04-26T01:35:02.974 DEBUG:teuthology.task.install:project ceph 2024-04-26T01:35:02.974 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-26T01:35:02.974 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-26T01:35:02.974 INFO:teuthology.task.install:Using flavor: default 2024-04-26T01:35:02.980 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T01:35:02.980 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T01:35:02.981 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-26T01:35:02.981 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:35:02.984 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-26T01:35:02.984 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:35:02.985 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-26T01:35:02.985 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:35:03.141 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-26T01:35:03.141 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-26T01:35:03.144 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-26T01:35:03.145 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-26T01:35:03.278 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T01:35:03.279 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-04-26T01:35:03.279 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T01:35:03.310 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-26T01:35:03.310 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-26T01:35:03.373 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T01:35:03.373 DEBUG:teuthology.orchestra.run.smithi001:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T01:35:03.417 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T01:35:03.418 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T01:35:03.418 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T01:35:03.446 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T01:35:03.446 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T01:35:03.507 DEBUG:teuthology.orchestra.run.smithi001:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T01:35:03.513 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T01:35:03.589 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T01:35:03.589 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-26T01:35:03.589 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T01:35:03.591 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T01:35:03.618 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T01:35:03.619 DEBUG:teuthology.orchestra.run.smithi173:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T01:35:03.623 DEBUG:teuthology.orchestra.run.smithi001:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T01:35:03.656 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2024-04-26T01:35:03.658 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2024-04-26T01:35:03.689 DEBUG:teuthology.orchestra.run.smithi173:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T01:35:03.697 INFO:teuthology.orchestra.run.smithi001.stdout:check_obsoletes = 1 2024-04-26T01:35:03.701 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum clean all 2024-04-26T01:35:03.765 DEBUG:teuthology.orchestra.run.smithi173:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T01:35:03.829 INFO:teuthology.orchestra.run.smithi173.stdout:check_obsoletes = 1 2024-04-26T01:35:03.831 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum clean all 2024-04-26T01:35:03.882 INFO:teuthology.orchestra.run.smithi097.stdout:48 files removed 2024-04-26T01:35:03.909 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T01:35:04.025 INFO:teuthology.orchestra.run.smithi001.stdout:48 files removed 2024-04-26T01:35:04.061 INFO:teuthology.orchestra.run.smithi173.stdout:48 files removed 2024-04-26T01:35:04.065 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T01:35:04.087 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T01:35:04.660 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 169 kB/s | 83 kB 00:00 2024-04-26T01:35:04.818 INFO:teuthology.orchestra.run.smithi173.stdout:ceph packages for x86_64 176 kB/s | 83 kB 00:00 2024-04-26T01:35:04.977 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 43 kB/s | 12 kB 00:00 2024-04-26T01:35:05.022 INFO:teuthology.orchestra.run.smithi001.stdout:ceph packages for x86_64 162 kB/s | 83 kB 00:00 2024-04-26T01:35:05.186 INFO:teuthology.orchestra.run.smithi173.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-04-26T01:35:05.427 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-04-26T01:35:05.573 INFO:teuthology.orchestra.run.smithi001.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-04-26T01:35:05.612 INFO:teuthology.orchestra.run.smithi173.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-26T01:35:06.049 INFO:teuthology.orchestra.run.smithi001.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-26T01:35:06.188 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-26T01:35:06.430 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-26T01:35:06.960 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - BaseOS 9.7 MB/s | 8.0 MB 00:00 2024-04-26T01:35:08.182 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-26T01:35:08.624 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-26T01:35:09.419 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-26T01:35:12.426 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 9.4 MB/s | 6.2 MB 00:00 2024-04-26T01:35:12.691 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-26T01:35:14.108 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-26T01:35:14.154 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - CRB 8.3 MB/s | 6.2 MB 00:00 2024-04-26T01:35:14.282 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-26T01:35:15.345 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-26T01:35:15.783 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 16 kB 00:00 2024-04-26T01:35:17.243 INFO:teuthology.orchestra.run.smithi001.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-26T01:35:20.484 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-26T01:35:22.644 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T01:35:22.645 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T01:35:22.656 INFO:teuthology.orchestra.run.smithi097.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T01:35:22.656 INFO:teuthology.orchestra.run.smithi097.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T01:35:22.657 INFO:teuthology.orchestra.run.smithi097.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T01:35:22.658 INFO:teuthology.orchestra.run.smithi097.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T01:35:22.660 INFO:teuthology.orchestra.run.smithi097.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T01:35:22.662 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T01:35:22.663 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T01:35:22.724 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-04-26T01:35:22.739 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================= 2024-04-26T01:35:22.739 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-04-26T01:35:22.739 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================= 2024-04-26T01:35:22.740 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-04-26T01:35:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T01:35:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-26T01:35:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-26T01:35:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-26T01:35:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-26T01:35:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-26T01:35:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-26T01:35:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-26T01:35:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-26T01:35:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-26T01:35:22.740 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-26T01:35:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-26T01:35:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-26T01:35:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-26T01:35:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-26T01:35:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-26T01:35:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-26T01:35:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T01:35:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T01:35:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T01:35:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-26T01:35:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-26T01:35:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-26T01:35:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T01:35:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T01:35:22.741 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T01:35:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T01:35:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T01:35:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T01:35:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-26T01:35:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-26T01:35:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-26T01:35:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-26T01:35:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-26T01:35:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-26T01:35:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-26T01:35:22.742 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T01:35:22.742 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2024-04-26T01:35:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-26T01:35:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-26T01:35:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T01:35:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T01:35:22.743 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-04-26T01:35:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T01:35:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T01:35:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T01:35:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-26T01:35:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-26T01:35:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-26T01:35:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-26T01:35:22.743 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-26T01:35:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-26T01:35:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-26T01:35:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-26T01:35:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T01:35:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T01:35:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T01:35:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T01:35:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T01:35:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T01:35:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-26T01:35:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-26T01:35:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-26T01:35:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T01:35:22.744 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T01:35:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T01:35:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T01:35:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T01:35:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-26T01:35:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T01:35:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-26T01:35:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T01:35:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T01:35:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T01:35:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T01:35:22.745 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T01:35:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T01:35:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T01:35:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T01:35:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-26T01:35:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T01:35:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T01:35:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T01:35:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T01:35:22.746 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T01:35:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T01:35:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T01:35:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T01:35:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-26T01:35:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-26T01:35:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T01:35:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T01:35:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T01:35:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T01:35:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T01:35:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T01:35:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T01:35:22.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T01:35:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T01:35:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T01:35:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T01:35:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T01:35:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T01:35:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T01:35:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T01:35:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T01:35:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T01:35:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T01:35:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T01:35:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T01:35:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T01:35:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T01:35:22.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T01:35:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T01:35:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T01:35:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T01:35:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T01:35:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T01:35:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T01:35:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T01:35:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T01:35:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T01:35:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T01:35:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T01:35:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T01:35:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T01:35:22.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T01:35:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T01:35:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T01:35:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T01:35:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T01:35:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T01:35:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T01:35:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T01:35:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T01:35:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T01:35:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T01:35:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T01:35:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T01:35:22.750 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T01:35:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T01:35:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T01:35:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T01:35:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T01:35:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T01:35:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T01:35:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T01:35:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T01:35:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T01:35:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T01:35:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T01:35:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T01:35:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T01:35:22.751 INFO:teuthology.orchestra.run.smithi097.stdout:Installing weak dependencies: 2024-04-26T01:35:22.751 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T01:35:22.752 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T01:35:22.752 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-04-26T01:35:22.752 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================= 2024-04-26T01:35:22.752 INFO:teuthology.orchestra.run.smithi097.stdout:Install 148 Packages 2024-04-26T01:35:22.752 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 4 Packages 2024-04-26T01:35:22.752 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T01:35:22.752 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 222 M 2024-04-26T01:35:22.753 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-04-26T01:35:23.326 INFO:teuthology.orchestra.run.smithi001.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-26T01:35:23.353 INFO:teuthology.orchestra.run.smithi097.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 32 kB/s | 6.6 kB 00:00 2024-04-26T01:35:23.645 INFO:teuthology.orchestra.run.smithi097.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 3.0 MB/s | 890 kB 00:00 2024-04-26T01:35:23.745 INFO:teuthology.orchestra.run.smithi097.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.4 MB/s | 145 kB 00:00 2024-04-26T01:35:24.062 INFO:teuthology.orchestra.run.smithi097.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 5.9 MB/s | 5.4 MB 00:00 2024-04-26T01:35:24.112 INFO:teuthology.orchestra.run.smithi097.stdout:(5/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 6.4 MB/s | 2.4 MB 00:00 2024-04-26T01:35:24.321 INFO:teuthology.orchestra.run.smithi097.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 5.8 MB/s | 1.5 MB 00:00 2024-04-26T01:35:24.696 INFO:teuthology.orchestra.run.smithi097.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 8.1 MB/s | 4.7 MB 00:00 2024-04-26T01:35:25.346 INFO:teuthology.orchestra.run.smithi097.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 9.6 MB/s | 21 MB 00:02 2024-04-26T01:35:25.446 INFO:teuthology.orchestra.run.smithi097.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 252 kB/s | 25 kB 00:00 2024-04-26T01:35:25.791 INFO:teuthology.orchestra.run.smithi001.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T01:35:25.792 INFO:teuthology.orchestra.run.smithi001.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T01:35:25.809 INFO:teuthology.orchestra.run.smithi001.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T01:35:25.810 INFO:teuthology.orchestra.run.smithi001.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T01:35:25.812 INFO:teuthology.orchestra.run.smithi001.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T01:35:25.813 INFO:teuthology.orchestra.run.smithi001.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T01:35:25.815 INFO:teuthology.orchestra.run.smithi001.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T01:35:25.819 INFO:teuthology.orchestra.run.smithi001.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T01:35:25.820 INFO:teuthology.orchestra.run.smithi001.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T01:35:25.889 INFO:teuthology.orchestra.run.smithi097.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 8.9 MB/s | 11 MB 00:01 2024-04-26T01:35:25.895 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2024-04-26T01:35:25.911 INFO:teuthology.orchestra.run.smithi001.stdout:======================================================================================= 2024-04-26T01:35:25.911 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repository Size 2024-04-26T01:35:25.911 INFO:teuthology.orchestra.run.smithi001.stdout:======================================================================================= 2024-04-26T01:35:25.911 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2024-04-26T01:35:25.911 INFO:teuthology.orchestra.run.smithi001.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T01:35:25.911 INFO:teuthology.orchestra.run.smithi001.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-26T01:35:25.912 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-26T01:35:25.912 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-26T01:35:25.912 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-26T01:35:25.912 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-26T01:35:25.912 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-26T01:35:25.912 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-26T01:35:25.913 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-26T01:35:25.913 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-26T01:35:25.913 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-26T01:35:25.913 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-26T01:35:25.913 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-26T01:35:25.913 INFO:teuthology.orchestra.run.smithi001.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-26T01:35:25.914 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-26T01:35:25.914 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-26T01:35:25.914 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-26T01:35:25.914 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T01:35:25.914 INFO:teuthology.orchestra.run.smithi001.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T01:35:25.915 INFO:teuthology.orchestra.run.smithi001.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T01:35:25.915 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-26T01:35:25.915 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-26T01:35:25.915 INFO:teuthology.orchestra.run.smithi001.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-26T01:35:25.915 INFO:teuthology.orchestra.run.smithi001.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T01:35:25.915 INFO:teuthology.orchestra.run.smithi001.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T01:35:25.916 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T01:35:25.916 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T01:35:25.916 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T01:35:25.916 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T01:35:25.916 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-26T01:35:25.916 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-26T01:35:25.917 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-26T01:35:25.917 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-26T01:35:25.917 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-26T01:35:25.917 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-26T01:35:25.917 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-26T01:35:25.917 INFO:teuthology.orchestra.run.smithi001.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T01:35:25.918 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrading: 2024-04-26T01:35:25.918 INFO:teuthology.orchestra.run.smithi001.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-26T01:35:25.918 INFO:teuthology.orchestra.run.smithi001.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-26T01:35:25.918 INFO:teuthology.orchestra.run.smithi001.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T01:35:25.918 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T01:35:25.919 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2024-04-26T01:35:25.919 INFO:teuthology.orchestra.run.smithi001.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T01:35:25.919 INFO:teuthology.orchestra.run.smithi001.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T01:35:25.919 INFO:teuthology.orchestra.run.smithi001.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T01:35:25.919 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-26T01:35:25.919 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-26T01:35:25.919 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-26T01:35:25.919 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-26T01:35:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-26T01:35:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-26T01:35:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-26T01:35:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-26T01:35:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T01:35:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T01:35:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T01:35:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T01:35:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T01:35:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T01:35:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-26T01:35:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-26T01:35:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-26T01:35:25.920 INFO:teuthology.orchestra.run.smithi001.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T01:35:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T01:35:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T01:35:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T01:35:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T01:35:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-26T01:35:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T01:35:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-26T01:35:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T01:35:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T01:35:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T01:35:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T01:35:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T01:35:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T01:35:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T01:35:25.921 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T01:35:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-26T01:35:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T01:35:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T01:35:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T01:35:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T01:35:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T01:35:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T01:35:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T01:35:25.922 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T01:35:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-26T01:35:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-26T01:35:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T01:35:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T01:35:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T01:35:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T01:35:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T01:35:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T01:35:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T01:35:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T01:35:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T01:35:25.923 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T01:35:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T01:35:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T01:35:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T01:35:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T01:35:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T01:35:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T01:35:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T01:35:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T01:35:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T01:35:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T01:35:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T01:35:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T01:35:25.924 INFO:teuthology.orchestra.run.smithi001.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T01:35:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T01:35:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T01:35:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T01:35:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T01:35:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T01:35:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T01:35:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T01:35:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T01:35:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T01:35:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T01:35:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T01:35:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T01:35:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T01:35:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T01:35:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T01:35:25.925 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T01:35:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T01:35:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T01:35:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T01:35:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T01:35:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T01:35:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T01:35:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T01:35:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T01:35:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T01:35:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T01:35:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T01:35:25.926 INFO:teuthology.orchestra.run.smithi001.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T01:35:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T01:35:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T01:35:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T01:35:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T01:35:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T01:35:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T01:35:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T01:35:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T01:35:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T01:35:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T01:35:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T01:35:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T01:35:25.927 INFO:teuthology.orchestra.run.smithi001.stdout:Installing weak dependencies: 2024-04-26T01:35:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T01:35:25.927 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:35:25.928 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2024-04-26T01:35:25.928 INFO:teuthology.orchestra.run.smithi001.stdout:======================================================================================= 2024-04-26T01:35:25.928 INFO:teuthology.orchestra.run.smithi001.stdout:Install 148 Packages 2024-04-26T01:35:25.928 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrade 4 Packages 2024-04-26T01:35:25.928 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:35:25.928 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 222 M 2024-04-26T01:35:25.928 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2024-04-26T01:35:25.990 INFO:teuthology.orchestra.run.smithi097.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.2 MB/s | 225 kB 00:00 2024-04-26T01:35:26.064 INFO:teuthology.orchestra.run.smithi097.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 361 kB/s | 27 kB 00:00 2024-04-26T01:35:26.139 INFO:teuthology.orchestra.run.smithi097.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 424 kB/s | 32 kB 00:00 2024-04-26T01:35:26.348 INFO:teuthology.orchestra.run.smithi097.stdout:(14/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 8.3 MB/s | 17 MB 00:02 2024-04-26T01:35:26.382 INFO:teuthology.orchestra.run.smithi097.stdout:(15/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 3.0 MB/s | 747 kB 00:00 2024-04-26T01:35:26.415 INFO:teuthology.orchestra.run.smithi097.stdout:(16/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 2.4 MB/s | 163 kB 00:00 2024-04-26T01:35:26.466 INFO:teuthology.orchestra.run.smithi097.stdout:(17/152): librados-devel-19.0.0-3156.gd69f13d8. 1.5 MB/s | 126 kB 00:00 2024-04-26T01:35:26.499 INFO:teuthology.orchestra.run.smithi097.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 5.8 MB/s | 500 kB 00:00 2024-04-26T01:35:26.583 INFO:teuthology.orchestra.run.smithi097.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 542 kB/s | 45 kB 00:00 2024-04-26T01:35:26.675 INFO:teuthology.orchestra.run.smithi097.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.4 MB/s | 135 kB 00:00 2024-04-26T01:35:26.767 INFO:teuthology.orchestra.run.smithi097.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.7 MB/s | 161 kB 00:00 2024-04-26T01:35:26.875 INFO:teuthology.orchestra.run.smithi097.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 2.9 MB/s | 320 kB 00:00 2024-04-26T01:35:26.975 INFO:teuthology.orchestra.run.smithi097.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 2.9 MB/s | 298 kB 00:00 2024-04-26T01:35:27.019 INFO:teuthology.orchestra.run.smithi001.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 18 kB/s | 6.6 kB 00:00 2024-04-26T01:35:27.051 INFO:teuthology.orchestra.run.smithi097.stdout:(24/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 9.2 MB/s | 5.4 MB 00:00 2024-04-26T01:35:27.076 INFO:teuthology.orchestra.run.smithi097.stdout:(25/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 991 kB/s | 99 kB 00:00 2024-04-26T01:35:27.135 INFO:teuthology.orchestra.run.smithi097.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.0 MB/s | 85 kB 00:00 2024-04-26T01:35:27.235 INFO:teuthology.orchestra.run.smithi097.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.7 MB/s | 170 kB 00:00 2024-04-26T01:35:27.335 INFO:teuthology.orchestra.run.smithi097.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 343 kB/s | 34 kB 00:00 2024-04-26T01:35:27.452 INFO:teuthology.orchestra.run.smithi097.stdout:(29/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 8.2 MB/s | 3.1 MB 00:00 2024-04-26T01:35:27.477 INFO:teuthology.orchestra.run.smithi097.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.0 MB/s | 142 kB 00:00 2024-04-26T01:35:27.928 INFO:teuthology.orchestra.run.smithi001.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 981 kB/s | 890 kB 00:00 2024-04-26T01:35:27.945 INFO:teuthology.orchestra.run.smithi097.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 7.2 MB/s | 3.5 MB 00:00 2024-04-26T01:35:28.045 INFO:teuthology.orchestra.run.smithi097.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.4 MB/s | 249 kB 00:00 2024-04-26T01:35:28.053 INFO:teuthology.orchestra.run.smithi001.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.1 MB/s | 145 kB 00:00 2024-04-26T01:35:28.137 INFO:teuthology.orchestra.run.smithi097.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 538 kB/s | 49 kB 00:00 2024-04-26T01:35:28.220 INFO:teuthology.orchestra.run.smithi097.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 198 kB/s | 16 kB 00:00 2024-04-26T01:35:28.320 INFO:teuthology.orchestra.run.smithi097.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.7 MB/s | 273 kB 00:00 2024-04-26T01:35:28.446 INFO:teuthology.orchestra.run.smithi097.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 6.0 MB/s | 762 kB 00:00 2024-04-26T01:35:28.540 INFO:teuthology.orchestra.run.smithi097.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 7.0 MB/s | 7.4 MB 00:01 2024-04-26T01:35:28.564 INFO:teuthology.orchestra.run.smithi097.stdout:(38/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 155 kB/s | 18 kB 00:00 2024-04-26T01:35:28.689 INFO:teuthology.orchestra.run.smithi097.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 535 kB/s | 80 kB 00:00 2024-04-26T01:35:28.739 INFO:teuthology.orchestra.run.smithi097.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-04-26T01:35:28.764 INFO:teuthology.orchestra.run.smithi097.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2024-04-26T01:35:28.770 INFO:teuthology.orchestra.run.smithi001.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.5 MB/s | 5.4 MB 00:02 2024-04-26T01:35:28.798 INFO:teuthology.orchestra.run.smithi097.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-04-26T01:35:28.840 INFO:teuthology.orchestra.run.smithi097.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-04-26T01:35:28.865 INFO:teuthology.orchestra.run.smithi097.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-26T01:35:28.887 INFO:teuthology.orchestra.run.smithi001.stdout:(5/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 2.8 MB/s | 2.4 MB 00:00 2024-04-26T01:35:28.890 INFO:teuthology.orchestra.run.smithi097.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-26T01:35:28.916 INFO:teuthology.orchestra.run.smithi097.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-26T01:35:28.941 INFO:teuthology.orchestra.run.smithi097.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-26T01:35:28.975 INFO:teuthology.orchestra.run.smithi097.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 12 MB/s | 689 kB 00:00 2024-04-26T01:35:29.016 INFO:teuthology.orchestra.run.smithi097.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-04-26T01:35:29.037 INFO:teuthology.orchestra.run.smithi001.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 5.6 MB/s | 1.5 MB 00:00 2024-04-26T01:35:29.042 INFO:teuthology.orchestra.run.smithi097.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-26T01:35:29.067 INFO:teuthology.orchestra.run.smithi097.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 428 kB/s | 21 kB 00:00 2024-04-26T01:35:29.093 INFO:teuthology.orchestra.run.smithi097.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-26T01:35:29.118 INFO:teuthology.orchestra.run.smithi097.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 657 kB/s | 33 kB 00:00 2024-04-26T01:35:29.160 INFO:teuthology.orchestra.run.smithi097.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 450 kB/s | 18 kB 00:00 2024-04-26T01:35:29.210 INFO:teuthology.orchestra.run.smithi097.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-04-26T01:35:29.235 INFO:teuthology.orchestra.run.smithi097.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 153 kB/s | 11 kB 00:00 2024-04-26T01:35:29.261 INFO:teuthology.orchestra.run.smithi097.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-26T01:35:29.286 INFO:teuthology.orchestra.run.smithi097.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 909 kB/s | 45 kB 00:00 2024-04-26T01:35:29.319 INFO:teuthology.orchestra.run.smithi097.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-26T01:35:29.345 INFO:teuthology.orchestra.run.smithi097.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 582 kB 00:00 2024-04-26T01:35:29.370 INFO:teuthology.orchestra.run.smithi097.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 513 kB/s | 25 kB 00:00 2024-04-26T01:35:29.395 INFO:teuthology.orchestra.run.smithi097.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-04-26T01:35:29.429 INFO:teuthology.orchestra.run.smithi097.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-26T01:35:29.454 INFO:teuthology.orchestra.run.smithi097.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-26T01:35:29.480 INFO:teuthology.orchestra.run.smithi097.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 700 kB/s | 35 kB 00:00 2024-04-26T01:35:29.604 INFO:teuthology.orchestra.run.smithi001.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 6.6 MB/s | 4.7 MB 00:00 2024-04-26T01:35:29.931 INFO:teuthology.orchestra.run.smithi097.stdout:(66/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 11 MB/s | 49 MB 00:04 2024-04-26T01:35:30.039 INFO:teuthology.orchestra.run.smithi097.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 8.1 MB/s | 4.7 MB 00:00 2024-04-26T01:35:30.106 INFO:teuthology.orchestra.run.smithi097.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.1 MB/s | 4.4 MB 00:00 2024-04-26T01:35:30.139 INFO:teuthology.orchestra.run.smithi097.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 7.7 MB/s | 1.6 MB 00:00 2024-04-26T01:35:30.165 INFO:teuthology.orchestra.run.smithi097.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 7.4 MB/s | 911 kB 00:00 2024-04-26T01:35:30.190 INFO:teuthology.orchestra.run.smithi097.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-04-26T01:35:30.232 INFO:teuthology.orchestra.run.smithi097.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-26T01:35:30.257 INFO:teuthology.orchestra.run.smithi097.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-26T01:35:30.332 INFO:teuthology.orchestra.run.smithi097.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.3 MB/s | 244 kB 00:00 2024-04-26T01:35:30.338 INFO:teuthology.orchestra.run.smithi001.stdout:(8/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 13 MB/s | 17 MB 00:01 2024-04-26T01:35:30.416 INFO:teuthology.orchestra.run.smithi097.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-04-26T01:35:30.421 INFO:teuthology.orchestra.run.smithi001.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 306 kB/s | 25 kB 00:00 2024-04-26T01:35:30.491 INFO:teuthology.orchestra.run.smithi097.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 6.0 MB 00:00 2024-04-26T01:35:30.516 INFO:teuthology.orchestra.run.smithi097.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.3 MB/s | 248 kB 00:00 2024-04-26T01:35:30.533 INFO:teuthology.orchestra.run.smithi097.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 223 kB/s | 26 kB 00:00 2024-04-26T01:35:30.550 INFO:teuthology.orchestra.run.smithi097.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 576 kB/s | 19 kB 00:00 2024-04-26T01:35:30.575 INFO:teuthology.orchestra.run.smithi097.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-26T01:35:30.601 INFO:teuthology.orchestra.run.smithi097.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 517 kB/s | 35 kB 00:00 2024-04-26T01:35:30.626 INFO:teuthology.orchestra.run.smithi097.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-26T01:35:30.651 INFO:teuthology.orchestra.run.smithi097.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-26T01:35:30.677 INFO:teuthology.orchestra.run.smithi097.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.6 MB/s | 77 kB 00:00 2024-04-26T01:35:30.702 INFO:teuthology.orchestra.run.smithi097.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 878 kB/s | 42 kB 00:00 2024-04-26T01:35:30.746 INFO:teuthology.orchestra.run.smithi001.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 9.3 MB/s | 11 MB 00:01 2024-04-26T01:35:30.769 INFO:teuthology.orchestra.run.smithi097.stdout:(86/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 5.1 MB 00:00 2024-04-26T01:35:30.794 INFO:teuthology.orchestra.run.smithi097.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-04-26T01:35:30.819 INFO:teuthology.orchestra.run.smithi097.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-04-26T01:35:30.830 INFO:teuthology.orchestra.run.smithi001.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.7 MB/s | 225 kB 00:00 2024-04-26T01:35:30.844 INFO:teuthology.orchestra.run.smithi097.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-26T01:35:30.870 INFO:teuthology.orchestra.run.smithi097.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-26T01:35:30.895 INFO:teuthology.orchestra.run.smithi097.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 715 kB/s | 54 kB 00:00 2024-04-26T01:35:30.922 INFO:teuthology.orchestra.run.smithi001.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 298 kB/s | 27 kB 00:00 2024-04-26T01:35:30.953 INFO:teuthology.orchestra.run.smithi097.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 749 kB/s | 42 kB 00:00 2024-04-26T01:35:30.978 INFO:teuthology.orchestra.run.smithi097.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 547 kB/s | 44 kB 00:00 2024-04-26T01:35:31.006 INFO:teuthology.orchestra.run.smithi001.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 385 kB/s | 32 kB 00:00 2024-04-26T01:35:31.020 INFO:teuthology.orchestra.run.smithi097.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-04-26T01:35:31.046 INFO:teuthology.orchestra.run.smithi097.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 557 kB/s | 36 kB 00:00 2024-04-26T01:35:31.088 INFO:teuthology.orchestra.run.smithi097.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 737 kB/s | 47 kB 00:00 2024-04-26T01:35:31.138 INFO:teuthology.orchestra.run.smithi097.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.3 MB/s | 305 kB 00:00 2024-04-26T01:35:31.147 INFO:teuthology.orchestra.run.smithi001.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 5.2 MB/s | 747 kB 00:00 2024-04-26T01:35:31.256 INFO:teuthology.orchestra.run.smithi001.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.5 MB/s | 163 kB 00:00 2024-04-26T01:35:31.297 INFO:teuthology.orchestra.run.smithi097.stdout:(98/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-04-26T01:35:31.322 INFO:teuthology.orchestra.run.smithi097.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 250 kB/s | 58 kB 00:00 2024-04-26T01:35:31.348 INFO:teuthology.orchestra.run.smithi097.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 454 kB/s | 94 kB 00:00 2024-04-26T01:35:31.348 INFO:teuthology.orchestra.run.smithi001.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.4 MB/s | 126 kB 00:00 2024-04-26T01:35:31.373 INFO:teuthology.orchestra.run.smithi097.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-04-26T01:35:31.398 INFO:teuthology.orchestra.run.smithi097.stdout:(102/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 848 kB/s | 64 kB 00:00 2024-04-26T01:35:31.423 INFO:teuthology.orchestra.run.smithi097.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-04-26T01:35:31.465 INFO:teuthology.orchestra.run.smithi097.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 9.2 MB/s | 863 kB 00:00 2024-04-26T01:35:31.465 INFO:teuthology.orchestra.run.smithi001.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 4.2 MB/s | 500 kB 00:00 2024-04-26T01:35:31.924 INFO:teuthology.orchestra.run.smithi097.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 42 kB/s | 19 kB 00:00 2024-04-26T01:35:32.174 INFO:teuthology.orchestra.run.smithi001.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 7.6 MB/s | 5.4 MB 00:00 2024-04-26T01:35:32.241 INFO:teuthology.orchestra.run.smithi001.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 687 kB/s | 45 kB 00:00 2024-04-26T01:35:32.316 INFO:teuthology.orchestra.run.smithi001.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.8 MB/s | 135 kB 00:00 2024-04-26T01:35:32.391 INFO:teuthology.orchestra.run.smithi097.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 320 kB/s | 309 kB 00:00 2024-04-26T01:35:32.400 INFO:teuthology.orchestra.run.smithi001.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.9 MB/s | 161 kB 00:00 2024-04-26T01:35:32.500 INFO:teuthology.orchestra.run.smithi001.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 3.2 MB/s | 320 kB 00:00 2024-04-26T01:35:32.507 INFO:teuthology.orchestra.run.smithi097.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 223 kB/s | 26 kB 00:00 2024-04-26T01:35:32.592 INFO:teuthology.orchestra.run.smithi001.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 3.2 MB/s | 298 kB 00:00 2024-04-26T01:35:32.624 INFO:teuthology.orchestra.run.smithi097.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 416 kB/s | 49 kB 00:00 2024-04-26T01:35:32.650 INFO:teuthology.orchestra.run.smithi097.stdout:(109/152): abseil-cpp-20211102.0-3.el9.x86_64.r 448 kB/s | 548 kB 00:01 2024-04-26T01:35:32.684 INFO:teuthology.orchestra.run.smithi001.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.1 MB/s | 99 kB 00:00 2024-04-26T01:35:32.784 INFO:teuthology.orchestra.run.smithi001.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 861 kB/s | 85 kB 00:00 2024-04-26T01:35:32.825 INFO:teuthology.orchestra.run.smithi097.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 336 kB/s | 67 kB 00:00 2024-04-26T01:35:32.942 INFO:teuthology.orchestra.run.smithi097.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 518 kB/s | 151 kB 00:00 2024-04-26T01:35:33.335 INFO:teuthology.orchestra.run.smithi001.stdout:(26/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 17 MB/s | 49 MB 00:02 2024-04-26T01:35:33.410 INFO:teuthology.orchestra.run.smithi001.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.3 MB/s | 170 kB 00:00 2024-04-26T01:35:33.502 INFO:teuthology.orchestra.run.smithi001.stdout:(28/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 4.3 MB/s | 3.1 MB 00:00 2024-04-26T01:35:33.527 INFO:teuthology.orchestra.run.smithi001.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3156.g 296 kB/s | 34 kB 00:00 2024-04-26T01:35:33.577 INFO:teuthology.orchestra.run.smithi001.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.9 MB/s | 142 kB 00:00 2024-04-26T01:35:33.668 INFO:teuthology.orchestra.run.smithi173.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-04-26T01:35:33.742 INFO:teuthology.orchestra.run.smithi097.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 685 kB/s | 548 kB 00:00 2024-04-26T01:35:33.836 INFO:teuthology.orchestra.run.smithi001.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 11 MB/s | 3.5 MB 00:00 2024-04-26T01:35:33.851 INFO:teuthology.orchestra.run.smithi097.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 401 kB/s | 43 kB 00:00 2024-04-26T01:35:33.911 INFO:teuthology.orchestra.run.smithi001.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 3.3 MB/s | 249 kB 00:00 2024-04-26T01:35:33.959 INFO:teuthology.orchestra.run.smithi097.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 297 kB/s | 32 kB 00:00 2024-04-26T01:35:33.985 INFO:teuthology.orchestra.run.smithi097.stdout:(115/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 723 kB/s | 838 kB 00:01 2024-04-26T01:35:34.003 INFO:teuthology.orchestra.run.smithi001.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 544 kB/s | 49 kB 00:00 2024-04-26T01:35:34.068 INFO:teuthology.orchestra.run.smithi097.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-04-26T01:35:34.078 INFO:teuthology.orchestra.run.smithi001.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 222 kB/s | 16 kB 00:00 2024-04-26T01:35:34.195 INFO:teuthology.orchestra.run.smithi001.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.3 MB/s | 273 kB 00:00 2024-04-26T01:35:34.285 INFO:teuthology.orchestra.run.smithi097.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 572 kB/s | 172 kB 00:00 2024-04-26T01:35:34.337 INFO:teuthology.orchestra.run.smithi001.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 9.7 MB/s | 7.4 MB 00:00 2024-04-26T01:35:34.396 INFO:teuthology.orchestra.run.smithi001.stdout:(37/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 3.7 MB/s | 762 kB 00:00 2024-04-26T01:35:34.421 INFO:teuthology.orchestra.run.smithi001.stdout:(38/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 221 kB/s | 18 kB 00:00 2024-04-26T01:35:34.619 INFO:teuthology.orchestra.run.smithi097.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 651 kB/s | 358 kB 00:00 2024-04-26T01:35:34.654 INFO:teuthology.orchestra.run.smithi001.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 312 kB/s | 80 kB 00:00 2024-04-26T01:35:34.728 INFO:teuthology.orchestra.run.smithi097.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 686 kB/s | 303 kB 00:00 2024-04-26T01:35:34.738 INFO:teuthology.orchestra.run.smithi001.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-04-26T01:35:34.752 INFO:teuthology.orchestra.run.smithi097.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 264 kB/s | 35 kB 00:00 2024-04-26T01:35:34.771 INFO:teuthology.orchestra.run.smithi001.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.2 MB/s | 803 kB 00:00 2024-04-26T01:35:34.805 INFO:teuthology.orchestra.run.smithi001.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-04-26T01:35:34.872 INFO:teuthology.orchestra.run.smithi001.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.2 MB 00:00 2024-04-26T01:35:34.898 INFO:teuthology.orchestra.run.smithi001.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-04-26T01:35:34.923 INFO:teuthology.orchestra.run.smithi001.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-04-26T01:35:34.948 INFO:teuthology.orchestra.run.smithi001.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-04-26T01:35:34.982 INFO:teuthology.orchestra.run.smithi001.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 215 kB 00:00 2024-04-26T01:35:35.024 INFO:teuthology.orchestra.run.smithi001.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 9.3 MB/s | 689 kB 00:00 2024-04-26T01:35:35.086 INFO:teuthology.orchestra.run.smithi097.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 573 kB/s | 206 kB 00:00 2024-04-26T01:35:35.099 INFO:teuthology.orchestra.run.smithi001.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 8.4 MB/s | 943 kB 00:00 2024-04-26T01:35:35.124 INFO:teuthology.orchestra.run.smithi001.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-04-26T01:35:35.158 INFO:teuthology.orchestra.run.smithi001.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 381 kB/s | 21 kB 00:00 2024-04-26T01:35:35.192 INFO:teuthology.orchestra.run.smithi001.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 4.7 MB/s | 313 kB 00:00 2024-04-26T01:35:35.217 INFO:teuthology.orchestra.run.smithi001.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 597 kB/s | 33 kB 00:00 2024-04-26T01:35:35.317 INFO:teuthology.orchestra.run.smithi001.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-26T01:35:35.342 INFO:teuthology.orchestra.run.smithi001.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 167 kB/s | 18 kB 00:00 2024-04-26T01:35:35.353 INFO:teuthology.orchestra.run.smithi097.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 542 kB/s | 144 kB 00:00 2024-04-26T01:35:35.367 INFO:teuthology.orchestra.run.smithi001.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 227 kB/s | 11 kB 00:00 2024-04-26T01:35:35.402 INFO:teuthology.orchestra.run.smithi001.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-26T01:35:35.426 INFO:teuthology.orchestra.run.smithi001.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 807 kB/s | 45 kB 00:00 2024-04-26T01:35:35.468 INFO:teuthology.orchestra.run.smithi001.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.9 MB/s | 662 kB 00:00 2024-04-26T01:35:35.470 INFO:teuthology.orchestra.run.smithi097.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 91 kB/s | 11 kB 00:00 2024-04-26T01:35:35.510 INFO:teuthology.orchestra.run.smithi001.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 7.5 MB/s | 582 kB 00:00 2024-04-26T01:35:35.544 INFO:teuthology.orchestra.run.smithi001.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 370 kB/s | 25 kB 00:00 2024-04-26T01:35:35.569 INFO:teuthology.orchestra.run.smithi001.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 827 kB/s | 47 kB 00:00 2024-04-26T01:35:35.578 INFO:teuthology.orchestra.run.smithi097.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-04-26T01:35:35.603 INFO:teuthology.orchestra.run.smithi001.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.0 MB/s | 292 kB 00:00 2024-04-26T01:35:35.636 INFO:teuthology.orchestra.run.smithi001.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 188 kB 00:00 2024-04-26T01:35:35.662 INFO:teuthology.orchestra.run.smithi001.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 631 kB/s | 35 kB 00:00 2024-04-26T01:35:35.679 INFO:teuthology.orchestra.run.smithi097.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2024-04-26T01:35:35.813 INFO:teuthology.orchestra.run.smithi097.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 146 kB/s | 19 kB 00:00 2024-04-26T01:35:35.862 INFO:teuthology.orchestra.run.smithi001.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-04-26T01:35:35.930 INFO:teuthology.orchestra.run.smithi097.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 168 kB/s | 20 kB 00:00 2024-04-26T01:35:35.979 INFO:teuthology.orchestra.run.smithi001.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 14 MB/s | 4.7 MB 00:00 2024-04-26T01:35:36.046 INFO:teuthology.orchestra.run.smithi001.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-04-26T01:35:36.096 INFO:teuthology.orchestra.run.smithi001.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 8.8 MB/s | 911 kB 00:00 2024-04-26T01:35:36.121 INFO:teuthology.orchestra.run.smithi001.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-26T01:35:36.230 INFO:teuthology.orchestra.run.smithi001.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-04-26T01:35:36.347 INFO:teuthology.orchestra.run.smithi001.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-04-26T01:35:36.372 INFO:teuthology.orchestra.run.smithi001.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 688 kB/s | 87 kB 00:00 2024-04-26T01:35:36.439 INFO:teuthology.orchestra.run.smithi001.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-26T01:35:36.582 INFO:teuthology.orchestra.run.smithi001.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 6.0 MB 00:00 2024-04-26T01:35:36.607 INFO:teuthology.orchestra.run.smithi001.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-04-26T01:35:36.649 INFO:teuthology.orchestra.run.smithi001.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 631 kB/s | 26 kB 00:00 2024-04-26T01:35:36.674 INFO:teuthology.orchestra.run.smithi001.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 292 kB/s | 19 kB 00:00 2024-04-26T01:35:36.716 INFO:teuthology.orchestra.run.smithi001.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 863 kB/s | 35 kB 00:00 2024-04-26T01:35:36.749 INFO:teuthology.orchestra.run.smithi001.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.3 MB/s | 172 kB 00:00 2024-04-26T01:35:36.816 INFO:teuthology.orchestra.run.smithi001.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-26T01:35:36.933 INFO:teuthology.orchestra.run.smithi001.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 5.1 MB 00:00 2024-04-26T01:35:36.967 INFO:teuthology.orchestra.run.smithi001.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.6 MB/s | 222 kB 00:00 2024-04-26T01:35:36.992 INFO:teuthology.orchestra.run.smithi001.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.4 MB/s | 77 kB 00:00 2024-04-26T01:35:37.009 INFO:teuthology.orchestra.run.smithi001.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-26T01:35:37.043 INFO:teuthology.orchestra.run.smithi001.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.4 MB/s | 269 kB 00:00 2024-04-26T01:35:37.068 INFO:teuthology.orchestra.run.smithi001.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.8 MB/s | 159 kB 00:00 2024-04-26T01:35:37.101 INFO:teuthology.orchestra.run.smithi001.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.9 MB/s | 279 kB 00:00 2024-04-26T01:35:37.127 INFO:teuthology.orchestra.run.smithi001.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 926 kB/s | 51 kB 00:00 2024-04-26T01:35:37.152 INFO:teuthology.orchestra.run.smithi001.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-26T01:35:37.213 INFO:teuthology.orchestra.run.smithi001.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 967 kB/s | 42 kB 00:00 2024-04-26T01:35:37.330 INFO:teuthology.orchestra.run.smithi097.stdout:(128/152): python3-grpcio-1.46.7-10.el9.x86_64. 810 kB/s | 2.0 MB 00:02 2024-04-26T01:35:37.344 INFO:teuthology.orchestra.run.smithi001.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 410 kB/s | 44 kB 00:00 2024-04-26T01:35:37.363 INFO:teuthology.orchestra.run.smithi097.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 729 kB/s | 1.0 MB 00:01 2024-04-26T01:35:37.436 INFO:teuthology.orchestra.run.smithi001.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.8 MB/s | 221 kB 00:00 2024-04-26T01:35:37.464 INFO:teuthology.orchestra.run.smithi097.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 347 kB/s | 46 kB 00:00 2024-04-26T01:35:37.598 INFO:teuthology.orchestra.run.smithi097.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 337 kB/s | 79 kB 00:00 2024-04-26T01:35:37.653 INFO:teuthology.orchestra.run.smithi001.stdout:(94/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-04-26T01:35:37.673 INFO:teuthology.orchestra.run.smithi097.stdout:(132/152): libarrow-9.0.0-10.el9.x86_64.rpm 787 kB/s | 4.4 MB 00:05 2024-04-26T01:35:37.678 INFO:teuthology.orchestra.run.smithi001.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 156 kB/s | 36 kB 00:00 2024-04-26T01:35:37.698 INFO:teuthology.orchestra.run.smithi097.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 246 kB/s | 58 kB 00:00 2024-04-26T01:35:37.704 INFO:teuthology.orchestra.run.smithi001.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 989 kB/s | 47 kB 00:00 2024-04-26T01:35:37.758 INFO:teuthology.orchestra.run.smithi001.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.8 MB/s | 305 kB 00:00 2024-04-26T01:35:37.782 INFO:teuthology.orchestra.run.smithi097.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-04-26T01:35:37.788 INFO:teuthology.orchestra.run.smithi001.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 724 kB/s | 58 kB 00:00 2024-04-26T01:35:37.813 INFO:teuthology.orchestra.run.smithi001.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.7 MB/s | 94 kB 00:00 2024-04-26T01:35:37.838 INFO:teuthology.orchestra.run.smithi001.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-26T01:35:37.864 INFO:teuthology.orchestra.run.smithi001.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 919 kB/s | 45 kB 00:00 2024-04-26T01:35:37.882 INFO:teuthology.orchestra.run.smithi097.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 490 kB/s | 90 kB 00:00 2024-04-26T01:35:37.889 INFO:teuthology.orchestra.run.smithi001.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 457 kB/s | 22 kB 00:00 2024-04-26T01:35:37.907 INFO:teuthology.orchestra.run.smithi097.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 225 kB/s | 28 kB 00:00 2024-04-26T01:35:37.939 INFO:teuthology.orchestra.run.smithi001.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 11 MB/s | 863 kB 00:00 2024-04-26T01:35:37.983 INFO:teuthology.orchestra.run.smithi097.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 307 kB/s | 31 kB 00:00 2024-04-26T01:35:38.091 INFO:teuthology.orchestra.run.smithi097.stdout:(138/152): python3-pecan-1.4.2-3.el9.noarch.rpm 552 kB/s | 272 kB 00:00 2024-04-26T01:35:38.166 INFO:teuthology.orchestra.run.smithi097.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 322 kB/s | 59 kB 00:00 2024-04-26T01:35:38.200 INFO:teuthology.orchestra.run.smithi097.stdout:(140/152): python3-routes-2.5.1-5.el9.noarch.rp 645 kB/s | 188 kB 00:00 2024-04-26T01:35:38.206 INFO:teuthology.orchestra.run.smithi001.stdout:(104/152): ceph-common-19.0.0-3156.gd69f13d8.el 1.8 MB/s | 21 MB 00:11 2024-04-26T01:35:38.225 INFO:teuthology.orchestra.run.smithi097.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 267 kB/s | 36 kB 00:00 2024-04-26T01:35:38.267 INFO:teuthology.orchestra.run.smithi097.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-04-26T01:35:38.323 INFO:teuthology.orchestra.run.smithi001.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 168 kB/s | 19 kB 00:00 2024-04-26T01:35:38.353 INFO:teuthology.orchestra.run.smithi001.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://ohioix.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 134.195.207.11) 2024-04-26T01:35:38.415 INFO:teuthology.orchestra.run.smithi001.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 651 kB/s | 309 kB 00:00 2024-04-26T01:35:38.442 INFO:teuthology.orchestra.run.smithi097.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 413 kB/s | 90 kB 00:00 2024-04-26T01:35:38.532 INFO:teuthology.orchestra.run.smithi001.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 865 kB/s | 548 kB 00:00 2024-04-26T01:35:38.536 INFO:teuthology.orchestra.run.smithi001.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for http://ohioix.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 134.195.207.11) 2024-04-26T01:35:38.551 INFO:teuthology.orchestra.run.smithi097.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 655 kB/s | 230 kB 00:00 2024-04-26T01:35:38.565 INFO:teuthology.orchestra.run.smithi001.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://ohioix.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 134.195.207.11) 2024-04-26T01:35:38.576 INFO:teuthology.orchestra.run.smithi097.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 149 kB/s | 20 kB 00:00 2024-04-26T01:35:38.665 INFO:teuthology.orchestra.run.smithi001.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 373 kB/s | 49 kB 00:00 2024-04-26T01:35:38.697 INFO:teuthology.orchestra.run.smithi173.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-26T01:35:38.748 INFO:teuthology.orchestra.run.smithi001.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for https://ohioix.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 134.195.207.11) 2024-04-26T01:35:38.782 INFO:teuthology.orchestra.run.smithi001.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 591 kB/s | 67 kB 00:00 2024-04-26T01:35:38.826 INFO:teuthology.orchestra.run.smithi097.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 696 kB/s | 192 kB 00:00 2024-04-26T01:35:38.860 INFO:teuthology.orchestra.run.smithi097.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 720 kB/s | 427 kB 00:00 2024-04-26T01:35:38.891 INFO:teuthology.orchestra.run.smithi001.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-26T01:35:38.922 INFO:teuthology.orchestra.run.smithi001.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://ohioix.mm.fcix.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 134.195.207.11) 2024-04-26T01:35:38.952 INFO:teuthology.orchestra.run.smithi001.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://ohioix.mm.fcix.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 134.195.207.11) 2024-04-26T01:35:39.041 INFO:teuthology.orchestra.run.smithi001.stdout:(111/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 47 kB/s | 26 kB 00:00 2024-04-26T01:35:39.125 INFO:teuthology.orchestra.run.smithi001.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 6.5 MB/s | 548 kB 00:00 2024-04-26T01:35:39.184 INFO:teuthology.orchestra.run.smithi001.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 759 kB/s | 43 kB 00:00 2024-04-26T01:35:39.242 INFO:teuthology.orchestra.run.smithi001.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 591 kB/s | 32 kB 00:00 2024-04-26T01:35:39.302 INFO:teuthology.orchestra.run.smithi097.stdout:(148/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 7.1 MB/s | 3.1 MB 00:00 2024-04-26T01:35:39.361 INFO:teuthology.orchestra.run.smithi097.stdout:(149/152): librados2-19.0.0-3156.gd69f13d8.el7. 6.4 MB/s | 3.4 MB 00:00 2024-04-26T01:35:39.393 INFO:teuthology.orchestra.run.smithi001.stdout:(115/152): libarrow-9.0.0-10.el9.x86_64.rpm 4.1 MB/s | 4.4 MB 00:01 2024-04-26T01:35:39.394 INFO:teuthology.orchestra.run.smithi097.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 15 MB/s | 1.3 MB 00:00 2024-04-26T01:35:39.426 INFO:teuthology.orchestra.run.smithi001.stdout:(116/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.5 MB/s | 838 kB 00:00 2024-04-26T01:35:39.451 INFO:teuthology.orchestra.run.smithi001.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 76 kB/s | 15 kB 00:00 2024-04-26T01:35:39.469 INFO:teuthology.orchestra.run.smithi097.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 23 MB/s | 2.4 MB 00:00 2024-04-26T01:35:39.493 INFO:teuthology.orchestra.run.smithi001.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.6 MB/s | 172 kB 00:00 2024-04-26T01:35:39.546 INFO:teuthology.orchestra.run.smithi001.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-04-26T01:35:39.578 INFO:teuthology.orchestra.run.smithi001.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 424 kB/s | 35 kB 00:00 2024-04-26T01:35:39.611 INFO:teuthology.orchestra.run.smithi001.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.1 MB/s | 206 kB 00:00 2024-04-26T01:35:39.670 INFO:teuthology.orchestra.run.smithi001.stdout:(122/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.4 MB/s | 303 kB 00:00 2024-04-26T01:35:39.695 INFO:teuthology.orchestra.run.smithi001.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-04-26T01:35:39.729 INFO:teuthology.orchestra.run.smithi001.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 185 kB/s | 11 kB 00:00 2024-04-26T01:35:39.754 INFO:teuthology.orchestra.run.smithi001.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 305 kB/s | 18 kB 00:00 2024-04-26T01:35:39.787 INFO:teuthology.orchestra.run.smithi001.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 401 kB/s | 23 kB 00:00 2024-04-26T01:35:39.813 INFO:teuthology.orchestra.run.smithi001.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 336 kB/s | 19 kB 00:00 2024-04-26T01:35:39.879 INFO:teuthology.orchestra.run.smithi001.stdout:(128/152): python3-grpcio-1.46.7-10.el9.x86_64. 6.8 MB/s | 2.0 MB 00:00 2024-04-26T01:35:39.905 INFO:teuthology.orchestra.run.smithi001.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 169 kB/s | 20 kB 00:00 2024-04-26T01:35:39.955 INFO:teuthology.orchestra.run.smithi001.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-04-26T01:35:39.980 INFO:teuthology.orchestra.run.smithi001.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 625 kB/s | 46 kB 00:00 2024-04-26T01:35:40.006 INFO:teuthology.orchestra.run.smithi001.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 795 kB/s | 79 kB 00:00 2024-04-26T01:35:40.031 INFO:teuthology.orchestra.run.smithi001.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 774 kB/s | 58 kB 00:00 2024-04-26T01:35:40.056 INFO:teuthology.orchestra.run.smithi001.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 332 kB/s | 16 kB 00:00 2024-04-26T01:35:40.081 INFO:teuthology.orchestra.run.smithi001.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-04-26T01:35:40.107 INFO:teuthology.orchestra.run.smithi001.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-26T01:35:40.132 INFO:teuthology.orchestra.run.smithi001.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 376 kB/s | 28 kB 00:00 2024-04-26T01:35:40.157 INFO:teuthology.orchestra.run.smithi001.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 410 kB/s | 31 kB 00:00 2024-04-26T01:35:40.191 INFO:teuthology.orchestra.run.smithi001.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-26T01:35:40.216 INFO:teuthology.orchestra.run.smithi001.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 619 kB/s | 36 kB 00:00 2024-04-26T01:35:40.241 INFO:teuthology.orchestra.run.smithi001.stdout:(141/152): python3-rsa-4.9-2.el9.noarch.rpm 546 kB/s | 59 kB 00:00 2024-04-26T01:35:40.267 INFO:teuthology.orchestra.run.smithi001.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 684 kB/s | 51 kB 00:00 2024-04-26T01:35:40.300 INFO:teuthology.orchestra.run.smithi001.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-26T01:35:40.318 INFO:teuthology.orchestra.run.smithi001.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-26T01:35:40.359 INFO:teuthology.orchestra.run.smithi001.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.6 MB/s | 427 kB 00:00 2024-04-26T01:35:40.384 INFO:teuthology.orchestra.run.smithi001.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 241 kB/s | 20 kB 00:00 2024-04-26T01:35:40.449 INFO:teuthology.orchestra.run.smithi001.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-04-26T01:35:40.527 INFO:teuthology.orchestra.run.smithi001.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-04-26T01:35:40.553 INFO:teuthology.orchestra.run.smithi097.stdout:(152/152): thrift-0.15.0-2.el9.x86_64.rpm 819 kB/s | 1.6 MB 00:01 2024-04-26T01:35:40.560 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-04-26T01:35:40.561 INFO:teuthology.orchestra.run.smithi097.stdout:Total 12 MB/s | 222 MB 00:17 2024-04-26T01:35:40.635 INFO:teuthology.orchestra.run.smithi001.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 13 MB/s | 1.3 MB 00:00 2024-04-26T01:35:40.744 INFO:teuthology.orchestra.run.smithi001.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-04-26T01:35:40.839 INFO:teuthology.orchestra.run.smithi173.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T01:35:40.840 INFO:teuthology.orchestra.run.smithi173.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T01:35:40.850 INFO:teuthology.orchestra.run.smithi173.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T01:35:40.851 INFO:teuthology.orchestra.run.smithi173.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T01:35:40.852 INFO:teuthology.orchestra.run.smithi173.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T01:35:40.853 INFO:teuthology.orchestra.run.smithi173.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T01:35:40.854 INFO:teuthology.orchestra.run.smithi173.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T01:35:40.856 INFO:teuthology.orchestra.run.smithi173.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T01:35:40.857 INFO:teuthology.orchestra.run.smithi173.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T01:35:40.917 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-26T01:35:40.943 INFO:teuthology.orchestra.run.smithi173.stdout:======================================================================================= 2024-04-26T01:35:40.943 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-04-26T01:35:40.943 INFO:teuthology.orchestra.run.smithi173.stdout:======================================================================================= 2024-04-26T01:35:40.943 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-04-26T01:35:40.943 INFO:teuthology.orchestra.run.smithi173.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T01:35:40.943 INFO:teuthology.orchestra.run.smithi173.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-26T01:35:40.943 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-26T01:35:40.943 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-26T01:35:40.944 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-26T01:35:40.944 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-26T01:35:40.944 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-26T01:35:40.944 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-26T01:35:40.944 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-26T01:35:40.944 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-26T01:35:40.944 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-26T01:35:40.944 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-26T01:35:40.944 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-26T01:35:40.944 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-26T01:35:40.944 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-26T01:35:40.945 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-26T01:35:40.945 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-26T01:35:40.945 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T01:35:40.945 INFO:teuthology.orchestra.run.smithi173.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T01:35:40.945 INFO:teuthology.orchestra.run.smithi173.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T01:35:40.945 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-26T01:35:40.945 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-26T01:35:40.945 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-26T01:35:40.945 INFO:teuthology.orchestra.run.smithi173.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T01:35:40.945 INFO:teuthology.orchestra.run.smithi173.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T01:35:40.945 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T01:35:40.945 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T01:35:40.945 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T01:35:40.946 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T01:35:40.946 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-26T01:35:40.946 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-26T01:35:40.946 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-26T01:35:40.946 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-26T01:35:40.946 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-26T01:35:40.946 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-26T01:35:40.946 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-26T01:35:40.946 INFO:teuthology.orchestra.run.smithi173.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T01:35:40.946 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrading: 2024-04-26T01:35:40.946 INFO:teuthology.orchestra.run.smithi173.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-26T01:35:40.946 INFO:teuthology.orchestra.run.smithi173.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-26T01:35:40.947 INFO:teuthology.orchestra.run.smithi173.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T01:35:40.947 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T01:35:40.947 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-04-26T01:35:40.947 INFO:teuthology.orchestra.run.smithi173.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T01:35:40.947 INFO:teuthology.orchestra.run.smithi173.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T01:35:40.947 INFO:teuthology.orchestra.run.smithi173.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T01:35:40.947 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-26T01:35:40.947 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-26T01:35:40.947 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-26T01:35:40.947 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-26T01:35:40.947 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-26T01:35:40.947 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-26T01:35:40.948 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-26T01:35:40.948 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-26T01:35:40.948 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T01:35:40.948 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T01:35:40.948 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T01:35:40.948 INFO:teuthology.orchestra.run.smithi173.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T01:35:40.948 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T01:35:40.948 INFO:teuthology.orchestra.run.smithi173.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T01:35:40.948 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-26T01:35:40.948 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-26T01:35:40.948 INFO:teuthology.orchestra.run.smithi173.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-26T01:35:40.948 INFO:teuthology.orchestra.run.smithi173.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T01:35:40.948 INFO:teuthology.orchestra.run.smithi173.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T01:35:40.948 INFO:teuthology.orchestra.run.smithi173.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T01:35:40.948 INFO:teuthology.orchestra.run.smithi173.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T01:35:40.949 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T01:35:40.949 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-26T01:35:40.949 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T01:35:40.949 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-26T01:35:40.949 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T01:35:40.949 INFO:teuthology.orchestra.run.smithi173.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T01:35:40.949 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T01:35:40.949 INFO:teuthology.orchestra.run.smithi173.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T01:35:40.949 INFO:teuthology.orchestra.run.smithi173.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T01:35:40.949 INFO:teuthology.orchestra.run.smithi173.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T01:35:40.949 INFO:teuthology.orchestra.run.smithi173.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T01:35:40.949 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T01:35:40.949 INFO:teuthology.orchestra.run.smithi173.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-26T01:35:40.950 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T01:35:40.950 INFO:teuthology.orchestra.run.smithi173.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T01:35:40.950 INFO:teuthology.orchestra.run.smithi173.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T01:35:40.950 INFO:teuthology.orchestra.run.smithi173.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T01:35:40.950 INFO:teuthology.orchestra.run.smithi173.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T01:35:40.950 INFO:teuthology.orchestra.run.smithi173.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T01:35:40.950 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T01:35:40.950 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T01:35:40.950 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-26T01:35:40.950 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-26T01:35:40.950 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T01:35:40.950 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T01:35:40.951 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T01:35:40.951 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T01:35:40.951 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T01:35:40.951 INFO:teuthology.orchestra.run.smithi173.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T01:35:40.951 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T01:35:40.951 INFO:teuthology.orchestra.run.smithi173.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T01:35:40.951 INFO:teuthology.orchestra.run.smithi173.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T01:35:40.951 INFO:teuthology.orchestra.run.smithi173.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T01:35:40.951 INFO:teuthology.orchestra.run.smithi173.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T01:35:40.951 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T01:35:40.951 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T01:35:40.951 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T01:35:40.951 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T01:35:40.951 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T01:35:40.952 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T01:35:40.952 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T01:35:40.952 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T01:35:40.952 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T01:35:40.952 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T01:35:40.952 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T01:35:40.952 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T01:35:40.952 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T01:35:40.952 INFO:teuthology.orchestra.run.smithi173.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T01:35:40.952 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T01:35:40.952 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T01:35:40.952 INFO:teuthology.orchestra.run.smithi173.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T01:35:40.952 INFO:teuthology.orchestra.run.smithi173.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T01:35:40.952 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T01:35:40.953 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T01:35:40.953 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T01:35:40.953 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T01:35:40.953 INFO:teuthology.orchestra.run.smithi173.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T01:35:40.953 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T01:35:40.953 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T01:35:40.953 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T01:35:40.953 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T01:35:40.953 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T01:35:40.953 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T01:35:40.953 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T01:35:40.953 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T01:35:40.953 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T01:35:40.953 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T01:35:40.954 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T01:35:40.954 INFO:teuthology.orchestra.run.smithi173.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T01:35:40.954 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T01:35:40.954 INFO:teuthology.orchestra.run.smithi173.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T01:35:40.954 INFO:teuthology.orchestra.run.smithi173.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T01:35:40.954 INFO:teuthology.orchestra.run.smithi173.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T01:35:40.954 INFO:teuthology.orchestra.run.smithi173.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T01:35:40.954 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T01:35:40.954 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T01:35:40.954 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T01:35:40.954 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T01:35:40.954 INFO:teuthology.orchestra.run.smithi173.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T01:35:40.954 INFO:teuthology.orchestra.run.smithi173.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T01:35:40.955 INFO:teuthology.orchestra.run.smithi173.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T01:35:40.955 INFO:teuthology.orchestra.run.smithi173.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T01:35:40.955 INFO:teuthology.orchestra.run.smithi173.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T01:35:40.955 INFO:teuthology.orchestra.run.smithi173.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T01:35:40.955 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T01:35:40.955 INFO:teuthology.orchestra.run.smithi173.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T01:35:40.955 INFO:teuthology.orchestra.run.smithi173.stdout:Installing weak dependencies: 2024-04-26T01:35:40.955 INFO:teuthology.orchestra.run.smithi173.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T01:35:40.955 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-26T01:35:40.955 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-04-26T01:35:40.955 INFO:teuthology.orchestra.run.smithi173.stdout:======================================================================================= 2024-04-26T01:35:40.955 INFO:teuthology.orchestra.run.smithi173.stdout:Install 148 Packages 2024-04-26T01:35:40.955 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrade 4 Packages 2024-04-26T01:35:40.955 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-26T01:35:40.956 INFO:teuthology.orchestra.run.smithi001.stdout:(151/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 6.4 MB/s | 3.1 MB 00:00 2024-04-26T01:35:40.956 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 222 M 2024-04-26T01:35:40.956 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-04-26T01:35:41.027 INFO:teuthology.orchestra.run.smithi001.stdout:(152/152): librados2-19.0.0-3156.gd69f13d8.el7. 5.3 MB/s | 3.4 MB 00:00 2024-04-26T01:35:41.045 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2024-04-26T01:35:41.045 INFO:teuthology.orchestra.run.smithi001.stdout:Total 15 MB/s | 222 MB 00:15 2024-04-26T01:35:41.649 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-04-26T01:35:41.698 INFO:teuthology.orchestra.run.smithi173.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 33 kB/s | 6.6 kB 00:00 2024-04-26T01:35:41.794 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-04-26T01:35:41.794 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-04-26T01:35:42.023 INFO:teuthology.orchestra.run.smithi173.stdout:(2/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 10 MB/s | 5.4 MB 00:00 2024-04-26T01:35:42.057 INFO:teuthology.orchestra.run.smithi173.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 4.3 MB/s | 145 kB 00:00 2024-04-26T01:35:42.082 INFO:teuthology.orchestra.run.smithi173.stdout:(4/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.3 MB/s | 890 kB 00:00 2024-04-26T01:35:42.207 INFO:teuthology.orchestra.run.smithi173.stdout:(5/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 16 MB/s | 2.4 MB 00:00 2024-04-26T01:35:42.475 INFO:teuthology.orchestra.run.smithi173.stdout:(6/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 18 MB/s | 4.7 MB 00:00 2024-04-26T01:35:42.516 INFO:teuthology.orchestra.run.smithi173.stdout:(7/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.4 MB/s | 1.5 MB 00:00 2024-04-26T01:35:43.358 INFO:teuthology.orchestra.run.smithi173.stdout:(8/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 19 MB/s | 17 MB 00:00 2024-04-26T01:35:43.417 INFO:teuthology.orchestra.run.smithi173.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 431 kB/s | 25 kB 00:00 2024-04-26T01:35:43.561 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-04-26T01:35:43.562 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-04-26T01:35:43.950 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2024-04-26T01:35:44.198 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2024-04-26T01:35:44.198 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2024-04-26T01:35:45.626 INFO:teuthology.orchestra.run.smithi173.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 3.4 MB/s | 11 MB 00:03 2024-04-26T01:35:46.043 INFO:teuthology.orchestra.run.smithi173.stdout:(11/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 19 MB/s | 49 MB 00:02 2024-04-26T01:35:46.068 INFO:teuthology.orchestra.run.smithi173.stdout:(12/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 509 kB/s | 225 kB 00:00 2024-04-26T01:35:46.093 INFO:teuthology.orchestra.run.smithi173.stdout:(13/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 541 kB/s | 27 kB 00:00 2024-04-26T01:35:46.118 INFO:teuthology.orchestra.run.smithi173.stdout:(14/152): libcephfs-devel-19.0.0-3156.gd69f13d8 635 kB/s | 32 kB 00:00 2024-04-26T01:35:46.177 INFO:teuthology.orchestra.run.smithi173.stdout:(15/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 8.8 MB/s | 747 kB 00:00 2024-04-26T01:35:46.227 INFO:teuthology.orchestra.run.smithi173.stdout:(16/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.5 MB/s | 163 kB 00:00 2024-04-26T01:35:46.253 INFO:teuthology.orchestra.run.smithi173.stdout:(17/152): librados-devel-19.0.0-3156.gd69f13d8. 1.6 MB/s | 126 kB 00:00 2024-04-26T01:35:46.336 INFO:teuthology.orchestra.run.smithi173.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 4.5 MB/s | 500 kB 00:00 2024-04-26T01:35:46.420 INFO:teuthology.orchestra.run.smithi173.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 541 kB/s | 45 kB 00:00 2024-04-26T01:35:46.554 INFO:teuthology.orchestra.run.smithi173.stdout:(20/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 18 MB/s | 5.4 MB 00:00 2024-04-26T01:35:46.579 INFO:teuthology.orchestra.run.smithi173.stdout:(21/152): python3-ceph-common-19.0.0-3156.gd69f 851 kB/s | 135 kB 00:00 2024-04-26T01:35:46.604 INFO:teuthology.orchestra.run.smithi173.stdout:(22/152): python3-cephfs-19.0.0-3156.gd69f13d8. 3.2 MB/s | 161 kB 00:00 2024-04-26T01:35:46.654 INFO:teuthology.orchestra.run.smithi173.stdout:(23/152): python3-rados-19.0.0-3156.gd69f13d8.e 4.2 MB/s | 320 kB 00:00 2024-04-26T01:35:46.688 INFO:teuthology.orchestra.run.smithi173.stdout:(24/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 3.5 MB/s | 298 kB 00:00 2024-04-26T01:35:46.713 INFO:teuthology.orchestra.run.smithi173.stdout:(25/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.7 MB/s | 99 kB 00:00 2024-04-26T01:35:46.730 INFO:teuthology.orchestra.run.smithi173.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.0 MB/s | 85 kB 00:00 2024-04-26T01:35:46.780 INFO:teuthology.orchestra.run.smithi173.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 3.3 MB/s | 170 kB 00:00 2024-04-26T01:35:46.822 INFO:teuthology.orchestra.run.smithi173.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 823 kB/s | 34 kB 00:00 2024-04-26T01:35:46.873 INFO:teuthology.orchestra.run.smithi173.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 2.8 MB/s | 142 kB 00:00 2024-04-26T01:35:46.979 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2024-04-26T01:35:46.980 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2024-04-26T01:35:47.090 INFO:teuthology.orchestra.run.smithi173.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 16 MB/s | 3.5 MB 00:00 2024-04-26T01:35:47.348 INFO:teuthology.orchestra.run.smithi173.stdout:(31/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 4.9 MB/s | 3.1 MB 00:00 2024-04-26T01:35:47.482 INFO:teuthology.orchestra.run.smithi173.stdout:(32/152): ceph-mgr-diskprediction-local-19.0.0- 19 MB/s | 7.4 MB 00:00 2024-04-26T01:35:47.507 INFO:teuthology.orchestra.run.smithi173.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3156.gd6 1.5 MB/s | 249 kB 00:00 2024-04-26T01:35:47.532 INFO:teuthology.orchestra.run.smithi173.stdout:(34/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 986 kB/s | 49 kB 00:00 2024-04-26T01:35:47.557 INFO:teuthology.orchestra.run.smithi173.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3156.gd 328 kB/s | 16 kB 00:00 2024-04-26T01:35:47.585 INFO:teuthology.orchestra.run.smithi173.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 5.1 MB/s | 273 kB 00:00 2024-04-26T01:35:47.627 INFO:teuthology.orchestra.run.smithi173.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 441 kB/s | 18 kB 00:00 2024-04-26T01:35:47.694 INFO:teuthology.orchestra.run.smithi173.stdout:(38/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 5.5 MB/s | 762 kB 00:00 2024-04-26T01:35:47.752 INFO:teuthology.orchestra.run.smithi173.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 641 kB/s | 80 kB 00:00 2024-04-26T01:35:47.811 INFO:teuthology.orchestra.run.smithi173.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.8 MB/s | 803 kB 00:00 2024-04-26T01:35:47.836 INFO:teuthology.orchestra.run.smithi173.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-04-26T01:35:47.862 INFO:teuthology.orchestra.run.smithi173.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-26T01:35:47.904 INFO:teuthology.orchestra.run.smithi173.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-26T01:35:47.929 INFO:teuthology.orchestra.run.smithi173.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-26T01:35:47.954 INFO:teuthology.orchestra.run.smithi173.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-26T01:35:47.979 INFO:teuthology.orchestra.run.smithi173.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-26T01:35:48.005 INFO:teuthology.orchestra.run.smithi173.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-26T01:35:48.030 INFO:teuthology.orchestra.run.smithi173.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 14 MB/s | 689 kB 00:00 2024-04-26T01:35:48.064 INFO:teuthology.orchestra.run.smithi173.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 943 kB 00:00 2024-04-26T01:35:48.089 INFO:teuthology.orchestra.run.smithi173.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-26T01:35:48.115 INFO:teuthology.orchestra.run.smithi173.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 432 kB/s | 21 kB 00:00 2024-04-26T01:35:48.140 INFO:teuthology.orchestra.run.smithi173.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-26T01:35:48.164 INFO:teuthology.orchestra.run.smithi173.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 661 kB/s | 33 kB 00:00 2024-04-26T01:35:48.207 INFO:teuthology.orchestra.run.smithi173.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 454 kB/s | 18 kB 00:00 2024-04-26T01:35:48.257 INFO:teuthology.orchestra.run.smithi173.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-04-26T01:35:48.282 INFO:teuthology.orchestra.run.smithi173.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 152 kB/s | 11 kB 00:00 2024-04-26T01:35:48.308 INFO:teuthology.orchestra.run.smithi173.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-26T01:35:48.334 INFO:teuthology.orchestra.run.smithi173.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 906 kB/s | 45 kB 00:00 2024-04-26T01:35:48.375 INFO:teuthology.orchestra.run.smithi173.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-04-26T01:35:48.408 INFO:teuthology.orchestra.run.smithi173.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 7.7 MB/s | 582 kB 00:00 2024-04-26T01:35:48.433 INFO:teuthology.orchestra.run.smithi173.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 443 kB/s | 25 kB 00:00 2024-04-26T01:35:48.450 INFO:teuthology.orchestra.run.smithi173.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-26T01:35:48.476 INFO:teuthology.orchestra.run.smithi173.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.8 MB/s | 292 kB 00:00 2024-04-26T01:35:48.509 INFO:teuthology.orchestra.run.smithi173.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-26T01:35:48.535 INFO:teuthology.orchestra.run.smithi173.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 603 kB/s | 35 kB 00:00 2024-04-26T01:35:48.685 INFO:teuthology.orchestra.run.smithi173.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-04-26T01:35:48.752 INFO:teuthology.orchestra.run.smithi173.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-04-26T01:35:48.794 INFO:teuthology.orchestra.run.smithi173.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-26T01:35:48.827 INFO:teuthology.orchestra.run.smithi173.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-26T01:35:48.854 INFO:teuthology.orchestra.run.smithi173.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 133 kB 00:00 2024-04-26T01:35:48.928 INFO:teuthology.orchestra.run.smithi173.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 14 MB/s | 1.0 MB 00:00 2024-04-26T01:35:48.970 INFO:teuthology.orchestra.run.smithi173.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 2.2 MB/s | 87 kB 00:00 2024-04-26T01:35:49.045 INFO:teuthology.orchestra.run.smithi173.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 27 MB/s | 5.8 MB 00:00 2024-04-26T01:35:49.112 INFO:teuthology.orchestra.run.smithi173.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.7 MB/s | 244 kB 00:00 2024-04-26T01:35:49.154 INFO:teuthology.orchestra.run.smithi173.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 6.2 MB/s | 248 kB 00:00 2024-04-26T01:35:49.230 INFO:teuthology.orchestra.run.smithi173.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 6.0 MB 00:00 2024-04-26T01:35:49.255 INFO:teuthology.orchestra.run.smithi173.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 257 kB/s | 26 kB 00:00 2024-04-26T01:35:49.280 INFO:teuthology.orchestra.run.smithi173.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-04-26T01:35:49.305 INFO:teuthology.orchestra.run.smithi173.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-26T01:35:49.330 INFO:teuthology.orchestra.run.smithi173.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-04-26T01:35:49.373 INFO:teuthology.orchestra.run.smithi173.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 6.1 MB/s | 250 kB 00:00 2024-04-26T01:35:49.415 INFO:teuthology.orchestra.run.smithi173.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 5.5 MB/s | 222 kB 00:00 2024-04-26T01:35:49.481 INFO:teuthology.orchestra.run.smithi173.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-04-26T01:35:49.487 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-04-26T01:35:49.507 INFO:teuthology.orchestra.run.smithi173.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 844 kB/s | 77 kB 00:00 2024-04-26T01:35:49.532 INFO:teuthology.orchestra.run.smithi173.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-04-26T01:35:49.557 INFO:teuthology.orchestra.run.smithi173.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-26T01:35:49.583 INFO:teuthology.orchestra.run.smithi173.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-26T01:35:49.608 INFO:teuthology.orchestra.run.smithi173.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-26T01:35:49.624 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T01:35:49.633 INFO:teuthology.orchestra.run.smithi173.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-26T01:35:49.658 INFO:teuthology.orchestra.run.smithi173.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-26T01:35:49.692 INFO:teuthology.orchestra.run.smithi173.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 1.3 MB/s | 42 kB 00:00 2024-04-26T01:35:49.742 INFO:teuthology.orchestra.run.smithi173.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 909 kB/s | 44 kB 00:00 2024-04-26T01:35:49.793 INFO:teuthology.orchestra.run.smithi173.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.5 MB/s | 221 kB 00:00 2024-04-26T01:35:49.857 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T01:35:49.909 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T01:35:49.958 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T01:35:50.000 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T01:35:50.010 INFO:teuthology.orchestra.run.smithi173.stdout:(94/152): ceph-common-19.0.0-3156.gd69f13d8.el7 2.5 MB/s | 21 MB 00:08 2024-04-26T01:35:50.035 INFO:teuthology.orchestra.run.smithi173.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 149 kB/s | 36 kB 00:00 2024-04-26T01:35:50.053 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T01:35:50.244 INFO:teuthology.orchestra.run.smithi173.stdout:(96/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-04-26T01:35:50.269 INFO:teuthology.orchestra.run.smithi173.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 184 kB/s | 47 kB 00:00 2024-04-26T01:35:50.294 INFO:teuthology.orchestra.run.smithi173.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 305 kB 00:00 2024-04-26T01:35:50.311 INFO:teuthology.orchestra.run.smithi173.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 874 kB/s | 58 kB 00:00 2024-04-26T01:35:50.336 INFO:teuthology.orchestra.run.smithi173.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 2.2 MB/s | 94 kB 00:00 2024-04-26T01:35:50.354 INFO:teuthology.orchestra.run.smithi173.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-26T01:35:50.370 INFO:teuthology.orchestra.run.smithi173.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 763 kB/s | 45 kB 00:00 2024-04-26T01:35:50.387 INFO:teuthology.orchestra.run.smithi173.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 668 kB/s | 22 kB 00:00 2024-04-26T01:35:50.388 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T01:35:50.421 INFO:teuthology.orchestra.run.smithi173.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-04-26T01:35:50.479 INFO:teuthology.orchestra.run.smithi173.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-04-26T01:35:50.501 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:35:50.538 INFO:teuthology.orchestra.run.smithi173.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 2.0 MB/s | 309 kB 00:00 2024-04-26T01:35:50.571 INFO:teuthology.orchestra.run.smithi173.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 777 kB/s | 26 kB 00:00 2024-04-26T01:35:50.606 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:35:50.613 INFO:teuthology.orchestra.run.smithi173.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-04-26T01:35:50.635 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:35:50.647 INFO:teuthology.orchestra.run.smithi173.stdout:(109/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-04-26T01:35:50.672 INFO:teuthology.orchestra.run.smithi173.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-26T01:35:50.697 INFO:teuthology.orchestra.run.smithi173.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-04-26T01:35:50.726 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:35:50.757 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-26T01:35:50.798 INFO:teuthology.orchestra.run.smithi173.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.5 MB/s | 838 kB 00:00 2024-04-26T01:35:50.823 INFO:teuthology.orchestra.run.smithi173.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-04-26T01:35:50.848 INFO:teuthology.orchestra.run.smithi173.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 865 kB/s | 43 kB 00:00 2024-04-26T01:35:50.873 INFO:teuthology.orchestra.run.smithi173.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-04-26T01:35:50.899 INFO:teuthology.orchestra.run.smithi173.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-04-26T01:35:50.909 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T01:35:50.941 INFO:teuthology.orchestra.run.smithi173.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-26T01:35:50.962 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T01:35:50.966 INFO:teuthology.orchestra.run.smithi173.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-04-26T01:35:50.990 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:35:51.008 INFO:teuthology.orchestra.run.smithi173.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 847 kB/s | 35 kB 00:00 2024-04-26T01:35:51.050 INFO:teuthology.orchestra.run.smithi173.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-26T01:35:51.075 INFO:teuthology.orchestra.run.smithi173.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-04-26T01:35:51.094 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:35:51.116 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-26T01:35:51.142 INFO:teuthology.orchestra.run.smithi173.stdout:(122/152): libarrow-9.0.0-10.el9.x86_64.rpm 6.7 MB/s | 4.4 MB 00:00 2024-04-26T01:35:51.167 INFO:teuthology.orchestra.run.smithi173.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-26T01:35:51.178 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-26T01:35:51.184 INFO:teuthology.orchestra.run.smithi173.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 255 kB/s | 11 kB 00:00 2024-04-26T01:35:51.200 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T01:35:51.209 INFO:teuthology.orchestra.run.smithi173.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 424 kB/s | 18 kB 00:00 2024-04-26T01:35:51.220 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T01:35:51.235 INFO:teuthology.orchestra.run.smithi173.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-04-26T01:35:51.260 INFO:teuthology.orchestra.run.smithi173.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-04-26T01:35:51.286 INFO:teuthology.orchestra.run.smithi173.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-04-26T01:35:51.328 INFO:teuthology.orchestra.run.smithi173.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-04-26T01:35:51.369 INFO:teuthology.orchestra.run.smithi173.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 1.9 MB/s | 79 kB 00:00 2024-04-26T01:35:51.383 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T01:35:51.411 INFO:teuthology.orchestra.run.smithi173.stdout:(131/152): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-04-26T01:35:51.416 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T01:35:51.496 INFO:teuthology.orchestra.run.smithi173.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 4.3 MB/s | 1.0 MB 00:00 2024-04-26T01:35:51.523 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T01:35:51.528 INFO:teuthology.orchestra.run.smithi173.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-26T01:35:51.562 INFO:teuthology.orchestra.run.smithi173.stdout:(134/152): python3-grpcio-1.46.7-10.el9.x86_64. 4.0 MB/s | 2.0 MB 00:00 2024-04-26T01:35:51.588 INFO:teuthology.orchestra.run.smithi173.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 180 kB/s | 16 kB 00:00 2024-04-26T01:35:51.613 INFO:teuthology.orchestra.run.smithi173.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-04-26T01:35:51.620 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-26T01:35:51.638 INFO:teuthology.orchestra.run.smithi173.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-04-26T01:35:51.657 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T01:35:51.663 INFO:teuthology.orchestra.run.smithi173.stdout:(138/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 670 kB/s | 90 kB 00:00 2024-04-26T01:35:51.689 INFO:teuthology.orchestra.run.smithi173.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-26T01:35:51.692 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T01:35:51.714 INFO:teuthology.orchestra.run.smithi173.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-04-26T01:35:51.739 INFO:teuthology.orchestra.run.smithi173.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-04-26T01:35:51.757 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T01:35:51.764 INFO:teuthology.orchestra.run.smithi173.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-04-26T01:35:51.791 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T01:35:51.815 INFO:teuthology.orchestra.run.smithi173.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-26T01:35:51.834 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T01:35:51.840 INFO:teuthology.orchestra.run.smithi173.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-26T01:35:51.857 INFO:teuthology.orchestra.run.smithi173.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 479 kB/s | 20 kB 00:00 2024-04-26T01:35:51.867 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T01:35:51.890 INFO:teuthology.orchestra.run.smithi173.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-26T01:35:51.895 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T01:35:51.915 INFO:teuthology.orchestra.run.smithi173.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-26T01:35:51.964 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T01:35:52.034 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T01:35:52.049 INFO:teuthology.orchestra.run.smithi173.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 8.2 MB/s | 1.6 MB 00:00 2024-04-26T01:35:52.064 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T01:35:52.099 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T01:35:52.116 INFO:teuthology.orchestra.run.smithi173.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 20 MB/s | 1.3 MB 00:00 2024-04-26T01:35:52.176 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T01:35:52.208 INFO:teuthology.orchestra.run.smithi173.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 26 MB/s | 2.4 MB 00:00 2024-04-26T01:35:52.211 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-26T01:35:52.254 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-26T01:35:52.258 INFO:teuthology.orchestra.run.smithi173.stdout:(151/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 9.2 MB/s | 3.1 MB 00:00 2024-04-26T01:35:52.299 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-26T01:35:52.375 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T01:35:52.458 INFO:teuthology.orchestra.run.smithi173.stdout:(152/152): librados2-19.0.0-3156.gd69f13d8.el7. 6.0 MB/s | 3.4 MB 00:00 2024-04-26T01:35:52.463 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-04-26T01:35:52.463 INFO:teuthology.orchestra.run.smithi173.stdout:Total 19 MB/s | 222 MB 00:11 2024-04-26T01:35:52.590 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T01:35:52.665 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T01:35:52.810 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T01:35:52.875 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T01:35:53.408 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T01:35:53.496 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T01:35:53.511 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2024-04-26T01:35:53.533 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T01:35:53.555 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-04-26T01:35:53.575 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T01:35:53.614 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-26T01:35:53.629 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T01:35:53.655 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T01:35:53.697 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-04-26T01:35:53.697 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-04-26T01:35:53.704 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T01:35:53.816 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T01:35:53.834 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T01:35:53.875 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T01:35:53.920 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T01:35:53.930 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T01:35:53.962 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T01:35:54.004 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T01:35:54.042 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T01:35:54.087 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T01:35:54.098 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T01:35:54.134 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T01:35:54.170 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T01:35:54.213 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T01:35:54.255 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T01:35:54.393 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T01:35:54.435 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T01:35:54.465 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T01:35:54.507 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T01:35:54.539 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T01:35:54.602 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T01:35:54.640 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T01:35:54.642 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:35:54.680 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T01:35:54.706 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T01:35:54.731 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-26T01:35:54.806 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T01:35:54.821 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:35:54.852 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:35:54.933 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T01:35:54.963 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T01:35:54.991 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T01:35:55.001 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:35:55.026 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T01:35:55.035 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-26T01:35:55.064 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T01:35:55.096 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T01:35:55.134 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T01:35:55.169 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T01:35:55.202 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T01:35:55.244 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T01:35:55.278 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T01:35:55.366 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T01:35:55.374 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T01:35:55.409 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T01:35:55.440 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T01:35:55.444 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T01:35:55.460 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-04-26T01:35:55.461 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-04-26T01:35:55.474 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:35:55.487 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T01:35:55.594 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T01:35:55.638 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:35:55.658 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T01:35:55.701 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-26T01:35:55.803 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-26T01:35:55.835 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T01:35:55.904 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T01:35:56.239 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T01:35:56.303 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T01:35:56.390 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T01:35:56.450 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T01:35:56.489 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T01:35:56.507 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T01:35:56.590 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T01:35:56.641 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-26T01:35:56.655 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T01:35:56.686 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T01:35:56.735 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T01:35:56.752 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T01:35:56.791 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T01:35:56.842 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T01:35:56.898 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T01:35:56.964 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T01:35:56.967 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T01:35:56.996 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T01:35:57.008 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T01:35:57.042 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T01:35:57.074 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T01:35:57.160 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T01:35:57.235 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T01:35:57.269 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T01:35:57.329 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T01:35:57.447 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T01:35:57.497 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-26T01:35:57.549 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-26T01:35:57.588 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-26T01:35:57.703 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T01:35:57.726 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T01:35:57.776 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T01:35:57.829 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T01:35:57.892 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T01:35:58.087 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T01:35:58.153 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T01:35:58.889 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T01:35:59.027 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T01:35:59.068 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T01:35:59.077 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T01:35:59.115 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T01:35:59.163 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-26T01:35:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T01:35:59.195 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T01:35:59.225 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T01:35:59.228 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T01:35:59.541 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T01:35:59.556 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T01:35:59.582 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-26T01:35:59.623 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T01:35:59.673 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-26T01:35:59.704 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-26T01:35:59.706 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T01:35:59.750 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T01:35:59.808 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T01:35:59.846 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T01:35:59.877 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T01:35:59.945 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T01:35:59.984 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T01:36:00.046 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T01:36:00.105 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T01:36:00.228 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-26T01:36:00.260 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-26T01:36:00.331 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-26T01:36:00.342 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T01:36:00.353 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-26T01:36:00.400 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T01:36:00.438 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T01:36:00.496 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T01:36:00.638 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T01:36:00.686 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T01:36:00.735 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T01:36:00.768 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T01:36:00.809 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-26T01:36:01.032 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T01:36:01.222 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T01:36:01.266 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T01:36:01.306 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T01:36:01.352 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T01:36:01.422 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T01:36:01.488 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T01:36:01.493 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-04-26T01:36:01.539 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T01:36:01.587 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T01:36:01.592 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T01:36:01.634 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T01:36:01.641 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T01:36:01.687 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T01:36:01.718 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T01:36:01.721 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T01:36:01.746 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T01:36:01.763 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T01:36:01.789 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T01:36:01.912 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T01:36:01.994 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T01:36:02.039 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T01:36:02.111 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T01:36:02.141 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T01:36:02.264 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:36:02.273 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T01:36:02.343 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T01:36:02.388 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:36:02.415 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:36:02.507 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:36:02.536 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-26T01:36:02.650 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:36:02.690 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T01:36:02.691 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:36:02.742 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T01:36:02.769 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:36:02.874 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:36:02.895 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-26T01:36:02.958 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-26T01:36:02.988 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T01:36:03.011 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T01:36:03.188 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T01:36:03.221 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T01:36:03.234 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:36:03.278 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-26T01:36:03.328 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T01:36:03.363 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T01:36:03.375 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-26T01:36:03.375 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T01:36:03.375 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T01:36:03.375 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T01:36:03.405 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-26T01:36:03.424 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-26T01:36:03.425 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T01:36:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T01:36:03.462 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T01:36:03.505 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T01:36:03.570 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T01:36:03.594 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T01:36:03.604 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T01:36:03.647 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T01:36:03.671 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T01:36:03.680 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T01:36:03.708 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T01:36:03.778 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T01:36:03.780 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T01:36:03.826 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T01:36:03.830 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T01:36:03.860 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T01:36:03.896 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T01:36:03.972 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T01:36:04.006 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-26T01:36:04.050 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-26T01:36:04.074 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T01:36:04.078 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-26T01:36:04.106 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T01:36:04.150 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T01:36:04.210 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T01:36:04.215 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T01:36:04.285 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T01:36:04.420 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T01:36:04.482 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T01:36:05.011 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T01:36:05.093 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T01:36:05.125 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T01:36:05.129 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T01:36:05.171 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T01:36:05.210 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-26T01:36:05.252 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T01:36:05.277 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T01:36:05.420 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T01:36:05.479 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T01:36:05.534 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T01:36:05.608 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T01:36:05.638 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T01:36:05.666 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T01:36:05.713 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T01:36:05.749 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T01:36:05.791 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T01:36:05.834 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T01:36:05.971 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T01:36:06.023 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T01:36:06.053 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T01:36:06.094 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T01:36:06.189 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T01:36:06.227 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T01:36:06.267 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T01:36:06.293 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T01:36:06.318 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-26T01:36:06.393 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T01:36:06.520 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T01:36:06.550 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T01:36:06.578 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T01:36:06.612 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T01:36:06.651 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T01:36:06.683 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T01:36:06.713 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T01:36:06.749 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T01:36:06.780 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T01:36:06.822 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T01:36:06.882 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T01:36:06.969 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T01:36:06.977 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T01:36:07.012 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T01:36:07.043 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T01:36:07.079 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T01:36:07.090 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T01:36:07.119 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T01:36:07.165 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T01:36:07.198 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T01:36:07.245 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T01:36:07.567 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T01:36:07.616 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-26T01:36:07.741 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-26T01:36:07.774 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-26T01:36:07.837 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T01:36:07.977 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T01:36:08.033 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T01:36:08.075 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T01:36:08.178 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T01:36:08.242 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T01:36:08.331 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T01:36:08.369 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T01:36:08.432 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-26T01:36:08.469 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-26T01:36:08.547 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T01:36:08.570 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-26T01:36:08.575 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T01:36:08.596 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-26T01:36:08.653 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T01:36:09.308 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T01:36:09.424 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T01:36:10.671 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T01:36:10.753 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T01:36:10.781 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T01:36:10.814 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T01:36:11.126 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T01:36:11.169 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-26T01:36:11.243 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-26T01:36:11.274 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-26T01:36:11.310 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T01:36:11.347 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:36:11.384 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:36:11.789 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-26T01:36:11.814 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-26T01:36:11.885 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-26T01:36:11.907 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-26T01:36:12.062 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:36:12.104 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-26T01:36:12.255 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-26T01:36:12.256 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T01:36:12.256 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T01:36:12.256 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:36:12.296 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-26T01:36:14.189 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:36:14.228 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:36:14.767 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:36:14.806 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-26T01:36:14.865 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-26T01:36:14.865 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2024-04-26T01:36:14.865 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2024-04-26T01:36:14.865 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2024-04-26T01:36:14.865 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2024-04-26T01:36:14.865 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2024-04-26T01:36:14.865 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /root 2024-04-26T01:36:14.865 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2024-04-26T01:36:14.865 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T01:36:14.880 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-26T01:36:14.880 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T01:36:14.880 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T01:36:14.880 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-26T01:36:14.908 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-26T01:36:15.042 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-26T01:36:15.057 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-26T01:36:15.058 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:15.058 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T01:36:15.058 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T01:36:16.528 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-26T01:36:16.553 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-26T01:36:16.553 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:16.553 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T01:36:16.554 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T01:36:16.554 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T01:36:16.554 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T01:36:16.779 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-26T01:36:16.801 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-26T01:36:16.802 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:16.802 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T01:36:16.802 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T01:36:16.802 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T01:36:16.802 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T01:36:17.238 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-26T01:36:17.260 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-26T01:36:17.261 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:17.261 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T01:36:17.261 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T01:36:17.261 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T01:36:17.261 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T01:36:17.368 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T01:36:17.416 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T01:36:17.471 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T01:36:17.657 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T01:36:17.705 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T01:36:17.783 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T01:36:17.873 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T01:36:18.014 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T01:36:18.055 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T01:36:18.079 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T01:36:18.098 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T01:36:18.499 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-26T01:36:18.530 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-26T01:36:18.562 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-26T01:36:18.615 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-26T01:36:18.669 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-26T01:36:19.707 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-26T01:36:19.738 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-26T01:36:20.704 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-26T01:36:20.730 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-26T01:36:20.833 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-26T01:36:20.993 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-26T01:36:21.023 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-26T01:36:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-26T01:36:21.050 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:21.050 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T01:36:21.050 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T01:36:21.050 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T01:36:21.050 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T01:36:21.092 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-26T01:36:21.102 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-26T01:36:21.204 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-26T01:36:21.256 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-26T01:36:22.180 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T01:36:22.218 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-26T01:36:22.241 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-26T01:36:22.241 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:22.241 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T01:36:22.241 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T01:36:22.242 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T01:36:22.242 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T01:36:22.285 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-26T01:36:22.307 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-26T01:36:22.307 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:22.307 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T01:36:22.307 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T01:36:22.353 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-26T01:36:22.375 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-26T01:36:22.375 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:22.375 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T01:36:22.375 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T01:36:22.375 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T01:36:22.375 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T01:36:22.670 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-26T01:36:22.693 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-26T01:36:22.693 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:22.693 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T01:36:22.693 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T01:36:22.694 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T01:36:22.694 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T01:36:25.833 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-26T01:36:25.833 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /sys 2024-04-26T01:36:25.833 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /proc 2024-04-26T01:36:25.834 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /mnt 2024-04-26T01:36:25.834 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /var/tmp 2024-04-26T01:36:25.834 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /home 2024-04-26T01:36:25.834 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /root 2024-04-26T01:36:25.834 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /tmp 2024-04-26T01:36:25.834 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:36:26.070 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-26T01:36:26.122 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-26T01:36:26.122 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:26.122 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T01:36:26.122 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:36:26.743 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-26T01:36:26.743 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /sys 2024-04-26T01:36:26.743 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /proc 2024-04-26T01:36:26.743 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /mnt 2024-04-26T01:36:26.743 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /var/tmp 2024-04-26T01:36:26.743 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /home 2024-04-26T01:36:26.744 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /root 2024-04-26T01:36:26.744 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /tmp 2024-04-26T01:36:26.744 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-26T01:36:26.919 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-26T01:36:26.935 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-26T01:36:26.935 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:26.935 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T01:36:26.935 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-26T01:36:27.291 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-26T01:36:27.332 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-26T01:36:27.373 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-26T01:36:27.405 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T01:36:27.432 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T01:36:27.617 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-26T01:36:27.661 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T01:36:27.691 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-26T01:36:27.812 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-26T01:36:27.904 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-26T01:36:27.905 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:27.905 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T01:36:27.905 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T01:36:27.905 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T01:36:27.905 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:36:28.171 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-26T01:36:28.241 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-26T01:36:28.241 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:28.241 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T01:36:28.241 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T01:36:28.242 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T01:36:28.242 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:36:28.286 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:36:28.330 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:36:28.350 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:36:28.422 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-26T01:36:28.447 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-26T01:36:28.448 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:28.448 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T01:36:28.448 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T01:36:28.448 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T01:36:28.448 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-26T01:36:28.664 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-26T01:36:28.686 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-26T01:36:28.686 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:28.686 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T01:36:28.686 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T01:36:28.687 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T01:36:28.687 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-26T01:36:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-26T01:36:28.839 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-26T01:36:28.839 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:28.839 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T01:36:28.840 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T01:36:28.840 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T01:36:28.840 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:36:28.981 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T01:36:29.046 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T01:36:29.121 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T01:36:29.140 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-26T01:36:29.162 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-26T01:36:29.163 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:29.163 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T01:36:29.163 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T01:36:29.163 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T01:36:29.163 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-26T01:36:29.179 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-26T01:36:29.262 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T01:36:29.270 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T01:36:29.302 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T01:36:29.327 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T01:36:29.382 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T01:36:29.385 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T01:36:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T01:36:29.415 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T01:36:29.416 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T01:36:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T01:36:29.500 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T01:36:29.533 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T01:36:29.533 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T01:36:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T01:36:29.569 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T01:36:29.616 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T01:36:29.616 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T01:36:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T01:36:29.690 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:36:29.694 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T01:36:29.782 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T01:36:29.876 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T01:36:29.923 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T01:36:29.930 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T01:36:29.982 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T01:36:29.992 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T01:36:30.014 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T01:36:30.024 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T01:36:30.042 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T01:36:30.402 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-26T01:36:30.432 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-26T01:36:30.466 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-26T01:36:30.526 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-26T01:36:30.563 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-26T01:36:30.712 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-26T01:36:30.761 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-26T01:36:30.819 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-26T01:36:30.931 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-26T01:36:30.962 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-26T01:36:31.601 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-26T01:36:31.631 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-26T01:36:32.337 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-26T01:36:32.381 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-26T01:36:32.597 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-26T01:36:32.624 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-26T01:36:32.726 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-26T01:36:32.888 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-26T01:36:32.918 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-26T01:36:32.941 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-26T01:36:32.941 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:32.941 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T01:36:32.941 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T01:36:32.941 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T01:36:32.942 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-26T01:36:32.985 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-26T01:36:32.996 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-26T01:36:33.093 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-26T01:36:33.150 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-26T01:36:33.545 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-26T01:36:33.581 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-26T01:36:33.735 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-26T01:36:33.921 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-26T01:36:33.966 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-26T01:36:34.048 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-26T01:36:34.048 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:34.048 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T01:36:34.048 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T01:36:34.048 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T01:36:34.048 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:36:34.076 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T01:36:34.104 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-26T01:36:34.108 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-26T01:36:34.126 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-26T01:36:34.126 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:34.126 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T01:36:34.126 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T01:36:34.126 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T01:36:34.127 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-26T01:36:34.150 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-26T01:36:34.171 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-26T01:36:34.193 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-26T01:36:34.193 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:34.193 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T01:36:34.193 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-26T01:36:34.238 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-26T01:36:34.261 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-26T01:36:34.261 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:34.261 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T01:36:34.261 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T01:36:34.261 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T01:36:34.261 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-26T01:36:34.301 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-26T01:36:34.387 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-26T01:36:34.565 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-26T01:36:34.586 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-26T01:36:34.587 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:34.587 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T01:36:34.587 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T01:36:34.587 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T01:36:34.587 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-26T01:36:35.488 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T01:36:35.519 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-26T01:36:35.593 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-26T01:36:35.593 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:35.593 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T01:36:35.593 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T01:36:35.593 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T01:36:35.594 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:36:35.645 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-26T01:36:35.717 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-26T01:36:35.717 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:35.717 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T01:36:35.717 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:36:35.794 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:36:35.794 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-26T01:36:35.794 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-26T01:36:35.794 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-26T01:36:35.794 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-26T01:36:35.794 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-26T01:36:35.794 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-26T01:36:35.794 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-26T01:36:35.794 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:36:35.794 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:36:35.795 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-26T01:36:35.795 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-26T01:36:35.795 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-26T01:36:35.795 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:36:35.795 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-26T01:36:35.795 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-26T01:36:35.795 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-26T01:36:35.795 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-26T01:36:35.795 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-26T01:36:35.795 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-26T01:36:35.795 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-26T01:36:35.795 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-26T01:36:35.795 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-26T01:36:35.796 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-26T01:36:35.796 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-26T01:36:35.796 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-26T01:36:35.796 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-26T01:36:35.796 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-26T01:36:35.796 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-26T01:36:35.796 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-26T01:36:35.796 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-26T01:36:35.796 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-26T01:36:35.796 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-26T01:36:35.796 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-26T01:36:35.796 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-26T01:36:35.796 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-26T01:36:35.796 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-26T01:36:35.797 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-26T01:36:35.797 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-26T01:36:35.797 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-26T01:36:35.797 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T01:36:35.797 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T01:36:35.797 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T01:36:35.797 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T01:36:35.797 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T01:36:35.797 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T01:36:35.797 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T01:36:35.797 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T01:36:35.797 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T01:36:35.797 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T01:36:35.797 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T01:36:35.798 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T01:36:35.798 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T01:36:35.798 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T01:36:35.798 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T01:36:35.798 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T01:36:35.798 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T01:36:35.798 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T01:36:35.798 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T01:36:35.798 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T01:36:35.798 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T01:36:35.798 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T01:36:35.798 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T01:36:35.798 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T01:36:35.799 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T01:36:35.799 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T01:36:35.799 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T01:36:35.799 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T01:36:35.799 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T01:36:35.799 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T01:36:35.799 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T01:36:35.799 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T01:36:35.799 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T01:36:35.799 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T01:36:35.799 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T01:36:35.799 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T01:36:35.799 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T01:36:35.800 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T01:36:35.800 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T01:36:35.800 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T01:36:35.800 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T01:36:35.800 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T01:36:35.800 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T01:36:35.800 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T01:36:35.800 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T01:36:35.800 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T01:36:35.800 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T01:36:35.800 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T01:36:35.800 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T01:36:35.800 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T01:36:35.800 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T01:36:35.801 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T01:36:35.801 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T01:36:35.801 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T01:36:35.801 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T01:36:35.801 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T01:36:35.801 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T01:36:35.801 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T01:36:35.801 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T01:36:35.801 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T01:36:35.801 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T01:36:35.801 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T01:36:35.801 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T01:36:35.801 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T01:36:35.802 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T01:36:35.802 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T01:36:35.802 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T01:36:35.802 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T01:36:35.802 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-26T01:36:35.802 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-26T01:36:35.802 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T01:36:35.802 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T01:36:35.802 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T01:36:35.802 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-26T01:36:35.802 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T01:36:35.802 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T01:36:35.802 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T01:36:35.802 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T01:36:35.803 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T01:36:35.803 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T01:36:35.803 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T01:36:35.803 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T01:36:35.803 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T01:36:35.803 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T01:36:35.803 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T01:36:35.803 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T01:36:35.803 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T01:36:35.803 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T01:36:35.803 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T01:36:35.803 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T01:36:35.803 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T01:36:35.803 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T01:36:35.804 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T01:36:35.804 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T01:36:35.804 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T01:36:35.804 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T01:36:35.804 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T01:36:35.804 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T01:36:35.804 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T01:36:35.804 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T01:36:35.804 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T01:36:35.804 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T01:36:35.804 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T01:36:35.804 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T01:36:35.804 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T01:36:35.805 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T01:36:35.805 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T01:36:35.805 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T01:36:35.805 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T01:36:35.805 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-26T01:36:35.805 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T01:36:35.805 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-26T01:36:35.805 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T01:36:35.805 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T01:36:35.805 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T01:36:35.805 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T01:36:35.806 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-26T01:36:35.876 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-26T01:36:35.876 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:35.876 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T01:36:35.876 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T01:36:35.876 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T01:36:35.877 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:36:36.280 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-26T01:36:36.363 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-26T01:36:36.363 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:36:36.363 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T01:36:36.363 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T01:36:36.364 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T01:36:36.364 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:36:37.824 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:36:37.825 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T01:36:37.825 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2024-04-26T01:36:37.825 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.825 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.825 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T01:36:37.825 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T01:36:37.825 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-04-26T01:36:37.825 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T01:36:37.825 INFO:teuthology.orchestra.run.smithi097.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T01:36:37.826 INFO:teuthology.orchestra.run.smithi097.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T01:36:37.826 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T01:36:37.826 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.826 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.826 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.826 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.826 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:37.826 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.826 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.826 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.826 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:37.826 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:37.827 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:37.827 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:37.827 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:37.827 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.827 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.827 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:37.827 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.827 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.827 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.827 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:37.827 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:37.827 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.828 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.828 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:37.828 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T01:36:37.828 INFO:teuthology.orchestra.run.smithi097.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T01:36:37.828 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T01:36:37.828 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T01:36:37.828 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T01:36:37.828 INFO:teuthology.orchestra.run.smithi097.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T01:36:37.828 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T01:36:37.828 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T01:36:37.828 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T01:36:37.829 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-26T01:36:37.829 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-26T01:36:37.829 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.829 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.829 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.829 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T01:36:37.829 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T01:36:37.829 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T01:36:37.829 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T01:36:37.829 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T01:36:37.829 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.829 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.829 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T01:36:37.829 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.830 INFO:teuthology.orchestra.run.smithi097.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T01:36:37.830 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T01:36:37.830 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T01:36:37.830 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T01:36:37.830 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T01:36:37.830 INFO:teuthology.orchestra.run.smithi097.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T01:36:37.830 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T01:36:37.830 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T01:36:37.830 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T01:36:37.831 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T01:36:37.831 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T01:36:37.831 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-26T01:36:37.831 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T01:36:37.831 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T01:36:37.831 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T01:36:37.831 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T01:36:37.831 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T01:36:37.831 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T01:36:37.831 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T01:36:37.831 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T01:36:37.831 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T01:36:37.832 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T01:36:37.832 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T01:36:37.832 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.832 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.832 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.832 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T01:36:37.832 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T01:36:37.832 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T01:36:37.832 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T01:36:37.832 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T01:36:37.832 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T01:36:37.832 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T01:36:37.832 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T01:36:37.833 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T01:36:37.833 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T01:36:37.833 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T01:36:37.833 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T01:36:37.833 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T01:36:37.833 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T01:36:37.833 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T01:36:37.833 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T01:36:37.833 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T01:36:37.833 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T01:36:37.833 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T01:36:37.833 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T01:36:37.833 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T01:36:37.833 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T01:36:37.834 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T01:36:37.834 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T01:36:37.834 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T01:36:37.834 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T01:36:37.834 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T01:36:37.834 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T01:36:37.834 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T01:36:37.834 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T01:36:37.834 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T01:36:37.834 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T01:36:37.834 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T01:36:37.834 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T01:36:37.835 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T01:36:37.835 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T01:36:37.835 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T01:36:37.835 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T01:36:37.835 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T01:36:37.835 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T01:36:37.835 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.835 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.835 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T01:36:37.835 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T01:36:37.835 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T01:36:37.835 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.835 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T01:36:37.835 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T01:36:37.835 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T01:36:37.836 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T01:36:37.836 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T01:36:37.836 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T01:36:37.836 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T01:36:37.836 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T01:36:37.836 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T01:36:37.836 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T01:36:37.836 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T01:36:37.836 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T01:36:37.836 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T01:36:37.836 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T01:36:37.836 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T01:36:37.836 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.836 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.836 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:37.837 INFO:teuthology.orchestra.run.smithi097.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T01:36:37.837 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T01:36:37.837 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T01:36:37.837 INFO:teuthology.orchestra.run.smithi097.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T01:36:37.837 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T01:36:37.837 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T01:36:37.837 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T01:36:37.837 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-26T01:36:37.837 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-04-26T01:36:38.198 DEBUG:teuthology.parallel:result is None 2024-04-26T01:36:39.184 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-26T01:36:39.225 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-26T01:36:39.266 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-26T01:36:39.298 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T01:36:39.325 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T01:36:39.516 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-26T01:36:39.555 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T01:36:39.584 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-26T01:36:40.226 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:36:40.258 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:36:40.278 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:36:41.106 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-26T01:36:41.178 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T01:36:41.220 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T01:36:41.303 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T01:36:41.341 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T01:36:41.341 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T01:36:41.416 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T01:36:41.449 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T01:36:41.449 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T01:36:41.537 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T01:36:41.606 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:36:41.661 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-26T01:36:41.713 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-26T01:36:41.801 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-26T01:36:41.849 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T01:36:41.898 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T01:36:42.289 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-26T01:36:42.340 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T01:36:42.368 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-26T01:36:43.289 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:36:43.332 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:36:43.393 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:36:44.836 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-26T01:36:44.947 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T01:36:45.029 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T01:36:45.206 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T01:36:45.254 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T01:36:45.254 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T01:36:45.383 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T01:36:45.427 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T01:36:45.428 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T01:36:45.571 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T01:36:45.715 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:36:46.022 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:36:46.023 INFO:teuthology.orchestra.run.smithi001.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T01:36:46.023 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:36:47.259 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:36:47.259 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-26T01:36:47.259 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-26T01:36:47.259 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-26T01:36:47.259 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-26T01:36:47.259 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-26T01:36:47.259 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-26T01:36:47.259 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-26T01:36:47.259 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:36:47.259 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:36:47.259 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-26T01:36:47.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-26T01:36:47.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-26T01:36:47.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:36:47.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-26T01:36:47.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-26T01:36:47.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-26T01:36:47.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-26T01:36:47.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-26T01:36:47.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-26T01:36:47.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-26T01:36:47.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-26T01:36:47.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-26T01:36:47.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-26T01:36:47.260 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-26T01:36:47.261 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-26T01:36:47.261 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-26T01:36:47.261 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-26T01:36:47.261 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-26T01:36:47.261 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-26T01:36:47.261 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-26T01:36:47.272 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-26T01:36:47.272 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-26T01:36:47.272 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-26T01:36:47.272 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-26T01:36:47.272 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-26T01:36:47.272 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-26T01:36:47.272 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-26T01:36:47.273 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-26T01:36:47.273 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-26T01:36:47.273 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T01:36:47.273 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T01:36:47.273 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T01:36:47.273 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T01:36:47.273 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T01:36:47.273 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T01:36:47.273 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T01:36:47.273 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T01:36:47.273 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T01:36:47.273 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T01:36:47.273 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T01:36:47.273 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T01:36:47.274 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T01:36:47.274 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T01:36:47.274 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T01:36:47.274 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T01:36:47.274 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T01:36:47.274 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T01:36:47.274 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T01:36:47.274 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T01:36:47.274 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T01:36:47.274 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T01:36:47.274 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T01:36:47.274 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T01:36:47.274 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T01:36:47.274 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T01:36:47.275 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T01:36:47.275 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T01:36:47.275 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T01:36:47.275 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T01:36:47.275 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T01:36:47.275 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T01:36:47.275 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T01:36:47.275 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T01:36:47.275 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T01:36:47.275 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T01:36:47.275 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T01:36:47.275 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T01:36:47.275 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T01:36:47.275 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T01:36:47.276 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T01:36:47.276 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T01:36:47.276 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T01:36:47.276 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T01:36:47.276 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T01:36:47.276 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T01:36:47.276 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T01:36:47.276 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T01:36:47.276 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T01:36:47.276 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T01:36:47.276 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T01:36:47.276 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T01:36:47.276 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T01:36:47.276 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T01:36:47.277 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T01:36:47.277 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T01:36:47.277 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T01:36:47.277 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T01:36:47.277 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T01:36:47.277 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T01:36:47.277 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T01:36:47.277 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T01:36:47.277 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T01:36:47.277 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T01:36:47.277 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T01:36:47.277 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T01:36:47.277 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T01:36:47.277 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T01:36:47.277 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-26T01:36:47.278 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-26T01:36:47.278 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T01:36:47.278 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T01:36:47.278 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T01:36:47.278 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-26T01:36:47.278 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T01:36:47.278 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T01:36:47.278 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T01:36:47.278 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T01:36:47.278 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T01:36:47.278 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T01:36:47.278 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T01:36:47.278 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T01:36:47.278 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T01:36:47.279 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T01:36:47.279 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T01:36:47.279 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T01:36:47.279 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T01:36:47.279 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T01:36:47.279 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T01:36:47.279 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T01:36:47.279 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T01:36:47.279 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T01:36:47.279 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T01:36:47.279 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T01:36:47.279 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T01:36:47.279 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T01:36:47.280 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T01:36:47.280 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T01:36:47.280 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T01:36:47.280 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T01:36:47.280 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T01:36:47.280 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T01:36:47.280 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T01:36:47.280 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T01:36:47.280 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T01:36:47.280 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T01:36:47.280 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T01:36:47.280 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T01:36:47.280 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T01:36:47.280 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-26T01:36:47.281 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T01:36:47.281 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-26T01:36:47.281 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T01:36:47.281 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T01:36:47.281 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T01:36:47.281 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T01:36:49.336 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:36:49.336 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-26T01:36:49.336 INFO:teuthology.orchestra.run.smithi173.stdout:Upgraded: 2024-04-26T01:36:49.336 INFO:teuthology.orchestra.run.smithi173.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.336 INFO:teuthology.orchestra.run.smithi173.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.336 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T01:36:49.336 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T01:36:49.336 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-04-26T01:36:49.336 INFO:teuthology.orchestra.run.smithi173.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T01:36:49.336 INFO:teuthology.orchestra.run.smithi173.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T01:36:49.336 INFO:teuthology.orchestra.run.smithi173.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T01:36:49.337 INFO:teuthology.orchestra.run.smithi173.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T01:36:49.337 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.337 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.337 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.337 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.337 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:49.337 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.337 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.337 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.337 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:49.337 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:49.337 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:49.337 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:49.338 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:49.338 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.338 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.338 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:49.338 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.338 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.338 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.338 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:49.338 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:49.338 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.338 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.338 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:49.338 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T01:36:49.339 INFO:teuthology.orchestra.run.smithi173.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T01:36:49.339 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T01:36:49.339 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T01:36:49.339 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T01:36:49.339 INFO:teuthology.orchestra.run.smithi173.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T01:36:49.339 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T01:36:49.339 INFO:teuthology.orchestra.run.smithi173.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T01:36:49.339 INFO:teuthology.orchestra.run.smithi173.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T01:36:49.339 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-26T01:36:49.339 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-26T01:36:49.339 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.339 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.339 INFO:teuthology.orchestra.run.smithi173.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.340 INFO:teuthology.orchestra.run.smithi173.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T01:36:49.340 INFO:teuthology.orchestra.run.smithi173.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T01:36:49.340 INFO:teuthology.orchestra.run.smithi173.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T01:36:49.340 INFO:teuthology.orchestra.run.smithi173.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T01:36:49.340 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T01:36:49.340 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.340 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.340 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T01:36:49.340 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.340 INFO:teuthology.orchestra.run.smithi173.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T01:36:49.340 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T01:36:49.340 INFO:teuthology.orchestra.run.smithi173.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T01:36:49.340 INFO:teuthology.orchestra.run.smithi173.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T01:36:49.340 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T01:36:49.341 INFO:teuthology.orchestra.run.smithi173.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T01:36:49.341 INFO:teuthology.orchestra.run.smithi173.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T01:36:49.341 INFO:teuthology.orchestra.run.smithi173.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T01:36:49.341 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T01:36:49.341 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T01:36:49.341 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T01:36:49.341 INFO:teuthology.orchestra.run.smithi173.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-26T01:36:49.341 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T01:36:49.341 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T01:36:49.341 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T01:36:49.341 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T01:36:49.341 INFO:teuthology.orchestra.run.smithi173.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T01:36:49.341 INFO:teuthology.orchestra.run.smithi173.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T01:36:49.341 INFO:teuthology.orchestra.run.smithi173.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T01:36:49.342 INFO:teuthology.orchestra.run.smithi173.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T01:36:49.342 INFO:teuthology.orchestra.run.smithi173.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T01:36:49.342 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T01:36:49.342 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T01:36:49.342 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.342 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.342 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.342 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T01:36:49.342 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T01:36:49.342 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T01:36:49.342 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T01:36:49.342 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T01:36:49.342 INFO:teuthology.orchestra.run.smithi173.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T01:36:49.343 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T01:36:49.343 INFO:teuthology.orchestra.run.smithi173.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T01:36:49.343 INFO:teuthology.orchestra.run.smithi173.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T01:36:49.343 INFO:teuthology.orchestra.run.smithi173.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T01:36:49.343 INFO:teuthology.orchestra.run.smithi173.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T01:36:49.343 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T01:36:49.343 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T01:36:49.343 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T01:36:49.343 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T01:36:49.343 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T01:36:49.343 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T01:36:49.343 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T01:36:49.343 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T01:36:49.343 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T01:36:49.343 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T01:36:49.344 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T01:36:49.344 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T01:36:49.344 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T01:36:49.344 INFO:teuthology.orchestra.run.smithi173.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T01:36:49.344 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T01:36:49.344 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T01:36:49.344 INFO:teuthology.orchestra.run.smithi173.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T01:36:49.344 INFO:teuthology.orchestra.run.smithi173.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T01:36:49.344 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T01:36:49.344 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T01:36:49.344 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T01:36:49.344 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T01:36:49.344 INFO:teuthology.orchestra.run.smithi173.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T01:36:49.344 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T01:36:49.345 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T01:36:49.345 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T01:36:49.345 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T01:36:49.345 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T01:36:49.345 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T01:36:49.345 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.345 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.345 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T01:36:49.345 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T01:36:49.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T01:36:49.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T01:36:49.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T01:36:49.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T01:36:49.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T01:36:49.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T01:36:49.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T01:36:49.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T01:36:49.346 INFO:teuthology.orchestra.run.smithi173.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T01:36:49.347 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T01:36:49.347 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T01:36:49.347 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T01:36:49.347 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T01:36:49.347 INFO:teuthology.orchestra.run.smithi173.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T01:36:49.347 INFO:teuthology.orchestra.run.smithi173.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T01:36:49.347 INFO:teuthology.orchestra.run.smithi173.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T01:36:49.347 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.347 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.347 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:49.347 INFO:teuthology.orchestra.run.smithi173.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T01:36:49.347 INFO:teuthology.orchestra.run.smithi173.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T01:36:49.348 INFO:teuthology.orchestra.run.smithi173.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T01:36:49.348 INFO:teuthology.orchestra.run.smithi173.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T01:36:49.348 INFO:teuthology.orchestra.run.smithi173.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T01:36:49.348 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T01:36:49.348 INFO:teuthology.orchestra.run.smithi173.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T01:36:49.348 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-26T01:36:49.348 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-26T01:36:49.728 DEBUG:teuthology.parallel:result is None 2024-04-26T01:36:51.750 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-26T01:36:51.750 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-26T01:36:51.750 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-26T01:36:51.750 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-26T01:36:51.750 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-26T01:36:51.750 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-26T01:36:51.750 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-26T01:36:51.750 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:36:51.751 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:36:51.751 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-26T01:36:51.751 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-26T01:36:51.751 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-26T01:36:51.751 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:36:51.751 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-26T01:36:51.751 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-26T01:36:51.751 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-26T01:36:51.751 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-26T01:36:51.751 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-26T01:36:51.751 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-26T01:36:51.751 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-26T01:36:51.751 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-26T01:36:51.752 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-26T01:36:51.752 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-26T01:36:51.752 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-26T01:36:51.752 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-26T01:36:51.752 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-26T01:36:51.752 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-26T01:36:51.752 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-26T01:36:51.752 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-26T01:36:51.752 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-26T01:36:51.752 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-26T01:36:51.752 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-26T01:36:51.752 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-26T01:36:51.752 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-26T01:36:51.752 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-26T01:36:51.753 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-26T01:36:51.753 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-26T01:36:51.753 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-26T01:36:51.753 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-26T01:36:51.753 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T01:36:51.753 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T01:36:51.753 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T01:36:51.753 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T01:36:51.753 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T01:36:51.753 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T01:36:51.753 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T01:36:51.753 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T01:36:51.753 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T01:36:51.753 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T01:36:51.754 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T01:36:51.754 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T01:36:51.754 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T01:36:51.754 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T01:36:51.754 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T01:36:51.754 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T01:36:51.754 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T01:36:51.754 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T01:36:51.754 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T01:36:51.754 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T01:36:51.754 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T01:36:51.754 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T01:36:51.754 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T01:36:51.755 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T01:36:51.755 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T01:36:51.755 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T01:36:51.755 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T01:36:51.755 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T01:36:51.755 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T01:36:51.755 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T01:36:51.755 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T01:36:51.755 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T01:36:51.755 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T01:36:51.755 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T01:36:51.755 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T01:36:51.755 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T01:36:51.755 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T01:36:51.756 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T01:36:51.756 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T01:36:51.756 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T01:36:51.756 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T01:36:51.756 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T01:36:51.756 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T01:36:51.756 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T01:36:51.756 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T01:36:51.756 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T01:36:51.756 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T01:36:51.756 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T01:36:51.756 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T01:36:51.756 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T01:36:51.756 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T01:36:51.757 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T01:36:51.757 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T01:36:51.757 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T01:36:51.757 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T01:36:51.757 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T01:36:51.757 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T01:36:51.757 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T01:36:51.757 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T01:36:51.757 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T01:36:51.757 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T01:36:51.757 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T01:36:51.757 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T01:36:51.757 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T01:36:51.757 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T01:36:51.758 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T01:36:51.758 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T01:36:51.758 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T01:36:51.758 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-26T01:36:51.758 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-26T01:36:51.758 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T01:36:51.758 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T01:36:51.758 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T01:36:51.758 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-26T01:36:51.758 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T01:36:51.758 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T01:36:51.758 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T01:36:51.758 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T01:36:51.758 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T01:36:51.759 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T01:36:51.759 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T01:36:51.759 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T01:36:51.759 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T01:36:51.759 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T01:36:51.759 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T01:36:51.759 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T01:36:51.759 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T01:36:51.759 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T01:36:51.759 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T01:36:51.760 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T01:36:51.760 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T01:36:51.760 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T01:36:51.760 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T01:36:51.760 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T01:36:51.760 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T01:36:51.760 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T01:36:51.761 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T01:36:51.761 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T01:36:51.761 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T01:36:51.761 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T01:36:51.761 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T01:36:51.761 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T01:36:51.762 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T01:36:51.762 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T01:36:51.762 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T01:36:51.762 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T01:36:51.762 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T01:36:51.762 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T01:36:51.763 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T01:36:51.763 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-26T01:36:51.763 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T01:36:51.763 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-26T01:36:51.763 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T01:36:51.763 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T01:36:51.763 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T01:36:51.764 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T01:36:54.151 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:36:54.152 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:36:54.152 INFO:teuthology.orchestra.run.smithi001.stdout:Upgraded: 2024-04-26T01:36:54.152 INFO:teuthology.orchestra.run.smithi001.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.152 INFO:teuthology.orchestra.run.smithi001.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.152 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T01:36:54.152 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T01:36:54.152 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2024-04-26T01:36:54.152 INFO:teuthology.orchestra.run.smithi001.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T01:36:54.152 INFO:teuthology.orchestra.run.smithi001.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T01:36:54.152 INFO:teuthology.orchestra.run.smithi001.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T01:36:54.152 INFO:teuthology.orchestra.run.smithi001.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T01:36:54.152 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.153 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.153 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.153 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.153 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:54.153 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.153 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.153 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.153 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:54.153 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:54.153 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:54.153 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:54.153 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:54.153 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.154 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.154 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:54.154 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.154 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.154 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.154 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:54.154 INFO:teuthology.orchestra.run.smithi001.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:54.154 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.154 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.154 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:36:54.154 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T01:36:54.154 INFO:teuthology.orchestra.run.smithi001.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T01:36:54.155 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T01:36:54.155 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T01:36:54.155 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T01:36:54.155 INFO:teuthology.orchestra.run.smithi001.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T01:36:54.155 INFO:teuthology.orchestra.run.smithi001.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T01:36:54.155 INFO:teuthology.orchestra.run.smithi001.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T01:36:54.155 INFO:teuthology.orchestra.run.smithi001.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T01:36:54.155 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-26T01:36:54.155 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-26T01:36:54.155 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.155 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.155 INFO:teuthology.orchestra.run.smithi001.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.156 INFO:teuthology.orchestra.run.smithi001.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T01:36:54.156 INFO:teuthology.orchestra.run.smithi001.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T01:36:54.156 INFO:teuthology.orchestra.run.smithi001.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T01:36:54.156 INFO:teuthology.orchestra.run.smithi001.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T01:36:54.156 INFO:teuthology.orchestra.run.smithi001.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T01:36:54.156 INFO:teuthology.orchestra.run.smithi001.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.156 INFO:teuthology.orchestra.run.smithi001.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.156 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T01:36:54.156 INFO:teuthology.orchestra.run.smithi001.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.156 INFO:teuthology.orchestra.run.smithi001.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T01:36:54.156 INFO:teuthology.orchestra.run.smithi001.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T01:36:54.156 INFO:teuthology.orchestra.run.smithi001.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T01:36:54.156 INFO:teuthology.orchestra.run.smithi001.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T01:36:54.156 INFO:teuthology.orchestra.run.smithi001.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T01:36:54.157 INFO:teuthology.orchestra.run.smithi001.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T01:36:54.157 INFO:teuthology.orchestra.run.smithi001.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T01:36:54.157 INFO:teuthology.orchestra.run.smithi001.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T01:36:54.157 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T01:36:54.157 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T01:36:54.157 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T01:36:54.157 INFO:teuthology.orchestra.run.smithi001.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-26T01:36:54.157 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T01:36:54.157 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T01:36:54.157 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T01:36:54.157 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T01:36:54.157 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T01:36:54.157 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T01:36:54.158 INFO:teuthology.orchestra.run.smithi001.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T01:36:54.158 INFO:teuthology.orchestra.run.smithi001.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T01:36:54.158 INFO:teuthology.orchestra.run.smithi001.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T01:36:54.158 INFO:teuthology.orchestra.run.smithi001.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T01:36:54.158 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T01:36:54.158 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.158 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.158 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.158 INFO:teuthology.orchestra.run.smithi001.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T01:36:54.158 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T01:36:54.158 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T01:36:54.158 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T01:36:54.158 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T01:36:54.158 INFO:teuthology.orchestra.run.smithi001.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T01:36:54.159 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T01:36:54.162 INFO:teuthology.orchestra.run.smithi001.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T01:36:54.162 INFO:teuthology.orchestra.run.smithi001.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T01:36:54.162 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T01:36:54.162 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T01:36:54.162 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T01:36:54.162 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T01:36:54.162 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T01:36:54.162 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T01:36:54.163 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T01:36:54.163 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T01:36:54.163 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T01:36:54.163 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T01:36:54.163 INFO:teuthology.orchestra.run.smithi001.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T01:36:54.163 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T01:36:54.163 INFO:teuthology.orchestra.run.smithi001.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T01:36:54.163 INFO:teuthology.orchestra.run.smithi001.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T01:36:54.163 INFO:teuthology.orchestra.run.smithi001.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T01:36:54.163 INFO:teuthology.orchestra.run.smithi001.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T01:36:54.163 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T01:36:54.163 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T01:36:54.164 INFO:teuthology.orchestra.run.smithi001.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T01:36:54.164 INFO:teuthology.orchestra.run.smithi001.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T01:36:54.164 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T01:36:54.164 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T01:36:54.164 INFO:teuthology.orchestra.run.smithi001.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T01:36:54.164 INFO:teuthology.orchestra.run.smithi001.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T01:36:54.164 INFO:teuthology.orchestra.run.smithi001.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T01:36:54.164 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T01:36:54.164 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T01:36:54.164 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T01:36:54.164 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T01:36:54.164 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T01:36:54.165 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T01:36:54.165 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.165 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.165 INFO:teuthology.orchestra.run.smithi001.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T01:36:54.165 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T01:36:54.165 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T01:36:54.165 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.165 INFO:teuthology.orchestra.run.smithi001.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T01:36:54.165 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T01:36:54.165 INFO:teuthology.orchestra.run.smithi001.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T01:36:54.165 INFO:teuthology.orchestra.run.smithi001.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T01:36:54.165 INFO:teuthology.orchestra.run.smithi001.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T01:36:54.165 INFO:teuthology.orchestra.run.smithi001.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T01:36:54.166 INFO:teuthology.orchestra.run.smithi001.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T01:36:54.166 INFO:teuthology.orchestra.run.smithi001.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T01:36:54.166 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T01:36:54.166 INFO:teuthology.orchestra.run.smithi001.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T01:36:54.166 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T01:36:54.166 INFO:teuthology.orchestra.run.smithi001.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T01:36:54.166 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T01:36:54.166 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T01:36:54.166 INFO:teuthology.orchestra.run.smithi001.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T01:36:54.166 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.167 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.167 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:36:54.167 INFO:teuthology.orchestra.run.smithi001.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T01:36:54.167 INFO:teuthology.orchestra.run.smithi001.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T01:36:54.167 INFO:teuthology.orchestra.run.smithi001.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T01:36:54.167 INFO:teuthology.orchestra.run.smithi001.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T01:36:54.167 INFO:teuthology.orchestra.run.smithi001.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T01:36:54.167 INFO:teuthology.orchestra.run.smithi001.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T01:36:54.168 INFO:teuthology.orchestra.run.smithi001.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T01:36:54.168 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:36:54.168 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2024-04-26T01:36:54.754 DEBUG:teuthology.parallel:result is None 2024-04-26T01:36:54.755 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:36:54.905 DEBUG:teuthology.orchestra.run.smithi001:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T01:36:54.959 INFO:teuthology.orchestra.run.smithi001.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-26T01:36:54.960 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-26T01:36:54.960 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-26T01:36:54.962 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:36:55.210 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T01:36:55.236 INFO:teuthology.orchestra.run.smithi097.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-26T01:36:55.236 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-26T01:36:55.236 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-26T01:36:55.238 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:36:55.453 DEBUG:teuthology.orchestra.run.smithi173:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T01:36:55.479 INFO:teuthology.orchestra.run.smithi173.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-26T01:36:55.479 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-26T01:36:55.479 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-26T01:36:55.481 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T01:36:55.481 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-04-26T01:36:55.481 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T01:36:55.557 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T01:36:55.557 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T01:36:55.588 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-26T01:36:55.588 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T01:36:55.618 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T01:36:55.618 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-04-26T01:36:55.618 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T01:36:55.711 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T01:36:55.823 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T01:36:55.823 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T01:36:55.852 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T01:36:55.917 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-26T01:36:55.917 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T01:36:55.945 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T01:36:56.013 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T01:36:56.013 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-04-26T01:36:56.013 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T01:36:56.079 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T01:36:56.183 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T01:36:56.183 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T01:36:56.207 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T01:36:56.270 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-26T01:36:56.270 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T01:36:56.298 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T01:36:56.365 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T01:36:56.365 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-04-26T01:36:56.365 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T01:36:56.431 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T01:36:56.534 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T01:36:56.535 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T01:36:56.559 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T01:36:56.623 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-26T01:36:56.623 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T01:36:56.650 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T01:36:56.717 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-26T01:36:56.819 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2}} 2024-04-26T01:36:56.819 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:36:56.820 INFO:tasks.cephadm:Cluster fsid is 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:36:56.820 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-26T01:36:56.820 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.1', 'mon.b': '172.21.15.97', 'mon.c': '172.21.15.173'} 2024-04-26T01:36:56.820 INFO:tasks.cephadm:First mon is mon.a on smithi001 2024-04-26T01:36:56.820 INFO:tasks.cephadm:First mgr is x 2024-04-26T01:36:56.820 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-26T01:36:56.820 DEBUG:teuthology.orchestra.run.smithi001:> sudo hostname $(hostname -s) 2024-04-26T01:36:56.896 DEBUG:teuthology.orchestra.run.smithi097:> sudo hostname $(hostname -s) 2024-04-26T01:36:56.919 DEBUG:teuthology.orchestra.run.smithi173:> sudo hostname $(hostname -s) 2024-04-26T01:36:56.947 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-26T01:36:56.947 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:36:57.091 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-26T01:36:57.421 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:36:57.422 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-26T01:36:57.422 DEBUG:teuthology.orchestra.run.smithi001:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T01:36:58.293 INFO:teuthology.orchestra.run.smithi001.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 26 01:36 /home/ubuntu/cephtest/cephadm 2024-04-26T01:36:58.293 DEBUG:teuthology.orchestra.run.smithi097:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T01:36:58.948 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 26 01:36 /home/ubuntu/cephtest/cephadm 2024-04-26T01:36:58.948 DEBUG:teuthology.orchestra.run.smithi173:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T01:36:59.469 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 26 01:36 /home/ubuntu/cephtest/cephadm 2024-04-26T01:36:59.470 DEBUG:teuthology.orchestra.run.smithi001:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T01:36:59.519 DEBUG:teuthology.orchestra.run.smithi097:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T01:36:59.538 DEBUG:teuthology.orchestra.run.smithi173:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T01:36:59.573 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-26T01:36:59.573 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-26T01:36:59.580 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-26T01:36:59.582 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-26T01:37:01.123 INFO:teuthology.orchestra.run.smithi097.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-26T01:37:01.324 INFO:teuthology.orchestra.run.smithi173.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-26T01:37:01.697 INFO:teuthology.orchestra.run.smithi001.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-26T01:37:43.844 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-26T01:37:43.844 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-26T01:37:43.844 INFO:teuthology.orchestra.run.smithi173.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-26T01:37:43.845 INFO:teuthology.orchestra.run.smithi173.stdout: "repo_digests": [ 2024-04-26T01:37:43.845 INFO:teuthology.orchestra.run.smithi173.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-26T01:37:43.845 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-04-26T01:37:43.845 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-26T01:37:47.740 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T01:37:47.740 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-26T01:37:47.740 INFO:teuthology.orchestra.run.smithi001.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-26T01:37:47.740 INFO:teuthology.orchestra.run.smithi001.stdout: "repo_digests": [ 2024-04-26T01:37:47.740 INFO:teuthology.orchestra.run.smithi001.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-26T01:37:47.741 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-04-26T01:37:47.741 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T01:37:51.205 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-26T01:37:51.205 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-26T01:37:51.205 INFO:teuthology.orchestra.run.smithi097.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-26T01:37:51.205 INFO:teuthology.orchestra.run.smithi097.stdout: "repo_digests": [ 2024-04-26T01:37:51.205 INFO:teuthology.orchestra.run.smithi097.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-26T01:37:51.205 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-26T01:37:51.205 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-26T01:37:51.229 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /etc/ceph 2024-04-26T01:37:51.326 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph 2024-04-26T01:37:51.352 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph 2024-04-26T01:37:51.384 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 777 /etc/ceph 2024-04-26T01:37:51.478 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /etc/ceph 2024-04-26T01:37:51.502 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 777 /etc/ceph 2024-04-26T01:37:51.530 INFO:tasks.cephadm:Writing seed config... 2024-04-26T01:37:51.531 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-26T01:37:51.531 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-26T01:37:51.532 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-26T01:37:51.532 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-26T01:37:51.532 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-26T01:37:51.532 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-26T01:37:51.532 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-26T01:37:51.532 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-26T01:37:51.532 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-26T01:37:51.532 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-26T01:37:51.532 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-26T01:37:51.532 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-26T01:37:51.532 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-26T01:37:51.532 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-26T01:37:51.532 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-26T01:37:51.533 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-26T01:37:51.533 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-26T01:37:51.533 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-26T01:37:51.533 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-26T01:37:51.533 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-26T01:37:51.533 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-26T01:37:51.533 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-26T01:37:51.533 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-26T01:37:51.533 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-26T01:37:51.533 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-26T01:37:51.533 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-26T01:37:51.533 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-26T01:37:51.533 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-26T01:37:51.533 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-26T01:37:51.533 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-26T01:37:51.534 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-26T01:37:51.534 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-26T01:37:51.534 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-26T01:37:51.534 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-26T01:37:51.534 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-26T01:37:51.534 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-26T01:37:51.534 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-26T01:37:51.534 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-26T01:37:51.534 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-26T01:37:51.534 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-26T01:37:51.534 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-26T01:37:51.534 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-26T01:37:51.534 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-26T01:37:51.534 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-26T01:37:51.535 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-26T01:37:51.535 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-26T01:37:51.535 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-26T01:37:51.535 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-26T01:37:51.535 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-26T01:37:51.535 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-26T01:37:51.536 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-04-26T01:37:51.536 DEBUG:teuthology.orchestra.run.smithi001:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-26T01:37:51.595 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 77cfcc80-036d-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-26T01:37:51.596 DEBUG:teuthology.orchestra.run.smithi001:mon.a> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mon.a.service 2024-04-26T01:37:51.642 DEBUG:teuthology.orchestra.run.smithi001:mgr.x> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mgr.x.service 2024-04-26T01:37:51.689 INFO:tasks.cephadm:Bootstrapping... 2024-04-26T01:37:51.690 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.1 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-26T01:37:51.967 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2024-04-26T01:37:51.968 INFO:teuthology.orchestra.run.smithi001.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', '77cfcc80-036d-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.1', '--skip-admin-label'] 2024-04-26T01:37:51.968 INFO:teuthology.orchestra.run.smithi001.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-26T01:37:51.968 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying podman|docker is present... 2024-04-26T01:37:52.027 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T01:37:52.028 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying lvm2 is present... 2024-04-26T01:37:52.028 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying time synchronization is in place... 2024-04-26T01:37:52.047 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T01:37:52.047 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T01:37:52.065 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T01:37:52.065 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-04-26T01:37:52.084 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout enabled 2024-04-26T01:37:52.104 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout active 2024-04-26T01:37:52.105 INFO:teuthology.orchestra.run.smithi001.stdout:Unit chronyd.service is enabled and running 2024-04-26T01:37:52.105 INFO:teuthology.orchestra.run.smithi001.stdout:Repeating the final host check... 2024-04-26T01:37:52.198 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T01:37:52.199 INFO:teuthology.orchestra.run.smithi001.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-26T01:37:52.199 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl is present 2024-04-26T01:37:52.199 INFO:teuthology.orchestra.run.smithi001.stdout:lvcreate is present 2024-04-26T01:37:52.224 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T01:37:52.225 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T01:37:52.249 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T01:37:52.249 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-04-26T01:37:52.276 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout enabled 2024-04-26T01:37:52.298 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout active 2024-04-26T01:37:52.299 INFO:teuthology.orchestra.run.smithi001.stdout:Unit chronyd.service is enabled and running 2024-04-26T01:37:52.299 INFO:teuthology.orchestra.run.smithi001.stdout:Host looks OK 2024-04-26T01:37:52.299 INFO:teuthology.orchestra.run.smithi001.stdout:Cluster fsid: 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:37:52.300 INFO:teuthology.orchestra.run.smithi001.stdout:Acquiring lock 140176336354464 on /run/cephadm/77cfcc80-036d-11ef-bc93-c7b262605968.lock 2024-04-26T01:37:52.300 INFO:teuthology.orchestra.run.smithi001.stdout:Lock 140176336354464 acquired on /run/cephadm/77cfcc80-036d-11ef-bc93-c7b262605968.lock 2024-04-26T01:37:52.302 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying IP 172.21.15.1 port 3300 ... 2024-04-26T01:37:52.303 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying IP 172.21.15.1 port 6789 ... 2024-04-26T01:37:52.303 INFO:teuthology.orchestra.run.smithi001.stdout:Base mon IP(s) is [172.21.15.1:3300, 172.21.15.1:6789], mon addrv is [v2:172.21.15.1:3300,v1:172.21.15.1:6789] 2024-04-26T01:37:52.318 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.1 metric 100 2024-04-26T01:37:52.319 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.1 metric 100 2024-04-26T01:37:52.332 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-26T01:37:52.332 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-26T01:37:52.332 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-26T01:37:52.345 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-26T01:37:52.345 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-26T01:37:52.345 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T01:37:52.345 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-26T01:37:52.345 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-04-26T01:37:52.346 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T01:37:52.348 INFO:teuthology.orchestra.run.smithi001.stdout:Mon IP `172.21.15.1` is in CIDR network `172.21.0.0/20` 2024-04-26T01:37:52.348 INFO:teuthology.orchestra.run.smithi001.stdout:Mon IP `172.21.15.1` is in CIDR network `172.21.0.0/20` 2024-04-26T01:37:52.349 INFO:teuthology.orchestra.run.smithi001.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-26T01:37:52.352 INFO:teuthology.orchestra.run.smithi001.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-26T01:37:52.355 INFO:teuthology.orchestra.run.smithi001.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-26T01:37:52.994 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-26T01:37:52.994 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-26T01:37:52.994 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Getting image source signatures 2024-04-26T01:37:52.995 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-26T01:37:52.995 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-26T01:37:52.995 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-26T01:37:52.995 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-26T01:37:54.204 INFO:teuthology.orchestra.run.smithi001.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-26T01:37:54.205 INFO:teuthology.orchestra.run.smithi001.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-26T01:37:54.205 INFO:teuthology.orchestra.run.smithi001.stdout:Extracting ceph user uid/gid from container image... 2024-04-26T01:37:55.366 INFO:teuthology.orchestra.run.smithi001.stdout:stat: stdout 167 167 2024-04-26T01:37:55.367 INFO:teuthology.orchestra.run.smithi001.stdout:Creating initial keys... 2024-04-26T01:37:56.543 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph-authtool: stdout AQBzBStmHrrtNBAAHOmDH+3QjZFZ1E0nXmGO0Q== 2024-04-26T01:37:57.800 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph-authtool: stdout AQB1BStm8b0QABAA+SN8Li5PfVXvSuvubBCOWA== 2024-04-26T01:37:58.920 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph-authtool: stdout AQB2BStmmLKjERAAAMw98VJ4x86asUPiqcSEpg== 2024-04-26T01:37:58.922 INFO:teuthology.orchestra.run.smithi001.stdout:Creating initial monmap... 2024-04-26T01:38:00.072 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T01:38:00.073 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-26T01:38:00.073 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:38:00.073 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T01:38:00.073 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool for a [v2:172.21.15.1:3300,v1:172.21.15.1:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T01:38:00.073 INFO:teuthology.orchestra.run.smithi001.stdout:setting min_mon_release = quincy 2024-04-26T01:38:00.073 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: set fsid to 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:38:00.073 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T01:38:00.073 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:38:00.073 INFO:teuthology.orchestra.run.smithi001.stdout:Creating mon... 2024-04-26T01:38:01.183 INFO:teuthology.orchestra.run.smithi001.stdout:create mon.a on 2024-04-26T01:38:01.574 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-26T01:38:01.823 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-26T01:38:02.149 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-77cfcc80-036d-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-77cfcc80-036d-11ef-bc93-c7b262605968.target. 2024-04-26T01:38:02.149 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-77cfcc80-036d-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-77cfcc80-036d-11ef-bc93-c7b262605968.target. 2024-04-26T01:38:02.548 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mon.a 2024-04-26T01:38:02.548 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to reset failed state of unit ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-26T01:38:02.841 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-77cfcc80-036d-11ef-bc93-c7b262605968.target.wants/ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-77cfcc80-036d-11ef-bc93-c7b262605968@.service. 2024-04-26T01:38:03.510 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:03 smithi001 podman[27439]: 2024-04-26 01:38:03.451803482 +0000 UTC m=+0.341194321 container start a1ff1b41424de7a5a1d2896d54528124a922a2a3ed467edae5625232274725ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-26T01:38:03.510 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:03 smithi001 bash[27439]: a1ff1b41424de7a5a1d2896d54528124a922a2a3ed467edae5625232274725ef 2024-04-26T01:38:03.540 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T01:38:03.540 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-04-26T01:38:03.555 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T01:38:03.555 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-04-26T01:38:03.556 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-04-26T01:38:03.556 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T01:38:03.556 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mon to start... 2024-04-26T01:38:03.556 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mon... 2024-04-26T01:38:03.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:03 smithi001 systemd[1]: Started Ceph mon.a for 77cfcc80-036d-11ef-bc93-c7b262605968. 2024-04-26T01:38:03.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:03 smithi001 ceph-mon[27457]: mkfs 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:38:03.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:03 smithi001 ceph-mon[27457]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T01:38:04.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:04 smithi001 ceph-mon[27457]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T01:38:04.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:04 smithi001 ceph-mon[27457]: monmap epoch 1 2024-04-26T01:38:04.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:04 smithi001 ceph-mon[27457]: fsid 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:38:04.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:04 smithi001 ceph-mon[27457]: last_changed 2024-04-26T01:37:59.408242+0000 2024-04-26T01:38:04.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:04 smithi001 ceph-mon[27457]: created 2024-04-26T01:37:59.408242+0000 2024-04-26T01:38:04.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:04 smithi001 ceph-mon[27457]: min_mon_release 19 (squid) 2024-04-26T01:38:04.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:04 smithi001 ceph-mon[27457]: election_strategy: 1 2024-04-26T01:38:04.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:04 smithi001 ceph-mon[27457]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-04-26T01:38:04.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:04 smithi001 ceph-mon[27457]: fsmap 2024-04-26T01:38:04.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:04 smithi001 ceph-mon[27457]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T01:38:04.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:04 smithi001 ceph-mon[27457]: mgrmap e1: no daemons active 2024-04-26T01:38:04.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:04 smithi001 ceph-mon[27457]: from='client.? 172.21.15.1:0/608110615' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-26T01:38:04.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout cluster: 2024-04-26T01:38:04.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout id: 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:38:04.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-26T01:38:04.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-04-26T01:38:04.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout services: 2024-04-26T01:38:04.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.804652s) 2024-04-26T01:38:04.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-26T01:38:04.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-26T01:38:04.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-04-26T01:38:04.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout data: 2024-04-26T01:38:04.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-26T01:38:04.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-26T01:38:04.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-26T01:38:04.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout pgs: 2024-04-26T01:38:04.959 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-04-26T01:38:04.959 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.239+0000 7f42b0600700 1 Processor -- start 2024-04-26T01:38:04.959 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.239+0000 7f42b0600700 1 -- start start 2024-04-26T01:38:04.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.240+0000 7f42b0600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42a8109130 0x7f42a810b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:04.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.240+0000 7f42b0600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f42a8063ce0 con 0x7f42a8109130 2024-04-26T01:38:04.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.241+0000 7f42adc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42a8109130 0x7f42a810b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:04.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.241+0000 7f42adc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42a8109130 0x7f42a810b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39148/0 (socket says 172.21.15.1:39148) 2024-04-26T01:38:04.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.241+0000 7f42adc00700 1 -- 172.21.15.1:0/1679201524 learned_addr learned my addr 172.21.15.1:0/1679201524 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:04.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.242+0000 7f42adc00700 1 -- 172.21.15.1:0/1679201524 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a810ba60 con 0x7f42a8109130 2024-04-26T01:38:04.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.242+0000 7f42adc00700 1 --2- 172.21.15.1:0/1679201524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42a8109130 0x7f42a810b520 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f429c0098a0 tx=0x7f429c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1979eabcc0df75 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:04.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.242+0000 7f42a7e00700 1 -- 172.21.15.1:0/1679201524 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f429c004160 con 0x7f42a8109130 2024-04-26T01:38:04.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.242+0000 7f42a7e00700 1 -- 172.21.15.1:0/1679201524 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f429c004300 con 0x7f42a8109130 2024-04-26T01:38:04.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.242+0000 7f42a7e00700 1 -- 172.21.15.1:0/1679201524 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f429c0036a0 con 0x7f42a8109130 2024-04-26T01:38:04.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.243+0000 7f42b0600700 1 -- 172.21.15.1:0/1679201524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42a8109130 msgr2=0x7f42a810b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:04.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.243+0000 7f42b0600700 1 --2- 172.21.15.1:0/1679201524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42a8109130 0x7f42a810b520 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f429c0098a0 tx=0x7f429c009bb0 comp rx=0 tx=0).stop 2024-04-26T01:38:04.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.243+0000 7f42b0600700 1 -- 172.21.15.1:0/1679201524 shutdown_connections 2024-04-26T01:38:04.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.243+0000 7f42b0600700 1 --2- 172.21.15.1:0/1679201524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42a8109130 0x7f42a810b520 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:04.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.243+0000 7f42b0600700 1 -- 172.21.15.1:0/1679201524 >> 172.21.15.1:0/1679201524 conn(0x7f42a8102b80 msgr2=0x7f42a8104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:04.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.243+0000 7f42b0600700 1 -- 172.21.15.1:0/1679201524 shutdown_connections 2024-04-26T01:38:04.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.243+0000 7f42b0600700 1 -- 172.21.15.1:0/1679201524 wait complete. 2024-04-26T01:38:04.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.244+0000 7f42b0600700 1 Processor -- start 2024-04-26T01:38:04.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.245+0000 7f42b0600700 1 -- start start 2024-04-26T01:38:04.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.245+0000 7f42b0600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42a8109130 0x7f42a819def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:04.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.245+0000 7f42b0600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f42a810bed0 con 0x7f42a8109130 2024-04-26T01:38:04.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.245+0000 7f42adc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42a8109130 0x7f42a819def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:04.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.245+0000 7f42adc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42a8109130 0x7f42a819def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39152/0 (socket says 172.21.15.1:39152) 2024-04-26T01:38:04.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.245+0000 7f42adc00700 1 -- 172.21.15.1:0/608110615 learned_addr learned my addr 172.21.15.1:0/608110615 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:04.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.246+0000 7f42adc00700 1 -- 172.21.15.1:0/608110615 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a819e430 con 0x7f42a8109130 2024-04-26T01:38:04.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.246+0000 7f42adc00700 1 --2- 172.21.15.1:0/608110615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42a8109130 0x7f42a819def0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f429c000c00 tx=0x7f429c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:04.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.246+0000 7f42a6000700 1 -- 172.21.15.1:0/608110615 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f429c003db0 con 0x7f42a8109130 2024-04-26T01:38:04.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.246+0000 7f42a6000700 1 -- 172.21.15.1:0/608110615 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f429c011630 con 0x7f42a8109130 2024-04-26T01:38:04.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.246+0000 7f42b0600700 1 -- 172.21.15.1:0/608110615 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a819e6c0 con 0x7f42a8109130 2024-04-26T01:38:04.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.246+0000 7f42a6000700 1 -- 172.21.15.1:0/608110615 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f429c011950 con 0x7f42a8109130 2024-04-26T01:38:04.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.246+0000 7f42b0600700 1 -- 172.21.15.1:0/608110615 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a81a13b0 con 0x7f42a8109130 2024-04-26T01:38:04.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.246+0000 7f42a6000700 1 -- 172.21.15.1:0/608110615 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f429c01a460 con 0x7f42a8109130 2024-04-26T01:38:04.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.247+0000 7f42a6000700 1 -- 172.21.15.1:0/608110615 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f429c01b780 con 0x7f42a8109130 2024-04-26T01:38:04.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.247+0000 7f42b0600700 1 -- 172.21.15.1:0/608110615 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42a80640a0 con 0x7f42a8109130 2024-04-26T01:38:04.964 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.251+0000 7f42a6000700 1 -- 172.21.15.1:0/608110615 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f429c01b980 con 0x7f42a8109130 2024-04-26T01:38:04.964 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.333+0000 7f42b0600700 1 -- 172.21.15.1:0/608110615 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f42a802e1d0 con 0x7f42a8109130 2024-04-26T01:38:04.964 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.333+0000 7f42a6000700 1 -- 172.21.15.1:0/608110615 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f429c01bb60 con 0x7f42a8109130 2024-04-26T01:38:04.964 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.335+0000 7f42b0600700 1 -- 172.21.15.1:0/608110615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42a8109130 msgr2=0x7f42a819def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:04.964 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.335+0000 7f42b0600700 1 --2- 172.21.15.1:0/608110615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42a8109130 0x7f42a819def0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f429c000c00 tx=0x7f429c004750 comp rx=0 tx=0).stop 2024-04-26T01:38:04.965 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.335+0000 7f42b0600700 1 -- 172.21.15.1:0/608110615 shutdown_connections 2024-04-26T01:38:04.965 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.335+0000 7f42b0600700 1 --2- 172.21.15.1:0/608110615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f42a8109130 0x7f42a819def0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:04.965 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.335+0000 7f42b0600700 1 -- 172.21.15.1:0/608110615 >> 172.21.15.1:0/608110615 conn(0x7f42a8102b80 msgr2=0x7f42a8104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:04.965 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.335+0000 7f42b0600700 1 -- 172.21.15.1:0/608110615 shutdown_connections 2024-04-26T01:38:04.965 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:04.335+0000 7f42b0600700 1 -- 172.21.15.1:0/608110615 wait complete. 2024-04-26T01:38:04.965 INFO:teuthology.orchestra.run.smithi001.stdout:mon is available 2024-04-26T01:38:04.965 INFO:teuthology.orchestra.run.smithi001.stdout:Assimilating anything we can from ceph.conf... 2024-04-26T01:38:06.084 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:05 smithi001 ceph-mon[27457]: from='client.? 172.21.15.1:0/3110037153' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T01:38:06.084 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:05 smithi001 ceph-mon[27457]: from='client.? 172.21.15.1:0/3110037153' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-26T01:38:06.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-04-26T01:38:06.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T01:38:06.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout fsid = 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:38:06.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T01:38:06.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.1:3300,v1:172.21.15.1:6789] 2024-04-26T01:38:06.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T01:38:06.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T01:38:06.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T01:38:06.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T01:38:06.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-04-26T01:38:06.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T01:38:06.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T01:38:06.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-04-26T01:38:06.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T01:38:06.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-26T01:38:06.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T01:38:06.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T01:38:06.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T01:38:06.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T01:38:06.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T01:38:06.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.688+0000 7f53e4600700 1 Processor -- start 2024-04-26T01:38:06.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.688+0000 7f53e4600700 1 -- start start 2024-04-26T01:38:06.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.689+0000 7f53e4600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53dc10a830 0x7f53dc10ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:06.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.689+0000 7f53e4600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc10b220 con 0x7f53dc10a830 2024-04-26T01:38:06.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.689+0000 7f53e1c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53dc10a830 0x7f53dc10ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:06.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.689+0000 7f53e1c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53dc10a830 0x7f53dc10ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39166/0 (socket says 172.21.15.1:39166) 2024-04-26T01:38:06.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.689+0000 7f53e1c00700 1 -- 172.21.15.1:0/4262082349 learned_addr learned my addr 172.21.15.1:0/4262082349 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:06.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.690+0000 7f53e1c00700 1 -- 172.21.15.1:0/4262082349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53dc10ba00 con 0x7f53dc10a830 2024-04-26T01:38:06.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.690+0000 7f53e1c00700 1 --2- 172.21.15.1:0/4262082349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53dc10a830 0x7f53dc10ac50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f53d00098a0 tx=0x7f53d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a194426f12dcb404 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:06.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.690+0000 7f53dbe00700 1 -- 172.21.15.1:0/4262082349 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53d0004160 con 0x7f53dc10a830 2024-04-26T01:38:06.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.691+0000 7f53dbe00700 1 -- 172.21.15.1:0/4262082349 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d0004300 con 0x7f53dc10a830 2024-04-26T01:38:06.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.691+0000 7f53dbe00700 1 -- 172.21.15.1:0/4262082349 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53d00036a0 con 0x7f53dc10a830 2024-04-26T01:38:06.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.691+0000 7f53e4600700 1 -- 172.21.15.1:0/4262082349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53dc10a830 msgr2=0x7f53dc10ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:06.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.691+0000 7f53e4600700 1 --2- 172.21.15.1:0/4262082349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53dc10a830 0x7f53dc10ac50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f53d00098a0 tx=0x7f53d0009bb0 comp rx=0 tx=0).stop 2024-04-26T01:38:06.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.692+0000 7f53e4600700 1 -- 172.21.15.1:0/4262082349 shutdown_connections 2024-04-26T01:38:06.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.692+0000 7f53e4600700 1 --2- 172.21.15.1:0/4262082349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53dc10a830 0x7f53dc10ac50 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:06.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.692+0000 7f53e4600700 1 -- 172.21.15.1:0/4262082349 >> 172.21.15.1:0/4262082349 conn(0x7f53dc07dbc0 msgr2=0x7f53dc07dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:06.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.692+0000 7f53e4600700 1 -- 172.21.15.1:0/4262082349 shutdown_connections 2024-04-26T01:38:06.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.692+0000 7f53e4600700 1 -- 172.21.15.1:0/4262082349 wait complete. 2024-04-26T01:38:06.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.693+0000 7f53e4600700 1 Processor -- start 2024-04-26T01:38:06.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.693+0000 7f53e4600700 1 -- start start 2024-04-26T01:38:06.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.694+0000 7f53e4600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53dc10a830 0x7f53dc19def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:06.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.694+0000 7f53e4600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc10bf30 con 0x7f53dc10a830 2024-04-26T01:38:06.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.694+0000 7f53e1c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53dc10a830 0x7f53dc19def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:06.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.694+0000 7f53e1c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53dc10a830 0x7f53dc19def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39182/0 (socket says 172.21.15.1:39182) 2024-04-26T01:38:06.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.694+0000 7f53e1c00700 1 -- 172.21.15.1:0/3110037153 learned_addr learned my addr 172.21.15.1:0/3110037153 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:06.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.695+0000 7f53e1c00700 1 -- 172.21.15.1:0/3110037153 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53dc19e430 con 0x7f53dc10a830 2024-04-26T01:38:06.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.695+0000 7f53e1c00700 1 --2- 172.21.15.1:0/3110037153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53dc10a830 0x7f53dc19def0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f53d0000c00 tx=0x7f53d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:06.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.696+0000 7f53da000700 1 -- 172.21.15.1:0/3110037153 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53d0003db0 con 0x7f53dc10a830 2024-04-26T01:38:06.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.696+0000 7f53da000700 1 -- 172.21.15.1:0/3110037153 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d0011630 con 0x7f53dc10a830 2024-04-26T01:38:06.357 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.696+0000 7f53da000700 1 -- 172.21.15.1:0/3110037153 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53d0011950 con 0x7f53dc10a830 2024-04-26T01:38:06.357 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.696+0000 7f53e4600700 1 -- 172.21.15.1:0/3110037153 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53dc19e6c0 con 0x7f53dc10a830 2024-04-26T01:38:06.357 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.696+0000 7f53e4600700 1 -- 172.21.15.1:0/3110037153 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53dc1a13b0 con 0x7f53dc10a830 2024-04-26T01:38:06.357 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.696+0000 7f53da000700 1 -- 172.21.15.1:0/3110037153 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f53d001a460 con 0x7f53dc10a830 2024-04-26T01:38:06.357 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.696+0000 7f53da000700 1 -- 172.21.15.1:0/3110037153 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f53d001b780 con 0x7f53dc10a830 2024-04-26T01:38:06.357 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.697+0000 7f53e4600700 1 -- 172.21.15.1:0/3110037153 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53dc0640a0 con 0x7f53dc10a830 2024-04-26T01:38:06.358 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.700+0000 7f53da000700 1 -- 172.21.15.1:0/3110037153 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f53d001b980 con 0x7f53dc10a830 2024-04-26T01:38:06.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.779+0000 7f53e4600700 1 -- 172.21.15.1:0/3110037153 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f53dc19eba0 con 0x7f53dc10a830 2024-04-26T01:38:06.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.787+0000 7f53da000700 1 -- 172.21.15.1:0/3110037153 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+490 (secure 0 0 0) 0x7f53dc19eba0 con 0x7f53dc10a830 2024-04-26T01:38:06.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.788+0000 7f53e4600700 1 -- 172.21.15.1:0/3110037153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53dc10a830 msgr2=0x7f53dc19def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:06.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.788+0000 7f53e4600700 1 --2- 172.21.15.1:0/3110037153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53dc10a830 0x7f53dc19def0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f53d0000c00 tx=0x7f53d0004750 comp rx=0 tx=0).stop 2024-04-26T01:38:06.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.789+0000 7f53e4600700 1 -- 172.21.15.1:0/3110037153 shutdown_connections 2024-04-26T01:38:06.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.789+0000 7f53e4600700 1 --2- 172.21.15.1:0/3110037153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53dc10a830 0x7f53dc19def0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:06.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.789+0000 7f53e4600700 1 -- 172.21.15.1:0/3110037153 >> 172.21.15.1:0/3110037153 conn(0x7f53dc07dbc0 msgr2=0x7f53dc194b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:06.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.789+0000 7f53e4600700 1 -- 172.21.15.1:0/3110037153 shutdown_connections 2024-04-26T01:38:06.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:05.789+0000 7f53e4600700 1 -- 172.21.15.1:0/3110037153 wait complete. 2024-04-26T01:38:06.359 INFO:teuthology.orchestra.run.smithi001.stdout:Generating new minimal ceph.conf... 2024-04-26T01:38:07.377 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:07 smithi001 ceph-mon[27457]: from='client.? 172.21.15.1:0/3208526475' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:38:07.740 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.002+0000 7f486a600700 1 Processor -- start 2024-04-26T01:38:07.740 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.003+0000 7f486a600700 1 -- start start 2024-04-26T01:38:07.740 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.003+0000 7f486a600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f486407dd70 0x7f486407c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:07.740 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.003+0000 7f486a600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f486407c7a0 con 0x7f486407dd70 2024-04-26T01:38:07.740 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.004+0000 7f4863400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f486407dd70 0x7f486407c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:07.740 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.004+0000 7f4863400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f486407dd70 0x7f486407c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39190/0 (socket says 172.21.15.1:39190) 2024-04-26T01:38:07.741 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.004+0000 7f4863400700 1 -- 172.21.15.1:0/2650758432 learned_addr learned my addr 172.21.15.1:0/2650758432 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:07.741 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.004+0000 7f4863400700 1 -- 172.21.15.1:0/2650758432 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486407c920 con 0x7f486407dd70 2024-04-26T01:38:07.741 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.004+0000 7f4863400700 1 --2- 172.21.15.1:0/2650758432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f486407dd70 0x7f486407c1d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f48540098a0 tx=0x7f4854009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f71ed64e3912e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:07.741 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.005+0000 7f4862000700 1 -- 172.21.15.1:0/2650758432 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4854004160 con 0x7f486407dd70 2024-04-26T01:38:07.741 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.005+0000 7f4862000700 1 -- 172.21.15.1:0/2650758432 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4854004300 con 0x7f486407dd70 2024-04-26T01:38:07.741 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.005+0000 7f4862000700 1 -- 172.21.15.1:0/2650758432 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4854011af0 con 0x7f486407dd70 2024-04-26T01:38:07.741 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.005+0000 7f486a600700 1 -- 172.21.15.1:0/2650758432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f486407dd70 msgr2=0x7f486407c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:07.741 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.005+0000 7f486a600700 1 --2- 172.21.15.1:0/2650758432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f486407dd70 0x7f486407c1d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f48540098a0 tx=0x7f4854009bb0 comp rx=0 tx=0).stop 2024-04-26T01:38:07.741 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.006+0000 7f486a600700 1 -- 172.21.15.1:0/2650758432 shutdown_connections 2024-04-26T01:38:07.741 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.006+0000 7f486a600700 1 --2- 172.21.15.1:0/2650758432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f486407dd70 0x7f486407c1d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:07.741 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.006+0000 7f486a600700 1 -- 172.21.15.1:0/2650758432 >> 172.21.15.1:0/2650758432 conn(0x7f4864103c70 msgr2=0x7f4864106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:07.741 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.006+0000 7f486a600700 1 -- 172.21.15.1:0/2650758432 shutdown_connections 2024-04-26T01:38:07.741 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.006+0000 7f486a600700 1 -- 172.21.15.1:0/2650758432 wait complete. 2024-04-26T01:38:07.741 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.007+0000 7f486a600700 1 Processor -- start 2024-04-26T01:38:07.742 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.007+0000 7f486a600700 1 -- start start 2024-04-26T01:38:07.742 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.007+0000 7f486a600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f486407dd70 0x7f486419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:07.742 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.007+0000 7f486a600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f486410eb80 con 0x7f486407dd70 2024-04-26T01:38:07.742 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.008+0000 7f4863400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f486407dd70 0x7f486419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:07.742 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.008+0000 7f4863400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f486407dd70 0x7f486419e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39204/0 (socket says 172.21.15.1:39204) 2024-04-26T01:38:07.742 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.008+0000 7f4863400700 1 -- 172.21.15.1:0/3208526475 learned_addr learned my addr 172.21.15.1:0/3208526475 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:07.742 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.008+0000 7f4863400700 1 -- 172.21.15.1:0/3208526475 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486419ec40 con 0x7f486407dd70 2024-04-26T01:38:07.742 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.008+0000 7f4863400700 1 --2- 172.21.15.1:0/3208526475 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f486407dd70 0x7f486419e700 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4854011ac0 tx=0x7f4854011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:07.742 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.009+0000 7f485be00700 1 -- 172.21.15.1:0/3208526475 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4854003970 con 0x7f486407dd70 2024-04-26T01:38:07.742 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.009+0000 7f485be00700 1 -- 172.21.15.1:0/3208526475 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4854003b10 con 0x7f486407dd70 2024-04-26T01:38:07.742 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.009+0000 7f485be00700 1 -- 172.21.15.1:0/3208526475 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4854024610 con 0x7f486407dd70 2024-04-26T01:38:07.742 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.009+0000 7f486a600700 1 -- 172.21.15.1:0/3208526475 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486419eed0 con 0x7f486407dd70 2024-04-26T01:38:07.742 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.009+0000 7f486a600700 1 -- 172.21.15.1:0/3208526475 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48641a1bc0 con 0x7f486407dd70 2024-04-26T01:38:07.742 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.009+0000 7f485be00700 1 -- 172.21.15.1:0/3208526475 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4854024bd0 con 0x7f486407dd70 2024-04-26T01:38:07.743 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.010+0000 7f485be00700 1 -- 172.21.15.1:0/3208526475 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f48540193c0 con 0x7f486407dd70 2024-04-26T01:38:07.743 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.010+0000 7f486a600700 1 -- 172.21.15.1:0/3208526475 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48640640a0 con 0x7f486407dd70 2024-04-26T01:38:07.743 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.013+0000 7f485be00700 1 -- 172.21.15.1:0/3208526475 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f485401a9e0 con 0x7f486407dd70 2024-04-26T01:38:07.743 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.092+0000 7f486a600700 1 -- 172.21.15.1:0/3208526475 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f48641a2100 con 0x7f486407dd70 2024-04-26T01:38:07.743 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.093+0000 7f485be00700 1 -- 172.21.15.1:0/3208526475 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7f485401a770 con 0x7f486407dd70 2024-04-26T01:38:07.743 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.095+0000 7f486a600700 1 -- 172.21.15.1:0/3208526475 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f486407dd70 msgr2=0x7f486419e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:07.743 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.095+0000 7f486a600700 1 --2- 172.21.15.1:0/3208526475 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f486407dd70 0x7f486419e700 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4854011ac0 tx=0x7f4854011ba0 comp rx=0 tx=0).stop 2024-04-26T01:38:07.743 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.095+0000 7f486a600700 1 -- 172.21.15.1:0/3208526475 shutdown_connections 2024-04-26T01:38:07.743 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.095+0000 7f486a600700 1 --2- 172.21.15.1:0/3208526475 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f486407dd70 0x7f486419e700 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:07.743 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.095+0000 7f486a600700 1 -- 172.21.15.1:0/3208526475 >> 172.21.15.1:0/3208526475 conn(0x7f4864103c70 msgr2=0x7f4864104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:07.743 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.095+0000 7f486a600700 1 -- 172.21.15.1:0/3208526475 shutdown_connections 2024-04-26T01:38:07.743 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:07.095+0000 7f486a600700 1 -- 172.21.15.1:0/3208526475 wait complete. 2024-04-26T01:38:07.743 INFO:teuthology.orchestra.run.smithi001.stdout:Restarting the monitor... 2024-04-26T01:38:08.127 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:07 smithi001 systemd[1]: Stopping Ceph mon.a for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T01:38:13.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:12 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-a[27453]: 2024-04-26T01:38:12.900+0000 7f5203600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T01:38:13.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:12 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-a[27453]: 2024-04-26T01:38:12.900+0000 7f5203600700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-26T01:38:13.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:12 smithi001 podman[27697]: 2024-04-26 01:38:12.962824249 +0000 UTC m=+4.795283756 container died a1ff1b41424de7a5a1d2896d54528124a922a2a3ed467edae5625232274725ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-04-26T01:38:17.326 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:17 smithi001 podman[27697]: 2024-04-26 01:38:17.084716301 +0000 UTC m=+8.917175808 container cleanup a1ff1b41424de7a5a1d2896d54528124a922a2a3ed467edae5625232274725ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T01:38:17.326 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:17 smithi001 bash[27697]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-a 2024-04-26T01:38:17.627 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:17 smithi001 podman[27711]: 2024-04-26 01:38:17.328113795 +0000 UTC m=+4.353556321 container remove a1ff1b41424de7a5a1d2896d54528124a922a2a3ed467edae5625232274725ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T01:38:18.307 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:17 smithi001 systemd[1]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-26T01:38:18.307 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:17 smithi001 systemd[1]: Stopped Ceph mon.a for 77cfcc80-036d-11ef-bc93-c7b262605968. 2024-04-26T01:38:18.307 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:17 smithi001 systemd[1]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mon.a.service: Consumed 2.050s CPU time. 2024-04-26T01:38:18.307 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:17 smithi001 systemd[1]: Starting Ceph mon.a for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T01:38:18.597 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 podman[27802]: 2024-04-26 01:38:18.219198646 +0000 UTC m=+0.042452078 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:38:18.597 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 podman[27802]: 2024-04-26 01:38:18.363881373 +0000 UTC m=+0.187134850 container create 42fcca49f2d3af9466a6e2432a77e3ed5c221f4cca3e86821263bd49aa4eefe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T01:38:18.597 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 podman[27802]: 2024-04-26 01:38:18.600529841 +0000 UTC m=+0.423783327 container init 42fcca49f2d3af9466a6e2432a77e3ed5c221f4cca3e86821263bd49aa4eefe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-26T01:38:18.687 INFO:teuthology.orchestra.run.smithi001.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-26T01:38:18.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 podman[27802]: 2024-04-26 01:38:18.610379393 +0000 UTC m=+0.433632863 container start 42fcca49f2d3af9466a6e2432a77e3ed5c221f4cca3e86821263bd49aa4eefe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T01:38:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T01:38:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-26T01:38:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: pidfile_write: ignore empty --pid-file 2024-04-26T01:38:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: load: jerasure load: lrc 2024-04-26T01:38:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: RocksDB version: 7.9.2 2024-04-26T01:38:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Git sha 0 2024-04-26T01:38:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-26T01:38:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: DB SUMMARY 2024-04-26T01:38:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: DB Session ID: UUW77DUFVBWP845ZKOEF 2024-04-26T01:38:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: CURRENT file: CURRENT 2024-04-26T01:38:18.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T01:38:18.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-26T01:38:18.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-26T01:38:18.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106006 ; 2024-04-26T01:38:18.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.error_if_exists: 0 2024-04-26T01:38:18.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.create_if_missing: 0 2024-04-26T01:38:18.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.paranoid_checks: 1 2024-04-26T01:38:18.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T01:38:18.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T01:38:18.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T01:38:18.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.env: 0x561df99c1cc0 2024-04-26T01:38:18.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T01:38:18.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.info_log: 0x561e2ae13c40 2024-04-26T01:38:18.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T01:38:18.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.statistics: (nil) 2024-04-26T01:38:18.881 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.use_fsync: 0 2024-04-26T01:38:18.881 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_log_file_size: 0 2024-04-26T01:38:18.881 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T01:38:18.881 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T01:38:18.881 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T01:38:18.881 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T01:38:18.881 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.allow_fallocate: 1 2024-04-26T01:38:18.882 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T01:38:18.882 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T01:38:18.882 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.use_direct_reads: 0 2024-04-26T01:38:18.882 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T01:38:18.882 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T01:38:18.882 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.db_log_dir: 2024-04-26T01:38:18.882 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.wal_dir: 2024-04-26T01:38:18.882 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T01:38:18.883 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T01:38:18.883 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T01:38:18.883 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T01:38:18.883 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T01:38:18.883 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T01:38:18.883 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T01:38:18.883 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T01:38:18.884 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.write_buffer_manager: 0x561e2ada70e0 2024-04-26T01:38:18.884 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T01:38:18.884 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T01:38:18.884 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T01:38:18.884 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T01:38:18.884 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T01:38:18.884 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T01:38:18.884 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T01:38:18.884 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T01:38:18.884 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.unordered_write: 0 2024-04-26T01:38:18.884 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T01:38:18.884 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T01:38:18.884 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T01:38:18.885 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T01:38:18.885 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.row_cache: None 2024-04-26T01:38:18.885 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.wal_filter: None 2024-04-26T01:38:18.885 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T01:38:18.885 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T01:38:18.885 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.two_write_queues: 0 2024-04-26T01:38:18.885 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T01:38:18.885 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.wal_compression: 0 2024-04-26T01:38:18.885 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.atomic_flush: 0 2024-04-26T01:38:18.885 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T01:38:18.885 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T01:38:18.885 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T01:38:18.885 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.log_readahead_size: 0 2024-04-26T01:38:18.886 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T01:38:18.886 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T01:38:18.886 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T01:38:18.886 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T01:38:18.886 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T01:38:18.886 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T01:38:18.886 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T01:38:18.886 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_background_jobs: 2 2024-04-26T01:38:18.886 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_background_compactions: -1 2024-04-26T01:38:18.886 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_subcompactions: 1 2024-04-26T01:38:18.886 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T01:38:18.886 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T01:38:18.886 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T01:38:18.887 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T01:38:18.887 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T01:38:18.887 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T01:38:18.887 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T01:38:18.887 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T01:38:18.887 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_open_files: -1 2024-04-26T01:38:18.887 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T01:38:18.887 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T01:38:18.887 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T01:38:18.887 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T01:38:18.887 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_background_flushes: -1 2024-04-26T01:38:18.887 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Compression algorithms supported: 2024-04-26T01:38:18.887 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T01:38:18.887 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: kZSTD supported: 0 2024-04-26T01:38:18.887 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: kXpressCompression supported: 0 2024-04-26T01:38:18.888 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T01:38:18.888 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: kZlibCompression supported: 1 2024-04-26T01:38:18.888 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: kSnappyCompression supported: 1 2024-04-26T01:38:18.888 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: kLZ4Compression supported: 1 2024-04-26T01:38:18.888 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: kBZip2Compression supported: 0 2024-04-26T01:38:18.888 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T01:38:18.888 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T01:38:18.888 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-26T01:38:18.888 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T01:38:18.888 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T01:38:18.888 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.merge_operator: 2024-04-26T01:38:18.888 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compaction_filter: None 2024-04-26T01:38:18.888 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T01:38:18.888 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T01:38:18.889 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T01:38:18.889 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T01:38:18.889 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561e2ae13e00) 2024-04-26T01:38:18.889 INFO:journalctl@ceph.mon.a.smithi001.stdout: cache_index_and_filter_blocks: 1 2024-04-26T01:38:18.889 INFO:journalctl@ceph.mon.a.smithi001.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T01:38:18.889 INFO:journalctl@ceph.mon.a.smithi001.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T01:38:18.889 INFO:journalctl@ceph.mon.a.smithi001.stdout: pin_top_level_index_and_filter: 1 2024-04-26T01:38:18.889 INFO:journalctl@ceph.mon.a.smithi001.stdout: index_type: 0 2024-04-26T01:38:18.889 INFO:journalctl@ceph.mon.a.smithi001.stdout: data_block_index_type: 0 2024-04-26T01:38:18.889 INFO:journalctl@ceph.mon.a.smithi001.stdout: index_shortening: 1 2024-04-26T01:38:18.889 INFO:journalctl@ceph.mon.a.smithi001.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T01:38:18.889 INFO:journalctl@ceph.mon.a.smithi001.stdout: checksum: 4 2024-04-26T01:38:18.889 INFO:journalctl@ceph.mon.a.smithi001.stdout: no_block_cache: 0 2024-04-26T01:38:18.890 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache: 0x561e2add91f0 2024-04-26T01:38:18.890 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache_name: BinnedLRUCache 2024-04-26T01:38:18.890 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache_options: 2024-04-26T01:38:18.890 INFO:journalctl@ceph.mon.a.smithi001.stdout: capacity : 536870912 2024-04-26T01:38:18.890 INFO:journalctl@ceph.mon.a.smithi001.stdout: num_shard_bits : 4 2024-04-26T01:38:18.890 INFO:journalctl@ceph.mon.a.smithi001.stdout: strict_capacity_limit : 0 2024-04-26T01:38:18.890 INFO:journalctl@ceph.mon.a.smithi001.stdout: high_pri_pool_ratio: 0.000 2024-04-26T01:38:18.890 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache_compressed: (nil) 2024-04-26T01:38:18.890 INFO:journalctl@ceph.mon.a.smithi001.stdout: persistent_cache: (nil) 2024-04-26T01:38:18.890 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_size: 4096 2024-04-26T01:38:18.890 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_size_deviation: 10 2024-04-26T01:38:18.890 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_restart_interval: 16 2024-04-26T01:38:18.890 INFO:journalctl@ceph.mon.a.smithi001.stdout: index_block_restart_interval: 1 2024-04-26T01:38:18.891 INFO:journalctl@ceph.mon.a.smithi001.stdout: metadata_block_size: 4096 2024-04-26T01:38:18.891 INFO:journalctl@ceph.mon.a.smithi001.stdout: partition_filters: 0 2024-04-26T01:38:18.891 INFO:journalctl@ceph.mon.a.smithi001.stdout: use_delta_encoding: 1 2024-04-26T01:38:18.891 INFO:journalctl@ceph.mon.a.smithi001.stdout: filter_policy: bloomfilter 2024-04-26T01:38:18.891 INFO:journalctl@ceph.mon.a.smithi001.stdout: whole_key_filtering: 1 2024-04-26T01:38:18.891 INFO:journalctl@ceph.mon.a.smithi001.stdout: verify_compression: 0 2024-04-26T01:38:18.891 INFO:journalctl@ceph.mon.a.smithi001.stdout: read_amp_bytes_per_bit: 0 2024-04-26T01:38:18.891 INFO:journalctl@ceph.mon.a.smithi001.stdout: format_version: 5 2024-04-26T01:38:18.891 INFO:journalctl@ceph.mon.a.smithi001.stdout: enable_index_compression: 1 2024-04-26T01:38:18.891 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_align: 0 2024-04-26T01:38:18.891 INFO:journalctl@ceph.mon.a.smithi001.stdout: max_auto_readahead_size: 262144 2024-04-26T01:38:18.891 INFO:journalctl@ceph.mon.a.smithi001.stdout: prepopulate_block_cache: 0 2024-04-26T01:38:18.891 INFO:journalctl@ceph.mon.a.smithi001.stdout: initial_auto_readahead_size: 8192 2024-04-26T01:38:18.891 INFO:journalctl@ceph.mon.a.smithi001.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T01:38:18.892 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T01:38:18.892 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T01:38:18.892 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compression: NoCompression 2024-04-26T01:38:18.892 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T01:38:18.892 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T01:38:18.892 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T01:38:18.892 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.num_levels: 7 2024-04-26T01:38:18.892 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T01:38:18.892 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T01:38:18.892 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T01:38:18.892 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T01:38:18.892 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T01:38:18.892 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T01:38:18.892 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T01:38:18.892 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T01:38:18.893 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T01:38:18.893 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T01:38:18.893 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T01:38:18.893 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T01:38:18.893 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T01:38:18.893 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T01:38:18.893 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T01:38:18.893 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T01:38:18.893 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T01:38:18.893 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T01:38:18.893 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T01:38:18.893 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T01:38:18.893 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T01:38:18.893 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T01:38:18.893 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T01:38:18.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T01:38:18.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T01:38:18.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T01:38:18.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T01:38:18.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T01:38:18.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T01:38:18.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T01:38:18.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T01:38:18.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T01:38:18.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T01:38:18.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T01:38:18.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T01:38:18.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T01:38:18.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T01:38:18.895 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T01:38:18.895 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T01:38:18.895 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T01:38:18.895 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T01:38:18.896 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T01:38:18.896 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T01:38:18.896 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T01:38:18.896 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T01:38:18.896 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T01:38:18.896 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T01:38:18.896 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T01:38:18.896 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T01:38:18.896 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T01:38:18.896 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T01:38:18.896 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T01:38:18.896 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T01:38:18.896 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T01:38:18.896 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.inplace_update_support: 0 2024-04-26T01:38:18.897 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T01:38:18.897 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T01:38:18.897 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T01:38:18.897 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T01:38:18.897 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.bloom_locality: 0 2024-04-26T01:38:18.897 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.max_successive_merges: 0 2024-04-26T01:38:18.897 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T01:38:18.897 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T01:38:18.897 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T01:38:18.897 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T01:38:18.897 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.ttl: 2592000 2024-04-26T01:38:18.897 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T01:38:18.897 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T01:38:18.897 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T01:38:18.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.enable_blob_files: false 2024-04-26T01:38:18.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.min_blob_size: 0 2024-04-26T01:38:18.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T01:38:18.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T01:38:18.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T01:38:18.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T01:38:18.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T01:38:18.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T01:38:18.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T01:38:18.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T01:38:18.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-26T01:38:18.898 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-26T01:38:18.899 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 461bc4e4-f8a7-4b20-bd3e-bf1ee5a8ac7c 2024-04-26T01:38:18.899 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714095498668407, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-26T01:38:18.899 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-26T01:38:18.899 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714095498670242, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101058, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98947, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14405, "raw_average_key_size": 49, "raw_value_size": 91126, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714095498, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "461bc4e4-f8a7-4b20-bd3e-bf1ee5a8ac7c", "db_session_id": "UUW77DUFVBWP845ZKOEF", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-26T01:38:18.899 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714095498670493, "job": 1, "event": "recovery_finished"} 2024-04-26T01:38:18.899 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-26T01:38:18.899 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T01:38:18.899 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561e2aeee000 2024-04-26T01:38:18.899 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: DB pointer 0x561e2aed8000 2024-04-26T01:38:18.899 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T01:38:18.899 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T01:38:18.899 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** DB Stats ** 2024-04-26T01:38:18.899 INFO:journalctl@ceph.mon.a.smithi001.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T01:38:18.899 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T01:38:18.900 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T01:38:18.900 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T01:38:18.900 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T01:38:18.900 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T01:38:18.900 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T01:38:18.900 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-04-26T01:38:18.900 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** Compaction Stats [default] ** 2024-04-26T01:38:18.900 INFO:journalctl@ceph.mon.a.smithi001.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T01:38:18.900 INFO:journalctl@ceph.mon.a.smithi001.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T01:38:18.900 INFO:journalctl@ceph.mon.a.smithi001.stdout: L0 2/0 100.55 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 83.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T01:38:18.900 INFO:journalctl@ceph.mon.a.smithi001.stdout: Sum 2/0 100.55 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 83.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T01:38:18.900 INFO:journalctl@ceph.mon.a.smithi001.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 83.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T01:38:18.900 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-04-26T01:38:18.900 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** Compaction Stats [default] ** 2024-04-26T01:38:18.901 INFO:journalctl@ceph.mon.a.smithi001.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T01:38:18.901 INFO:journalctl@ceph.mon.a.smithi001.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T01:38:18.901 INFO:journalctl@ceph.mon.a.smithi001.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 83.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T01:38:18.901 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-04-26T01:38:18.901 INFO:journalctl@ceph.mon.a.smithi001.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T01:38:18.901 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-04-26T01:38:18.901 INFO:journalctl@ceph.mon.a.smithi001.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T01:38:18.901 INFO:journalctl@ceph.mon.a.smithi001.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T01:38:18.901 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T01:38:18.901 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T01:38:18.901 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T01:38:18.901 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T01:38:18.901 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative compaction: 0.00 GB write, 14.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T01:38:18.901 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval compaction: 0.00 GB write, 14.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T01:38:18.902 INFO:journalctl@ceph.mon.a.smithi001.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T01:38:18.902 INFO:journalctl@ceph.mon.a.smithi001.stdout: Block cache BinnedLRUCache@0x561e2add91f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 2.7e-05 secs_since: 0 2024-04-26T01:38:18.902 INFO:journalctl@ceph.mon.a.smithi001.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-26T01:38:18.902 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-04-26T01:38:18.902 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T01:38:18.902 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: starting mon.a rank 0 at public addrs [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] at bind addrs [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:38:18.902 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: mon.a@-1(???) e1 preinit fsid 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:38:18.902 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: mon.a@-1(???).mds e1 new map 2024-04-26T01:38:18.902 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: mon.a@-1(???).mds e1 print_map 2024-04-26T01:38:18.902 INFO:journalctl@ceph.mon.a.smithi001.stdout: e1 2024-04-26T01:38:18.902 INFO:journalctl@ceph.mon.a.smithi001.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T01:38:18.902 INFO:journalctl@ceph.mon.a.smithi001.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T01:38:18.902 INFO:journalctl@ceph.mon.a.smithi001.stdout: legacy client fscid: -1 2024-04-26T01:38:18.903 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-04-26T01:38:18.903 INFO:journalctl@ceph.mon.a.smithi001.stdout: No filesystems configured 2024-04-26T01:38:18.903 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T01:38:18.903 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T01:38:18.903 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T01:38:18.903 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T01:38:18.903 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-26T01:38:18.903 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 bash[27802]: 42fcca49f2d3af9466a6e2432a77e3ed5c221f4cca3e86821263bd49aa4eefe5 2024-04-26T01:38:18.903 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 systemd[1]: Started Ceph mon.a for 77cfcc80-036d-11ef-bc93-c7b262605968. 2024-04-26T01:38:18.904 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T01:38:18.904 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: monmap epoch 1 2024-04-26T01:38:18.904 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: fsid 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:38:18.904 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: last_changed 2024-04-26T01:37:59.408242+0000 2024-04-26T01:38:18.904 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: created 2024-04-26T01:37:59.408242+0000 2024-04-26T01:38:18.904 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: min_mon_release 19 (squid) 2024-04-26T01:38:18.904 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: election_strategy: 1 2024-04-26T01:38:18.904 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-04-26T01:38:18.904 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: fsmap 2024-04-26T01:38:18.904 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T01:38:18.904 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:18 smithi001 ceph-mon[27822]: mgrmap e1: no daemons active 2024-04-26T01:38:20.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.433+0000 7fe2fe200700 1 Processor -- start 2024-04-26T01:38:20.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.434+0000 7fe2fe200700 1 -- start start 2024-04-26T01:38:20.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.434+0000 7fe2fe200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f807dd50 0x7fe2f807c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:20.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.434+0000 7fe2fe200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f807c780 con 0x7fe2f807dd50 2024-04-26T01:38:20.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.435+0000 7fe2f7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f807dd50 0x7fe2f807c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:20.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.435+0000 7fe2f7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f807dd50 0x7fe2f807c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55280/0 (socket says 172.21.15.1:55280) 2024-04-26T01:38:20.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.435+0000 7fe2f7400700 1 -- 172.21.15.1:0/3464289725 learned_addr learned my addr 172.21.15.1:0/3464289725 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:20.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.435+0000 7fe2f7400700 1 -- 172.21.15.1:0/3464289725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f807c900 con 0x7fe2f807dd50 2024-04-26T01:38:20.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.436+0000 7fe2f7400700 1 --2- 172.21.15.1:0/3464289725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f807dd50 0x7fe2f807c1b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80098a0 tx=0x7fe2e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be92e8f3c3af0334 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:20.457 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.436+0000 7fe2f6000700 1 -- 172.21.15.1:0/3464289725 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2e8004160 con 0x7fe2f807dd50 2024-04-26T01:38:20.458 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.436+0000 7fe2f6000700 1 -- 172.21.15.1:0/3464289725 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe2e8004300 con 0x7fe2f807dd50 2024-04-26T01:38:20.458 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.436+0000 7fe2f6000700 1 -- 172.21.15.1:0/3464289725 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2e8011af0 con 0x7fe2f807dd50 2024-04-26T01:38:20.458 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.437+0000 7fe2fe200700 1 -- 172.21.15.1:0/3464289725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f807dd50 msgr2=0x7fe2f807c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:20.458 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.437+0000 7fe2fe200700 1 --2- 172.21.15.1:0/3464289725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f807dd50 0x7fe2f807c1b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80098a0 tx=0x7fe2e8009bb0 comp rx=0 tx=0).stop 2024-04-26T01:38:20.458 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.437+0000 7fe2fe200700 1 -- 172.21.15.1:0/3464289725 shutdown_connections 2024-04-26T01:38:20.458 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.437+0000 7fe2fe200700 1 --2- 172.21.15.1:0/3464289725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f807dd50 0x7fe2f807c1b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:20.458 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.437+0000 7fe2fe200700 1 -- 172.21.15.1:0/3464289725 >> 172.21.15.1:0/3464289725 conn(0x7fe2f8103c70 msgr2=0x7fe2f8106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:20.458 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.437+0000 7fe2fe200700 1 -- 172.21.15.1:0/3464289725 shutdown_connections 2024-04-26T01:38:20.458 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.437+0000 7fe2fe200700 1 -- 172.21.15.1:0/3464289725 wait complete. 2024-04-26T01:38:20.458 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.438+0000 7fe2fe200700 1 Processor -- start 2024-04-26T01:38:20.458 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.438+0000 7fe2fe200700 1 -- start start 2024-04-26T01:38:20.458 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.438+0000 7fe2fe200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f807dd50 0x7fe2f819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:20.458 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.438+0000 7fe2fe200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f810eb40 con 0x7fe2f807dd50 2024-04-26T01:38:20.459 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.439+0000 7fe2f7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f807dd50 0x7fe2f819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:20.459 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.439+0000 7fe2f7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f807dd50 0x7fe2f819e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55296/0 (socket says 172.21.15.1:55296) 2024-04-26T01:38:20.459 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.439+0000 7fe2f7400700 1 -- 172.21.15.1:0/3833927710 learned_addr learned my addr 172.21.15.1:0/3833927710 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:20.459 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.439+0000 7fe2f7400700 1 -- 172.21.15.1:0/3833927710 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f819ec50 con 0x7fe2f807dd50 2024-04-26T01:38:20.459 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.439+0000 7fe2f7400700 1 --2- 172.21.15.1:0/3833927710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f807dd50 0x7fe2f819e710 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8011ac0 tx=0x7fe2e8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:20.459 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.440+0000 7fe2efe00700 1 -- 172.21.15.1:0/3833927710 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2e8003970 con 0x7fe2f807dd50 2024-04-26T01:38:20.460 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.440+0000 7fe2efe00700 1 -- 172.21.15.1:0/3833927710 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe2e8003b10 con 0x7fe2f807dd50 2024-04-26T01:38:20.460 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.440+0000 7fe2efe00700 1 -- 172.21.15.1:0/3833927710 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2e8024610 con 0x7fe2f807dd50 2024-04-26T01:38:20.460 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.440+0000 7fe2fe200700 1 -- 172.21.15.1:0/3833927710 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f819eee0 con 0x7fe2f807dd50 2024-04-26T01:38:20.460 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.440+0000 7fe2fe200700 1 -- 172.21.15.1:0/3833927710 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f81a1bd0 con 0x7fe2f807dd50 2024-04-26T01:38:20.461 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.440+0000 7fe2efe00700 1 -- 172.21.15.1:0/3833927710 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe2e8024c20 con 0x7fe2f807dd50 2024-04-26T01:38:20.461 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.440+0000 7fe2efe00700 1 -- 172.21.15.1:0/3833927710 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe2e80199b0 con 0x7fe2f807dd50 2024-04-26T01:38:20.461 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.441+0000 7fe2fe200700 1 -- 172.21.15.1:0/3833927710 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2f80640a0 con 0x7fe2f807dd50 2024-04-26T01:38:20.461 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.444+0000 7fe2efe00700 1 -- 172.21.15.1:0/3833927710 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe2e801a9e0 con 0x7fe2f807dd50 2024-04-26T01:38:20.461 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.522+0000 7fe2fe200700 1 -- 172.21.15.1:0/3833927710 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fe2f81a1da0 con 0x7fe2f807dd50 2024-04-26T01:38:20.461 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.525+0000 7fe2efe00700 1 -- 172.21.15.1:0/3833927710 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fe2e8019bb0 con 0x7fe2f807dd50 2024-04-26T01:38:20.461 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.527+0000 7fe2fe200700 1 -- 172.21.15.1:0/3833927710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f807dd50 msgr2=0x7fe2f819e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:20.461 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.527+0000 7fe2fe200700 1 --2- 172.21.15.1:0/3833927710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f807dd50 0x7fe2f819e710 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8011ac0 tx=0x7fe2e8011ba0 comp rx=0 tx=0).stop 2024-04-26T01:38:20.461 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.527+0000 7fe2fe200700 1 -- 172.21.15.1:0/3833927710 shutdown_connections 2024-04-26T01:38:20.461 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.527+0000 7fe2fe200700 1 --2- 172.21.15.1:0/3833927710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f807dd50 0x7fe2f819e710 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:20.461 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.527+0000 7fe2fe200700 1 -- 172.21.15.1:0/3833927710 >> 172.21.15.1:0/3833927710 conn(0x7fe2f8103c70 msgr2=0x7fe2f8104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:20.461 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.527+0000 7fe2fe200700 1 -- 172.21.15.1:0/3833927710 shutdown_connections 2024-04-26T01:38:20.461 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:19.527+0000 7fe2fe200700 1 -- 172.21.15.1:0/3833927710 wait complete. 2024-04-26T01:38:20.462 INFO:teuthology.orchestra.run.smithi001.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-26T01:38:20.484 INFO:teuthology.orchestra.run.smithi001.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-26T01:38:20.484 INFO:teuthology.orchestra.run.smithi001.stdout:Creating mgr... 2024-04-26T01:38:20.486 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-26T01:38:20.487 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-26T01:38:20.877 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:20 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3833927710' entity='client.admin' 2024-04-26T01:38:20.907 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mgr.x 2024-04-26T01:38:20.908 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to reset failed state of unit ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-26T01:38:21.170 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-77cfcc80-036d-11ef-bc93-c7b262605968.target.wants/ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-77cfcc80-036d-11ef-bc93-c7b262605968@.service. 2024-04-26T01:38:21.595 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:21 smithi001 systemd[1]: Starting Ceph mgr.x for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T01:38:21.858 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:21 smithi001 podman[28044]: 2024-04-26 01:38:21.503462521 +0000 UTC m=+0.033901932 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:38:21.858 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:21 smithi001 podman[28044]: 2024-04-26 01:38:21.616557014 +0000 UTC m=+0.146996456 container create 8387a05c9c89fb71da1a5240a3872aee14cf844486d3c5d73b9ac5501ca5ba55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T01:38:21.858 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:21 smithi001 podman[28044]: 2024-04-26 01:38:21.861676302 +0000 UTC m=+0.392115750 container init 8387a05c9c89fb71da1a5240a3872aee14cf844486d3c5d73b9ac5501ca5ba55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5) 2024-04-26T01:38:22.035 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T01:38:22.035 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-04-26T01:38:22.054 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T01:38:22.054 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-04-26T01:38:22.054 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-04-26T01:38:22.055 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T01:38:22.077 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T01:38:22.077 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-04-26T01:38:22.098 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T01:38:22.098 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-04-26T01:38:22.098 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-04-26T01:38:22.098 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-26T01:38:22.099 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr to start... 2024-04-26T01:38:22.099 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr... 2024-04-26T01:38:22.123 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:21 smithi001 podman[28044]: 2024-04-26 01:38:21.872502919 +0000 UTC m=+0.402942367 container start 8387a05c9c89fb71da1a5240a3872aee14cf844486d3c5d73b9ac5501ca5ba55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T01:38:22.123 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:22 smithi001 bash[28044]: 8387a05c9c89fb71da1a5240a3872aee14cf844486d3c5d73b9ac5501ca5ba55 2024-04-26T01:38:22.123 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:22 smithi001 systemd[1]: Started Ceph mgr.x for 77cfcc80-036d-11ef-bc93-c7b262605968. 2024-04-26T01:38:22.377 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:22 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:22.126+0000 7f7749637200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:38:22.701 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:22 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:22.394+0000 7f7749637200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:38:22.701 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:22 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:22.532+0000 7f7749637200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:38:22.955 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:22 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2830197684' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T01:38:23.333 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:22 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:22.958+0000 7f7749637200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:38:23.333 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:23 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:23.085+0000 7f7749637200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:38:23.756 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-04-26T01:38:23.761 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:38:23.762 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "77cfcc80-036d-11ef-bc93-c7b262605968", 2024-04-26T01:38:23.762 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T01:38:23.762 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T01:38:23.762 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T01:38:23.762 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T01:38:23.762 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:23.762 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T01:38:23.762 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T01:38:23.762 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2024-04-26T01:38:23.762 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:38:23.762 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T01:38:23.762 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T01:38:23.762 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:38:23.762 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-04-26T01:38:23.763 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T01:38:23.763 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:38:23.763 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T01:38:23.763 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T01:38:23.763 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:23.763 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T01:38:23.763 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:38:23.763 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T01:38:23.763 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T01:38:23.763 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T01:38:23.763 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T01:38:23.763 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T01:38:23.763 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T01:38:23.763 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:23.764 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T01:38:23.764 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T01:38:23.764 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T01:38:23.764 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T01:38:23.764 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T01:38:23.764 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T01:38:23.764 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T01:38:23.764 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T01:38:23.764 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T01:38:23.764 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:23.764 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T01:38:23.764 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:38:23.764 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T01:38:23.764 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T01:38:23.764 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:23.765 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T01:38:23.765 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T01:38:23.765 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T01:38:23.765 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T01:38:23.765 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T01:38:23.765 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T01:38:23.765 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T01:38:23.765 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:38:23.765 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:38:23.770 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:23.770 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T01:38:23.770 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:38:23.770 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T01:38:03.531805+0000", 2024-04-26T01:38:23.770 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:38:23.770 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:23.770 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T01:38:23.770 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:38:23.771 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.715+0000 7fc84b400700 1 Processor -- start 2024-04-26T01:38:23.771 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.715+0000 7fc84b400700 1 -- start start 2024-04-26T01:38:23.771 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.715+0000 7fc84b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc844106ee0 0x7fc844107300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:23.771 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.715+0000 7fc84b400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc844063d20 con 0x7fc844106ee0 2024-04-26T01:38:23.771 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.716+0000 7fc848a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc844106ee0 0x7fc844107300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:23.771 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.716+0000 7fc848a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc844106ee0 0x7fc844107300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53774/0 (socket says 172.21.15.1:53774) 2024-04-26T01:38:23.771 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.716+0000 7fc848a00700 1 -- 172.21.15.1:0/3570212417 learned_addr learned my addr 172.21.15.1:0/3570212417 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:23.771 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.717+0000 7fc848a00700 1 -- 172.21.15.1:0/3570212417 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc844107840 con 0x7fc844106ee0 2024-04-26T01:38:23.771 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.717+0000 7fc848a00700 1 --2- 172.21.15.1:0/3570212417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc844106ee0 0x7fc844107300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc8380098a0 tx=0x7fc838009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faffdc08fb920933 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:23.771 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.717+0000 7fc843400700 1 -- 172.21.15.1:0/3570212417 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc838004160 con 0x7fc844106ee0 2024-04-26T01:38:23.771 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.717+0000 7fc843400700 1 -- 172.21.15.1:0/3570212417 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc838004300 con 0x7fc844106ee0 2024-04-26T01:38:23.771 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.717+0000 7fc843400700 1 -- 172.21.15.1:0/3570212417 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc838011af0 con 0x7fc844106ee0 2024-04-26T01:38:23.771 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.718+0000 7fc84b400700 1 -- 172.21.15.1:0/3570212417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc844106ee0 msgr2=0x7fc844107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:23.772 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.718+0000 7fc84b400700 1 --2- 172.21.15.1:0/3570212417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc844106ee0 0x7fc844107300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc8380098a0 tx=0x7fc838009bb0 comp rx=0 tx=0).stop 2024-04-26T01:38:23.772 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.718+0000 7fc84b400700 1 -- 172.21.15.1:0/3570212417 shutdown_connections 2024-04-26T01:38:23.772 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.718+0000 7fc84b400700 1 --2- 172.21.15.1:0/3570212417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc844106ee0 0x7fc844107300 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:23.772 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.718+0000 7fc84b400700 1 -- 172.21.15.1:0/3570212417 >> 172.21.15.1:0/3570212417 conn(0x7fc844102b80 msgr2=0x7fc844104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:23.772 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.718+0000 7fc84b400700 1 -- 172.21.15.1:0/3570212417 shutdown_connections 2024-04-26T01:38:23.772 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.718+0000 7fc84b400700 1 -- 172.21.15.1:0/3570212417 wait complete. 2024-04-26T01:38:23.772 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.720+0000 7fc84b400700 1 Processor -- start 2024-04-26T01:38:23.772 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.720+0000 7fc84b400700 1 -- start start 2024-04-26T01:38:23.772 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.720+0000 7fc84b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc844106ee0 0x7fc84419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:23.772 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.720+0000 7fc84b400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc844108800 con 0x7fc844106ee0 2024-04-26T01:38:23.772 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.721+0000 7fc848a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc844106ee0 0x7fc84419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:23.772 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.721+0000 7fc848a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc844106ee0 0x7fc84419e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53786/0 (socket says 172.21.15.1:53786) 2024-04-26T01:38:23.772 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.721+0000 7fc848a00700 1 -- 172.21.15.1:0/2830197684 learned_addr learned my addr 172.21.15.1:0/2830197684 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:23.772 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.721+0000 7fc848a00700 1 -- 172.21.15.1:0/2830197684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84419ec50 con 0x7fc844106ee0 2024-04-26T01:38:23.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.721+0000 7fc848a00700 1 --2- 172.21.15.1:0/2830197684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc844106ee0 0x7fc84419e710 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc838011ac0 tx=0x7fc838011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:23.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.722+0000 7fc841600700 1 -- 172.21.15.1:0/2830197684 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc838003970 con 0x7fc844106ee0 2024-04-26T01:38:23.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.722+0000 7fc841600700 1 -- 172.21.15.1:0/2830197684 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc838003b10 con 0x7fc844106ee0 2024-04-26T01:38:23.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.722+0000 7fc841600700 1 -- 172.21.15.1:0/2830197684 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc838024610 con 0x7fc844106ee0 2024-04-26T01:38:23.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.722+0000 7fc84b400700 1 -- 172.21.15.1:0/2830197684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84419eee0 con 0x7fc844106ee0 2024-04-26T01:38:23.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.722+0000 7fc84b400700 1 -- 172.21.15.1:0/2830197684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8441a1bd0 con 0x7fc844106ee0 2024-04-26T01:38:23.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.722+0000 7fc841600700 1 -- 172.21.15.1:0/2830197684 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc838024bd0 con 0x7fc844106ee0 2024-04-26T01:38:23.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.722+0000 7fc841600700 1 -- 172.21.15.1:0/2830197684 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc838019700 con 0x7fc844106ee0 2024-04-26T01:38:23.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.723+0000 7fc84b400700 1 -- 172.21.15.1:0/2830197684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8440640a0 con 0x7fc844106ee0 2024-04-26T01:38:23.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.726+0000 7fc841600700 1 -- 172.21.15.1:0/2830197684 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc83802ddf0 con 0x7fc844106ee0 2024-04-26T01:38:23.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.806+0000 7fc84b400700 1 -- 172.21.15.1:0/2830197684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc84402e180 con 0x7fc844106ee0 2024-04-26T01:38:23.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.806+0000 7fc841600700 1 -- 172.21.15.1:0/2830197684 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc83801a9e0 con 0x7fc844106ee0 2024-04-26T01:38:23.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.808+0000 7fc84b400700 1 -- 172.21.15.1:0/2830197684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc844106ee0 msgr2=0x7fc84419e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:23.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.808+0000 7fc84b400700 1 --2- 172.21.15.1:0/2830197684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc844106ee0 0x7fc84419e710 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc838011ac0 tx=0x7fc838011ba0 comp rx=0 tx=0).stop 2024-04-26T01:38:23.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.808+0000 7fc84b400700 1 -- 172.21.15.1:0/2830197684 shutdown_connections 2024-04-26T01:38:23.774 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.808+0000 7fc84b400700 1 --2- 172.21.15.1:0/2830197684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc844106ee0 0x7fc84419e710 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:23.774 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.808+0000 7fc84b400700 1 -- 172.21.15.1:0/2830197684 >> 172.21.15.1:0/2830197684 conn(0x7fc844102b80 msgr2=0x7fc8441953f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:23.774 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.808+0000 7fc84b400700 1 -- 172.21.15.1:0/2830197684 shutdown_connections 2024-04-26T01:38:23.774 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:22.808+0000 7fc84b400700 1 -- 172.21.15.1:0/2830197684 wait complete. 2024-04-26T01:38:23.774 INFO:teuthology.orchestra.run.smithi001.stdout:mgr not available, waiting (1/15)... 2024-04-26T01:38:25.127 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:24 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:24.720+0000 7f7749637200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:38:25.128 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:24 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:24.953+0000 7f7749637200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:38:25.561 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:25 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:25.170+0000 7f7749637200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:38:25.561 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:25 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:25.564+0000 7f7749637200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:38:25.877 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:25 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:25.688+0000 7f7749637200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:38:26.214 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:25 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:25.952+0000 7f7749637200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:38:26.472 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:26 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:26.217+0000 7f7749637200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:38:26.836 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:26 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/232548410' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T01:38:27.127 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:27 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:27.040+0000 7f7749637200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:38:27.173 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-04-26T01:38:27.173 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:38:27.173 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "77cfcc80-036d-11ef-bc93-c7b262605968", 2024-04-26T01:38:27.173 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T01:38:27.173 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T01:38:27.173 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T01:38:27.173 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T01:38:27.173 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:27.173 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T01:38:27.173 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T01:38:27.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2024-04-26T01:38:27.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:38:27.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T01:38:27.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T01:38:27.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:38:27.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-26T01:38:27.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T01:38:27.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:38:27.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T01:38:27.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T01:38:27.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:27.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T01:38:27.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:38:27.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T01:38:27.175 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T01:38:27.175 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T01:38:27.175 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T01:38:27.175 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T01:38:27.175 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T01:38:27.175 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:27.175 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T01:38:27.175 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T01:38:27.175 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T01:38:27.175 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T01:38:27.175 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T01:38:27.175 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T01:38:27.175 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T01:38:27.175 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T01:38:27.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T01:38:27.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:27.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T01:38:27.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:38:27.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T01:38:27.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T01:38:27.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:27.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T01:38:27.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T01:38:27.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T01:38:27.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T01:38:27.177 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T01:38:27.177 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T01:38:27.177 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T01:38:27.177 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:38:27.177 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:38:27.177 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:27.177 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T01:38:27.177 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:38:27.177 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T01:38:03.531805+0000", 2024-04-26T01:38:27.177 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:38:27.177 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:27.177 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T01:38:27.177 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:38:27.177 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.385+0000 7f83e7000700 1 Processor -- start 2024-04-26T01:38:27.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.385+0000 7f83e7000700 1 -- start start 2024-04-26T01:38:27.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.386+0000 7f83e7000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83e0106f20 0x7f83e0107340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:27.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.386+0000 7f83e7000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f83e0063ce0 con 0x7f83e0106f20 2024-04-26T01:38:27.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.386+0000 7f83dfe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83e0106f20 0x7f83e0107340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:27.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.386+0000 7f83dfe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83e0106f20 0x7f83e0107340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53798/0 (socket says 172.21.15.1:53798) 2024-04-26T01:38:27.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.386+0000 7f83dfe00700 1 -- 172.21.15.1:0/1803227110 learned_addr learned my addr 172.21.15.1:0/1803227110 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:27.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.387+0000 7f83dfe00700 1 -- 172.21.15.1:0/1803227110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e0107880 con 0x7f83e0106f20 2024-04-26T01:38:27.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.387+0000 7f83dfe00700 1 --2- 172.21.15.1:0/1803227110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83e0106f20 0x7f83e0107340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f83d00098a0 tx=0x7f83d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54e470c96ff49fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:27.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.387+0000 7f83dea00700 1 -- 172.21.15.1:0/1803227110 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83d0004160 con 0x7f83e0106f20 2024-04-26T01:38:27.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.387+0000 7f83dea00700 1 -- 172.21.15.1:0/1803227110 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f83d0004300 con 0x7f83e0106f20 2024-04-26T01:38:27.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.387+0000 7f83dea00700 1 -- 172.21.15.1:0/1803227110 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83d0011af0 con 0x7f83e0106f20 2024-04-26T01:38:27.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.388+0000 7f83e7000700 1 -- 172.21.15.1:0/1803227110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83e0106f20 msgr2=0x7f83e0107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:27.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.388+0000 7f83e7000700 1 --2- 172.21.15.1:0/1803227110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83e0106f20 0x7f83e0107340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f83d00098a0 tx=0x7f83d0009bb0 comp rx=0 tx=0).stop 2024-04-26T01:38:27.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.389+0000 7f83e7000700 1 -- 172.21.15.1:0/1803227110 shutdown_connections 2024-04-26T01:38:27.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.389+0000 7f83e7000700 1 --2- 172.21.15.1:0/1803227110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83e0106f20 0x7f83e0107340 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:27.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.389+0000 7f83e7000700 1 -- 172.21.15.1:0/1803227110 >> 172.21.15.1:0/1803227110 conn(0x7f83e0102b80 msgr2=0x7f83e0104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:27.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.389+0000 7f83e7000700 1 -- 172.21.15.1:0/1803227110 shutdown_connections 2024-04-26T01:38:27.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.389+0000 7f83e7000700 1 -- 172.21.15.1:0/1803227110 wait complete. 2024-04-26T01:38:27.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.390+0000 7f83e7000700 1 Processor -- start 2024-04-26T01:38:27.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.390+0000 7f83e7000700 1 -- start start 2024-04-26T01:38:27.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.390+0000 7f83e7000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83e0106f20 0x7f83e019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:27.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.390+0000 7f83e7000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f83e0108840 con 0x7f83e0106f20 2024-04-26T01:38:27.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.391+0000 7f83dfe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83e0106f20 0x7f83e019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:27.180 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.391+0000 7f83dfe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83e0106f20 0x7f83e019e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53808/0 (socket says 172.21.15.1:53808) 2024-04-26T01:38:27.180 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.391+0000 7f83dfe00700 1 -- 172.21.15.1:0/232548410 learned_addr learned my addr 172.21.15.1:0/232548410 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:27.180 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.391+0000 7f83dfe00700 1 -- 172.21.15.1:0/232548410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e019ec80 con 0x7f83e0106f20 2024-04-26T01:38:27.180 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.391+0000 7f83dfe00700 1 --2- 172.21.15.1:0/232548410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83e0106f20 0x7f83e019e740 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009fb0 tx=0x7f83d0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:27.180 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.392+0000 7f83dcc00700 1 -- 172.21.15.1:0/232548410 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83d00036a0 con 0x7f83e0106f20 2024-04-26T01:38:27.180 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.392+0000 7f83dcc00700 1 -- 172.21.15.1:0/232548410 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f83d001a430 con 0x7f83e0106f20 2024-04-26T01:38:27.180 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.392+0000 7f83e7000700 1 -- 172.21.15.1:0/232548410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e019ef10 con 0x7f83e0106f20 2024-04-26T01:38:27.180 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.392+0000 7f83e7000700 1 -- 172.21.15.1:0/232548410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e01a1c00 con 0x7f83e0106f20 2024-04-26T01:38:27.180 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.392+0000 7f83dcc00700 1 -- 172.21.15.1:0/232548410 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83d0019550 con 0x7f83e0106f20 2024-04-26T01:38:27.180 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.392+0000 7f83dcc00700 1 -- 172.21.15.1:0/232548410 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f83d0019be0 con 0x7f83e0106f20 2024-04-26T01:38:27.180 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.392+0000 7f83dcc00700 1 -- 172.21.15.1:0/232548410 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f83d0024730 con 0x7f83e0106f20 2024-04-26T01:38:27.181 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.393+0000 7f83e7000700 1 -- 172.21.15.1:0/232548410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83e00640a0 con 0x7f83e0106f20 2024-04-26T01:38:27.181 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.396+0000 7f83dcc00700 1 -- 172.21.15.1:0/232548410 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f83d0043e50 con 0x7f83e0106f20 2024-04-26T01:38:27.181 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.475+0000 7f83e7000700 1 -- 172.21.15.1:0/232548410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f83e002e180 con 0x7f83e0106f20 2024-04-26T01:38:27.181 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.476+0000 7f83dcc00700 1 -- 172.21.15.1:0/232548410 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f83d0032030 con 0x7f83e0106f20 2024-04-26T01:38:27.181 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.477+0000 7f83e7000700 1 -- 172.21.15.1:0/232548410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83e0106f20 msgr2=0x7f83e019e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:27.181 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.477+0000 7f83e7000700 1 --2- 172.21.15.1:0/232548410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83e0106f20 0x7f83e019e740 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009fb0 tx=0x7f83d0011940 comp rx=0 tx=0).stop 2024-04-26T01:38:27.181 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.477+0000 7f83e7000700 1 -- 172.21.15.1:0/232548410 shutdown_connections 2024-04-26T01:38:27.181 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.477+0000 7f83e7000700 1 --2- 172.21.15.1:0/232548410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83e0106f20 0x7f83e019e740 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:27.181 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.477+0000 7f83e7000700 1 -- 172.21.15.1:0/232548410 >> 172.21.15.1:0/232548410 conn(0x7f83e0102b80 msgr2=0x7f83e01953e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:27.181 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.477+0000 7f83e7000700 1 -- 172.21.15.1:0/232548410 shutdown_connections 2024-04-26T01:38:27.181 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:26.478+0000 7f83e7000700 1 -- 172.21.15.1:0/232548410 wait complete. 2024-04-26T01:38:27.182 INFO:teuthology.orchestra.run.smithi001.stdout:mgr not available, waiting (2/15)... 2024-04-26T01:38:27.877 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:27 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:27.615+0000 7f7749637200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:38:27.877 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:27 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:27.744+0000 7f7749637200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:38:28.377 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:28 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:28.015+0000 7f7749637200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:38:29.127 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:28 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:28.679+0000 7f7749637200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:38:29.127 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:28 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:28.805+0000 7f7749637200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:38:29.462 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:29 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:29.134+0000 7f7749637200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:38:29.462 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:29 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:29.332+0000 7f7749637200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:38:29.462 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:29 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:29.465+0000 7f7749637200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:38:29.788 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:29 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:29.624+0000 7f7749637200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:38:30.127 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:30 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/447766767' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T01:38:30.128 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:30 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:30.005+0000 7f7749637200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:38:30.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-04-26T01:38:30.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:38:30.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "77cfcc80-036d-11ef-bc93-c7b262605968", 2024-04-26T01:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T01:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T01:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T01:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T01:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T01:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T01:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2024-04-26T01:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T01:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T01:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-04-26T01:38:30.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T01:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T01:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T01:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T01:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T01:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T01:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T01:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T01:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T01:38:30.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T01:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T01:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T01:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T01:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T01:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T01:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T01:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T01:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T01:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T01:38:30.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T01:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T01:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T01:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T01:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T01:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T01:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T01:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T01:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T01:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T01:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:30.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T01:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T01:38:03.531805+0000", 2024-04-26T01:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T01:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.857+0000 7f144ce00700 1 Processor -- start 2024-04-26T01:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.857+0000 7f144ce00700 1 -- start start 2024-04-26T01:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.858+0000 7f144ce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1448109170 0x7f144810b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.858+0000 7f144ce00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1448063ca0 con 0x7f1448109170 2024-04-26T01:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.858+0000 7f1445e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1448109170 0x7f144810b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.858+0000 7f1445e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1448109170 0x7f144810b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53810/0 (socket says 172.21.15.1:53810) 2024-04-26T01:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.858+0000 7f1445e00700 1 -- 172.21.15.1:0/1513591992 learned_addr learned my addr 172.21.15.1:0/1513591992 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:30.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.859+0000 7f1445e00700 1 -- 172.21.15.1:0/1513591992 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144810baa0 con 0x7f1448109170 2024-04-26T01:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.859+0000 7f1445e00700 1 --2- 172.21.15.1:0/1513591992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1448109170 0x7f144810b560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f14380098a0 tx=0x7f1438009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272ca481ab167d6a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.859+0000 7f1444a00700 1 -- 172.21.15.1:0/1513591992 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1438004160 con 0x7f1448109170 2024-04-26T01:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.859+0000 7f1444a00700 1 -- 172.21.15.1:0/1513591992 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1438004300 con 0x7f1448109170 2024-04-26T01:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.859+0000 7f1444a00700 1 -- 172.21.15.1:0/1513591992 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1438011af0 con 0x7f1448109170 2024-04-26T01:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.860+0000 7f144ce00700 1 -- 172.21.15.1:0/1513591992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1448109170 msgr2=0x7f144810b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.860+0000 7f144ce00700 1 --2- 172.21.15.1:0/1513591992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1448109170 0x7f144810b560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f14380098a0 tx=0x7f1438009bb0 comp rx=0 tx=0).stop 2024-04-26T01:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.860+0000 7f144ce00700 1 -- 172.21.15.1:0/1513591992 shutdown_connections 2024-04-26T01:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.860+0000 7f144ce00700 1 --2- 172.21.15.1:0/1513591992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1448109170 0x7f144810b560 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.860+0000 7f144ce00700 1 -- 172.21.15.1:0/1513591992 >> 172.21.15.1:0/1513591992 conn(0x7f1448102b80 msgr2=0x7f1448104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.860+0000 7f144ce00700 1 -- 172.21.15.1:0/1513591992 shutdown_connections 2024-04-26T01:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.860+0000 7f144ce00700 1 -- 172.21.15.1:0/1513591992 wait complete. 2024-04-26T01:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.862+0000 7f144ce00700 1 Processor -- start 2024-04-26T01:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.862+0000 7f144ce00700 1 -- start start 2024-04-26T01:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.862+0000 7f144ce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1448109170 0x7f144819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:30.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.862+0000 7f144ce00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f144810c940 con 0x7f1448109170 2024-04-26T01:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.862+0000 7f1445e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1448109170 0x7f144819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.863+0000 7f1445e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1448109170 0x7f144819e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53814/0 (socket says 172.21.15.1:53814) 2024-04-26T01:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.863+0000 7f1445e00700 1 -- 172.21.15.1:0/447766767 learned_addr learned my addr 172.21.15.1:0/447766767 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.863+0000 7f1445e00700 1 -- 172.21.15.1:0/447766767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144819ec60 con 0x7f1448109170 2024-04-26T01:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.863+0000 7f1445e00700 1 --2- 172.21.15.1:0/447766767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1448109170 0x7f144819e720 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1438009fb0 tx=0x7f1438011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.863+0000 7f143ea00700 1 -- 172.21.15.1:0/447766767 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14380036a0 con 0x7f1448109170 2024-04-26T01:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.863+0000 7f143ea00700 1 -- 172.21.15.1:0/447766767 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f143801a430 con 0x7f1448109170 2024-04-26T01:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.863+0000 7f144ce00700 1 -- 172.21.15.1:0/447766767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144819eef0 con 0x7f1448109170 2024-04-26T01:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.864+0000 7f143ea00700 1 -- 172.21.15.1:0/447766767 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1438019550 con 0x7f1448109170 2024-04-26T01:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.864+0000 7f144ce00700 1 -- 172.21.15.1:0/447766767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14481a1be0 con 0x7f1448109170 2024-04-26T01:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.864+0000 7f143ea00700 1 -- 172.21.15.1:0/447766767 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1438019b90 con 0x7f1448109170 2024-04-26T01:38:30.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.864+0000 7f143ea00700 1 -- 172.21.15.1:0/447766767 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1438024730 con 0x7f1448109170 2024-04-26T01:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.865+0000 7f144ce00700 1 -- 172.21.15.1:0/447766767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14480640a0 con 0x7f1448109170 2024-04-26T01:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.869+0000 7f143ea00700 1 -- 172.21.15.1:0/447766767 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f1438043e50 con 0x7f1448109170 2024-04-26T01:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.949+0000 7f144ce00700 1 -- 172.21.15.1:0/447766767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f14481a2120 con 0x7f1448109170 2024-04-26T01:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.949+0000 7f143ea00700 1 -- 172.21.15.1:0/447766767 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f1438032030 con 0x7f1448109170 2024-04-26T01:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.951+0000 7f144ce00700 1 -- 172.21.15.1:0/447766767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1448109170 msgr2=0x7f144819e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.951+0000 7f144ce00700 1 --2- 172.21.15.1:0/447766767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1448109170 0x7f144819e720 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1438009fb0 tx=0x7f1438011940 comp rx=0 tx=0).stop 2024-04-26T01:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.951+0000 7f144ce00700 1 -- 172.21.15.1:0/447766767 shutdown_connections 2024-04-26T01:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.951+0000 7f144ce00700 1 --2- 172.21.15.1:0/447766767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1448109170 0x7f144819e720 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.951+0000 7f144ce00700 1 -- 172.21.15.1:0/447766767 >> 172.21.15.1:0/447766767 conn(0x7f1448102b80 msgr2=0x7f1448103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.951+0000 7f144ce00700 1 -- 172.21.15.1:0/447766767 shutdown_connections 2024-04-26T01:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:29.952+0000 7f144ce00700 1 -- 172.21.15.1:0/447766767 wait complete. 2024-04-26T01:38:30.647 INFO:teuthology.orchestra.run.smithi001.stdout:mgr not available, waiting (3/15)... 2024-04-26T01:38:30.877 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:30 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:30.799+0000 7f7749637200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:38:31.377 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:31 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:31.092+0000 7f7749637200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:38:31.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:31 smithi001 ceph-mon[27822]: Activating manager daemon x 2024-04-26T01:38:31.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:31 smithi001 ceph-mon[27822]: mgrmap e2: x(active, starting, since 0.0119611s) 2024-04-26T01:38:31.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:31 smithi001 ceph-mon[27822]: from='mgr.14100 172.21.15.1:0/3396561924' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T01:38:31.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:31 smithi001 ceph-mon[27822]: from='mgr.14100 172.21.15.1:0/3396561924' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T01:38:31.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:31 smithi001 ceph-mon[27822]: from='mgr.14100 172.21.15.1:0/3396561924' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T01:38:31.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:31 smithi001 ceph-mon[27822]: from='mgr.14100 172.21.15.1:0/3396561924' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:38:31.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:31 smithi001 ceph-mon[27822]: from='mgr.14100 172.21.15.1:0/3396561924' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T01:38:31.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:31 smithi001 ceph-mon[27822]: Manager daemon x is now available 2024-04-26T01:38:31.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:31 smithi001 ceph-mon[27822]: from='mgr.14100 172.21.15.1:0/3396561924' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:38:32.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:32 smithi001 ceph-mon[27822]: from='mgr.14100 172.21.15.1:0/3396561924' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:38:32.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:32 smithi001 ceph-mon[27822]: from='mgr.14100 172.21.15.1:0/3396561924' entity='mgr.x' 2024-04-26T01:38:32.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:32 smithi001 ceph-mon[27822]: from='mgr.14100 172.21.15.1:0/3396561924' entity='mgr.x' 2024-04-26T01:38:32.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:32 smithi001 ceph-mon[27822]: from='mgr.14100 172.21.15.1:0/3396561924' entity='mgr.x' 2024-04-26T01:38:32.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:32 smithi001 ceph-mon[27822]: mgrmap e3: x(active, since 1.02138s) 2024-04-26T01:38:34.239 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:34 smithi001 ceph-mon[27822]: mgrmap e4: x(active, since 2s) 2024-04-26T01:38:34.239 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:34 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1987989967' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T01:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-04-26T01:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "77cfcc80-036d-11ef-bc93-c7b262605968", 2024-04-26T01:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T01:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T01:38:34.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T01:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T01:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T01:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T01:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2024-04-26T01:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T01:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T01:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-26T01:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T01:38:34.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T01:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T01:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T01:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T01:38:34.259 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T01:38:34.268 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T01:38:34.268 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T01:38:34.268 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T01:38:34.268 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T01:38:34.268 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:34.268 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T01:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T01:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T01:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T01:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T01:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T01:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T01:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T01:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T01:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T01:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:38:34.269 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T01:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T01:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T01:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T01:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T01:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T01:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T01:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T01:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T01:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:34.270 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T01:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T01:38:03.531805+0000", 2024-04-26T01:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T01:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.300+0000 7fe2e7200700 1 Processor -- start 2024-04-26T01:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.300+0000 7fe2e7200700 1 -- start start 2024-04-26T01:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.301+0000 7fe2e7200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2e0108420 0x7fe2e0108840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.301+0000 7fe2e7200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e0108e10 con 0x7fe2e0108420 2024-04-26T01:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.301+0000 7fe2e5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2e0108420 0x7fe2e0108840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.301+0000 7fe2e5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2e0108420 0x7fe2e0108840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45208/0 (socket says 172.21.15.1:45208) 2024-04-26T01:38:34.271 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.301+0000 7fe2e5e00700 1 -- 172.21.15.1:0/463408908 learned_addr learned my addr 172.21.15.1:0/463408908 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.302+0000 7fe2e5e00700 1 -- 172.21.15.1:0/463408908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e0109620 con 0x7fe2e0108420 2024-04-26T01:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.302+0000 7fe2e5e00700 1 --2- 172.21.15.1:0/463408908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2e0108420 0x7fe2e0108840 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40098a0 tx=0x7fe2d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64d87733ebcbe9c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.303+0000 7fe2e4a00700 1 -- 172.21.15.1:0/463408908 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2d4004160 con 0x7fe2e0108420 2024-04-26T01:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.303+0000 7fe2e4a00700 1 -- 172.21.15.1:0/463408908 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe2d4004300 con 0x7fe2e0108420 2024-04-26T01:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.303+0000 7fe2e4a00700 1 -- 172.21.15.1:0/463408908 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2d4011af0 con 0x7fe2e0108420 2024-04-26T01:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.304+0000 7fe2e7200700 1 -- 172.21.15.1:0/463408908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2e0108420 msgr2=0x7fe2e0108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.304+0000 7fe2e7200700 1 --2- 172.21.15.1:0/463408908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2e0108420 0x7fe2e0108840 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40098a0 tx=0x7fe2d4009bb0 comp rx=0 tx=0).stop 2024-04-26T01:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.304+0000 7fe2e7200700 1 -- 172.21.15.1:0/463408908 shutdown_connections 2024-04-26T01:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.304+0000 7fe2e7200700 1 --2- 172.21.15.1:0/463408908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2e0108420 0x7fe2e0108840 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.304+0000 7fe2e7200700 1 -- 172.21.15.1:0/463408908 >> 172.21.15.1:0/463408908 conn(0x7fe2e0103a00 msgr2=0x7fe2e0105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.304+0000 7fe2e7200700 1 -- 172.21.15.1:0/463408908 shutdown_connections 2024-04-26T01:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.304+0000 7fe2e7200700 1 -- 172.21.15.1:0/463408908 wait complete. 2024-04-26T01:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.305+0000 7fe2e7200700 1 Processor -- start 2024-04-26T01:38:34.272 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.305+0000 7fe2e7200700 1 -- start start 2024-04-26T01:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.306+0000 7fe2e7200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2e019e7e0 0x7fe2e019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.306+0000 7fe2e7200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e010c850 con 0x7fe2e019e7e0 2024-04-26T01:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.306+0000 7fe2e5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2e019e7e0 0x7fe2e019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.306+0000 7fe2e5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2e019e7e0 0x7fe2e019ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45214/0 (socket says 172.21.15.1:45214) 2024-04-26T01:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.306+0000 7fe2e5e00700 1 -- 172.21.15.1:0/1987989967 learned_addr learned my addr 172.21.15.1:0/1987989967 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.306+0000 7fe2e5e00700 1 -- 172.21.15.1:0/1987989967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e019f140 con 0x7fe2e019e7e0 2024-04-26T01:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.307+0000 7fe2e5e00700 1 --2- 172.21.15.1:0/1987989967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2e019e7e0 0x7fe2e019ec00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4000c00 tx=0x7fe2d4004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.307+0000 7fe2daa00700 1 -- 172.21.15.1:0/1987989967 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2d40037d0 con 0x7fe2e019e7e0 2024-04-26T01:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.307+0000 7fe2daa00700 1 -- 172.21.15.1:0/1987989967 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe2d4003970 con 0x7fe2e019e7e0 2024-04-26T01:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.307+0000 7fe2daa00700 1 -- 172.21.15.1:0/1987989967 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2d401b440 con 0x7fe2e019e7e0 2024-04-26T01:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.307+0000 7fe2e7200700 1 -- 172.21.15.1:0/1987989967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2e019f3d0 con 0x7fe2e019e7e0 2024-04-26T01:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.307+0000 7fe2e7200700 1 -- 172.21.15.1:0/1987989967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2e01a1eb0 con 0x7fe2e019e7e0 2024-04-26T01:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.308+0000 7fe2daa00700 1 -- 172.21.15.1:0/1987989967 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fe2d401a440 con 0x7fe2e019e7e0 2024-04-26T01:38:34.273 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.308+0000 7fe2e7200700 1 -- 172.21.15.1:0/1987989967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2e010c9d0 con 0x7fe2e019e7e0 2024-04-26T01:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.309+0000 7fe2daa00700 1 --2- 172.21.15.1:0/1987989967 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7fe2c8039560 0x7fe2c803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.309+0000 7fe2daa00700 1 -- 172.21.15.1:0/1987989967 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe2d404da10 con 0x7fe2e019e7e0 2024-04-26T01:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.310+0000 7fe2e5400700 1 --2- 172.21.15.1:0/1987989967 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7fe2c8039560 0x7fe2c803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.310+0000 7fe2e5400700 1 --2- 172.21.15.1:0/1987989967 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7fe2c8039560 0x7fe2c803ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0006fd0 tx=0x7fe2d0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.316+0000 7fe2daa00700 1 -- 172.21.15.1:0/1987989967 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe2d40172b0 con 0x7fe2e019e7e0 2024-04-26T01:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.632+0000 7fe2e7200700 1 -- 172.21.15.1:0/1987989967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe2e01a2780 con 0x7fe2e019e7e0 2024-04-26T01:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.633+0000 7fe2daa00700 1 -- 172.21.15.1:0/1987989967 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fe2d4018e00 con 0x7fe2e019e7e0 2024-04-26T01:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.635+0000 7fe2e7200700 1 -- 172.21.15.1:0/1987989967 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7fe2c8039560 msgr2=0x7fe2c803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.635+0000 7fe2e7200700 1 --2- 172.21.15.1:0/1987989967 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7fe2c8039560 0x7fe2c803ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0006fd0 tx=0x7fe2d0006e40 comp rx=0 tx=0).stop 2024-04-26T01:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.635+0000 7fe2e7200700 1 -- 172.21.15.1:0/1987989967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2e019e7e0 msgr2=0x7fe2e019ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.635+0000 7fe2e7200700 1 --2- 172.21.15.1:0/1987989967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2e019e7e0 0x7fe2e019ec00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4000c00 tx=0x7fe2d4004260 comp rx=0 tx=0).stop 2024-04-26T01:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.635+0000 7fe2e7200700 1 -- 172.21.15.1:0/1987989967 shutdown_connections 2024-04-26T01:38:34.274 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.635+0000 7fe2e7200700 1 --2- 172.21.15.1:0/1987989967 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7fe2c8039560 0x7fe2c803ba20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:34.275 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.635+0000 7fe2e7200700 1 --2- 172.21.15.1:0/1987989967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2e019e7e0 0x7fe2e019ec00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:34.275 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.635+0000 7fe2e7200700 1 -- 172.21.15.1:0/1987989967 >> 172.21.15.1:0/1987989967 conn(0x7fe2e0103a00 msgr2=0x7fe2e01045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:34.275 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.636+0000 7fe2e7200700 1 -- 172.21.15.1:0/1987989967 shutdown_connections 2024-04-26T01:38:34.275 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:33.636+0000 7fe2e7200700 1 -- 172.21.15.1:0/1987989967 wait complete. 2024-04-26T01:38:34.275 INFO:teuthology.orchestra.run.smithi001.stdout:mgr is available 2024-04-26T01:38:35.515 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:35 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2238298338' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T01:38:35.795 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-04-26T01:38:35.795 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T01:38:35.795 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout fsid = 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:38:35.795 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T01:38:35.795 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T01:38:35.795 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T01:38:35.795 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T01:38:35.795 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T01:38:35.795 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-04-26T01:38:35.795 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T01:38:35.796 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T01:38:35.796 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-04-26T01:38:35.796 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T01:38:35.801 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-26T01:38:35.801 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T01:38:35.801 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T01:38:35.802 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T01:38:35.802 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T01:38:35.802 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T01:38:35.802 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.907+0000 7f2d21200700 1 Processor -- start 2024-04-26T01:38:35.802 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.908+0000 7f2d21200700 1 -- start start 2024-04-26T01:38:35.803 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.908+0000 7f2d21200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d1c108600 0x7f2d1c108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:35.803 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.908+0000 7f2d21200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c108ff0 con 0x7f2d1c108600 2024-04-26T01:38:35.803 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.908+0000 7f2d19e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d1c108600 0x7f2d1c108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:35.803 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.909+0000 7f2d19e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d1c108600 0x7f2d1c108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45226/0 (socket says 172.21.15.1:45226) 2024-04-26T01:38:35.803 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.909+0000 7f2d19e00700 1 -- 172.21.15.1:0/1409608676 learned_addr learned my addr 172.21.15.1:0/1409608676 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:35.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.909+0000 7f2d19e00700 1 -- 172.21.15.1:0/1409608676 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c109800 con 0x7f2d1c108600 2024-04-26T01:38:35.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.909+0000 7f2d19e00700 1 --2- 172.21.15.1:0/1409608676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d1c108600 0x7f2d1c108a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0098a0 tx=0x7f2d0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad58cf2e8dd7132 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:35.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.910+0000 7f2d19400700 1 -- 172.21.15.1:0/1409608676 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d0c004160 con 0x7f2d1c108600 2024-04-26T01:38:35.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.910+0000 7f2d19400700 1 -- 172.21.15.1:0/1409608676 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2d0c004300 con 0x7f2d1c108600 2024-04-26T01:38:35.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.910+0000 7f2d19400700 1 -- 172.21.15.1:0/1409608676 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d0c011af0 con 0x7f2d1c108600 2024-04-26T01:38:35.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.910+0000 7f2d21200700 1 -- 172.21.15.1:0/1409608676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d1c108600 msgr2=0x7f2d1c108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:35.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.910+0000 7f2d21200700 1 --2- 172.21.15.1:0/1409608676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d1c108600 0x7f2d1c108a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0098a0 tx=0x7f2d0c009bb0 comp rx=0 tx=0).stop 2024-04-26T01:38:35.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.911+0000 7f2d21200700 1 -- 172.21.15.1:0/1409608676 shutdown_connections 2024-04-26T01:38:35.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.911+0000 7f2d21200700 1 --2- 172.21.15.1:0/1409608676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d1c108600 0x7f2d1c108a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:35.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.911+0000 7f2d21200700 1 -- 172.21.15.1:0/1409608676 >> 172.21.15.1:0/1409608676 conn(0x7f2d1c103be0 msgr2=0x7f2d1c106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:35.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.911+0000 7f2d21200700 1 -- 172.21.15.1:0/1409608676 shutdown_connections 2024-04-26T01:38:35.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.911+0000 7f2d21200700 1 -- 172.21.15.1:0/1409608676 wait complete. 2024-04-26T01:38:35.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.912+0000 7f2d21200700 1 Processor -- start 2024-04-26T01:38:35.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.913+0000 7f2d21200700 1 -- start start 2024-04-26T01:38:35.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.913+0000 7f2d21200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d1c108600 0x7f2d1c19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:35.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.913+0000 7f2d21200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c10ca30 con 0x7f2d1c108600 2024-04-26T01:38:35.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.913+0000 7f2d19e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d1c108600 0x7f2d1c19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:35.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.913+0000 7f2d19e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d1c108600 0x7f2d1c19c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45242/0 (socket says 172.21.15.1:45242) 2024-04-26T01:38:35.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.913+0000 7f2d19e00700 1 -- 172.21.15.1:0/2238298338 learned_addr learned my addr 172.21.15.1:0/2238298338 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:35.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.914+0000 7f2d19e00700 1 -- 172.21.15.1:0/2238298338 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c19cac0 con 0x7f2d1c108600 2024-04-26T01:38:35.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.914+0000 7f2d19e00700 1 --2- 172.21.15.1:0/2238298338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d1c108600 0x7f2d1c19c580 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0117b0 tx=0x7f2d0c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:35.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.914+0000 7f2d13400700 1 -- 172.21.15.1:0/2238298338 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d0c003af0 con 0x7f2d1c108600 2024-04-26T01:38:35.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.914+0000 7f2d13400700 1 -- 172.21.15.1:0/2238298338 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2d0c020070 con 0x7f2d1c108600 2024-04-26T01:38:35.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.914+0000 7f2d21200700 1 -- 172.21.15.1:0/2238298338 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1c19cd50 con 0x7f2d1c108600 2024-04-26T01:38:35.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.914+0000 7f2d13400700 1 -- 172.21.15.1:0/2238298338 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d0c019630 con 0x7f2d1c108600 2024-04-26T01:38:35.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.915+0000 7f2d21200700 1 -- 172.21.15.1:0/2238298338 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d1c19fa40 con 0x7f2d1c108600 2024-04-26T01:38:35.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.916+0000 7f2d13400700 1 -- 172.21.15.1:0/2238298338 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f2d0c01a9c0 con 0x7f2d1c108600 2024-04-26T01:38:35.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.916+0000 7f2d21200700 1 -- 172.21.15.1:0/2238298338 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d1c10cbb0 con 0x7f2d1c108600 2024-04-26T01:38:35.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.916+0000 7f2d13400700 1 --2- 172.21.15.1:0/2238298338 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7f2d00039600 0x7f2d0003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:35.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.916+0000 7f2d13400700 1 -- 172.21.15.1:0/2238298338 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2d0c04da90 con 0x7f2d1c108600 2024-04-26T01:38:35.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.917+0000 7f2d11400700 1 --2- 172.21.15.1:0/2238298338 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7f2d00039600 0x7f2d0003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:35.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.917+0000 7f2d11400700 1 --2- 172.21.15.1:0/2238298338 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7f2d00039600 0x7f2d0003bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400ad30 tx=0x7f2d040093f0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:35.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:34.922+0000 7f2d13400700 1 -- 172.21.15.1:0/2238298338 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d0c00e260 con 0x7f2d1c108600 2024-04-26T01:38:35.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:35.151+0000 7f2d21200700 1 -- 172.21.15.1:0/2238298338 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f2d1c19fc10 con 0x7f2d1c108600 2024-04-26T01:38:35.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:35.153+0000 7f2d13400700 1 -- 172.21.15.1:0/2238298338 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7f2d0c024e60 con 0x7f2d1c108600 2024-04-26T01:38:35.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:35.156+0000 7f2d21200700 1 -- 172.21.15.1:0/2238298338 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7f2d00039600 msgr2=0x7f2d0003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:35.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:35.156+0000 7f2d21200700 1 --2- 172.21.15.1:0/2238298338 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7f2d00039600 0x7f2d0003bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400ad30 tx=0x7f2d040093f0 comp rx=0 tx=0).stop 2024-04-26T01:38:35.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:35.156+0000 7f2d21200700 1 -- 172.21.15.1:0/2238298338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d1c108600 msgr2=0x7f2d1c19c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:35.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:35.156+0000 7f2d21200700 1 --2- 172.21.15.1:0/2238298338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d1c108600 0x7f2d1c19c580 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0117b0 tx=0x7f2d0c011b70 comp rx=0 tx=0).stop 2024-04-26T01:38:35.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:35.156+0000 7f2d21200700 1 -- 172.21.15.1:0/2238298338 shutdown_connections 2024-04-26T01:38:35.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:35.156+0000 7f2d21200700 1 --2- 172.21.15.1:0/2238298338 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7f2d00039600 0x7f2d0003bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:35.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:35.156+0000 7f2d21200700 1 --2- 172.21.15.1:0/2238298338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d1c108600 0x7f2d1c19c580 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:35.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:35.156+0000 7f2d21200700 1 -- 172.21.15.1:0/2238298338 >> 172.21.15.1:0/2238298338 conn(0x7f2d1c103be0 msgr2=0x7f2d1c104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:35.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:35.156+0000 7f2d21200700 1 -- 172.21.15.1:0/2238298338 shutdown_connections 2024-04-26T01:38:35.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:35.156+0000 7f2d21200700 1 -- 172.21.15.1:0/2238298338 wait complete. 2024-04-26T01:38:35.807 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling cephadm module... 2024-04-26T01:38:37.092 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:36 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/661607181' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-26T01:38:37.093 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:36 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: ignoring --setuser ceph since I am not root 2024-04-26T01:38:37.093 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:36 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: ignoring --setgroup ceph since I am not root 2024-04-26T01:38:37.093 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:37 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:37.095+0000 7fea07a96200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:38:37.378 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:37 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:37.341+0000 7fea07a96200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:38:37.693 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.447+0000 7f9f7c600700 1 Processor -- start 2024-04-26T01:38:37.693 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.447+0000 7f9f7c600700 1 -- start start 2024-04-26T01:38:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.447+0000 7f9f7c600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f74109080 0x7f9f7410b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.447+0000 7f9f7c600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7407ceb0 con 0x7f9f74109080 2024-04-26T01:38:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.448+0000 7f9f79c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f74109080 0x7f9f7410b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.448+0000 7f9f79c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f74109080 0x7f9f7410b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45250/0 (socket says 172.21.15.1:45250) 2024-04-26T01:38:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.448+0000 7f9f79c00700 1 -- 172.21.15.1:0/1889683491 learned_addr learned my addr 172.21.15.1:0/1889683491 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.448+0000 7f9f79c00700 1 -- 172.21.15.1:0/1889683491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7410b9b0 con 0x7f9f74109080 2024-04-26T01:38:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.449+0000 7f9f79c00700 1 --2- 172.21.15.1:0/1889683491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f74109080 0x7f9f7410b470 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9f680098a0 tx=0x7f9f68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7609060cbcdd645 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.449+0000 7f9f73e00700 1 -- 172.21.15.1:0/1889683491 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f68004160 con 0x7f9f74109080 2024-04-26T01:38:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.449+0000 7f9f73e00700 1 -- 172.21.15.1:0/1889683491 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9f68004300 con 0x7f9f74109080 2024-04-26T01:38:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.449+0000 7f9f73e00700 1 -- 172.21.15.1:0/1889683491 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f68011af0 con 0x7f9f74109080 2024-04-26T01:38:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.450+0000 7f9f7c600700 1 -- 172.21.15.1:0/1889683491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f74109080 msgr2=0x7f9f7410b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.450+0000 7f9f7c600700 1 --2- 172.21.15.1:0/1889683491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f74109080 0x7f9f7410b470 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9f680098a0 tx=0x7f9f68009bb0 comp rx=0 tx=0).stop 2024-04-26T01:38:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.450+0000 7f9f7c600700 1 -- 172.21.15.1:0/1889683491 shutdown_connections 2024-04-26T01:38:37.695 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.450+0000 7f9f7c600700 1 --2- 172.21.15.1:0/1889683491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f74109080 0x7f9f7410b470 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:37.695 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.450+0000 7f9f7c600700 1 -- 172.21.15.1:0/1889683491 >> 172.21.15.1:0/1889683491 conn(0x7f9f74102b10 msgr2=0x7f9f74104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:37.695 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.450+0000 7f9f7c600700 1 -- 172.21.15.1:0/1889683491 shutdown_connections 2024-04-26T01:38:37.697 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.450+0000 7f9f7c600700 1 -- 172.21.15.1:0/1889683491 wait complete. 2024-04-26T01:38:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.452+0000 7f9f7c600700 1 Processor -- start 2024-04-26T01:38:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.452+0000 7f9f7c600700 1 -- start start 2024-04-26T01:38:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.452+0000 7f9f7c600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f74109080 0x7f9f74108900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.452+0000 7f9f7c600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7410c910 con 0x7f9f74109080 2024-04-26T01:38:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.452+0000 7f9f79c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f74109080 0x7f9f74108900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.453+0000 7f9f79c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f74109080 0x7f9f74108900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45264/0 (socket says 172.21.15.1:45264) 2024-04-26T01:38:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.453+0000 7f9f79c00700 1 -- 172.21.15.1:0/661607181 learned_addr learned my addr 172.21.15.1:0/661607181 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.453+0000 7f9f79c00700 1 -- 172.21.15.1:0/661607181 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f74108e40 con 0x7f9f74109080 2024-04-26T01:38:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.453+0000 7f9f79c00700 1 --2- 172.21.15.1:0/661607181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f74109080 0x7f9f74108900 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9f68011ac0 tx=0x7f9f68011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.453+0000 7f9f72000700 1 -- 172.21.15.1:0/661607181 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f68003970 con 0x7f9f74109080 2024-04-26T01:38:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.453+0000 7f9f72000700 1 -- 172.21.15.1:0/661607181 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9f68003b10 con 0x7f9f74109080 2024-04-26T01:38:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.454+0000 7f9f72000700 1 -- 172.21.15.1:0/661607181 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f68024610 con 0x7f9f74109080 2024-04-26T01:38:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.454+0000 7f9f7c600700 1 -- 172.21.15.1:0/661607181 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f74107030 con 0x7f9f74109080 2024-04-26T01:38:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.454+0000 7f9f7c600700 1 -- 172.21.15.1:0/661607181 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f74107570 con 0x7f9f74109080 2024-04-26T01:38:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.455+0000 7f9f7c600700 1 -- 172.21.15.1:0/661607181 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f740640a0 con 0x7f9f74109080 2024-04-26T01:38:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.455+0000 7f9f72000700 1 -- 172.21.15.1:0/661607181 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f9f6801a9e0 con 0x7f9f74109080 2024-04-26T01:38:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.455+0000 7f9f72000700 1 --2- 172.21.15.1:0/661607181 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7f9f5c0395b0 0x7f9f5c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.456+0000 7f9f72000700 1 -- 172.21.15.1:0/661607181 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9f6804d860 con 0x7f9f74109080 2024-04-26T01:38:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.456+0000 7f9f79200700 1 --2- 172.21.15.1:0/661607181 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7f9f5c0395b0 0x7f9f5c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.457+0000 7f9f79200700 1 --2- 172.21.15.1:0/661607181 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7f9f5c0395b0 0x7f9f5c03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9f60006fd0 tx=0x7f9f60006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.461+0000 7f9f72000700 1 -- 172.21.15.1:0/661607181 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9f6800b3e0 con 0x7f9f74109080 2024-04-26T01:38:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.756+0000 7f9f7c600700 1 -- 172.21.15.1:0/661607181 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f9f74107d60 con 0x7f9f74109080 2024-04-26T01:38:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.810+0000 7f9f72000700 1 -- 172.21.15.1:0/661607181 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f9f6804d280 con 0x7f9f74109080 2024-04-26T01:38:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.813+0000 7f9f7c600700 1 -- 172.21.15.1:0/661607181 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7f9f5c0395b0 msgr2=0x7f9f5c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.813+0000 7f9f7c600700 1 --2- 172.21.15.1:0/661607181 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7f9f5c0395b0 0x7f9f5c03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9f60006fd0 tx=0x7f9f60006e40 comp rx=0 tx=0).stop 2024-04-26T01:38:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.813+0000 7f9f7c600700 1 -- 172.21.15.1:0/661607181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f74109080 msgr2=0x7f9f74108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.813+0000 7f9f7c600700 1 --2- 172.21.15.1:0/661607181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f74109080 0x7f9f74108900 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9f68011ac0 tx=0x7f9f68011ba0 comp rx=0 tx=0).stop 2024-04-26T01:38:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.814+0000 7f9f7c600700 1 -- 172.21.15.1:0/661607181 shutdown_connections 2024-04-26T01:38:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.814+0000 7f9f7c600700 1 --2- 172.21.15.1:0/661607181 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7f9f5c0395b0 0x7f9f5c03ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.814+0000 7f9f7c600700 1 --2- 172.21.15.1:0/661607181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f74109080 0x7f9f74108900 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.814+0000 7f9f7c600700 1 -- 172.21.15.1:0/661607181 >> 172.21.15.1:0/661607181 conn(0x7f9f74102b10 msgr2=0x7f9f741036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.814+0000 7f9f7c600700 1 -- 172.21.15.1:0/661607181 shutdown_connections 2024-04-26T01:38:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:36.814+0000 7f9f7c600700 1 -- 172.21.15.1:0/661607181 wait complete. 2024-04-26T01:38:37.815 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:37 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:37.476+0000 7fea07a96200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:38:38.076 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:37 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/661607181' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-26T01:38:38.077 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:37 smithi001 ceph-mon[27822]: mgrmap e5: x(active, since 5s) 2024-04-26T01:38:38.077 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:37 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:37.891+0000 7fea07a96200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:38:38.077 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:38 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:38.006+0000 7fea07a96200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:38:39.025 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:38 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3872816506' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T01:38:39.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:38:39.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-26T01:38:39.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T01:38:39.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T01:38:39.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T01:38:39.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:38:39.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.341+0000 7ff346400700 1 Processor -- start 2024-04-26T01:38:39.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.341+0000 7ff346400700 1 -- start start 2024-04-26T01:38:39.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.342+0000 7ff346400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff340106f00 0x7ff340107320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:39.297 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.342+0000 7ff346400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff340063d00 con 0x7ff340106f00 2024-04-26T01:38:39.297 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.342+0000 7ff33f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff340106f00 0x7ff340107320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:39.297 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.342+0000 7ff33f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff340106f00 0x7ff340107320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45298/0 (socket says 172.21.15.1:45298) 2024-04-26T01:38:39.297 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.342+0000 7ff33f400700 1 -- 172.21.15.1:0/2635237960 learned_addr learned my addr 172.21.15.1:0/2635237960 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:39.297 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.343+0000 7ff33f400700 1 -- 172.21.15.1:0/2635237960 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff340107860 con 0x7ff340106f00 2024-04-26T01:38:39.297 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.343+0000 7ff33f400700 1 --2- 172.21.15.1:0/2635237960 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff340106f00 0x7ff340107320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff3300098a0 tx=0x7ff330009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9620e0abb87b5b42 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:39.297 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.344+0000 7ff33e000700 1 -- 172.21.15.1:0/2635237960 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff330004160 con 0x7ff340106f00 2024-04-26T01:38:39.297 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.344+0000 7ff33e000700 1 -- 172.21.15.1:0/2635237960 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff330004300 con 0x7ff340106f00 2024-04-26T01:38:39.297 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.344+0000 7ff33e000700 1 -- 172.21.15.1:0/2635237960 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff330011af0 con 0x7ff340106f00 2024-04-26T01:38:39.298 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.344+0000 7ff346400700 1 -- 172.21.15.1:0/2635237960 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff340106f00 msgr2=0x7ff340107320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:39.298 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.344+0000 7ff346400700 1 --2- 172.21.15.1:0/2635237960 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff340106f00 0x7ff340107320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff3300098a0 tx=0x7ff330009bb0 comp rx=0 tx=0).stop 2024-04-26T01:38:39.298 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.345+0000 7ff346400700 1 -- 172.21.15.1:0/2635237960 shutdown_connections 2024-04-26T01:38:39.298 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.345+0000 7ff346400700 1 --2- 172.21.15.1:0/2635237960 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff340106f00 0x7ff340107320 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:39.298 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.345+0000 7ff346400700 1 -- 172.21.15.1:0/2635237960 >> 172.21.15.1:0/2635237960 conn(0x7ff340102b80 msgr2=0x7ff340104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:39.298 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.345+0000 7ff346400700 1 -- 172.21.15.1:0/2635237960 shutdown_connections 2024-04-26T01:38:39.298 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.345+0000 7ff346400700 1 -- 172.21.15.1:0/2635237960 wait complete. 2024-04-26T01:38:39.298 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.346+0000 7ff346400700 1 Processor -- start 2024-04-26T01:38:39.298 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.346+0000 7ff346400700 1 -- start start 2024-04-26T01:38:39.298 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.346+0000 7ff346400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff340106f00 0x7ff34019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:39.298 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.346+0000 7ff346400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff340108820 con 0x7ff340106f00 2024-04-26T01:38:39.298 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.347+0000 7ff33f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff340106f00 0x7ff34019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:39.298 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.347+0000 7ff33f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff340106f00 0x7ff34019e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45310/0 (socket says 172.21.15.1:45310) 2024-04-26T01:38:39.299 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.347+0000 7ff33f400700 1 -- 172.21.15.1:0/3872816506 learned_addr learned my addr 172.21.15.1:0/3872816506 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:39.299 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.347+0000 7ff33f400700 1 -- 172.21.15.1:0/3872816506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34019ec90 con 0x7ff340106f00 2024-04-26T01:38:39.299 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.347+0000 7ff33f400700 1 --2- 172.21.15.1:0/3872816506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff340106f00 0x7ff34019e750 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff330009fb0 tx=0x7ff330011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:39.299 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.348+0000 7ff337e00700 1 -- 172.21.15.1:0/3872816506 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3300036a0 con 0x7ff340106f00 2024-04-26T01:38:39.299 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.348+0000 7ff346400700 1 -- 172.21.15.1:0/3872816506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34019ef20 con 0x7ff340106f00 2024-04-26T01:38:39.299 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.348+0000 7ff337e00700 1 -- 172.21.15.1:0/3872816506 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff33001a430 con 0x7ff340106f00 2024-04-26T01:38:39.299 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.348+0000 7ff337e00700 1 -- 172.21.15.1:0/3872816506 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff330019550 con 0x7ff340106f00 2024-04-26T01:38:39.299 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.348+0000 7ff346400700 1 -- 172.21.15.1:0/3872816506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3401a1c10 con 0x7ff340106f00 2024-04-26T01:38:39.299 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.349+0000 7ff346400700 1 -- 172.21.15.1:0/3872816506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3400640a0 con 0x7ff340106f00 2024-04-26T01:38:39.299 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.349+0000 7ff337e00700 1 -- 172.21.15.1:0/3872816506 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7ff33001a8e0 con 0x7ff340106f00 2024-04-26T01:38:39.299 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.350+0000 7ff337e00700 1 --2- 172.21.15.1:0/3872816506 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7ff324039610 0x7ff32403bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:39.299 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.350+0000 7ff337e00700 1 -- 172.21.15.1:0/3872816506 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff33004d8c0 con 0x7ff340106f00 2024-04-26T01:38:39.299 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.350+0000 7ff33ea00700 1 -- 172.21.15.1:0/3872816506 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7ff324039610 msgr2=0x7ff32403bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/810889357 2024-04-26T01:38:39.299 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.350+0000 7ff33ea00700 1 --2- 172.21.15.1:0/3872816506 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7ff324039610 0x7ff32403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:38:39.300 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.355+0000 7ff337e00700 1 -- 172.21.15.1:0/3872816506 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff33000b3e0 con 0x7ff340106f00 2024-04-26T01:38:39.300 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.551+0000 7ff33ea00700 1 -- 172.21.15.1:0/3872816506 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7ff324039610 msgr2=0x7ff32403bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/810889357 2024-04-26T01:38:39.300 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.551+0000 7ff33ea00700 1 --2- 172.21.15.1:0/3872816506 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7ff324039610 0x7ff32403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:38:39.300 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.682+0000 7ff346400700 1 -- 172.21.15.1:0/3872816506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7ff3400008d0 con 0x7ff340106f00 2024-04-26T01:38:39.300 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.683+0000 7ff337e00700 1 -- 172.21.15.1:0/3872816506 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7ff33001a690 con 0x7ff340106f00 2024-04-26T01:38:39.300 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.685+0000 7ff346400700 1 -- 172.21.15.1:0/3872816506 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7ff324039610 msgr2=0x7ff32403bad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:38:39.300 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.685+0000 7ff346400700 1 --2- 172.21.15.1:0/3872816506 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7ff324039610 0x7ff32403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:39.300 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.685+0000 7ff346400700 1 -- 172.21.15.1:0/3872816506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff340106f00 msgr2=0x7ff34019e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:39.300 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.685+0000 7ff346400700 1 --2- 172.21.15.1:0/3872816506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff340106f00 0x7ff34019e750 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff330009fb0 tx=0x7ff330011940 comp rx=0 tx=0).stop 2024-04-26T01:38:39.300 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.686+0000 7ff346400700 1 -- 172.21.15.1:0/3872816506 shutdown_connections 2024-04-26T01:38:39.300 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.686+0000 7ff346400700 1 --2- 172.21.15.1:0/3872816506 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7ff324039610 0x7ff32403bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:39.300 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.686+0000 7ff346400700 1 --2- 172.21.15.1:0/3872816506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff340106f00 0x7ff34019e750 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:39.300 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.686+0000 7ff346400700 1 -- 172.21.15.1:0/3872816506 >> 172.21.15.1:0/3872816506 conn(0x7ff340102b80 msgr2=0x7ff3401953c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:39.300 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.686+0000 7ff346400700 1 -- 172.21.15.1:0/3872816506 shutdown_connections 2024-04-26T01:38:39.300 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:38.686+0000 7ff346400700 1 -- 172.21.15.1:0/3872816506 wait complete. 2024-04-26T01:38:39.301 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for the mgr to restart... 2024-04-26T01:38:39.301 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr epoch 5... 2024-04-26T01:38:39.835 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:39 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:39.625+0000 7fea07a96200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:38:40.086 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:39 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:39.865+0000 7fea07a96200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:38:40.086 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:40 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:40.086+0000 7fea07a96200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:38:40.861 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:40 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:40.480+0000 7fea07a96200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:38:40.861 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:40 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:40.606+0000 7fea07a96200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:38:41.124 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:40 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:40.864+0000 7fea07a96200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:38:41.377 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:41 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:41.127+0000 7fea07a96200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:38:42.377 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:41 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:41.937+0000 7fea07a96200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:38:42.877 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:42 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:42.511+0000 7fea07a96200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:38:42.878 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:42 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:42.641+0000 7fea07a96200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:38:43.377 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:42 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:42.916+0000 7fea07a96200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:38:43.877 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:43 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:43.582+0000 7fea07a96200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:38:43.878 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:43 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:43.710+0000 7fea07a96200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:38:44.354 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:44 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:44.043+0000 7fea07a96200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:38:44.354 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:44 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:44.226+0000 7fea07a96200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:38:44.354 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:44 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:44.357+0000 7fea07a96200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:38:44.627 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:44 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:44.513+0000 7fea07a96200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:38:45.377 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:44 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:44.890+0000 7fea07a96200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:38:45.967 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:45 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:45.676+0000 7fea07a96200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:38:46.377 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:45 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:38:45.970+0000 7fea07a96200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:38:46.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:46 smithi001 ceph-mon[27822]: Active manager daemon x restarted 2024-04-26T01:38:46.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:46 smithi001 ceph-mon[27822]: Activating manager daemon x 2024-04-26T01:38:46.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:46 smithi001 ceph-mon[27822]: osdmap e2: 0 total, 0 up, 0 in 2024-04-26T01:38:46.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:46 smithi001 ceph-mon[27822]: mgrmap e6: x(active, starting, since 0.0103419s) 2024-04-26T01:38:46.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:46 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:38:46.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:46 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T01:38:46.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:46 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T01:38:46.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:46 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T01:38:46.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:46 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T01:38:46.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:46 smithi001 ceph-mon[27822]: Manager daemon x is now available 2024-04-26T01:38:47.335 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:47 smithi001 ceph-mon[27822]: Found migration_current of "None". Setting to last migration. 2024-04-26T01:38:47.335 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:47 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' 2024-04-26T01:38:47.335 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:47 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' 2024-04-26T01:38:47.335 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:47 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:38:47.335 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:47 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:38:47.335 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:47 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:38:47.335 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:47 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:38:47.335 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:47 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' 2024-04-26T01:38:47.335 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:47 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' 2024-04-26T01:38:47.336 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:47 smithi001 ceph-mon[27822]: mgrmap e7: x(active, since 1.01329s) 2024-04-26T01:38:47.654 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:38:47.655 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-26T01:38:47.655 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T01:38:47.655 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:38:47.655 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.880+0000 7fb8f8800700 1 Processor -- start 2024-04-26T01:38:47.656 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.880+0000 7fb8f8800700 1 -- start start 2024-04-26T01:38:47.656 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.880+0000 7fb8f8800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0109c20 0x7fb8f010a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:47.656 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.880+0000 7fb8f8800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f010a610 con 0x7fb8f0109c20 2024-04-26T01:38:47.659 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.881+0000 7fb8f5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0109c20 0x7fb8f010a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:47.659 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.881+0000 7fb8f5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0109c20 0x7fb8f010a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45318/0 (socket says 172.21.15.1:45318) 2024-04-26T01:38:47.659 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.881+0000 7fb8f5e00700 1 -- 172.21.15.1:0/2374068087 learned_addr learned my addr 172.21.15.1:0/2374068087 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:47.660 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.881+0000 7fb8f5e00700 1 -- 172.21.15.1:0/2374068087 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f010a790 con 0x7fb8f0109c20 2024-04-26T01:38:47.660 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.882+0000 7fb8f5e00700 1 --2- 172.21.15.1:0/2374068087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0109c20 0x7fb8f010a040 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40098a0 tx=0x7fb8e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a6a9881eeef9d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:47.660 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.882+0000 7fb8f4a00700 1 -- 172.21.15.1:0/2374068087 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8e4004160 con 0x7fb8f0109c20 2024-04-26T01:38:47.660 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.882+0000 7fb8f4a00700 1 -- 172.21.15.1:0/2374068087 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb8e4004300 con 0x7fb8f0109c20 2024-04-26T01:38:47.660 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.882+0000 7fb8f4a00700 1 -- 172.21.15.1:0/2374068087 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8e4011af0 con 0x7fb8f0109c20 2024-04-26T01:38:47.660 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.883+0000 7fb8f8800700 1 -- 172.21.15.1:0/2374068087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0109c20 msgr2=0x7fb8f010a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:47.660 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.883+0000 7fb8f8800700 1 --2- 172.21.15.1:0/2374068087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0109c20 0x7fb8f010a040 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40098a0 tx=0x7fb8e4009bb0 comp rx=0 tx=0).stop 2024-04-26T01:38:47.660 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.883+0000 7fb8f8800700 1 -- 172.21.15.1:0/2374068087 shutdown_connections 2024-04-26T01:38:47.660 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.883+0000 7fb8f8800700 1 --2- 172.21.15.1:0/2374068087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0109c20 0x7fb8f010a040 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:47.660 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.883+0000 7fb8f8800700 1 -- 172.21.15.1:0/2374068087 >> 172.21.15.1:0/2374068087 conn(0x7fb8f0102240 msgr2=0x7fb8f0104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:47.660 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.883+0000 7fb8f8800700 1 -- 172.21.15.1:0/2374068087 shutdown_connections 2024-04-26T01:38:47.660 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.883+0000 7fb8f8800700 1 -- 172.21.15.1:0/2374068087 wait complete. 2024-04-26T01:38:47.660 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.884+0000 7fb8f8800700 1 Processor -- start 2024-04-26T01:38:47.660 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.884+0000 7fb8f8800700 1 -- start start 2024-04-26T01:38:47.661 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.885+0000 7fb8f8800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0109c20 0x7fb8f0109240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:47.661 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.885+0000 7fb8f8800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f01073c0 con 0x7fb8f0109c20 2024-04-26T01:38:47.661 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.885+0000 7fb8f5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0109c20 0x7fb8f0109240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:47.661 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.885+0000 7fb8f5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0109c20 0x7fb8f0109240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45326/0 (socket says 172.21.15.1:45326) 2024-04-26T01:38:47.661 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.885+0000 7fb8f5e00700 1 -- 172.21.15.1:0/3027979338 learned_addr learned my addr 172.21.15.1:0/3027979338 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:47.661 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.885+0000 7fb8f5e00700 1 -- 172.21.15.1:0/3027979338 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f0109780 con 0x7fb8f0109c20 2024-04-26T01:38:47.661 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.886+0000 7fb8f5e00700 1 --2- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0109c20 0x7fb8f0109240 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009fb0 tx=0x7fb8e4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:47.661 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.886+0000 7fb8eaa00700 1 -- 172.21.15.1:0/3027979338 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8e40036a0 con 0x7fb8f0109c20 2024-04-26T01:38:47.661 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.886+0000 7fb8eaa00700 1 -- 172.21.15.1:0/3027979338 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb8e401a430 con 0x7fb8f0109c20 2024-04-26T01:38:47.661 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.886+0000 7fb8eaa00700 1 -- 172.21.15.1:0/3027979338 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8e4019550 con 0x7fb8f0109c20 2024-04-26T01:38:47.661 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.886+0000 7fb8f8800700 1 -- 172.21.15.1:0/3027979338 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f0107970 con 0x7fb8f0109c20 2024-04-26T01:38:47.665 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.886+0000 7fb8f8800700 1 -- 172.21.15.1:0/3027979338 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f0107eb0 con 0x7fb8f0109c20 2024-04-26T01:38:47.665 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.887+0000 7fb8eaa00700 1 -- 172.21.15.1:0/3027979338 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fb8e401a8e0 con 0x7fb8f0109c20 2024-04-26T01:38:47.665 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.888+0000 7fb8eaa00700 1 --2- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7fb8d80395b0 0x7fb8d803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:47.665 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.888+0000 7fb8eaa00700 1 -- 172.21.15.1:0/3027979338 --> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb8d803c180 con 0x7fb8d80395b0 2024-04-26T01:38:47.665 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.888+0000 7fb8eaa00700 1 -- 172.21.15.1:0/3027979338 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb8e404d840 con 0x7fb8f0109c20 2024-04-26T01:38:47.665 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.888+0000 7fb8f5400700 1 -- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7fb8d80395b0 msgr2=0x7fb8d803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/810889357 2024-04-26T01:38:47.666 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:39.888+0000 7fb8f5400700 1 --2- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7fb8d80395b0 0x7fb8d803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:38:47.666 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:40.089+0000 7fb8f5400700 1 -- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7fb8d80395b0 msgr2=0x7fb8d803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/810889357 2024-04-26T01:38:47.666 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:40.089+0000 7fb8f5400700 1 --2- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7fb8d80395b0 0x7fb8d803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:38:47.666 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:40.490+0000 7fb8f5400700 1 -- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7fb8d80395b0 msgr2=0x7fb8d803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/810889357 2024-04-26T01:38:47.666 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:40.490+0000 7fb8f5400700 1 --2- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7fb8d80395b0 0x7fb8d803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T01:38:47.666 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:41.291+0000 7fb8f5400700 1 -- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7fb8d80395b0 msgr2=0x7fb8d803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/810889357 2024-04-26T01:38:47.666 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:41.291+0000 7fb8f5400700 1 --2- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7fb8d80395b0 0x7fb8d803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T01:38:47.666 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:42.893+0000 7fb8f5400700 1 -- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7fb8d80395b0 msgr2=0x7fb8d803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/810889357 2024-04-26T01:38:47.666 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:42.893+0000 7fb8f5400700 1 --2- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7fb8d80395b0 0x7fb8d803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T01:38:47.666 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:45.980+0000 7fb8eaa00700 1 -- 172.21.15.1:0/3027979338 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7fb8e4003840 con 0x7fb8f0109c20 2024-04-26T01:38:47.666 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:45.980+0000 7fb8eaa00700 1 -- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7fb8d80395b0 msgr2=0x7fb8d803ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:38:47.666 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:45.980+0000 7fb8eaa00700 1 --2- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7fb8d80395b0 0x7fb8d803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:47.666 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:46.985+0000 7fb8eaa00700 1 -- 172.21.15.1:0/3027979338 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fb8e404f4d0 con 0x7fb8f0109c20 2024-04-26T01:38:47.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:46.985+0000 7fb8eaa00700 1 --2- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7fb8d803cea0 0x7fb8d803f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:47.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:46.985+0000 7fb8eaa00700 1 -- 172.21.15.1:0/3027979338 --> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb8e400bd20 con 0x7fb8d803cea0 2024-04-26T01:38:47.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:46.987+0000 7fb8f5400700 1 --2- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7fb8d803cea0 0x7fb8d803f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:47.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:46.988+0000 7fb8f5400700 1 --2- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7fb8d803cea0 0x7fb8d803f290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0003a10 tx=0x7fb8e00092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:47.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:46.990+0000 7fb8eaa00700 1 -- 172.21.15.1:0/3027979338 <== mgr.14120 v2:172.21.15.1:6800/1870580982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fb8e400bd20 con 0x7fb8d803cea0 2024-04-26T01:38:47.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:46.998+0000 7fb8f8800700 1 -- 172.21.15.1:0/3027979338 --> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fb8f00640a0 con 0x7fb8d803cea0 2024-04-26T01:38:47.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:46.999+0000 7fb8eaa00700 1 -- 172.21.15.1:0/3027979338 <== mgr.14120 v2:172.21.15.1:6800/1870580982 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fb8f00640a0 con 0x7fb8d803cea0 2024-04-26T01:38:47.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:46.999+0000 7fb8f8800700 1 -- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7fb8d803cea0 msgr2=0x7fb8d803f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:47.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:46.999+0000 7fb8f8800700 1 --2- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7fb8d803cea0 0x7fb8d803f290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0003a10 tx=0x7fb8e00092b0 comp rx=0 tx=0).stop 2024-04-26T01:38:47.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:46.999+0000 7fb8f8800700 1 -- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0109c20 msgr2=0x7fb8f0109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:47.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:46.999+0000 7fb8f8800700 1 --2- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0109c20 0x7fb8f0109240 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009fb0 tx=0x7fb8e4011940 comp rx=0 tx=0).stop 2024-04-26T01:38:47.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:46.999+0000 7fb8f8800700 1 -- 172.21.15.1:0/3027979338 shutdown_connections 2024-04-26T01:38:47.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:46.999+0000 7fb8f8800700 1 --2- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0109c20 0x7fb8f0109240 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:47.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:46.999+0000 7fb8f8800700 1 --2- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7fb8d803cea0 0x7fb8d803f290 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:47.668 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:46.999+0000 7fb8f8800700 1 --2- 172.21.15.1:0/3027979338 >> [v2:172.21.15.1:6800/810889357,v1:172.21.15.1:6801/810889357] conn(0x7fb8d80395b0 0x7fb8d803ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:47.668 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:46.999+0000 7fb8f8800700 1 -- 172.21.15.1:0/3027979338 >> 172.21.15.1:0/3027979338 conn(0x7fb8f0102240 msgr2=0x7fb8f0102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:47.668 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:47.000+0000 7fb8f8800700 1 -- 172.21.15.1:0/3027979338 shutdown_connections 2024-04-26T01:38:47.668 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:47.000+0000 7fb8f8800700 1 -- 172.21.15.1:0/3027979338 wait complete. 2024-04-26T01:38:47.668 INFO:teuthology.orchestra.run.smithi001.stdout:mgr epoch 5 is available 2024-04-26T01:38:47.668 INFO:teuthology.orchestra.run.smithi001.stdout:Setting orchestrator backend to cephadm... 2024-04-26T01:38:48.297 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:48 smithi001 ceph-mon[27822]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T01:38:48.297 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:48 smithi001 ceph-mon[27822]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T01:38:49.270 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:49 smithi001 ceph-mon[27822]: mgrmap e8: x(active, since 2s) 2024-04-26T01:38:49.271 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:49 smithi001 ceph-mon[27822]: [26/Apr/2024:01:38:48] ENGINE Bus STARTING 2024-04-26T01:38:49.271 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:49 smithi001 ceph-mon[27822]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:38:49.271 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:49 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' 2024-04-26T01:38:49.271 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:49 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:38:49.271 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:49 smithi001 ceph-mon[27822]: [26/Apr/2024:01:38:48] ENGINE Serving on https://172.21.15.1:7150 2024-04-26T01:38:49.271 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:49 smithi001 ceph-mon[27822]: [26/Apr/2024:01:38:48] ENGINE Serving on http://172.21.15.1:8765 2024-04-26T01:38:49.271 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:49 smithi001 ceph-mon[27822]: [26/Apr/2024:01:38:48] ENGINE Bus STARTED 2024-04-26T01:38:49.271 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:49 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:38:49.282 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.363+0000 7ff04b600700 1 Processor -- start 2024-04-26T01:38:49.282 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.364+0000 7ff04b600700 1 -- start start 2024-04-26T01:38:49.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.364+0000 7ff04b600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0440825d0 0x7ff0440829f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:49.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.364+0000 7ff04b600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff044082fc0 con 0x7ff0440825d0 2024-04-26T01:38:49.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.364+0000 7ff048c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0440825d0 0x7ff0440829f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:49.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.365+0000 7ff048c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0440825d0 0x7ff0440829f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56714/0 (socket says 172.21.15.1:56714) 2024-04-26T01:38:49.285 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.365+0000 7ff048c00700 1 -- 172.21.15.1:0/4276820835 learned_addr learned my addr 172.21.15.1:0/4276820835 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:49.285 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.365+0000 7ff048c00700 1 -- 172.21.15.1:0/4276820835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04410fe10 con 0x7ff0440825d0 2024-04-26T01:38:49.285 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.365+0000 7ff048c00700 1 --2- 172.21.15.1:0/4276820835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0440825d0 0x7ff0440829f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff0380098a0 tx=0x7ff038009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9aa839e6cd1015 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:49.285 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.366+0000 7ff043400700 1 -- 172.21.15.1:0/4276820835 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff038004160 con 0x7ff0440825d0 2024-04-26T01:38:49.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.366+0000 7ff043400700 1 -- 172.21.15.1:0/4276820835 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff038004300 con 0x7ff0440825d0 2024-04-26T01:38:49.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.366+0000 7ff043400700 1 -- 172.21.15.1:0/4276820835 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff038011af0 con 0x7ff0440825d0 2024-04-26T01:38:49.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.366+0000 7ff04b600700 1 -- 172.21.15.1:0/4276820835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0440825d0 msgr2=0x7ff0440829f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:49.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.366+0000 7ff04b600700 1 --2- 172.21.15.1:0/4276820835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0440825d0 0x7ff0440829f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff0380098a0 tx=0x7ff038009bb0 comp rx=0 tx=0).stop 2024-04-26T01:38:49.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.367+0000 7ff04b600700 1 -- 172.21.15.1:0/4276820835 shutdown_connections 2024-04-26T01:38:49.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.367+0000 7ff04b600700 1 --2- 172.21.15.1:0/4276820835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0440825d0 0x7ff0440829f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:49.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.367+0000 7ff04b600700 1 -- 172.21.15.1:0/4276820835 >> 172.21.15.1:0/4276820835 conn(0x7ff04407dbc0 msgr2=0x7ff044080000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:49.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.367+0000 7ff04b600700 1 -- 172.21.15.1:0/4276820835 shutdown_connections 2024-04-26T01:38:49.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.367+0000 7ff04b600700 1 -- 172.21.15.1:0/4276820835 wait complete. 2024-04-26T01:38:49.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.368+0000 7ff04b600700 1 Processor -- start 2024-04-26T01:38:49.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.368+0000 7ff04b600700 1 -- start start 2024-04-26T01:38:49.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.369+0000 7ff04b600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0441a2d70 0x7ff0441a3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:49.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.369+0000 7ff04b600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff044113040 con 0x7ff0441a2d70 2024-04-26T01:38:49.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.369+0000 7ff048c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0441a2d70 0x7ff0441a3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:49.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.369+0000 7ff048c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0441a2d70 0x7ff0441a3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56728/0 (socket says 172.21.15.1:56728) 2024-04-26T01:38:49.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.369+0000 7ff048c00700 1 -- 172.21.15.1:0/1431354140 learned_addr learned my addr 172.21.15.1:0/1431354140 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:49.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.369+0000 7ff048c00700 1 -- 172.21.15.1:0/1431354140 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0441a36d0 con 0x7ff0441a2d70 2024-04-26T01:38:49.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.370+0000 7ff048c00700 1 --2- 172.21.15.1:0/1431354140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0441a2d70 0x7ff0441a3190 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff0380117b0 tx=0x7ff038011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.370+0000 7ff041600700 1 -- 172.21.15.1:0/1431354140 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0380036a0 con 0x7ff0441a2d70 2024-04-26T01:38:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.370+0000 7ff041600700 1 -- 172.21.15.1:0/1431354140 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff03801a460 con 0x7ff0441a2d70 2024-04-26T01:38:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.370+0000 7ff04b600700 1 -- 172.21.15.1:0/1431354140 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0441a3960 con 0x7ff0441a2d70 2024-04-26T01:38:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.370+0000 7ff041600700 1 -- 172.21.15.1:0/1431354140 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff03801b440 con 0x7ff0441a2d70 2024-04-26T01:38:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.370+0000 7ff04b600700 1 -- 172.21.15.1:0/1431354140 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0441a6440 con 0x7ff0441a2d70 2024-04-26T01:38:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.371+0000 7ff041600700 1 -- 172.21.15.1:0/1431354140 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff03801a980 con 0x7ff0441a2d70 2024-04-26T01:38:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.371+0000 7ff04b600700 1 -- 172.21.15.1:0/1431354140 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0441131c0 con 0x7ff0441a2d70 2024-04-26T01:38:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.372+0000 7ff041600700 1 --2- 172.21.15.1:0/1431354140 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7ff02c039560 0x7ff02c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.372+0000 7ff041600700 1 -- 172.21.15.1:0/1431354140 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff03804dcf0 con 0x7ff0441a2d70 2024-04-26T01:38:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.372+0000 7ff043e00700 1 --2- 172.21.15.1:0/1431354140 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7ff02c039560 0x7ff02c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.373+0000 7ff043e00700 1 --2- 172.21.15.1:0/1431354140 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7ff02c039560 0x7ff02c03ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff034006fd0 tx=0x7ff034006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.378+0000 7ff041600700 1 -- 172.21.15.1:0/1431354140 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff03801a6c0 con 0x7ff0441a2d70 2024-04-26T01:38:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.621+0000 7ff04b600700 1 -- 172.21.15.1:0/1431354140 --> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7ff04407eef0 con 0x7ff02c039560 2024-04-26T01:38:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.628+0000 7ff041600700 1 -- 172.21.15.1:0/1431354140 <== mgr.14120 v2:172.21.15.1:6800/1870580982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff04407eef0 con 0x7ff02c039560 2024-04-26T01:38:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.630+0000 7ff04b600700 1 -- 172.21.15.1:0/1431354140 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7ff02c039560 msgr2=0x7ff02c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.630+0000 7ff04b600700 1 --2- 172.21.15.1:0/1431354140 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7ff02c039560 0x7ff02c03ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff034006fd0 tx=0x7ff034006e40 comp rx=0 tx=0).stop 2024-04-26T01:38:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.631+0000 7ff04b600700 1 -- 172.21.15.1:0/1431354140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0441a2d70 msgr2=0x7ff0441a3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.631+0000 7ff04b600700 1 --2- 172.21.15.1:0/1431354140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0441a2d70 0x7ff0441a3190 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff0380117b0 tx=0x7ff038011940 comp rx=0 tx=0).stop 2024-04-26T01:38:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.631+0000 7ff04b600700 1 -- 172.21.15.1:0/1431354140 shutdown_connections 2024-04-26T01:38:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.631+0000 7ff04b600700 1 --2- 172.21.15.1:0/1431354140 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7ff02c039560 0x7ff02c03ba20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.631+0000 7ff04b600700 1 --2- 172.21.15.1:0/1431354140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0441a2d70 0x7ff0441a3190 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.631+0000 7ff04b600700 1 -- 172.21.15.1:0/1431354140 >> 172.21.15.1:0/1431354140 conn(0x7ff04407dbc0 msgr2=0x7ff04407e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.631+0000 7ff04b600700 1 -- 172.21.15.1:0/1431354140 shutdown_connections 2024-04-26T01:38:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:48.631+0000 7ff04b600700 1 -- 172.21.15.1:0/1431354140 wait complete. 2024-04-26T01:38:50.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-26T01:38:50.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.949+0000 7f4006200700 1 Processor -- start 2024-04-26T01:38:50.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.949+0000 7f4006200700 1 -- start start 2024-04-26T01:38:50.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.950+0000 7f4006200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f400010a8c0 0x7f400010ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:50.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.950+0000 7f4006200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f400010b2b0 con 0x7f400010a8c0 2024-04-26T01:38:50.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.950+0000 7f3fff400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f400010a8c0 0x7f400010ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:50.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.950+0000 7f3fff400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f400010a8c0 0x7f400010ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56738/0 (socket says 172.21.15.1:56738) 2024-04-26T01:38:50.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.950+0000 7f3fff400700 1 -- 172.21.15.1:0/2794937909 learned_addr learned my addr 172.21.15.1:0/2794937909 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:50.773 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.951+0000 7f3fff400700 1 -- 172.21.15.1:0/2794937909 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f400010ba00 con 0x7f400010a8c0 2024-04-26T01:38:50.774 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.951+0000 7f3fff400700 1 --2- 172.21.15.1:0/2794937909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f400010a8c0 0x7f400010ace0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00098a0 tx=0x7f3ff0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec88dd17e8b6f5b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:50.774 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.952+0000 7f3ffea00700 1 -- 172.21.15.1:0/2794937909 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ff0004160 con 0x7f400010a8c0 2024-04-26T01:38:50.778 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.952+0000 7f3ffea00700 1 -- 172.21.15.1:0/2794937909 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3ff0004300 con 0x7f400010a8c0 2024-04-26T01:38:50.778 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.952+0000 7f3ffea00700 1 -- 172.21.15.1:0/2794937909 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ff0011af0 con 0x7f400010a8c0 2024-04-26T01:38:50.778 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.952+0000 7f4006200700 1 -- 172.21.15.1:0/2794937909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f400010a8c0 msgr2=0x7f400010ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:50.778 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.952+0000 7f4006200700 1 --2- 172.21.15.1:0/2794937909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f400010a8c0 0x7f400010ace0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00098a0 tx=0x7f3ff0009bb0 comp rx=0 tx=0).stop 2024-04-26T01:38:50.778 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.952+0000 7f4006200700 1 -- 172.21.15.1:0/2794937909 shutdown_connections 2024-04-26T01:38:50.778 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.952+0000 7f4006200700 1 --2- 172.21.15.1:0/2794937909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f400010a8c0 0x7f400010ace0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:50.778 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.952+0000 7f4006200700 1 -- 172.21.15.1:0/2794937909 >> 172.21.15.1:0/2794937909 conn(0x7f400007dbd0 msgr2=0x7f400007e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:50.778 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.953+0000 7f4006200700 1 -- 172.21.15.1:0/2794937909 shutdown_connections 2024-04-26T01:38:50.778 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.953+0000 7f4006200700 1 -- 172.21.15.1:0/2794937909 wait complete. 2024-04-26T01:38:50.778 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.954+0000 7f4006200700 1 Processor -- start 2024-04-26T01:38:50.779 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.954+0000 7f4006200700 1 -- start start 2024-04-26T01:38:50.779 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.954+0000 7f4006200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f400010a8c0 0x7f400019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:50.779 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.954+0000 7f4006200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f400010ec30 con 0x7f400010a8c0 2024-04-26T01:38:50.779 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.955+0000 7f3fff400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f400010a8c0 0x7f400019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:50.779 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.955+0000 7f3fff400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f400010a8c0 0x7f400019e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56750/0 (socket says 172.21.15.1:56750) 2024-04-26T01:38:50.779 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.955+0000 7f3fff400700 1 -- 172.21.15.1:0/405450159 learned_addr learned my addr 172.21.15.1:0/405450159 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:50.779 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.955+0000 7f3fff400700 1 -- 172.21.15.1:0/405450159 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f400019ecc0 con 0x7f400010a8c0 2024-04-26T01:38:50.779 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.955+0000 7f3fff400700 1 --2- 172.21.15.1:0/405450159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f400010a8c0 0x7f400019e780 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00117b0 tx=0x7f3ff0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:50.779 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.955+0000 7f3ffcc00700 1 -- 172.21.15.1:0/405450159 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ff0003af0 con 0x7f400010a8c0 2024-04-26T01:38:50.779 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.956+0000 7f3ffcc00700 1 -- 172.21.15.1:0/405450159 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3ff0020070 con 0x7f400010a8c0 2024-04-26T01:38:50.779 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.956+0000 7f3ffcc00700 1 -- 172.21.15.1:0/405450159 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ff0019630 con 0x7f400010a8c0 2024-04-26T01:38:50.779 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.956+0000 7f4006200700 1 -- 172.21.15.1:0/405450159 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f400019ef50 con 0x7f400010a8c0 2024-04-26T01:38:50.779 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.956+0000 7f4006200700 1 -- 172.21.15.1:0/405450159 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40001a1c40 con 0x7f400010a8c0 2024-04-26T01:38:50.779 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.957+0000 7f3ffcc00700 1 -- 172.21.15.1:0/405450159 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3ff001a430 con 0x7f400010a8c0 2024-04-26T01:38:50.779 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.957+0000 7f4006200700 1 -- 172.21.15.1:0/405450159 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f400010edb0 con 0x7f400010a8c0 2024-04-26T01:38:50.780 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.958+0000 7f3ffcc00700 1 --2- 172.21.15.1:0/405450159 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f3fe4039600 0x7f3fe403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:50.780 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.958+0000 7f3ffcc00700 1 -- 172.21.15.1:0/405450159 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3ff004db50 con 0x7f400010a8c0 2024-04-26T01:38:50.780 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.958+0000 7f3ff7e00700 1 --2- 172.21.15.1:0/405450159 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f3fe4039600 0x7f3fe403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:50.780 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.959+0000 7f3ff7e00700 1 --2- 172.21.15.1:0/405450159 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f3fe4039600 0x7f3fe403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8006fd0 tx=0x7f3fe8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:50.780 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:49.964+0000 7f3ffcc00700 1 -- 172.21.15.1:0/405450159 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ff000e260 con 0x7f400010a8c0 2024-04-26T01:38:50.780 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:50.195+0000 7f4006200700 1 -- 172.21.15.1:0/405450159 --> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f4000108cc0 con 0x7f3fe4039600 2024-04-26T01:38:50.780 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:50.197+0000 7f3ffcc00700 1 -- 172.21.15.1:0/405450159 <== mgr.14120 v2:172.21.15.1:6800/1870580982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f4000108cc0 con 0x7f3fe4039600 2024-04-26T01:38:50.780 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:50.200+0000 7f4006200700 1 -- 172.21.15.1:0/405450159 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f3fe4039600 msgr2=0x7f3fe403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:50.780 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:50.200+0000 7f4006200700 1 --2- 172.21.15.1:0/405450159 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f3fe4039600 0x7f3fe403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8006fd0 tx=0x7f3fe8006e40 comp rx=0 tx=0).stop 2024-04-26T01:38:50.780 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:50.200+0000 7f4006200700 1 -- 172.21.15.1:0/405450159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f400010a8c0 msgr2=0x7f400019e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:50.780 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:50.200+0000 7f4006200700 1 --2- 172.21.15.1:0/405450159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f400010a8c0 0x7f400019e780 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00117b0 tx=0x7f3ff0011b70 comp rx=0 tx=0).stop 2024-04-26T01:38:50.780 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:50.201+0000 7f4006200700 1 -- 172.21.15.1:0/405450159 shutdown_connections 2024-04-26T01:38:50.780 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:50.201+0000 7f4006200700 1 --2- 172.21.15.1:0/405450159 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f3fe4039600 0x7f3fe403bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:50.780 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:50.201+0000 7f4006200700 1 --2- 172.21.15.1:0/405450159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f400010a8c0 0x7f400019e780 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:50.781 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:50.201+0000 7f4006200700 1 -- 172.21.15.1:0/405450159 >> 172.21.15.1:0/405450159 conn(0x7f400007dbd0 msgr2=0x7f40001085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:50.781 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:50.201+0000 7f4006200700 1 -- 172.21.15.1:0/405450159 shutdown_connections 2024-04-26T01:38:50.781 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:50.201+0000 7f4006200700 1 -- 172.21.15.1:0/405450159 wait complete. 2024-04-26T01:38:50.782 INFO:teuthology.orchestra.run.smithi001.stdout:Generating ssh key... 2024-04-26T01:38:51.055 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:50 smithi001 ceph-mon[27822]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:38:52.129 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: Generating public/private rsa key pair. 2024-04-26T01:38:52.129 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: Your identification has been saved in /tmp/tmpneyvtyq9/key. 2024-04-26T01:38:52.129 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: Your public key has been saved in /tmp/tmpneyvtyq9/key.pub. 2024-04-26T01:38:52.129 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: The key fingerprint is: 2024-04-26T01:38:52.129 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: SHA256:Im4Tk+4FYXqO3dr0ogJqBn7AEhypJSxZrOa8tFp+sHk ceph-77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:38:52.129 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: The key's randomart image is: 2024-04-26T01:38:52.129 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: +---[RSA 3072]----+ 2024-04-26T01:38:52.130 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: |.+o | 2024-04-26T01:38:52.130 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: |++o | 2024-04-26T01:38:52.130 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: |++. o | 2024-04-26T01:38:52.130 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: |+o o o | 2024-04-26T01:38:52.130 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: |=.. B . S | 2024-04-26T01:38:52.130 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: |+*.B * . | 2024-04-26T01:38:52.130 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: |*.*+B + | 2024-04-26T01:38:52.130 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: |.O++E*.. | 2024-04-26T01:38:52.130 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: |= o++.... | 2024-04-26T01:38:52.130 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:38:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: +----[SHA256]-----+ 2024-04-26T01:38:52.513 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.364+0000 7f6e3f800700 1 Processor -- start 2024-04-26T01:38:52.514 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.364+0000 7f6e3f800700 1 -- start start 2024-04-26T01:38:52.514 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.365+0000 7f6e3f800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e3807ddd0 0x7f6e3807c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:52.514 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.365+0000 7f6e3f800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3807c800 con 0x7f6e3807ddd0 2024-04-26T01:38:52.514 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.365+0000 7f6e3ce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e3807ddd0 0x7f6e3807c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:52.514 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.365+0000 7f6e3ce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e3807ddd0 0x7f6e3807c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56752/0 (socket says 172.21.15.1:56752) 2024-04-26T01:38:52.514 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.365+0000 7f6e3ce00700 1 -- 172.21.15.1:0/2531195216 learned_addr learned my addr 172.21.15.1:0/2531195216 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:52.514 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.365+0000 7f6e3ce00700 1 -- 172.21.15.1:0/2531195216 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3807c980 con 0x7f6e3807ddd0 2024-04-26T01:38:52.514 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.366+0000 7f6e3ce00700 1 --2- 172.21.15.1:0/2531195216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e3807ddd0 0x7f6e3807c230 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0098a0 tx=0x7f6e2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42194e770e78ddcc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:52.514 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.366+0000 7f6e37400700 1 -- 172.21.15.1:0/2531195216 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e2c004160 con 0x7f6e3807ddd0 2024-04-26T01:38:52.514 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.366+0000 7f6e37400700 1 -- 172.21.15.1:0/2531195216 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6e2c004300 con 0x7f6e3807ddd0 2024-04-26T01:38:52.514 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.366+0000 7f6e37400700 1 -- 172.21.15.1:0/2531195216 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e2c011af0 con 0x7f6e3807ddd0 2024-04-26T01:38:52.514 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.367+0000 7f6e3f800700 1 -- 172.21.15.1:0/2531195216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e3807ddd0 msgr2=0x7f6e3807c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:52.515 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.367+0000 7f6e3f800700 1 --2- 172.21.15.1:0/2531195216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e3807ddd0 0x7f6e3807c230 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0098a0 tx=0x7f6e2c009bb0 comp rx=0 tx=0).stop 2024-04-26T01:38:52.518 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.367+0000 7f6e3f800700 1 -- 172.21.15.1:0/2531195216 shutdown_connections 2024-04-26T01:38:52.518 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.367+0000 7f6e3f800700 1 --2- 172.21.15.1:0/2531195216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e3807ddd0 0x7f6e3807c230 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:52.519 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.367+0000 7f6e3f800700 1 -- 172.21.15.1:0/2531195216 >> 172.21.15.1:0/2531195216 conn(0x7f6e38103c70 msgr2=0x7f6e38106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:52.519 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.367+0000 7f6e3f800700 1 -- 172.21.15.1:0/2531195216 shutdown_connections 2024-04-26T01:38:52.519 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.367+0000 7f6e3f800700 1 -- 172.21.15.1:0/2531195216 wait complete. 2024-04-26T01:38:52.519 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.368+0000 7f6e3f800700 1 Processor -- start 2024-04-26T01:38:52.519 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.368+0000 7f6e3f800700 1 -- start start 2024-04-26T01:38:52.519 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.369+0000 7f6e3f800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e3807ddd0 0x7f6e3819e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:52.519 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.369+0000 7f6e3f800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3810eb70 con 0x7f6e3807ddd0 2024-04-26T01:38:52.519 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.369+0000 7f6e3ce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e3807ddd0 0x7f6e3819e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:52.519 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.369+0000 7f6e3ce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e3807ddd0 0x7f6e3819e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56756/0 (socket says 172.21.15.1:56756) 2024-04-26T01:38:52.519 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.369+0000 7f6e3ce00700 1 -- 172.21.15.1:0/2985274772 learned_addr learned my addr 172.21.15.1:0/2985274772 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:52.520 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.369+0000 7f6e3ce00700 1 -- 172.21.15.1:0/2985274772 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3819ecb0 con 0x7f6e3807ddd0 2024-04-26T01:38:52.520 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.370+0000 7f6e3ce00700 1 --2- 172.21.15.1:0/2985274772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e3807ddd0 0x7f6e3819e770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0117b0 tx=0x7f6e2c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:52.520 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.370+0000 7f6e35600700 1 -- 172.21.15.1:0/2985274772 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e2c003af0 con 0x7f6e3807ddd0 2024-04-26T01:38:52.520 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.370+0000 7f6e35600700 1 -- 172.21.15.1:0/2985274772 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6e2c020070 con 0x7f6e3807ddd0 2024-04-26T01:38:52.520 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.370+0000 7f6e3f800700 1 -- 172.21.15.1:0/2985274772 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3819ef40 con 0x7f6e3807ddd0 2024-04-26T01:38:52.520 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.370+0000 7f6e35600700 1 -- 172.21.15.1:0/2985274772 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e2c019550 con 0x7f6e3807ddd0 2024-04-26T01:38:52.520 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.370+0000 7f6e3f800700 1 -- 172.21.15.1:0/2985274772 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e381a1c30 con 0x7f6e3807ddd0 2024-04-26T01:38:52.520 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.371+0000 7f6e3f800700 1 -- 172.21.15.1:0/2985274772 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e380640a0 con 0x7f6e3807ddd0 2024-04-26T01:38:52.520 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.371+0000 7f6e35600700 1 -- 172.21.15.1:0/2985274772 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6e2c019810 con 0x7f6e3807ddd0 2024-04-26T01:38:52.520 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.372+0000 7f6e35600700 1 --2- 172.21.15.1:0/2985274772 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f6e200395b0 0x7f6e2003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:52.520 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.372+0000 7f6e35600700 1 -- 172.21.15.1:0/2985274772 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6e2c04dad0 con 0x7f6e3807ddd0 2024-04-26T01:38:52.520 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.372+0000 7f6e37e00700 1 --2- 172.21.15.1:0/2985274772 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f6e200395b0 0x7f6e2003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:52.520 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.373+0000 7f6e37e00700 1 --2- 172.21.15.1:0/2985274772 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f6e200395b0 0x7f6e2003ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6e28006fd0 tx=0x7f6e28006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:52.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.377+0000 7f6e35600700 1 -- 172.21.15.1:0/2985274772 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e2c00e260 con 0x7f6e3807ddd0 2024-04-26T01:38:52.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.612+0000 7f6e3f800700 1 -- 172.21.15.1:0/2985274772 --> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f6e38104ed0 con 0x7f6e200395b0 2024-04-26T01:38:52.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.796+0000 7f6e35600700 1 -- 172.21.15.1:0/2985274772 <== mgr.14120 v2:172.21.15.1:6800/1870580982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6e38104ed0 con 0x7f6e200395b0 2024-04-26T01:38:52.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.799+0000 7f6e3f800700 1 -- 172.21.15.1:0/2985274772 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f6e200395b0 msgr2=0x7f6e2003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:52.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.799+0000 7f6e3f800700 1 --2- 172.21.15.1:0/2985274772 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f6e200395b0 0x7f6e2003ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6e28006fd0 tx=0x7f6e28006e40 comp rx=0 tx=0).stop 2024-04-26T01:38:52.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.799+0000 7f6e3f800700 1 -- 172.21.15.1:0/2985274772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e3807ddd0 msgr2=0x7f6e3819e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:52.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.799+0000 7f6e3f800700 1 --2- 172.21.15.1:0/2985274772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e3807ddd0 0x7f6e3819e770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0117b0 tx=0x7f6e2c011b70 comp rx=0 tx=0).stop 2024-04-26T01:38:52.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.799+0000 7f6e3f800700 1 -- 172.21.15.1:0/2985274772 shutdown_connections 2024-04-26T01:38:52.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.799+0000 7f6e3f800700 1 --2- 172.21.15.1:0/2985274772 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f6e200395b0 0x7f6e2003ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:52.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.799+0000 7f6e3f800700 1 --2- 172.21.15.1:0/2985274772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e3807ddd0 0x7f6e3819e770 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:52.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.799+0000 7f6e3f800700 1 -- 172.21.15.1:0/2985274772 >> 172.21.15.1:0/2985274772 conn(0x7f6e38103c70 msgr2=0x7f6e381047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:52.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.799+0000 7f6e3f800700 1 -- 172.21.15.1:0/2985274772 shutdown_connections 2024-04-26T01:38:52.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:51.800+0000 7f6e3f800700 1 -- 172.21.15.1:0/2985274772 wait complete. 2024-04-26T01:38:53.038 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:52 smithi001 ceph-mon[27822]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:38:53.039 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:52 smithi001 ceph-mon[27822]: Generating ssh key... 2024-04-26T01:38:53.039 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:52 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' 2024-04-26T01:38:53.039 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:52 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' 2024-04-26T01:38:54.127 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:53 smithi001 ceph-mon[27822]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:38:54.275 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCa5A5IQvnMrpVirYsaCjt9dyCZTEK2sAxmAtBUYdoTzM7TcEiQw1v6QEgjQLogT3f5u81yYpmsMu8O0XcInh5ov31EDYpE+gHS/zmpoSih+ivMuETd7bvNeC9oX2tdrGugXHX4IlXzN2HjsOg4MWLw5fVebLO2CVgIbgi+7Q3vaucSqrKhDplHKMo173Qu9nathtbNj29CN8/G8KVsI5lNw053CsvcDET43mafxe5gAcKiIyRu3B7Bq9mTfyFvKz2aTCIZCWTTF7r7xj0GwTR0VV7+1EPDV7y0pOyDys42s6yNV8MiVuufGy4FjFUHHR2RXUyh4Y/mlBTEvRXBAXfHnkGoJdS6RzjhViKeQwGAxt+vSqOLeXXn9AADRXIc3FLo+LfcqED9r5K8pMke6HPVeH8LuoEbcUw6XVl6hHpt3Ah6Zi90Rj1BgmdsiG55fOcXwIiLe00b+9bdtq12clO+ajfAdqDT7NYuRzJpFjmgGoGBHBvMlyiOvaC49fHBNNc= ceph-77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:38:54.275 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.210+0000 7f7d4d200700 1 Processor -- start 2024-04-26T01:38:54.276 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.210+0000 7f7d4d200700 1 -- start start 2024-04-26T01:38:54.277 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.211+0000 7f7d4d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d481090b0 0x7f7d4810b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:54.277 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.211+0000 7f7d4d200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7d48063d60 con 0x7f7d481090b0 2024-04-26T01:38:54.277 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.211+0000 7f7d45e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d481090b0 0x7f7d4810b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:54.277 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.211+0000 7f7d45e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d481090b0 0x7f7d4810b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48880/0 (socket says 172.21.15.1:48880) 2024-04-26T01:38:54.277 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.211+0000 7f7d45e00700 1 -- 172.21.15.1:0/2156926459 learned_addr learned my addr 172.21.15.1:0/2156926459 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:54.277 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.212+0000 7f7d45e00700 1 -- 172.21.15.1:0/2156926459 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4810b9e0 con 0x7f7d481090b0 2024-04-26T01:38:54.278 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.212+0000 7f7d45e00700 1 --2- 172.21.15.1:0/2156926459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d481090b0 0x7f7d4810b4a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7d380098a0 tx=0x7f7d38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ad4743279032ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:54.278 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.212+0000 7f7d44a00700 1 -- 172.21.15.1:0/2156926459 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d38004160 con 0x7f7d481090b0 2024-04-26T01:38:54.279 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.212+0000 7f7d44a00700 1 -- 172.21.15.1:0/2156926459 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7d38004300 con 0x7f7d481090b0 2024-04-26T01:38:54.279 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.212+0000 7f7d44a00700 1 -- 172.21.15.1:0/2156926459 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d38011af0 con 0x7f7d481090b0 2024-04-26T01:38:54.279 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.213+0000 7f7d4d200700 1 -- 172.21.15.1:0/2156926459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d481090b0 msgr2=0x7f7d4810b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:54.279 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.213+0000 7f7d4d200700 1 --2- 172.21.15.1:0/2156926459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d481090b0 0x7f7d4810b4a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7d380098a0 tx=0x7f7d38009bb0 comp rx=0 tx=0).stop 2024-04-26T01:38:54.279 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.213+0000 7f7d4d200700 1 -- 172.21.15.1:0/2156926459 shutdown_connections 2024-04-26T01:38:54.279 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.213+0000 7f7d4d200700 1 --2- 172.21.15.1:0/2156926459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d481090b0 0x7f7d4810b4a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:54.279 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.213+0000 7f7d4d200700 1 -- 172.21.15.1:0/2156926459 >> 172.21.15.1:0/2156926459 conn(0x7f7d48102b80 msgr2=0x7f7d48104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:54.279 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.213+0000 7f7d4d200700 1 -- 172.21.15.1:0/2156926459 shutdown_connections 2024-04-26T01:38:54.279 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.213+0000 7f7d4d200700 1 -- 172.21.15.1:0/2156926459 wait complete. 2024-04-26T01:38:54.280 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.214+0000 7f7d4d200700 1 Processor -- start 2024-04-26T01:38:54.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.214+0000 7f7d4d200700 1 -- start start 2024-04-26T01:38:54.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.215+0000 7f7d4d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d481090b0 0x7f7d4819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:54.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.215+0000 7f7d4d200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4810c940 con 0x7f7d481090b0 2024-04-26T01:38:54.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.215+0000 7f7d45e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d481090b0 0x7f7d4819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:54.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.215+0000 7f7d45e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d481090b0 0x7f7d4819c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48882/0 (socket says 172.21.15.1:48882) 2024-04-26T01:38:54.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.215+0000 7f7d45e00700 1 -- 172.21.15.1:0/1534269425 learned_addr learned my addr 172.21.15.1:0/1534269425 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:38:54.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.215+0000 7f7d45e00700 1 -- 172.21.15.1:0/1534269425 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4819ca40 con 0x7f7d481090b0 2024-04-26T01:38:54.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.216+0000 7f7d45e00700 1 --2- 172.21.15.1:0/1534269425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d481090b0 0x7f7d4819c500 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7d38011ac0 tx=0x7f7d38011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:54.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.216+0000 7f7d3ea00700 1 -- 172.21.15.1:0/1534269425 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d38003970 con 0x7f7d481090b0 2024-04-26T01:38:54.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.216+0000 7f7d3ea00700 1 -- 172.21.15.1:0/1534269425 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7d38003b10 con 0x7f7d481090b0 2024-04-26T01:38:54.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.216+0000 7f7d3ea00700 1 -- 172.21.15.1:0/1534269425 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d38024610 con 0x7f7d481090b0 2024-04-26T01:38:54.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.216+0000 7f7d4d200700 1 -- 172.21.15.1:0/1534269425 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d4819ccd0 con 0x7f7d481090b0 2024-04-26T01:38:54.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.216+0000 7f7d4d200700 1 -- 172.21.15.1:0/1534269425 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d4819f9c0 con 0x7f7d481090b0 2024-04-26T01:38:54.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.217+0000 7f7d4d200700 1 -- 172.21.15.1:0/1534269425 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d480640a0 con 0x7f7d481090b0 2024-04-26T01:38:54.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.218+0000 7f7d3ea00700 1 -- 172.21.15.1:0/1534269425 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7d3801a9e0 con 0x7f7d481090b0 2024-04-26T01:38:54.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.218+0000 7f7d3ea00700 1 --2- 172.21.15.1:0/1534269425 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f7d2c039600 0x7f7d2c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:38:54.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.218+0000 7f7d3ea00700 1 -- 172.21.15.1:0/1534269425 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7d3804d980 con 0x7f7d481090b0 2024-04-26T01:38:54.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.218+0000 7f7d45400700 1 --2- 172.21.15.1:0/1534269425 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f7d2c039600 0x7f7d2c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:38:54.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.219+0000 7f7d45400700 1 --2- 172.21.15.1:0/1534269425 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f7d2c039600 0x7f7d2c03bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7d34006fd0 tx=0x7f7d34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:38:54.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.224+0000 7f7d3ea00700 1 -- 172.21.15.1:0/1534269425 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d3800b3e0 con 0x7f7d481090b0 2024-04-26T01:38:54.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.455+0000 7f7d4d200700 1 -- 172.21.15.1:0/1534269425 --> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f7d48103da0 con 0x7f7d2c039600 2024-04-26T01:38:54.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.456+0000 7f7d3ea00700 1 -- 172.21.15.1:0/1534269425 <== mgr.14120 v2:172.21.15.1:6800/1870580982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f7d48103da0 con 0x7f7d2c039600 2024-04-26T01:38:54.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.459+0000 7f7d4d200700 1 -- 172.21.15.1:0/1534269425 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f7d2c039600 msgr2=0x7f7d2c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:54.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.459+0000 7f7d4d200700 1 --2- 172.21.15.1:0/1534269425 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f7d2c039600 0x7f7d2c03bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7d34006fd0 tx=0x7f7d34006e40 comp rx=0 tx=0).stop 2024-04-26T01:38:54.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.459+0000 7f7d4d200700 1 -- 172.21.15.1:0/1534269425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d481090b0 msgr2=0x7f7d4819c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:38:54.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.459+0000 7f7d4d200700 1 --2- 172.21.15.1:0/1534269425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d481090b0 0x7f7d4819c500 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7d38011ac0 tx=0x7f7d38011ba0 comp rx=0 tx=0).stop 2024-04-26T01:38:54.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.460+0000 7f7d4d200700 1 -- 172.21.15.1:0/1534269425 shutdown_connections 2024-04-26T01:38:54.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.460+0000 7f7d4d200700 1 --2- 172.21.15.1:0/1534269425 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f7d2c039600 0x7f7d2c03bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:54.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.460+0000 7f7d4d200700 1 --2- 172.21.15.1:0/1534269425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d481090b0 0x7f7d4819c500 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:38:54.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.460+0000 7f7d4d200700 1 -- 172.21.15.1:0/1534269425 >> 172.21.15.1:0/1534269425 conn(0x7f7d48102b80 msgr2=0x7f7d48103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:38:54.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.460+0000 7f7d4d200700 1 -- 172.21.15.1:0/1534269425 shutdown_connections 2024-04-26T01:38:54.288 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:53.460+0000 7f7d4d200700 1 -- 172.21.15.1:0/1534269425 wait complete. 2024-04-26T01:38:54.288 INFO:teuthology.orchestra.run.smithi001.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-26T01:38:54.288 INFO:teuthology.orchestra.run.smithi001.stdout:Adding key to root@localhost authorized_keys... 2024-04-26T01:38:54.296 INFO:teuthology.orchestra.run.smithi001.stdout:Adding host smithi001... 2024-04-26T01:38:56.127 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:55 smithi001 ceph-mon[27822]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi001", "addr": "172.21.15.1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:38:57.127 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:38:56 smithi001 ceph-mon[27822]: Deploying cephadm binary to smithi001 2024-04-26T01:39:02.148 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Added host 'smithi001' with addr '172.21.15.1' 2024-04-26T01:39:02.148 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.885+0000 7faede200700 1 Processor -- start 2024-04-26T01:39:02.148 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.885+0000 7faede200700 1 -- start start 2024-04-26T01:39:02.148 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.886+0000 7faede200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed807ddb0 0x7faed807c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:02.149 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.886+0000 7faede200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faed807c7e0 con 0x7faed807ddb0 2024-04-26T01:39:02.149 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.887+0000 7faedce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed807ddb0 0x7faed807c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:02.149 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.887+0000 7faedce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed807ddb0 0x7faed807c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48892/0 (socket says 172.21.15.1:48892) 2024-04-26T01:39:02.149 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.887+0000 7faedce00700 1 -- 172.21.15.1:0/2459185160 learned_addr learned my addr 172.21.15.1:0/2459185160 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:02.149 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.887+0000 7faedce00700 1 -- 172.21.15.1:0/2459185160 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed807c960 con 0x7faed807ddb0 2024-04-26T01:39:02.149 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.887+0000 7faedce00700 1 --2- 172.21.15.1:0/2459185160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed807ddb0 0x7faed807c210 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faecc0098a0 tx=0x7faecc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953c9d54a80a1c8c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:02.149 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.888+0000 7faed7400700 1 -- 172.21.15.1:0/2459185160 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faecc004160 con 0x7faed807ddb0 2024-04-26T01:39:02.149 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.888+0000 7faed7400700 1 -- 172.21.15.1:0/2459185160 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faecc004300 con 0x7faed807ddb0 2024-04-26T01:39:02.149 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.888+0000 7faed7400700 1 -- 172.21.15.1:0/2459185160 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faecc011af0 con 0x7faed807ddb0 2024-04-26T01:39:02.149 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.888+0000 7faede200700 1 -- 172.21.15.1:0/2459185160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed807ddb0 msgr2=0x7faed807c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:02.149 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.888+0000 7faede200700 1 --2- 172.21.15.1:0/2459185160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed807ddb0 0x7faed807c210 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faecc0098a0 tx=0x7faecc009bb0 comp rx=0 tx=0).stop 2024-04-26T01:39:02.149 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.888+0000 7faede200700 1 -- 172.21.15.1:0/2459185160 shutdown_connections 2024-04-26T01:39:02.149 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.888+0000 7faede200700 1 --2- 172.21.15.1:0/2459185160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed807ddb0 0x7faed807c210 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:02.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.888+0000 7faede200700 1 -- 172.21.15.1:0/2459185160 >> 172.21.15.1:0/2459185160 conn(0x7faed8103a10 msgr2=0x7faed8105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:02.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.889+0000 7faede200700 1 -- 172.21.15.1:0/2459185160 shutdown_connections 2024-04-26T01:39:02.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.889+0000 7faede200700 1 -- 172.21.15.1:0/2459185160 wait complete. 2024-04-26T01:39:02.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.890+0000 7faede200700 1 Processor -- start 2024-04-26T01:39:02.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.890+0000 7faede200700 1 -- start start 2024-04-26T01:39:02.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.890+0000 7faede200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed819e810 0x7faed819ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:02.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.890+0000 7faede200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faed810e960 con 0x7faed819e810 2024-04-26T01:39:02.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.891+0000 7faedce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed819e810 0x7faed819ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:02.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.891+0000 7faedce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed819e810 0x7faed819ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48904/0 (socket says 172.21.15.1:48904) 2024-04-26T01:39:02.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.891+0000 7faedce00700 1 -- 172.21.15.1:0/4107425136 learned_addr learned my addr 172.21.15.1:0/4107425136 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:02.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.891+0000 7faedce00700 1 -- 172.21.15.1:0/4107425136 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed819f170 con 0x7faed819e810 2024-04-26T01:39:02.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.891+0000 7faedce00700 1 --2- 172.21.15.1:0/4107425136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed819e810 0x7faed819ec30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faecc000c00 tx=0x7faecc004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:02.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.892+0000 7faed5600700 1 -- 172.21.15.1:0/4107425136 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faecc0037d0 con 0x7faed819e810 2024-04-26T01:39:02.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.892+0000 7faed5600700 1 -- 172.21.15.1:0/4107425136 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faecc003970 con 0x7faed819e810 2024-04-26T01:39:02.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.892+0000 7faede200700 1 -- 172.21.15.1:0/4107425136 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed819f400 con 0x7faed819e810 2024-04-26T01:39:02.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.892+0000 7faed5600700 1 -- 172.21.15.1:0/4107425136 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faecc01b440 con 0x7faed819e810 2024-04-26T01:39:02.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.892+0000 7faede200700 1 -- 172.21.15.1:0/4107425136 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed81a1e90 con 0x7faed819e810 2024-04-26T01:39:02.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.893+0000 7faed5600700 1 -- 172.21.15.1:0/4107425136 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7faecc01a440 con 0x7faed819e810 2024-04-26T01:39:02.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.893+0000 7faede200700 1 -- 172.21.15.1:0/4107425136 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faed80640a0 con 0x7faed819e810 2024-04-26T01:39:02.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.894+0000 7faed5600700 1 --2- 172.21.15.1:0/4107425136 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7faec00395b0 0x7faec003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:02.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.894+0000 7faed5600700 1 -- 172.21.15.1:0/4107425136 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faecc04db40 con 0x7faed819e810 2024-04-26T01:39:02.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.894+0000 7faed7e00700 1 --2- 172.21.15.1:0/4107425136 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7faec00395b0 0x7faec003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:02.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.895+0000 7faed7e00700 1 --2- 172.21.15.1:0/4107425136 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7faec00395b0 0x7faec003ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faec4006fd0 tx=0x7faec4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:02.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:54.899+0000 7faed5600700 1 -- 172.21.15.1:0/4107425136 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faecc0172b0 con 0x7faed819e810 2024-04-26T01:39:02.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:38:55.132+0000 7faede200700 1 -- 172.21.15.1:0/4107425136 --> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi001", "addr": "172.21.15.1", "target": ["mon-mgr", ""]}) -- 0x7faed8104ee0 con 0x7faec00395b0 2024-04-26T01:39:02.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:01.469+0000 7faed5600700 1 -- 172.21.15.1:0/4107425136 <== mgr.14120 v2:172.21.15.1:6800/1870580982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7faed8104ee0 con 0x7faec00395b0 2024-04-26T01:39:02.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:01.474+0000 7faede200700 1 -- 172.21.15.1:0/4107425136 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7faec00395b0 msgr2=0x7faec003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:02.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:01.474+0000 7faede200700 1 --2- 172.21.15.1:0/4107425136 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7faec00395b0 0x7faec003ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faec4006fd0 tx=0x7faec4006e40 comp rx=0 tx=0).stop 2024-04-26T01:39:02.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:01.474+0000 7faede200700 1 -- 172.21.15.1:0/4107425136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed819e810 msgr2=0x7faed819ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:02.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:01.474+0000 7faede200700 1 --2- 172.21.15.1:0/4107425136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed819e810 0x7faed819ec30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faecc000c00 tx=0x7faecc004260 comp rx=0 tx=0).stop 2024-04-26T01:39:02.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:01.474+0000 7faede200700 1 -- 172.21.15.1:0/4107425136 shutdown_connections 2024-04-26T01:39:02.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:01.474+0000 7faede200700 1 --2- 172.21.15.1:0/4107425136 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7faec00395b0 0x7faec003ba70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:02.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:01.474+0000 7faede200700 1 --2- 172.21.15.1:0/4107425136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed819e810 0x7faed819ec30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:02.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:01.474+0000 7faede200700 1 -- 172.21.15.1:0/4107425136 >> 172.21.15.1:0/4107425136 conn(0x7faed8103a10 msgr2=0x7faed8104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:02.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:01.474+0000 7faede200700 1 -- 172.21.15.1:0/4107425136 shutdown_connections 2024-04-26T01:39:02.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:01.475+0000 7faede200700 1 -- 172.21.15.1:0/4107425136 wait complete. 2024-04-26T01:39:02.152 INFO:teuthology.orchestra.run.smithi001.stdout:Deploying unmanaged mon service... 2024-04-26T01:39:02.751 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:02 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' 2024-04-26T01:39:02.752 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:02 smithi001 ceph-mon[27822]: Added host smithi001 2024-04-26T01:39:02.752 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:02 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:39:03.665 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-26T01:39:03.665 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.754+0000 7fd08ea00700 1 Processor -- start 2024-04-26T01:39:03.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.754+0000 7fd08ea00700 1 -- start start 2024-04-26T01:39:03.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.755+0000 7fd08ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0881090d0 0x7fd08810b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:03.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.755+0000 7fd08ea00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd088063d40 con 0x7fd0881090d0 2024-04-26T01:39:03.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.756+0000 7fd087e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0881090d0 0x7fd08810b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:03.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.756+0000 7fd087e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0881090d0 0x7fd08810b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48506/0 (socket says 172.21.15.1:48506) 2024-04-26T01:39:03.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.756+0000 7fd087e00700 1 -- 172.21.15.1:0/838831988 learned_addr learned my addr 172.21.15.1:0/838831988 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:03.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.756+0000 7fd087e00700 1 -- 172.21.15.1:0/838831988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08810ba00 con 0x7fd0881090d0 2024-04-26T01:39:03.668 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.757+0000 7fd087e00700 1 --2- 172.21.15.1:0/838831988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0881090d0 0x7fd08810b4c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd078009e80 tx=0x7fd07800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420a05bbb7d5e509 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:03.668 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.757+0000 7fd086a00700 1 -- 172.21.15.1:0/838831988 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd078017070 con 0x7fd0881090d0 2024-04-26T01:39:03.668 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.757+0000 7fd086a00700 1 -- 172.21.15.1:0/838831988 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd07800b8c0 con 0x7fd0881090d0 2024-04-26T01:39:03.668 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.757+0000 7fd086a00700 1 -- 172.21.15.1:0/838831988 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd078004490 con 0x7fd0881090d0 2024-04-26T01:39:03.668 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.758+0000 7fd08ea00700 1 -- 172.21.15.1:0/838831988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0881090d0 msgr2=0x7fd08810b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:03.668 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.758+0000 7fd08ea00700 1 --2- 172.21.15.1:0/838831988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0881090d0 0x7fd08810b4c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd078009e80 tx=0x7fd07800b330 comp rx=0 tx=0).stop 2024-04-26T01:39:03.672 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.758+0000 7fd08ea00700 1 -- 172.21.15.1:0/838831988 shutdown_connections 2024-04-26T01:39:03.672 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.758+0000 7fd08ea00700 1 --2- 172.21.15.1:0/838831988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0881090d0 0x7fd08810b4c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:03.672 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.758+0000 7fd08ea00700 1 -- 172.21.15.1:0/838831988 >> 172.21.15.1:0/838831988 conn(0x7fd088102b80 msgr2=0x7fd088104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:03.672 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.758+0000 7fd08ea00700 1 -- 172.21.15.1:0/838831988 shutdown_connections 2024-04-26T01:39:03.672 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.758+0000 7fd08ea00700 1 -- 172.21.15.1:0/838831988 wait complete. 2024-04-26T01:39:03.672 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.760+0000 7fd08ea00700 1 Processor -- start 2024-04-26T01:39:03.672 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.760+0000 7fd08ea00700 1 -- start start 2024-04-26T01:39:03.673 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.760+0000 7fd08ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0881090d0 0x7fd08819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:03.673 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.760+0000 7fd08ea00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd08810c8f0 con 0x7fd0881090d0 2024-04-26T01:39:03.673 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.760+0000 7fd087e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0881090d0 0x7fd08819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:03.673 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.761+0000 7fd087e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0881090d0 0x7fd08819c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48508/0 (socket says 172.21.15.1:48508) 2024-04-26T01:39:03.673 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.761+0000 7fd087e00700 1 -- 172.21.15.1:0/225083668 learned_addr learned my addr 172.21.15.1:0/225083668 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:03.673 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.761+0000 7fd087e00700 1 -- 172.21.15.1:0/225083668 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08819c9a0 con 0x7fd0881090d0 2024-04-26T01:39:03.673 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.761+0000 7fd087e00700 1 --2- 172.21.15.1:0/225083668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0881090d0 0x7fd08819c460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd07800b7b0 tx=0x7fd07800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:03.673 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.761+0000 7fd084c00700 1 -- 172.21.15.1:0/225083668 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd078020020 con 0x7fd0881090d0 2024-04-26T01:39:03.673 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.762+0000 7fd084c00700 1 -- 172.21.15.1:0/225083668 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd078003d10 con 0x7fd0881090d0 2024-04-26T01:39:03.673 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.762+0000 7fd084c00700 1 -- 172.21.15.1:0/225083668 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd078017040 con 0x7fd0881090d0 2024-04-26T01:39:03.673 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.762+0000 7fd08ea00700 1 -- 172.21.15.1:0/225083668 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08819cc30 con 0x7fd0881090d0 2024-04-26T01:39:03.673 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.762+0000 7fd08ea00700 1 -- 172.21.15.1:0/225083668 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd08819f920 con 0x7fd0881090d0 2024-04-26T01:39:03.673 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.763+0000 7fd084c00700 1 -- 172.21.15.1:0/225083668 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd078023020 con 0x7fd0881090d0 2024-04-26T01:39:03.673 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.763+0000 7fd08ea00700 1 -- 172.21.15.1:0/225083668 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0880640a0 con 0x7fd0881090d0 2024-04-26T01:39:03.674 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.763+0000 7fd084c00700 1 --2- 172.21.15.1:0/225083668 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7fd06c039600 0x7fd06c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:03.674 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.763+0000 7fd084c00700 1 -- 172.21.15.1:0/225083668 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd07800f050 con 0x7fd0881090d0 2024-04-26T01:39:03.674 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.764+0000 7fd087400700 1 --2- 172.21.15.1:0/225083668 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7fd06c039600 0x7fd06c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:03.674 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.765+0000 7fd087400700 1 --2- 172.21.15.1:0/225083668 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7fd06c039600 0x7fd06c03bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd074006fd0 tx=0x7fd074006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:03.674 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:02.770+0000 7fd084c00700 1 -- 172.21.15.1:0/225083668 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd07802a020 con 0x7fd0881090d0 2024-04-26T01:39:03.674 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:03.020+0000 7fd08ea00700 1 -- 172.21.15.1:0/225083668 --> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fd088103d30 con 0x7fd06c039600 2024-04-26T01:39:03.674 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:03.027+0000 7fd084c00700 1 -- 172.21.15.1:0/225083668 <== mgr.14120 v2:172.21.15.1:6800/1870580982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd088103d30 con 0x7fd06c039600 2024-04-26T01:39:03.674 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:03.030+0000 7fd08ea00700 1 -- 172.21.15.1:0/225083668 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7fd06c039600 msgr2=0x7fd06c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:03.674 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:03.030+0000 7fd08ea00700 1 --2- 172.21.15.1:0/225083668 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7fd06c039600 0x7fd06c03bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd074006fd0 tx=0x7fd074006e40 comp rx=0 tx=0).stop 2024-04-26T01:39:03.674 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:03.030+0000 7fd08ea00700 1 -- 172.21.15.1:0/225083668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0881090d0 msgr2=0x7fd08819c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:03.674 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:03.030+0000 7fd08ea00700 1 --2- 172.21.15.1:0/225083668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0881090d0 0x7fd08819c460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd07800b7b0 tx=0x7fd07800be90 comp rx=0 tx=0).stop 2024-04-26T01:39:03.674 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:03.030+0000 7fd08ea00700 1 -- 172.21.15.1:0/225083668 shutdown_connections 2024-04-26T01:39:03.674 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:03.030+0000 7fd08ea00700 1 --2- 172.21.15.1:0/225083668 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7fd06c039600 0x7fd06c03bac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:03.675 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:03.030+0000 7fd08ea00700 1 --2- 172.21.15.1:0/225083668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0881090d0 0x7fd08819c460 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:03.675 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:03.030+0000 7fd08ea00700 1 -- 172.21.15.1:0/225083668 >> 172.21.15.1:0/225083668 conn(0x7fd088102b80 msgr2=0x7fd088103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:03.675 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:03.030+0000 7fd08ea00700 1 -- 172.21.15.1:0/225083668 shutdown_connections 2024-04-26T01:39:03.675 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:03.030+0000 7fd08ea00700 1 -- 172.21.15.1:0/225083668 wait complete. 2024-04-26T01:39:03.675 INFO:teuthology.orchestra.run.smithi001.stdout:Deploying unmanaged mgr service... 2024-04-26T01:39:04.207 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:04 smithi001 ceph-mon[27822]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:39:04.208 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:04 smithi001 ceph-mon[27822]: Saving service mon spec with placement count:5 2024-04-26T01:39:04.208 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:04 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' 2024-04-26T01:39:05.525 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-26T01:39:05.525 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.521+0000 7f20a9800700 1 Processor -- start 2024-04-26T01:39:05.526 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.521+0000 7f20a9800700 1 -- start start 2024-04-26T01:39:05.526 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.522+0000 7f20a9800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a4106e70 0x7f20a41092a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:05.527 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.522+0000 7f20a9800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f20a407ceb0 con 0x7f20a4106e70 2024-04-26T01:39:05.527 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.522+0000 7f20a2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a4106e70 0x7f20a41092a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:05.527 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.522+0000 7f20a2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a4106e70 0x7f20a41092a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48522/0 (socket says 172.21.15.1:48522) 2024-04-26T01:39:05.527 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.522+0000 7f20a2a00700 1 -- 172.21.15.1:0/3111307135 learned_addr learned my addr 172.21.15.1:0/3111307135 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:05.527 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.523+0000 7f20a2a00700 1 -- 172.21.15.1:0/3111307135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a41097e0 con 0x7f20a4106e70 2024-04-26T01:39:05.528 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.523+0000 7f20a2a00700 1 --2- 172.21.15.1:0/3111307135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a4106e70 0x7f20a41092a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f20940098a0 tx=0x7f2094009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eba3d194f1b6fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:05.529 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.523+0000 7f20a1600700 1 -- 172.21.15.1:0/3111307135 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2094004160 con 0x7f20a4106e70 2024-04-26T01:39:05.529 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.523+0000 7f20a1600700 1 -- 172.21.15.1:0/3111307135 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2094004300 con 0x7f20a4106e70 2024-04-26T01:39:05.529 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.523+0000 7f20a1600700 1 -- 172.21.15.1:0/3111307135 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2094011af0 con 0x7f20a4106e70 2024-04-26T01:39:05.529 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.524+0000 7f20a9800700 1 -- 172.21.15.1:0/3111307135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a4106e70 msgr2=0x7f20a41092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:05.529 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.524+0000 7f20a9800700 1 --2- 172.21.15.1:0/3111307135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a4106e70 0x7f20a41092a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f20940098a0 tx=0x7f2094009bb0 comp rx=0 tx=0).stop 2024-04-26T01:39:05.529 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.524+0000 7f20a9800700 1 -- 172.21.15.1:0/3111307135 shutdown_connections 2024-04-26T01:39:05.529 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.524+0000 7f20a9800700 1 --2- 172.21.15.1:0/3111307135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a4106e70 0x7f20a41092a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:05.529 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.524+0000 7f20a9800700 1 -- 172.21.15.1:0/3111307135 >> 172.21.15.1:0/3111307135 conn(0x7f20a4102b10 msgr2=0x7f20a4104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:05.529 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.524+0000 7f20a9800700 1 -- 172.21.15.1:0/3111307135 shutdown_connections 2024-04-26T01:39:05.529 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.524+0000 7f20a9800700 1 -- 172.21.15.1:0/3111307135 wait complete. 2024-04-26T01:39:05.529 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.525+0000 7f20a9800700 1 Processor -- start 2024-04-26T01:39:05.529 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.526+0000 7f20a9800700 1 -- start start 2024-04-26T01:39:05.531 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.526+0000 7f20a9800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a4106e70 0x7f20a41a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:05.532 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.526+0000 7f20a9800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f20a410a7a0 con 0x7f20a4106e70 2024-04-26T01:39:05.532 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.526+0000 7f20a2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a4106e70 0x7f20a41a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:05.532 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.526+0000 7f20a2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a4106e70 0x7f20a41a2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48530/0 (socket says 172.21.15.1:48530) 2024-04-26T01:39:05.532 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.526+0000 7f20a2a00700 1 -- 172.21.15.1:0/1311911153 learned_addr learned my addr 172.21.15.1:0/1311911153 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:05.532 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.527+0000 7f20a2a00700 1 -- 172.21.15.1:0/1311911153 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a41a3040 con 0x7f20a4106e70 2024-04-26T01:39:05.532 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.527+0000 7f20a2a00700 1 --2- 172.21.15.1:0/1311911153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a4106e70 0x7f20a41a2b00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2094011ac0 tx=0x7f2094011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:05.532 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.527+0000 7f209b400700 1 -- 172.21.15.1:0/1311911153 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2094003970 con 0x7f20a4106e70 2024-04-26T01:39:05.532 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.527+0000 7f209b400700 1 -- 172.21.15.1:0/1311911153 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2094003b10 con 0x7f20a4106e70 2024-04-26T01:39:05.532 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.527+0000 7f20a9800700 1 -- 172.21.15.1:0/1311911153 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a41a32d0 con 0x7f20a4106e70 2024-04-26T01:39:05.532 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.527+0000 7f209b400700 1 -- 172.21.15.1:0/1311911153 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2094024610 con 0x7f20a4106e70 2024-04-26T01:39:05.532 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.527+0000 7f20a9800700 1 -- 172.21.15.1:0/1311911153 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a41a5fc0 con 0x7f20a4106e70 2024-04-26T01:39:05.532 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.528+0000 7f209b400700 1 -- 172.21.15.1:0/1311911153 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f209401a9e0 con 0x7f20a4106e70 2024-04-26T01:39:05.532 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.528+0000 7f20a9800700 1 -- 172.21.15.1:0/1311911153 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20a40640a0 con 0x7f20a4106e70 2024-04-26T01:39:05.533 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.529+0000 7f209b400700 1 --2- 172.21.15.1:0/1311911153 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f2088039600 0x7f208803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:05.533 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.529+0000 7f209b400700 1 -- 172.21.15.1:0/1311911153 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f209404d9b0 con 0x7f20a4106e70 2024-04-26T01:39:05.533 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.529+0000 7f20a2000700 1 --2- 172.21.15.1:0/1311911153 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f2088039600 0x7f208803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:05.533 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.530+0000 7f20a2000700 1 --2- 172.21.15.1:0/1311911153 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f2088039600 0x7f208803bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2090006fd0 tx=0x7f2090006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:05.533 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.535+0000 7f209b400700 1 -- 172.21.15.1:0/1311911153 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f209400b3e0 con 0x7f20a4106e70 2024-04-26T01:39:05.534 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.787+0000 7f20a9800700 1 -- 172.21.15.1:0/1311911153 --> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f20a40008d0 con 0x7f2088039600 2024-04-26T01:39:05.534 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.796+0000 7f209b400700 1 -- 172.21.15.1:0/1311911153 <== mgr.14120 v2:172.21.15.1:6800/1870580982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f20a40008d0 con 0x7f2088039600 2024-04-26T01:39:05.534 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.799+0000 7f20a9800700 1 -- 172.21.15.1:0/1311911153 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f2088039600 msgr2=0x7f208803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:05.535 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.799+0000 7f20a9800700 1 --2- 172.21.15.1:0/1311911153 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f2088039600 0x7f208803bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2090006fd0 tx=0x7f2090006e40 comp rx=0 tx=0).stop 2024-04-26T01:39:05.535 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.799+0000 7f20a9800700 1 -- 172.21.15.1:0/1311911153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a4106e70 msgr2=0x7f20a41a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:05.535 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.799+0000 7f20a9800700 1 --2- 172.21.15.1:0/1311911153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a4106e70 0x7f20a41a2b00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2094011ac0 tx=0x7f2094011ba0 comp rx=0 tx=0).stop 2024-04-26T01:39:05.535 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.800+0000 7f20a9800700 1 -- 172.21.15.1:0/1311911153 shutdown_connections 2024-04-26T01:39:05.535 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.800+0000 7f20a9800700 1 --2- 172.21.15.1:0/1311911153 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f2088039600 0x7f208803bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:05.535 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.800+0000 7f20a9800700 1 --2- 172.21.15.1:0/1311911153 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a4106e70 0x7f20a41a2b00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:05.535 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.800+0000 7f20a9800700 1 -- 172.21.15.1:0/1311911153 >> 172.21.15.1:0/1311911153 conn(0x7f20a4102b10 msgr2=0x7f20a4104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:05.535 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.800+0000 7f20a9800700 1 -- 172.21.15.1:0/1311911153 shutdown_connections 2024-04-26T01:39:05.535 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:04.800+0000 7f20a9800700 1 -- 172.21.15.1:0/1311911153 wait complete. 2024-04-26T01:39:06.051 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:05 smithi001 ceph-mon[27822]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:39:06.051 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:05 smithi001 ceph-mon[27822]: Saving service mgr spec with placement count:2 2024-04-26T01:39:06.051 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:05 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' 2024-04-26T01:39:07.377 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:07 smithi001 ceph-mon[27822]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:39:07.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:07 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' 2024-04-26T01:39:07.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:07 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2087660502' entity='client.admin' 2024-04-26T01:39:07.434 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.489+0000 7f6fa1200700 1 Processor -- start 2024-04-26T01:39:07.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.490+0000 7f6fa1200700 1 -- start start 2024-04-26T01:39:07.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.490+0000 7f6fa1200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f9c108610 0x7f6f9c108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:07.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.490+0000 7f6fa1200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c109000 con 0x7f6f9c108610 2024-04-26T01:39:07.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.491+0000 7f6f99e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f9c108610 0x7f6f9c108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:07.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.491+0000 7f6f99e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f9c108610 0x7f6f9c108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48546/0 (socket says 172.21.15.1:48546) 2024-04-26T01:39:07.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.491+0000 7f6f99e00700 1 -- 172.21.15.1:0/2675198046 learned_addr learned my addr 172.21.15.1:0/2675198046 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:07.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.491+0000 7f6f99e00700 1 -- 172.21.15.1:0/2675198046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9c109810 con 0x7f6f9c108610 2024-04-26T01:39:07.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.492+0000 7f6f99e00700 1 --2- 172.21.15.1:0/2675198046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f9c108610 0x7f6f9c108a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0098a0 tx=0x7f6f8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8fa2e85fe5e8fc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:07.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.492+0000 7f6f98a00700 1 -- 172.21.15.1:0/2675198046 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f8c004160 con 0x7f6f9c108610 2024-04-26T01:39:07.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.492+0000 7f6f98a00700 1 -- 172.21.15.1:0/2675198046 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f8c004300 con 0x7f6f9c108610 2024-04-26T01:39:07.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.492+0000 7f6f98a00700 1 -- 172.21.15.1:0/2675198046 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f8c011a20 con 0x7f6f9c108610 2024-04-26T01:39:07.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.493+0000 7f6fa1200700 1 -- 172.21.15.1:0/2675198046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f9c108610 msgr2=0x7f6f9c108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:07.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.493+0000 7f6fa1200700 1 --2- 172.21.15.1:0/2675198046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f9c108610 0x7f6f9c108a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0098a0 tx=0x7f6f8c009bb0 comp rx=0 tx=0).stop 2024-04-26T01:39:07.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.493+0000 7f6fa1200700 1 -- 172.21.15.1:0/2675198046 shutdown_connections 2024-04-26T01:39:07.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.493+0000 7f6fa1200700 1 --2- 172.21.15.1:0/2675198046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f9c108610 0x7f6f9c108a30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:07.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.493+0000 7f6fa1200700 1 -- 172.21.15.1:0/2675198046 >> 172.21.15.1:0/2675198046 conn(0x7f6f9c103c70 msgr2=0x7f6f9c106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:07.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.493+0000 7f6fa1200700 1 -- 172.21.15.1:0/2675198046 shutdown_connections 2024-04-26T01:39:07.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.493+0000 7f6fa1200700 1 -- 172.21.15.1:0/2675198046 wait complete. 2024-04-26T01:39:07.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.494+0000 7f6fa1200700 1 Processor -- start 2024-04-26T01:39:07.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.494+0000 7f6fa1200700 1 -- start start 2024-04-26T01:39:07.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.495+0000 7f6fa1200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f9c108610 0x7f6f9c07da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:07.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.495+0000 7f6fa1200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9c10cbc0 con 0x7f6f9c108610 2024-04-26T01:39:07.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.495+0000 7f6f99e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f9c108610 0x7f6f9c07da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:07.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.495+0000 7f6f99e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f9c108610 0x7f6f9c07da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48548/0 (socket says 172.21.15.1:48548) 2024-04-26T01:39:07.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.495+0000 7f6f99e00700 1 -- 172.21.15.1:0/2087660502 learned_addr learned my addr 172.21.15.1:0/2087660502 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:07.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.495+0000 7f6f99e00700 1 -- 172.21.15.1:0/2087660502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9c07dfa0 con 0x7f6f9c108610 2024-04-26T01:39:07.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.495+0000 7f6f99e00700 1 --2- 172.21.15.1:0/2087660502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f9c108610 0x7f6f9c07da60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c000c00 tx=0x7f6f8c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:07.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.496+0000 7f6f92a00700 1 -- 172.21.15.1:0/2087660502 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f8c003aa0 con 0x7f6f9c108610 2024-04-26T01:39:07.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.496+0000 7f6f92a00700 1 -- 172.21.15.1:0/2087660502 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f8c01b8f0 con 0x7f6f9c108610 2024-04-26T01:39:07.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.496+0000 7f6f92a00700 1 -- 172.21.15.1:0/2087660502 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f8c023890 con 0x7f6f9c108610 2024-04-26T01:39:07.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.496+0000 7f6fa1200700 1 -- 172.21.15.1:0/2087660502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f9c07c190 con 0x7f6f9c108610 2024-04-26T01:39:07.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.496+0000 7f6fa1200700 1 -- 172.21.15.1:0/2087660502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f9c07c6d0 con 0x7f6f9c108610 2024-04-26T01:39:07.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.497+0000 7f6f92a00700 1 -- 172.21.15.1:0/2087660502 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6f8c01b440 con 0x7f6f9c108610 2024-04-26T01:39:07.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.497+0000 7f6fa1200700 1 -- 172.21.15.1:0/2087660502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f9c0640a0 con 0x7f6f9c108610 2024-04-26T01:39:07.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.498+0000 7f6f92a00700 1 --2- 172.21.15.1:0/2087660502 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f6f80039600 0x7f6f8003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:07.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.498+0000 7f6f92a00700 1 -- 172.21.15.1:0/2087660502 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6f8c04deb0 con 0x7f6f9c108610 2024-04-26T01:39:07.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.498+0000 7f6f99400700 1 --2- 172.21.15.1:0/2087660502 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f6f80039600 0x7f6f8003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:07.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.499+0000 7f6f99400700 1 --2- 172.21.15.1:0/2087660502 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f6f80039600 0x7f6f8003bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6f88006fd0 tx=0x7f6f88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:07.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.503+0000 7f6f92a00700 1 -- 172.21.15.1:0/2087660502 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f8c01f030 con 0x7f6f9c108610 2024-04-26T01:39:07.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.735+0000 7f6fa1200700 1 -- 172.21.15.1:0/2087660502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f6f9c07c950 con 0x7f6f9c108610 2024-04-26T01:39:07.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.738+0000 7f6f92a00700 1 -- 172.21.15.1:0/2087660502 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f6f8c0292a0 con 0x7f6f9c108610 2024-04-26T01:39:07.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.741+0000 7f6fa1200700 1 -- 172.21.15.1:0/2087660502 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f6f80039600 msgr2=0x7f6f8003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:07.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.741+0000 7f6fa1200700 1 --2- 172.21.15.1:0/2087660502 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f6f80039600 0x7f6f8003bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6f88006fd0 tx=0x7f6f88006e40 comp rx=0 tx=0).stop 2024-04-26T01:39:07.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.741+0000 7f6fa1200700 1 -- 172.21.15.1:0/2087660502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f9c108610 msgr2=0x7f6f9c07da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:07.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.741+0000 7f6fa1200700 1 --2- 172.21.15.1:0/2087660502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f9c108610 0x7f6f9c07da60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c000c00 tx=0x7f6f8c011aa0 comp rx=0 tx=0).stop 2024-04-26T01:39:07.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.741+0000 7f6fa1200700 1 -- 172.21.15.1:0/2087660502 shutdown_connections 2024-04-26T01:39:07.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.741+0000 7f6fa1200700 1 --2- 172.21.15.1:0/2087660502 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f6f80039600 0x7f6f8003bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:07.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.741+0000 7f6fa1200700 1 --2- 172.21.15.1:0/2087660502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f9c108610 0x7f6f9c07da60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:07.439 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.741+0000 7f6fa1200700 1 -- 172.21.15.1:0/2087660502 >> 172.21.15.1:0/2087660502 conn(0x7f6f9c103c70 msgr2=0x7f6f9c104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:07.439 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.742+0000 7f6fa1200700 1 -- 172.21.15.1:0/2087660502 shutdown_connections 2024-04-26T01:39:07.439 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:06.742+0000 7f6fa1200700 1 -- 172.21.15.1:0/2087660502 wait complete. 2024-04-26T01:39:08.377 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:08 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' 2024-04-26T01:39:09.098 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.145+0000 7f1298400700 1 Processor -- start 2024-04-26T01:39:09.099 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.146+0000 7f1298400700 1 -- start start 2024-04-26T01:39:09.099 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.146+0000 7f1298400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1290106ee0 0x7f1290107300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:09.099 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.146+0000 7f1298400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1290063d20 con 0x7f1290106ee0 2024-04-26T01:39:09.099 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.147+0000 7f1295a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1290106ee0 0x7f1290107300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:09.099 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.147+0000 7f1295a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1290106ee0 0x7f1290107300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48554/0 (socket says 172.21.15.1:48554) 2024-04-26T01:39:09.099 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.147+0000 7f1295a00700 1 -- 172.21.15.1:0/1548574728 learned_addr learned my addr 172.21.15.1:0/1548574728 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:09.101 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.147+0000 7f1295a00700 1 -- 172.21.15.1:0/1548574728 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1290107840 con 0x7f1290106ee0 2024-04-26T01:39:09.101 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.148+0000 7f1295a00700 1 --2- 172.21.15.1:0/1548574728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1290106ee0 0x7f1290107300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f12840098a0 tx=0x7f1284009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6d5815f5b12562 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:09.101 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.148+0000 7f128fe00700 1 -- 172.21.15.1:0/1548574728 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1284004160 con 0x7f1290106ee0 2024-04-26T01:39:09.101 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.148+0000 7f128fe00700 1 -- 172.21.15.1:0/1548574728 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1284004300 con 0x7f1290106ee0 2024-04-26T01:39:09.101 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.148+0000 7f128fe00700 1 -- 172.21.15.1:0/1548574728 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1284011a20 con 0x7f1290106ee0 2024-04-26T01:39:09.101 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.149+0000 7f1298400700 1 -- 172.21.15.1:0/1548574728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1290106ee0 msgr2=0x7f1290107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:09.102 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.149+0000 7f1298400700 1 --2- 172.21.15.1:0/1548574728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1290106ee0 0x7f1290107300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f12840098a0 tx=0x7f1284009bb0 comp rx=0 tx=0).stop 2024-04-26T01:39:09.102 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.149+0000 7f1298400700 1 -- 172.21.15.1:0/1548574728 shutdown_connections 2024-04-26T01:39:09.102 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.149+0000 7f1298400700 1 --2- 172.21.15.1:0/1548574728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1290106ee0 0x7f1290107300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:09.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.149+0000 7f1298400700 1 -- 172.21.15.1:0/1548574728 >> 172.21.15.1:0/1548574728 conn(0x7f1290102b80 msgr2=0x7f1290104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:09.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.149+0000 7f1298400700 1 -- 172.21.15.1:0/1548574728 shutdown_connections 2024-04-26T01:39:09.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.149+0000 7f1298400700 1 -- 172.21.15.1:0/1548574728 wait complete. 2024-04-26T01:39:09.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.150+0000 7f1298400700 1 Processor -- start 2024-04-26T01:39:09.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.150+0000 7f1298400700 1 -- start start 2024-04-26T01:39:09.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.151+0000 7f1298400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1290106ee0 0x7f129019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:09.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.151+0000 7f1298400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f12901088b0 con 0x7f1290106ee0 2024-04-26T01:39:09.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.151+0000 7f1295a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1290106ee0 0x7f129019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:09.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.151+0000 7f1295a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1290106ee0 0x7f129019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48570/0 (socket says 172.21.15.1:48570) 2024-04-26T01:39:09.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.151+0000 7f1295a00700 1 -- 172.21.15.1:0/516945413 learned_addr learned my addr 172.21.15.1:0/516945413 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:09.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.151+0000 7f1295a00700 1 -- 172.21.15.1:0/516945413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129019ebf0 con 0x7f1290106ee0 2024-04-26T01:39:09.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.152+0000 7f1295a00700 1 --2- 172.21.15.1:0/516945413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1290106ee0 0x7f129019e6b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1284004660 tx=0x7f1284011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:09.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.152+0000 7f128e000700 1 -- 172.21.15.1:0/516945413 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1284003aa0 con 0x7f1290106ee0 2024-04-26T01:39:09.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.152+0000 7f128e000700 1 -- 172.21.15.1:0/516945413 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f128401b8f0 con 0x7f1290106ee0 2024-04-26T01:39:09.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.152+0000 7f128e000700 1 -- 172.21.15.1:0/516945413 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1284023900 con 0x7f1290106ee0 2024-04-26T01:39:09.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.152+0000 7f1298400700 1 -- 172.21.15.1:0/516945413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129019ee80 con 0x7f1290106ee0 2024-04-26T01:39:09.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.152+0000 7f1298400700 1 -- 172.21.15.1:0/516945413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12901a1b70 con 0x7f1290106ee0 2024-04-26T01:39:09.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.153+0000 7f128e000700 1 -- 172.21.15.1:0/516945413 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f128401baa0 con 0x7f1290106ee0 2024-04-26T01:39:09.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.154+0000 7f1298400700 1 -- 172.21.15.1:0/516945413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12900640a0 con 0x7f1290106ee0 2024-04-26T01:39:09.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.154+0000 7f128e000700 1 --2- 172.21.15.1:0/516945413 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f1278039600 0x7f127803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:09.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.154+0000 7f128e000700 1 -- 172.21.15.1:0/516945413 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f128404dec0 con 0x7f1290106ee0 2024-04-26T01:39:09.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.154+0000 7f1295000700 1 --2- 172.21.15.1:0/516945413 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f1278039600 0x7f127803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:09.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.155+0000 7f1295000700 1 --2- 172.21.15.1:0/516945413 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f1278039600 0x7f127803bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f127c006fd0 tx=0x7f127c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:09.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.160+0000 7f128e000700 1 -- 172.21.15.1:0/516945413 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f128401f030 con 0x7f1290106ee0 2024-04-26T01:39:09.109 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.404+0000 7f1298400700 1 -- 172.21.15.1:0/516945413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f129002e2c0 con 0x7f1290106ee0 2024-04-26T01:39:09.109 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.408+0000 7f128e000700 1 -- 172.21.15.1:0/516945413 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f128402c3f0 con 0x7f1290106ee0 2024-04-26T01:39:09.109 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.411+0000 7f1298400700 1 -- 172.21.15.1:0/516945413 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f1278039600 msgr2=0x7f127803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:09.109 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.411+0000 7f1298400700 1 --2- 172.21.15.1:0/516945413 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f1278039600 0x7f127803bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f127c006fd0 tx=0x7f127c006e40 comp rx=0 tx=0).stop 2024-04-26T01:39:09.109 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.411+0000 7f1298400700 1 -- 172.21.15.1:0/516945413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1290106ee0 msgr2=0x7f129019e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:09.109 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.411+0000 7f1298400700 1 --2- 172.21.15.1:0/516945413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1290106ee0 0x7f129019e6b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1284004660 tx=0x7f1284011aa0 comp rx=0 tx=0).stop 2024-04-26T01:39:09.109 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.411+0000 7f1298400700 1 -- 172.21.15.1:0/516945413 shutdown_connections 2024-04-26T01:39:09.109 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.411+0000 7f1298400700 1 --2- 172.21.15.1:0/516945413 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f1278039600 0x7f127803bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:09.109 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.411+0000 7f1298400700 1 --2- 172.21.15.1:0/516945413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1290106ee0 0x7f129019e6b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:09.109 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.411+0000 7f1298400700 1 -- 172.21.15.1:0/516945413 >> 172.21.15.1:0/516945413 conn(0x7f1290102b80 msgr2=0x7f1290195350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:09.109 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.412+0000 7f1298400700 1 -- 172.21.15.1:0/516945413 shutdown_connections 2024-04-26T01:39:09.109 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:08.412+0000 7f1298400700 1 -- 172.21.15.1:0/516945413 wait complete. 2024-04-26T01:39:09.109 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling the dashboard module... 2024-04-26T01:39:09.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:09 smithi001 ceph-mon[27822]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:39:09.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:09 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/516945413' entity='client.admin' 2024-04-26T01:39:11.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:11 smithi001 ceph-mon[27822]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:39:11.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:11 smithi001 ceph-mon[27822]: from='mgr.14120 172.21.15.1:0/4179471221' entity='mgr.x' 2024-04-26T01:39:11.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:11 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/36686245' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-26T01:39:11.628 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:11 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: ignoring --setuser ceph since I am not root 2024-04-26T01:39:11.628 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:11 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: ignoring --setgroup ceph since I am not root 2024-04-26T01:39:12.104 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:11 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:11.725+0000 7f8184586200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:39:12.105 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:11 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:11.971+0000 7f8184586200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:39:12.105 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:12 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:12.107+0000 7f8184586200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:39:12.222 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.302+0000 7ff063000700 1 Processor -- start 2024-04-26T01:39:12.222 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.302+0000 7ff063000700 1 -- start start 2024-04-26T01:39:12.222 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.303+0000 7ff063000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff05c108610 0x7ff05c108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:12.222 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.303+0000 7ff063000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c109000 con 0x7ff05c108610 2024-04-26T01:39:12.222 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.304+0000 7ff05be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff05c108610 0x7ff05c108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:12.222 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.304+0000 7ff05be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff05c108610 0x7ff05c108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48582/0 (socket says 172.21.15.1:48582) 2024-04-26T01:39:12.222 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.304+0000 7ff05be00700 1 -- 172.21.15.1:0/1824772948 learned_addr learned my addr 172.21.15.1:0/1824772948 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:12.223 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.305+0000 7ff05be00700 1 -- 172.21.15.1:0/1824772948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05c109810 con 0x7ff05c108610 2024-04-26T01:39:12.223 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.305+0000 7ff05be00700 1 --2- 172.21.15.1:0/1824772948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff05c108610 0x7ff05c108a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0098a0 tx=0x7ff04c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff78a19a13949e59 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:12.223 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.305+0000 7ff05aa00700 1 -- 172.21.15.1:0/1824772948 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff04c004160 con 0x7ff05c108610 2024-04-26T01:39:12.223 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.305+0000 7ff05aa00700 1 -- 172.21.15.1:0/1824772948 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff04c004300 con 0x7ff05c108610 2024-04-26T01:39:12.223 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.305+0000 7ff05aa00700 1 -- 172.21.15.1:0/1824772948 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff04c011a20 con 0x7ff05c108610 2024-04-26T01:39:12.223 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.306+0000 7ff063000700 1 -- 172.21.15.1:0/1824772948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff05c108610 msgr2=0x7ff05c108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:12.223 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.306+0000 7ff063000700 1 --2- 172.21.15.1:0/1824772948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff05c108610 0x7ff05c108a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0098a0 tx=0x7ff04c009bb0 comp rx=0 tx=0).stop 2024-04-26T01:39:12.223 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.307+0000 7ff063000700 1 -- 172.21.15.1:0/1824772948 shutdown_connections 2024-04-26T01:39:12.223 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.307+0000 7ff063000700 1 --2- 172.21.15.1:0/1824772948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff05c108610 0x7ff05c108a30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:12.223 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.307+0000 7ff063000700 1 -- 172.21.15.1:0/1824772948 >> 172.21.15.1:0/1824772948 conn(0x7ff05c103c70 msgr2=0x7ff05c106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:12.223 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.307+0000 7ff063000700 1 -- 172.21.15.1:0/1824772948 shutdown_connections 2024-04-26T01:39:12.224 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.307+0000 7ff063000700 1 -- 172.21.15.1:0/1824772948 wait complete. 2024-04-26T01:39:12.224 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.308+0000 7ff063000700 1 Processor -- start 2024-04-26T01:39:12.224 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.308+0000 7ff063000700 1 -- start start 2024-04-26T01:39:12.224 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.308+0000 7ff063000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff05c108610 0x7ff05c19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:12.224 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.308+0000 7ff063000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c10cbc0 con 0x7ff05c108610 2024-04-26T01:39:12.224 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.309+0000 7ff05be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff05c108610 0x7ff05c19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:12.224 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.309+0000 7ff05be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff05c108610 0x7ff05c19c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48596/0 (socket says 172.21.15.1:48596) 2024-04-26T01:39:12.224 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.309+0000 7ff05be00700 1 -- 172.21.15.1:0/36686245 learned_addr learned my addr 172.21.15.1:0/36686245 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:12.224 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.309+0000 7ff05be00700 1 -- 172.21.15.1:0/36686245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05c19c9f0 con 0x7ff05c108610 2024-04-26T01:39:12.224 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.309+0000 7ff05be00700 1 --2- 172.21.15.1:0/36686245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff05c108610 0x7ff05c19c4b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0119f0 tx=0x7ff04c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:12.224 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.310+0000 7ff058c00700 1 -- 172.21.15.1:0/36686245 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff04c0039e0 con 0x7ff05c108610 2024-04-26T01:39:12.224 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.310+0000 7ff058c00700 1 -- 172.21.15.1:0/36686245 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff04c01b8f0 con 0x7ff05c108610 2024-04-26T01:39:12.224 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.310+0000 7ff058c00700 1 -- 172.21.15.1:0/36686245 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff04c0237a0 con 0x7ff05c108610 2024-04-26T01:39:12.224 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.310+0000 7ff063000700 1 -- 172.21.15.1:0/36686245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05c19cc80 con 0x7ff05c108610 2024-04-26T01:39:12.225 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.310+0000 7ff063000700 1 -- 172.21.15.1:0/36686245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff05c19f970 con 0x7ff05c108610 2024-04-26T01:39:12.225 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.312+0000 7ff063000700 1 -- 172.21.15.1:0/36686245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff05c10cd40 con 0x7ff05c108610 2024-04-26T01:39:12.225 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.312+0000 7ff058c00700 1 -- 172.21.15.1:0/36686245 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff04c01b440 con 0x7ff05c108610 2024-04-26T01:39:12.225 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.312+0000 7ff058c00700 1 --2- 172.21.15.1:0/36686245 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7ff0400395b0 0x7ff04003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:12.225 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.312+0000 7ff058c00700 1 -- 172.21.15.1:0/36686245 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff04c04dfc0 con 0x7ff05c108610 2024-04-26T01:39:12.225 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.313+0000 7ff05b400700 1 --2- 172.21.15.1:0/36686245 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7ff0400395b0 0x7ff04003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:12.225 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.313+0000 7ff05b400700 1 --2- 172.21.15.1:0/36686245 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7ff0400395b0 0x7ff04003ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff048006fd0 tx=0x7ff048006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:12.225 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.318+0000 7ff058c00700 1 -- 172.21.15.1:0/36686245 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff04c01f030 con 0x7ff05c108610 2024-04-26T01:39:12.225 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:10.631+0000 7ff063000700 1 -- 172.21.15.1:0/36686245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7ff05c19ff20 con 0x7ff05c108610 2024-04-26T01:39:12.225 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:11.409+0000 7ff058c00700 1 -- 172.21.15.1:0/36686245 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7ff04c016260 con 0x7ff05c108610 2024-04-26T01:39:12.225 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:11.412+0000 7ff058c00700 1 -- 172.21.15.1:0/36686245 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7ff04c019750 con 0x7ff05c108610 2024-04-26T01:39:12.225 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:11.414+0000 7ff063000700 1 -- 172.21.15.1:0/36686245 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7ff0400395b0 msgr2=0x7ff04003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:12.225 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:11.414+0000 7ff063000700 1 --2- 172.21.15.1:0/36686245 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7ff0400395b0 0x7ff04003ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff048006fd0 tx=0x7ff048006e40 comp rx=0 tx=0).stop 2024-04-26T01:39:12.226 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:11.414+0000 7ff063000700 1 -- 172.21.15.1:0/36686245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff05c108610 msgr2=0x7ff05c19c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:12.226 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:11.414+0000 7ff063000700 1 --2- 172.21.15.1:0/36686245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff05c108610 0x7ff05c19c4b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0119f0 tx=0x7ff04c011ad0 comp rx=0 tx=0).stop 2024-04-26T01:39:12.226 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:11.414+0000 7ff063000700 1 -- 172.21.15.1:0/36686245 shutdown_connections 2024-04-26T01:39:12.226 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:11.414+0000 7ff063000700 1 --2- 172.21.15.1:0/36686245 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7ff0400395b0 0x7ff04003ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:12.226 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:11.414+0000 7ff063000700 1 --2- 172.21.15.1:0/36686245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff05c108610 0x7ff05c19c4b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:12.226 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:11.414+0000 7ff063000700 1 -- 172.21.15.1:0/36686245 >> 172.21.15.1:0/36686245 conn(0x7ff05c103c70 msgr2=0x7ff05c104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:12.226 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:11.414+0000 7ff063000700 1 -- 172.21.15.1:0/36686245 shutdown_connections 2024-04-26T01:39:12.226 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:11.415+0000 7ff063000700 1 -- 172.21.15.1:0/36686245 wait complete. 2024-04-26T01:39:12.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:12 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/36686245' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-26T01:39:12.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:12 smithi001 ceph-mon[27822]: mgrmap e9: x(active, since 25s) 2024-04-26T01:39:12.628 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:12 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:12.523+0000 7f8184586200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:39:13.014 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:12 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:12.642+0000 7f8184586200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:39:13.762 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:13 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/206764325' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T01:39:13.788 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:39:13.788 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-26T01:39:13.788 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T01:39:13.788 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T01:39:13.789 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T01:39:13.789 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:39:13.789 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.808+0000 7fcdd3600700 1 Processor -- start 2024-04-26T01:39:13.789 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.808+0000 7fcdd3600700 1 -- start start 2024-04-26T01:39:13.789 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.808+0000 7fcdd3600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdcc108610 0x7fcdcc108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:13.789 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.808+0000 7fcdd3600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc109000 con 0x7fcdcc108610 2024-04-26T01:39:13.790 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.809+0000 7fcdd0c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdcc108610 0x7fcdcc108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:13.790 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.809+0000 7fcdd0c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdcc108610 0x7fcdcc108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39374/0 (socket says 172.21.15.1:39374) 2024-04-26T01:39:13.790 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.809+0000 7fcdd0c00700 1 -- 172.21.15.1:0/1884834414 learned_addr learned my addr 172.21.15.1:0/1884834414 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:13.790 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.809+0000 7fcdd0c00700 1 -- 172.21.15.1:0/1884834414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdcc109810 con 0x7fcdcc108610 2024-04-26T01:39:13.790 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.809+0000 7fcdd0c00700 1 --2- 172.21.15.1:0/1884834414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdcc108610 0x7fcdcc108a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc0098a0 tx=0x7fcdbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984d9bb03d1cba4a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:13.790 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.810+0000 7fcdcbe00700 1 -- 172.21.15.1:0/1884834414 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdbc004160 con 0x7fcdcc108610 2024-04-26T01:39:13.791 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.810+0000 7fcdcbe00700 1 -- 172.21.15.1:0/1884834414 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdbc004300 con 0x7fcdcc108610 2024-04-26T01:39:13.791 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.810+0000 7fcdcbe00700 1 -- 172.21.15.1:0/1884834414 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdbc011a20 con 0x7fcdcc108610 2024-04-26T01:39:13.791 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.810+0000 7fcdd3600700 1 -- 172.21.15.1:0/1884834414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdcc108610 msgr2=0x7fcdcc108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:13.791 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.810+0000 7fcdd3600700 1 --2- 172.21.15.1:0/1884834414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdcc108610 0x7fcdcc108a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc0098a0 tx=0x7fcdbc009bb0 comp rx=0 tx=0).stop 2024-04-26T01:39:13.791 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.811+0000 7fcdd3600700 1 -- 172.21.15.1:0/1884834414 shutdown_connections 2024-04-26T01:39:13.791 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.811+0000 7fcdd3600700 1 --2- 172.21.15.1:0/1884834414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdcc108610 0x7fcdcc108a30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:13.791 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.811+0000 7fcdd3600700 1 -- 172.21.15.1:0/1884834414 >> 172.21.15.1:0/1884834414 conn(0x7fcdcc103c70 msgr2=0x7fcdcc106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:13.791 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.811+0000 7fcdd3600700 1 -- 172.21.15.1:0/1884834414 shutdown_connections 2024-04-26T01:39:13.791 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.811+0000 7fcdd3600700 1 -- 172.21.15.1:0/1884834414 wait complete. 2024-04-26T01:39:13.791 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.812+0000 7fcdd3600700 1 Processor -- start 2024-04-26T01:39:13.791 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.812+0000 7fcdd3600700 1 -- start start 2024-04-26T01:39:13.791 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.812+0000 7fcdd3600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdcc108610 0x7fcdcc07da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:13.791 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.812+0000 7fcdd3600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc10cbc0 con 0x7fcdcc108610 2024-04-26T01:39:13.791 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.813+0000 7fcdd0c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdcc108610 0x7fcdcc07da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:13.792 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.813+0000 7fcdd0c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdcc108610 0x7fcdcc07da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39378/0 (socket says 172.21.15.1:39378) 2024-04-26T01:39:13.792 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.813+0000 7fcdd0c00700 1 -- 172.21.15.1:0/206764325 learned_addr learned my addr 172.21.15.1:0/206764325 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:13.792 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.813+0000 7fcdd0c00700 1 -- 172.21.15.1:0/206764325 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdcc07dfa0 con 0x7fcdcc108610 2024-04-26T01:39:13.792 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.813+0000 7fcdd0c00700 1 --2- 172.21.15.1:0/206764325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdcc108610 0x7fcdcc07da60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc000c00 tx=0x7fcdbc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:13.792 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.814+0000 7fcdca000700 1 -- 172.21.15.1:0/206764325 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdbc003aa0 con 0x7fcdcc108610 2024-04-26T01:39:13.792 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.814+0000 7fcdca000700 1 -- 172.21.15.1:0/206764325 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdbc01b8f0 con 0x7fcdcc108610 2024-04-26T01:39:13.792 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.814+0000 7fcdca000700 1 -- 172.21.15.1:0/206764325 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdbc023890 con 0x7fcdcc108610 2024-04-26T01:39:13.793 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.814+0000 7fcdd3600700 1 -- 172.21.15.1:0/206764325 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdcc07c190 con 0x7fcdcc108610 2024-04-26T01:39:13.793 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.814+0000 7fcdd3600700 1 -- 172.21.15.1:0/206764325 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdcc07c6d0 con 0x7fcdcc108610 2024-04-26T01:39:13.793 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.815+0000 7fcdca000700 1 -- 172.21.15.1:0/206764325 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fcdbc01b440 con 0x7fcdcc108610 2024-04-26T01:39:13.793 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.815+0000 7fcdd3600700 1 -- 172.21.15.1:0/206764325 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdcc0640a0 con 0x7fcdcc108610 2024-04-26T01:39:13.793 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.816+0000 7fcdca000700 1 --2- 172.21.15.1:0/206764325 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7fcdb0039600 0x7fcdb003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:13.794 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.816+0000 7fcdca000700 1 -- 172.21.15.1:0/206764325 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcdbc04dea0 con 0x7fcdcc108610 2024-04-26T01:39:13.794 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.816+0000 7fcdc3e00700 1 -- 172.21.15.1:0/206764325 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7fcdb0039600 msgr2=0x7fcdb003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1870580982 2024-04-26T01:39:13.794 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.816+0000 7fcdc3e00700 1 --2- 172.21.15.1:0/206764325 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7fcdb0039600 0x7fcdb003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:39:13.794 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:12.821+0000 7fcdca000700 1 -- 172.21.15.1:0/206764325 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcdbc01f030 con 0x7fcdcc108610 2024-04-26T01:39:13.794 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:13.017+0000 7fcdc3e00700 1 -- 172.21.15.1:0/206764325 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7fcdb0039600 msgr2=0x7fcdb003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1870580982 2024-04-26T01:39:13.794 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:13.017+0000 7fcdc3e00700 1 --2- 172.21.15.1:0/206764325 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7fcdb0039600 0x7fcdb003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:39:13.795 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:13.111+0000 7fcdd3600700 1 -- 172.21.15.1:0/206764325 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fcdcc07c950 con 0x7fcdcc108610 2024-04-26T01:39:13.795 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:13.112+0000 7fcdca000700 1 -- 172.21.15.1:0/206764325 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fcdbc01bda0 con 0x7fcdcc108610 2024-04-26T01:39:13.795 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:13.115+0000 7fcdd3600700 1 -- 172.21.15.1:0/206764325 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7fcdb0039600 msgr2=0x7fcdb003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:39:13.795 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:13.115+0000 7fcdd3600700 1 --2- 172.21.15.1:0/206764325 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7fcdb0039600 0x7fcdb003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:13.795 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:13.115+0000 7fcdd3600700 1 -- 172.21.15.1:0/206764325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdcc108610 msgr2=0x7fcdcc07da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:13.796 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:13.115+0000 7fcdd3600700 1 --2- 172.21.15.1:0/206764325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdcc108610 0x7fcdcc07da60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc000c00 tx=0x7fcdbc011aa0 comp rx=0 tx=0).stop 2024-04-26T01:39:13.796 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:13.115+0000 7fcdd3600700 1 -- 172.21.15.1:0/206764325 shutdown_connections 2024-04-26T01:39:13.796 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:13.115+0000 7fcdd3600700 1 --2- 172.21.15.1:0/206764325 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7fcdb0039600 0x7fcdb003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:13.796 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:13.115+0000 7fcdd3600700 1 --2- 172.21.15.1:0/206764325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdcc108610 0x7fcdcc07da60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:13.796 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:13.115+0000 7fcdd3600700 1 -- 172.21.15.1:0/206764325 >> 172.21.15.1:0/206764325 conn(0x7fcdcc103c70 msgr2=0x7fcdcc104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:13.796 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:13.115+0000 7fcdd3600700 1 -- 172.21.15.1:0/206764325 shutdown_connections 2024-04-26T01:39:13.797 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:13.115+0000 7fcdd3600700 1 -- 172.21.15.1:0/206764325 wait complete. 2024-04-26T01:39:13.797 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for the mgr to restart... 2024-04-26T01:39:13.797 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr epoch 9... 2024-04-26T01:39:14.377 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:14 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:14.253+0000 7f8184586200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:39:14.711 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:14 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:14.493+0000 7f8184586200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:39:14.711 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:14 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:14.713+0000 7f8184586200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:39:15.378 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:15 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:15.107+0000 7f8184586200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:39:15.378 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:15 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:15.233+0000 7f8184586200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:39:15.751 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:15 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:15.491+0000 7f8184586200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:39:15.751 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:15 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:15.754+0000 7f8184586200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:39:16.828 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:16 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:16.573+0000 7f8184586200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:39:17.447 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:17 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:17.158+0000 7f8184586200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:39:17.447 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:17 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:17.291+0000 7f8184586200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:39:17.763 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:17 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:17.570+0000 7f8184586200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:39:18.628 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:18 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:18.239+0000 7f8184586200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:39:18.628 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:18 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:18.367+0000 7f8184586200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:39:19.019 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:18 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:18.704+0000 7f8184586200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:39:19.020 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:18 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:18.891+0000 7f8184586200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:39:19.020 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:19 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:19.022+0000 7f8184586200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:39:19.377 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:19 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:19.180+0000 7f8184586200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:39:19.877 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:19 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:19.556+0000 7f8184586200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:39:20.627 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:20 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:20.345+0000 7f8184586200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:39:21.128 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:39:20 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:39:20.642+0000 7f8184586200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:39:21.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:20 smithi001 ceph-mon[27822]: Active manager daemon x restarted 2024-04-26T01:39:21.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:20 smithi001 ceph-mon[27822]: Activating manager daemon x 2024-04-26T01:39:21.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:20 smithi001 ceph-mon[27822]: osdmap e3: 0 total, 0 up, 0 in 2024-04-26T01:39:21.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:20 smithi001 ceph-mon[27822]: mgrmap e10: x(active, starting, since 0.00934322s) 2024-04-26T01:39:21.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:39:21.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T01:39:21.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T01:39:21.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T01:39:21.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T01:39:21.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:20 smithi001 ceph-mon[27822]: Manager daemon x is now available 2024-04-26T01:39:22.095 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:21 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:39:22.095 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:21 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:39:22.095 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:21 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:39:22.096 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:21 smithi001 ceph-mon[27822]: [26/Apr/2024:01:39:21] ENGINE Bus STARTING 2024-04-26T01:39:22.096 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:21 smithi001 ceph-mon[27822]: [26/Apr/2024:01:39:21] ENGINE Serving on http://172.21.15.1:8765 2024-04-26T01:39:22.096 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:21 smithi001 ceph-mon[27822]: [26/Apr/2024:01:39:21] ENGINE Serving on https://172.21.15.1:7150 2024-04-26T01:39:22.096 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:21 smithi001 ceph-mon[27822]: [26/Apr/2024:01:39:21] ENGINE Bus STARTED 2024-04-26T01:39:22.096 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:21 smithi001 ceph-mon[27822]: mgrmap e11: x(active, since 1.01165s) 2024-04-26T01:39:22.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:39:22.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-26T01:39:22.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T01:39:22.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:39:22.715 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.436+0000 7f3711a00700 1 Processor -- start 2024-04-26T01:39:22.715 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.436+0000 7f3711a00700 1 -- start start 2024-04-26T01:39:22.715 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.436+0000 7f3711a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f370c106d20 0x7f370c107140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:22.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.436+0000 7f3711a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f370c063ce0 con 0x7f370c106d20 2024-04-26T01:39:22.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.436+0000 7f370be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f370c106d20 0x7f370c107140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:22.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.437+0000 7f370be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f370c106d20 0x7f370c107140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39384/0 (socket says 172.21.15.1:39384) 2024-04-26T01:39:22.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.437+0000 7f370be00700 1 -- 172.21.15.1:0/3519532472 learned_addr learned my addr 172.21.15.1:0/3519532472 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:22.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.437+0000 7f370be00700 1 -- 172.21.15.1:0/3519532472 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370c107680 con 0x7f370c106d20 2024-04-26T01:39:22.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.437+0000 7f370be00700 1 --2- 172.21.15.1:0/3519532472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f370c106d20 0x7f370c107140 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0098a0 tx=0x7f36fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aab8a1c9d443766 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:22.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.437+0000 7f370aa00700 1 -- 172.21.15.1:0/3519532472 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36fc004160 con 0x7f370c106d20 2024-04-26T01:39:22.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.438+0000 7f370aa00700 1 -- 172.21.15.1:0/3519532472 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36fc004300 con 0x7f370c106d20 2024-04-26T01:39:22.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.438+0000 7f370aa00700 1 -- 172.21.15.1:0/3519532472 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36fc011a20 con 0x7f370c106d20 2024-04-26T01:39:22.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.438+0000 7f3711a00700 1 -- 172.21.15.1:0/3519532472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f370c106d20 msgr2=0x7f370c107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:22.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.438+0000 7f3711a00700 1 --2- 172.21.15.1:0/3519532472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f370c106d20 0x7f370c107140 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0098a0 tx=0x7f36fc009bb0 comp rx=0 tx=0).stop 2024-04-26T01:39:22.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.438+0000 7f3711a00700 1 -- 172.21.15.1:0/3519532472 shutdown_connections 2024-04-26T01:39:22.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.438+0000 7f3711a00700 1 --2- 172.21.15.1:0/3519532472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f370c106d20 0x7f370c107140 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:22.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.438+0000 7f3711a00700 1 -- 172.21.15.1:0/3519532472 >> 172.21.15.1:0/3519532472 conn(0x7f370c102960 msgr2=0x7f370c104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:22.722 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.438+0000 7f3711a00700 1 -- 172.21.15.1:0/3519532472 shutdown_connections 2024-04-26T01:39:22.722 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.439+0000 7f3711a00700 1 -- 172.21.15.1:0/3519532472 wait complete. 2024-04-26T01:39:22.722 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.440+0000 7f3711a00700 1 Processor -- start 2024-04-26T01:39:22.722 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.440+0000 7f3711a00700 1 -- start start 2024-04-26T01:39:22.722 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.440+0000 7f3711a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f370c106d20 0x7f370c19e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:22.722 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.440+0000 7f3711a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1086f0 con 0x7f370c106d20 2024-04-26T01:39:22.722 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.440+0000 7f370be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f370c106d20 0x7f370c19e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:22.722 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.440+0000 7f370be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f370c106d20 0x7f370c19e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39390/0 (socket says 172.21.15.1:39390) 2024-04-26T01:39:22.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.440+0000 7f370be00700 1 -- 172.21.15.1:0/3085073258 learned_addr learned my addr 172.21.15.1:0/3085073258 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:22.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.441+0000 7f370be00700 1 -- 172.21.15.1:0/3085073258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370c19ea30 con 0x7f370c106d20 2024-04-26T01:39:22.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.441+0000 7f370be00700 1 --2- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f370c106d20 0x7f370c19e4f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f36fc004660 tx=0x7f36fc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:22.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.441+0000 7f3708c00700 1 -- 172.21.15.1:0/3085073258 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36fc003aa0 con 0x7f370c106d20 2024-04-26T01:39:22.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.441+0000 7f3708c00700 1 -- 172.21.15.1:0/3085073258 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36fc01b8f0 con 0x7f370c106d20 2024-04-26T01:39:22.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.441+0000 7f3711a00700 1 -- 172.21.15.1:0/3085073258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f370c19ecc0 con 0x7f370c106d20 2024-04-26T01:39:22.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.441+0000 7f3708c00700 1 -- 172.21.15.1:0/3085073258 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36fc023900 con 0x7f370c106d20 2024-04-26T01:39:22.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.442+0000 7f3711a00700 1 -- 172.21.15.1:0/3085073258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f370c1a19b0 con 0x7f370c106d20 2024-04-26T01:39:22.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.443+0000 7f3708c00700 1 -- 172.21.15.1:0/3085073258 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f36fc01baa0 con 0x7f370c106d20 2024-04-26T01:39:22.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.443+0000 7f3708c00700 1 --2- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f36f0039600 0x7f36f003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:22.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.443+0000 7f3708c00700 1 -- 172.21.15.1:0/3085073258 --> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f36f003c1d0 con 0x7f36f0039600 2024-04-26T01:39:22.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.443+0000 7f3708c00700 1 -- 172.21.15.1:0/3085073258 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f36fc04df60 con 0x7f370c106d20 2024-04-26T01:39:22.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.443+0000 7f370b400700 1 -- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f36f0039600 msgr2=0x7f36f003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1870580982 2024-04-26T01:39:22.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.443+0000 7f370b400700 1 --2- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f36f0039600 0x7f36f003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:39:22.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.644+0000 7f370b400700 1 -- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f36f0039600 msgr2=0x7f36f003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1870580982 2024-04-26T01:39:22.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:14.644+0000 7f370b400700 1 --2- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f36f0039600 0x7f36f003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:39:22.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:15.046+0000 7f370b400700 1 -- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f36f0039600 msgr2=0x7f36f003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1870580982 2024-04-26T01:39:22.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:15.046+0000 7f370b400700 1 --2- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f36f0039600 0x7f36f003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T01:39:22.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:15.847+0000 7f370b400700 1 -- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f36f0039600 msgr2=0x7f36f003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1870580982 2024-04-26T01:39:22.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:15.847+0000 7f370b400700 1 --2- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f36f0039600 0x7f36f003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T01:39:22.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:17.449+0000 7f370b400700 1 -- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f36f0039600 msgr2=0x7f36f003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1870580982 2024-04-26T01:39:22.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:17.449+0000 7f370b400700 1 --2- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f36f0039600 0x7f36f003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T01:39:22.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:20.651+0000 7f3708c00700 1 -- 172.21.15.1:0/3085073258 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f36fc02c3f0 con 0x7f370c106d20 2024-04-26T01:39:22.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:20.651+0000 7f3708c00700 1 -- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f36f0039600 msgr2=0x7f36f003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:39:22.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:20.651+0000 7f3708c00700 1 --2- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f36f0039600 0x7f36f003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:22.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:21.653+0000 7f3708c00700 1 -- 172.21.15.1:0/3085073258 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f36fc00ec80 con 0x7f370c106d20 2024-04-26T01:39:22.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:21.653+0000 7f3708c00700 1 --2- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f36f003cfe0 0x7f36f003f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:22.725 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:21.654+0000 7f3708c00700 1 -- 172.21.15.1:0/3085073258 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f36fc02de50 con 0x7f36f003cfe0 2024-04-26T01:39:22.725 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:21.654+0000 7f370b400700 1 --2- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f36f003cfe0 0x7f36f003f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:22.725 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:21.657+0000 7f370b400700 1 --2- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f36f003cfe0 0x7f36f003f3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f36f8003a10 tx=0x7f36f80092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:22.725 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:21.658+0000 7f3708c00700 1 -- 172.21.15.1:0/3085073258 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f36fc02de50 con 0x7f36f003cfe0 2024-04-26T01:39:22.725 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:21.665+0000 7f3702000700 1 -- 172.21.15.1:0/3085073258 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f36e4002600 con 0x7f36f003cfe0 2024-04-26T01:39:22.725 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:21.666+0000 7f3708c00700 1 -- 172.21.15.1:0/3085073258 <== mgr.14152 v2:172.21.15.1:6800/2140737121 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f36e4002600 con 0x7f36f003cfe0 2024-04-26T01:39:22.725 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:21.666+0000 7f3702000700 1 -- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f36f003cfe0 msgr2=0x7f36f003f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:22.725 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:21.666+0000 7f3702000700 1 --2- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f36f003cfe0 0x7f36f003f3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f36f8003a10 tx=0x7f36f80092b0 comp rx=0 tx=0).stop 2024-04-26T01:39:22.725 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:21.666+0000 7f3702000700 1 -- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f370c106d20 msgr2=0x7f370c19e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:22.725 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:21.666+0000 7f3702000700 1 --2- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f370c106d20 0x7f370c19e4f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f36fc004660 tx=0x7f36fc011aa0 comp rx=0 tx=0).stop 2024-04-26T01:39:22.726 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:21.666+0000 7f3702000700 1 -- 172.21.15.1:0/3085073258 shutdown_connections 2024-04-26T01:39:22.726 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:21.666+0000 7f3702000700 1 --2- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f36f003cfe0 0x7f36f003f3d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:22.726 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:21.666+0000 7f3702000700 1 --2- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f370c106d20 0x7f370c19e4f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:22.726 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:21.666+0000 7f3702000700 1 --2- 172.21.15.1:0/3085073258 >> [v2:172.21.15.1:6800/1870580982,v1:172.21.15.1:6801/1870580982] conn(0x7f36f0039600 0x7f36f003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:22.726 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:21.666+0000 7f3702000700 1 -- 172.21.15.1:0/3085073258 >> 172.21.15.1:0/3085073258 conn(0x7f370c102960 msgr2=0x7f370c195190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:22.726 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:21.667+0000 7f3702000700 1 -- 172.21.15.1:0/3085073258 shutdown_connections 2024-04-26T01:39:22.726 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:21.667+0000 7f3702000700 1 -- 172.21.15.1:0/3085073258 wait complete. 2024-04-26T01:39:22.727 INFO:teuthology.orchestra.run.smithi001.stdout:mgr epoch 9 is available 2024-04-26T01:39:22.727 INFO:teuthology.orchestra.run.smithi001.stdout:Generating a dashboard self-signed certificate... 2024-04-26T01:39:22.986 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:22 smithi001 ceph-mon[27822]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T01:39:22.986 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:22 smithi001 ceph-mon[27822]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T01:39:23.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:23 smithi001 ceph-mon[27822]: mgrmap e12: x(active, since 2s) 2024-04-26T01:39:23.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:23 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:24.959 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-26T01:39:24.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.649+0000 7f0df7200700 1 Processor -- start 2024-04-26T01:39:24.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.649+0000 7f0df7200700 1 -- start start 2024-04-26T01:39:24.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.649+0000 7f0df7200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df010a870 0x7f0df010ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:24.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.649+0000 7f0df7200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0df010b260 con 0x7f0df010a870 2024-04-26T01:39:24.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.650+0000 7f0defe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df010a870 0x7f0df010ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:24.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.650+0000 7f0defe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df010a870 0x7f0df010ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45424/0 (socket says 172.21.15.1:45424) 2024-04-26T01:39:24.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.650+0000 7f0defe00700 1 -- 172.21.15.1:0/2615517404 learned_addr learned my addr 172.21.15.1:0/2615517404 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:24.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.651+0000 7f0defe00700 1 -- 172.21.15.1:0/2615517404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df010ba20 con 0x7f0df010a870 2024-04-26T01:39:24.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.651+0000 7f0defe00700 1 --2- 172.21.15.1:0/2615517404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df010a870 0x7f0df010ac90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0de00098a0 tx=0x7f0de0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25962beeb5e4aa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:24.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.651+0000 7f0deea00700 1 -- 172.21.15.1:0/2615517404 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0de0004160 con 0x7f0df010a870 2024-04-26T01:39:24.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.651+0000 7f0deea00700 1 -- 172.21.15.1:0/2615517404 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0de0004300 con 0x7f0df010a870 2024-04-26T01:39:24.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.651+0000 7f0deea00700 1 -- 172.21.15.1:0/2615517404 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0de0011a20 con 0x7f0df010a870 2024-04-26T01:39:24.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.652+0000 7f0df7200700 1 -- 172.21.15.1:0/2615517404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df010a870 msgr2=0x7f0df010ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:24.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.652+0000 7f0df7200700 1 --2- 172.21.15.1:0/2615517404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df010a870 0x7f0df010ac90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0de00098a0 tx=0x7f0de0009bb0 comp rx=0 tx=0).stop 2024-04-26T01:39:24.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.652+0000 7f0df7200700 1 -- 172.21.15.1:0/2615517404 shutdown_connections 2024-04-26T01:39:24.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.652+0000 7f0df7200700 1 --2- 172.21.15.1:0/2615517404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df010a870 0x7f0df010ac90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:24.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.652+0000 7f0df7200700 1 -- 172.21.15.1:0/2615517404 >> 172.21.15.1:0/2615517404 conn(0x7f0df007dca0 msgr2=0x7f0df01083c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:24.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.652+0000 7f0df7200700 1 -- 172.21.15.1:0/2615517404 shutdown_connections 2024-04-26T01:39:24.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.652+0000 7f0df7200700 1 -- 172.21.15.1:0/2615517404 wait complete. 2024-04-26T01:39:24.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.653+0000 7f0df7200700 1 Processor -- start 2024-04-26T01:39:24.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.654+0000 7f0df7200700 1 -- start start 2024-04-26T01:39:24.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.654+0000 7f0df7200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df010a870 0x7f0df019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:24.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.654+0000 7f0df7200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0df010edd0 con 0x7f0df010a870 2024-04-26T01:39:24.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.654+0000 7f0defe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df010a870 0x7f0df019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:24.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.654+0000 7f0defe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df010a870 0x7f0df019e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45430/0 (socket says 172.21.15.1:45430) 2024-04-26T01:39:24.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.654+0000 7f0defe00700 1 -- 172.21.15.1:0/3053844281 learned_addr learned my addr 172.21.15.1:0/3053844281 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:24.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.655+0000 7f0defe00700 1 -- 172.21.15.1:0/3053844281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df019ec00 con 0x7f0df010a870 2024-04-26T01:39:24.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.655+0000 7f0defe00700 1 --2- 172.21.15.1:0/3053844281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df010a870 0x7f0df019e6c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0de0000c00 tx=0x7f0de0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:24.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.655+0000 7f0decc00700 1 -- 172.21.15.1:0/3053844281 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0de0003aa0 con 0x7f0df010a870 2024-04-26T01:39:24.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.655+0000 7f0decc00700 1 -- 172.21.15.1:0/3053844281 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0de001b8f0 con 0x7f0df010a870 2024-04-26T01:39:24.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.655+0000 7f0decc00700 1 -- 172.21.15.1:0/3053844281 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0de0023890 con 0x7f0df010a870 2024-04-26T01:39:24.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.655+0000 7f0df7200700 1 -- 172.21.15.1:0/3053844281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df019ee90 con 0x7f0df010a870 2024-04-26T01:39:24.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.656+0000 7f0df7200700 1 -- 172.21.15.1:0/3053844281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df01a1b80 con 0x7f0df010a870 2024-04-26T01:39:24.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.657+0000 7f0df7200700 1 -- 172.21.15.1:0/3053844281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df010ef50 con 0x7f0df010a870 2024-04-26T01:39:24.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.657+0000 7f0decc00700 1 -- 172.21.15.1:0/3053844281 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f0de001b440 con 0x7f0df010a870 2024-04-26T01:39:24.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.658+0000 7f0decc00700 1 --2- 172.21.15.1:0/3053844281 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0dd4039660 0x7f0dd403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:24.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.658+0000 7f0decc00700 1 -- 172.21.15.1:0/3053844281 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0de004dfe0 con 0x7f0df010a870 2024-04-26T01:39:24.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.658+0000 7f0def400700 1 --2- 172.21.15.1:0/3053844281 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0dd4039660 0x7f0dd403bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:24.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.659+0000 7f0def400700 1 --2- 172.21.15.1:0/3053844281 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0dd4039660 0x7f0dd403bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc006fd0 tx=0x7f0ddc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:24.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.664+0000 7f0decc00700 1 -- 172.21.15.1:0/3053844281 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0de001f030 con 0x7f0df010a870 2024-04-26T01:39:24.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:23.902+0000 7f0df7200700 1 -- 172.21.15.1:0/3053844281 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f0df0108220 con 0x7f0dd4039660 2024-04-26T01:39:24.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:24.010+0000 7f0decc00700 1 -- 172.21.15.1:0/3053844281 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f0df0108220 con 0x7f0dd4039660 2024-04-26T01:39:24.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:24.014+0000 7f0df7200700 1 -- 172.21.15.1:0/3053844281 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0dd4039660 msgr2=0x7f0dd403bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:24.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:24.014+0000 7f0df7200700 1 --2- 172.21.15.1:0/3053844281 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0dd4039660 0x7f0dd403bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc006fd0 tx=0x7f0ddc006e40 comp rx=0 tx=0).stop 2024-04-26T01:39:24.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:24.014+0000 7f0df7200700 1 -- 172.21.15.1:0/3053844281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df010a870 msgr2=0x7f0df019e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:24.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:24.014+0000 7f0df7200700 1 --2- 172.21.15.1:0/3053844281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df010a870 0x7f0df019e6c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0de0000c00 tx=0x7f0de0011aa0 comp rx=0 tx=0).stop 2024-04-26T01:39:24.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:24.014+0000 7f0df7200700 1 -- 172.21.15.1:0/3053844281 shutdown_connections 2024-04-26T01:39:24.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:24.014+0000 7f0df7200700 1 --2- 172.21.15.1:0/3053844281 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0dd4039660 0x7f0dd403bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:24.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:24.014+0000 7f0df7200700 1 --2- 172.21.15.1:0/3053844281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df010a870 0x7f0df019e6c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:24.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:24.014+0000 7f0df7200700 1 -- 172.21.15.1:0/3053844281 >> 172.21.15.1:0/3053844281 conn(0x7f0df007dca0 msgr2=0x7f0df0107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:24.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:24.014+0000 7f0df7200700 1 -- 172.21.15.1:0/3053844281 shutdown_connections 2024-04-26T01:39:24.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:24.014+0000 7f0df7200700 1 -- 172.21.15.1:0/3053844281 wait complete. 2024-04-26T01:39:24.964 INFO:teuthology.orchestra.run.smithi001.stdout:Creating initial admin user... 2024-04-26T01:39:25.276 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:25 smithi001 ceph-mon[27822]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:39:25.276 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:25 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:25.276 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:25 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:26.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:26 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:26.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:26 smithi001 ceph-mon[27822]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:39:26.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:26 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:26.994 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$kNPeHrb15wqejuAGu.1AT.85PInx2k4tDbVDvuHnai7D.xF3loIPu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714095566, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-26T01:39:26.995 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.577+0000 7f9ab6400700 1 Processor -- start 2024-04-26T01:39:26.995 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.577+0000 7f9ab6400700 1 -- start start 2024-04-26T01:39:26.996 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.577+0000 7f9ab6400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab010a870 0x7f9ab010ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:26.996 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.578+0000 7f9ab6400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab010b260 con 0x7f9ab010a870 2024-04-26T01:39:26.996 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.578+0000 7f9aaf400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab010a870 0x7f9ab010ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:26.997 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.578+0000 7f9aaf400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab010a870 0x7f9ab010ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45434/0 (socket says 172.21.15.1:45434) 2024-04-26T01:39:26.997 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.578+0000 7f9aaf400700 1 -- 172.21.15.1:0/1140061910 learned_addr learned my addr 172.21.15.1:0/1140061910 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:26.997 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.578+0000 7f9aaf400700 1 -- 172.21.15.1:0/1140061910 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab010ba20 con 0x7f9ab010a870 2024-04-26T01:39:26.997 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.579+0000 7f9aaf400700 1 --2- 172.21.15.1:0/1140061910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab010a870 0x7f9ab010ac90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00098a0 tx=0x7f9aa0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cee5d2236aa4c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:26.998 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.579+0000 7f9aae000700 1 -- 172.21.15.1:0/1140061910 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9aa0004160 con 0x7f9ab010a870 2024-04-26T01:39:26.998 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.579+0000 7f9aae000700 1 -- 172.21.15.1:0/1140061910 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aa0004300 con 0x7f9ab010a870 2024-04-26T01:39:26.998 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.579+0000 7f9aae000700 1 -- 172.21.15.1:0/1140061910 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9aa0011a20 con 0x7f9ab010a870 2024-04-26T01:39:26.998 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.580+0000 7f9ab6400700 1 -- 172.21.15.1:0/1140061910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab010a870 msgr2=0x7f9ab010ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:26.998 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.580+0000 7f9ab6400700 1 --2- 172.21.15.1:0/1140061910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab010a870 0x7f9ab010ac90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00098a0 tx=0x7f9aa0009bb0 comp rx=0 tx=0).stop 2024-04-26T01:39:26.999 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.580+0000 7f9ab6400700 1 -- 172.21.15.1:0/1140061910 shutdown_connections 2024-04-26T01:39:26.999 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.580+0000 7f9ab6400700 1 --2- 172.21.15.1:0/1140061910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab010a870 0x7f9ab010ac90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:26.999 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.580+0000 7f9ab6400700 1 -- 172.21.15.1:0/1140061910 >> 172.21.15.1:0/1140061910 conn(0x7f9ab007dca0 msgr2=0x7f9ab01083c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:26.999 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.580+0000 7f9ab6400700 1 -- 172.21.15.1:0/1140061910 shutdown_connections 2024-04-26T01:39:27.000 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.580+0000 7f9ab6400700 1 -- 172.21.15.1:0/1140061910 wait complete. 2024-04-26T01:39:27.000 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.581+0000 7f9ab6400700 1 Processor -- start 2024-04-26T01:39:27.000 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.581+0000 7f9ab6400700 1 -- start start 2024-04-26T01:39:27.000 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.582+0000 7f9ab6400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab010a870 0x7f9ab019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:27.000 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.582+0000 7f9ab6400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab010edd0 con 0x7f9ab010a870 2024-04-26T01:39:27.001 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.582+0000 7f9aaf400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab010a870 0x7f9ab019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:27.001 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.582+0000 7f9aaf400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab010a870 0x7f9ab019e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45448/0 (socket says 172.21.15.1:45448) 2024-04-26T01:39:27.001 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.582+0000 7f9aaf400700 1 -- 172.21.15.1:0/156632465 learned_addr learned my addr 172.21.15.1:0/156632465 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:27.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.582+0000 7f9aaf400700 1 -- 172.21.15.1:0/156632465 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab019ec40 con 0x7f9ab010a870 2024-04-26T01:39:27.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.583+0000 7f9aaf400700 1 --2- 172.21.15.1:0/156632465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab010a870 0x7f9ab019e700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0000c00 tx=0x7f9aa0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:27.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.583+0000 7f9aa7e00700 1 -- 172.21.15.1:0/156632465 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9aa0003aa0 con 0x7f9ab010a870 2024-04-26T01:39:27.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.583+0000 7f9aa7e00700 1 -- 172.21.15.1:0/156632465 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aa001b8f0 con 0x7f9ab010a870 2024-04-26T01:39:27.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.583+0000 7f9aa7e00700 1 -- 172.21.15.1:0/156632465 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9aa0023890 con 0x7f9ab010a870 2024-04-26T01:39:27.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.583+0000 7f9ab6400700 1 -- 172.21.15.1:0/156632465 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab019eed0 con 0x7f9ab010a870 2024-04-26T01:39:27.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.583+0000 7f9ab6400700 1 -- 172.21.15.1:0/156632465 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab01a1bc0 con 0x7f9ab010a870 2024-04-26T01:39:27.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.584+0000 7f9aa7e00700 1 -- 172.21.15.1:0/156632465 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f9aa001b440 con 0x7f9ab010a870 2024-04-26T01:39:27.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.584+0000 7f9ab6400700 1 -- 172.21.15.1:0/156632465 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab010ef50 con 0x7f9ab010a870 2024-04-26T01:39:27.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.585+0000 7f9aa7e00700 1 --2- 172.21.15.1:0/156632465 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f9a94039660 0x7f9a9403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:27.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.585+0000 7f9aa7e00700 1 -- 172.21.15.1:0/156632465 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9aa004dff0 con 0x7f9ab010a870 2024-04-26T01:39:27.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.585+0000 7f9aaea00700 1 --2- 172.21.15.1:0/156632465 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f9a94039660 0x7f9a9403bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:27.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.586+0000 7f9aaea00700 1 --2- 172.21.15.1:0/156632465 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f9a94039660 0x7f9a9403bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c006fd0 tx=0x7f9a9c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:27.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.591+0000 7f9aa7e00700 1 -- 172.21.15.1:0/156632465 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9aa001f030 con 0x7f9ab010a870 2024-04-26T01:39:27.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:25.861+0000 7f9ab6400700 1 -- 172.21.15.1:0/156632465 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f9ab01081d0 con 0x7f9a94039660 2024-04-26T01:39:27.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:26.121+0000 7f9aa7e00700 1 -- 172.21.15.1:0/156632465 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f9ab01081d0 con 0x7f9a94039660 2024-04-26T01:39:27.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:26.124+0000 7f9ab6400700 1 -- 172.21.15.1:0/156632465 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f9a94039660 msgr2=0x7f9a9403bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:27.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:26.124+0000 7f9ab6400700 1 --2- 172.21.15.1:0/156632465 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f9a94039660 0x7f9a9403bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c006fd0 tx=0x7f9a9c006e40 comp rx=0 tx=0).stop 2024-04-26T01:39:27.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:26.124+0000 7f9ab6400700 1 -- 172.21.15.1:0/156632465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab010a870 msgr2=0x7f9ab019e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:27.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:26.124+0000 7f9ab6400700 1 --2- 172.21.15.1:0/156632465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab010a870 0x7f9ab019e700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0000c00 tx=0x7f9aa0011aa0 comp rx=0 tx=0).stop 2024-04-26T01:39:27.006 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:26.125+0000 7f9ab6400700 1 -- 172.21.15.1:0/156632465 shutdown_connections 2024-04-26T01:39:27.007 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:26.125+0000 7f9ab6400700 1 --2- 172.21.15.1:0/156632465 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f9a94039660 0x7f9a9403bb20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:27.007 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:26.125+0000 7f9ab6400700 1 --2- 172.21.15.1:0/156632465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ab010a870 0x7f9ab019e700 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:27.007 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:26.125+0000 7f9ab6400700 1 -- 172.21.15.1:0/156632465 >> 172.21.15.1:0/156632465 conn(0x7f9ab007dca0 msgr2=0x7f9ab0107ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:27.007 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:26.125+0000 7f9ab6400700 1 -- 172.21.15.1:0/156632465 shutdown_connections 2024-04-26T01:39:27.007 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:26.125+0000 7f9ab6400700 1 -- 172.21.15.1:0/156632465 wait complete. 2024-04-26T01:39:27.007 INFO:teuthology.orchestra.run.smithi001.stdout:Fetching dashboard port number... 2024-04-26T01:39:28.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:28 smithi001 ceph-mon[27822]: mgrmap e13: x(active, since 6s) 2024-04-26T01:39:28.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:28 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/344890533' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-26T01:39:29.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 8443 2024-04-26T01:39:29.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.696+0000 7fc17a000700 1 Processor -- start 2024-04-26T01:39:29.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.696+0000 7fc17a000700 1 -- start start 2024-04-26T01:39:29.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.697+0000 7fc17a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc17410a870 0x7fc17410ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:29.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.697+0000 7fc17a000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc17410b260 con 0x7fc17410a870 2024-04-26T01:39:29.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.697+0000 7fc173400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc17410a870 0x7fc17410ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:29.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.697+0000 7fc173400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc17410a870 0x7fc17410ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45454/0 (socket says 172.21.15.1:45454) 2024-04-26T01:39:29.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.697+0000 7fc173400700 1 -- 172.21.15.1:0/3277111975 learned_addr learned my addr 172.21.15.1:0/3277111975 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:29.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.698+0000 7fc173400700 1 -- 172.21.15.1:0/3277111975 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17410ba20 con 0x7fc17410a870 2024-04-26T01:39:29.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.698+0000 7fc173400700 1 --2- 172.21.15.1:0/3277111975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc17410a870 0x7fc17410ac90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc1640098a0 tx=0x7fc164009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b28c6f15797c5e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:29.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.698+0000 7fc172000700 1 -- 172.21.15.1:0/3277111975 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc164004160 con 0x7fc17410a870 2024-04-26T01:39:29.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.699+0000 7fc172000700 1 -- 172.21.15.1:0/3277111975 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc164004300 con 0x7fc17410a870 2024-04-26T01:39:29.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.699+0000 7fc172000700 1 -- 172.21.15.1:0/3277111975 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc164011a20 con 0x7fc17410a870 2024-04-26T01:39:29.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.699+0000 7fc17a000700 1 -- 172.21.15.1:0/3277111975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc17410a870 msgr2=0x7fc17410ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:29.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.699+0000 7fc17a000700 1 --2- 172.21.15.1:0/3277111975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc17410a870 0x7fc17410ac90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc1640098a0 tx=0x7fc164009bb0 comp rx=0 tx=0).stop 2024-04-26T01:39:29.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.699+0000 7fc17a000700 1 -- 172.21.15.1:0/3277111975 shutdown_connections 2024-04-26T01:39:29.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.699+0000 7fc17a000700 1 --2- 172.21.15.1:0/3277111975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc17410a870 0x7fc17410ac90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:29.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.699+0000 7fc17a000700 1 -- 172.21.15.1:0/3277111975 >> 172.21.15.1:0/3277111975 conn(0x7fc17407dca0 msgr2=0x7fc1741083c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:29.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.700+0000 7fc17a000700 1 -- 172.21.15.1:0/3277111975 shutdown_connections 2024-04-26T01:39:29.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.700+0000 7fc17a000700 1 -- 172.21.15.1:0/3277111975 wait complete. 2024-04-26T01:39:29.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.701+0000 7fc17a000700 1 Processor -- start 2024-04-26T01:39:29.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.701+0000 7fc17a000700 1 -- start start 2024-04-26T01:39:29.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.702+0000 7fc17a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc17410a870 0x7fc1741a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:29.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.702+0000 7fc17a000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc17410edd0 con 0x7fc17410a870 2024-04-26T01:39:29.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.702+0000 7fc173400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc17410a870 0x7fc1741a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:29.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.702+0000 7fc173400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc17410a870 0x7fc1741a2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45460/0 (socket says 172.21.15.1:45460) 2024-04-26T01:39:29.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.702+0000 7fc173400700 1 -- 172.21.15.1:0/344890533 learned_addr learned my addr 172.21.15.1:0/344890533 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:29.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.702+0000 7fc173400700 1 -- 172.21.15.1:0/344890533 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1741a3020 con 0x7fc17410a870 2024-04-26T01:39:29.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.703+0000 7fc173400700 1 --2- 172.21.15.1:0/344890533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc17410a870 0x7fc1741a2ae0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc164000c00 tx=0x7fc164011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:29.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.703+0000 7fc16be00700 1 -- 172.21.15.1:0/344890533 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc164003aa0 con 0x7fc17410a870 2024-04-26T01:39:29.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.703+0000 7fc16be00700 1 -- 172.21.15.1:0/344890533 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc16401b8f0 con 0x7fc17410a870 2024-04-26T01:39:29.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.703+0000 7fc17a000700 1 -- 172.21.15.1:0/344890533 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1741a32b0 con 0x7fc17410a870 2024-04-26T01:39:29.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.703+0000 7fc16be00700 1 -- 172.21.15.1:0/344890533 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc164023890 con 0x7fc17410a870 2024-04-26T01:39:29.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.703+0000 7fc17a000700 1 -- 172.21.15.1:0/344890533 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1741a5fa0 con 0x7fc17410a870 2024-04-26T01:39:29.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.705+0000 7fc16be00700 1 -- 172.21.15.1:0/344890533 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7fc16402c3f0 con 0x7fc17410a870 2024-04-26T01:39:29.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.705+0000 7fc17a000700 1 -- 172.21.15.1:0/344890533 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc17410ef50 con 0x7fc17410a870 2024-04-26T01:39:29.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.705+0000 7fc16be00700 1 --2- 172.21.15.1:0/344890533 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fc158039700 0x7fc15803bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:29.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.705+0000 7fc16be00700 1 -- 172.21.15.1:0/344890533 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc16404e010 con 0x7fc17410a870 2024-04-26T01:39:29.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.705+0000 7fc172a00700 1 --2- 172.21.15.1:0/344890533 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fc158039700 0x7fc15803bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:29.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.706+0000 7fc172a00700 1 --2- 172.21.15.1:0/344890533 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fc158039700 0x7fc15803bbc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc160006fd0 tx=0x7fc160006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:29.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.711+0000 7fc16be00700 1 -- 172.21.15.1:0/344890533 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc16401f030 con 0x7fc17410a870 2024-04-26T01:39:29.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.971+0000 7fc17a000700 1 -- 172.21.15.1:0/344890533 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fc1741a6580 con 0x7fc17410a870 2024-04-26T01:39:29.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.972+0000 7fc16be00700 1 -- 172.21.15.1:0/344890533 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fc1640105e0 con 0x7fc17410a870 2024-04-26T01:39:29.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.975+0000 7fc17a000700 1 -- 172.21.15.1:0/344890533 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fc158039700 msgr2=0x7fc15803bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:29.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.975+0000 7fc17a000700 1 --2- 172.21.15.1:0/344890533 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fc158039700 0x7fc15803bbc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc160006fd0 tx=0x7fc160006e40 comp rx=0 tx=0).stop 2024-04-26T01:39:29.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.975+0000 7fc17a000700 1 -- 172.21.15.1:0/344890533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc17410a870 msgr2=0x7fc1741a2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:29.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.975+0000 7fc17a000700 1 --2- 172.21.15.1:0/344890533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc17410a870 0x7fc1741a2ae0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc164000c00 tx=0x7fc164011aa0 comp rx=0 tx=0).stop 2024-04-26T01:39:29.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.975+0000 7fc17a000700 1 -- 172.21.15.1:0/344890533 shutdown_connections 2024-04-26T01:39:29.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.975+0000 7fc17a000700 1 --2- 172.21.15.1:0/344890533 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fc158039700 0x7fc15803bbc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:29.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.975+0000 7fc17a000700 1 --2- 172.21.15.1:0/344890533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc17410a870 0x7fc1741a2ae0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:29.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.975+0000 7fc17a000700 1 -- 172.21.15.1:0/344890533 >> 172.21.15.1:0/344890533 conn(0x7fc17407dca0 msgr2=0x7fc174107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:29.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.975+0000 7fc17a000700 1 -- 172.21.15.1:0/344890533 shutdown_connections 2024-04-26T01:39:29.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:27.976+0000 7fc17a000700 1 -- 172.21.15.1:0/344890533 wait complete. 2024-04-26T01:39:29.524 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T01:39:29.525 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-04-26T01:39:29.556 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T01:39:29.556 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-04-26T01:39:29.556 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-04-26T01:39:29.556 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-26T01:39:29.564 INFO:teuthology.orchestra.run.smithi001.stdout:Ceph Dashboard is now available at: 2024-04-26T01:39:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:39:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: URL: https://smithi001.front.sepia.ceph.com:8443/ 2024-04-26T01:39:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: User: admin 2024-04-26T01:39:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: Password: 1e5lgtii6e 2024-04-26T01:39:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:39:29.565 INFO:teuthology.orchestra.run.smithi001.stdout:Saving cluster configuration to /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config directory 2024-04-26T01:39:29.567 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling autotune for osd_memory_target 2024-04-26T01:39:31.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.170+0000 7f23e3e00700 1 Processor -- start 2024-04-26T01:39:31.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.170+0000 7f23e3e00700 1 -- start start 2024-04-26T01:39:31.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.171+0000 7f23e3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23dc107540 0x7f23dc107960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:31.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.171+0000 7f23e3e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc107f30 con 0x7f23dc107540 2024-04-26T01:39:31.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.171+0000 7f23e1400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23dc107540 0x7f23dc107960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:31.097 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.172+0000 7f23e1400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23dc107540 0x7f23dc107960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45462/0 (socket says 172.21.15.1:45462) 2024-04-26T01:39:31.098 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.172+0000 7f23e1400700 1 -- 172.21.15.1:0/115846729 learned_addr learned my addr 172.21.15.1:0/115846729 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:31.098 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.172+0000 7f23e1400700 1 -- 172.21.15.1:0/115846729 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc108740 con 0x7f23dc107540 2024-04-26T01:39:31.098 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.172+0000 7f23e1400700 1 --2- 172.21.15.1:0/115846729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23dc107540 0x7f23dc107960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f23d00098a0 tx=0x7f23d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06dbe446818aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:31.098 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.173+0000 7f23dbe00700 1 -- 172.21.15.1:0/115846729 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23d0004160 con 0x7f23dc107540 2024-04-26T01:39:31.098 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.173+0000 7f23dbe00700 1 -- 172.21.15.1:0/115846729 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23d0004300 con 0x7f23dc107540 2024-04-26T01:39:31.098 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.173+0000 7f23dbe00700 1 -- 172.21.15.1:0/115846729 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23d0011a20 con 0x7f23dc107540 2024-04-26T01:39:31.098 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.173+0000 7f23e3e00700 1 -- 172.21.15.1:0/115846729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23dc107540 msgr2=0x7f23dc107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:31.098 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.173+0000 7f23e3e00700 1 --2- 172.21.15.1:0/115846729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23dc107540 0x7f23dc107960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f23d00098a0 tx=0x7f23d0009bb0 comp rx=0 tx=0).stop 2024-04-26T01:39:31.100 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.174+0000 7f23e3e00700 1 -- 172.21.15.1:0/115846729 shutdown_connections 2024-04-26T01:39:31.100 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.174+0000 7f23e3e00700 1 --2- 172.21.15.1:0/115846729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23dc107540 0x7f23dc107960 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:31.100 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.174+0000 7f23e3e00700 1 -- 172.21.15.1:0/115846729 >> 172.21.15.1:0/115846729 conn(0x7f23dc102b80 msgr2=0x7f23dc104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:31.100 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.174+0000 7f23e3e00700 1 -- 172.21.15.1:0/115846729 shutdown_connections 2024-04-26T01:39:31.100 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.174+0000 7f23e3e00700 1 -- 172.21.15.1:0/115846729 wait complete. 2024-04-26T01:39:31.100 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.175+0000 7f23e3e00700 1 Processor -- start 2024-04-26T01:39:31.100 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.175+0000 7f23e3e00700 1 -- start start 2024-04-26T01:39:31.100 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.175+0000 7f23e3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23dc107540 0x7f23dc19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:31.100 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.175+0000 7f23e3e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc10baf0 con 0x7f23dc107540 2024-04-26T01:39:31.100 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.176+0000 7f23e1400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23dc107540 0x7f23dc19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:31.100 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.176+0000 7f23e1400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23dc107540 0x7f23dc19e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45472/0 (socket says 172.21.15.1:45472) 2024-04-26T01:39:31.100 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.176+0000 7f23e1400700 1 -- 172.21.15.1:0/3082138290 learned_addr learned my addr 172.21.15.1:0/3082138290 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:31.101 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.176+0000 7f23e1400700 1 -- 172.21.15.1:0/3082138290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc19ecd0 con 0x7f23dc107540 2024-04-26T01:39:31.102 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.176+0000 7f23e1400700 1 --2- 172.21.15.1:0/3082138290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23dc107540 0x7f23dc19e790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f23d0000c00 tx=0x7f23d0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:31.103 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.177+0000 7f23da000700 1 -- 172.21.15.1:0/3082138290 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23d0003aa0 con 0x7f23dc107540 2024-04-26T01:39:31.103 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.177+0000 7f23da000700 1 -- 172.21.15.1:0/3082138290 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23d001b8f0 con 0x7f23dc107540 2024-04-26T01:39:31.103 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.177+0000 7f23e3e00700 1 -- 172.21.15.1:0/3082138290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23dc19ef60 con 0x7f23dc107540 2024-04-26T01:39:31.103 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.177+0000 7f23da000700 1 -- 172.21.15.1:0/3082138290 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23d0023890 con 0x7f23dc107540 2024-04-26T01:39:31.103 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.177+0000 7f23e3e00700 1 -- 172.21.15.1:0/3082138290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23dc1a1c50 con 0x7f23dc107540 2024-04-26T01:39:31.103 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.178+0000 7f23da000700 1 -- 172.21.15.1:0/3082138290 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f23d001b440 con 0x7f23dc107540 2024-04-26T01:39:31.103 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.178+0000 7f23e3e00700 1 -- 172.21.15.1:0/3082138290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23dc10bc70 con 0x7f23dc107540 2024-04-26T01:39:31.103 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.179+0000 7f23da000700 1 --2- 172.21.15.1:0/3082138290 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f23c4039700 0x7f23c403bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:31.103 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.179+0000 7f23da000700 1 -- 172.21.15.1:0/3082138290 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f23d004e040 con 0x7f23dc107540 2024-04-26T01:39:31.103 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.179+0000 7f23e0a00700 1 --2- 172.21.15.1:0/3082138290 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f23c4039700 0x7f23c403bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:31.103 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.180+0000 7f23e0a00700 1 --2- 172.21.15.1:0/3082138290 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f23c4039700 0x7f23c403bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f23c8006fd0 tx=0x7f23c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:31.103 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.184+0000 7f23da000700 1 -- 172.21.15.1:0/3082138290 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23d001f030 con 0x7f23dc107540 2024-04-26T01:39:31.103 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.417+0000 7f23e3e00700 1 -- 172.21.15.1:0/3082138290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f23dc02e260 con 0x7f23dc107540 2024-04-26T01:39:31.103 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.418+0000 7f23da000700 1 -- 172.21.15.1:0/3082138290 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f23d00189e0 con 0x7f23dc107540 2024-04-26T01:39:31.104 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.422+0000 7f23e3e00700 1 -- 172.21.15.1:0/3082138290 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f23c4039700 msgr2=0x7f23c403bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:31.104 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.422+0000 7f23e3e00700 1 --2- 172.21.15.1:0/3082138290 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f23c4039700 0x7f23c403bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f23c8006fd0 tx=0x7f23c8006e40 comp rx=0 tx=0).stop 2024-04-26T01:39:31.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.422+0000 7f23e3e00700 1 -- 172.21.15.1:0/3082138290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23dc107540 msgr2=0x7f23dc19e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:31.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.422+0000 7f23e3e00700 1 --2- 172.21.15.1:0/3082138290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23dc107540 0x7f23dc19e790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f23d0000c00 tx=0x7f23d0011aa0 comp rx=0 tx=0).stop 2024-04-26T01:39:31.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.422+0000 7f23e3e00700 1 -- 172.21.15.1:0/3082138290 shutdown_connections 2024-04-26T01:39:31.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.422+0000 7f23e3e00700 1 --2- 172.21.15.1:0/3082138290 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f23c4039700 0x7f23c403bbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:31.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.422+0000 7f23e3e00700 1 --2- 172.21.15.1:0/3082138290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23dc107540 0x7f23dc19e790 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:31.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.422+0000 7f23e3e00700 1 -- 172.21.15.1:0/3082138290 >> 172.21.15.1:0/3082138290 conn(0x7f23dc102b80 msgr2=0x7f23dc103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:31.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.422+0000 7f23e3e00700 1 -- 172.21.15.1:0/3082138290 shutdown_connections 2024-04-26T01:39:31.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:30.422+0000 7f23e3e00700 1 -- 172.21.15.1:0/3082138290 wait complete. 2024-04-26T01:39:32.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.754+0000 7fd7fba00700 1 Processor -- start 2024-04-26T01:39:32.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.754+0000 7fd7fba00700 1 -- start start 2024-04-26T01:39:32.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.755+0000 7fd7fba00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7f410a830 0x7fd7f410ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:32.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.755+0000 7fd7fba00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f410b220 con 0x7fd7f410a830 2024-04-26T01:39:32.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.755+0000 7fd7f9000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7f410a830 0x7fd7f410ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:32.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.755+0000 7fd7f9000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7f410a830 0x7fd7f410ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45484/0 (socket says 172.21.15.1:45484) 2024-04-26T01:39:32.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.756+0000 7fd7f9000700 1 -- 172.21.15.1:0/3053921733 learned_addr learned my addr 172.21.15.1:0/3053921733 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:32.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.756+0000 7fd7f9000700 1 -- 172.21.15.1:0/3053921733 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f410ba00 con 0x7fd7f410a830 2024-04-26T01:39:32.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.756+0000 7fd7f9000700 1 --2- 172.21.15.1:0/3053921733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7f410a830 0x7fd7f410ac50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80098a0 tx=0x7fd7e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd50c841100ddc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:32.857 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.757+0000 7fd7f3400700 1 -- 172.21.15.1:0/3053921733 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7e8004160 con 0x7fd7f410a830 2024-04-26T01:39:32.858 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.757+0000 7fd7f3400700 1 -- 172.21.15.1:0/3053921733 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7e8004300 con 0x7fd7f410a830 2024-04-26T01:39:32.858 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.757+0000 7fd7f3400700 1 -- 172.21.15.1:0/3053921733 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7e8011a20 con 0x7fd7f410a830 2024-04-26T01:39:32.858 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.757+0000 7fd7fba00700 1 -- 172.21.15.1:0/3053921733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7f410a830 msgr2=0x7fd7f410ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:32.858 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.757+0000 7fd7fba00700 1 --2- 172.21.15.1:0/3053921733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7f410a830 0x7fd7f410ac50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80098a0 tx=0x7fd7e8009bb0 comp rx=0 tx=0).stop 2024-04-26T01:39:32.858 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.757+0000 7fd7fba00700 1 -- 172.21.15.1:0/3053921733 shutdown_connections 2024-04-26T01:39:32.858 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.757+0000 7fd7fba00700 1 --2- 172.21.15.1:0/3053921733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7f410a830 0x7fd7f410ac50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:32.858 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.757+0000 7fd7fba00700 1 -- 172.21.15.1:0/3053921733 >> 172.21.15.1:0/3053921733 conn(0x7fd7f407dbc0 msgr2=0x7fd7f407dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:32.858 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.758+0000 7fd7fba00700 1 -- 172.21.15.1:0/3053921733 shutdown_connections 2024-04-26T01:39:32.858 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.758+0000 7fd7fba00700 1 -- 172.21.15.1:0/3053921733 wait complete. 2024-04-26T01:39:32.858 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.759+0000 7fd7fba00700 1 Processor -- start 2024-04-26T01:39:32.858 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.759+0000 7fd7fba00700 1 -- start start 2024-04-26T01:39:32.858 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.759+0000 7fd7fba00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7f410a830 0x7fd7f419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:32.858 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.759+0000 7fd7fba00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f410edb0 con 0x7fd7f410a830 2024-04-26T01:39:32.859 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.759+0000 7fd7f9000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7f410a830 0x7fd7f419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:32.859 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.760+0000 7fd7f9000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7f410a830 0x7fd7f419e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45488/0 (socket says 172.21.15.1:45488) 2024-04-26T01:39:32.859 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.760+0000 7fd7f9000700 1 -- 172.21.15.1:0/502561855 learned_addr learned my addr 172.21.15.1:0/502561855 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:32.859 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.760+0000 7fd7f9000700 1 -- 172.21.15.1:0/502561855 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f419ec40 con 0x7fd7f410a830 2024-04-26T01:39:32.859 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.760+0000 7fd7f9000700 1 --2- 172.21.15.1:0/502561855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7f410a830 0x7fd7f419e700 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80119f0 tx=0x7fd7e8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:32.859 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.760+0000 7fd7f1600700 1 -- 172.21.15.1:0/502561855 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7e80039e0 con 0x7fd7f410a830 2024-04-26T01:39:32.859 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.761+0000 7fd7f1600700 1 -- 172.21.15.1:0/502561855 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7e801b8f0 con 0x7fd7f410a830 2024-04-26T01:39:32.859 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.761+0000 7fd7f1600700 1 -- 172.21.15.1:0/502561855 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7e80237a0 con 0x7fd7f410a830 2024-04-26T01:39:32.859 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.761+0000 7fd7fba00700 1 -- 172.21.15.1:0/502561855 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f419eed0 con 0x7fd7f410a830 2024-04-26T01:39:32.859 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.761+0000 7fd7fba00700 1 -- 172.21.15.1:0/502561855 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f41a1bc0 con 0x7fd7f410a830 2024-04-26T01:39:32.859 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.762+0000 7fd7fba00700 1 -- 172.21.15.1:0/502561855 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f410ef30 con 0x7fd7f410a830 2024-04-26T01:39:32.859 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.762+0000 7fd7f1600700 1 -- 172.21.15.1:0/502561855 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7fd7e801b440 con 0x7fd7f410a830 2024-04-26T01:39:32.859 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.762+0000 7fd7f1600700 1 --2- 172.21.15.1:0/502561855 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fd7dc0396b0 0x7fd7dc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:32.859 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.762+0000 7fd7f1600700 1 -- 172.21.15.1:0/502561855 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd7e804e150 con 0x7fd7f410a830 2024-04-26T01:39:32.860 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.763+0000 7fd7f3e00700 1 --2- 172.21.15.1:0/502561855 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fd7dc0396b0 0x7fd7dc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:32.860 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.763+0000 7fd7f3e00700 1 --2- 172.21.15.1:0/502561855 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fd7dc0396b0 0x7fd7dc03bb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0006fd0 tx=0x7fd7e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:32.860 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:31.768+0000 7fd7f1600700 1 -- 172.21.15.1:0/502561855 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd7e801f030 con 0x7fd7f410a830 2024-04-26T01:39:32.860 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:32.127+0000 7fd7fba00700 1 -- 172.21.15.1:0/502561855 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fd7f41a2170 con 0x7fd7f410a830 2024-04-26T01:39:32.860 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:32.133+0000 7fd7f1600700 1 -- 172.21.15.1:0/502561855 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7fd7e8018b60 con 0x7fd7f410a830 2024-04-26T01:39:32.860 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:32.137+0000 7fd7fba00700 1 -- 172.21.15.1:0/502561855 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fd7dc0396b0 msgr2=0x7fd7dc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:32.860 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:32.137+0000 7fd7fba00700 1 --2- 172.21.15.1:0/502561855 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fd7dc0396b0 0x7fd7dc03bb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0006fd0 tx=0x7fd7e0006e40 comp rx=0 tx=0).stop 2024-04-26T01:39:32.860 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:32.137+0000 7fd7fba00700 1 -- 172.21.15.1:0/502561855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7f410a830 msgr2=0x7fd7f419e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:32.860 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:32.137+0000 7fd7fba00700 1 --2- 172.21.15.1:0/502561855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7f410a830 0x7fd7f419e700 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80119f0 tx=0x7fd7e8011ad0 comp rx=0 tx=0).stop 2024-04-26T01:39:32.860 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:32.137+0000 7fd7fba00700 1 -- 172.21.15.1:0/502561855 shutdown_connections 2024-04-26T01:39:32.860 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:32.137+0000 7fd7fba00700 1 --2- 172.21.15.1:0/502561855 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fd7dc0396b0 0x7fd7dc03bb70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:32.860 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:32.137+0000 7fd7fba00700 1 --2- 172.21.15.1:0/502561855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7f410a830 0x7fd7f419e700 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:32.860 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:32.137+0000 7fd7fba00700 1 -- 172.21.15.1:0/502561855 >> 172.21.15.1:0/502561855 conn(0x7fd7f407dbc0 msgr2=0x7fd7f4108510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:32.861 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:32.137+0000 7fd7fba00700 1 -- 172.21.15.1:0/502561855 shutdown_connections 2024-04-26T01:39:32.861 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-04-26T01:39:32.137+0000 7fd7fba00700 1 -- 172.21.15.1:0/502561855 wait complete. 2024-04-26T01:39:32.861 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-26T01:39:32.861 INFO:teuthology.orchestra.run.smithi001.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-26T01:39:32.861 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:39:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-26T01:39:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:39:32.862 INFO:teuthology.orchestra.run.smithi001.stdout:Or, if you are only running a single cluster on this host: 2024-04-26T01:39:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:39:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-26T01:39:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:39:32.862 INFO:teuthology.orchestra.run.smithi001.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-26T01:39:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:39:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: ceph telemetry on 2024-04-26T01:39:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:39:32.862 INFO:teuthology.orchestra.run.smithi001.stdout:For more information see: 2024-04-26T01:39:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:39:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-26T01:39:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:39:32.863 INFO:teuthology.orchestra.run.smithi001.stdout:Bootstrap complete. 2024-04-26T01:39:32.952 INFO:tasks.cephadm:Fetching config... 2024-04-26T01:39:32.952 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-04-26T01:39:32.952 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-26T01:39:33.003 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-26T01:39:33.004 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-04-26T01:39:33.004 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-26T01:39:33.050 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-26T01:39:33.050 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-04-26T01:39:33.050 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-26T01:39:33.159 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-26T01:39:33.159 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-04-26T01:39:33.159 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-26T01:39:33.255 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-26T01:39:33.255 DEBUG:teuthology.orchestra.run.smithi001:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCa5A5IQvnMrpVirYsaCjt9dyCZTEK2sAxmAtBUYdoTzM7TcEiQw1v6QEgjQLogT3f5u81yYpmsMu8O0XcInh5ov31EDYpE+gHS/zmpoSih+ivMuETd7bvNeC9oX2tdrGugXHX4IlXzN2HjsOg4MWLw5fVebLO2CVgIbgi+7Q3vaucSqrKhDplHKMo173Qu9nathtbNj29CN8/G8KVsI5lNw053CsvcDET43mafxe5gAcKiIyRu3B7Bq9mTfyFvKz2aTCIZCWTTF7r7xj0GwTR0VV7+1EPDV7y0pOyDys42s6yNV8MiVuufGy4FjFUHHR2RXUyh4Y/mlBTEvRXBAXfHnkGoJdS6RzjhViKeQwGAxt+vSqOLeXXn9AADRXIc3FLo+LfcqED9r5K8pMke6HPVeH8LuoEbcUw6XVl6hHpt3Ah6Zi90Rj1BgmdsiG55fOcXwIiLe00b+9bdtq12clO+ajfAdqDT7NYuRzJpFjmgGoGBHBvMlyiOvaC49fHBNNc= ceph-77cfcc80-036d-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T01:39:33.377 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:33 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/502561855' entity='client.admin' 2024-04-26T01:39:33.388 INFO:teuthology.orchestra.run.smithi001.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCa5A5IQvnMrpVirYsaCjt9dyCZTEK2sAxmAtBUYdoTzM7TcEiQw1v6QEgjQLogT3f5u81yYpmsMu8O0XcInh5ov31EDYpE+gHS/zmpoSih+ivMuETd7bvNeC9oX2tdrGugXHX4IlXzN2HjsOg4MWLw5fVebLO2CVgIbgi+7Q3vaucSqrKhDplHKMo173Qu9nathtbNj29CN8/G8KVsI5lNw053CsvcDET43mafxe5gAcKiIyRu3B7Bq9mTfyFvKz2aTCIZCWTTF7r7xj0GwTR0VV7+1EPDV7y0pOyDys42s6yNV8MiVuufGy4FjFUHHR2RXUyh4Y/mlBTEvRXBAXfHnkGoJdS6RzjhViKeQwGAxt+vSqOLeXXn9AADRXIc3FLo+LfcqED9r5K8pMke6HPVeH8LuoEbcUw6XVl6hHpt3Ah6Zi90Rj1BgmdsiG55fOcXwIiLe00b+9bdtq12clO+ajfAdqDT7NYuRzJpFjmgGoGBHBvMlyiOvaC49fHBNNc= ceph-77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:39:33.433 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCa5A5IQvnMrpVirYsaCjt9dyCZTEK2sAxmAtBUYdoTzM7TcEiQw1v6QEgjQLogT3f5u81yYpmsMu8O0XcInh5ov31EDYpE+gHS/zmpoSih+ivMuETd7bvNeC9oX2tdrGugXHX4IlXzN2HjsOg4MWLw5fVebLO2CVgIbgi+7Q3vaucSqrKhDplHKMo173Qu9nathtbNj29CN8/G8KVsI5lNw053CsvcDET43mafxe5gAcKiIyRu3B7Bq9mTfyFvKz2aTCIZCWTTF7r7xj0GwTR0VV7+1EPDV7y0pOyDys42s6yNV8MiVuufGy4FjFUHHR2RXUyh4Y/mlBTEvRXBAXfHnkGoJdS6RzjhViKeQwGAxt+vSqOLeXXn9AADRXIc3FLo+LfcqED9r5K8pMke6HPVeH8LuoEbcUw6XVl6hHpt3Ah6Zi90Rj1BgmdsiG55fOcXwIiLe00b+9bdtq12clO+ajfAdqDT7NYuRzJpFjmgGoGBHBvMlyiOvaC49fHBNNc= ceph-77cfcc80-036d-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T01:39:33.472 INFO:teuthology.orchestra.run.smithi097.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCa5A5IQvnMrpVirYsaCjt9dyCZTEK2sAxmAtBUYdoTzM7TcEiQw1v6QEgjQLogT3f5u81yYpmsMu8O0XcInh5ov31EDYpE+gHS/zmpoSih+ivMuETd7bvNeC9oX2tdrGugXHX4IlXzN2HjsOg4MWLw5fVebLO2CVgIbgi+7Q3vaucSqrKhDplHKMo173Qu9nathtbNj29CN8/G8KVsI5lNw053CsvcDET43mafxe5gAcKiIyRu3B7Bq9mTfyFvKz2aTCIZCWTTF7r7xj0GwTR0VV7+1EPDV7y0pOyDys42s6yNV8MiVuufGy4FjFUHHR2RXUyh4Y/mlBTEvRXBAXfHnkGoJdS6RzjhViKeQwGAxt+vSqOLeXXn9AADRXIc3FLo+LfcqED9r5K8pMke6HPVeH8LuoEbcUw6XVl6hHpt3Ah6Zi90Rj1BgmdsiG55fOcXwIiLe00b+9bdtq12clO+ajfAdqDT7NYuRzJpFjmgGoGBHBvMlyiOvaC49fHBNNc= ceph-77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:39:33.484 DEBUG:teuthology.orchestra.run.smithi173:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCa5A5IQvnMrpVirYsaCjt9dyCZTEK2sAxmAtBUYdoTzM7TcEiQw1v6QEgjQLogT3f5u81yYpmsMu8O0XcInh5ov31EDYpE+gHS/zmpoSih+ivMuETd7bvNeC9oX2tdrGugXHX4IlXzN2HjsOg4MWLw5fVebLO2CVgIbgi+7Q3vaucSqrKhDplHKMo173Qu9nathtbNj29CN8/G8KVsI5lNw053CsvcDET43mafxe5gAcKiIyRu3B7Bq9mTfyFvKz2aTCIZCWTTF7r7xj0GwTR0VV7+1EPDV7y0pOyDys42s6yNV8MiVuufGy4FjFUHHR2RXUyh4Y/mlBTEvRXBAXfHnkGoJdS6RzjhViKeQwGAxt+vSqOLeXXn9AADRXIc3FLo+LfcqED9r5K8pMke6HPVeH8LuoEbcUw6XVl6hHpt3Ah6Zi90Rj1BgmdsiG55fOcXwIiLe00b+9bdtq12clO+ajfAdqDT7NYuRzJpFjmgGoGBHBvMlyiOvaC49fHBNNc= ceph-77cfcc80-036d-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T01:39:33.523 INFO:teuthology.orchestra.run.smithi173.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCa5A5IQvnMrpVirYsaCjt9dyCZTEK2sAxmAtBUYdoTzM7TcEiQw1v6QEgjQLogT3f5u81yYpmsMu8O0XcInh5ov31EDYpE+gHS/zmpoSih+ivMuETd7bvNeC9oX2tdrGugXHX4IlXzN2HjsOg4MWLw5fVebLO2CVgIbgi+7Q3vaucSqrKhDplHKMo173Qu9nathtbNj29CN8/G8KVsI5lNw053CsvcDET43mafxe5gAcKiIyRu3B7Bq9mTfyFvKz2aTCIZCWTTF7r7xj0GwTR0VV7+1EPDV7y0pOyDys42s6yNV8MiVuufGy4FjFUHHR2RXUyh4Y/mlBTEvRXBAXfHnkGoJdS6RzjhViKeQwGAxt+vSqOLeXXn9AADRXIc3FLo+LfcqED9r5K8pMke6HPVeH8LuoEbcUw6XVl6hHpt3Ah6Zi90Rj1BgmdsiG55fOcXwIiLe00b+9bdtq12clO+ajfAdqDT7NYuRzJpFjmgGoGBHBvMlyiOvaC49fHBNNc= ceph-77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:39:33.535 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-26T01:39:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:39:36.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.084+0000 7fec55a00700 1 -- 172.21.15.1:0/819482424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec501046f0 msgr2=0x7fec50104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:36.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.084+0000 7fec55a00700 1 --2- 172.21.15.1:0/819482424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec501046f0 0x7fec50104b10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fec40009910 tx=0x7fec40009c20 comp rx=0 tx=0).stop 2024-04-26T01:39:36.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.084+0000 7fec55a00700 1 -- 172.21.15.1:0/819482424 shutdown_connections 2024-04-26T01:39:36.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.084+0000 7fec55a00700 1 --2- 172.21.15.1:0/819482424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec501046f0 0x7fec50104b10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:36.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.084+0000 7fec55a00700 1 -- 172.21.15.1:0/819482424 >> 172.21.15.1:0/819482424 conn(0x7fec500ffc70 msgr2=0x7fec501020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:36.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.084+0000 7fec55a00700 1 -- 172.21.15.1:0/819482424 shutdown_connections 2024-04-26T01:39:36.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.084+0000 7fec55a00700 1 -- 172.21.15.1:0/819482424 wait complete. 2024-04-26T01:39:36.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.085+0000 7fec55a00700 1 Processor -- start 2024-04-26T01:39:36.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.085+0000 7fec55a00700 1 -- start start 2024-04-26T01:39:36.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.086+0000 7fec55a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec501046f0 0x7fec50199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:36.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.086+0000 7fec55a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fec5010a630 con 0x7fec501046f0 2024-04-26T01:39:36.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.086+0000 7fec4ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec501046f0 0x7fec50199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:36.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.086+0000 7fec4ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec501046f0 0x7fec50199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34270/0 (socket says 172.21.15.1:34270) 2024-04-26T01:39:36.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.086+0000 7fec4ea00700 1 -- 172.21.15.1:0/3764261529 learned_addr learned my addr 172.21.15.1:0/3764261529 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:36.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.087+0000 7fec4ea00700 1 -- 172.21.15.1:0/3764261529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5019a240 con 0x7fec501046f0 2024-04-26T01:39:36.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.087+0000 7fec4ea00700 1 --2- 172.21.15.1:0/3764261529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec501046f0 0x7fec50199d00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec40005800 tx=0x7fec40009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:36.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.087+0000 7fec47400700 1 -- 172.21.15.1:0/3764261529 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec40027070 con 0x7fec501046f0 2024-04-26T01:39:36.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.087+0000 7fec47400700 1 -- 172.21.15.1:0/3764261529 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec4000be10 con 0x7fec501046f0 2024-04-26T01:39:36.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.087+0000 7fec47400700 1 -- 172.21.15.1:0/3764261529 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec4000f460 con 0x7fec501046f0 2024-04-26T01:39:36.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.087+0000 7fec55a00700 1 -- 172.21.15.1:0/3764261529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5019a4d0 con 0x7fec501046f0 2024-04-26T01:39:36.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.088+0000 7fec55a00700 1 -- 172.21.15.1:0/3764261529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec5019d1b0 con 0x7fec501046f0 2024-04-26T01:39:36.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.089+0000 7fec47400700 1 -- 172.21.15.1:0/3764261529 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7fec4002a070 con 0x7fec501046f0 2024-04-26T01:39:36.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.089+0000 7fec55a00700 1 -- 172.21.15.1:0/3764261529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec5010adb0 con 0x7fec501046f0 2024-04-26T01:39:36.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.090+0000 7fec47400700 1 --2- 172.21.15.1:0/3764261529 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fec340396b0 0x7fec3403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:36.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.090+0000 7fec47400700 1 -- 172.21.15.1:0/3764261529 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fec4001e030 con 0x7fec501046f0 2024-04-26T01:39:36.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.091+0000 7fec4e000700 1 --2- 172.21.15.1:0/3764261529 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fec340396b0 0x7fec3403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:36.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.091+0000 7fec4e000700 1 --2- 172.21.15.1:0/3764261529 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fec340396b0 0x7fec3403bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fec3c006fd0 tx=0x7fec3c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:36.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.096+0000 7fec47400700 1 -- 172.21.15.1:0/3764261529 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec40013070 con 0x7fec501046f0 2024-04-26T01:39:36.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.343+0000 7fec55a00700 1 -- 172.21.15.1:0/3764261529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fec5002e090 con 0x7fec501046f0 2024-04-26T01:39:36.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.347+0000 7fec47400700 1 -- 172.21.15.1:0/3764261529 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fec40031020 con 0x7fec501046f0 2024-04-26T01:39:36.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.350+0000 7fec55a00700 1 -- 172.21.15.1:0/3764261529 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fec340396b0 msgr2=0x7fec3403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:36.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.350+0000 7fec55a00700 1 --2- 172.21.15.1:0/3764261529 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fec340396b0 0x7fec3403bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fec3c006fd0 tx=0x7fec3c006e40 comp rx=0 tx=0).stop 2024-04-26T01:39:36.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.350+0000 7fec55a00700 1 -- 172.21.15.1:0/3764261529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec501046f0 msgr2=0x7fec50199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:36.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.350+0000 7fec55a00700 1 --2- 172.21.15.1:0/3764261529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec501046f0 0x7fec50199d00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec40005800 tx=0x7fec40009f60 comp rx=0 tx=0).stop 2024-04-26T01:39:36.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.350+0000 7fec55a00700 1 -- 172.21.15.1:0/3764261529 shutdown_connections 2024-04-26T01:39:36.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.350+0000 7fec55a00700 1 --2- 172.21.15.1:0/3764261529 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fec340396b0 0x7fec3403bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:36.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.350+0000 7fec55a00700 1 --2- 172.21.15.1:0/3764261529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec501046f0 0x7fec50199d00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:36.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.350+0000 7fec55a00700 1 -- 172.21.15.1:0/3764261529 >> 172.21.15.1:0/3764261529 conn(0x7fec500ffc70 msgr2=0x7fec50100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:36.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.351+0000 7fec55a00700 1 -- 172.21.15.1:0/3764261529 shutdown_connections 2024-04-26T01:39:36.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:36.351+0000 7fec55a00700 1 -- 172.21.15.1:0/3764261529 wait complete. 2024-04-26T01:39:37.085 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-26T01:39:37.085 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-26T01:39:37.540 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:39:37.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:37 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3764261529' entity='client.admin' 2024-04-26T01:39:37.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:37 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:37.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:37 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:37.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:37 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi001", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:39:37.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:37 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:37.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:37 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:39:37.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:37 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.516+0000 7ff102e00700 1 -- 172.21.15.1:0/1175903019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0fc1024c0 msgr2=0x7ff0fc1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.516+0000 7ff102e00700 1 --2- 172.21.15.1:0/1175903019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0fc1024c0 0x7ff0fc1028e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec009910 tx=0x7ff0ec009c20 comp rx=0 tx=0).stop 2024-04-26T01:39:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.516+0000 7ff102e00700 1 -- 172.21.15.1:0/1175903019 shutdown_connections 2024-04-26T01:39:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.516+0000 7ff102e00700 1 --2- 172.21.15.1:0/1175903019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0fc1024c0 0x7ff0fc1028e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.516+0000 7ff102e00700 1 -- 172.21.15.1:0/1175903019 >> 172.21.15.1:0/1175903019 conn(0x7ff0fc0fda60 msgr2=0x7ff0fc0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.516+0000 7ff102e00700 1 -- 172.21.15.1:0/1175903019 shutdown_connections 2024-04-26T01:39:39.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.516+0000 7ff102e00700 1 -- 172.21.15.1:0/1175903019 wait complete. 2024-04-26T01:39:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.517+0000 7ff102e00700 1 Processor -- start 2024-04-26T01:39:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.517+0000 7ff102e00700 1 -- start start 2024-04-26T01:39:39.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.518+0000 7ff102e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0fc1024c0 0x7ff0fc19e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:39.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.518+0000 7ff102e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc108400 con 0x7ff0fc1024c0 2024-04-26T01:39:39.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.518+0000 7ff0fbe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0fc1024c0 0x7ff0fc19e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:39.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.518+0000 7ff0fbe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0fc1024c0 0x7ff0fc19e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34286/0 (socket says 172.21.15.1:34286) 2024-04-26T01:39:39.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.518+0000 7ff0fbe00700 1 -- 172.21.15.1:0/2164610586 learned_addr learned my addr 172.21.15.1:0/2164610586 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.518+0000 7ff0fbe00700 1 -- 172.21.15.1:0/2164610586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0fc19e670 con 0x7ff0fc1024c0 2024-04-26T01:39:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.519+0000 7ff0fbe00700 1 --2- 172.21.15.1:0/2164610586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0fc1024c0 0x7ff0fc19e130 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec005950 tx=0x7ff0ec004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.519+0000 7ff0f9600700 1 -- 172.21.15.1:0/2164610586 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0ec026070 con 0x7ff0fc1024c0 2024-04-26T01:39:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.519+0000 7ff0f9600700 1 -- 172.21.15.1:0/2164610586 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0ec00bc00 con 0x7ff0fc1024c0 2024-04-26T01:39:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.519+0000 7ff0f9600700 1 -- 172.21.15.1:0/2164610586 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0ec003a10 con 0x7ff0fc1024c0 2024-04-26T01:39:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.519+0000 7ff102e00700 1 -- 172.21.15.1:0/2164610586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0fc19e900 con 0x7ff0fc1024c0 2024-04-26T01:39:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.519+0000 7ff102e00700 1 -- 172.21.15.1:0/2164610586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0fc1a15f0 con 0x7ff0fc1024c0 2024-04-26T01:39:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.520+0000 7ff102e00700 1 -- 172.21.15.1:0/2164610586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0fc108b80 con 0x7ff0fc1024c0 2024-04-26T01:39:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.520+0000 7ff0f9600700 1 -- 172.21.15.1:0/2164610586 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7ff0ec029070 con 0x7ff0fc1024c0 2024-04-26T01:39:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.521+0000 7ff0f9600700 1 --2- 172.21.15.1:0/2164610586 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ff0e0039650 0x7ff0e003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:39.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.521+0000 7ff0f9600700 1 -- 172.21.15.1:0/2164610586 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff0ec01e070 con 0x7ff0fc1024c0 2024-04-26T01:39:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.521+0000 7ff0f3400700 1 --2- 172.21.15.1:0/2164610586 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ff0e0039650 0x7ff0e003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.522+0000 7ff0f3400700 1 --2- 172.21.15.1:0/2164610586 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ff0e0039650 0x7ff0e003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4006fd0 tx=0x7ff0e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:39.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.527+0000 7ff0f9600700 1 -- 172.21.15.1:0/2164610586 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff0ec013070 con 0x7ff0fc1024c0 2024-04-26T01:39:39.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.762+0000 7ff102e00700 1 -- 172.21.15.1:0/2164610586 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7ff0fc02e090 con 0x7ff0e0039650 2024-04-26T01:39:39.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.770+0000 7ff0f9600700 1 -- 172.21.15.1:0/2164610586 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff0fc02e090 con 0x7ff0e0039650 2024-04-26T01:39:39.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.772+0000 7ff102e00700 1 -- 172.21.15.1:0/2164610586 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ff0e0039650 msgr2=0x7ff0e003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:39.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.772+0000 7ff102e00700 1 --2- 172.21.15.1:0/2164610586 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ff0e0039650 0x7ff0e003bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4006fd0 tx=0x7ff0e4006e40 comp rx=0 tx=0).stop 2024-04-26T01:39:39.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.772+0000 7ff102e00700 1 -- 172.21.15.1:0/2164610586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0fc1024c0 msgr2=0x7ff0fc19e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:39.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.772+0000 7ff102e00700 1 --2- 172.21.15.1:0/2164610586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0fc1024c0 0x7ff0fc19e130 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec005950 tx=0x7ff0ec004fa0 comp rx=0 tx=0).stop 2024-04-26T01:39:39.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.772+0000 7ff102e00700 1 -- 172.21.15.1:0/2164610586 shutdown_connections 2024-04-26T01:39:39.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.772+0000 7ff102e00700 1 --2- 172.21.15.1:0/2164610586 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ff0e0039650 0x7ff0e003bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:39.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.772+0000 7ff102e00700 1 --2- 172.21.15.1:0/2164610586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0fc1024c0 0x7ff0fc19e130 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:39.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.772+0000 7ff102e00700 1 -- 172.21.15.1:0/2164610586 >> 172.21.15.1:0/2164610586 conn(0x7ff0fc0fda60 msgr2=0x7ff0fc0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:39.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.773+0000 7ff102e00700 1 -- 172.21.15.1:0/2164610586 shutdown_connections 2024-04-26T01:39:39.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:39.773+0000 7ff102e00700 1 -- 172.21.15.1:0/2164610586 wait complete. 2024-04-26T01:39:40.477 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi097 2024-04-26T01:39:40.477 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T01:39:40.477 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/etc/ceph/ceph.conf 2024-04-26T01:39:40.495 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T01:39:40.495 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:39:40.553 INFO:tasks.cephadm:Adding host smithi097 to orchestrator... 2024-04-26T01:39:40.553 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph orch host add smithi097 2024-04-26T01:39:40.970 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:39:41.116 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:40 smithi001 ceph-mon[27822]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:39:41.116 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:40 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:41.116 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:40 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:39:41.117 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:40 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:39:41.117 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:40 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:39:41.117 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:40 smithi001 ceph-mon[27822]: Updating smithi001:/etc/ceph/ceph.conf 2024-04-26T01:39:41.117 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:40 smithi001 ceph-mon[27822]: Updating smithi001:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:39:42.127 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:41 smithi001 ceph-mon[27822]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:39:42.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:41 smithi001 ceph-mon[27822]: Updating smithi001:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:39:43.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.037+0000 7faec7200700 1 -- 172.21.15.1:0/3898319115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faec010a670 msgr2=0x7faec010aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:43.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.037+0000 7faec7200700 1 --2- 172.21.15.1:0/3898319115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faec010a670 0x7faec010aa50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faeb0009910 tx=0x7faeb0009c20 comp rx=0 tx=0).stop 2024-04-26T01:39:43.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.037+0000 7faec7200700 1 -- 172.21.15.1:0/3898319115 shutdown_connections 2024-04-26T01:39:43.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.037+0000 7faec7200700 1 --2- 172.21.15.1:0/3898319115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faec010a670 0x7faec010aa50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:43.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.037+0000 7faec7200700 1 -- 172.21.15.1:0/3898319115 >> 172.21.15.1:0/3898319115 conn(0x7faec0100160 msgr2=0x7faec0102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:43.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.037+0000 7faec7200700 1 -- 172.21.15.1:0/3898319115 shutdown_connections 2024-04-26T01:39:43.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.037+0000 7faec7200700 1 -- 172.21.15.1:0/3898319115 wait complete. 2024-04-26T01:39:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.038+0000 7faec7200700 1 Processor -- start 2024-04-26T01:39:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.039+0000 7faec7200700 1 -- start start 2024-04-26T01:39:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.039+0000 7faec7200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faec010a670 0x7faec019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.039+0000 7faec7200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faec010eb60 con 0x7faec010a670 2024-04-26T01:39:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.039+0000 7faebfe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faec010a670 0x7faec019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.039+0000 7faebfe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faec010a670 0x7faec019a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42548/0 (socket says 172.21.15.1:42548) 2024-04-26T01:39:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.039+0000 7faebfe00700 1 -- 172.21.15.1:0/3429473929 learned_addr learned my addr 172.21.15.1:0/3429473929 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:43.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.040+0000 7faebfe00700 1 -- 172.21.15.1:0/3429473929 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec019a600 con 0x7faec010a670 2024-04-26T01:39:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.040+0000 7faebfe00700 1 --2- 172.21.15.1:0/3429473929 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faec010a670 0x7faec019a0c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faeb00056d0 tx=0x7faeb00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.040+0000 7faebcc00700 1 -- 172.21.15.1:0/3429473929 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faeb0026070 con 0x7faec010a670 2024-04-26T01:39:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.040+0000 7faebcc00700 1 -- 172.21.15.1:0/3429473929 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faeb000bba0 con 0x7faec010a670 2024-04-26T01:39:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.040+0000 7faebcc00700 1 -- 172.21.15.1:0/3429473929 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faeb0003920 con 0x7faec010a670 2024-04-26T01:39:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.040+0000 7faec7200700 1 -- 172.21.15.1:0/3429473929 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec019a8f0 con 0x7faec010a670 2024-04-26T01:39:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.040+0000 7faec7200700 1 -- 172.21.15.1:0/3429473929 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec019e990 con 0x7faec010a670 2024-04-26T01:39:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.042+0000 7faebcc00700 1 -- 172.21.15.1:0/3429473929 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7faeb0029070 con 0x7faec010a670 2024-04-26T01:39:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.042+0000 7faec7200700 1 -- 172.21.15.1:0/3429473929 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faec0105db0 con 0x7faec010a670 2024-04-26T01:39:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.042+0000 7faebcc00700 1 --2- 172.21.15.1:0/3429473929 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7faea4039650 0x7faea403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.042+0000 7faebcc00700 1 -- 172.21.15.1:0/3429473929 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faeb001e070 con 0x7faec010a670 2024-04-26T01:39:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.042+0000 7faebf400700 1 --2- 172.21.15.1:0/3429473929 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7faea4039650 0x7faea403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.043+0000 7faebf400700 1 --2- 172.21.15.1:0/3429473929 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7faea4039650 0x7faea403bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faeac006fd0 tx=0x7faeac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:43.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.048+0000 7faebcc00700 1 -- 172.21.15.1:0/3429473929 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faeb0013070 con 0x7faec010a670 2024-04-26T01:39:43.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:43.284+0000 7faec7200700 1 -- 172.21.15.1:0/3429473929 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi097", "target": ["mon-mgr", ""]}) -- 0x7faec019b420 con 0x7faea4039650 2024-04-26T01:39:43.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:43 smithi001 ceph-mon[27822]: Updating smithi001:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:39:43.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:43 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:43.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:43 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:43.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:43 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:44.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:44 smithi001 ceph-mon[27822]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:39:44.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:44 smithi001 ceph-mon[27822]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi097", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:39:45.877 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:45 smithi001 ceph-mon[27822]: Deploying cephadm binary to smithi097 2024-04-26T01:39:46.779 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:46 smithi001 ceph-mon[27822]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:39:48.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:48.089+0000 7faebcc00700 1 -- 172.21.15.1:0/3429473929 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7faec019b420 con 0x7faea4039650 2024-04-26T01:39:48.087 INFO:teuthology.orchestra.run.smithi001.stdout:Added host 'smithi097' with addr '172.21.15.97' 2024-04-26T01:39:48.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:48.092+0000 7faec7200700 1 -- 172.21.15.1:0/3429473929 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7faea4039650 msgr2=0x7faea403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:48.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:48.092+0000 7faec7200700 1 --2- 172.21.15.1:0/3429473929 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7faea4039650 0x7faea403bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faeac006fd0 tx=0x7faeac006e40 comp rx=0 tx=0).stop 2024-04-26T01:39:48.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:48.092+0000 7faec7200700 1 -- 172.21.15.1:0/3429473929 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faec010a670 msgr2=0x7faec019a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:48.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:48.092+0000 7faec7200700 1 --2- 172.21.15.1:0/3429473929 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faec010a670 0x7faec019a0c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faeb00056d0 tx=0x7faeb00057b0 comp rx=0 tx=0).stop 2024-04-26T01:39:48.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:48.092+0000 7faec7200700 1 -- 172.21.15.1:0/3429473929 shutdown_connections 2024-04-26T01:39:48.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:48.092+0000 7faec7200700 1 --2- 172.21.15.1:0/3429473929 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7faea4039650 0x7faea403bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:48.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:48.092+0000 7faec7200700 1 --2- 172.21.15.1:0/3429473929 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faec010a670 0x7faec019a0c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:48.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:48.092+0000 7faec7200700 1 -- 172.21.15.1:0/3429473929 >> 172.21.15.1:0/3429473929 conn(0x7faec0100160 msgr2=0x7faec0100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:48.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:48.093+0000 7faec7200700 1 -- 172.21.15.1:0/3429473929 shutdown_connections 2024-04-26T01:39:48.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:48.093+0000 7faec7200700 1 -- 172.21.15.1:0/3429473929 wait complete. 2024-04-26T01:39:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:48 smithi001 ceph-mon[27822]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:39:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:48 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:48 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:39:48.795 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T01:39:49.239 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:39:49.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:49 smithi001 ceph-mon[27822]: Added host smithi097 2024-04-26T01:39:49.773 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:49 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:49.773 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:49 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:50.755 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:50 smithi001 ceph-mon[27822]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:39:51.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.253+0000 7f51bba00700 1 -- 172.21.15.1:0/301868442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b4074d60 msgr2=0x7f51b4075140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:51.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.253+0000 7f51bba00700 1 --2- 172.21.15.1:0/301868442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b4074d60 0x7f51b4075140 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009910 tx=0x7f51a8009c20 comp rx=0 tx=0).stop 2024-04-26T01:39:51.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.253+0000 7f51bba00700 1 -- 172.21.15.1:0/301868442 shutdown_connections 2024-04-26T01:39:51.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.253+0000 7f51bba00700 1 --2- 172.21.15.1:0/301868442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b4074d60 0x7f51b4075140 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:51.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.253+0000 7f51bba00700 1 -- 172.21.15.1:0/301868442 >> 172.21.15.1:0/301868442 conn(0x7f51b40fdf80 msgr2=0x7f51b41003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:51.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.253+0000 7f51bba00700 1 -- 172.21.15.1:0/301868442 shutdown_connections 2024-04-26T01:39:51.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.253+0000 7f51bba00700 1 -- 172.21.15.1:0/301868442 wait complete. 2024-04-26T01:39:51.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.254+0000 7f51bba00700 1 Processor -- start 2024-04-26T01:39:51.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.254+0000 7f51bba00700 1 -- start start 2024-04-26T01:39:51.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.255+0000 7f51bba00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b4074d60 0x7f51b419a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:51.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.255+0000 7f51bba00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f51b410eb90 con 0x7f51b4074d60 2024-04-26T01:39:51.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.255+0000 7f51b9000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b4074d60 0x7f51b419a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:51.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.255+0000 7f51b9000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b4074d60 0x7f51b419a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42574/0 (socket says 172.21.15.1:42574) 2024-04-26T01:39:51.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.255+0000 7f51b9000700 1 -- 172.21.15.1:0/3864285577 learned_addr learned my addr 172.21.15.1:0/3864285577 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:51.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.255+0000 7f51b9000700 1 -- 172.21.15.1:0/3864285577 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b419a630 con 0x7f51b4074d60 2024-04-26T01:39:51.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.256+0000 7f51b9000700 1 --2- 172.21.15.1:0/3864285577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b4074d60 0x7f51b419a0f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f51a8006010 tx=0x7f51a8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:51.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.256+0000 7f51b1600700 1 -- 172.21.15.1:0/3864285577 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51a8026070 con 0x7f51b4074d60 2024-04-26T01:39:51.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.256+0000 7f51b1600700 1 -- 172.21.15.1:0/3864285577 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51a800bc00 con 0x7f51b4074d60 2024-04-26T01:39:51.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.256+0000 7f51b1600700 1 -- 172.21.15.1:0/3864285577 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51a8003910 con 0x7f51b4074d60 2024-04-26T01:39:51.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.256+0000 7f51bba00700 1 -- 172.21.15.1:0/3864285577 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b419a8c0 con 0x7f51b4074d60 2024-04-26T01:39:51.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.256+0000 7f51bba00700 1 -- 172.21.15.1:0/3864285577 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b419ad20 con 0x7f51b4074d60 2024-04-26T01:39:51.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.257+0000 7f51b1600700 1 -- 172.21.15.1:0/3864285577 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f51a8029070 con 0x7f51b4074d60 2024-04-26T01:39:51.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.257+0000 7f51bba00700 1 -- 172.21.15.1:0/3864285577 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b40764a0 con 0x7f51b4074d60 2024-04-26T01:39:51.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.258+0000 7f51b1600700 1 --2- 172.21.15.1:0/3864285577 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f519c0396b0 0x7f519c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:51.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.258+0000 7f51b1600700 1 -- 172.21.15.1:0/3864285577 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f51a801e070 con 0x7f51b4074d60 2024-04-26T01:39:51.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.258+0000 7f51b3e00700 1 --2- 172.21.15.1:0/3864285577 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f519c0396b0 0x7f519c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:51.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.259+0000 7f51b3e00700 1 --2- 172.21.15.1:0/3864285577 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f519c0396b0 0x7f519c03bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f51a0006fd0 tx=0x7f51a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:51.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.264+0000 7f51b1600700 1 -- 172.21.15.1:0/3864285577 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f51a8013070 con 0x7f51b4074d60 2024-04-26T01:39:51.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.501+0000 7f51bba00700 1 -- 172.21.15.1:0/3864285577 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51b419b4c0 con 0x7f519c0396b0 2024-04-26T01:39:51.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.503+0000 7f51b1600700 1 -- 172.21.15.1:0/3864285577 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+158 (secure 0 0 0) 0x7f51b419b4c0 con 0x7f519c0396b0 2024-04-26T01:39:51.500 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:39:51.500 INFO:teuthology.orchestra.run.smithi001.stdout:[{"addr": "172.21.15.1", "hostname": "smithi001", "labels": [], "status": ""}, {"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}] 2024-04-26T01:39:51.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.506+0000 7f51bba00700 1 -- 172.21.15.1:0/3864285577 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f519c0396b0 msgr2=0x7f519c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:51.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.506+0000 7f51bba00700 1 --2- 172.21.15.1:0/3864285577 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f519c0396b0 0x7f519c03bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f51a0006fd0 tx=0x7f51a0006e40 comp rx=0 tx=0).stop 2024-04-26T01:39:51.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.506+0000 7f51bba00700 1 -- 172.21.15.1:0/3864285577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b4074d60 msgr2=0x7f51b419a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:51.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.506+0000 7f51bba00700 1 --2- 172.21.15.1:0/3864285577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b4074d60 0x7f51b419a0f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f51a8006010 tx=0x7f51a8005780 comp rx=0 tx=0).stop 2024-04-26T01:39:51.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.506+0000 7f51bba00700 1 -- 172.21.15.1:0/3864285577 shutdown_connections 2024-04-26T01:39:51.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.506+0000 7f51bba00700 1 --2- 172.21.15.1:0/3864285577 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f519c0396b0 0x7f519c03bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:51.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.506+0000 7f51bba00700 1 --2- 172.21.15.1:0/3864285577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b4074d60 0x7f51b419a0f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:51.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.506+0000 7f51bba00700 1 -- 172.21.15.1:0/3864285577 >> 172.21.15.1:0/3864285577 conn(0x7f51b40fdf80 msgr2=0x7f51b4106770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:51.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.507+0000 7f51bba00700 1 -- 172.21.15.1:0/3864285577 shutdown_connections 2024-04-26T01:39:51.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:51.507+0000 7f51bba00700 1 -- 172.21.15.1:0/3864285577 wait complete. 2024-04-26T01:39:52.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:51 smithi001 ceph-mon[27822]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:39:52.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:51 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:52.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:51 smithi001 ceph-mon[27822]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:39:52.238 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi173 2024-04-26T01:39:52.239 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-26T01:39:52.239 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/etc/ceph/ceph.conf 2024-04-26T01:39:52.257 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-26T01:39:52.257 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:39:52.315 INFO:tasks.cephadm:Adding host smithi173 to orchestrator... 2024-04-26T01:39:52.315 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph orch host add smithi173 2024-04-26T01:39:52.791 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:39:54.109 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:53 smithi001 ceph-mon[27822]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:39:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.905+0000 7f9fd1e00700 1 -- 172.21.15.1:0/943880269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fcc1084e0 msgr2=0x7f9fcc1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.905+0000 7f9fd1e00700 1 --2- 172.21.15.1:0/943880269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fcc1084e0 0x7f9fcc1088c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc009910 tx=0x7f9fbc009c20 comp rx=0 tx=0).stop 2024-04-26T01:39:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.905+0000 7f9fd1e00700 1 -- 172.21.15.1:0/943880269 shutdown_connections 2024-04-26T01:39:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.905+0000 7f9fd1e00700 1 --2- 172.21.15.1:0/943880269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fcc1084e0 0x7f9fcc1088c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.906+0000 7f9fd1e00700 1 -- 172.21.15.1:0/943880269 >> 172.21.15.1:0/943880269 conn(0x7f9fcc076ea0 msgr2=0x7f9fcc0772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.906+0000 7f9fd1e00700 1 -- 172.21.15.1:0/943880269 shutdown_connections 2024-04-26T01:39:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.906+0000 7f9fd1e00700 1 -- 172.21.15.1:0/943880269 wait complete. 2024-04-26T01:39:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.907+0000 7f9fd1e00700 1 Processor -- start 2024-04-26T01:39:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.907+0000 7f9fd1e00700 1 -- start start 2024-04-26T01:39:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.908+0000 7f9fd1e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fcc1084e0 0x7f9fcc197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.908+0000 7f9fd1e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc10c920 con 0x7f9fcc1084e0 2024-04-26T01:39:54.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.908+0000 7f9fcaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fcc1084e0 0x7f9fcc197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:54.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.908+0000 7f9fcaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fcc1084e0 0x7f9fcc197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59976/0 (socket says 172.21.15.1:59976) 2024-04-26T01:39:54.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.908+0000 7f9fcaa00700 1 -- 172.21.15.1:0/2223317258 learned_addr learned my addr 172.21.15.1:0/2223317258 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:39:54.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.909+0000 7f9fcaa00700 1 -- 172.21.15.1:0/2223317258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fcc198400 con 0x7f9fcc1084e0 2024-04-26T01:39:54.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.909+0000 7f9fcaa00700 1 --2- 172.21.15.1:0/2223317258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fcc1084e0 0x7f9fcc197ec0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc0056d0 tx=0x7f9fbc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:54.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.909+0000 7f9fd0a00700 1 -- 172.21.15.1:0/2223317258 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fbc026070 con 0x7f9fcc1084e0 2024-04-26T01:39:54.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.909+0000 7f9fd0a00700 1 -- 172.21.15.1:0/2223317258 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fbc00bba0 con 0x7f9fcc1084e0 2024-04-26T01:39:54.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.909+0000 7f9fd1e00700 1 -- 172.21.15.1:0/2223317258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fcc1986f0 con 0x7f9fcc1084e0 2024-04-26T01:39:54.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.909+0000 7f9fd0a00700 1 -- 172.21.15.1:0/2223317258 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fbc003920 con 0x7f9fcc1084e0 2024-04-26T01:39:54.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.909+0000 7f9fd1e00700 1 -- 172.21.15.1:0/2223317258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fcc19c790 con 0x7f9fcc1084e0 2024-04-26T01:39:54.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.911+0000 7f9fd0a00700 1 -- 172.21.15.1:0/2223317258 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f9fbc029070 con 0x7f9fcc1084e0 2024-04-26T01:39:54.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.911+0000 7f9fd1e00700 1 -- 172.21.15.1:0/2223317258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fcc103b90 con 0x7f9fcc1084e0 2024-04-26T01:39:54.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.911+0000 7f9fd0a00700 1 --2- 172.21.15.1:0/2223317258 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f9fb00396b0 0x7f9fb003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:39:54.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.911+0000 7f9fd0a00700 1 -- 172.21.15.1:0/2223317258 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9fbc01e070 con 0x7f9fcc1084e0 2024-04-26T01:39:54.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.911+0000 7f9fca000700 1 --2- 172.21.15.1:0/2223317258 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f9fb00396b0 0x7f9fb003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:39:54.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.912+0000 7f9fca000700 1 --2- 172.21.15.1:0/2223317258 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f9fb00396b0 0x7f9fb003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8006fd0 tx=0x7f9fb8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:39:54.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:54.917+0000 7f9fd0a00700 1 -- 172.21.15.1:0/2223317258 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9fbc013070 con 0x7f9fcc1084e0 2024-04-26T01:39:55.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:55.157+0000 7f9fd1e00700 1 -- 172.21.15.1:0/2223317258 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi173", "target": ["mon-mgr", ""]}) -- 0x7f9fcc199220 con 0x7f9fb00396b0 2024-04-26T01:39:55.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:54 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:55.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:54 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:55.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:54 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:55.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:54 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:55.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:54 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:39:55.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:54 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:39:55.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:54 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:39:55.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:54 smithi001 ceph-mon[27822]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-26T01:39:55.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:54 smithi001 ceph-mon[27822]: Updating smithi097:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:39:56.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:56 smithi001 ceph-mon[27822]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:39:56.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:56 smithi001 ceph-mon[27822]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:39:56.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:56 smithi001 ceph-mon[27822]: Updating smithi097:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:39:56.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:56 smithi001 ceph-mon[27822]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi173", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:39:56.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:56 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:56.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:56 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:56.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:56 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:39:57.627 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:57 smithi001 ceph-mon[27822]: Deploying cephadm binary to smithi173 2024-04-26T01:39:58.627 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:39:58 smithi001 ceph-mon[27822]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:39:59.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:59.992+0000 7f9fd0a00700 1 -- 172.21.15.1:0/2223317258 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f9fcc199220 con 0x7f9fb00396b0 2024-04-26T01:39:59.990 INFO:teuthology.orchestra.run.smithi001.stdout:Added host 'smithi173' with addr '172.21.15.173' 2024-04-26T01:39:59.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:59.996+0000 7f9fd1e00700 1 -- 172.21.15.1:0/2223317258 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f9fb00396b0 msgr2=0x7f9fb003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:59.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:59.996+0000 7f9fd1e00700 1 --2- 172.21.15.1:0/2223317258 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f9fb00396b0 0x7f9fb003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8006fd0 tx=0x7f9fb8006e40 comp rx=0 tx=0).stop 2024-04-26T01:39:59.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:59.997+0000 7f9fd1e00700 1 -- 172.21.15.1:0/2223317258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fcc1084e0 msgr2=0x7f9fcc197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:39:59.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:59.997+0000 7f9fd1e00700 1 --2- 172.21.15.1:0/2223317258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fcc1084e0 0x7f9fcc197ec0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc0056d0 tx=0x7f9fbc0057b0 comp rx=0 tx=0).stop 2024-04-26T01:39:59.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:59.997+0000 7f9fd1e00700 1 -- 172.21.15.1:0/2223317258 shutdown_connections 2024-04-26T01:39:59.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:59.997+0000 7f9fd1e00700 1 --2- 172.21.15.1:0/2223317258 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f9fb00396b0 0x7f9fb003bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:59.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:59.997+0000 7f9fd1e00700 1 --2- 172.21.15.1:0/2223317258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fcc1084e0 0x7f9fcc197ec0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:39:59.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:59.997+0000 7f9fd1e00700 1 -- 172.21.15.1:0/2223317258 >> 172.21.15.1:0/2223317258 conn(0x7f9fcc076ea0 msgr2=0x7f9fcc0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:39:59.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:59.998+0000 7f9fd1e00700 1 -- 172.21.15.1:0/2223317258 shutdown_connections 2024-04-26T01:39:59.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:39:59.998+0000 7f9fd1e00700 1 -- 172.21.15.1:0/2223317258 wait complete. 2024-04-26T01:40:00.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:00 smithi001 ceph-mon[27822]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:00.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:00 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:00.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:00 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:40:00.768 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T01:40:01.230 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:40:01.672 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:01 smithi001 ceph-mon[27822]: Added host smithi173 2024-04-26T01:40:01.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:01 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:01.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:01 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:02.711 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:02 smithi001 ceph-mon[27822]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:03.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.243+0000 7fa90fc00700 1 -- 172.21.15.1:0/2344718632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa908069e60 msgr2=0x7fa90806a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:03.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.243+0000 7fa90fc00700 1 --2- 172.21.15.1:0/2344718632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa908069e60 0x7fa90806a240 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009910 tx=0x7fa8fc009c20 comp rx=0 tx=0).stop 2024-04-26T01:40:03.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.243+0000 7fa90fc00700 1 -- 172.21.15.1:0/2344718632 shutdown_connections 2024-04-26T01:40:03.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.243+0000 7fa90fc00700 1 --2- 172.21.15.1:0/2344718632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa908069e60 0x7fa90806a240 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:03.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.243+0000 7fa90fc00700 1 -- 172.21.15.1:0/2344718632 >> 172.21.15.1:0/2344718632 conn(0x7fa908076ea0 msgr2=0x7fa9080772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:03.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.243+0000 7fa90fc00700 1 -- 172.21.15.1:0/2344718632 shutdown_connections 2024-04-26T01:40:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.244+0000 7fa90fc00700 1 -- 172.21.15.1:0/2344718632 wait complete. 2024-04-26T01:40:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.245+0000 7fa90fc00700 1 Processor -- start 2024-04-26T01:40:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.245+0000 7fa90fc00700 1 -- start start 2024-04-26T01:40:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.246+0000 7fa90fc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa908069e60 0x7fa90819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.246+0000 7fa90fc00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa90810eb90 con 0x7fa908069e60 2024-04-26T01:40:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.246+0000 7fa90d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa908069e60 0x7fa90819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.246+0000 7fa90d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa908069e60 0x7fa90819a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45082/0 (socket says 172.21.15.1:45082) 2024-04-26T01:40:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.246+0000 7fa90d200700 1 -- 172.21.15.1:0/3480511173 learned_addr learned my addr 172.21.15.1:0/3480511173 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:40:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.247+0000 7fa90d200700 1 -- 172.21.15.1:0/3480511173 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90819a650 con 0x7fa908069e60 2024-04-26T01:40:03.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.247+0000 7fa90d200700 1 --2- 172.21.15.1:0/3480511173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa908069e60 0x7fa90819a110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc006010 tx=0x7fa8fc005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:03.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.247+0000 7fa905600700 1 -- 172.21.15.1:0/3480511173 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8fc026070 con 0x7fa908069e60 2024-04-26T01:40:03.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.248+0000 7fa90fc00700 1 -- 172.21.15.1:0/3480511173 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa90819a8e0 con 0x7fa908069e60 2024-04-26T01:40:03.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.248+0000 7fa905600700 1 -- 172.21.15.1:0/3480511173 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8fc00bc00 con 0x7fa908069e60 2024-04-26T01:40:03.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.248+0000 7fa905600700 1 -- 172.21.15.1:0/3480511173 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8fc003910 con 0x7fa908069e60 2024-04-26T01:40:03.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.248+0000 7fa90fc00700 1 -- 172.21.15.1:0/3480511173 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa90819ad40 con 0x7fa908069e60 2024-04-26T01:40:03.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.249+0000 7fa905600700 1 -- 172.21.15.1:0/3480511173 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7fa8fc029070 con 0x7fa908069e60 2024-04-26T01:40:03.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.249+0000 7fa90fc00700 1 -- 172.21.15.1:0/3480511173 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa908101970 con 0x7fa908069e60 2024-04-26T01:40:03.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.250+0000 7fa905600700 1 --2- 172.21.15.1:0/3480511173 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa8f00396b0 0x7fa8f003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:03.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.250+0000 7fa905600700 1 -- 172.21.15.1:0/3480511173 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa8fc01e070 con 0x7fa908069e60 2024-04-26T01:40:03.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.250+0000 7fa907e00700 1 --2- 172.21.15.1:0/3480511173 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa8f00396b0 0x7fa8f003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:03.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.251+0000 7fa907e00700 1 --2- 172.21.15.1:0/3480511173 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa8f00396b0 0x7fa8f003bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4006fd0 tx=0x7fa8f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:03.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.256+0000 7fa905600700 1 -- 172.21.15.1:0/3480511173 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa8fc013070 con 0x7fa908069e60 2024-04-26T01:40:03.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.495+0000 7fa90fc00700 1 -- 172.21.15.1:0/3480511173 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa90819b470 con 0x7fa8f00396b0 2024-04-26T01:40:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.498+0000 7fa905600700 1 -- 172.21.15.1:0/3480511173 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7fa90819b470 con 0x7fa8f00396b0 2024-04-26T01:40:03.495 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:40:03.495 INFO:teuthology.orchestra.run.smithi001.stdout:[{"addr": "172.21.15.1", "hostname": "smithi001", "labels": [], "status": ""}, {"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}, {"addr": "172.21.15.173", "hostname": "smithi173", "labels": [], "status": ""}] 2024-04-26T01:40:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.500+0000 7fa90fc00700 1 -- 172.21.15.1:0/3480511173 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa8f00396b0 msgr2=0x7fa8f003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.500+0000 7fa90fc00700 1 --2- 172.21.15.1:0/3480511173 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa8f00396b0 0x7fa8f003bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4006fd0 tx=0x7fa8f4006e40 comp rx=0 tx=0).stop 2024-04-26T01:40:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.500+0000 7fa90fc00700 1 -- 172.21.15.1:0/3480511173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa908069e60 msgr2=0x7fa90819a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.500+0000 7fa90fc00700 1 --2- 172.21.15.1:0/3480511173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa908069e60 0x7fa90819a110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc006010 tx=0x7fa8fc005780 comp rx=0 tx=0).stop 2024-04-26T01:40:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.501+0000 7fa90fc00700 1 -- 172.21.15.1:0/3480511173 shutdown_connections 2024-04-26T01:40:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.501+0000 7fa90fc00700 1 --2- 172.21.15.1:0/3480511173 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa8f00396b0 0x7fa8f003bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.501+0000 7fa90fc00700 1 --2- 172.21.15.1:0/3480511173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa908069e60 0x7fa90819a110 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.501+0000 7fa90fc00700 1 -- 172.21.15.1:0/3480511173 >> 172.21.15.1:0/3480511173 conn(0x7fa908076ea0 msgr2=0x7fa9081099c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.501+0000 7fa90fc00700 1 -- 172.21.15.1:0/3480511173 shutdown_connections 2024-04-26T01:40:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:03.501+0000 7fa90fc00700 1 -- 172.21.15.1:0/3480511173 wait complete. 2024-04-26T01:40:03.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:03 smithi001 ceph-mon[27822]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:03.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:03.881 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:03 smithi001 ceph-mon[27822]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:40:04.311 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-26T01:40:04.311 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-26T01:40:04.774 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:40:05.981 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:05 smithi001 ceph-mon[27822]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:06.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.874+0000 7fb64b600700 1 -- 172.21.15.1:0/3996193057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb644069e60 msgr2=0x7fb64406a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:06.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.874+0000 7fb64b600700 1 --2- 172.21.15.1:0/3996193057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb644069e60 0x7fb64406a240 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb63800b0a0 tx=0x7fb63800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:40:06.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.874+0000 7fb64b600700 1 -- 172.21.15.1:0/3996193057 shutdown_connections 2024-04-26T01:40:06.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.874+0000 7fb64b600700 1 --2- 172.21.15.1:0/3996193057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb644069e60 0x7fb64406a240 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:06.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.874+0000 7fb64b600700 1 -- 172.21.15.1:0/3996193057 >> 172.21.15.1:0/3996193057 conn(0x7fb644076ea0 msgr2=0x7fb6440772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:06.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.874+0000 7fb64b600700 1 -- 172.21.15.1:0/3996193057 shutdown_connections 2024-04-26T01:40:06.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.874+0000 7fb64b600700 1 -- 172.21.15.1:0/3996193057 wait complete. 2024-04-26T01:40:06.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.876+0000 7fb64b600700 1 Processor -- start 2024-04-26T01:40:06.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.876+0000 7fb64b600700 1 -- start start 2024-04-26T01:40:06.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.877+0000 7fb64b600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb644069e60 0x7fb64419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:06.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.877+0000 7fb64b600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb64410eb90 con 0x7fb644069e60 2024-04-26T01:40:06.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.877+0000 7fb648c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb644069e60 0x7fb64419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:06.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.877+0000 7fb648c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb644069e60 0x7fb64419a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45104/0 (socket says 172.21.15.1:45104) 2024-04-26T01:40:06.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.877+0000 7fb648c00700 1 -- 172.21.15.1:0/360923931 learned_addr learned my addr 172.21.15.1:0/360923931 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:40:06.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.877+0000 7fb648c00700 1 -- 172.21.15.1:0/360923931 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64419a5b0 con 0x7fb644069e60 2024-04-26T01:40:06.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.878+0000 7fb648c00700 1 --2- 172.21.15.1:0/360923931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb644069e60 0x7fb64419a070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb63800d010 tx=0x7fb638003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:06.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.878+0000 7fb641600700 1 -- 172.21.15.1:0/360923931 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb638021020 con 0x7fb644069e60 2024-04-26T01:40:06.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.878+0000 7fb641600700 1 -- 172.21.15.1:0/360923931 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6380096a0 con 0x7fb644069e60 2024-04-26T01:40:06.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.878+0000 7fb641600700 1 -- 172.21.15.1:0/360923931 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb638017040 con 0x7fb644069e60 2024-04-26T01:40:06.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.878+0000 7fb64b600700 1 -- 172.21.15.1:0/360923931 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb64419a840 con 0x7fb644069e60 2024-04-26T01:40:06.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.878+0000 7fb64b600700 1 -- 172.21.15.1:0/360923931 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb64419e940 con 0x7fb644069e60 2024-04-26T01:40:06.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.879+0000 7fb641600700 1 -- 172.21.15.1:0/360923931 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7fb638024020 con 0x7fb644069e60 2024-04-26T01:40:06.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.879+0000 7fb64b600700 1 -- 172.21.15.1:0/360923931 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb644101970 con 0x7fb644069e60 2024-04-26T01:40:06.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.880+0000 7fb641600700 1 --2- 172.21.15.1:0/360923931 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb62c039700 0x7fb62c03bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:06.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.880+0000 7fb641600700 1 -- 172.21.15.1:0/360923931 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb63800f050 con 0x7fb644069e60 2024-04-26T01:40:06.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.880+0000 7fb643e00700 1 --2- 172.21.15.1:0/360923931 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb62c039700 0x7fb62c03bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:06.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.881+0000 7fb643e00700 1 --2- 172.21.15.1:0/360923931 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb62c039700 0x7fb62c03bbc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb630006fd0 tx=0x7fb630006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:06.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:06.886+0000 7fb641600700 1 -- 172.21.15.1:0/360923931 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb63802b020 con 0x7fb644069e60 2024-04-26T01:40:07.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:07.127+0000 7fb64b600700 1 -- 172.21.15.1:0/360923931 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fb64419b3d0 con 0x7fb644069e60 2024-04-26T01:40:07.124 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:06 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:07.124 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:06 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:07.124 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:06 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:07.124 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:06 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:07.124 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:06 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi173", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:40:07.124 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:06 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:07.124 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:06 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:40:07.125 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:06 smithi001 ceph-mon[27822]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-26T01:40:07.125 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:06 smithi001 ceph-mon[27822]: Updating smithi173:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:40:07.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:07.951+0000 7fb641600700 1 -- 172.21.15.1:0/360923931 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fb63802b200 con 0x7fb644069e60 2024-04-26T01:40:07.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:07.955+0000 7fb64b600700 1 -- 172.21.15.1:0/360923931 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb62c039700 msgr2=0x7fb62c03bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:07.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:07.955+0000 7fb64b600700 1 --2- 172.21.15.1:0/360923931 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb62c039700 0x7fb62c03bbc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb630006fd0 tx=0x7fb630006e40 comp rx=0 tx=0).stop 2024-04-26T01:40:07.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:07.955+0000 7fb64b600700 1 -- 172.21.15.1:0/360923931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb644069e60 msgr2=0x7fb64419a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:07.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:07.955+0000 7fb64b600700 1 --2- 172.21.15.1:0/360923931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb644069e60 0x7fb64419a070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb63800d010 tx=0x7fb638003ce0 comp rx=0 tx=0).stop 2024-04-26T01:40:07.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:07.955+0000 7fb64b600700 1 -- 172.21.15.1:0/360923931 shutdown_connections 2024-04-26T01:40:07.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:07.955+0000 7fb64b600700 1 --2- 172.21.15.1:0/360923931 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb62c039700 0x7fb62c03bbc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:07.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:07.955+0000 7fb64b600700 1 --2- 172.21.15.1:0/360923931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb644069e60 0x7fb64419a070 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:07.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:07.955+0000 7fb64b600700 1 -- 172.21.15.1:0/360923931 >> 172.21.15.1:0/360923931 conn(0x7fb644076ea0 msgr2=0x7fb6441099c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:07.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:07.956+0000 7fb64b600700 1 -- 172.21.15.1:0/360923931 shutdown_connections 2024-04-26T01:40:07.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:07.956+0000 7fb64b600700 1 -- 172.21.15.1:0/360923931 wait complete. 2024-04-26T01:40:07.954 INFO:teuthology.orchestra.run.smithi001.stderr:adjusted tunables profile to default 2024-04-26T01:40:08.277 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:07 smithi001 ceph-mon[27822]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:40:08.278 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:07 smithi001 ceph-mon[27822]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:08.278 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:07 smithi001 ceph-mon[27822]: Updating smithi173:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:40:08.278 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:07 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/360923931' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T01:40:08.278 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:07 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:08.278 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:07 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:08.278 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:07 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:08.652 INFO:tasks.cephadm:Adding mon.a on smithi001 2024-04-26T01:40:08.652 INFO:tasks.cephadm:Adding mon.b on smithi097 2024-04-26T01:40:08.652 INFO:tasks.cephadm:Adding mon.c on smithi173 2024-04-26T01:40:08.652 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi001:172.21.15.1=a;smithi097:172.21.15.97=b;smithi173:172.21.15.173=c' 2024-04-26T01:40:08.988 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:40:09.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:08 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/360923931' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T01:40:09.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:08 smithi001 ceph-mon[27822]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:40:10.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:09 smithi001 ceph-mon[27822]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:10.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.385+0000 7fb8dea41700 1 -- 172.21.15.173:0/3610939230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8d8108550 msgr2=0x7fb8d8108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:10.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.385+0000 7fb8dea41700 1 --2- 172.21.15.173:0/3610939230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8d8108550 0x7fb8d8108930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009910 tx=0x7fb8c8009c20 comp rx=0 tx=0).stop 2024-04-26T01:40:10.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.386+0000 7fb8dea41700 1 -- 172.21.15.173:0/3610939230 shutdown_connections 2024-04-26T01:40:10.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.386+0000 7fb8dea41700 1 --2- 172.21.15.173:0/3610939230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8d8108550 0x7fb8d8108930 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:10.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.386+0000 7fb8dea41700 1 -- 172.21.15.173:0/3610939230 >> 172.21.15.173:0/3610939230 conn(0x7fb8d8077ac0 msgr2=0x7fb8d8077ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:10.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.386+0000 7fb8dea41700 1 -- 172.21.15.173:0/3610939230 shutdown_connections 2024-04-26T01:40:10.383 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.386+0000 7fb8dea41700 1 -- 172.21.15.173:0/3610939230 wait complete. 2024-04-26T01:40:10.384 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.386+0000 7fb8dea41700 1 Processor -- start 2024-04-26T01:40:10.384 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.386+0000 7fb8dea41700 1 -- start start 2024-04-26T01:40:10.384 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.387+0000 7fb8dea41700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8d8108550 0x7fb8d8197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:10.385 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.387+0000 7fb8dea41700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d810c9b0 con 0x7fb8d8108550 2024-04-26T01:40:10.385 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.387+0000 7fb8d7fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8d8108550 0x7fb8d8197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:10.385 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.387+0000 7fb8d7fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8d8108550 0x7fb8d8197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.173:45040/0 (socket says 172.21.15.173:45040) 2024-04-26T01:40:10.385 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.387+0000 7fb8d7fff700 1 -- 172.21.15.173:0/1572732588 learned_addr learned my addr 172.21.15.173:0/1572732588 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-26T01:40:10.386 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.388+0000 7fb8d7fff700 1 -- 172.21.15.173:0/1572732588 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d8198370 con 0x7fb8d8108550 2024-04-26T01:40:10.386 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.388+0000 7fb8d7fff700 1 --2- 172.21.15.173:0/1572732588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8d8108550 0x7fb8d8197e30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80056d0 tx=0x7fb8c80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:10.386 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.389+0000 7fb8d57fa700 1 -- 172.21.15.173:0/1572732588 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8c8026070 con 0x7fb8d8108550 2024-04-26T01:40:10.386 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.389+0000 7fb8dea41700 1 -- 172.21.15.173:0/1572732588 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d8198660 con 0x7fb8d8108550 2024-04-26T01:40:10.386 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.389+0000 7fb8dea41700 1 -- 172.21.15.173:0/1572732588 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d819c700 con 0x7fb8d8108550 2024-04-26T01:40:10.386 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.389+0000 7fb8d57fa700 1 -- 172.21.15.173:0/1572732588 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8c800bba0 con 0x7fb8d8108550 2024-04-26T01:40:10.386 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.389+0000 7fb8d57fa700 1 -- 172.21.15.173:0/1572732588 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8c8030060 con 0x7fb8d8108550 2024-04-26T01:40:10.387 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.389+0000 7fb8dea41700 1 -- 172.21.15.173:0/1572732588 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d8103cf0 con 0x7fb8d8108550 2024-04-26T01:40:10.387 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.390+0000 7fb8d57fa700 1 -- 172.21.15.173:0/1572732588 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7fb8c8029070 con 0x7fb8d8108550 2024-04-26T01:40:10.388 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.390+0000 7fb8d57fa700 1 --2- 172.21.15.173:0/1572732588 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb8c0039650 0x7fb8c003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:10.388 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.390+0000 7fb8d57fa700 1 -- 172.21.15.173:0/1572732588 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb8c801e070 con 0x7fb8d8108550 2024-04-26T01:40:10.388 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.391+0000 7fb8d77fe700 1 --2- 172.21.15.173:0/1572732588 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb8c0039650 0x7fb8c003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:10.389 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.392+0000 7fb8d77fe700 1 --2- 172.21.15.173:0/1572732588 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb8c0039650 0x7fb8c003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc006fd0 tx=0x7fb8cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:10.404 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.407+0000 7fb8d57fa700 1 -- 172.21.15.173:0/1572732588 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb8c8013070 con 0x7fb8d8108550 2024-04-26T01:40:10.612 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.615+0000 7fb8dea41700 1 -- 172.21.15.173:0/1572732588 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi001:172.21.15.1=a;smithi097:172.21.15.97=b;smithi173:172.21.15.173=c", "target": ["mon-mgr", ""]}) -- 0x7fb8d8199190 con 0x7fb8c0039650 2024-04-26T01:40:10.627 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.630+0000 7fb8d57fa700 1 -- 172.21.15.173:0/1572732588 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb8d8199190 con 0x7fb8c0039650 2024-04-26T01:40:10.627 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled mon update... 2024-04-26T01:40:10.630 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.632+0000 7fb8dea41700 1 -- 172.21.15.173:0/1572732588 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb8c0039650 msgr2=0x7fb8c003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:10.630 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.632+0000 7fb8dea41700 1 --2- 172.21.15.173:0/1572732588 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb8c0039650 0x7fb8c003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc006fd0 tx=0x7fb8cc006e40 comp rx=0 tx=0).stop 2024-04-26T01:40:10.630 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.633+0000 7fb8dea41700 1 -- 172.21.15.173:0/1572732588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8d8108550 msgr2=0x7fb8d8197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:10.630 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.633+0000 7fb8dea41700 1 --2- 172.21.15.173:0/1572732588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8d8108550 0x7fb8d8197e30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80056d0 tx=0x7fb8c80057b0 comp rx=0 tx=0).stop 2024-04-26T01:40:10.630 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.633+0000 7fb8dea41700 1 -- 172.21.15.173:0/1572732588 shutdown_connections 2024-04-26T01:40:10.630 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.633+0000 7fb8dea41700 1 --2- 172.21.15.173:0/1572732588 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb8c0039650 0x7fb8c003bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:10.630 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.633+0000 7fb8dea41700 1 --2- 172.21.15.173:0/1572732588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8d8108550 0x7fb8d8197e30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:10.630 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.633+0000 7fb8dea41700 1 -- 172.21.15.173:0/1572732588 >> 172.21.15.173:0/1572732588 conn(0x7fb8d8077ac0 msgr2=0x7fb8d8103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:10.631 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.633+0000 7fb8dea41700 1 -- 172.21.15.173:0/1572732588 shutdown_connections 2024-04-26T01:40:10.631 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:10.633+0000 7fb8dea41700 1 -- 172.21.15.173:0/1572732588 wait complete. 2024-04-26T01:40:11.230 DEBUG:teuthology.orchestra.run.smithi097:mon.b> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mon.b.service 2024-04-26T01:40:11.232 DEBUG:teuthology.orchestra.run.smithi173:mon.c> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mon.c.service 2024-04-26T01:40:11.234 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T01:40:11.234 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T01:40:12.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:11 smithi001 ceph-mon[27822]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi001:172.21.15.1=a;smithi097:172.21.15.97=b;smithi173:172.21.15.173=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:40:12.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:11 smithi001 ceph-mon[27822]: Saving service mon spec with placement smithi001:172.21.15.1=a;smithi097:172.21.15.97=b;smithi173:172.21.15.173=c;count:3 2024-04-26T01:40:12.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:11 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:11 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:40:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:11 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:11 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:40:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:11 smithi001 ceph-mon[27822]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:11 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:11 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:40:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:11 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:11 smithi001 ceph-mon[27822]: Deploying daemon mon.c on smithi173 2024-04-26T01:40:13.139 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:40:14.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:13 smithi001 ceph-mon[27822]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:15.001 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.003+0000 7f718b15c700 1 -- 172.21.15.173:0/525121096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7184103770 msgr2=0x7f7184103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:15.001 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.003+0000 7f718b15c700 1 --2- 172.21.15.173:0/525121096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7184103770 0x7f7184103b50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7178009910 tx=0x7f7178009c20 comp rx=0 tx=0).stop 2024-04-26T01:40:15.001 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.003+0000 7f718b15c700 1 -- 172.21.15.173:0/525121096 shutdown_connections 2024-04-26T01:40:15.001 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.003+0000 7f718b15c700 1 --2- 172.21.15.173:0/525121096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7184103770 0x7f7184103b50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:15.002 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.003+0000 7f718b15c700 1 -- 172.21.15.173:0/525121096 >> 172.21.15.173:0/525121096 conn(0x7f71840fcf80 msgr2=0x7f71840ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:15.002 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.003+0000 7f718b15c700 1 -- 172.21.15.173:0/525121096 shutdown_connections 2024-04-26T01:40:15.002 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.003+0000 7f718b15c700 1 -- 172.21.15.173:0/525121096 wait complete. 2024-04-26T01:40:15.002 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.003+0000 7f718b15c700 1 Processor -- start 2024-04-26T01:40:15.002 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.003+0000 7f718b15c700 1 -- start start 2024-04-26T01:40:15.002 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.004+0000 7f718b15c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7184103770 0x7f7184197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:15.002 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.004+0000 7f718b15c700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f718410caa0 con 0x7f7184103770 2024-04-26T01:40:15.002 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.004+0000 7f7188ef8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7184103770 0x7f7184197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:15.002 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.004+0000 7f7188ef8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7184103770 0x7f7184197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.173:45066/0 (socket says 172.21.15.173:45066) 2024-04-26T01:40:15.002 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.004+0000 7f7188ef8700 1 -- 172.21.15.173:0/3056466945 learned_addr learned my addr 172.21.15.173:0/3056466945 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-26T01:40:15.003 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.005+0000 7f7188ef8700 1 -- 172.21.15.173:0/3056466945 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7184198500 con 0x7f7184103770 2024-04-26T01:40:15.003 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.005+0000 7f7188ef8700 1 --2- 172.21.15.173:0/3056466945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7184103770 0x7f7184197fc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7178006010 tx=0x7f7178005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:15.003 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.005+0000 7f7181ffb700 1 -- 172.21.15.173:0/3056466945 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7178026070 con 0x7f7184103770 2024-04-26T01:40:15.003 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.005+0000 7f718b15c700 1 -- 172.21.15.173:0/3056466945 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7184198790 con 0x7f7184103770 2024-04-26T01:40:15.003 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.006+0000 7f718b15c700 1 -- 172.21.15.173:0/3056466945 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7184198bf0 con 0x7f7184103770 2024-04-26T01:40:15.004 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.006+0000 7f7181ffb700 1 -- 172.21.15.173:0/3056466945 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f717800bc00 con 0x7f7184103770 2024-04-26T01:40:15.004 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.006+0000 7f7181ffb700 1 -- 172.21.15.173:0/3056466945 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7178004830 con 0x7f7184103770 2024-04-26T01:40:15.004 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.006+0000 7f718b15c700 1 -- 172.21.15.173:0/3056466945 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7168005330 con 0x7f7184103770 2024-04-26T01:40:15.004 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.007+0000 7f7181ffb700 1 -- 172.21.15.173:0/3056466945 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f7178029070 con 0x7f7184103770 2024-04-26T01:40:15.004 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.007+0000 7f7181ffb700 1 --2- 172.21.15.173:0/3056466945 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f71740397f0 0x7f717403bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:15.004 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.007+0000 7f7181ffb700 1 -- 172.21.15.173:0/3056466945 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f717801e070 con 0x7f7184103770 2024-04-26T01:40:15.005 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.008+0000 7f7183fff700 1 --2- 172.21.15.173:0/3056466945 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f71740397f0 0x7f717403bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:15.006 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.009+0000 7f7183fff700 1 --2- 172.21.15.173:0/3056466945 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f71740397f0 0x7f717403bcb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7170006fd0 tx=0x7f7170006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:15.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.023+0000 7f7181ffb700 1 -- 172.21.15.173:0/3056466945 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7178013070 con 0x7f7184103770 2024-04-26T01:40:15.160 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.162+0000 7f7181ffb700 1 -- 172.21.15.173:0/3056466945 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f717800ec70 con 0x7f7184103770 2024-04-26T01:40:15.314 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:15.317+0000 7f718b15c700 1 -- 172.21.15.173:0/3056466945 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f71680058f0 con 0x7f7184103770 2024-04-26T01:40:20.184 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:20.186+0000 7f7181ffb700 1 -- 172.21.15.173:0/3056466945 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f7178038410 con 0x7f7184103770 2024-04-26T01:40:20.184 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-26T01:40:20.185 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":2,"fsid":"77cfcc80-036d-11ef-bc93-c7b262605968","modified":"2024-04-26T01:40:15.162257Z","created":"2024-04-26T01:37:59.408242Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:3300","nonce":0},{"type":"v1","addr":"172.21.15.173:6789","nonce":0}]},"addr":"172.21.15.173:6789/0","public_addr":"172.21.15.173:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T01:40:20.187 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:20.190+0000 7f718b15c700 1 -- 172.21.15.173:0/3056466945 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f71740397f0 msgr2=0x7f717403bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:20.187 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:20.190+0000 7f718b15c700 1 --2- 172.21.15.173:0/3056466945 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f71740397f0 0x7f717403bcb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7170006fd0 tx=0x7f7170006e40 comp rx=0 tx=0).stop 2024-04-26T01:40:20.187 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:20.190+0000 7f718b15c700 1 -- 172.21.15.173:0/3056466945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7184103770 msgr2=0x7f7184197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:20.187 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:20.190+0000 7f718b15c700 1 --2- 172.21.15.173:0/3056466945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7184103770 0x7f7184197fc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7178006010 tx=0x7f7178005780 comp rx=0 tx=0).stop 2024-04-26T01:40:20.187 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:20.190+0000 7f718b15c700 1 -- 172.21.15.173:0/3056466945 shutdown_connections 2024-04-26T01:40:20.187 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:20.190+0000 7f718b15c700 1 --2- 172.21.15.173:0/3056466945 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f71740397f0 0x7f717403bcb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:20.187 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:20.190+0000 7f718b15c700 1 --2- 172.21.15.173:0/3056466945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7184103770 0x7f7184197fc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:20.188 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:20.190+0000 7f718b15c700 1 -- 172.21.15.173:0/3056466945 >> 172.21.15.173:0/3056466945 conn(0x7f71840fcf80 msgr2=0x7f7184106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:20.188 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:20.190+0000 7f718b15c700 1 -- 172.21.15.173:0/3056466945 shutdown_connections 2024-04-26T01:40:20.188 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:20.190+0000 7f718b15c700 1 -- 172.21.15.173:0/3056466945 wait complete. 2024-04-26T01:40:20.188 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 2 2024-04-26T01:40:20.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:20.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:40:20.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: mon.a calling monitor election 2024-04-26T01:40:20.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:40:20.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: from='client.? 172.21.15.173:0/3056466945' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T01:40:20.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:40:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:40:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: mon.c calling monitor election 2024-04-26T01:40:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:40:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:40:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:40:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T01:40:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: monmap epoch 2 2024-04-26T01:40:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: fsid 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: last_changed 2024-04-26T01:40:15.162257+0000 2024-04-26T01:40:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: created 2024-04-26T01:37:59.408242+0000 2024-04-26T01:40:20.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: min_mon_release 19 (squid) 2024-04-26T01:40:20.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: election_strategy: 1 2024-04-26T01:40:20.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-04-26T01:40:20.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-04-26T01:40:20.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: fsmap 2024-04-26T01:40:20.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:40:20.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: mgrmap e13: x(active, since 59s) 2024-04-26T01:40:20.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: overall HEALTH_OK 2024-04-26T01:40:20.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:20.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:20.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:20.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:40:21.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:21 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:21.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:21 smithi001 ceph-mon[27822]: Deploying daemon mon.b on smithi097 2024-04-26T01:40:21.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:21 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:40:21.789 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T01:40:21.790 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T01:40:22.037 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:40:22.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:22 smithi001 ceph-mon[27822]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:23.346 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.347+0000 7f4b524cd700 1 -- 172.21.15.173:0/859032425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b4c101500 msgr2=0x7f4b4c1018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:23.346 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.347+0000 7f4b524cd700 1 --2- 172.21.15.173:0/859032425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b4c101500 0x7f4b4c1018e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009910 tx=0x7f4b40009c20 comp rx=0 tx=0).stop 2024-04-26T01:40:23.346 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.347+0000 7f4b524cd700 1 -- 172.21.15.173:0/859032425 shutdown_connections 2024-04-26T01:40:23.346 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.347+0000 7f4b524cd700 1 --2- 172.21.15.173:0/859032425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b4c101500 0x7f4b4c1018e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:23.346 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.347+0000 7f4b524cd700 1 -- 172.21.15.173:0/859032425 >> 172.21.15.173:0/859032425 conn(0x7f4b4c0fcf80 msgr2=0x7f4b4c0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:23.346 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.347+0000 7f4b524cd700 1 -- 172.21.15.173:0/859032425 shutdown_connections 2024-04-26T01:40:23.346 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.347+0000 7f4b524cd700 1 -- 172.21.15.173:0/859032425 wait complete. 2024-04-26T01:40:23.346 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.347+0000 7f4b524cd700 1 Processor -- start 2024-04-26T01:40:23.347 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.347+0000 7f4b524cd700 1 -- start start 2024-04-26T01:40:23.347 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.348+0000 7f4b524cd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b4c1a46a0 0x7f4b4c1a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:23.347 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.348+0000 7f4b524cd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b4c1a4fc0 0x7f4b4c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:23.347 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.348+0000 7f4b524cd700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c1130b0 con 0x7f4b4c1a4fc0 2024-04-26T01:40:23.347 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.348+0000 7f4b524cd700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c112f30 con 0x7f4b4c1a46a0 2024-04-26T01:40:23.347 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.348+0000 7f4b4bfff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b4c1a46a0 0x7f4b4c1a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:23.347 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.348+0000 7f4b4bfff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b4c1a46a0 0x7f4b4c1a4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:41366/0 (socket says 172.21.15.173:41366) 2024-04-26T01:40:23.347 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.348+0000 7f4b4bfff700 1 -- 172.21.15.173:0/3676604745 learned_addr learned my addr 172.21.15.173:0/3676604745 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-26T01:40:23.347 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.348+0000 7f4b4bfff700 1 -- 172.21.15.173:0/3676604745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b4c1a46a0 msgr2=0x7f4b4c1a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-26T01:40:23.347 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.348+0000 7f4b4bfff700 1 -- 172.21.15.173:0/3676604745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b4c1a46a0 msgr2=0x7f4b4c1a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T01:40:23.347 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.348+0000 7f4b4bfff700 1 --2- 172.21.15.173:0/3676604745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b4c1a46a0 0x7f4b4c1a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T01:40:23.347 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.348+0000 7f4b4bfff700 1 --2- 172.21.15.173:0/3676604745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b4c1a46a0 0x7f4b4c1a4a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:40:23.347 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.348+0000 7f4b4b7fe700 1 --2- 172.21.15.173:0/3676604745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b4c1a4fc0 0x7f4b4c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:23.347 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.349+0000 7f4b4b7fe700 1 -- 172.21.15.173:0/3676604745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b4c1a46a0 msgr2=0x7f4b4c1a4a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:40:23.348 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.349+0000 7f4b4b7fe700 1 --2- 172.21.15.173:0/3676604745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b4c1a46a0 0x7f4b4c1a4a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:23.348 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.349+0000 7f4b4b7fe700 1 -- 172.21.15.173:0/3676604745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4c19eed0 con 0x7f4b4c1a4fc0 2024-04-26T01:40:23.349 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.349+0000 7f4b4b7fe700 1 --2- 172.21.15.173:0/3676604745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b4c1a4fc0 0x7f4b4c19e820 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00b4d0 tx=0x7f4b3c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:23.349 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.349+0000 7f4b497fa700 1 -- 172.21.15.173:0/3676604745 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4b3c011630 con 0x7f4b4c1a4fc0 2024-04-26T01:40:23.349 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.350+0000 7f4b524cd700 1 -- 172.21.15.173:0/3676604745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4c19f1c0 con 0x7f4b4c1a4fc0 2024-04-26T01:40:23.349 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.350+0000 7f4b524cd700 1 -- 172.21.15.173:0/3676604745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4c1a94f0 con 0x7f4b4c1a4fc0 2024-04-26T01:40:23.349 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.350+0000 7f4b497fa700 1 -- 172.21.15.173:0/3676604745 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b3c011c80 con 0x7f4b4c1a4fc0 2024-04-26T01:40:23.349 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.350+0000 7f4b497fa700 1 -- 172.21.15.173:0/3676604745 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4b3c01adc0 con 0x7f4b4c1a4fc0 2024-04-26T01:40:23.349 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.350+0000 7f4b524cd700 1 -- 172.21.15.173:0/3676604745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b4c06b530 con 0x7f4b4c1a4fc0 2024-04-26T01:40:23.349 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.351+0000 7f4b497fa700 1 -- 172.21.15.173:0/3676604745 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f4b3c018020 con 0x7f4b4c1a4fc0 2024-04-26T01:40:23.350 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.351+0000 7f4b497fa700 1 --2- 172.21.15.173:0/3676604745 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f4b340393f0 0x7f4b3403b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:23.350 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.351+0000 7f4b497fa700 1 -- 172.21.15.173:0/3676604745 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4b3c04c330 con 0x7f4b4c1a4fc0 2024-04-26T01:40:23.350 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.352+0000 7f4b4bfff700 1 --2- 172.21.15.173:0/3676604745 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f4b340393f0 0x7f4b3403b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:23.350 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.353+0000 7f4b4bfff700 1 --2- 172.21.15.173:0/3676604745 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f4b340393f0 0x7f4b3403b8b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000b5c0 tx=0x7f4b4001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:23.363 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.367+0000 7f4b497fa700 1 -- 172.21.15.173:0/3676604745 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b3c01f020 con 0x7f4b4c1a4fc0 2024-04-26T01:40:23.645 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.648+0000 7f4b524cd700 1 -- 172.21.15.173:0/3676604745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4b4c19fd50 con 0x7f4b4c1a4fc0 2024-04-26T01:40:23.646 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.649+0000 7f4b497fa700 1 -- 172.21.15.173:0/3676604745 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f4b3c00e260 con 0x7f4b4c1a4fc0 2024-04-26T01:40:23.646 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-26T01:40:23.646 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":2,"fsid":"77cfcc80-036d-11ef-bc93-c7b262605968","modified":"2024-04-26T01:40:15.162257Z","created":"2024-04-26T01:37:59.408242Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:3300","nonce":0},{"type":"v1","addr":"172.21.15.173:6789","nonce":0}]},"addr":"172.21.15.173:6789/0","public_addr":"172.21.15.173:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T01:40:23.648 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.651+0000 7f4b524cd700 1 -- 172.21.15.173:0/3676604745 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f4b340393f0 msgr2=0x7f4b3403b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:23.648 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.651+0000 7f4b524cd700 1 --2- 172.21.15.173:0/3676604745 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f4b340393f0 0x7f4b3403b8b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000b5c0 tx=0x7f4b4001a040 comp rx=0 tx=0).stop 2024-04-26T01:40:23.649 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.651+0000 7f4b524cd700 1 -- 172.21.15.173:0/3676604745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b4c1a4fc0 msgr2=0x7f4b4c19e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:23.649 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.651+0000 7f4b524cd700 1 --2- 172.21.15.173:0/3676604745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b4c1a4fc0 0x7f4b4c19e820 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00b4d0 tx=0x7f4b3c00b890 comp rx=0 tx=0).stop 2024-04-26T01:40:23.649 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.651+0000 7f4b524cd700 1 -- 172.21.15.173:0/3676604745 shutdown_connections 2024-04-26T01:40:23.649 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.651+0000 7f4b524cd700 1 --2- 172.21.15.173:0/3676604745 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f4b340393f0 0x7f4b3403b8b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:23.649 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.651+0000 7f4b524cd700 1 --2- 172.21.15.173:0/3676604745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b4c1a46a0 0x7f4b4c1a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:23.649 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.651+0000 7f4b524cd700 1 --2- 172.21.15.173:0/3676604745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b4c1a4fc0 0x7f4b4c19e820 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:23.649 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.651+0000 7f4b524cd700 1 -- 172.21.15.173:0/3676604745 >> 172.21.15.173:0/3676604745 conn(0x7f4b4c0fcf80 msgr2=0x7f4b4c0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:23.649 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.651+0000 7f4b524cd700 1 -- 172.21.15.173:0/3676604745 shutdown_connections 2024-04-26T01:40:23.649 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:23.651+0000 7f4b524cd700 1 -- 172.21.15.173:0/3676604745 wait complete. 2024-04-26T01:40:23.650 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 2 2024-04-26T01:40:24.017 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 podman[32835]: 2024-04-26 01:40:24.020643015 +0000 UTC m=+0.142983964 container create e232f88726108e2e1027d8409f615909f84d343e626049130fe45b6da6800de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-26T01:40:24.267 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 podman[32835]: 2024-04-26 01:40:24.206232303 +0000 UTC m=+0.328573254 container init e232f88726108e2e1027d8409f615909f84d343e626049130fe45b6da6800de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-b, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-04-26T01:40:24.267 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 podman[32835]: 2024-04-26 01:40:24.209065225 +0000 UTC m=+0.331406178 container start e232f88726108e2e1027d8409f615909f84d343e626049130fe45b6da6800de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-b, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-26T01:40:24.267 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T01:40:24.267 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-26T01:40:24.268 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: pidfile_write: ignore empty --pid-file 2024-04-26T01:40:24.268 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: load: jerasure load: lrc 2024-04-26T01:40:24.268 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: RocksDB version: 7.9.2 2024-04-26T01:40:24.268 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Git sha 0 2024-04-26T01:40:24.268 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-26T01:40:24.268 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: DB SUMMARY 2024-04-26T01:40:24.268 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: DB Session ID: 4W84TM6MGZ4H76I5QD79 2024-04-26T01:40:24.268 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: CURRENT file: CURRENT 2024-04-26T01:40:24.268 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T01:40:24.268 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-26T01:40:24.268 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-26T01:40:24.268 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-26T01:40:24.268 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.error_if_exists: 0 2024-04-26T01:40:24.268 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.create_if_missing: 0 2024-04-26T01:40:24.269 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.paranoid_checks: 1 2024-04-26T01:40:24.269 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T01:40:24.269 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T01:40:24.269 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T01:40:24.269 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.env: 0x559b9d174cc0 2024-04-26T01:40:24.269 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T01:40:24.269 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.info_log: 0x559b9ea65be0 2024-04-26T01:40:24.269 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T01:40:24.269 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.statistics: (nil) 2024-04-26T01:40:24.269 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.use_fsync: 0 2024-04-26T01:40:24.269 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_log_file_size: 0 2024-04-26T01:40:24.269 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T01:40:24.269 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T01:40:24.270 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T01:40:24.270 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T01:40:24.270 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.allow_fallocate: 1 2024-04-26T01:40:24.270 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T01:40:24.270 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T01:40:24.270 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.use_direct_reads: 0 2024-04-26T01:40:24.270 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T01:40:24.270 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T01:40:24.270 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.db_log_dir: 2024-04-26T01:40:24.270 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.wal_dir: 2024-04-26T01:40:24.270 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T01:40:24.270 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T01:40:24.270 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T01:40:24.270 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T01:40:24.271 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T01:40:24.271 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T01:40:24.271 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T01:40:24.271 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T01:40:24.271 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.write_buffer_manager: 0x559b9e9f90e0 2024-04-26T01:40:24.271 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T01:40:24.271 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T01:40:24.271 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T01:40:24.271 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T01:40:24.271 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T01:40:24.271 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T01:40:24.271 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T01:40:24.271 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T01:40:24.271 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.unordered_write: 0 2024-04-26T01:40:24.271 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T01:40:24.272 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T01:40:24.272 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T01:40:24.272 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T01:40:24.272 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.row_cache: None 2024-04-26T01:40:24.272 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.wal_filter: None 2024-04-26T01:40:24.273 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T01:40:24.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T01:40:24.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.two_write_queues: 0 2024-04-26T01:40:24.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T01:40:24.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.wal_compression: 0 2024-04-26T01:40:24.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.atomic_flush: 0 2024-04-26T01:40:24.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T01:40:24.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T01:40:24.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T01:40:24.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.log_readahead_size: 0 2024-04-26T01:40:24.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T01:40:24.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T01:40:24.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T01:40:24.275 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T01:40:24.275 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T01:40:24.275 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T01:40:24.275 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T01:40:24.275 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_background_jobs: 2 2024-04-26T01:40:24.275 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_background_compactions: -1 2024-04-26T01:40:24.275 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_subcompactions: 1 2024-04-26T01:40:24.275 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T01:40:24.275 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T01:40:24.275 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T01:40:24.275 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T01:40:24.275 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T01:40:24.275 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T01:40:24.275 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T01:40:24.276 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T01:40:24.276 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_open_files: -1 2024-04-26T01:40:24.276 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T01:40:24.276 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T01:40:24.276 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T01:40:24.276 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T01:40:24.276 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_background_flushes: -1 2024-04-26T01:40:24.276 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Compression algorithms supported: 2024-04-26T01:40:24.276 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T01:40:24.276 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: kZSTD supported: 0 2024-04-26T01:40:24.276 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: kXpressCompression supported: 0 2024-04-26T01:40:24.276 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T01:40:24.276 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: kZlibCompression supported: 1 2024-04-26T01:40:24.276 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: kSnappyCompression supported: 1 2024-04-26T01:40:24.276 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: kLZ4Compression supported: 1 2024-04-26T01:40:24.277 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: kBZip2Compression supported: 0 2024-04-26T01:40:24.277 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T01:40:24.277 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T01:40:24.277 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-26T01:40:24.277 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T01:40:24.277 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T01:40:24.277 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.merge_operator: 2024-04-26T01:40:24.277 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compaction_filter: None 2024-04-26T01:40:24.277 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T01:40:24.277 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T01:40:24.277 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T01:40:24.277 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T01:40:24.277 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559b9ea65dc0) 2024-04-26T01:40:24.277 INFO:journalctl@ceph.mon.b.smithi097.stdout: cache_index_and_filter_blocks: 1 2024-04-26T01:40:24.277 INFO:journalctl@ceph.mon.b.smithi097.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T01:40:24.278 INFO:journalctl@ceph.mon.b.smithi097.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T01:40:24.278 INFO:journalctl@ceph.mon.b.smithi097.stdout: pin_top_level_index_and_filter: 1 2024-04-26T01:40:24.278 INFO:journalctl@ceph.mon.b.smithi097.stdout: index_type: 0 2024-04-26T01:40:24.278 INFO:journalctl@ceph.mon.b.smithi097.stdout: data_block_index_type: 0 2024-04-26T01:40:24.278 INFO:journalctl@ceph.mon.b.smithi097.stdout: index_shortening: 1 2024-04-26T01:40:24.278 INFO:journalctl@ceph.mon.b.smithi097.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T01:40:24.278 INFO:journalctl@ceph.mon.b.smithi097.stdout: checksum: 4 2024-04-26T01:40:24.278 INFO:journalctl@ceph.mon.b.smithi097.stdout: no_block_cache: 0 2024-04-26T01:40:24.278 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_cache: 0x559b9ea2b1f0 2024-04-26T01:40:24.278 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_cache_name: BinnedLRUCache 2024-04-26T01:40:24.278 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_cache_options: 2024-04-26T01:40:24.278 INFO:journalctl@ceph.mon.b.smithi097.stdout: capacity : 536870912 2024-04-26T01:40:24.278 INFO:journalctl@ceph.mon.b.smithi097.stdout: num_shard_bits : 4 2024-04-26T01:40:24.278 INFO:journalctl@ceph.mon.b.smithi097.stdout: strict_capacity_limit : 0 2024-04-26T01:40:24.279 INFO:journalctl@ceph.mon.b.smithi097.stdout: high_pri_pool_ratio: 0.000 2024-04-26T01:40:24.279 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_cache_compressed: (nil) 2024-04-26T01:40:24.279 INFO:journalctl@ceph.mon.b.smithi097.stdout: persistent_cache: (nil) 2024-04-26T01:40:24.279 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_size: 4096 2024-04-26T01:40:24.279 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_size_deviation: 10 2024-04-26T01:40:24.279 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_restart_interval: 16 2024-04-26T01:40:24.279 INFO:journalctl@ceph.mon.b.smithi097.stdout: index_block_restart_interval: 1 2024-04-26T01:40:24.279 INFO:journalctl@ceph.mon.b.smithi097.stdout: metadata_block_size: 4096 2024-04-26T01:40:24.279 INFO:journalctl@ceph.mon.b.smithi097.stdout: partition_filters: 0 2024-04-26T01:40:24.279 INFO:journalctl@ceph.mon.b.smithi097.stdout: use_delta_encoding: 1 2024-04-26T01:40:24.279 INFO:journalctl@ceph.mon.b.smithi097.stdout: filter_policy: bloomfilter 2024-04-26T01:40:24.279 INFO:journalctl@ceph.mon.b.smithi097.stdout: whole_key_filtering: 1 2024-04-26T01:40:24.279 INFO:journalctl@ceph.mon.b.smithi097.stdout: verify_compression: 0 2024-04-26T01:40:24.279 INFO:journalctl@ceph.mon.b.smithi097.stdout: read_amp_bytes_per_bit: 0 2024-04-26T01:40:24.279 INFO:journalctl@ceph.mon.b.smithi097.stdout: format_version: 5 2024-04-26T01:40:24.280 INFO:journalctl@ceph.mon.b.smithi097.stdout: enable_index_compression: 1 2024-04-26T01:40:24.280 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_align: 0 2024-04-26T01:40:24.280 INFO:journalctl@ceph.mon.b.smithi097.stdout: max_auto_readahead_size: 262144 2024-04-26T01:40:24.280 INFO:journalctl@ceph.mon.b.smithi097.stdout: prepopulate_block_cache: 0 2024-04-26T01:40:24.280 INFO:journalctl@ceph.mon.b.smithi097.stdout: initial_auto_readahead_size: 8192 2024-04-26T01:40:24.280 INFO:journalctl@ceph.mon.b.smithi097.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T01:40:24.280 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T01:40:24.280 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T01:40:24.280 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compression: NoCompression 2024-04-26T01:40:24.280 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T01:40:24.280 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T01:40:24.280 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T01:40:24.280 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.num_levels: 7 2024-04-26T01:40:24.280 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T01:40:24.281 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T01:40:24.281 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T01:40:24.281 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T01:40:24.281 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T01:40:24.281 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T01:40:24.281 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T01:40:24.281 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T01:40:24.281 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T01:40:24.281 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T01:40:24.281 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T01:40:24.281 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T01:40:24.281 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T01:40:24.281 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T01:40:24.281 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T01:40:24.281 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T01:40:24.282 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T01:40:24.282 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T01:40:24.282 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T01:40:24.282 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T01:40:24.282 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T01:40:24.282 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T01:40:24.282 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T01:40:24.282 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T01:40:24.282 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T01:40:24.282 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T01:40:24.282 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T01:40:24.282 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T01:40:24.282 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T01:40:24.283 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T01:40:24.283 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T01:40:24.283 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T01:40:24.283 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T01:40:24.283 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T01:40:24.283 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T01:40:24.283 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T01:40:24.283 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T01:40:24.283 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T01:40:24.283 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T01:40:24.283 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T01:40:24.283 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T01:40:24.283 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T01:40:24.283 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T01:40:24.283 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T01:40:24.284 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T01:40:24.284 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T01:40:24.284 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T01:40:24.284 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T01:40:24.284 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T01:40:24.284 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T01:40:24.284 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T01:40:24.284 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T01:40:24.284 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T01:40:24.284 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T01:40:24.284 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.inplace_update_support: 0 2024-04-26T01:40:24.284 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T01:40:24.284 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T01:40:24.284 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T01:40:24.284 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T01:40:24.285 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.bloom_locality: 0 2024-04-26T01:40:24.285 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.max_successive_merges: 0 2024-04-26T01:40:24.285 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T01:40:24.285 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T01:40:24.285 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T01:40:24.285 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T01:40:24.285 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.ttl: 2592000 2024-04-26T01:40:24.285 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T01:40:24.285 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T01:40:24.285 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T01:40:24.285 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.enable_blob_files: false 2024-04-26T01:40:24.285 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.min_blob_size: 0 2024-04-26T01:40:24.285 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T01:40:24.286 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T01:40:24.286 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T01:40:24.286 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T01:40:24.286 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T01:40:24.286 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T01:40:24.286 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T01:40:24.286 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T01:40:24.286 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-26T01:40:24.286 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-26T01:40:24.286 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 25842cce-1af9-45f0-8c16-f9a1c0c1495b 2024-04-26T01:40:24.286 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714095624238701, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-26T01:40:24.286 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-26T01:40:24.286 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714095624239126, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714095624, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "25842cce-1af9-45f0-8c16-f9a1c0c1495b", "db_session_id": "4W84TM6MGZ4H76I5QD79", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-26T01:40:24.286 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714095624239205, "job": 1, "event": "recovery_finished"} 2024-04-26T01:40:24.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-26T01:40:24.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T01:40:24.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559b9eb40000 2024-04-26T01:40:24.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: DB pointer 0x559b9eb2a000 2024-04-26T01:40:24.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T01:40:24.287 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T01:40:24.287 INFO:journalctl@ceph.mon.b.smithi097.stdout: ** DB Stats ** 2024-04-26T01:40:24.287 INFO:journalctl@ceph.mon.b.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T01:40:24.287 INFO:journalctl@ceph.mon.b.smithi097.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T01:40:24.287 INFO:journalctl@ceph.mon.b.smithi097.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T01:40:24.287 INFO:journalctl@ceph.mon.b.smithi097.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T01:40:24.287 INFO:journalctl@ceph.mon.b.smithi097.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T01:40:24.287 INFO:journalctl@ceph.mon.b.smithi097.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T01:40:24.287 INFO:journalctl@ceph.mon.b.smithi097.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T01:40:24.288 INFO:journalctl@ceph.mon.b.smithi097.stdout: 2024-04-26T01:40:24.288 INFO:journalctl@ceph.mon.b.smithi097.stdout: ** Compaction Stats [default] ** 2024-04-26T01:40:24.288 INFO:journalctl@ceph.mon.b.smithi097.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T01:40:24.288 INFO:journalctl@ceph.mon.b.smithi097.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T01:40:24.288 INFO:journalctl@ceph.mon.b.smithi097.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T01:40:24.288 INFO:journalctl@ceph.mon.b.smithi097.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T01:40:24.288 INFO:journalctl@ceph.mon.b.smithi097.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T01:40:24.288 INFO:journalctl@ceph.mon.b.smithi097.stdout: 2024-04-26T01:40:24.288 INFO:journalctl@ceph.mon.b.smithi097.stdout: ** Compaction Stats [default] ** 2024-04-26T01:40:24.288 INFO:journalctl@ceph.mon.b.smithi097.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T01:40:24.288 INFO:journalctl@ceph.mon.b.smithi097.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T01:40:24.288 INFO:journalctl@ceph.mon.b.smithi097.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T01:40:24.288 INFO:journalctl@ceph.mon.b.smithi097.stdout: 2024-04-26T01:40:24.288 INFO:journalctl@ceph.mon.b.smithi097.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T01:40:24.288 INFO:journalctl@ceph.mon.b.smithi097.stdout: 2024-04-26T01:40:24.289 INFO:journalctl@ceph.mon.b.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T01:40:24.289 INFO:journalctl@ceph.mon.b.smithi097.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T01:40:24.289 INFO:journalctl@ceph.mon.b.smithi097.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T01:40:24.289 INFO:journalctl@ceph.mon.b.smithi097.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T01:40:24.289 INFO:journalctl@ceph.mon.b.smithi097.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T01:40:24.289 INFO:journalctl@ceph.mon.b.smithi097.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T01:40:24.289 INFO:journalctl@ceph.mon.b.smithi097.stdout: Cumulative compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T01:40:24.289 INFO:journalctl@ceph.mon.b.smithi097.stdout: Interval compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T01:40:24.289 INFO:journalctl@ceph.mon.b.smithi097.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T01:40:24.289 INFO:journalctl@ceph.mon.b.smithi097.stdout: Block cache BinnedLRUCache@0x559b9ea2b1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-26T01:40:24.289 INFO:journalctl@ceph.mon.b.smithi097.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-26T01:40:24.289 INFO:journalctl@ceph.mon.b.smithi097.stdout: 2024-04-26T01:40:24.289 INFO:journalctl@ceph.mon.b.smithi097.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T01:40:24.289 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-26T01:40:24.289 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: using public_addr v2:172.21.15.97:0/0 -> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] 2024-04-26T01:40:24.289 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: starting mon.b rank -1 at public addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] at bind addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:24.290 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: mon.b@-1(???) e0 preinit fsid 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:24.290 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: mon.b@-1(synchronizing).mds e1 new map 2024-04-26T01:40:24.290 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-26T01:40:24.290 INFO:journalctl@ceph.mon.b.smithi097.stdout: e1 2024-04-26T01:40:24.290 INFO:journalctl@ceph.mon.b.smithi097.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T01:40:24.290 INFO:journalctl@ceph.mon.b.smithi097.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T01:40:24.290 INFO:journalctl@ceph.mon.b.smithi097.stdout: legacy client fscid: -1 2024-04-26T01:40:24.290 INFO:journalctl@ceph.mon.b.smithi097.stdout: 2024-04-26T01:40:24.290 INFO:journalctl@ceph.mon.b.smithi097.stdout: No filesystems configured 2024-04-26T01:40:24.290 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-26T01:40:24.290 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-26T01:40:24.290 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-26T01:40:24.290 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-26T01:40:24.290 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-26T01:40:24.290 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-26T01:40:24.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T01:40:24.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T01:40:24.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T01:40:24.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T01:40:24.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: Added host smithi097 2024-04-26T01:40:24.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:24.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:24.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:40:24.291 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:24.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:40:24.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:24.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:40:24.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-26T01:40:24.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: Updating smithi097:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:40:24.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:40:24.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:24.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: Updating smithi097:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:40:24.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi173", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:40:24.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: Deploying cephadm binary to smithi173 2024-04-26T01:40:24.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:24.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:24.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:40:24.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: Added host smithi173 2024-04-26T01:40:24.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:24.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:24.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:40:24.293 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:24.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi173", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:40:24.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:24.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:40:24.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-26T01:40:24.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: Updating smithi173:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:40:24.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:40:24.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:24.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: Updating smithi173:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:40:24.294 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 bash[32835]: e232f88726108e2e1027d8409f615909f84d343e626049130fe45b6da6800de2 2024-04-26T01:40:24.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/360923931' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T01:40:24.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/360923931' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T01:40:24.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:40:24.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:24.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi001:172.21.15.1=a;smithi097:172.21.15.97=b;smithi173:172.21.15.173=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:40:24.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: Saving service mon spec with placement smithi001:172.21.15.1=a;smithi097:172.21.15.97=b;smithi173:172.21.15.173=c;count:3 2024-04-26T01:40:24.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:40:24.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:24.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:40:24.295 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:24.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:40:24.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:24.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: Deploying daemon mon.c on smithi173 2024-04-26T01:40:24.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:24.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:24.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:40:24.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: mon.a calling monitor election 2024-04-26T01:40:24.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:40:24.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='client.? 172.21.15.173:0/3056466945' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T01:40:24.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:40:24.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:24.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:40:24.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: mon.c calling monitor election 2024-04-26T01:40:24.296 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:40:24.297 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:24.297 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:40:24.297 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:40:24.297 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T01:40:24.297 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: monmap epoch 2 2024-04-26T01:40:24.297 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: fsid 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:24.297 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: last_changed 2024-04-26T01:40:15.162257+0000 2024-04-26T01:40:24.297 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: created 2024-04-26T01:37:59.408242+0000 2024-04-26T01:40:24.297 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: min_mon_release 19 (squid) 2024-04-26T01:40:24.297 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: election_strategy: 1 2024-04-26T01:40:24.297 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-04-26T01:40:24.297 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-04-26T01:40:24.297 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: fsmap 2024-04-26T01:40:24.297 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:40:24.298 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: mgrmap e13: x(active, since 59s) 2024-04-26T01:40:24.298 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: overall HEALTH_OK 2024-04-26T01:40:24.298 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.298 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.298 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:24.298 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:40:24.298 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:24.298 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: Deploying daemon mon.b on smithi097 2024-04-26T01:40:24.298 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:40:24.298 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:24.298 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:24.298 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: from='client.? 172.21.15.173:0/3676604745' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T01:40:24.298 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 ceph-mon[32853]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-26T01:40:24.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:24 smithi173 ceph-mon[32553]: from='client.? 172.21.15.173:0/3676604745' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T01:40:24.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:24 smithi001 ceph-mon[27822]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:24.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:24 smithi001 ceph-mon[27822]: from='client.? 172.21.15.173:0/3676604745' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T01:40:24.731 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:24 smithi097 systemd[1]: Started Ceph mon.b for 77cfcc80-036d-11ef-bc93-c7b262605968. 2024-04-26T01:40:25.159 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T01:40:25.160 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T01:40:25.402 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:40:29.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: mon.c calling monitor election 2024-04-26T01:40:29.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:40:29.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: mon.a calling monitor election 2024-04-26T01:40:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:40:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T01:40:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: monmap epoch 3 2024-04-26T01:40:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: fsid 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:29.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: last_changed 2024-04-26T01:40:24.277378+0000 2024-04-26T01:40:29.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: created 2024-04-26T01:37:59.408242+0000 2024-04-26T01:40:29.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: min_mon_release 19 (squid) 2024-04-26T01:40:29.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: election_strategy: 1 2024-04-26T01:40:29.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-04-26T01:40:29.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-04-26T01:40:29.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-04-26T01:40:29.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: fsmap 2024-04-26T01:40:29.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:40:29.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: mgrmap e13: x(active, since 68s) 2024-04-26T01:40:29.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: overall HEALTH_OK 2024-04-26T01:40:29.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:29.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:29.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:29.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: mon.c calling monitor election 2024-04-26T01:40:29.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:40:29.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: mon.a calling monitor election 2024-04-26T01:40:29.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:29.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:40:29.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:29.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:29.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:29.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:29.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:29.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:29.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T01:40:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: monmap epoch 3 2024-04-26T01:40:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: fsid 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: last_changed 2024-04-26T01:40:24.277378+0000 2024-04-26T01:40:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: created 2024-04-26T01:37:59.408242+0000 2024-04-26T01:40:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: min_mon_release 19 (squid) 2024-04-26T01:40:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: election_strategy: 1 2024-04-26T01:40:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-04-26T01:40:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-04-26T01:40:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-04-26T01:40:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: fsmap 2024-04-26T01:40:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:40:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: mgrmap e13: x(active, since 68s) 2024-04-26T01:40:29.818 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: overall HEALTH_OK 2024-04-26T01:40:29.818 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:29.818 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:29.818 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:29.918 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.918+0000 7f8994693700 1 -- 172.21.15.173:0/4171523660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f89780035c0 msgr2=0x7f8978005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:29.919 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.918+0000 7f8994693700 1 --2- 172.21.15.173:0/4171523660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f89780035c0 0x7f8978005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f898c101360 tx=0x7f897c00feb0 comp rx=0 tx=0).stop 2024-04-26T01:40:29.919 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.918+0000 7f8994693700 1 -- 172.21.15.173:0/4171523660 shutdown_connections 2024-04-26T01:40:29.919 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.918+0000 7f8994693700 1 --2- 172.21.15.173:0/4171523660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f89780035c0 0x7f8978005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:29.919 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.918+0000 7f8994693700 1 -- 172.21.15.173:0/4171523660 >> 172.21.15.173:0/4171523660 conn(0x7f898c0fcf20 msgr2=0x7f898c0ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:29.919 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.918+0000 7f8994693700 1 -- 172.21.15.173:0/4171523660 shutdown_connections 2024-04-26T01:40:29.919 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.918+0000 7f8994693700 1 -- 172.21.15.173:0/4171523660 wait complete. 2024-04-26T01:40:29.919 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.919+0000 7f8994693700 1 Processor -- start 2024-04-26T01:40:29.919 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.919+0000 7f8994693700 1 -- start start 2024-04-26T01:40:29.919 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.919+0000 7f8994693700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89780035c0 0x7f898c1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:29.919 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.919+0000 7f8994693700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f898c198630 0x7f898c19c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:29.919 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.919+0000 7f8994693700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f898c19cf00 0x7f898c19d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:29.920 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.919+0000 7f8994693700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f898c109340 con 0x7f898c198630 2024-04-26T01:40:29.920 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.919+0000 7f8994693700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f898c1091c0 con 0x7f898c19cf00 2024-04-26T01:40:29.920 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.919+0000 7f8994693700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f898c1094c0 con 0x7f89780035c0 2024-04-26T01:40:29.920 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.919+0000 7f899242f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89780035c0 0x7f898c1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:29.920 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.919+0000 7f899242f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89780035c0 0x7f898c1980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:53434/0 (socket says 172.21.15.173:53434) 2024-04-26T01:40:29.920 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.919+0000 7f899242f700 1 -- 172.21.15.173:0/2391570223 learned_addr learned my addr 172.21.15.173:0/2391570223 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-26T01:40:29.920 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.919+0000 7f899242f700 1 -- 172.21.15.173:0/2391570223 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89780035c0 msgr2=0x7f898c1980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-26T01:40:29.920 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.919+0000 7f899242f700 1 -- 172.21.15.173:0/2391570223 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89780035c0 msgr2=0x7f898c1980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T01:40:29.920 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.919+0000 7f899242f700 1 --2- 172.21.15.173:0/2391570223 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89780035c0 0x7f898c1980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T01:40:29.920 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.919+0000 7f8991c2e700 1 --2- 172.21.15.173:0/2391570223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f898c198630 0x7f898c19c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:29.920 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.919+0000 7f899242f700 1 --2- 172.21.15.173:0/2391570223 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89780035c0 0x7f898c1980f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:40:29.920 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.919+0000 7f8992c30700 1 --2- 172.21.15.173:0/2391570223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f898c19cf00 0x7f898c19d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:29.920 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.920+0000 7f8992c30700 1 -- 172.21.15.173:0/2391570223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f898c19cf00 msgr2=0x7f898c19d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-26T01:40:29.920 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.920+0000 7f8992c30700 1 -- 172.21.15.173:0/2391570223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f898c19cf00 msgr2=0x7f898c19d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T01:40:29.921 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.920+0000 7f8992c30700 1 --2- 172.21.15.173:0/2391570223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f898c19cf00 0x7f898c19d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T01:40:29.921 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.920+0000 7f8992c30700 1 --2- 172.21.15.173:0/2391570223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f898c19cf00 0x7f898c19d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:40:29.921 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.920+0000 7f8991c2e700 1 -- 172.21.15.173:0/2391570223 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89780035c0 msgr2=0x7f898c1980f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:40:29.921 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.920+0000 7f8991c2e700 1 --2- 172.21.15.173:0/2391570223 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89780035c0 0x7f898c1980f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:29.921 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.920+0000 7f8991c2e700 1 -- 172.21.15.173:0/2391570223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f898c19cf00 msgr2=0x7f898c19d3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:40:29.921 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.920+0000 7f8991c2e700 1 --2- 172.21.15.173:0/2391570223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f898c19cf00 0x7f898c19d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:29.921 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.920+0000 7f8991c2e700 1 -- 172.21.15.173:0/2391570223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898c1a4c70 con 0x7f898c198630 2024-04-26T01:40:29.921 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.920+0000 7f8991c2e700 1 --2- 172.21.15.173:0/2391570223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f898c198630 0x7f898c19c9c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f898800b550 tx=0x7f898800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:29.921 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.920+0000 7f89837fe700 1 -- 172.21.15.173:0/2391570223 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8988004230 con 0x7f898c198630 2024-04-26T01:40:29.921 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.920+0000 7f8994693700 1 -- 172.21.15.173:0/2391570223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898c1a4ea0 con 0x7f898c198630 2024-04-26T01:40:29.921 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.920+0000 7f89837fe700 1 -- 172.21.15.173:0/2391570223 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89880043d0 con 0x7f898c198630 2024-04-26T01:40:29.921 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.921+0000 7f89837fe700 1 -- 172.21.15.173:0/2391570223 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8988011b90 con 0x7f898c198630 2024-04-26T01:40:29.921 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.921+0000 7f8994693700 1 -- 172.21.15.173:0/2391570223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f898c1a53d0 con 0x7f898c198630 2024-04-26T01:40:29.921 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.922+0000 7f89837fe700 1 -- 172.21.15.173:0/2391570223 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f898801a400 con 0x7f898c198630 2024-04-26T01:40:29.923 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.922+0000 7f89837fe700 1 --2- 172.21.15.173:0/2391570223 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8970039ae0 0x7f897003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:29.923 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.922+0000 7f89837fe700 1 -- 172.21.15.173:0/2391570223 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f898804c490 con 0x7f898c198630 2024-04-26T01:40:29.923 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.922+0000 7f8994693700 1 -- 172.21.15.173:0/2391570223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f898c0640a0 con 0x7f898c198630 2024-04-26T01:40:29.923 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.922+0000 7f899242f700 1 --2- 172.21.15.173:0/2391570223 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8970039ae0 0x7f897003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:29.923 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.923+0000 7f899242f700 1 --2- 172.21.15.173:0/2391570223 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8970039ae0 0x7f897003bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f897c00fa60 tx=0x7f897c005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:29.927 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:29.930+0000 7f89837fe700 1 -- 172.21.15.173:0/2391570223 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f89880202a0 con 0x7f898c198630 2024-04-26T01:40:30.218 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:30.221+0000 7f8994693700 1 -- 172.21.15.173:0/2391570223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f898c1a5690 con 0x7f898c198630 2024-04-26T01:40:30.219 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:30.222+0000 7f89837fe700 1 -- 172.21.15.173:0/2391570223 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1275 (secure 0 0 0) 0x7f8988003be0 con 0x7f898c198630 2024-04-26T01:40:30.219 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-26T01:40:30.219 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":3,"fsid":"77cfcc80-036d-11ef-bc93-c7b262605968","modified":"2024-04-26T01:40:24.277378Z","created":"2024-04-26T01:37:59.408242Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:3300","nonce":0},{"type":"v1","addr":"172.21.15.173:6789","nonce":0}]},"addr":"172.21.15.173:6789/0","public_addr":"172.21.15.173:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T01:40:30.221 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:30.224+0000 7f8994693700 1 -- 172.21.15.173:0/2391570223 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8970039ae0 msgr2=0x7f897003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:30.221 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:30.224+0000 7f8994693700 1 --2- 172.21.15.173:0/2391570223 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8970039ae0 0x7f897003bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f897c00fa60 tx=0x7f897c005a90 comp rx=0 tx=0).stop 2024-04-26T01:40:30.221 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:30.224+0000 7f8994693700 1 -- 172.21.15.173:0/2391570223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f898c198630 msgr2=0x7f898c19c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:30.222 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:30.224+0000 7f8994693700 1 --2- 172.21.15.173:0/2391570223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f898c198630 0x7f898c19c9c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f898800b550 tx=0x7f898800b910 comp rx=0 tx=0).stop 2024-04-26T01:40:30.222 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:30.224+0000 7f8994693700 1 -- 172.21.15.173:0/2391570223 shutdown_connections 2024-04-26T01:40:30.222 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:30.224+0000 7f8994693700 1 --2- 172.21.15.173:0/2391570223 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8970039ae0 0x7f897003bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:30.222 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:30.224+0000 7f8994693700 1 --2- 172.21.15.173:0/2391570223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f898c19cf00 0x7f898c19d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:30.222 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:30.224+0000 7f8994693700 1 --2- 172.21.15.173:0/2391570223 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89780035c0 0x7f898c1980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:30.222 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:30.224+0000 7f8994693700 1 --2- 172.21.15.173:0/2391570223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f898c198630 0x7f898c19c9c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:30.222 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:30.224+0000 7f8994693700 1 -- 172.21.15.173:0/2391570223 >> 172.21.15.173:0/2391570223 conn(0x7f898c0fcf20 msgr2=0x7f898c106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:30.222 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:30.224+0000 7f8994693700 1 -- 172.21.15.173:0/2391570223 shutdown_connections 2024-04-26T01:40:30.222 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:30.224+0000 7f8994693700 1 -- 172.21.15.173:0/2391570223 wait complete. 2024-04-26T01:40:30.223 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 3 2024-04-26T01:40:30.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:30 smithi173 ceph-mon[32553]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:30.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:30 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:30.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:30 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:40:30.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:30 smithi173 ceph-mon[32553]: from='client.? 172.21.15.173:0/2391570223' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T01:40:30.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:30 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:30.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:30 smithi001 ceph-mon[27822]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:30.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:30 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:30.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:30 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:40:30.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:30 smithi001 ceph-mon[27822]: from='client.? 172.21.15.173:0/2391570223' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T01:40:30.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:30 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:30.851 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-26T01:40:30.852 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-26T01:40:31.309 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:40:31.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: mon.c calling monitor election 2024-04-26T01:40:31.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:40:31.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: mon.a calling monitor election 2024-04-26T01:40:31.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:31.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:40:31.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:31.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:31.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:31.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:31.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:31.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:31.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:31.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T01:40:31.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: monmap epoch 3 2024-04-26T01:40:31.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: fsid 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:31.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: last_changed 2024-04-26T01:40:24.277378+0000 2024-04-26T01:40:31.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: created 2024-04-26T01:37:59.408242+0000 2024-04-26T01:40:31.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: min_mon_release 19 (squid) 2024-04-26T01:40:31.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: election_strategy: 1 2024-04-26T01:40:31.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-04-26T01:40:31.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-04-26T01:40:31.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-04-26T01:40:31.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: fsmap 2024-04-26T01:40:31.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:40:31.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: mgrmap e13: x(active, since 68s) 2024-04-26T01:40:31.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: overall HEALTH_OK 2024-04-26T01:40:31.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:31.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:31.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:31.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:31.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:31.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:40:31.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: from='client.? 172.21.15.173:0/2391570223' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T01:40:31.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:32.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: mon.b calling monitor election 2024-04-26T01:40:32.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:32.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: mon.b calling monitor election 2024-04-26T01:40:32.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: mon.a calling monitor election 2024-04-26T01:40:32.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: mon.c calling monitor election 2024-04-26T01:40:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T01:40:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: monmap epoch 3 2024-04-26T01:40:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: fsid 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: last_changed 2024-04-26T01:40:24.277378+0000 2024-04-26T01:40:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: created 2024-04-26T01:37:59.408242+0000 2024-04-26T01:40:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: min_mon_release 19 (squid) 2024-04-26T01:40:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: election_strategy: 1 2024-04-26T01:40:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-04-26T01:40:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-04-26T01:40:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-04-26T01:40:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: fsmap 2024-04-26T01:40:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:40:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: mgrmap e13: x(active, since 70s) 2024-04-26T01:40:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: overall HEALTH_OK 2024-04-26T01:40:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:32.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:32.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:32.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:31 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:40:32.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: mon.b calling monitor election 2024-04-26T01:40:32.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:32.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: mon.b calling monitor election 2024-04-26T01:40:32.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: mon.a calling monitor election 2024-04-26T01:40:32.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: mon.c calling monitor election 2024-04-26T01:40:32.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T01:40:32.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: monmap epoch 3 2024-04-26T01:40:32.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: fsid 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:32.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: last_changed 2024-04-26T01:40:24.277378+0000 2024-04-26T01:40:32.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: created 2024-04-26T01:37:59.408242+0000 2024-04-26T01:40:32.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: min_mon_release 19 (squid) 2024-04-26T01:40:32.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: election_strategy: 1 2024-04-26T01:40:32.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-04-26T01:40:32.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-04-26T01:40:32.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-04-26T01:40:32.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: fsmap 2024-04-26T01:40:32.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:40:32.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: mgrmap e13: x(active, since 70s) 2024-04-26T01:40:32.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: overall HEALTH_OK 2024-04-26T01:40:32.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:32.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:32.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:32.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:40:32.315 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: mon.b calling monitor election 2024-04-26T01:40:32.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:32.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: mon.b calling monitor election 2024-04-26T01:40:32.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: mon.a calling monitor election 2024-04-26T01:40:32.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: mon.c calling monitor election 2024-04-26T01:40:32.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T01:40:32.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: monmap epoch 3 2024-04-26T01:40:32.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: fsid 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:32.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: last_changed 2024-04-26T01:40:24.277378+0000 2024-04-26T01:40:32.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: created 2024-04-26T01:37:59.408242+0000 2024-04-26T01:40:32.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: min_mon_release 19 (squid) 2024-04-26T01:40:32.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: election_strategy: 1 2024-04-26T01:40:32.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-04-26T01:40:32.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: 1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.c 2024-04-26T01:40:32.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-04-26T01:40:32.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: fsmap 2024-04-26T01:40:32.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:40:32.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: mgrmap e13: x(active, since 70s) 2024-04-26T01:40:32.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: overall HEALTH_OK 2024-04-26T01:40:32.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:32.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:32.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:32.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:31 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:40:33.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.437+0000 7faa17a00700 1 -- 172.21.15.1:0/657646493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa101084e0 msgr2=0x7faa101088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:33.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.437+0000 7faa17a00700 1 --2- 172.21.15.1:0/657646493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa101084e0 0x7faa101088c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faa04009910 tx=0x7faa04009c20 comp rx=0 tx=0).stop 2024-04-26T01:40:33.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.437+0000 7faa17a00700 1 -- 172.21.15.1:0/657646493 shutdown_connections 2024-04-26T01:40:33.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.437+0000 7faa17a00700 1 --2- 172.21.15.1:0/657646493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa101084e0 0x7faa101088c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:33.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.437+0000 7faa17a00700 1 -- 172.21.15.1:0/657646493 >> 172.21.15.1:0/657646493 conn(0x7faa100fdf80 msgr2=0x7faa101003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:33.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.437+0000 7faa17a00700 1 -- 172.21.15.1:0/657646493 shutdown_connections 2024-04-26T01:40:33.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.437+0000 7faa17a00700 1 -- 172.21.15.1:0/657646493 wait complete. 2024-04-26T01:40:33.436 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:40:33 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:40:33.278+0000 7f8140a00700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-26T01:40:33.436 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:33 smithi001 ceph-mon[27822]: Updating smithi001:/etc/ceph/ceph.conf 2024-04-26T01:40:33.436 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:33 smithi001 ceph-mon[27822]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-26T01:40:33.436 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:33 smithi001 ceph-mon[27822]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-26T01:40:33.436 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:33 smithi001 ceph-mon[27822]: Updating smithi173:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:40:33.436 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:33 smithi001 ceph-mon[27822]: Updating smithi097:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:40:33.436 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:33 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:33.436 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:33 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:33.436 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:33 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:33.436 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:33 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:33.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:33 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:33.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:33 smithi001 ceph-mon[27822]: Updating smithi001:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:40:33.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:33 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:33.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:33 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:33.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:33 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:33.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.439+0000 7faa17a00700 1 Processor -- start 2024-04-26T01:40:33.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.439+0000 7faa17a00700 1 -- start start 2024-04-26T01:40:33.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.439+0000 7faa17a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa101084e0 0x7faa10074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:33.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.439+0000 7faa17a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa10075280 0x7faa10075700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:33.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.439+0000 7faa17a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa101a42f0 0x7faa101a66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:33.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.439+0000 7faa17a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faa1010c940 con 0x7faa101a42f0 2024-04-26T01:40:33.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.439+0000 7faa17a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faa1010c7c0 con 0x7faa101084e0 2024-04-26T01:40:33.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.439+0000 7faa17a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faa1010cac0 con 0x7faa10075280 2024-04-26T01:40:33.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.440+0000 7faa15000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa101084e0 0x7faa10074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:33.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.440+0000 7faa15a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa101a42f0 0x7faa101a66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:33.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.440+0000 7faa15000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa101084e0 0x7faa10074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:56906/0 (socket says 172.21.15.1:56906) 2024-04-26T01:40:33.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.440+0000 7faa15000700 1 -- 172.21.15.1:0/882249590 learned_addr learned my addr 172.21.15.1:0/882249590 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:40:33.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.440+0000 7faa15000700 1 -- 172.21.15.1:0/882249590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa10075280 msgr2=0x7faa10075700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:33.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.440+0000 7faa0fe00700 1 --2- 172.21.15.1:0/882249590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa10075280 0x7faa10075700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:33.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.441+0000 7faa15000700 1 --2- 172.21.15.1:0/882249590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa10075280 0x7faa10075700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:33.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.441+0000 7faa15000700 1 -- 172.21.15.1:0/882249590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa101a42f0 msgr2=0x7faa101a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:33.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.441+0000 7faa15000700 1 --2- 172.21.15.1:0/882249590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa101a42f0 0x7faa101a66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:33.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.441+0000 7faa15000700 1 -- 172.21.15.1:0/882249590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa101a6e10 con 0x7faa101084e0 2024-04-26T01:40:33.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.441+0000 7faa15a00700 1 --2- 172.21.15.1:0/882249590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa101a42f0 0x7faa101a66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:40:33.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.441+0000 7faa0fe00700 1 --2- 172.21.15.1:0/882249590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa10075280 0x7faa10075700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:40:33.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.441+0000 7faa15000700 1 --2- 172.21.15.1:0/882249590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa101084e0 0x7faa10074d40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7faa04004f80 tx=0x7faa04004fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:33.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.442+0000 7faa0d600700 1 -- 172.21.15.1:0/882249590 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa04026070 con 0x7faa101084e0 2024-04-26T01:40:33.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.442+0000 7faa0d600700 1 -- 172.21.15.1:0/882249590 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa0400b6f0 con 0x7faa101084e0 2024-04-26T01:40:33.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.442+0000 7faa17a00700 1 -- 172.21.15.1:0/882249590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa101a7100 con 0x7faa101084e0 2024-04-26T01:40:33.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.442+0000 7faa0d600700 1 -- 172.21.15.1:0/882249590 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa040038e0 con 0x7faa101084e0 2024-04-26T01:40:33.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.442+0000 7faa17a00700 1 -- 172.21.15.1:0/882249590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa101a7690 con 0x7faa101084e0 2024-04-26T01:40:33.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.443+0000 7faa0d600700 1 -- 172.21.15.1:0/882249590 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7faa04029070 con 0x7faa101084e0 2024-04-26T01:40:33.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.444+0000 7faa17a00700 1 -- 172.21.15.1:0/882249590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa100640a0 con 0x7faa101084e0 2024-04-26T01:40:33.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.444+0000 7faa0d600700 1 --2- 172.21.15.1:0/882249590 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa9f8039ae0 0x7fa9f803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:33.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.444+0000 7faa0d600700 1 -- 172.21.15.1:0/882249590 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7faa0404d400 con 0x7faa101084e0 2024-04-26T01:40:33.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.445+0000 7faa0fe00700 1 --2- 172.21.15.1:0/882249590 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa9f8039ae0 0x7fa9f803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:33.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.446+0000 7faa0fe00700 1 --2- 172.21.15.1:0/882249590 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa9f8039ae0 0x7fa9f803bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0079b0 tx=0x7fa9fc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:33.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.450+0000 7faa0d600700 1 -- 172.21.15.1:0/882249590 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa04013070 con 0x7faa101084e0 2024-04-26T01:40:33.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:33 smithi097 ceph-mon[32853]: Updating smithi001:/etc/ceph/ceph.conf 2024-04-26T01:40:33.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:33 smithi097 ceph-mon[32853]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-26T01:40:33.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:33 smithi097 ceph-mon[32853]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-26T01:40:33.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:33 smithi097 ceph-mon[32853]: Updating smithi173:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:40:33.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:33 smithi097 ceph-mon[32853]: Updating smithi097:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:40:33.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:33 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:33.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:33 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:33.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:33 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:33.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:33 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:33.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:33 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:33.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:33 smithi097 ceph-mon[32853]: Updating smithi001:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:40:33.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:33 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:33.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:33 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:33.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:33 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:33.494 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:33 smithi173 ceph-mon[32553]: Updating smithi001:/etc/ceph/ceph.conf 2024-04-26T01:40:33.495 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:33 smithi173 ceph-mon[32553]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-26T01:40:33.495 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:33 smithi173 ceph-mon[32553]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-26T01:40:33.495 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:33 smithi173 ceph-mon[32553]: Updating smithi173:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:40:33.495 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:33 smithi173 ceph-mon[32553]: Updating smithi097:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:40:33.495 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:33 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:33.495 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:33 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:33.495 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:33 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:33.495 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:33 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:33.495 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:33 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:40:33.495 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:33 smithi173 ceph-mon[32553]: Updating smithi001:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:40:33.495 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:33 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:33.495 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:33 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:33.495 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:33 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:33.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.688+0000 7faa17a00700 1 -- 172.21.15.1:0/882249590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7faa101a7940 con 0x7faa101084e0 2024-04-26T01:40:33.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.689+0000 7faa0d600700 1 -- 172.21.15.1:0/882249590 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7faa04030020 con 0x7faa101084e0 2024-04-26T01:40:33.686 INFO:teuthology.orchestra.run.smithi001.stdout:# minimal ceph.conf for 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:33.686 INFO:teuthology.orchestra.run.smithi001.stdout:[global] 2024-04-26T01:40:33.686 INFO:teuthology.orchestra.run.smithi001.stdout: fsid = 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:40:33.686 INFO:teuthology.orchestra.run.smithi001.stdout: mon_host = [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-04-26T01:40:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.692+0000 7faa17a00700 1 -- 172.21.15.1:0/882249590 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa9f8039ae0 msgr2=0x7fa9f803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.692+0000 7faa17a00700 1 --2- 172.21.15.1:0/882249590 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa9f8039ae0 0x7fa9f803bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0079b0 tx=0x7fa9fc008040 comp rx=0 tx=0).stop 2024-04-26T01:40:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.692+0000 7faa17a00700 1 -- 172.21.15.1:0/882249590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa101084e0 msgr2=0x7faa10074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.692+0000 7faa17a00700 1 --2- 172.21.15.1:0/882249590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa101084e0 0x7faa10074d40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7faa04004f80 tx=0x7faa04004fb0 comp rx=0 tx=0).stop 2024-04-26T01:40:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.692+0000 7faa17a00700 1 -- 172.21.15.1:0/882249590 shutdown_connections 2024-04-26T01:40:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.692+0000 7faa17a00700 1 --2- 172.21.15.1:0/882249590 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa9f8039ae0 0x7fa9f803bfa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.692+0000 7faa17a00700 1 --2- 172.21.15.1:0/882249590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa101a42f0 0x7faa101a66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.692+0000 7faa17a00700 1 --2- 172.21.15.1:0/882249590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa101084e0 0x7faa10074d40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.692+0000 7faa17a00700 1 --2- 172.21.15.1:0/882249590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa10075280 0x7faa10075700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.692+0000 7faa17a00700 1 -- 172.21.15.1:0/882249590 >> 172.21.15.1:0/882249590 conn(0x7faa100fdf80 msgr2=0x7faa10103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.692+0000 7faa17a00700 1 -- 172.21.15.1:0/882249590 shutdown_connections 2024-04-26T01:40:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:33.692+0000 7faa17a00700 1 -- 172.21.15.1:0/882249590 wait complete. 2024-04-26T01:40:34.425 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-26T01:40:34.425 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-04-26T01:40:34.425 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T01:40:34.501 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-04-26T01:40:34.501 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:40:34.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:34 smithi173 ceph-mon[32553]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:34.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:34 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:34.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:34 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:34.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:34 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:34.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:34 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:34.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:34 smithi173 ceph-mon[32553]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T01:40:34.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:34 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:40:34.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:34 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:40:34.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:34 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:34.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:34 smithi173 ceph-mon[32553]: Reconfiguring daemon mon.a on smithi001 2024-04-26T01:40:34.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:34 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/882249590' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:34.587 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:34 smithi001 ceph-mon[27822]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:34.587 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:34 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:34.587 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:34 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:34.587 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:34 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:34.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:34 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:34.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:34 smithi001 ceph-mon[27822]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T01:40:34.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:34 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:40:34.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:34 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:40:34.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:34 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:34.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:34 smithi001 ceph-mon[27822]: Reconfiguring daemon mon.a on smithi001 2024-04-26T01:40:34.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:34 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/882249590' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:34.636 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T01:40:34.636 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T01:40:34.661 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:34 smithi097 ceph-mon[32853]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:34.662 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:34 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:34.662 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:34 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:34.662 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:34 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:34.662 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:34 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:34.662 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:34 smithi097 ceph-mon[32853]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T01:40:34.662 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:34 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:40:34.662 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:34 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:40:34.662 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:34 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:34.662 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:34 smithi097 ceph-mon[32853]: Reconfiguring daemon mon.a on smithi001 2024-04-26T01:40:34.662 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:34 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/882249590' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:34.666 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T01:40:34.666 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:40:34.731 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-26T01:40:34.732 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T01:40:34.760 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-26T01:40:34.760 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:40:34.824 INFO:tasks.cephadm:Adding mgr.x on smithi001 2024-04-26T01:40:34.825 INFO:tasks.cephadm:Adding mgr.y on smithi097 2024-04-26T01:40:34.825 INFO:tasks.cephadm:Adding mgr.z on smithi173 2024-04-26T01:40:34.825 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi001=x;smithi097=y;smithi173=z' 2024-04-26T01:40:35.092 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:40:36.364 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:36 smithi173 ceph-mon[32553]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:36.364 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:36 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:36.364 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:36 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:36.364 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:36 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:40:36.365 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:36 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:40:36.365 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:36 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:36.562 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:36 smithi097 ceph-mon[32853]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:36.562 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:36 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:36.562 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:36 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:36.562 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:36 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:40:36.562 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:36 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:40:36.562 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:36 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:36.602 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.606+0000 7f8e7b0fb700 1 -- 172.21.15.173:0/1106564267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e741030f0 msgr2=0x7f8e741054e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:36.603 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.606+0000 7f8e7b0fb700 1 --2- 172.21.15.173:0/1106564267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e741030f0 0x7f8e741054e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8e68009910 tx=0x7f8e68009c20 comp rx=0 tx=0).stop 2024-04-26T01:40:36.603 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.606+0000 7f8e7b0fb700 1 -- 172.21.15.173:0/1106564267 shutdown_connections 2024-04-26T01:40:36.603 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.606+0000 7f8e7b0fb700 1 --2- 172.21.15.173:0/1106564267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e741030f0 0x7f8e741054e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:36.603 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.606+0000 7f8e7b0fb700 1 -- 172.21.15.173:0/1106564267 >> 172.21.15.173:0/1106564267 conn(0x7f8e740fca80 msgr2=0x7f8e740feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:36.603 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.606+0000 7f8e7b0fb700 1 -- 172.21.15.173:0/1106564267 shutdown_connections 2024-04-26T01:40:36.603 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.606+0000 7f8e7b0fb700 1 -- 172.21.15.173:0/1106564267 wait complete. 2024-04-26T01:40:36.605 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.607+0000 7f8e7b0fb700 1 Processor -- start 2024-04-26T01:40:36.605 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.607+0000 7f8e7b0fb700 1 -- start start 2024-04-26T01:40:36.605 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.607+0000 7f8e7b0fb700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e741030f0 0x7f8e74197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:36.605 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.607+0000 7f8e7b0fb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e74198290 0x7f8e7419d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:36.605 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.607+0000 7f8e7b0fb700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e74198710 0x7f8e7419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:36.605 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.607+0000 7f8e7b0fb700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8e74108500 con 0x7f8e741030f0 2024-04-26T01:40:36.605 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.607+0000 7f8e7b0fb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8e74108380 con 0x7f8e74198290 2024-04-26T01:40:36.605 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.607+0000 7f8e7b0fb700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8e74108680 con 0x7f8e74198710 2024-04-26T01:40:36.605 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.607+0000 7f8e79698700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e74198710 0x7f8e7419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:36.605 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.607+0000 7f8e79698700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e74198710 0x7f8e7419f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:54324/0 (socket says 172.21.15.173:54324) 2024-04-26T01:40:36.605 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.607+0000 7f8e79698700 1 -- 172.21.15.173:0/2018618643 learned_addr learned my addr 172.21.15.173:0/2018618643 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-26T01:40:36.606 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.607+0000 7f8e73fff700 1 --2- 172.21.15.173:0/2018618643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e74198290 0x7f8e7419d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:36.606 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.608+0000 7f8e79698700 1 -- 172.21.15.173:0/2018618643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e74198290 msgr2=0x7f8e7419d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:36.606 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.608+0000 7f8e79698700 1 --2- 172.21.15.173:0/2018618643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e74198290 0x7f8e7419d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:36.606 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.608+0000 7f8e79698700 1 -- 172.21.15.173:0/2018618643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e741030f0 msgr2=0x7f8e74197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:36.606 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.608+0000 7f8e79698700 1 --2- 172.21.15.173:0/2018618643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e741030f0 0x7f8e74197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:36.606 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.608+0000 7f8e79698700 1 -- 172.21.15.173:0/2018618643 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e741a0160 con 0x7f8e74198710 2024-04-26T01:40:36.606 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.608+0000 7f8e79698700 1 --2- 172.21.15.173:0/2018618643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e74198710 0x7f8e7419f890 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00ba50 tx=0x7f8e6c00be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:36.606 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.608+0000 7f8e71ffb700 1 -- 172.21.15.173:0/2018618643 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c019070 con 0x7f8e74198710 2024-04-26T01:40:36.606 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.608+0000 7f8e71ffb700 1 -- 172.21.15.173:0/2018618643 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e6c014440 con 0x7f8e74198710 2024-04-26T01:40:36.606 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.608+0000 7f8e7b0fb700 1 -- 172.21.15.173:0/2018618643 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e741a0450 con 0x7f8e74198710 2024-04-26T01:40:36.606 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.608+0000 7f8e71ffb700 1 -- 172.21.15.173:0/2018618643 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c007590 con 0x7f8e74198710 2024-04-26T01:40:36.606 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.608+0000 7f8e7b0fb700 1 -- 172.21.15.173:0/2018618643 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e741a0990 con 0x7f8e74198710 2024-04-26T01:40:36.606 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.609+0000 7f8e71ffb700 1 -- 172.21.15.173:0/2018618643 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f8e6c01c070 con 0x7f8e74198710 2024-04-26T01:40:36.607 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.609+0000 7f8e71ffb700 1 --2- 172.21.15.173:0/2018618643 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8e64039ae0 0x7f8e6403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:36.607 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.609+0000 7f8e71ffb700 1 -- 172.21.15.173:0/2018618643 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8e6c04c190 con 0x7f8e74198710 2024-04-26T01:40:36.607 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.609+0000 7f8e7b0fb700 1 -- 172.21.15.173:0/2018618643 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e740640a0 con 0x7f8e74198710 2024-04-26T01:40:36.607 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.610+0000 7f8e78e97700 1 --2- 172.21.15.173:0/2018618643 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8e64039ae0 0x7f8e6403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:36.608 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.611+0000 7f8e78e97700 1 --2- 172.21.15.173:0/2018618643 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8e64039ae0 0x7f8e6403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8e68005e00 tx=0x7f8e68005260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:36.610 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.614+0000 7f8e71ffb700 1 -- 172.21.15.173:0/2018618643 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e6c014910 con 0x7f8e74198710 2024-04-26T01:40:36.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:36 smithi001 ceph-mon[27822]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:36.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:36 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:36.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:36 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:36.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:36 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:40:36.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:36 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:40:36.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:36 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:36.821 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.824+0000 7f8e7b0fb700 1 -- 172.21.15.173:0/2018618643 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi097=y;smithi173=z", "target": ["mon-mgr", ""]}) -- 0x7f8e74062f40 con 0x7f8e64039ae0 2024-04-26T01:40:36.834 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.838+0000 7f8e71ffb700 1 -- 172.21.15.173:0/2018618643 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8e74062f40 con 0x7f8e64039ae0 2024-04-26T01:40:36.835 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled mgr update... 2024-04-26T01:40:36.837 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.841+0000 7f8e7b0fb700 1 -- 172.21.15.173:0/2018618643 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8e64039ae0 msgr2=0x7f8e6403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:36.837 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.841+0000 7f8e7b0fb700 1 --2- 172.21.15.173:0/2018618643 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8e64039ae0 0x7f8e6403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8e68005e00 tx=0x7f8e68005260 comp rx=0 tx=0).stop 2024-04-26T01:40:36.837 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.841+0000 7f8e7b0fb700 1 -- 172.21.15.173:0/2018618643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e74198710 msgr2=0x7f8e7419f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:36.837 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.841+0000 7f8e7b0fb700 1 --2- 172.21.15.173:0/2018618643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e74198710 0x7f8e7419f890 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00ba50 tx=0x7f8e6c00be10 comp rx=0 tx=0).stop 2024-04-26T01:40:36.837 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.841+0000 7f8e7b0fb700 1 -- 172.21.15.173:0/2018618643 shutdown_connections 2024-04-26T01:40:36.838 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.841+0000 7f8e7b0fb700 1 --2- 172.21.15.173:0/2018618643 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8e64039ae0 0x7f8e6403bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:36.838 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.841+0000 7f8e7b0fb700 1 --2- 172.21.15.173:0/2018618643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e74198710 0x7f8e7419f890 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:36.838 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.841+0000 7f8e7b0fb700 1 --2- 172.21.15.173:0/2018618643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e741030f0 0x7f8e74197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:36.838 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.841+0000 7f8e7b0fb700 1 --2- 172.21.15.173:0/2018618643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e74198290 0x7f8e7419d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:36.838 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.841+0000 7f8e7b0fb700 1 -- 172.21.15.173:0/2018618643 >> 172.21.15.173:0/2018618643 conn(0x7f8e740fca80 msgr2=0x7f8e740fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:36.838 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.841+0000 7f8e7b0fb700 1 -- 172.21.15.173:0/2018618643 shutdown_connections 2024-04-26T01:40:36.838 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:40:36.841+0000 7f8e7b0fb700 1 -- 172.21.15.173:0/2018618643 wait complete. 2024-04-26T01:40:37.452 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:37 smithi173 ceph-mon[32553]: Reconfiguring mon.b (monmap changed)... 2024-04-26T01:40:37.452 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:37 smithi173 ceph-mon[32553]: Reconfiguring daemon mon.b on smithi097 2024-04-26T01:40:37.452 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:37 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:37.452 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:37 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:37.453 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:37 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:37.453 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:37 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:40:37.453 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:37 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:40:37.453 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:37 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:37.482 DEBUG:teuthology.orchestra.run.smithi097:mgr.y> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mgr.y.service 2024-04-26T01:40:37.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:37 smithi097 ceph-mon[32853]: Reconfiguring mon.b (monmap changed)... 2024-04-26T01:40:37.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:37 smithi097 ceph-mon[32853]: Reconfiguring daemon mon.b on smithi097 2024-04-26T01:40:37.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:37 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:37.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:37 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:37.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:37 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:37.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:37 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:40:37.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:37 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:40:37.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:37 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:37.485 DEBUG:teuthology.orchestra.run.smithi173:mgr.z> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mgr.z.service 2024-04-26T01:40:37.487 INFO:tasks.cephadm:Deploying OSDs... 2024-04-26T01:40:37.488 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-04-26T01:40:37.488 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T01:40:37.549 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T01:40:37.549 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_1 2024-04-26T01:40:37.606 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T01:40:37.607 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:40:37.607 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 6h/6d Inode: 625 Links: 1 2024-04-26T01:40:37.607 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:40:37.607 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:40:37.607 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-04-26 01:39:34.600736453 +0000 2024-04-26T01:40:37.607 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-04-26 01:34:33.015049673 +0000 2024-04-26T01:40:37.607 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-04-26 01:34:33.015049673 +0000 2024-04-26T01:40:37.607 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-04-26 01:34:33.015049673 +0000 2024-04-26T01:40:37.607 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T01:40:37.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:37 smithi001 ceph-mon[27822]: Reconfiguring mon.b (monmap changed)... 2024-04-26T01:40:37.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:37 smithi001 ceph-mon[27822]: Reconfiguring daemon mon.b on smithi097 2024-04-26T01:40:37.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:37 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:37.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:37 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:37.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:37 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:37.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:37 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:40:37.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:37 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:40:37.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:37 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:37.696 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-04-26T01:40:37.696 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-04-26T01:40:37.696 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000228349 s, 2.2 MB/s 2024-04-26T01:40:37.701 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T01:40:37.803 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_2 2024-04-26T01:40:37.856 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T01:40:37.856 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:40:37.856 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 6h/6d Inode: 645 Links: 1 2024-04-26T01:40:37.856 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:40:37.856 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:40:37.856 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-04-26 01:39:34.703735175 +0000 2024-04-26T01:40:37.857 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-04-26 01:34:33.633041944 +0000 2024-04-26T01:40:37.857 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-04-26 01:34:33.633041944 +0000 2024-04-26T01:40:37.857 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-04-26 01:34:33.633041944 +0000 2024-04-26T01:40:37.857 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T01:40:37.940 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-04-26T01:40:37.941 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-04-26T01:40:37.941 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000310117 s, 1.7 MB/s 2024-04-26T01:40:37.944 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T01:40:38.002 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_3 2024-04-26T01:40:38.053 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T01:40:38.053 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:40:38.053 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 6h/6d Inode: 664 Links: 1 2024-04-26T01:40:38.053 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:40:38.053 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:40:38.053 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-04-26 01:39:34.802733945 +0000 2024-04-26T01:40:38.053 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-04-26 01:34:34.201034840 +0000 2024-04-26T01:40:38.053 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-04-26 01:34:34.201034840 +0000 2024-04-26T01:40:38.053 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-04-26 01:34:34.201034840 +0000 2024-04-26T01:40:38.053 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T01:40:38.127 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-04-26T01:40:38.127 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-04-26T01:40:38.127 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000268786 s, 1.9 MB/s 2024-04-26T01:40:38.130 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T01:40:38.187 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_4 2024-04-26T01:40:38.238 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T01:40:38.238 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:40:38.238 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 6h/6d Inode: 684 Links: 1 2024-04-26T01:40:38.239 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:40:38.239 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:40:38.239 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-04-26 01:39:34.892732828 +0000 2024-04-26T01:40:38.239 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-04-26 01:34:34.796027399 +0000 2024-04-26T01:40:38.239 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-04-26 01:34:34.796027399 +0000 2024-04-26T01:40:38.240 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-04-26 01:34:34.796027399 +0000 2024-04-26T01:40:38.240 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T01:40:38.326 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-04-26T01:40:38.326 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-04-26T01:40:38.327 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000293151 s, 1.7 MB/s 2024-04-26T01:40:38.330 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T01:40:38.387 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-26T01:40:38.387 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T01:40:38.404 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T01:40:38.404 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2024-04-26T01:40:38.459 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T01:40:38.460 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:40:38.460 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-26T01:40:38.460 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:40:38.460 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:40:38.460 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-26 01:40:23.694789391 +0000 2024-04-26T01:40:38.460 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-26 01:33:27.065398584 +0000 2024-04-26T01:40:38.460 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-26 01:33:27.065398584 +0000 2024-04-26T01:40:38.460 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-26 01:33:27.065398584 +0000 2024-04-26T01:40:38.461 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T01:40:38.522 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-26T01:40:38.522 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-26T01:40:38.522 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000198259 s, 2.6 MB/s 2024-04-26T01:40:38.524 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T01:40:38.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:38 smithi173 ceph-mon[32553]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:38.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:38 smithi173 ceph-mon[32553]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi097=y;smithi173=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:40:38.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:38 smithi173 ceph-mon[32553]: Saving service mgr spec with placement smithi001=x;smithi097=y;smithi173=z;count:3 2024-04-26T01:40:38.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:38 smithi173 ceph-mon[32553]: Reconfiguring mon.c (monmap changed)... 2024-04-26T01:40:38.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:38 smithi173 ceph-mon[32553]: Reconfiguring daemon mon.c on smithi173 2024-04-26T01:40:38.580 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2024-04-26T01:40:38.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:38 smithi001 ceph-mon[27822]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:38.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:38 smithi001 ceph-mon[27822]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi097=y;smithi173=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:40:38.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:38 smithi001 ceph-mon[27822]: Saving service mgr spec with placement smithi001=x;smithi097=y;smithi173=z;count:3 2024-04-26T01:40:38.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:38 smithi001 ceph-mon[27822]: Reconfiguring mon.c (monmap changed)... 2024-04-26T01:40:38.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:38 smithi001 ceph-mon[27822]: Reconfiguring daemon mon.c on smithi173 2024-04-26T01:40:38.635 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T01:40:38.635 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:40:38.635 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-26T01:40:38.635 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:40:38.635 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:40:38.635 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-26 01:40:23.694789391 +0000 2024-04-26T01:40:38.635 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-26 01:33:27.407392906 +0000 2024-04-26T01:40:38.635 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-26 01:33:27.407392906 +0000 2024-04-26T01:40:38.635 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-26 01:33:27.407392906 +0000 2024-04-26T01:40:38.635 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T01:40:38.695 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:38 smithi097 ceph-mon[32853]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:38.695 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:38 smithi097 ceph-mon[32853]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi097=y;smithi173=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:40:38.695 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:38 smithi097 ceph-mon[32853]: Saving service mgr spec with placement smithi001=x;smithi097=y;smithi173=z;count:3 2024-04-26T01:40:38.695 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:38 smithi097 ceph-mon[32853]: Reconfiguring mon.c (monmap changed)... 2024-04-26T01:40:38.695 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:38 smithi097 ceph-mon[32853]: Reconfiguring daemon mon.c on smithi173 2024-04-26T01:40:38.697 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-26T01:40:38.697 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-26T01:40:38.697 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.00018651 s, 2.7 MB/s 2024-04-26T01:40:38.699 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T01:40:38.755 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2024-04-26T01:40:38.810 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T01:40:38.810 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:40:38.811 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-26T01:40:38.811 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:40:38.811 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:40:38.811 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-26 01:40:23.695789375 +0000 2024-04-26T01:40:38.811 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-26 01:33:27.753387162 +0000 2024-04-26T01:40:38.811 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-26 01:33:27.753387162 +0000 2024-04-26T01:40:38.811 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-26 01:33:27.753387162 +0000 2024-04-26T01:40:38.811 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T01:40:38.872 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-26T01:40:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-26T01:40:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000310414 s, 1.6 MB/s 2024-04-26T01:40:38.874 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T01:40:38.930 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2024-04-26T01:40:38.985 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T01:40:38.985 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:40:38.985 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-26T01:40:38.985 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:40:38.985 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:40:38.985 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-26 01:40:23.695789375 +0000 2024-04-26T01:40:38.985 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-26 01:33:28.093381517 +0000 2024-04-26T01:40:38.986 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-26 01:33:28.093381517 +0000 2024-04-26T01:40:38.986 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-26 01:33:28.093381517 +0000 2024-04-26T01:40:38.986 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T01:40:39.047 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-26T01:40:39.047 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-26T01:40:39.047 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000191618 s, 2.7 MB/s 2024-04-26T01:40:39.049 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T01:40:39.104 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-26T01:40:39.104 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T01:40:39.119 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T01:40:39.119 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_1 2024-04-26T01:40:39.174 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T01:40:39.174 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:40:39.174 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-26T01:40:39.174 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:40:39.174 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:40:39.174 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-04-26 01:40:14.082874816 +0000 2024-04-26T01:40:39.174 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-04-26 01:33:35.309224026 +0000 2024-04-26T01:40:39.174 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-04-26 01:33:35.309224026 +0000 2024-04-26T01:40:39.174 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-04-26 01:33:35.309224026 +0000 2024-04-26T01:40:39.174 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T01:40:39.237 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-04-26T01:40:39.237 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-04-26T01:40:39.237 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000260349 s, 2.0 MB/s 2024-04-26T01:40:39.238 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T01:40:39.296 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_2 2024-04-26T01:40:39.350 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T01:40:39.350 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:40:39.350 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-26T01:40:39.351 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:40:39.351 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:40:39.351 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-04-26 01:40:14.083874793 +0000 2024-04-26T01:40:39.351 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-04-26 01:33:35.651215762 +0000 2024-04-26T01:40:39.351 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-04-26 01:33:35.651215762 +0000 2024-04-26T01:40:39.351 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-04-26 01:33:35.651215762 +0000 2024-04-26T01:40:39.351 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T01:40:39.413 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-04-26T01:40:39.414 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-04-26T01:40:39.414 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000224997 s, 2.3 MB/s 2024-04-26T01:40:39.415 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T01:40:39.471 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_3 2024-04-26T01:40:39.525 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T01:40:39.525 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:40:39.525 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-26T01:40:39.526 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:40:39.526 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:40:39.526 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-04-26 01:40:14.083874793 +0000 2024-04-26T01:40:39.526 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-04-26 01:33:35.986207667 +0000 2024-04-26T01:40:39.526 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-04-26 01:33:35.986207667 +0000 2024-04-26T01:40:39.526 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-04-26 01:33:35.986207667 +0000 2024-04-26T01:40:39.526 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T01:40:39.589 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-04-26T01:40:39.589 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-04-26T01:40:39.590 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000230061 s, 2.2 MB/s 2024-04-26T01:40:39.591 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T01:40:39.646 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_4 2024-04-26T01:40:39.701 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T01:40:39.701 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:40:39.701 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-26T01:40:39.702 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:40:39.702 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:40:39.702 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-04-26 01:40:14.083874793 +0000 2024-04-26T01:40:39.702 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-04-26 01:33:36.320199596 +0000 2024-04-26T01:40:39.702 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-04-26 01:33:36.320199596 +0000 2024-04-26T01:40:39.702 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-04-26 01:33:36.320199596 +0000 2024-04-26T01:40:39.702 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T01:40:39.764 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-04-26T01:40:39.764 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-04-26T01:40:39.764 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000184822 s, 2.8 MB/s 2024-04-26T01:40:39.765 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T01:40:39.822 INFO:tasks.cephadm:Deploying osd.0 on smithi001 with /dev/vg_nvme/lv_4... 2024-04-26T01:40:39.822 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T01:40:39.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:39 smithi097 ceph-mon[32853]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:39.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:39.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:39.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:40:39.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:39.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:40:39.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:39.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:40:39.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T01:40:39.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:40:39.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:39.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:39 smithi097 ceph-mon[32853]: Deploying daemon mgr.z on smithi173 2024-04-26T01:40:40.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:39 smithi173 ceph-mon[32553]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:40.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:40.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:40.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:40:40.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:40.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:40:40.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:40.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:40:40.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T01:40:40.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:40:40.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:40.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:39 smithi173 ceph-mon[32553]: Deploying daemon mgr.z on smithi173 2024-04-26T01:40:40.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:39 smithi001 ceph-mon[27822]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:40.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:40.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:40.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:40:40.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:40.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:40:40.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:40.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:40:40.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T01:40:40.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:40:40.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:40.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:39 smithi001 ceph-mon[27822]: Deploying daemon mgr.z on smithi173 2024-04-26T01:40:40.273 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:40:41.025 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:40 smithi173 systemd[1]: Starting Ceph mgr.z for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T01:40:41.311 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:41 smithi173 podman[34411]: 2024-04-26 01:40:41.028985228 +0000 UTC m=+0.093066853 container create 28ffc01326f14f2bfdf9fc08524a9a9d031ab86fe640eadcac71c7421b7b084c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-26T01:40:41.311 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:41 smithi173 podman[34411]: 2024-04-26 01:40:40.953848877 +0000 UTC m=+0.017930476 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:40:41.311 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:41 smithi173 podman[34411]: 2024-04-26 01:40:41.239766038 +0000 UTC m=+0.303847626 container init 28ffc01326f14f2bfdf9fc08524a9a9d031ab86fe640eadcac71c7421b7b084c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-26T01:40:41.311 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:41 smithi173 podman[34411]: 2024-04-26 01:40:41.242357593 +0000 UTC m=+0.306439177 container start 28ffc01326f14f2bfdf9fc08524a9a9d031ab86fe640eadcac71c7421b7b084c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T01:40:41.311 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:41 smithi173 ceph-mgr[34430]: -- 172.21.15.173:0/3902759892 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d6fa0fe5a0 con 0x55d6fa0d2800 2024-04-26T01:40:41.311 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:41 smithi173 bash[34411]: 28ffc01326f14f2bfdf9fc08524a9a9d031ab86fe640eadcac71c7421b7b084c 2024-04-26T01:40:41.565 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:41 smithi173 systemd[1]: Started Ceph mgr.z for 77cfcc80-036d-11ef-bc93-c7b262605968. 2024-04-26T01:40:41.565 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:41 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:41.395+0000 7f92f8dc7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:40:41.982 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:41 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:41.579+0000 7f92f8dc7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:40:41.982 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:41 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:41.679+0000 7f92f8dc7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:40:41.982 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:41 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:41.985+0000 7f92f8dc7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:40:42.315 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:42 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:42.069+0000 7f92f8dc7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:40:42.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:42 smithi001 ceph-mon[27822]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:42.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:42 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:42.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:42 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:42.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:42 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:42.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:42 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:40:42.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:42 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T01:40:42.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:42 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:40:42.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:42 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:42.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:42 smithi001 ceph-mon[27822]: Deploying daemon mgr.y on smithi097 2024-04-26T01:40:42.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:42 smithi097 ceph-mon[32853]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:42.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:42 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:42.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:42 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:42.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:42 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:42.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:42 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:40:42.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:42 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T01:40:42.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:42 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:40:42.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:42 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:42.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:42 smithi097 ceph-mon[32853]: Deploying daemon mgr.y on smithi097 2024-04-26T01:40:42.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:42 smithi173 ceph-mon[32553]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:42.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:42 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:42.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:42 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:42.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:42 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:42.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:42 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:40:42.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:42 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T01:40:42.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:42 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:40:42.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:42 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:42.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:42 smithi173 ceph-mon[32553]: Deploying daemon mgr.y on smithi097 2024-04-26T01:40:43.123 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T01:40:43.123 INFO:teuthology.orchestra.run.smithi001.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T01:40:43.123 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10+0 records in 2024-04-26T01:40:43.123 INFO:teuthology.orchestra.run.smithi001.stderr:10+0 records out 2024-04-26T01:40:43.123 INFO:teuthology.orchestra.run.smithi001.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0371568 s, 282 MB/s 2024-04-26T01:40:43.124 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping successful for: 2024-04-26T01:40:43.565 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:43 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:43.275+0000 7f92f8dc7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:40:43.565 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:43 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:43.443+0000 7f92f8dc7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:40:43.655 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:43 smithi097 systemd[1]: Starting Ceph mgr.y for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T01:40:43.897 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:43 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:43.614+0000 7f92f8dc7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:40:43.898 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:43 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:43.901+0000 7f92f8dc7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:40:43.919 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:43 smithi097 podman[34221]: 2024-04-26 01:40:43.561650258 +0000 UTC m=+0.026351922 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:40:43.919 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:43 smithi097 podman[34221]: 2024-04-26 01:40:43.701839498 +0000 UTC m=+0.166541151 container create b29fd37f9ca712deefc52882fb097e7f5fae919e5a4b74fcb79b060921304da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T01:40:43.919 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:43 smithi097 podman[34221]: 2024-04-26 01:40:43.895816971 +0000 UTC m=+0.360518625 container init b29fd37f9ca712deefc52882fb097e7f5fae919e5a4b74fcb79b060921304da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T01:40:43.919 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:43 smithi097 podman[34221]: 2024-04-26 01:40:43.898273327 +0000 UTC m=+0.362974984 container start b29fd37f9ca712deefc52882fb097e7f5fae919e5a4b74fcb79b060921304da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T01:40:43.936 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi001:vg_nvme/lv_4 2024-04-26T01:40:44.196 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:43 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:43.993+0000 7f92f8dc7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:40:44.197 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:44 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:44.200+0000 7f92f8dc7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:40:44.232 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:43 smithi097 ceph-mgr[34240]: -- 172.21.15.97:0/179418348 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5575762105a0 con 0x5575761e4c00 2024-04-26T01:40:44.232 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:43 smithi097 bash[34221]: b29fd37f9ca712deefc52882fb097e7f5fae919e5a4b74fcb79b060921304da1 2024-04-26T01:40:44.232 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:43 smithi097 systemd[1]: Started Ceph mgr.y for 77cfcc80-036d-11ef-bc93-c7b262605968. 2024-04-26T01:40:44.232 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:44 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:44.061+0000 7f0735c10200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:40:44.427 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:40:44.565 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:44 smithi001 ceph-mon[27822]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:44.565 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:44 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:44.565 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:44 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:44.565 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:44 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:44.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:44 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:44.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:44 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:40:44.566 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:44 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:44.396+0000 7f92f8dc7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:40:44.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:44 smithi173 ceph-mon[32553]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:44.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:44 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:44.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:44 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:44.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:44 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:44.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:44 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:44.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:44 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:40:44.695 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:44 smithi097 ceph-mon[32853]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:44.695 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:44 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:44.695 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:44 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:44.696 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:44 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:44.696 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:44 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:44.696 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:44 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:40:44.696 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:44 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:44.254+0000 7f0735c10200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:40:44.696 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:44 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:44.362+0000 7f0735c10200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:40:44.696 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:44 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:44.697+0000 7f0735c10200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:40:44.982 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:44 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:44.787+0000 7f0735c10200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:40:45.315 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:45 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:45.028+0000 7f92f8dc7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:40:45.765 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:45 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:45.461+0000 7f92f8dc7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:40:45.765 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:45 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:45.557+0000 7f92f8dc7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:40:45.765 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:45 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:45.768+0000 7f92f8dc7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:40:46.391 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:46 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:46.056+0000 7f0735c10200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:40:46.392 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:46 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:46.225+0000 7f0735c10200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:40:46.392 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:46 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:46.394+0000 7f0735c10200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:40:46.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.442+0000 7fbe9d400700 1 -- 172.21.15.1:0/452544681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe98104640 msgr2=0x7fbe9810aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:46.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.442+0000 7fbe9d400700 1 --2- 172.21.15.1:0/452544681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe98104640 0x7fbe9810aef0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000b0d0 tx=0x7fbe9000b3e0 comp rx=0 tx=0).stop 2024-04-26T01:40:46.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.442+0000 7fbe9d400700 1 -- 172.21.15.1:0/452544681 shutdown_connections 2024-04-26T01:40:46.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.442+0000 7fbe9d400700 1 --2- 172.21.15.1:0/452544681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe98104640 0x7fbe9810aef0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:46.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.442+0000 7fbe9d400700 1 --2- 172.21.15.1:0/452544681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe98102aa0 0x7fbe98102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:46.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.442+0000 7fbe9d400700 1 --2- 172.21.15.1:0/452544681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe98103ca0 0x7fbe98104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:46.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.442+0000 7fbe9d400700 1 -- 172.21.15.1:0/452544681 >> 172.21.15.1:0/452544681 conn(0x7fbe980fe040 msgr2=0x7fbe98100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:40:46.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.442+0000 7fbe9d400700 1 -- 172.21.15.1:0/452544681 shutdown_connections 2024-04-26T01:40:46.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.442+0000 7fbe9d400700 1 -- 172.21.15.1:0/452544681 wait complete. 2024-04-26T01:40:46.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.443+0000 7fbe9d400700 1 Processor -- start 2024-04-26T01:40:46.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.443+0000 7fbe9d400700 1 -- start start 2024-04-26T01:40:46.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.444+0000 7fbe9d400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe98102aa0 0x7fbe98198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:46.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.444+0000 7fbe9d400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe98103ca0 0x7fbe98199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:46.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.444+0000 7fbe9d400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe98104640 0x7fbe981a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:46.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.444+0000 7fbe9d400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9810e590 con 0x7fbe98102aa0 2024-04-26T01:40:46.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.444+0000 7fbe9d400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9810e410 con 0x7fbe98103ca0 2024-04-26T01:40:46.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.444+0000 7fbe9d400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9810e710 con 0x7fbe98104640 2024-04-26T01:40:46.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.444+0000 7fbe8de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe98103ca0 0x7fbe98199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:46.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.444+0000 7fbe8de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe98103ca0 0x7fbe98199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:54158/0 (socket says 172.21.15.1:54158) 2024-04-26T01:40:46.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.444+0000 7fbe8de00700 1 -- 172.21.15.1:0/2111927525 learned_addr learned my addr 172.21.15.1:0/2111927525 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:40:46.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.444+0000 7fbe96800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe98102aa0 0x7fbe98198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:46.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.444+0000 7fbe97200700 1 --2- 172.21.15.1:0/2111927525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe98104640 0x7fbe981a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:46.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.444+0000 7fbe96800700 1 -- 172.21.15.1:0/2111927525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe98104640 msgr2=0x7fbe981a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:46.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.444+0000 7fbe96800700 1 --2- 172.21.15.1:0/2111927525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe98104640 0x7fbe981a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:46.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.445+0000 7fbe96800700 1 -- 172.21.15.1:0/2111927525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe98103ca0 msgr2=0x7fbe98199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:40:46.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.445+0000 7fbe96800700 1 --2- 172.21.15.1:0/2111927525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe98103ca0 0x7fbe98199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:40:46.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.445+0000 7fbe96800700 1 -- 172.21.15.1:0/2111927525 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe981a0830 con 0x7fbe98102aa0 2024-04-26T01:40:46.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.445+0000 7fbe8de00700 1 --2- 172.21.15.1:0/2111927525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe98103ca0 0x7fbe98199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:40:46.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.445+0000 7fbe97200700 1 --2- 172.21.15.1:0/2111927525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe98104640 0x7fbe981a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:40:46.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.445+0000 7fbe96800700 1 --2- 172.21.15.1:0/2111927525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe98102aa0 0x7fbe98198bd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800e9f0 tx=0x7fbe8800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:46.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.445+0000 7fbe8fe00700 1 -- 172.21.15.1:0/2111927525 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe88004230 con 0x7fbe98102aa0 2024-04-26T01:40:46.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.445+0000 7fbe8fe00700 1 -- 172.21.15.1:0/2111927525 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe880043d0 con 0x7fbe98102aa0 2024-04-26T01:40:46.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.445+0000 7fbe9d400700 1 -- 172.21.15.1:0/2111927525 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe981a0b20 con 0x7fbe98102aa0 2024-04-26T01:40:46.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.445+0000 7fbe8fe00700 1 -- 172.21.15.1:0/2111927525 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe88003d30 con 0x7fbe98102aa0 2024-04-26T01:40:46.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.445+0000 7fbe9d400700 1 -- 172.21.15.1:0/2111927525 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe981a1010 con 0x7fbe98102aa0 2024-04-26T01:40:46.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.446+0000 7fbe8fe00700 1 -- 172.21.15.1:0/2111927525 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7fbe88021440 con 0x7fbe98102aa0 2024-04-26T01:40:46.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.447+0000 7fbe8fe00700 1 --2- 172.21.15.1:0/2111927525 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbe7c039890 0x7fbe7c03bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:40:46.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.447+0000 7fbe8fe00700 1 -- 172.21.15.1:0/2111927525 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbe8804cdc0 con 0x7fbe98102aa0 2024-04-26T01:40:46.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.447+0000 7fbe9d400700 1 -- 172.21.15.1:0/2111927525 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe98069730 con 0x7fbe98102aa0 2024-04-26T01:40:46.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.447+0000 7fbe8de00700 1 --2- 172.21.15.1:0/2111927525 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbe7c039890 0x7fbe7c03bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:40:46.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.448+0000 7fbe8de00700 1 --2- 172.21.15.1:0/2111927525 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbe7c039890 0x7fbe7c03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbe80011550 tx=0x7fbe80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:40:46.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.454+0000 7fbe8fe00700 1 -- 172.21.15.1:0/2111927525 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe8800cd90 con 0x7fbe98102aa0 2024-04-26T01:40:46.565 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:46 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:46.295+0000 7f92f8dc7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:40:46.565 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:46 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:46.387+0000 7f92f8dc7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:40:46.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:46 smithi173 ceph-mon[32553]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:46.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:46 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:46.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:46 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:46.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:46 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:46.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:46 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:46.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:46 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:46.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:46 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:40:46.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:46 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:46.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:46 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:46.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:46 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:40:46.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:46 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:40:46.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:46 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:46.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:46 smithi001 ceph-mon[27822]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:46.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:46 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:46.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:46 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:46.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:46 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:46.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:46 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:46.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:46 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:46.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:46 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:40:46.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:46 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:46.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:46 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:46.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:46 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:40:46.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:46 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:40:46.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:46 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:46.679 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:46 smithi097 ceph-mon[32853]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:46.679 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:46 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:46.679 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:46 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:46.679 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:46 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:46.679 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:46 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:46.679 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:46 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:46.679 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:46 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:40:46.679 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:46 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:46.679 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:46 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:46.679 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:46 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:40:46.679 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:46 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:40:46.680 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:46 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:46.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:46.699+0000 7fbe9d400700 1 -- 172.21.15.1:0/2111927525 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbe98062e80 con 0x7fbe7c039890 2024-04-26T01:40:46.963 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:46 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:46.628+0000 7f92f8dc7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:40:46.963 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:46 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:46.760+0000 7f92f8dc7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:40:46.963 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:46 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:46.853+0000 7f92f8dc7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:40:46.964 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:46 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:46.967+0000 7f92f8dc7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:40:46.972 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:46 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:46.682+0000 7f0735c10200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:40:46.972 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:46 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:46.774+0000 7f0735c10200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:40:46.972 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:46 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:46.975+0000 7f0735c10200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:40:47.232 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:47 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:47.165+0000 7f0735c10200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:40:47.565 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:47 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:47.270+0000 7f92f8dc7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:40:47.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:47 smithi173 ceph-mon[32553]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T01:40:47.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:47 smithi173 ceph-mon[32553]: Reconfiguring daemon mgr.x on smithi001 2024-04-26T01:40:47.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:47 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:40:47.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:47 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:40:47.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:47 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:47.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:47 smithi001 ceph-mon[27822]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T01:40:47.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:47 smithi001 ceph-mon[27822]: Reconfiguring daemon mgr.x on smithi001 2024-04-26T01:40:47.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:47 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:40:47.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:47 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:40:47.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:47 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:47.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:47 smithi097 ceph-mon[32853]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T01:40:47.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:47 smithi097 ceph-mon[32853]: Reconfiguring daemon mgr.x on smithi001 2024-04-26T01:40:47.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:47 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:40:47.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:47 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:40:47.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:47 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:48.173 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:47 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:47.758+0000 7f0735c10200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:40:48.315 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:47 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:47.866+0000 7f92f8dc7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:40:48.315 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:40:48 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:40:48.080+0000 7f92f8dc7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:40:48.474 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:48 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:48.176+0000 7f0735c10200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:40:48.474 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:48 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:48.272+0000 7f0735c10200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:40:48.474 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:48 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:48.476+0000 7f0735c10200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:40:48.474 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:48 smithi097 ceph-mon[32853]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:48.474 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:48 smithi097 ceph-mon[32853]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:40:48.474 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:48 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:48.475 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:48 smithi097 ceph-mon[32853]: from='mgr.? 172.21.15.173:0/762949028' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T01:40:48.475 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:48 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:48.475 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:48 smithi097 ceph-mon[32853]: from='mgr.? 172.21.15.173:0/762949028' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:40:48.475 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:48 smithi097 ceph-mon[32853]: from='mgr.? 172.21.15.173:0/762949028' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T01:40:48.475 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:48 smithi097 ceph-mon[32853]: from='mgr.? 172.21.15.173:0/762949028' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:40:48.475 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:48 smithi097 ceph-mon[32853]: Standby manager daemon z started 2024-04-26T01:40:48.475 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:48 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:40:48.475 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:48 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:48.475 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:48 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:40:48.475 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:48 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:48 smithi001 ceph-mon[27822]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:48 smithi001 ceph-mon[27822]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:40:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:48 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:48 smithi001 ceph-mon[27822]: from='mgr.? 172.21.15.173:0/762949028' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T01:40:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:48 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:48 smithi001 ceph-mon[27822]: from='mgr.? 172.21.15.173:0/762949028' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:40:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:48 smithi001 ceph-mon[27822]: from='mgr.? 172.21.15.173:0/762949028' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T01:40:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:48 smithi001 ceph-mon[27822]: from='mgr.? 172.21.15.173:0/762949028' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:40:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:48 smithi001 ceph-mon[27822]: Standby manager daemon z started 2024-04-26T01:40:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:48 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:40:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:48 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:48 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:40:48.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:48 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:48.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:48 smithi173 ceph-mon[32553]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:48.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:48 smithi173 ceph-mon[32553]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:40:48.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:48 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:48.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:48 smithi173 ceph-mon[32553]: from='mgr.? 172.21.15.173:0/762949028' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T01:40:48.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:48 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:48.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:48 smithi173 ceph-mon[32553]: from='mgr.? 172.21.15.173:0/762949028' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:40:48.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:48 smithi173 ceph-mon[32553]: from='mgr.? 172.21.15.173:0/762949028' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T01:40:48.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:48 smithi173 ceph-mon[32553]: from='mgr.? 172.21.15.173:0/762949028' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:40:48.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:48 smithi173 ceph-mon[32553]: Standby manager daemon z started 2024-04-26T01:40:48.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:48 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:40:48.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:48 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:40:48.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:48 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:40:48.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:48 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:40:49.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:49.097+0000 7fbe8fe00700 1 -- 172.21.15.1:0/2111927525 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 14) ==== 92601+0+0 (secure 0 0 0) 0x7fbe880107b0 con 0x7fbe98102aa0 2024-04-26T01:40:49.326 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:48 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:48.998+0000 7f0735c10200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:40:49.326 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:49 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:49.090+0000 7f0735c10200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:40:49.667 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:49 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:49.329+0000 7f0735c10200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:40:49.667 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:49 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:49.461+0000 7f0735c10200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:40:49.667 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:49 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:49.554+0000 7f0735c10200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:40:49.667 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:49 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:49.670+0000 7f0735c10200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:40:50.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:50 smithi097 ceph-mon[32853]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:50.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:50 smithi097 ceph-mon[32853]: mgrmap e14: x(active, since 88s), standbys: z 2024-04-26T01:40:50.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:50 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T01:40:50.232 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:49 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:49.973+0000 7f0735c10200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:40:50.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:50 smithi001 ceph-mon[27822]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:50.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:50 smithi001 ceph-mon[27822]: mgrmap e14: x(active, since 88s), standbys: z 2024-04-26T01:40:50.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:50 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T01:40:50.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:50 smithi173 ceph-mon[32553]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:50.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:50 smithi173 ceph-mon[32553]: mgrmap e14: x(active, since 88s), standbys: z 2024-04-26T01:40:50.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:50 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T01:40:50.982 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:50 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:50.574+0000 7f0735c10200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:40:50.982 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:40:50 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:40:50.789+0000 7f0735c10200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:40:51.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:40:51.123+0000 7fbe8fe00700 1 -- 172.21.15.1:0/2111927525 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mgrmap(e 15) ==== 138589+0+0 (secure 0 0 0) 0x7fbe88016370 con 0x7fbe98102aa0 2024-04-26T01:40:51.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:51 smithi001 ceph-mon[27822]: Standby manager daemon y started 2024-04-26T01:40:51.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:51 smithi001 ceph-mon[27822]: from='mgr.? 172.21.15.97:0/2441753241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T01:40:51.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:51 smithi001 ceph-mon[27822]: from='mgr.? 172.21.15.97:0/2441753241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:40:51.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:51 smithi001 ceph-mon[27822]: from='mgr.? 172.21.15.97:0/2441753241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T01:40:51.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:51 smithi001 ceph-mon[27822]: from='mgr.? 172.21.15.97:0/2441753241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:40:51.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:51 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1962601557' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb10ae12-09ee-4d6b-bc78-8c6016613b81"}]: dispatch 2024-04-26T01:40:51.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:51 smithi001 ceph-mon[27822]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb10ae12-09ee-4d6b-bc78-8c6016613b81"}]: dispatch 2024-04-26T01:40:51.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:51 smithi001 ceph-mon[27822]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb10ae12-09ee-4d6b-bc78-8c6016613b81"}]': finished 2024-04-26T01:40:51.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:51 smithi001 ceph-mon[27822]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T01:40:51.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:51 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:40:51.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:51 smithi097 ceph-mon[32853]: Standby manager daemon y started 2024-04-26T01:40:51.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:51 smithi097 ceph-mon[32853]: from='mgr.? 172.21.15.97:0/2441753241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T01:40:51.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:51 smithi097 ceph-mon[32853]: from='mgr.? 172.21.15.97:0/2441753241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:40:51.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:51 smithi097 ceph-mon[32853]: from='mgr.? 172.21.15.97:0/2441753241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T01:40:51.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:51 smithi097 ceph-mon[32853]: from='mgr.? 172.21.15.97:0/2441753241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:40:51.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:51 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1962601557' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb10ae12-09ee-4d6b-bc78-8c6016613b81"}]: dispatch 2024-04-26T01:40:51.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:51 smithi097 ceph-mon[32853]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb10ae12-09ee-4d6b-bc78-8c6016613b81"}]: dispatch 2024-04-26T01:40:51.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:51 smithi097 ceph-mon[32853]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb10ae12-09ee-4d6b-bc78-8c6016613b81"}]': finished 2024-04-26T01:40:51.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:51 smithi097 ceph-mon[32853]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T01:40:51.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:51 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:40:51.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:51 smithi173 ceph-mon[32553]: Standby manager daemon y started 2024-04-26T01:40:51.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:51 smithi173 ceph-mon[32553]: from='mgr.? 172.21.15.97:0/2441753241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T01:40:51.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:51 smithi173 ceph-mon[32553]: from='mgr.? 172.21.15.97:0/2441753241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:40:51.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:51 smithi173 ceph-mon[32553]: from='mgr.? 172.21.15.97:0/2441753241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T01:40:51.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:51 smithi173 ceph-mon[32553]: from='mgr.? 172.21.15.97:0/2441753241' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:40:51.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:51 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1962601557' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb10ae12-09ee-4d6b-bc78-8c6016613b81"}]: dispatch 2024-04-26T01:40:51.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:51 smithi173 ceph-mon[32553]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb10ae12-09ee-4d6b-bc78-8c6016613b81"}]: dispatch 2024-04-26T01:40:51.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:51 smithi173 ceph-mon[32553]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb10ae12-09ee-4d6b-bc78-8c6016613b81"}]': finished 2024-04-26T01:40:51.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:51 smithi173 ceph-mon[32553]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T01:40:51.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:51 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:40:52.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:52 smithi001 ceph-mon[27822]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:52.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:52 smithi001 ceph-mon[27822]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-04-26T01:40:52.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T01:40:52.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:52 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/4058193285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:40:52.432 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:52 smithi173 ceph-mon[32553]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:52.432 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:52 smithi173 ceph-mon[32553]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-04-26T01:40:52.432 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T01:40:52.432 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:52 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/4058193285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:40:52.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:52 smithi097 ceph-mon[32853]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:52.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:52 smithi097 ceph-mon[32853]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-04-26T01:40:52.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T01:40:52.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:52 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/4058193285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:40:54.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:54 smithi001 ceph-mon[27822]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:54.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:54 smithi097 ceph-mon[32853]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:54.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:54 smithi173 ceph-mon[32553]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:56.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:56 smithi001 ceph-mon[27822]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:56.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:56 smithi097 ceph-mon[32853]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:56.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:56 smithi173 ceph-mon[32553]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:58.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:40:58 smithi001 ceph-mon[27822]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:58.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:40:58 smithi097 ceph-mon[32853]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:40:58.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:40:58 smithi173 ceph-mon[32553]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:00.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:00 smithi001 ceph-mon[27822]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:00.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:00 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T01:41:00.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:00 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:00.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:00 smithi097 ceph-mon[32853]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:00.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:00 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T01:41:00.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:00 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:00.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:00 smithi173 ceph-mon[32553]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:00.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:00 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T01:41:00.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:00 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:01.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:01 smithi001 ceph-mon[27822]: Deploying daemon osd.0 on smithi001 2024-04-26T01:41:01.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:01 smithi097 ceph-mon[32853]: Deploying daemon osd.0 on smithi001 2024-04-26T01:41:01.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:01 smithi173 ceph-mon[32553]: Deploying daemon osd.0 on smithi001 2024-04-26T01:41:02.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:02 smithi001 ceph-mon[27822]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:02.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:02 smithi097 ceph-mon[32853]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:02.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:02 smithi173 ceph-mon[32553]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:04.331 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:04 smithi001 ceph-mon[27822]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:04.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:04 smithi097 ceph-mon[32853]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:04.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:04 smithi173 ceph-mon[32553]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:06.323 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:06 smithi001 ceph-mon[27822]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:06.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:06 smithi097 ceph-mon[32853]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:06.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:06 smithi173 ceph-mon[32553]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:08.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:08 smithi097 ceph-mon[32853]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:08.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:08 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:41:08.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:08 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:08.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:08 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:08.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:08 smithi173 ceph-mon[32553]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:08.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:08 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:41:08.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:08 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:08.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:08 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:08.627 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:08 smithi001 ceph-mon[27822]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:08.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:08 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:41:08.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:08 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:08.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:08 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:10.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:10 smithi097 ceph-mon[32853]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:10.521 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:10 smithi001 ceph-mon[27822]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:10.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:10 smithi173 ceph-mon[32553]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:12.305 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:12 smithi097 ceph-mon[32853]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:12.305 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:12 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:12.305 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:12 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:12.305 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:12 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:12.305 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:12 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:41:12.305 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:12 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:12.305 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:12 smithi097 ceph-mon[32853]: from='osd.0 [v2:172.21.15.1:6802/3176633965,v1:172.21.15.1:6803/3176633965]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T01:41:12.306 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:12 smithi097 ceph-mon[32853]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T01:41:12.315 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:12 smithi173 ceph-mon[32553]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:12.315 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:12 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:12.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:12 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:12.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:12 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:12.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:12 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:41:12.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:12 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:12.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:12 smithi173 ceph-mon[32553]: from='osd.0 [v2:172.21.15.1:6802/3176633965,v1:172.21.15.1:6803/3176633965]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T01:41:12.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:12 smithi173 ceph-mon[32553]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T01:41:12.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:12 smithi001 ceph-mon[27822]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:12.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:12 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:12.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:12 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:12.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:12 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:12.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:12 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:41:12.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:12 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:12.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:12 smithi001 ceph-mon[27822]: from='osd.0 [v2:172.21.15.1:6802/3176633965,v1:172.21.15.1:6803/3176633965]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T01:41:12.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:12 smithi001 ceph-mon[27822]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T01:41:12.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:41:12.834+0000 7fbe8fe00700 1 -- 172.21.15.1:0/2111927525 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbe98062e80 con 0x7fbe7c039890 2024-04-26T01:41:12.832 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 0 on host 'smithi001' 2024-04-26T01:41:12.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:41:12.838+0000 7fbe9d400700 1 -- 172.21.15.1:0/2111927525 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbe7c039890 msgr2=0x7fbe7c03bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:12.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:41:12.838+0000 7fbe9d400700 1 --2- 172.21.15.1:0/2111927525 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbe7c039890 0x7fbe7c03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbe80011550 tx=0x7fbe80009260 comp rx=0 tx=0).stop 2024-04-26T01:41:12.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:41:12.838+0000 7fbe9d400700 1 -- 172.21.15.1:0/2111927525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe98102aa0 msgr2=0x7fbe98198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:12.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:41:12.838+0000 7fbe9d400700 1 --2- 172.21.15.1:0/2111927525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe98102aa0 0x7fbe98198bd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800e9f0 tx=0x7fbe8800ed00 comp rx=0 tx=0).stop 2024-04-26T01:41:12.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:41:12.838+0000 7fbe9d400700 1 -- 172.21.15.1:0/2111927525 shutdown_connections 2024-04-26T01:41:12.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:41:12.838+0000 7fbe9d400700 1 --2- 172.21.15.1:0/2111927525 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbe7c039890 0x7fbe7c03bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:12.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:41:12.838+0000 7fbe9d400700 1 --2- 172.21.15.1:0/2111927525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe98104640 0x7fbe981a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:12.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:41:12.838+0000 7fbe9d400700 1 --2- 172.21.15.1:0/2111927525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe98102aa0 0x7fbe98198bd0 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800e9f0 tx=0x7fbe8800ed00 comp rx=0 tx=0).stop 2024-04-26T01:41:12.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:41:12.838+0000 7fbe9d400700 1 --2- 172.21.15.1:0/2111927525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe98103ca0 0x7fbe98199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:12.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:41:12.838+0000 7fbe9d400700 1 -- 172.21.15.1:0/2111927525 >> 172.21.15.1:0/2111927525 conn(0x7fbe980fe040 msgr2=0x7fbe98107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:12.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:41:12.839+0000 7fbe9d400700 1 -- 172.21.15.1:0/2111927525 shutdown_connections 2024-04-26T01:41:12.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:41:12.839+0000 7fbe9d400700 1 -- 172.21.15.1:0/2111927525 wait complete. 2024-04-26T01:41:13.479 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:13 smithi097 ceph-mon[32853]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T01:41:13.479 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:13 smithi097 ceph-mon[32853]: from='osd.0 [v2:172.21.15.1:6802/3176633965,v1:172.21.15.1:6803/3176633965]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-04-26T01:41:13.479 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:13 smithi097 ceph-mon[32853]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T01:41:13.479 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:13 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:41:13.479 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:13 smithi097 ceph-mon[32853]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-04-26T01:41:13.479 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:13 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:13.480 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:13 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:41:13.480 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:13 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:13.546 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:13 smithi001 ceph-mon[27822]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T01:41:13.547 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:13 smithi001 ceph-mon[27822]: from='osd.0 [v2:172.21.15.1:6802/3176633965,v1:172.21.15.1:6803/3176633965]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-04-26T01:41:13.547 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:13 smithi001 ceph-mon[27822]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T01:41:13.547 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:13 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:41:13.547 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:13 smithi001 ceph-mon[27822]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-04-26T01:41:13.547 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:13 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:13.547 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:13 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:41:13.547 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:13 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:13.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:13 smithi173 ceph-mon[32553]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T01:41:13.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:13 smithi173 ceph-mon[32553]: from='osd.0 [v2:172.21.15.1:6802/3176633965,v1:172.21.15.1:6803/3176633965]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-04-26T01:41:13.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:13 smithi173 ceph-mon[32553]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T01:41:13.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:13 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:41:13.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:13 smithi173 ceph-mon[32553]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-04-26T01:41:13.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:13 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:13.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:13 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:41:13.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:13 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:13.609 DEBUG:teuthology.orchestra.run.smithi001:osd.0> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.0.service 2024-04-26T01:41:13.614 INFO:tasks.cephadm:Deploying osd.1 on smithi097 with /dev/vg_nvme/lv_4... 2024-04-26T01:41:13.615 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T01:41:13.845 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:41:14.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:14 smithi097 ceph-mon[32853]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:14.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:14 smithi097 ceph-mon[32853]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-04-26T01:41:14.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:14 smithi097 ceph-mon[32853]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T01:41:14.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:14 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:41:14.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:14 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:41:14.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:14 smithi173 ceph-mon[32553]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:14.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:14 smithi173 ceph-mon[32553]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-04-26T01:41:14.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:14 smithi173 ceph-mon[32553]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T01:41:14.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:14 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:41:14.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:14 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:41:14.582 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:14 smithi001 ceph-mon[27822]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:14.582 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:14 smithi001 ceph-mon[27822]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-04-26T01:41:14.582 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:14 smithi001 ceph-mon[27822]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T01:41:14.582 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:14 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:41:14.582 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:14 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:41:15.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:15 smithi001 ceph-mon[27822]: purged_snaps scrub starts 2024-04-26T01:41:15.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:15 smithi001 ceph-mon[27822]: purged_snaps scrub ok 2024-04-26T01:41:15.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:15 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:41:15.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:15 smithi001 ceph-mon[27822]: osd.0 [v2:172.21.15.1:6802/3176633965,v1:172.21.15.1:6803/3176633965] boot 2024-04-26T01:41:15.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:15 smithi001 ceph-mon[27822]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T01:41:15.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:15 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:41:15.481 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:15 smithi097 ceph-mon[32853]: purged_snaps scrub starts 2024-04-26T01:41:15.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:15 smithi097 ceph-mon[32853]: purged_snaps scrub ok 2024-04-26T01:41:15.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:15 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:41:15.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:15 smithi097 ceph-mon[32853]: osd.0 [v2:172.21.15.1:6802/3176633965,v1:172.21.15.1:6803/3176633965] boot 2024-04-26T01:41:15.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:15 smithi097 ceph-mon[32853]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T01:41:15.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:15 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:41:15.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:15 smithi173 ceph-mon[32553]: purged_snaps scrub starts 2024-04-26T01:41:15.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:15 smithi173 ceph-mon[32553]: purged_snaps scrub ok 2024-04-26T01:41:15.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:15 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:41:15.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:15 smithi173 ceph-mon[32553]: osd.0 [v2:172.21.15.1:6802/3176633965,v1:172.21.15.1:6803/3176633965] boot 2024-04-26T01:41:15.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:15 smithi173 ceph-mon[32553]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T01:41:15.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:15 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:41:15.696 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T01:41:15.696 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T01:41:15.696 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-26T01:41:15.696 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-26T01:41:15.696 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112567 s, 932 MB/s 2024-04-26T01:41:15.696 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 2024-04-26T01:41:15.696 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-26T01:41:16.286 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:16 smithi097 ceph-mon[32853]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:16.286 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:16 smithi097 ceph-mon[32853]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T01:41:16.313 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_4 2024-04-26T01:41:16.546 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:41:16.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:16 smithi173 ceph-mon[32553]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:16.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:16 smithi173 ceph-mon[32553]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T01:41:16.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:16 smithi001 ceph-mon[27822]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:41:16.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:16 smithi001 ceph-mon[27822]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T01:41:18.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.059+0000 7f8a42525700 1 -- 172.21.15.97:0/1048457618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a3c102fe0 msgr2=0x7f8a3c103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:18.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.059+0000 7f8a42525700 1 --2- 172.21.15.97:0/1048457618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a3c102fe0 0x7f8a3c103400 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009990 tx=0x7f8a30009ca0 comp rx=0 tx=0).stop 2024-04-26T01:41:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.059+0000 7f8a42525700 1 -- 172.21.15.97:0/1048457618 shutdown_connections 2024-04-26T01:41:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.059+0000 7f8a42525700 1 --2- 172.21.15.97:0/1048457618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a3c102600 0x7f8a3c109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.059+0000 7f8a42525700 1 --2- 172.21.15.97:0/1048457618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a3c102fe0 0x7f8a3c103400 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.059+0000 7f8a42525700 1 --2- 172.21.15.97:0/1048457618 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a3c101c40 0x7f8a3c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.059+0000 7f8a42525700 1 -- 172.21.15.97:0/1048457618 >> 172.21.15.97:0/1048457618 conn(0x7f8a3c0fc6f0 msgr2=0x7f8a3c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.059+0000 7f8a42525700 1 -- 172.21.15.97:0/1048457618 shutdown_connections 2024-04-26T01:41:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.059+0000 7f8a42525700 1 -- 172.21.15.97:0/1048457618 wait complete. 2024-04-26T01:41:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.060+0000 7f8a42525700 1 Processor -- start 2024-04-26T01:41:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.060+0000 7f8a42525700 1 -- start start 2024-04-26T01:41:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.060+0000 7f8a42525700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a3c101c40 0x7f8a3c198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.060+0000 7f8a42525700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a3c102600 0x7f8a3c1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:18.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.060+0000 7f8a42525700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a3c102fe0 0x7f8a3c1a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:18.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.060+0000 7f8a42525700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c10e630 con 0x7f8a3c102600 2024-04-26T01:41:18.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.060+0000 7f8a42525700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c10e4b0 con 0x7f8a3c102fe0 2024-04-26T01:41:18.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.060+0000 7f8a42525700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c10e7b0 con 0x7f8a3c101c40 2024-04-26T01:41:18.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.060+0000 7f8a40ac2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a3c102fe0 0x7f8a3c1a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:18.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.060+0000 7f8a40ac2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a3c102fe0 0x7f8a3c1a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35150/0 (socket says 172.21.15.97:35150) 2024-04-26T01:41:18.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.060+0000 7f8a40ac2700 1 -- 172.21.15.97:0/3652169279 learned_addr learned my addr 172.21.15.97:0/3652169279 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T01:41:18.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.060+0000 7f8a3bfff700 1 --2- 172.21.15.97:0/3652169279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a3c101c40 0x7f8a3c198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:18.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.061+0000 7f8a40ac2700 1 -- 172.21.15.97:0/3652169279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a3c101c40 msgr2=0x7f8a3c198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:18.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.061+0000 7f8a40ac2700 1 --2- 172.21.15.97:0/3652169279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a3c101c40 0x7f8a3c198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:18.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.061+0000 7f8a40ac2700 1 -- 172.21.15.97:0/3652169279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a3c102600 msgr2=0x7f8a3c1992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:18.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.061+0000 7f8a40ac2700 1 --2- 172.21.15.97:0/3652169279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a3c102600 0x7f8a3c1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:18.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.061+0000 7f8a40ac2700 1 -- 172.21.15.97:0/3652169279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3c1a09f0 con 0x7f8a3c102fe0 2024-04-26T01:41:18.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.061+0000 7f8a40ac2700 1 --2- 172.21.15.97:0/3652169279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a3c102fe0 0x7f8a3c1a0320 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400b970 tx=0x7f8a3400bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:18.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.061+0000 7f8a397fa700 1 -- 172.21.15.97:0/3652169279 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34019070 con 0x7f8a3c102fe0 2024-04-26T01:41:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.061+0000 7f8a397fa700 1 -- 172.21.15.97:0/3652169279 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a3400cde0 con 0x7f8a3c102fe0 2024-04-26T01:41:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.061+0000 7f8a42525700 1 -- 172.21.15.97:0/3652169279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3c1a0ce0 con 0x7f8a3c102fe0 2024-04-26T01:41:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.061+0000 7f8a397fa700 1 -- 172.21.15.97:0/3652169279 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34007370 con 0x7f8a3c102fe0 2024-04-26T01:41:18.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.061+0000 7f8a42525700 1 -- 172.21.15.97:0/3652169279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a3c1a1240 con 0x7f8a3c102fe0 2024-04-26T01:41:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.062+0000 7f8a42525700 1 -- 172.21.15.97:0/3652169279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a3c069730 con 0x7f8a3c102fe0 2024-04-26T01:41:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.063+0000 7f8a397fa700 1 -- 172.21.15.97:0/3652169279 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) ==== 138589+0+0 (secure 0 0 0) 0x7f8a3401c070 con 0x7f8a3c102fe0 2024-04-26T01:41:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.063+0000 7f8a397fa700 1 --2- 172.21.15.97:0/3652169279 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8a240a3c30 0x7f8a240a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.064+0000 7f8a397fa700 1 -- 172.21.15.97:0/3652169279 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f8a340cde10 con 0x7f8a3c102fe0 2024-04-26T01:41:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.064+0000 7f8a3bfff700 1 --2- 172.21.15.97:0/3652169279 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8a240a3c30 0x7f8a240a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.065+0000 7f8a3bfff700 1 --2- 172.21.15.97:0/3652169279 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8a240a3c30 0x7f8a240a60f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8a30003790 tx=0x7f8a30005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.068+0000 7f8a397fa700 1 -- 172.21.15.97:0/3652169279 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a3409c260 con 0x7f8a3c102fe0 2024-04-26T01:41:18.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:18.274+0000 7f8a42525700 1 -- 172.21.15.97:0/3652169279 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8a3c062e80 con 0x7f8a240a3c30 2024-04-26T01:41:18.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:18 smithi097 ceph-mon[32853]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:18.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:18 smithi173 ceph-mon[32553]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:18.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:18 smithi001 ceph-mon[27822]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:19.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:19 smithi097 ceph-mon[32853]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:41:19.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:19 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:41:19.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:19 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:41:19.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:19 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:19.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:19 smithi097 ceph-mon[32853]: Detected new or changed devices on smithi001 2024-04-26T01:41:19.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:19 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:19.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:19 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:19.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:19 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:41:19.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:19 smithi097 ceph-mon[32853]: Adjusting osd_memory_target on smithi001 to 17094M 2024-04-26T01:41:19.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:19 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:19.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:19 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:19.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:19 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:41:19.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:19 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:19.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:19 smithi173 ceph-mon[32553]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:41:19.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:19 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:41:19.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:19 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:41:19.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:19 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:19.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:19 smithi173 ceph-mon[32553]: Detected new or changed devices on smithi001 2024-04-26T01:41:19.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:19 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:19.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:19 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:19.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:19 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:41:19.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:19 smithi173 ceph-mon[32553]: Adjusting osd_memory_target on smithi001 to 17094M 2024-04-26T01:41:19.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:19 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:19.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:19 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:19.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:19 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:41:19.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:19 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:19.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:19 smithi001 ceph-mon[27822]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:41:19.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:19 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:41:19.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:19 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:41:19.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:19 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:19.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:19 smithi001 ceph-mon[27822]: Detected new or changed devices on smithi001 2024-04-26T01:41:19.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:19 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:19.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:19 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:19.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:19 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:41:19.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:19 smithi001 ceph-mon[27822]: Adjusting osd_memory_target on smithi001 to 17094M 2024-04-26T01:41:19.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:19 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:19.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:19 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:19.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:19 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:41:19.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:19 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:20.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:20 smithi097 ceph-mon[32853]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:20.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:20 smithi173 ceph-mon[32553]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:20.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:20 smithi001 ceph-mon[27822]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:21.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:21 smithi001 ceph-mon[27822]: from='client.? 172.21.15.97:0/1716416384' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acd12888-6ff6-40bc-99e7-e5c395e85b1c"}]: dispatch 2024-04-26T01:41:21.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:21 smithi001 ceph-mon[27822]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acd12888-6ff6-40bc-99e7-e5c395e85b1c"}]: dispatch 2024-04-26T01:41:21.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:21 smithi001 ceph-mon[27822]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "acd12888-6ff6-40bc-99e7-e5c395e85b1c"}]': finished 2024-04-26T01:41:21.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:21 smithi001 ceph-mon[27822]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T01:41:21.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:21 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:41:21.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:21 smithi097 ceph-mon[32853]: from='client.? 172.21.15.97:0/1716416384' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acd12888-6ff6-40bc-99e7-e5c395e85b1c"}]: dispatch 2024-04-26T01:41:21.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:21 smithi097 ceph-mon[32853]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acd12888-6ff6-40bc-99e7-e5c395e85b1c"}]: dispatch 2024-04-26T01:41:21.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:21 smithi097 ceph-mon[32853]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "acd12888-6ff6-40bc-99e7-e5c395e85b1c"}]': finished 2024-04-26T01:41:21.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:21 smithi097 ceph-mon[32853]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T01:41:21.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:21 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:41:21.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:21 smithi173 ceph-mon[32553]: from='client.? 172.21.15.97:0/1716416384' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acd12888-6ff6-40bc-99e7-e5c395e85b1c"}]: dispatch 2024-04-26T01:41:21.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:21 smithi173 ceph-mon[32553]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acd12888-6ff6-40bc-99e7-e5c395e85b1c"}]: dispatch 2024-04-26T01:41:21.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:21 smithi173 ceph-mon[32553]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "acd12888-6ff6-40bc-99e7-e5c395e85b1c"}]': finished 2024-04-26T01:41:21.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:21 smithi173 ceph-mon[32553]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T01:41:21.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:21 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:41:22.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:22 smithi001 ceph-mon[27822]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:22.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:22 smithi001 ceph-mon[27822]: from='client.? 172.21.15.97:0/4148679028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:41:22.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:22 smithi097 ceph-mon[32853]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:22.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:22 smithi097 ceph-mon[32853]: from='client.? 172.21.15.97:0/4148679028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:41:22.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:22 smithi173 ceph-mon[32553]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:22.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:22 smithi173 ceph-mon[32553]: from='client.? 172.21.15.97:0/4148679028' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:41:24.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:24 smithi097 ceph-mon[32853]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:24.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:24 smithi173 ceph-mon[32553]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:24.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:24 smithi001 ceph-mon[27822]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:26.471 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:26 smithi097 ceph-mon[32853]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:26.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:26 smithi173 ceph-mon[32553]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:26.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:26 smithi001 ceph-mon[27822]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:28.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:28 smithi097 ceph-mon[32853]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:28.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:28 smithi173 ceph-mon[32553]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:28.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:28 smithi001 ceph-mon[27822]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:29.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:29 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T01:41:29.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:29 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:29.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:29 smithi097 ceph-mon[32853]: Deploying daemon osd.1 on smithi097 2024-04-26T01:41:29.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T01:41:29.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:29.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:29 smithi173 ceph-mon[32553]: Deploying daemon osd.1 on smithi097 2024-04-26T01:41:29.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T01:41:29.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:29.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:29 smithi001 ceph-mon[27822]: Deploying daemon osd.1 on smithi097 2024-04-26T01:41:30.650 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:30 smithi097 ceph-mon[32853]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:30.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:30 smithi173 ceph-mon[32553]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:30.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:30 smithi001 ceph-mon[27822]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:32.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:32 smithi097 ceph-mon[32853]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:32.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:32 smithi001 ceph-mon[27822]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:32.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:32 smithi173 ceph-mon[32553]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:34.578 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:34 smithi097 ceph-mon[32853]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:34.578 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:34 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:34.578 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:34 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:34.579 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:34 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:41:34.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:34 smithi173 ceph-mon[32553]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:34.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:34 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:34.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:34 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:34.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:34 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:41:34.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:34 smithi001 ceph-mon[27822]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:34.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:34 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:34.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:34 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:34 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:41:36.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:36 smithi097 ceph-mon[32853]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:36.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:36 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:36.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:36 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:36.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:36 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:36.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:36 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:41:36.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:36 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:36.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:36 smithi173 ceph-mon[32553]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:36.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:36 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:36.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:36 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:36.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:36 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:36.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:36 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:41:36.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:36 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:36.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:36 smithi001 ceph-mon[27822]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:36.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:36 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:36.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:36 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:36 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:36 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:41:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:36 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:37.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:37.767+0000 7f8a397fa700 1 -- 172.21.15.97:0/3652169279 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8a3c062e80 con 0x7f8a240a3c30 2024-04-26T01:41:37.765 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 1 on host 'smithi097' 2024-04-26T01:41:37.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:37.770+0000 7f8a42525700 1 -- 172.21.15.97:0/3652169279 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8a240a3c30 msgr2=0x7f8a240a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:37.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:37.770+0000 7f8a42525700 1 --2- 172.21.15.97:0/3652169279 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8a240a3c30 0x7f8a240a60f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8a30003790 tx=0x7f8a30005fb0 comp rx=0 tx=0).stop 2024-04-26T01:41:37.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:37.770+0000 7f8a42525700 1 -- 172.21.15.97:0/3652169279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a3c102fe0 msgr2=0x7f8a3c1a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:37.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:37.770+0000 7f8a42525700 1 --2- 172.21.15.97:0/3652169279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a3c102fe0 0x7f8a3c1a0320 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400b970 tx=0x7f8a3400bc80 comp rx=0 tx=0).stop 2024-04-26T01:41:37.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:37.770+0000 7f8a42525700 1 -- 172.21.15.97:0/3652169279 shutdown_connections 2024-04-26T01:41:37.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:37.770+0000 7f8a42525700 1 --2- 172.21.15.97:0/3652169279 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8a240a3c30 0x7f8a240a60f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:37.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:37.770+0000 7f8a42525700 1 --2- 172.21.15.97:0/3652169279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a3c102fe0 0x7f8a3c1a0320 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:37.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:37.770+0000 7f8a42525700 1 --2- 172.21.15.97:0/3652169279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a3c101c40 0x7f8a3c198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:37.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:37.770+0000 7f8a42525700 1 --2- 172.21.15.97:0/3652169279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a3c102600 0x7f8a3c1992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:37.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:37.770+0000 7f8a42525700 1 -- 172.21.15.97:0/3652169279 >> 172.21.15.97:0/3652169279 conn(0x7f8a3c0fc6f0 msgr2=0x7f8a3c106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:37.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:37.770+0000 7f8a42525700 1 -- 172.21.15.97:0/3652169279 shutdown_connections 2024-04-26T01:41:37.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:41:37.770+0000 7f8a42525700 1 -- 172.21.15.97:0/3652169279 wait complete. 2024-04-26T01:41:38.367 DEBUG:teuthology.orchestra.run.smithi097:osd.1> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.1.service 2024-04-26T01:41:38.370 INFO:tasks.cephadm:Deploying osd.2 on smithi173 with /dev/vg_nvme/lv_4... 2024-04-26T01:41:38.370 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T01:41:38.614 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:41:38.666 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:38 smithi097 ceph-mon[32853]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:38.666 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:38 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:38.666 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:38 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:38.666 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:38 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:41:38.666 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:38 smithi097 ceph-mon[32853]: from='osd.1 [v2:172.21.15.97:6800/3755022621,v1:172.21.15.97:6801/3755022621]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T01:41:38.666 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:38 smithi097 ceph-mon[32853]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T01:41:38.720 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:38 smithi173 ceph-mon[32553]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:38.720 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:38 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:38.720 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:38 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:38.720 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:38 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:41:38.720 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:38 smithi173 ceph-mon[32553]: from='osd.1 [v2:172.21.15.97:6800/3755022621,v1:172.21.15.97:6801/3755022621]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T01:41:38.720 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:38 smithi173 ceph-mon[32553]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T01:41:38.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:38 smithi001 ceph-mon[27822]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:38.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:38 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:38.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:38 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:38.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:38 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:41:38.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:38 smithi001 ceph-mon[27822]: from='osd.1 [v2:172.21.15.97:6800/3755022621,v1:172.21.15.97:6801/3755022621]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T01:41:38.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:38 smithi001 ceph-mon[27822]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T01:41:39.788 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:39 smithi097 ceph-mon[32853]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:39.789 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:39 smithi097 ceph-mon[32853]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T01:41:39.789 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:39 smithi097 ceph-mon[32853]: from='osd.1 [v2:172.21.15.97:6800/3755022621,v1:172.21.15.97:6801/3755022621]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:41:39.789 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:39 smithi097 ceph-mon[32853]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T01:41:39.789 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:41:39.789 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:39 smithi097 ceph-mon[32853]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:41:40.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:39 smithi173 ceph-mon[32553]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:40.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:39 smithi173 ceph-mon[32553]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T01:41:40.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:39 smithi173 ceph-mon[32553]: from='osd.1 [v2:172.21.15.97:6800/3755022621,v1:172.21.15.97:6801/3755022621]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:41:40.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:39 smithi173 ceph-mon[32553]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T01:41:40.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:41:40.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:39 smithi173 ceph-mon[32553]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:41:40.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:39 smithi001 ceph-mon[27822]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:40.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:39 smithi001 ceph-mon[27822]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T01:41:40.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:39 smithi001 ceph-mon[27822]: from='osd.1 [v2:172.21.15.97:6800/3755022621,v1:172.21.15.97:6801/3755022621]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:41:40.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:39 smithi001 ceph-mon[27822]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T01:41:40.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:41:40.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:39 smithi001 ceph-mon[27822]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:41:40.318 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T01:41:40.318 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T01:41:40.318 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-04-26T01:41:40.318 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-04-26T01:41:40.318 INFO:teuthology.orchestra.run.smithi173.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114447 s, 916 MB/s 2024-04-26T01:41:40.318 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-04-26T01:41:40.850 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_4 2024-04-26T01:41:40.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:40 smithi097 ceph-mon[32853]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T01:41:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:40 smithi097 ceph-mon[32853]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T01:41:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:40 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:41:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:40 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:41:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:40 smithi097 ceph-mon[32853]: from='osd.1 ' entity='osd.1' 2024-04-26T01:41:40.983 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 01:41:40 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-1[37441]: 2024-04-26T01:41:40.634+0000 7fb235ac6700 -1 osd.1 0 waiting for initial osdmap 2024-04-26T01:41:40.983 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 01:41:40 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-1[37441]: 2024-04-26T01:41:40.816+0000 7fb22d8b1700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:41:41.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:40 smithi173 ceph-mon[32553]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T01:41:41.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:40 smithi173 ceph-mon[32553]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T01:41:41.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:40 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:41:41.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:40 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:41:41.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:40 smithi173 ceph-mon[32553]: from='osd.1 ' entity='osd.1' 2024-04-26T01:41:41.093 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:41:41.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:40 smithi001 ceph-mon[27822]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T01:41:41.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:40 smithi001 ceph-mon[27822]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T01:41:41.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:40 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:41:41.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:40 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:41:41.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:40 smithi001 ceph-mon[27822]: from='osd.1 ' entity='osd.1' 2024-04-26T01:41:41.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:41 smithi173 ceph-mon[32553]: purged_snaps scrub starts 2024-04-26T01:41:41.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:41 smithi173 ceph-mon[32553]: purged_snaps scrub ok 2024-04-26T01:41:41.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:41 smithi173 ceph-mon[32553]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:41.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:41 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:41:41.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:41 smithi173 ceph-mon[32553]: Detected new or changed devices on smithi097 2024-04-26T01:41:41.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:41 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:41.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:41 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:41.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:41 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:41:41.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:41 smithi173 ceph-mon[32553]: Adjusting osd_memory_target on smithi097 to 17088M 2024-04-26T01:41:41.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:41 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:41.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:41 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:41.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:41 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:41:41.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:41 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:41.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:41 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:41:42.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:41 smithi001 ceph-mon[27822]: purged_snaps scrub starts 2024-04-26T01:41:42.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:41 smithi001 ceph-mon[27822]: purged_snaps scrub ok 2024-04-26T01:41:42.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:41 smithi001 ceph-mon[27822]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:42.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:41 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:41:42.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:41 smithi001 ceph-mon[27822]: Detected new or changed devices on smithi097 2024-04-26T01:41:42.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:41 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:42.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:41 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:42.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:41 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:41:42.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:41 smithi001 ceph-mon[27822]: Adjusting osd_memory_target on smithi097 to 17088M 2024-04-26T01:41:42.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:41 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:42.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:41 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:42.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:41 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:41:42.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:41 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:42.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:41 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:41:42.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:41 smithi097 ceph-mon[32853]: purged_snaps scrub starts 2024-04-26T01:41:42.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:41 smithi097 ceph-mon[32853]: purged_snaps scrub ok 2024-04-26T01:41:42.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:41 smithi097 ceph-mon[32853]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:42.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:41 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:41:42.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:41 smithi097 ceph-mon[32853]: Detected new or changed devices on smithi097 2024-04-26T01:41:42.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:41 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:42.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:41 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:42.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:41 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:41:42.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:41 smithi097 ceph-mon[32853]: Adjusting osd_memory_target on smithi097 to 17088M 2024-04-26T01:41:42.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:41 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:42.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:41 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:42.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:41 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:41:42.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:41 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:42.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:41 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:41:42.729 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.729+0000 7f6eddd50700 1 -- 172.21.15.173:0/1909238696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ed8102330 msgr2=0x7f6ed8102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:42.729 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.729+0000 7f6eddd50700 1 --2- 172.21.15.173:0/1909238696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ed8102330 0x7f6ed8102790 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000aaa0 tx=0x7f6ec000adb0 comp rx=0 tx=0).stop 2024-04-26T01:41:42.729 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.729+0000 7f6eddd50700 1 -- 172.21.15.173:0/1909238696 shutdown_connections 2024-04-26T01:41:42.729 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.729+0000 7f6eddd50700 1 --2- 172.21.15.173:0/1909238696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ed8102cd0 0x7f6ed8109580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:42.729 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.729+0000 7f6eddd50700 1 --2- 172.21.15.173:0/1909238696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed8101130 0x7f6ed8101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:42.729 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.729+0000 7f6eddd50700 1 --2- 172.21.15.173:0/1909238696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ed8102330 0x7f6ed8102790 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:42.729 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.729+0000 7f6eddd50700 1 -- 172.21.15.173:0/1909238696 >> 172.21.15.173:0/1909238696 conn(0x7f6ed80fc6b0 msgr2=0x7f6ed80feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:41:42.729 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.729+0000 7f6eddd50700 1 -- 172.21.15.173:0/1909238696 shutdown_connections 2024-04-26T01:41:42.730 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.729+0000 7f6eddd50700 1 -- 172.21.15.173:0/1909238696 wait complete. 2024-04-26T01:41:42.730 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.730+0000 7f6eddd50700 1 Processor -- start 2024-04-26T01:41:42.730 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.730+0000 7f6eddd50700 1 -- start start 2024-04-26T01:41:42.730 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.730+0000 7f6eddd50700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ed8101130 0x7f6ed81050c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:42.730 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.730+0000 7f6eddd50700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ed8102330 0x7f6ed8103710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:42.730 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.730+0000 7f6eddd50700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed8102cd0 0x7f6ed8103c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:42.730 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.730+0000 7f6eddd50700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed810c3b0 con 0x7f6ed8102330 2024-04-26T01:41:42.730 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.730+0000 7f6eddd50700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed810c230 con 0x7f6ed8102cd0 2024-04-26T01:41:42.730 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.730+0000 7f6eddd50700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed810c530 con 0x7f6ed8101130 2024-04-26T01:41:42.730 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.730+0000 7f6ed77fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ed8101130 0x7f6ed81050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:42.730 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.730+0000 7f6ed77fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ed8101130 0x7f6ed81050c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:42244/0 (socket says 172.21.15.173:42244) 2024-04-26T01:41:42.730 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.730+0000 7f6ed77fe700 1 -- 172.21.15.173:0/607093148 learned_addr learned my addr 172.21.15.173:0/607093148 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-26T01:41:42.730 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.731+0000 7f6ed7fff700 1 --2- 172.21.15.173:0/607093148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed8102cd0 0x7f6ed8103c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:42.731 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.731+0000 7f6ed77fe700 1 -- 172.21.15.173:0/607093148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed8102cd0 msgr2=0x7f6ed8103c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:42.731 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.731+0000 7f6ed77fe700 1 --2- 172.21.15.173:0/607093148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed8102cd0 0x7f6ed8103c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:42.731 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.731+0000 7f6ed77fe700 1 -- 172.21.15.173:0/607093148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ed8102330 msgr2=0x7f6ed8103710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:41:42.731 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.731+0000 7f6ed77fe700 1 --2- 172.21.15.173:0/607093148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ed8102330 0x7f6ed8103710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:41:42.731 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.731+0000 7f6ed77fe700 1 -- 172.21.15.173:0/607093148 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed8104500 con 0x7f6ed8101130 2024-04-26T01:41:42.731 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.731+0000 7f6ed77fe700 1 --2- 172.21.15.173:0/607093148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ed8101130 0x7f6ed81050c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800d980 tx=0x7f6ec800dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:42.731 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.731+0000 7f6ed4ff9700 1 -- 172.21.15.173:0/607093148 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8017070 con 0x7f6ed8101130 2024-04-26T01:41:42.731 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.731+0000 7f6ed4ff9700 1 -- 172.21.15.173:0/607093148 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec8010460 con 0x7f6ed8101130 2024-04-26T01:41:42.731 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.731+0000 7f6eddd50700 1 -- 172.21.15.173:0/607093148 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed81a2e30 con 0x7f6ed8101130 2024-04-26T01:41:42.731 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.731+0000 7f6ed4ff9700 1 -- 172.21.15.173:0/607093148 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec800de60 con 0x7f6ed8101130 2024-04-26T01:41:42.731 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.731+0000 7f6eddd50700 1 -- 172.21.15.173:0/607093148 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed81a31d0 con 0x7f6ed8101130 2024-04-26T01:41:42.732 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.732+0000 7f6eddd50700 1 -- 172.21.15.173:0/607093148 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed8069760 con 0x7f6ed8101130 2024-04-26T01:41:42.732 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.733+0000 7f6ed4ff9700 1 -- 172.21.15.173:0/607093148 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 15) ==== 138589+0+0 (secure 0 0 0) 0x7f6ec8009b40 con 0x7f6ed8101130 2024-04-26T01:41:42.732 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.734+0000 7f6ed4ff9700 1 --2- 172.21.15.173:0/607093148 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f6ec40a3bb0 0x7f6ec40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:41:42.733 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.734+0000 7f6ed4ff9700 1 -- 172.21.15.173:0/607093148 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f6ec80ce720 con 0x7f6ed8101130 2024-04-26T01:41:42.733 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.734+0000 7f6ed6ffd700 1 --2- 172.21.15.173:0/607093148 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f6ec40a3bb0 0x7f6ec40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:41:42.733 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.735+0000 7f6ed6ffd700 1 --2- 172.21.15.173:0/607093148 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f6ec40a3bb0 0x7f6ec40a6070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000c6d0 tx=0x7f6ec0003cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:41:42.736 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.738+0000 7f6ed4ff9700 1 -- 172.21.15.173:0/607093148 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ed8069760 con 0x7f6ed8101130 2024-04-26T01:41:42.941 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:41:42.944+0000 7f6eddd50700 1 -- 172.21.15.173:0/607093148 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6ed8108f00 con 0x7f6ec40a3bb0 2024-04-26T01:41:43.018 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:42 smithi097 ceph-mon[32853]: osd.1 [v2:172.21.15.97:6800/3755022621,v1:172.21.15.97:6801/3755022621] boot 2024-04-26T01:41:43.018 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:42 smithi097 ceph-mon[32853]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T01:41:43.019 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:42 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:41:43.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:42 smithi173 ceph-mon[32553]: osd.1 [v2:172.21.15.97:6800/3755022621,v1:172.21.15.97:6801/3755022621] boot 2024-04-26T01:41:43.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:42 smithi173 ceph-mon[32553]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T01:41:43.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:42 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:41:43.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:42 smithi001 ceph-mon[27822]: osd.1 [v2:172.21.15.97:6800/3755022621,v1:172.21.15.97:6801/3755022621] boot 2024-04-26T01:41:43.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:42 smithi001 ceph-mon[27822]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T01:41:43.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:42 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:41:43.835 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:43 smithi173 ceph-mon[32553]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:43.835 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:43 smithi173 ceph-mon[32553]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T01:41:43.835 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:43 smithi173 ceph-mon[32553]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:41:43.835 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:43 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:41:43.835 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:43 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:41:43.835 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:43 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:44.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:43 smithi001 ceph-mon[27822]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:44.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:43 smithi001 ceph-mon[27822]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T01:41:44.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:43 smithi001 ceph-mon[27822]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:41:44.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:43 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:41:44.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:43 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:41:44.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:43 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:44.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:43 smithi097 ceph-mon[32853]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:41:44.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:43 smithi097 ceph-mon[32853]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T01:41:44.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:43 smithi097 ceph-mon[32853]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:41:44.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:43 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:41:44.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:43 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:41:44.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:43 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:46.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:45 smithi173 ceph-mon[32553]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:41:46.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:45 smithi173 ceph-mon[32553]: from='client.? 172.21.15.173:0/1862607251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79971508-e256-4145-8111-a75666e886fc"}]: dispatch 2024-04-26T01:41:46.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:45 smithi173 ceph-mon[32553]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79971508-e256-4145-8111-a75666e886fc"}]: dispatch 2024-04-26T01:41:46.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:45 smithi173 ceph-mon[32553]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79971508-e256-4145-8111-a75666e886fc"}]': finished 2024-04-26T01:41:46.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:45 smithi173 ceph-mon[32553]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T01:41:46.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:45 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:41:46.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:45 smithi001 ceph-mon[27822]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:41:46.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:45 smithi001 ceph-mon[27822]: from='client.? 172.21.15.173:0/1862607251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79971508-e256-4145-8111-a75666e886fc"}]: dispatch 2024-04-26T01:41:46.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:45 smithi001 ceph-mon[27822]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79971508-e256-4145-8111-a75666e886fc"}]: dispatch 2024-04-26T01:41:46.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:45 smithi001 ceph-mon[27822]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79971508-e256-4145-8111-a75666e886fc"}]': finished 2024-04-26T01:41:46.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:45 smithi001 ceph-mon[27822]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T01:41:46.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:45 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:41:46.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:45 smithi097 ceph-mon[32853]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:41:46.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:45 smithi097 ceph-mon[32853]: from='client.? 172.21.15.173:0/1862607251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79971508-e256-4145-8111-a75666e886fc"}]: dispatch 2024-04-26T01:41:46.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:45 smithi097 ceph-mon[32853]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79971508-e256-4145-8111-a75666e886fc"}]: dispatch 2024-04-26T01:41:46.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:45 smithi097 ceph-mon[32853]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79971508-e256-4145-8111-a75666e886fc"}]': finished 2024-04-26T01:41:46.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:45 smithi097 ceph-mon[32853]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T01:41:46.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:45 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:41:47.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:46 smithi173 ceph-mon[32553]: from='client.? 172.21.15.173:0/2154229537' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:41:47.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:46 smithi001 ceph-mon[27822]: from='client.? 172.21.15.173:0/2154229537' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:41:47.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:46 smithi097 ceph-mon[32853]: from='client.? 172.21.15.173:0/2154229537' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:41:48.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:47 smithi173 ceph-mon[32553]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:41:48.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:47 smithi001 ceph-mon[27822]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:41:48.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:47 smithi097 ceph-mon[32853]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:41:49.980 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:49 smithi173 ceph-mon[32553]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:41:50.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:49 smithi001 ceph-mon[27822]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:41:50.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:49 smithi097 ceph-mon[32853]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:41:52.029 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:51 smithi173 ceph-mon[32553]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:41:52.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:51 smithi001 ceph-mon[27822]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:41:52.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:51 smithi097 ceph-mon[32853]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:41:53.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T01:41:53.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:53.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T01:41:53.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:53.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T01:41:53.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:41:54.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:53 smithi173 ceph-mon[32553]: Deploying daemon osd.2 on smithi173 2024-04-26T01:41:54.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:53 smithi173 ceph-mon[32553]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:41:54.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:53 smithi001 ceph-mon[27822]: Deploying daemon osd.2 on smithi173 2024-04-26T01:41:54.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:53 smithi001 ceph-mon[27822]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:41:54.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:53 smithi097 ceph-mon[32853]: Deploying daemon osd.2 on smithi173 2024-04-26T01:41:54.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:53 smithi097 ceph-mon[32853]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:41:56.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:55 smithi173 ceph-mon[32553]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:41:56.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:55 smithi001 ceph-mon[27822]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:41:56.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:55 smithi097 ceph-mon[32853]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:41:58.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:57 smithi173 ceph-mon[32553]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:41:58.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:57 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:58.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:57 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:41:58.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:57 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:58.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:57 smithi001 ceph-mon[27822]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:41:58.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:57 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:58.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:57 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:41:58.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:57 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:58.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:57 smithi097 ceph-mon[32853]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:41:58.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:57 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:41:58.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:57 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:41:58.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:57 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:00.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:41:59 smithi173 ceph-mon[32553]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:42:00.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:41:59 smithi001 ceph-mon[27822]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:42:00.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:41:59 smithi097 ceph-mon[32853]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:42:01.225 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:00 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:01.225 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:00 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:01.225 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:00 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:01.225 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:00 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:01.225 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:00 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:01.225 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:42:01.228+0000 7f6ed4ff9700 1 -- 172.21.15.173:0/607093148 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6ed8108f00 con 0x7f6ec40a3bb0 2024-04-26T01:42:01.225 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 2 on host 'smithi173' 2024-04-26T01:42:01.227 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:42:01.230+0000 7f6eddd50700 1 -- 172.21.15.173:0/607093148 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f6ec40a3bb0 msgr2=0x7f6ec40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:01.227 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:42:01.230+0000 7f6eddd50700 1 --2- 172.21.15.173:0/607093148 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f6ec40a3bb0 0x7f6ec40a6070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000c6d0 tx=0x7f6ec0003cf0 comp rx=0 tx=0).stop 2024-04-26T01:42:01.227 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:42:01.230+0000 7f6eddd50700 1 -- 172.21.15.173:0/607093148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ed8101130 msgr2=0x7f6ed81050c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:01.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:42:01.230+0000 7f6eddd50700 1 --2- 172.21.15.173:0/607093148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ed8101130 0x7f6ed81050c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800d980 tx=0x7f6ec800dd40 comp rx=0 tx=0).stop 2024-04-26T01:42:01.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:42:01.230+0000 7f6eddd50700 1 -- 172.21.15.173:0/607093148 shutdown_connections 2024-04-26T01:42:01.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:42:01.230+0000 7f6eddd50700 1 --2- 172.21.15.173:0/607093148 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f6ec40a3bb0 0x7f6ec40a6070 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:01.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:42:01.230+0000 7f6eddd50700 1 --2- 172.21.15.173:0/607093148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed8102cd0 0x7f6ed8103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:01.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:42:01.230+0000 7f6eddd50700 1 --2- 172.21.15.173:0/607093148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ed8101130 0x7f6ed81050c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:01.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:42:01.230+0000 7f6eddd50700 1 --2- 172.21.15.173:0/607093148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ed8102330 0x7f6ed8103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:01.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:42:01.230+0000 7f6eddd50700 1 -- 172.21.15.173:0/607093148 >> 172.21.15.173:0/607093148 conn(0x7f6ed80fc6b0 msgr2=0x7f6ed81077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:01.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:42:01.230+0000 7f6eddd50700 1 -- 172.21.15.173:0/607093148 shutdown_connections 2024-04-26T01:42:01.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:42:01.231+0000 7f6eddd50700 1 -- 172.21.15.173:0/607093148 wait complete. 2024-04-26T01:42:01.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:00 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:01.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:00 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:01.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:00 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:01.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:00 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:01.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:00 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:01.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:00 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:01.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:00 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:01.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:00 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:01.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:00 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:01.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:00 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:01.784 DEBUG:teuthology.orchestra.run.smithi173:osd.2> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.2.service 2024-04-26T01:42:01.786 INFO:tasks.cephadm:Deploying osd.3 on smithi001 with /dev/vg_nvme/lv_3... 2024-04-26T01:42:01.787 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T01:42:02.248 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:42:02.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:02 smithi097 ceph-mon[32853]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:42:02.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:02 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:02.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:02 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:02 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:02 smithi097 ceph-mon[32853]: from='osd.2 [v2:172.21.15.173:6800/734861771,v1:172.21.15.173:6801/734861771]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T01:42:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:02 smithi097 ceph-mon[32853]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T01:42:02.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:02 smithi173 ceph-mon[32553]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:42:02.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:02 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:02.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:02 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:02.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:02 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:02.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:02 smithi173 ceph-mon[32553]: from='osd.2 [v2:172.21.15.173:6800/734861771,v1:172.21.15.173:6801/734861771]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T01:42:02.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:02 smithi173 ceph-mon[32553]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T01:42:02.587 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:02 smithi001 ceph-mon[27822]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:42:02.587 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:02 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:02.587 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:02 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:02.587 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:02 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:02.587 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:02 smithi001 ceph-mon[27822]: from='osd.2 [v2:172.21.15.173:6800/734861771,v1:172.21.15.173:6801/734861771]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T01:42:02.587 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:02 smithi001 ceph-mon[27822]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T01:42:03.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:03 smithi097 ceph-mon[32853]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T01:42:03.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:03 smithi097 ceph-mon[32853]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T01:42:03.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:42:03.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:03 smithi097 ceph-mon[32853]: from='osd.2 [v2:172.21.15.173:6800/734861771,v1:172.21.15.173:6801/734861771]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:42:03.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:03 smithi097 ceph-mon[32853]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:42:03.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:03 smithi173 ceph-mon[32553]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T01:42:03.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:03 smithi173 ceph-mon[32553]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T01:42:03.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:42:03.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:03 smithi173 ceph-mon[32553]: from='osd.2 [v2:172.21.15.173:6800/734861771,v1:172.21.15.173:6801/734861771]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:42:03.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:03 smithi173 ceph-mon[32553]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:42:03.567 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 01:42:03 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-2[37698]: 2024-04-26T01:42:03.211+0000 7fd79aec3700 -1 osd.2 0 waiting for initial osdmap 2024-04-26T01:42:03.567 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 01:42:03 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-2[37698]: 2024-04-26T01:42:03.218+0000 7fd796cb6700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:42:03.580 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:03 smithi001 ceph-mon[27822]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T01:42:03.580 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:03 smithi001 ceph-mon[27822]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T01:42:03.580 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:42:03.580 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:03 smithi001 ceph-mon[27822]: from='osd.2 [v2:172.21.15.173:6800/734861771,v1:172.21.15.173:6801/734861771]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:42:03.580 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:03 smithi001 ceph-mon[27822]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:42:04.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:04 smithi097 ceph-mon[32853]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:42:04.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:04 smithi097 ceph-mon[32853]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-26T01:42:04.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:04 smithi097 ceph-mon[32853]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T01:42:04.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:04 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:42:04.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:04 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:42:04.501 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:04 smithi001 ceph-mon[27822]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:42:04.501 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:04 smithi001 ceph-mon[27822]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-26T01:42:04.501 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:04 smithi001 ceph-mon[27822]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T01:42:04.501 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:04 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:42:04.501 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:04 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:42:04.563 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:04 smithi173 ceph-mon[32553]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:42:04.563 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:04 smithi173 ceph-mon[32553]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-26T01:42:04.563 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:04 smithi173 ceph-mon[32553]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T01:42:04.563 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:04 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:42:04.563 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:04 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:42:05.076 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T01:42:05.076 INFO:teuthology.orchestra.run.smithi001.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T01:42:05.077 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10+0 records in 2024-04-26T01:42:05.077 INFO:teuthology.orchestra.run.smithi001.stderr:10+0 records out 2024-04-26T01:42:05.077 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0357999 s, 293 MB/s 2024-04-26T01:42:05.077 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping successful for: 2024-04-26T01:42:05.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:05 smithi001 ceph-mon[27822]: purged_snaps scrub starts 2024-04-26T01:42:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:05 smithi001 ceph-mon[27822]: purged_snaps scrub ok 2024-04-26T01:42:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:05 smithi001 ceph-mon[27822]: osd.2 [v2:172.21.15.173:6800/734861771,v1:172.21.15.173:6801/734861771] boot 2024-04-26T01:42:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:05 smithi001 ceph-mon[27822]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T01:42:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:05 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:42:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:05 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:05 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:05 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:42:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:05 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:05 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:05 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:05 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:05 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T01:42:05.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:05 smithi097 ceph-mon[32853]: purged_snaps scrub starts 2024-04-26T01:42:05.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:05 smithi097 ceph-mon[32853]: purged_snaps scrub ok 2024-04-26T01:42:05.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:05 smithi097 ceph-mon[32853]: osd.2 [v2:172.21.15.173:6800/734861771,v1:172.21.15.173:6801/734861771] boot 2024-04-26T01:42:05.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:05 smithi097 ceph-mon[32853]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T01:42:05.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:05 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:42:05.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:05 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:05.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:05 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:05.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:05 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:42:05.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:05 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:05.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:05 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:05.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:05 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:05.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:05 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:05.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:05 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T01:42:05.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:05 smithi173 ceph-mon[32553]: purged_snaps scrub starts 2024-04-26T01:42:05.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:05 smithi173 ceph-mon[32553]: purged_snaps scrub ok 2024-04-26T01:42:05.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:05 smithi173 ceph-mon[32553]: osd.2 [v2:172.21.15.173:6800/734861771,v1:172.21.15.173:6801/734861771] boot 2024-04-26T01:42:05.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:05 smithi173 ceph-mon[32553]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T01:42:05.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:05 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:42:05.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:05 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:05.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:05 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:05.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:05 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:42:05.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:05 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:05.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:05 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:05.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:05 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:05.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:05 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:05.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:05 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T01:42:06.108 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi001:vg_nvme/lv_3 2024-04-26T01:42:06.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:06 smithi001 ceph-mon[27822]: Detected new or changed devices on smithi173 2024-04-26T01:42:06.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:06 smithi001 ceph-mon[27822]: Adjusting osd_memory_target on smithi173 to 17088M 2024-04-26T01:42:06.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:06 smithi001 ceph-mon[27822]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:42:06.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:06 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T01:42:06.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:06 smithi001 ceph-mon[27822]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T01:42:06.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:06 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T01:42:06.557 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:42:06.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:06 smithi173 ceph-mon[32553]: Detected new or changed devices on smithi173 2024-04-26T01:42:06.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:06 smithi173 ceph-mon[32553]: Adjusting osd_memory_target on smithi173 to 17088M 2024-04-26T01:42:06.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:06 smithi173 ceph-mon[32553]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:42:06.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:06 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T01:42:06.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:06 smithi173 ceph-mon[32553]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T01:42:06.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:06 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T01:42:06.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:06 smithi097 ceph-mon[32853]: Detected new or changed devices on smithi173 2024-04-26T01:42:06.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:06 smithi097 ceph-mon[32853]: Adjusting osd_memory_target on smithi173 to 17088M 2024-04-26T01:42:06.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:06 smithi097 ceph-mon[32853]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:42:06.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:06 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T01:42:06.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:06 smithi097 ceph-mon[32853]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T01:42:06.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:06 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T01:42:07.353 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:07 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T01:42:07.353 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:07 smithi001 ceph-mon[27822]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T01:42:07.353 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:07 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:07.353 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:07 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:07.353 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 01:42:07 smithi001 sudo[44658]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T01:42:07.353 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 01:42:07 smithi001 sudo[44658]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:42:07.354 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 01:42:07 smithi001 sudo[44658]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:42:07.354 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 01:42:07 smithi001 sudo[44658]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:42:07.354 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 01:42:07 smithi001 sudo[44661]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T01:42:07.354 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 01:42:07 smithi001 sudo[44661]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:42:07.354 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 01:42:07 smithi001 sudo[44661]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:42:07.354 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 01:42:07 smithi001 sudo[44661]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:42:07.432 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:07 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T01:42:07.432 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:07 smithi173 ceph-mon[32553]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T01:42:07.432 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:07 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:07.432 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:07 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:07.432 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 01:42:07 smithi173 sudo[41299]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T01:42:07.433 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 01:42:07 smithi173 sudo[41299]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:42:07.433 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 01:42:07 smithi173 sudo[41299]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:42:07.433 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 01:42:07 smithi173 sudo[41299]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:42:07.433 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 01:42:07 smithi173 sudo[41303]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T01:42:07.433 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 01:42:07 smithi173 sudo[41303]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:42:07.433 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 01:42:07 smithi173 sudo[41303]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:42:07.433 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 01:42:07 smithi173 sudo[41303]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:42:07.472 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:07 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T01:42:07.472 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:07 smithi097 ceph-mon[32853]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T01:42:07.472 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:07 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:07.473 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:07 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:07.473 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 01:42:07 smithi097 sudo[41137]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T01:42:07.473 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 01:42:07 smithi097 sudo[41137]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:42:07.473 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 01:42:07 smithi097 sudo[41137]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:42:07.473 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 01:42:07 smithi097 sudo[41137]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:42:07.473 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 01:42:07 smithi097 sudo[41140]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T01:42:07.473 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 01:42:07 smithi097 sudo[41140]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:42:07.473 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 01:42:07 smithi097 sudo[41140]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:42:07.473 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 01:42:07 smithi097 sudo[41140]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:42:07.618 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:07 smithi001 sudo[44677]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T01:42:07.618 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:07 smithi001 sudo[44677]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:42:07.618 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:07 smithi001 sudo[44677]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:42:07.618 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:07 smithi001 sudo[44677]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:42:07.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:07 smithi001 sudo[44680]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T01:42:07.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:07 smithi001 sudo[44680]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:42:07.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:07 smithi001 sudo[44680]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:42:07.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:07 smithi001 sudo[44680]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:42:07.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:07 smithi097 sudo[41143]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T01:42:07.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:07 smithi097 sudo[41143]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:42:07.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:07 smithi097 sudo[41143]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:42:07.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:07 smithi097 sudo[41143]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:42:07.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:07 smithi097 sudo[41146]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T01:42:07.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:07 smithi097 sudo[41146]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:42:07.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:07 smithi097 sudo[41146]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:42:07.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:07 smithi097 sudo[41146]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:42:07.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:07 smithi173 sudo[41311]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T01:42:07.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:07 smithi173 sudo[41311]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:42:07.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:07 smithi173 sudo[41311]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:42:07.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:07 smithi173 sudo[41311]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:42:07.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:07 smithi173 sudo[41314]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T01:42:07.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:07 smithi173 sudo[41314]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:42:07.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:07 smithi173 sudo[41314]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:42:07.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:07 smithi173 sudo[41314]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:42:08.558 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:08 smithi001 ceph-mon[27822]: pgmap v93: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:08.558 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:08 smithi001 ceph-mon[27822]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T01:42:08.558 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:08 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:08.558 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:08 smithi001 ceph-mon[27822]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:42:08.558 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:08 smithi001 ceph-mon[27822]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:42:08.558 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:08 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:42:08.558 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:08 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:08.558 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:08 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:08.558 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:08 smithi001 ceph-mon[27822]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:42:08.558 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:08 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:42:08.558 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:08 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:08.559 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:08 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:08.559 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:08 smithi001 ceph-mon[27822]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:42:08.559 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:08 smithi001 ceph-mon[27822]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:42:08.559 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:08 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:42:08.559 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:08 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:08.559 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:08 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:08.559 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:08 smithi001 ceph-mon[27822]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:42:08.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:08 smithi173 ceph-mon[32553]: pgmap v93: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:08.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:08 smithi173 ceph-mon[32553]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T01:42:08.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:08 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:08.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:08 smithi173 ceph-mon[32553]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:42:08.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:08 smithi173 ceph-mon[32553]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:42:08.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:08 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:42:08.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:08 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:08 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:08 smithi173 ceph-mon[32553]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:42:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:08 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:42:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:08 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:08 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:08 smithi173 ceph-mon[32553]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:42:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:08 smithi173 ceph-mon[32553]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:42:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:08 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:42:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:08 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:08 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:08 smithi173 ceph-mon[32553]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:42:08.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:08 smithi097 ceph-mon[32853]: pgmap v93: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:08.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:08 smithi097 ceph-mon[32853]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T01:42:08.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:08 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:08.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:08 smithi097 ceph-mon[32853]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:42:08.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:08 smithi097 ceph-mon[32853]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:42:08.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:08 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:42:08.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:08 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:08.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:08 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:08.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:08 smithi097 ceph-mon[32853]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:42:08.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:08 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:42:08.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:08 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:08.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:08 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:08.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:08 smithi097 ceph-mon[32853]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:42:08.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:08 smithi097 ceph-mon[32853]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:42:08.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:08 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:42:08.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:08 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:42:08.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:08 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:42:08.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:08 smithi097 ceph-mon[32853]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:42:08.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.737+0000 7f691fe00700 1 -- 172.21.15.1:0/1665925086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692010a8b0 msgr2=0x7f692010cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:08.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.737+0000 7f691fe00700 1 --2- 172.21.15.1:0/1665925086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692010a8b0 0x7f692010cd40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f691800b0a0 tx=0x7f691800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:42:08.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.737+0000 7f691fe00700 1 -- 172.21.15.1:0/1665925086 shutdown_connections 2024-04-26T01:42:08.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.737+0000 7f691fe00700 1 --2- 172.21.15.1:0/1665925086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692010a8b0 0x7f692010cd40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:08.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.737+0000 7f691fe00700 1 --2- 172.21.15.1:0/1665925086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6920076a70 0x7f6920074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:08.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.737+0000 7f691fe00700 1 --2- 172.21.15.1:0/1665925086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69200754a0 0x7f6920075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:08.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.737+0000 7f691fe00700 1 -- 172.21.15.1:0/1665925086 >> 172.21.15.1:0/1665925086 conn(0x7f69200fdf80 msgr2=0x7f69201003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:08.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.738+0000 7f691fe00700 1 -- 172.21.15.1:0/1665925086 shutdown_connections 2024-04-26T01:42:08.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.738+0000 7f691fe00700 1 -- 172.21.15.1:0/1665925086 wait complete. 2024-04-26T01:42:08.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.739+0000 7f691fe00700 1 Processor -- start 2024-04-26T01:42:08.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.739+0000 7f691fe00700 1 -- start start 2024-04-26T01:42:08.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.739+0000 7f691fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69200754a0 0x7f692019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:08.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.739+0000 7f691fe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6920076a70 0x7f692019b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:08.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.739+0000 7f691fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692010a8b0 0x7f69201a2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:08.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.739+0000 7f691fe00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6920110670 con 0x7f69200754a0 2024-04-26T01:42:08.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.739+0000 7f691fe00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f69201104f0 con 0x7f692010a8b0 2024-04-26T01:42:08.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.739+0000 7f691fe00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f69201107f0 con 0x7f6920076a70 2024-04-26T01:42:08.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.740+0000 7f691ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69200754a0 0x7f692019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:08.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.740+0000 7f691ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69200754a0 0x7f692019acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39122/0 (socket says 172.21.15.1:39122) 2024-04-26T01:42:08.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.740+0000 7f691ea00700 1 -- 172.21.15.1:0/1774222010 learned_addr learned my addr 172.21.15.1:0/1774222010 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:42:08.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.740+0000 7f691f400700 1 --2- 172.21.15.1:0/1774222010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692010a8b0 0x7f69201a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:08.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.740+0000 7f6916000700 1 --2- 172.21.15.1:0/1774222010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6920076a70 0x7f692019b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:08.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.740+0000 7f691f400700 1 -- 172.21.15.1:0/1774222010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6920076a70 msgr2=0x7f692019b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:08.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.740+0000 7f691f400700 1 --2- 172.21.15.1:0/1774222010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6920076a70 0x7f692019b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:08.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.740+0000 7f691f400700 1 -- 172.21.15.1:0/1774222010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69200754a0 msgr2=0x7f692019acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:08.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.740+0000 7f691f400700 1 --2- 172.21.15.1:0/1774222010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69200754a0 0x7f692019acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:08.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.740+0000 7f691f400700 1 -- 172.21.15.1:0/1774222010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69201a28f0 con 0x7f692010a8b0 2024-04-26T01:42:08.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.740+0000 7f691f400700 1 --2- 172.21.15.1:0/1774222010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692010a8b0 0x7f69201a2220 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6918006130 tx=0x7f6918004ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:08.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.741+0000 7f6917e00700 1 -- 172.21.15.1:0/1774222010 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6918021070 con 0x7f692010a8b0 2024-04-26T01:42:08.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.741+0000 7f6917e00700 1 -- 172.21.15.1:0/1774222010 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6918007b80 con 0x7f692010a8b0 2024-04-26T01:42:08.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.741+0000 7f6917e00700 1 -- 172.21.15.1:0/1774222010 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6918005040 con 0x7f692010a8b0 2024-04-26T01:42:08.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.741+0000 7f691fe00700 1 -- 172.21.15.1:0/1774222010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69201a2b80 con 0x7f692010a8b0 2024-04-26T01:42:08.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.741+0000 7f691fe00700 1 -- 172.21.15.1:0/1774222010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69201a3110 con 0x7f692010a8b0 2024-04-26T01:42:08.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.742+0000 7f691fe00700 1 -- 172.21.15.1:0/1774222010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6920069730 con 0x7f692010a8b0 2024-04-26T01:42:08.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.744+0000 7f6917e00700 1 -- 172.21.15.1:0/1774222010 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) ==== 138589+0+0 (secure 0 0 0) 0x7f691801d450 con 0x7f692010a8b0 2024-04-26T01:42:08.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.746+0000 7f6917e00700 1 --2- 172.21.15.1:0/1774222010 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f69000a3c30 0x7f69000a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:08.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.746+0000 7f6917e00700 1 -- 172.21.15.1:0/1774222010 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f691800f020 con 0x7f692010a8b0 2024-04-26T01:42:08.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.746+0000 7f691ea00700 1 --2- 172.21.15.1:0/1774222010 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f69000a3c30 0x7f69000a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:08.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.747+0000 7f691ea00700 1 --2- 172.21.15.1:0/1774222010 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f69000a3c30 0x7f69000a60f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6920100aa0 tx=0x7f6910009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:08.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.748+0000 7f6917e00700 1 -- 172.21.15.1:0/1774222010 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f691802a030 con 0x7f692010a8b0 2024-04-26T01:42:08.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:08.990+0000 7f691fe00700 1 -- 172.21.15.1:0/1774222010 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6920062f40 con 0x7f69000a3c30 2024-04-26T01:42:09.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:09.281+0000 7f6917e00700 1 -- 172.21.15.1:0/1774222010 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f6918099f90 con 0x7f692010a8b0 2024-04-26T01:42:09.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:09 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:42:09.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:09 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:42:09.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:09 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:09.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:09 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:42:09.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:09 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:42:09.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:09 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:09.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:09 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:42:09.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:09 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:42:09.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:09 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:10.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:10 smithi173 ceph-mon[32553]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:10.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:10 smithi173 ceph-mon[32553]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:42:10.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:10 smithi173 ceph-mon[32553]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T01:42:10.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:10 smithi001 ceph-mon[27822]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:10.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:10 smithi001 ceph-mon[27822]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:42:10.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:10 smithi001 ceph-mon[27822]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T01:42:10.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:10 smithi097 ceph-mon[32853]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:10.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:10 smithi097 ceph-mon[32853]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:42:10.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:10 smithi097 ceph-mon[32853]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T01:42:12.549 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:12 smithi097 ceph-mon[32853]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:12.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:12 smithi173 ceph-mon[32553]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:12.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:12 smithi001 ceph-mon[27822]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:13.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:13 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1129052195' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8ccc4a9-0f31-446a-8bcb-4691532397c2"}]: dispatch 2024-04-26T01:42:13.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:13 smithi173 ceph-mon[32553]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8ccc4a9-0f31-446a-8bcb-4691532397c2"}]: dispatch 2024-04-26T01:42:13.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:13 smithi173 ceph-mon[32553]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8ccc4a9-0f31-446a-8bcb-4691532397c2"}]': finished 2024-04-26T01:42:13.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:13 smithi173 ceph-mon[32553]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T01:42:13.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:13 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:42:13.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:13 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1129052195' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8ccc4a9-0f31-446a-8bcb-4691532397c2"}]: dispatch 2024-04-26T01:42:13.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:13 smithi001 ceph-mon[27822]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8ccc4a9-0f31-446a-8bcb-4691532397c2"}]: dispatch 2024-04-26T01:42:13.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:13 smithi001 ceph-mon[27822]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8ccc4a9-0f31-446a-8bcb-4691532397c2"}]': finished 2024-04-26T01:42:13.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:13 smithi001 ceph-mon[27822]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T01:42:13.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:13 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:42:13.720 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:13 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1129052195' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8ccc4a9-0f31-446a-8bcb-4691532397c2"}]: dispatch 2024-04-26T01:42:13.720 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:13 smithi097 ceph-mon[32853]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8ccc4a9-0f31-446a-8bcb-4691532397c2"}]: dispatch 2024-04-26T01:42:13.720 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:13 smithi097 ceph-mon[32853]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8ccc4a9-0f31-446a-8bcb-4691532397c2"}]': finished 2024-04-26T01:42:13.720 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:13 smithi097 ceph-mon[32853]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T01:42:13.720 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:13 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:42:14.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:14 smithi173 ceph-mon[32553]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:14.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:14 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/2086240608' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:42:14.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:14 smithi001 ceph-mon[27822]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:14.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:14 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2086240608' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:42:14.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:14 smithi097 ceph-mon[32853]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:14.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:14 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/2086240608' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:42:16.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:16 smithi173 ceph-mon[32553]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:16.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:16 smithi001 ceph-mon[27822]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:16.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:16 smithi097 ceph-mon[32853]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:18.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:18 smithi173 ceph-mon[32553]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:18.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:18 smithi001 ceph-mon[27822]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:18.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:18 smithi097 ceph-mon[32853]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:20.585 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:20 smithi001 ceph-mon[27822]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:20.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:20 smithi097 ceph-mon[32853]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:20.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:20 smithi173 ceph-mon[32553]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:22.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:22 smithi001 ceph-mon[27822]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:22.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:22 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T01:42:22.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:22 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:22.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:22 smithi097 ceph-mon[32853]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:22.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:22 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T01:42:22.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:22 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:22.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:22 smithi173 ceph-mon[32553]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:22.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:22 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T01:42:22.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:22 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:23.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:23 smithi097 ceph-mon[32853]: Deploying daemon osd.3 on smithi001 2024-04-26T01:42:23.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:23 smithi001 ceph-mon[27822]: Deploying daemon osd.3 on smithi001 2024-04-26T01:42:23.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:23 smithi173 ceph-mon[32553]: Deploying daemon osd.3 on smithi001 2024-04-26T01:42:24.601 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:24 smithi001 ceph-mon[27822]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:24.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:24 smithi097 ceph-mon[32853]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:24.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:24 smithi173 ceph-mon[32553]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:26.503 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:26 smithi001 ceph-mon[27822]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:26.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:26 smithi097 ceph-mon[32853]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:26.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:26 smithi173 ceph-mon[32553]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:28.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:28 smithi001 ceph-mon[27822]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:28.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:28 smithi097 ceph-mon[32853]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:28.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:28 smithi173 ceph-mon[32553]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:30.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:30 smithi001 ceph-mon[27822]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:30.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:30 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:30.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:30 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:30.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:30 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:30.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:30 smithi097 ceph-mon[32853]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:30.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:30 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:30.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:30 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:30.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:30 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:30.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:30 smithi173 ceph-mon[32553]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:30.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:30 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:30.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:30 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:30.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:30 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:32.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:32 smithi001 ceph-mon[27822]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:32.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:32 smithi097 ceph-mon[32853]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:32.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:32 smithi173 ceph-mon[32553]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:34.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:34.842+0000 7f6917e00700 1 -- 172.21.15.1:0/1774222010 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6920062f40 con 0x7f69000a3c30 2024-04-26T01:42:34.839 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 3 on host 'smithi001' 2024-04-26T01:42:34.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:34.846+0000 7f691fe00700 1 -- 172.21.15.1:0/1774222010 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f69000a3c30 msgr2=0x7f69000a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:34.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:34.846+0000 7f691fe00700 1 --2- 172.21.15.1:0/1774222010 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f69000a3c30 0x7f69000a60f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6920100aa0 tx=0x7f6910009260 comp rx=0 tx=0).stop 2024-04-26T01:42:34.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:34.846+0000 7f691fe00700 1 -- 172.21.15.1:0/1774222010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692010a8b0 msgr2=0x7f69201a2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:34.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:34.846+0000 7f691fe00700 1 --2- 172.21.15.1:0/1774222010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692010a8b0 0x7f69201a2220 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6918006130 tx=0x7f6918004ce0 comp rx=0 tx=0).stop 2024-04-26T01:42:34.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:34.846+0000 7f691fe00700 1 -- 172.21.15.1:0/1774222010 shutdown_connections 2024-04-26T01:42:34.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:34.846+0000 7f691fe00700 1 --2- 172.21.15.1:0/1774222010 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f69000a3c30 0x7f69000a60f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:34.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:34.846+0000 7f691fe00700 1 --2- 172.21.15.1:0/1774222010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f692010a8b0 0x7f69201a2220 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:34.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:34.846+0000 7f691fe00700 1 --2- 172.21.15.1:0/1774222010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69200754a0 0x7f692019acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:34.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:34.846+0000 7f691fe00700 1 --2- 172.21.15.1:0/1774222010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6920076a70 0x7f692019b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:34.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:34.846+0000 7f691fe00700 1 -- 172.21.15.1:0/1774222010 >> 172.21.15.1:0/1774222010 conn(0x7f69200fdf80 msgr2=0x7f69201067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:34.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:34.846+0000 7f691fe00700 1 -- 172.21.15.1:0/1774222010 shutdown_connections 2024-04-26T01:42:34.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:42:34.846+0000 7f691fe00700 1 -- 172.21.15.1:0/1774222010 wait complete. 2024-04-26T01:42:34.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:34 smithi001 ceph-mon[27822]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:34.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:34 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:34.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:34 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:34.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:34 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:34.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:34 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:34.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:34 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:34.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:34 smithi001 ceph-mon[27822]: from='osd.3 [v2:172.21.15.1:6810/3349961961,v1:172.21.15.1:6811/3349961961]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T01:42:34.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:34 smithi001 ceph-mon[27822]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T01:42:34.890 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:34 smithi173 ceph-mon[32553]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:34.890 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:34 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:34.890 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:34 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:34.890 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:34 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:34.890 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:34 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:34.890 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:34 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:34.890 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:34 smithi173 ceph-mon[32553]: from='osd.3 [v2:172.21.15.1:6810/3349961961,v1:172.21.15.1:6811/3349961961]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T01:42:34.890 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:34 smithi173 ceph-mon[32553]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T01:42:34.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:34 smithi097 ceph-mon[32853]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:34.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:34 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:34.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:34 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:34.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:34 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:34.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:34 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:34.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:34 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:34.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:34 smithi097 ceph-mon[32853]: from='osd.3 [v2:172.21.15.1:6810/3349961961,v1:172.21.15.1:6811/3349961961]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T01:42:34.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:34 smithi097 ceph-mon[32853]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T01:42:35.532 DEBUG:teuthology.orchestra.run.smithi001:osd.3> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.3.service 2024-04-26T01:42:35.536 INFO:tasks.cephadm:Deploying osd.4 on smithi097 with /dev/vg_nvme/lv_3... 2024-04-26T01:42:35.536 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T01:42:35.779 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:35 smithi001 ceph-mon[27822]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T01:42:35.780 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:35 smithi001 ceph-mon[27822]: from='osd.3 [v2:172.21.15.1:6810/3349961961,v1:172.21.15.1:6811/3349961961]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-04-26T01:42:35.780 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:35 smithi001 ceph-mon[27822]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T01:42:35.780 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:35 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:42:35.780 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:35 smithi001 ceph-mon[27822]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-04-26T01:42:35.780 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:35 smithi001 ceph-mon[27822]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:35.780 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:35 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:35.780 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:35 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:35.780 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:35 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:35.780 INFO:journalctl@ceph.osd.3.smithi001.stdout:Apr 26 01:42:35 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-3[47007]: 2024-04-26T01:42:35.636+0000 7fd217200700 -1 osd.3 0 waiting for initial osdmap 2024-04-26T01:42:35.780 INFO:journalctl@ceph.osd.3.smithi001.stdout:Apr 26 01:42:35 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-3[47007]: 2024-04-26T01:42:35.646+0000 7fd212000700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:42:35.781 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:42:35.903 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:35 smithi097 ceph-mon[32853]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T01:42:35.903 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:35 smithi097 ceph-mon[32853]: from='osd.3 [v2:172.21.15.1:6810/3349961961,v1:172.21.15.1:6811/3349961961]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-04-26T01:42:35.904 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:35 smithi097 ceph-mon[32853]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T01:42:35.904 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:35 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:42:35.904 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:35 smithi097 ceph-mon[32853]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-04-26T01:42:35.904 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:35 smithi097 ceph-mon[32853]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:35.904 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:35 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:35.904 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:35 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:35.904 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:35 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:36.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:35 smithi173 ceph-mon[32553]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T01:42:36.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:35 smithi173 ceph-mon[32553]: from='osd.3 [v2:172.21.15.1:6810/3349961961,v1:172.21.15.1:6811/3349961961]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-04-26T01:42:36.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:35 smithi173 ceph-mon[32553]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T01:42:36.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:35 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:42:36.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:35 smithi173 ceph-mon[32553]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-04-26T01:42:36.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:35 smithi173 ceph-mon[32553]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:36.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:35 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:36.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:35 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:36.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:35 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:36.790 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:36 smithi001 ceph-mon[27822]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-04-26T01:42:36.790 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:36 smithi001 ceph-mon[27822]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T01:42:36.790 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:36 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:42:36.790 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:36 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:42:36.950 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:36 smithi097 ceph-mon[32853]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-04-26T01:42:36.950 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:36 smithi097 ceph-mon[32853]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T01:42:36.950 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:36 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:42:36.950 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:36 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:42:37.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:36 smithi173 ceph-mon[32553]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-04-26T01:42:37.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:36 smithi173 ceph-mon[32553]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T01:42:37.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:36 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:42:37.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:36 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:42:37.723 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T01:42:37.723 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T01:42:37.723 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-26T01:42:37.724 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-26T01:42:37.724 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0171839 s, 610 MB/s 2024-04-26T01:42:37.724 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-26T01:42:37.757 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:37 smithi001 ceph-mon[27822]: purged_snaps scrub starts 2024-04-26T01:42:37.757 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:37 smithi001 ceph-mon[27822]: purged_snaps scrub ok 2024-04-26T01:42:37.757 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:37 smithi001 ceph-mon[27822]: osd.3 [v2:172.21.15.1:6810/3349961961,v1:172.21.15.1:6811/3349961961] boot 2024-04-26T01:42:37.757 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:37 smithi001 ceph-mon[27822]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T01:42:37.757 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:37 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:42:37.757 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:37 smithi001 ceph-mon[27822]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:37.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:37 smithi097 ceph-mon[32853]: purged_snaps scrub starts 2024-04-26T01:42:37.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:37 smithi097 ceph-mon[32853]: purged_snaps scrub ok 2024-04-26T01:42:37.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:37 smithi097 ceph-mon[32853]: osd.3 [v2:172.21.15.1:6810/3349961961,v1:172.21.15.1:6811/3349961961] boot 2024-04-26T01:42:37.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:37 smithi097 ceph-mon[32853]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T01:42:37.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:37 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:42:37.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:37 smithi097 ceph-mon[32853]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:38.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:37 smithi173 ceph-mon[32553]: purged_snaps scrub starts 2024-04-26T01:42:38.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:37 smithi173 ceph-mon[32553]: purged_snaps scrub ok 2024-04-26T01:42:38.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:37 smithi173 ceph-mon[32553]: osd.3 [v2:172.21.15.1:6810/3349961961,v1:172.21.15.1:6811/3349961961] boot 2024-04-26T01:42:38.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:37 smithi173 ceph-mon[32553]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T01:42:38.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:37 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:42:38.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:37 smithi173 ceph-mon[32553]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:42:38.322 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_3 2024-04-26T01:42:38.561 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:42:38.929 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:38 smithi097 ceph-mon[32853]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T01:42:39.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:38 smithi173 ceph-mon[32553]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T01:42:39.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:38 smithi001 ceph-mon[27822]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T01:42:39.881 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:39 smithi001 ceph-mon[27822]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T01:42:39.881 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:39 smithi001 ceph-mon[27822]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:42:39.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:39 smithi097 ceph-mon[32853]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T01:42:39.923 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:39 smithi097 ceph-mon[32853]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:42:40.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:39 smithi173 ceph-mon[32553]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T01:42:40.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:39 smithi173 ceph-mon[32553]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:42:40.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.165+0000 7fef88f5e700 1 -- 172.21.15.97:0/1279833196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef84102aa0 msgr2=0x7fef84102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:40.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.165+0000 7fef88f5e700 1 --2- 172.21.15.97:0/1279833196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef84102aa0 0x7fef84102ec0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fef74009920 tx=0x7fef74009c30 comp rx=0 tx=0).stop 2024-04-26T01:42:40.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.165+0000 7fef88f5e700 1 -- 172.21.15.97:0/1279833196 shutdown_connections 2024-04-26T01:42:40.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.165+0000 7fef88f5e700 1 --2- 172.21.15.97:0/1279833196 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef84104640 0x7fef8410aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:40.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.165+0000 7fef88f5e700 1 --2- 172.21.15.97:0/1279833196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef84102aa0 0x7fef84102ec0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:40.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.165+0000 7fef88f5e700 1 --2- 172.21.15.97:0/1279833196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef84103ca0 0x7fef84104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:40.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.165+0000 7fef88f5e700 1 -- 172.21.15.97:0/1279833196 >> 172.21.15.97:0/1279833196 conn(0x7fef840fe040 msgr2=0x7fef84100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:40.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.165+0000 7fef88f5e700 1 -- 172.21.15.97:0/1279833196 shutdown_connections 2024-04-26T01:42:40.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.165+0000 7fef88f5e700 1 -- 172.21.15.97:0/1279833196 wait complete. 2024-04-26T01:42:40.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.165+0000 7fef88f5e700 1 Processor -- start 2024-04-26T01:42:40.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.165+0000 7fef88f5e700 1 -- start start 2024-04-26T01:42:40.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.166+0000 7fef88f5e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef84102aa0 0x7fef840767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:40.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.166+0000 7fef88f5e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef84103ca0 0x7fef84074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:40.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.166+0000 7fef88f5e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef84104640 0x7fef84075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:40.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.166+0000 7fef88f5e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fef8410e5b0 con 0x7fef84103ca0 2024-04-26T01:42:40.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.166+0000 7fef88f5e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fef8410e430 con 0x7fef84104640 2024-04-26T01:42:40.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.166+0000 7fef88f5e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fef8410e730 con 0x7fef84102aa0 2024-04-26T01:42:40.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.166+0000 7fef82d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef84104640 0x7fef84075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:40.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.166+0000 7fef82d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef84104640 0x7fef84075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40530/0 (socket says 172.21.15.97:40530) 2024-04-26T01:42:40.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.166+0000 7fef82d9d700 1 -- 172.21.15.97:0/2475325267 learned_addr learned my addr 172.21.15.97:0/2475325267 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T01:42:40.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.166+0000 7fef8259c700 1 --2- 172.21.15.97:0/2475325267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef84102aa0 0x7fef840767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:40.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.166+0000 7fef82d9d700 1 -- 172.21.15.97:0/2475325267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef84102aa0 msgr2=0x7fef840767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:40.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.166+0000 7fef82d9d700 1 --2- 172.21.15.97:0/2475325267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef84102aa0 0x7fef840767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:40.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.166+0000 7fef82d9d700 1 -- 172.21.15.97:0/2475325267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef84103ca0 msgr2=0x7fef84074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:40.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.166+0000 7fef82d9d700 1 --2- 172.21.15.97:0/2475325267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef84103ca0 0x7fef84074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:40.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.166+0000 7fef82d9d700 1 -- 172.21.15.97:0/2475325267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef84075c20 con 0x7fef84104640 2024-04-26T01:42:40.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.166+0000 7fef82d9d700 1 --2- 172.21.15.97:0/2475325267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef84104640 0x7fef84075370 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00c110 tx=0x7fef7c00c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:40.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.166+0000 7fef737fe700 1 -- 172.21.15.97:0/2475325267 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c0092e0 con 0x7fef84104640 2024-04-26T01:42:40.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.166+0000 7fef737fe700 1 -- 172.21.15.97:0/2475325267 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef7c004bb0 con 0x7fef84104640 2024-04-26T01:42:40.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.166+0000 7fef88f5e700 1 -- 172.21.15.97:0/2475325267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef841a2e30 con 0x7fef84104640 2024-04-26T01:42:40.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.166+0000 7fef737fe700 1 -- 172.21.15.97:0/2475325267 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c007ce0 con 0x7fef84104640 2024-04-26T01:42:40.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.166+0000 7fef88f5e700 1 -- 172.21.15.97:0/2475325267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef841a3310 con 0x7fef84104640 2024-04-26T01:42:40.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.167+0000 7fef88f5e700 1 -- 172.21.15.97:0/2475325267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef84069730 con 0x7fef84104640 2024-04-26T01:42:40.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.168+0000 7fef737fe700 1 -- 172.21.15.97:0/2475325267 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fef7c01a020 con 0x7fef84104640 2024-04-26T01:42:40.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.169+0000 7fef737fe700 1 --2- 172.21.15.97:0/2475325267 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fef6c0a3c40 0x7fef6c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:42:40.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.169+0000 7fef737fe700 1 -- 172.21.15.97:0/2475325267 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7fef7c0ced10 con 0x7fef84104640 2024-04-26T01:42:40.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.170+0000 7fef8259c700 1 --2- 172.21.15.97:0/2475325267 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fef6c0a3c40 0x7fef6c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:42:40.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.170+0000 7fef8259c700 1 --2- 172.21.15.97:0/2475325267 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fef6c0a3c40 0x7fef6c0a6100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fef740098f0 tx=0x7fef74019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:42:40.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.174+0000 7fef737fe700 1 -- 172.21.15.97:0/2475325267 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef7c010040 con 0x7fef84104640 2024-04-26T01:42:40.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:40.379+0000 7fef88f5e700 1 -- 172.21.15.97:0/2475325267 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fef84062e80 con 0x7fef6c0a3c40 2024-04-26T01:42:40.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:40 smithi097 ceph-mon[32853]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T01:42:40.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:40 smithi097 ceph-mon[32853]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:42:40.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:40 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:42:40.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:40 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:42:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:40 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:40 smithi097 ceph-mon[32853]: Detected new or changed devices on smithi001 2024-04-26T01:42:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:40 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:40 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:40 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:42:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:40 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:42:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:40 smithi097 ceph-mon[32853]: Adjusting osd_memory_target on smithi001 to 8547M 2024-04-26T01:42:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:40 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:40 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:40 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:40 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:41.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:40 smithi173 ceph-mon[32553]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T01:42:41.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:40 smithi173 ceph-mon[32553]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:42:41.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:40 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:42:41.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:40 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:42:41.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:40 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:41.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:40 smithi173 ceph-mon[32553]: Detected new or changed devices on smithi001 2024-04-26T01:42:41.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:40 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:41.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:40 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:41.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:40 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:42:41.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:40 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:42:41.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:40 smithi173 ceph-mon[32553]: Adjusting osd_memory_target on smithi001 to 8547M 2024-04-26T01:42:41.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:40 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:41.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:40 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:41.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:40 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:41.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:40 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:41.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:40 smithi001 ceph-mon[27822]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T01:42:41.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:40 smithi001 ceph-mon[27822]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:42:41.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:40 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:42:41.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:40 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:42:41.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:40 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:41.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:40 smithi001 ceph-mon[27822]: Detected new or changed devices on smithi001 2024-04-26T01:42:41.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:40 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:41.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:40 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:41.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:40 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:42:41.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:40 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:42:41.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:40 smithi001 ceph-mon[27822]: Adjusting osd_memory_target on smithi001 to 8547M 2024-04-26T01:42:41.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:40 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:41.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:40 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:41.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:40 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:42:41.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:40 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:41.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:41 smithi097 ceph-mon[32853]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T01:42:41.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:41 smithi097 ceph-mon[32853]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:42:42.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:41 smithi173 ceph-mon[32553]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T01:42:42.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:41 smithi173 ceph-mon[32553]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:42:42.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:41 smithi001 ceph-mon[27822]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T01:42:42.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:41 smithi001 ceph-mon[27822]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:42:44.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:43 smithi173 ceph-mon[32553]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:42:44.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:43 smithi173 ceph-mon[32553]: from='client.? 172.21.15.97:0/3098882266' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b724648-4df7-40fc-89d5-55f170cdd806"}]: dispatch 2024-04-26T01:42:44.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:43 smithi173 ceph-mon[32553]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b724648-4df7-40fc-89d5-55f170cdd806"}]: dispatch 2024-04-26T01:42:44.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:43 smithi173 ceph-mon[32553]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b724648-4df7-40fc-89d5-55f170cdd806"}]': finished 2024-04-26T01:42:44.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:43 smithi173 ceph-mon[32553]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T01:42:44.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:43 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:42:44.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:43 smithi001 ceph-mon[27822]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:42:44.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:43 smithi001 ceph-mon[27822]: from='client.? 172.21.15.97:0/3098882266' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b724648-4df7-40fc-89d5-55f170cdd806"}]: dispatch 2024-04-26T01:42:44.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:43 smithi001 ceph-mon[27822]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b724648-4df7-40fc-89d5-55f170cdd806"}]: dispatch 2024-04-26T01:42:44.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:43 smithi001 ceph-mon[27822]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b724648-4df7-40fc-89d5-55f170cdd806"}]': finished 2024-04-26T01:42:44.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:43 smithi001 ceph-mon[27822]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T01:42:44.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:43 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:42:44.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:43 smithi097 ceph-mon[32853]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:42:44.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:43 smithi097 ceph-mon[32853]: from='client.? 172.21.15.97:0/3098882266' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b724648-4df7-40fc-89d5-55f170cdd806"}]: dispatch 2024-04-26T01:42:44.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:43 smithi097 ceph-mon[32853]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b724648-4df7-40fc-89d5-55f170cdd806"}]: dispatch 2024-04-26T01:42:44.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:43 smithi097 ceph-mon[32853]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b724648-4df7-40fc-89d5-55f170cdd806"}]': finished 2024-04-26T01:42:44.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:43 smithi097 ceph-mon[32853]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T01:42:44.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:43 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:42:45.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:44 smithi173 ceph-mon[32553]: from='client.? 172.21.15.97:0/3080710102' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:42:45.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:44 smithi001 ceph-mon[27822]: from='client.? 172.21.15.97:0/3080710102' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:42:45.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:44 smithi097 ceph-mon[32853]: from='client.? 172.21.15.97:0/3080710102' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:42:46.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:45 smithi173 ceph-mon[32553]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:42:46.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:45 smithi001 ceph-mon[27822]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:42:46.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:45 smithi097 ceph-mon[32853]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:42:47.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:47 smithi097 ceph-mon[32853]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-26T01:42:48.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:47 smithi173 ceph-mon[32553]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-26T01:42:48.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:47 smithi001 ceph-mon[27822]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-26T01:42:50.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:49 smithi173 ceph-mon[32553]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:42:50.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:49 smithi001 ceph-mon[27822]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:42:50.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:49 smithi097 ceph-mon[32853]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:42:50.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:50 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T01:42:50.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:50 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:51.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:50 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T01:42:51.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:50 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:51.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:50 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T01:42:51.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:50 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:42:51.970 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:51 smithi097 ceph-mon[32853]: Deploying daemon osd.4 on smithi097 2024-04-26T01:42:51.970 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:51 smithi097 ceph-mon[32853]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:42:52.065 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:51 smithi001 ceph-mon[27822]: Deploying daemon osd.4 on smithi097 2024-04-26T01:42:52.065 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:51 smithi001 ceph-mon[27822]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:42:52.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:51 smithi173 ceph-mon[32553]: Deploying daemon osd.4 on smithi097 2024-04-26T01:42:52.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:51 smithi173 ceph-mon[32553]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:42:53.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:53 smithi097 ceph-mon[32853]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:42:54.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:53 smithi173 ceph-mon[32553]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:42:54.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:53 smithi001 ceph-mon[27822]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:42:56.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:55 smithi173 ceph-mon[32553]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:42:56.091 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:55 smithi097 ceph-mon[32853]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:42:56.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:55 smithi001 ceph-mon[27822]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:42:57.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:57 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:57.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:57 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:57.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:57 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:57.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:57 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:57.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:57 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:57.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:57 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:57.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:57 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:57.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:57 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:42:57.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:57 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:42:58.606 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:42:58 smithi097 ceph-mon[32853]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:42:58.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:42:58 smithi001 ceph-mon[27822]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:42:58.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:42:58 smithi173 ceph-mon[32553]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:42:59.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:59.619+0000 7fef737fe700 1 -- 172.21.15.97:0/2475325267 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fef84062e80 con 0x7fef6c0a3c40 2024-04-26T01:42:59.617 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 4 on host 'smithi097' 2024-04-26T01:42:59.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:59.622+0000 7fef88f5e700 1 -- 172.21.15.97:0/2475325267 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fef6c0a3c40 msgr2=0x7fef6c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:59.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:59.622+0000 7fef88f5e700 1 --2- 172.21.15.97:0/2475325267 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fef6c0a3c40 0x7fef6c0a6100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fef740098f0 tx=0x7fef74019040 comp rx=0 tx=0).stop 2024-04-26T01:42:59.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:59.622+0000 7fef88f5e700 1 -- 172.21.15.97:0/2475325267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef84104640 msgr2=0x7fef84075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:42:59.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:59.622+0000 7fef88f5e700 1 --2- 172.21.15.97:0/2475325267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef84104640 0x7fef84075370 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00c110 tx=0x7fef7c00c420 comp rx=0 tx=0).stop 2024-04-26T01:42:59.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:59.622+0000 7fef88f5e700 1 -- 172.21.15.97:0/2475325267 shutdown_connections 2024-04-26T01:42:59.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:59.622+0000 7fef88f5e700 1 --2- 172.21.15.97:0/2475325267 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fef6c0a3c40 0x7fef6c0a6100 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:59.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:59.622+0000 7fef88f5e700 1 --2- 172.21.15.97:0/2475325267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef84104640 0x7fef84075370 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:59.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:59.622+0000 7fef88f5e700 1 --2- 172.21.15.97:0/2475325267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef84102aa0 0x7fef840767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:59.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:59.622+0000 7fef88f5e700 1 --2- 172.21.15.97:0/2475325267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef84103ca0 0x7fef84074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:42:59.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:59.622+0000 7fef88f5e700 1 -- 172.21.15.97:0/2475325267 >> 172.21.15.97:0/2475325267 conn(0x7fef840fe040 msgr2=0x7fef841070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:42:59.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:59.622+0000 7fef88f5e700 1 -- 172.21.15.97:0/2475325267 shutdown_connections 2024-04-26T01:42:59.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:42:59.622+0000 7fef88f5e700 1 -- 172.21.15.97:0/2475325267 wait complete. 2024-04-26T01:43:00.251 DEBUG:teuthology.orchestra.run.smithi097:osd.4> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.4.service 2024-04-26T01:43:00.253 INFO:tasks.cephadm:Deploying osd.5 on smithi173 with /dev/vg_nvme/lv_3... 2024-04-26T01:43:00.253 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T01:43:00.497 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:43:00.812 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:00 smithi173 ceph-mon[32553]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:43:00.813 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:00 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:00.813 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:00 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:00.813 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:00 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:00.813 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:00 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:00.813 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:00 smithi173 ceph-mon[32553]: from='osd.4 [v2:172.21.15.97:6808/2494639397,v1:172.21.15.97:6809/2494639397]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T01:43:00.813 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:00 smithi173 ceph-mon[32553]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T01:43:00.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:00 smithi001 ceph-mon[27822]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:43:00.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:00 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:00.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:00 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:00.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:00 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:00.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:00 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:00.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:00 smithi001 ceph-mon[27822]: from='osd.4 [v2:172.21.15.97:6808/2494639397,v1:172.21.15.97:6809/2494639397]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T01:43:00.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:00 smithi001 ceph-mon[27822]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T01:43:00.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:00 smithi097 ceph-mon[32853]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:43:00.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:00 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:00.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:00 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:00.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:00 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:00.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:00 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:00.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:00 smithi097 ceph-mon[32853]: from='osd.4 [v2:172.21.15.97:6808/2494639397,v1:172.21.15.97:6809/2494639397]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T01:43:00.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:00 smithi097 ceph-mon[32853]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T01:43:01.770 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:01 smithi173 ceph-mon[32553]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T01:43:01.770 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:01 smithi173 ceph-mon[32553]: from='osd.4 [v2:172.21.15.97:6808/2494639397,v1:172.21.15.97:6809/2494639397]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:43:01.770 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:01 smithi173 ceph-mon[32553]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T01:43:01.770 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:01 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:43:01.770 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:01 smithi173 ceph-mon[32553]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:43:01.770 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:01 smithi173 ceph-mon[32553]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:43:01.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:01 smithi097 ceph-mon[32853]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T01:43:01.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:01 smithi097 ceph-mon[32853]: from='osd.4 [v2:172.21.15.97:6808/2494639397,v1:172.21.15.97:6809/2494639397]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:43:01.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:01 smithi097 ceph-mon[32853]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T01:43:01.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:01 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:43:01.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:01 smithi097 ceph-mon[32853]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:43:01.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:01 smithi097 ceph-mon[32853]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:43:01.983 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 26 01:43:01 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-4[43984]: 2024-04-26T01:43:01.710+0000 7fe934e33700 -1 osd.4 0 waiting for initial osdmap 2024-04-26T01:43:01.983 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 26 01:43:01 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-4[43984]: 2024-04-26T01:43:01.716+0000 7fe930c26700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:43:02.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:01 smithi001 ceph-mon[27822]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T01:43:02.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:01 smithi001 ceph-mon[27822]: from='osd.4 [v2:172.21.15.97:6808/2494639397,v1:172.21.15.97:6809/2494639397]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:43:02.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:01 smithi001 ceph-mon[27822]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T01:43:02.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:01 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:43:02.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:01 smithi001 ceph-mon[27822]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:43:02.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:01 smithi001 ceph-mon[27822]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:43:02.358 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T01:43:02.358 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T01:43:02.358 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-04-26T01:43:02.358 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-04-26T01:43:02.358 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114709 s, 914 MB/s 2024-04-26T01:43:02.358 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-04-26T01:43:02.931 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:02 smithi173 ceph-mon[32553]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T01:43:02.931 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:02 smithi173 ceph-mon[32553]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T01:43:02.931 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:02 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:43:02.931 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:02 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:43:02.957 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_3 2024-04-26T01:43:02.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:02 smithi097 ceph-mon[32853]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T01:43:02.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:02 smithi097 ceph-mon[32853]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T01:43:02.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:02 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:43:02.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:02 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:43:03.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:02 smithi001 ceph-mon[27822]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T01:43:03.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:02 smithi001 ceph-mon[27822]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T01:43:03.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:02 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:43:03.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:02 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:43:03.195 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:43:03.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:03 smithi173 ceph-mon[32553]: purged_snaps scrub starts 2024-04-26T01:43:03.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:03 smithi173 ceph-mon[32553]: purged_snaps scrub ok 2024-04-26T01:43:03.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:03 smithi173 ceph-mon[32553]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:43:03.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:03 smithi173 ceph-mon[32553]: osd.4 [v2:172.21.15.97:6808/2494639397,v1:172.21.15.97:6809/2494639397] boot 2024-04-26T01:43:03.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:03 smithi173 ceph-mon[32553]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T01:43:03.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:43:03.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:03 smithi173 ceph-mon[32553]: Detected new or changed devices on smithi097 2024-04-26T01:43:03.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:03.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:03.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:43:03.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:43:03.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:03 smithi173 ceph-mon[32553]: Adjusting osd_memory_target on smithi097 to 8544M 2024-04-26T01:43:03.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:03.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:03.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:03.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:03.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:03.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:03.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:03.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:03.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:03 smithi097 ceph-mon[32853]: purged_snaps scrub starts 2024-04-26T01:43:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:03 smithi097 ceph-mon[32853]: purged_snaps scrub ok 2024-04-26T01:43:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:03 smithi097 ceph-mon[32853]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:43:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:03 smithi097 ceph-mon[32853]: osd.4 [v2:172.21.15.97:6808/2494639397,v1:172.21.15.97:6809/2494639397] boot 2024-04-26T01:43:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:03 smithi097 ceph-mon[32853]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T01:43:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:43:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:03 smithi097 ceph-mon[32853]: Detected new or changed devices on smithi097 2024-04-26T01:43:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:43:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:43:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:03 smithi097 ceph-mon[32853]: Adjusting osd_memory_target on smithi097 to 8544M 2024-04-26T01:43:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:03.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:03.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:03.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:03.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:03.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:03.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:03 smithi001 ceph-mon[27822]: purged_snaps scrub starts 2024-04-26T01:43:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:03 smithi001 ceph-mon[27822]: purged_snaps scrub ok 2024-04-26T01:43:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:03 smithi001 ceph-mon[27822]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:43:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:03 smithi001 ceph-mon[27822]: osd.4 [v2:172.21.15.97:6808/2494639397,v1:172.21.15.97:6809/2494639397] boot 2024-04-26T01:43:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:03 smithi001 ceph-mon[27822]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T01:43:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:43:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:03 smithi001 ceph-mon[27822]: Detected new or changed devices on smithi097 2024-04-26T01:43:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:43:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:43:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:03 smithi001 ceph-mon[27822]: Adjusting osd_memory_target on smithi097 to 8544M 2024-04-26T01:43:04.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:04.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:04.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:04.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:04.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:04.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:04.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:04.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:04.710 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.713+0000 7f7431474700 1 -- 172.21.15.173:0/866234581 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f742c105e50 msgr2=0x7f742c108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:43:04.711 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.713+0000 7f7431474700 1 --2- 172.21.15.173:0/866234581 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f742c105e50 0x7f742c108240 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7414009990 tx=0x7f7414009ca0 comp rx=0 tx=0).stop 2024-04-26T01:43:04.711 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.713+0000 7f7431474700 1 -- 172.21.15.173:0/866234581 shutdown_connections 2024-04-26T01:43:04.711 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.713+0000 7f7431474700 1 --2- 172.21.15.173:0/866234581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f742c108780 0x7f742c10abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:04.711 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.713+0000 7f7431474700 1 --2- 172.21.15.173:0/866234581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742c103520 0x7f742c105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:04.711 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.713+0000 7f7431474700 1 --2- 172.21.15.173:0/866234581 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f742c105e50 0x7f742c108240 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:04.711 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.713+0000 7f7431474700 1 -- 172.21.15.173:0/866234581 >> 172.21.15.173:0/866234581 conn(0x7f742c0fcf30 msgr2=0x7f742c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:43:04.711 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.713+0000 7f7431474700 1 -- 172.21.15.173:0/866234581 shutdown_connections 2024-04-26T01:43:04.711 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.713+0000 7f7431474700 1 -- 172.21.15.173:0/866234581 wait complete. 2024-04-26T01:43:04.713 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.714+0000 7f7431474700 1 Processor -- start 2024-04-26T01:43:04.713 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.714+0000 7f7431474700 1 -- start start 2024-04-26T01:43:04.713 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.714+0000 7f7431474700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f742c103520 0x7f742c198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:43:04.713 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.714+0000 7f7431474700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f742c105e50 0x7f742c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:43:04.713 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.714+0000 7f7431474700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742c108780 0x7f742c1a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:43:04.713 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.714+0000 7f7431474700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f742c10e610 con 0x7f742c103520 2024-04-26T01:43:04.714 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.714+0000 7f7431474700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f742c10e490 con 0x7f742c108780 2024-04-26T01:43:04.714 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.714+0000 7f7431474700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f742c10e790 con 0x7f742c105e50 2024-04-26T01:43:04.714 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.714+0000 7f74227fc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f742c105e50 0x7f742c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:43:04.714 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.714+0000 7f74227fc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f742c105e50 0x7f742c1992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:36742/0 (socket says 172.21.15.173:36742) 2024-04-26T01:43:04.714 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.714+0000 7f74227fc700 1 -- 172.21.15.173:0/202958901 learned_addr learned my addr 172.21.15.173:0/202958901 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-26T01:43:04.714 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.714+0000 7f742b7fe700 1 --2- 172.21.15.173:0/202958901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742c108780 0x7f742c1a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:43:04.714 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.714+0000 7f74227fc700 1 -- 172.21.15.173:0/202958901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742c108780 msgr2=0x7f742c1a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:43:04.714 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.714+0000 7f74227fc700 1 --2- 172.21.15.173:0/202958901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742c108780 0x7f742c1a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:04.714 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.714+0000 7f74227fc700 1 -- 172.21.15.173:0/202958901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f742c103520 msgr2=0x7f742c198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:43:04.714 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.714+0000 7f74227fc700 1 --2- 172.21.15.173:0/202958901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f742c103520 0x7f742c198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:04.714 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.714+0000 7f74227fc700 1 -- 172.21.15.173:0/202958901 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742c1a0990 con 0x7f742c105e50 2024-04-26T01:43:04.714 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.715+0000 7f74227fc700 1 --2- 172.21.15.173:0/202958901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f742c105e50 0x7f742c1992c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f74140038a0 tx=0x7f74140047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:43:04.714 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.715+0000 7f7428ff9700 1 -- 172.21.15.173:0/202958901 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414027070 con 0x7f742c105e50 2024-04-26T01:43:04.714 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.715+0000 7f7428ff9700 1 -- 172.21.15.173:0/202958901 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7414022470 con 0x7f742c105e50 2024-04-26T01:43:04.715 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.715+0000 7f7431474700 1 -- 172.21.15.173:0/202958901 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f742c1a0c20 con 0x7f742c105e50 2024-04-26T01:43:04.715 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.715+0000 7f7428ff9700 1 -- 172.21.15.173:0/202958901 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741400f6c0 con 0x7f742c105e50 2024-04-26T01:43:04.715 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.715+0000 7f7431474700 1 -- 172.21.15.173:0/202958901 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f742c1a1070 con 0x7f742c105e50 2024-04-26T01:43:04.715 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.716+0000 7f7431474700 1 -- 172.21.15.173:0/202958901 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f742c069730 con 0x7f742c105e50 2024-04-26T01:43:04.716 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.717+0000 7f7428ff9700 1 -- 172.21.15.173:0/202958901 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f741402a070 con 0x7f742c105e50 2024-04-26T01:43:04.716 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.717+0000 7f7428ff9700 1 --2- 172.21.15.173:0/202958901 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f74180a3c40 0x7f74180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:43:04.716 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.718+0000 7f7428ff9700 1 -- 172.21.15.173:0/202958901 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f74140d0200 con 0x7f742c105e50 2024-04-26T01:43:04.716 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.718+0000 7f742affd700 1 --2- 172.21.15.173:0/202958901 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f74180a3c40 0x7f74180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:43:04.716 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.719+0000 7f742affd700 1 --2- 172.21.15.173:0/202958901 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f74180a3c40 0x7f74180a6100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f741c0097b0 tx=0x7f741c006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:43:04.720 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.722+0000 7f7428ff9700 1 -- 172.21.15.173:0/202958901 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7414013070 con 0x7f742c105e50 2024-04-26T01:43:04.925 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:04.927+0000 7f7431474700 1 -- 172.21.15.173:0/202958901 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f742c062e80 con 0x7f74180a3c40 2024-04-26T01:43:05.315 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:05 smithi173 ceph-mon[32553]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T01:43:05.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:05 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:43:05.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:05 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:43:05.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:05 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:05.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:05 smithi097 ceph-mon[32853]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T01:43:05.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:05 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:43:05.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:05 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:43:05.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:05 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:05.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:05 smithi001 ceph-mon[27822]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T01:43:05.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:05 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:43:05.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:05 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:43:05.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:05 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:06.354 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:06 smithi173 ceph-mon[32553]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:06.355 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:06 smithi173 ceph-mon[32553]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:43:06.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:06 smithi097 ceph-mon[32853]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:06.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:06 smithi097 ceph-mon[32853]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:43:06.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:06 smithi001 ceph-mon[27822]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:06.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:06 smithi001 ceph-mon[27822]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:43:08.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:08 smithi097 ceph-mon[32853]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:08 smithi097 ceph-mon[32853]: from='client.? 172.21.15.173:0/3894874063' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5a24a9c-3007-4b87-8562-40c98c41ef8b"}]: dispatch 2024-04-26T01:43:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:08 smithi097 ceph-mon[32853]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5a24a9c-3007-4b87-8562-40c98c41ef8b"}]: dispatch 2024-04-26T01:43:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:08 smithi097 ceph-mon[32853]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5a24a9c-3007-4b87-8562-40c98c41ef8b"}]': finished 2024-04-26T01:43:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:08 smithi097 ceph-mon[32853]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T01:43:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:08 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:43:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:08 smithi097 ceph-mon[32853]: from='client.? 172.21.15.173:0/2496508979' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:43:08.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:08 smithi173 ceph-mon[32553]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:08.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:08 smithi173 ceph-mon[32553]: from='client.? 172.21.15.173:0/3894874063' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5a24a9c-3007-4b87-8562-40c98c41ef8b"}]: dispatch 2024-04-26T01:43:08.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:08 smithi173 ceph-mon[32553]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5a24a9c-3007-4b87-8562-40c98c41ef8b"}]: dispatch 2024-04-26T01:43:08.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:08 smithi173 ceph-mon[32553]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5a24a9c-3007-4b87-8562-40c98c41ef8b"}]': finished 2024-04-26T01:43:08.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:08 smithi173 ceph-mon[32553]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T01:43:08.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:08 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:43:08.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:08 smithi173 ceph-mon[32553]: from='client.? 172.21.15.173:0/2496508979' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:43:08.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:08 smithi001 ceph-mon[27822]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:08.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:08 smithi001 ceph-mon[27822]: from='client.? 172.21.15.173:0/3894874063' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5a24a9c-3007-4b87-8562-40c98c41ef8b"}]: dispatch 2024-04-26T01:43:08.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:08 smithi001 ceph-mon[27822]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5a24a9c-3007-4b87-8562-40c98c41ef8b"}]: dispatch 2024-04-26T01:43:08.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:08 smithi001 ceph-mon[27822]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5a24a9c-3007-4b87-8562-40c98c41ef8b"}]': finished 2024-04-26T01:43:08.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:08 smithi001 ceph-mon[27822]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T01:43:08.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:08 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:43:08.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:08 smithi001 ceph-mon[27822]: from='client.? 172.21.15.173:0/2496508979' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:43:10.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:10 smithi097 ceph-mon[32853]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:10.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:10 smithi173 ceph-mon[32553]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:10.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:10 smithi001 ceph-mon[27822]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:12.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:12 smithi097 ceph-mon[32853]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:12.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:12 smithi173 ceph-mon[32553]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:12.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:12 smithi001 ceph-mon[27822]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:14.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:14 smithi097 ceph-mon[32853]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:14.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:14 smithi173 ceph-mon[32553]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:14.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:14 smithi001 ceph-mon[27822]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:15.315 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:15 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T01:43:15.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:15 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:15.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:15 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T01:43:15.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:15 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:15.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:15 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T01:43:15.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:15 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:16.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:16 smithi097 ceph-mon[32853]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:16.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:16 smithi097 ceph-mon[32853]: Deploying daemon osd.5 on smithi173 2024-04-26T01:43:16.511 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:16 smithi173 ceph-mon[32553]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:16.511 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:16 smithi173 ceph-mon[32553]: Deploying daemon osd.5 on smithi173 2024-04-26T01:43:16.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:16 smithi001 ceph-mon[27822]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:16.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:16 smithi001 ceph-mon[27822]: Deploying daemon osd.5 on smithi173 2024-04-26T01:43:18.439 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:18 smithi173 ceph-mon[32553]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:18.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:18 smithi097 ceph-mon[32853]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:18.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:18 smithi001 ceph-mon[27822]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:20.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:20 smithi097 ceph-mon[32853]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:20.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:20 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:20.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:20 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:20.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:20 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:20.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:20 smithi173 ceph-mon[32553]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:20.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:20 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:20.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:20 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:20.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:20 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:20.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:20 smithi001 ceph-mon[27822]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:20.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:20.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:20.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:22.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:22 smithi173 ceph-mon[32553]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:22.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:22 smithi001 ceph-mon[27822]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:22.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:22 smithi097 ceph-mon[32853]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:23.560 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:23.562+0000 7f7428ff9700 1 -- 172.21.15.173:0/202958901 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f742c062e80 con 0x7f74180a3c40 2024-04-26T01:43:23.560 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 5 on host 'smithi173' 2024-04-26T01:43:23.562 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:23.565+0000 7f7431474700 1 -- 172.21.15.173:0/202958901 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f74180a3c40 msgr2=0x7f74180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:43:23.562 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:23.565+0000 7f7431474700 1 --2- 172.21.15.173:0/202958901 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f74180a3c40 0x7f74180a6100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f741c0097b0 tx=0x7f741c006d20 comp rx=0 tx=0).stop 2024-04-26T01:43:23.562 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:23.565+0000 7f7431474700 1 -- 172.21.15.173:0/202958901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f742c105e50 msgr2=0x7f742c1992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:43:23.562 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:23.565+0000 7f7431474700 1 --2- 172.21.15.173:0/202958901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f742c105e50 0x7f742c1992c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f74140038a0 tx=0x7f74140047b0 comp rx=0 tx=0).stop 2024-04-26T01:43:23.562 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:23.565+0000 7f7431474700 1 -- 172.21.15.173:0/202958901 shutdown_connections 2024-04-26T01:43:23.562 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:23.565+0000 7f7431474700 1 --2- 172.21.15.173:0/202958901 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f74180a3c40 0x7f74180a6100 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:23.562 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:23.565+0000 7f7431474700 1 --2- 172.21.15.173:0/202958901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742c108780 0x7f742c1a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:23.562 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:23.565+0000 7f7431474700 1 --2- 172.21.15.173:0/202958901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f742c103520 0x7f742c198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:23.562 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:23.565+0000 7f7431474700 1 --2- 172.21.15.173:0/202958901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f742c105e50 0x7f742c1992c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:23.562 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:23.565+0000 7f7431474700 1 -- 172.21.15.173:0/202958901 >> 172.21.15.173:0/202958901 conn(0x7f742c0fcf30 msgr2=0x7f742c103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:43:23.563 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:23.565+0000 7f7431474700 1 -- 172.21.15.173:0/202958901 shutdown_connections 2024-04-26T01:43:23.563 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:43:23.565+0000 7f7431474700 1 -- 172.21.15.173:0/202958901 wait complete. 2024-04-26T01:43:23.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:23 smithi097 ceph-mon[32853]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:23.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:23 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:23.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:23 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:23.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:23 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:23.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:23 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:23.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:23 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:23.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:23 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:23.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:23 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:23.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:23 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:24.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:23 smithi173 ceph-mon[32553]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:24.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:23 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:24.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:23 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:24.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:23 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:24.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:23 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:24.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:23 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:24.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:23 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:24.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:23 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:24.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:23 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:24.078 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:23 smithi001 ceph-mon[27822]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:24.078 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:23 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:24.078 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:23 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:24.078 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:23 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:24.078 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:23 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:24.078 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:23 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:24.079 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:23 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:24.079 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:23 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:24.079 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:23 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:24.208 DEBUG:teuthology.orchestra.run.smithi173:osd.5> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.5.service 2024-04-26T01:43:24.210 INFO:tasks.cephadm:Deploying osd.6 on smithi001 with /dev/vg_nvme/lv_2... 2024-04-26T01:43:24.210 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T01:43:24.663 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:43:25.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:24 smithi173 ceph-mon[32553]: from='osd.5 [v2:172.21.15.173:6808/1026507957,v1:172.21.15.173:6809/1026507957]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T01:43:25.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:24 smithi173 ceph-mon[32553]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T01:43:25.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:24 smithi001 ceph-mon[27822]: from='osd.5 [v2:172.21.15.173:6808/1026507957,v1:172.21.15.173:6809/1026507957]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T01:43:25.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:24 smithi001 ceph-mon[27822]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T01:43:25.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:24 smithi097 ceph-mon[32853]: from='osd.5 [v2:172.21.15.173:6808/1026507957,v1:172.21.15.173:6809/1026507957]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T01:43:25.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:24 smithi097 ceph-mon[32853]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T01:43:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:25 smithi001 ceph-mon[27822]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:25 smithi001 ceph-mon[27822]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T01:43:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:25 smithi001 ceph-mon[27822]: from='osd.5 [v2:172.21.15.173:6808/1026507957,v1:172.21.15.173:6809/1026507957]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:43:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:25 smithi001 ceph-mon[27822]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T01:43:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:25 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:43:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:25 smithi001 ceph-mon[27822]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:43:26.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:25 smithi097 ceph-mon[32853]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:26.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:25 smithi097 ceph-mon[32853]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T01:43:26.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:25 smithi097 ceph-mon[32853]: from='osd.5 [v2:172.21.15.173:6808/1026507957,v1:172.21.15.173:6809/1026507957]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:43:26.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:25 smithi097 ceph-mon[32853]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T01:43:26.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:25 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:43:26.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:25 smithi097 ceph-mon[32853]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:43:26.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:25 smithi173 ceph-mon[32553]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:26.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:25 smithi173 ceph-mon[32553]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T01:43:26.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:25 smithi173 ceph-mon[32553]: from='osd.5 [v2:172.21.15.173:6808/1026507957,v1:172.21.15.173:6809/1026507957]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:43:26.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:25 smithi173 ceph-mon[32553]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T01:43:26.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:25 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:43:26.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:25 smithi173 ceph-mon[32553]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:43:27.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:26 smithi173 ceph-mon[32553]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-26T01:43:27.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:26 smithi173 ceph-mon[32553]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T01:43:27.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:26 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:43:27.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:26 smithi173 ceph-mon[32553]: from='osd.5 ' entity='osd.5' 2024-04-26T01:43:27.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:26 smithi173 ceph-mon[32553]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T01:43:27.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:26 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:43:27.066 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 01:43:26 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T01:43:26.737+0000 7ff3c5902700 -1 osd.5 0 waiting for initial osdmap 2024-04-26T01:43:27.066 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 01:43:26 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T01:43:26.896+0000 7ff3bd6ed700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:43:27.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:26 smithi001 ceph-mon[27822]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-26T01:43:27.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:26 smithi001 ceph-mon[27822]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T01:43:27.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:26 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:43:27.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:26 smithi001 ceph-mon[27822]: from='osd.5 ' entity='osd.5' 2024-04-26T01:43:27.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:26 smithi001 ceph-mon[27822]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T01:43:27.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:26 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:43:27.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:26 smithi097 ceph-mon[32853]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-26T01:43:27.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:26 smithi097 ceph-mon[32853]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T01:43:27.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:26 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:43:27.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:26 smithi097 ceph-mon[32853]: from='osd.5 ' entity='osd.5' 2024-04-26T01:43:27.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:26 smithi097 ceph-mon[32853]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T01:43:27.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:26 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:43:27.492 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T01:43:27.492 INFO:teuthology.orchestra.run.smithi001.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T01:43:27.492 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10+0 records in 2024-04-26T01:43:27.492 INFO:teuthology.orchestra.run.smithi001.stderr:10+0 records out 2024-04-26T01:43:27.493 INFO:teuthology.orchestra.run.smithi001.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0376705 s, 278 MB/s 2024-04-26T01:43:27.493 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping successful for: 2024-04-26T01:43:28.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:27 smithi001 ceph-mon[27822]: purged_snaps scrub starts 2024-04-26T01:43:28.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:27 smithi001 ceph-mon[27822]: purged_snaps scrub ok 2024-04-26T01:43:28.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:27 smithi001 ceph-mon[27822]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:28.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:27 smithi001 ceph-mon[27822]: Detected new or changed devices on smithi173 2024-04-26T01:43:28.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:27 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:28.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:27 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:28.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:27 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:43:28.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:27 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:43:28.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:27 smithi001 ceph-mon[27822]: Adjusting osd_memory_target on smithi173 to 8544M 2024-04-26T01:43:28.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:27 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:28.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:27 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:28.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:27 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:28.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:27 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:28.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:27 smithi001 ceph-mon[27822]: osd.5 [v2:172.21.15.173:6808/1026507957,v1:172.21.15.173:6809/1026507957] boot 2024-04-26T01:43:28.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:27 smithi001 ceph-mon[27822]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T01:43:28.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:27 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:43:28.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:27 smithi097 ceph-mon[32853]: purged_snaps scrub starts 2024-04-26T01:43:28.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:27 smithi097 ceph-mon[32853]: purged_snaps scrub ok 2024-04-26T01:43:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:27 smithi097 ceph-mon[32853]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:27 smithi097 ceph-mon[32853]: Detected new or changed devices on smithi173 2024-04-26T01:43:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:27 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:27 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:27 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:43:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:27 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:43:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:27 smithi097 ceph-mon[32853]: Adjusting osd_memory_target on smithi173 to 8544M 2024-04-26T01:43:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:27 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:27 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:27 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:27 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:27 smithi097 ceph-mon[32853]: osd.5 [v2:172.21.15.173:6808/1026507957,v1:172.21.15.173:6809/1026507957] boot 2024-04-26T01:43:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:27 smithi097 ceph-mon[32853]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T01:43:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:27 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:43:28.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:27 smithi173 ceph-mon[32553]: purged_snaps scrub starts 2024-04-26T01:43:28.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:27 smithi173 ceph-mon[32553]: purged_snaps scrub ok 2024-04-26T01:43:28.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:27 smithi173 ceph-mon[32553]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:28.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:27 smithi173 ceph-mon[32553]: Detected new or changed devices on smithi173 2024-04-26T01:43:28.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:27 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:28.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:27 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:28.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:27 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:43:28.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:27 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:43:28.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:27 smithi173 ceph-mon[32553]: Adjusting osd_memory_target on smithi173 to 8544M 2024-04-26T01:43:28.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:27 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:28.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:27 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:28.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:27 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:28.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:27 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:28.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:27 smithi173 ceph-mon[32553]: osd.5 [v2:172.21.15.173:6808/1026507957,v1:172.21.15.173:6809/1026507957] boot 2024-04-26T01:43:28.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:27 smithi173 ceph-mon[32553]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T01:43:28.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:27 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:43:28.398 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi001:vg_nvme/lv_2 2024-04-26T01:43:28.866 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:43:30.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:29 smithi173 ceph-mon[32553]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:30.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:29 smithi173 ceph-mon[32553]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T01:43:30.099 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:29 smithi001 ceph-mon[27822]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:30.099 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:29 smithi001 ceph-mon[27822]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T01:43:30.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:29 smithi097 ceph-mon[32853]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:43:30.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:29 smithi097 ceph-mon[32853]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T01:43:30.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.902+0000 7feb73400700 1 -- 172.21.15.1:0/669984988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c104690 msgr2=0x7feb6c10af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:43:30.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.902+0000 7feb73400700 1 --2- 172.21.15.1:0/669984988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c104690 0x7feb6c10af40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7feb64009920 tx=0x7feb64009c30 comp rx=0 tx=0).stop 2024-04-26T01:43:30.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.902+0000 7feb73400700 1 -- 172.21.15.1:0/669984988 shutdown_connections 2024-04-26T01:43:30.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.902+0000 7feb73400700 1 --2- 172.21.15.1:0/669984988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c104690 0x7feb6c10af40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:30.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.902+0000 7feb73400700 1 --2- 172.21.15.1:0/669984988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb6c102af0 0x7feb6c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:30.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.902+0000 7feb73400700 1 --2- 172.21.15.1:0/669984988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb6c103cf0 0x7feb6c104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:30.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.902+0000 7feb73400700 1 -- 172.21.15.1:0/669984988 >> 172.21.15.1:0/669984988 conn(0x7feb6c0fe090 msgr2=0x7feb6c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:43:30.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.902+0000 7feb73400700 1 -- 172.21.15.1:0/669984988 shutdown_connections 2024-04-26T01:43:30.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.902+0000 7feb73400700 1 -- 172.21.15.1:0/669984988 wait complete. 2024-04-26T01:43:30.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.904+0000 7feb73400700 1 Processor -- start 2024-04-26T01:43:30.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.904+0000 7feb73400700 1 -- start start 2024-04-26T01:43:30.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.904+0000 7feb73400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb6c102af0 0x7feb6c0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:43:30.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.904+0000 7feb73400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c103cf0 0x7feb6c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:43:30.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.904+0000 7feb73400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb6c104690 0x7feb6c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:43:30.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.904+0000 7feb73400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c10e5e0 con 0x7feb6c102af0 2024-04-26T01:43:30.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.904+0000 7feb73400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c10e460 con 0x7feb6c103cf0 2024-04-26T01:43:30.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.904+0000 7feb73400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c10e760 con 0x7feb6c104690 2024-04-26T01:43:30.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.905+0000 7feb70a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb6c102af0 0x7feb6c0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:43:30.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.905+0000 7feb70a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb6c102af0 0x7feb6c0767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58266/0 (socket says 172.21.15.1:58266) 2024-04-26T01:43:30.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.905+0000 7feb70a00700 1 -- 172.21.15.1:0/3222404792 learned_addr learned my addr 172.21.15.1:0/3222404792 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:43:30.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.905+0000 7feb6be00700 1 --2- 172.21.15.1:0/3222404792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c103cf0 0x7feb6c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:43:30.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.905+0000 7feb71400700 1 --2- 172.21.15.1:0/3222404792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb6c104690 0x7feb6c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:43:30.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.905+0000 7feb70a00700 1 -- 172.21.15.1:0/3222404792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb6c104690 msgr2=0x7feb6c075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:43:30.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.905+0000 7feb70a00700 1 --2- 172.21.15.1:0/3222404792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb6c104690 0x7feb6c075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:30.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.905+0000 7feb70a00700 1 -- 172.21.15.1:0/3222404792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c103cf0 msgr2=0x7feb6c074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:43:30.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.905+0000 7feb70a00700 1 --2- 172.21.15.1:0/3222404792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c103cf0 0x7feb6c074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:30.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.905+0000 7feb70a00700 1 -- 172.21.15.1:0/3222404792 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c075c20 con 0x7feb6c102af0 2024-04-26T01:43:30.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.905+0000 7feb6be00700 1 --2- 172.21.15.1:0/3222404792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c103cf0 0x7feb6c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:43:30.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.905+0000 7feb70a00700 1 --2- 172.21.15.1:0/3222404792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb6c102af0 0x7feb6c0767e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7feb6000e3f0 tx=0x7feb6000e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:43:30.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.906+0000 7feb69600700 1 -- 172.21.15.1:0/3222404792 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb600092d0 con 0x7feb6c102af0 2024-04-26T01:43:30.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.906+0000 7feb73400700 1 -- 172.21.15.1:0/3222404792 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6c1a2d80 con 0x7feb6c102af0 2024-04-26T01:43:30.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.906+0000 7feb69600700 1 -- 172.21.15.1:0/3222404792 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb60019070 con 0x7feb6c102af0 2024-04-26T01:43:30.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.906+0000 7feb69600700 1 -- 172.21.15.1:0/3222404792 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb600146e0 con 0x7feb6c102af0 2024-04-26T01:43:30.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.906+0000 7feb73400700 1 -- 172.21.15.1:0/3222404792 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6c1a3200 con 0x7feb6c102af0 2024-04-26T01:43:30.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.907+0000 7feb73400700 1 -- 172.21.15.1:0/3222404792 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6c069730 con 0x7feb6c102af0 2024-04-26T01:43:30.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.909+0000 7feb69600700 1 -- 172.21.15.1:0/3222404792 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7feb60012050 con 0x7feb6c102af0 2024-04-26T01:43:30.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.911+0000 7feb69600700 1 --2- 172.21.15.1:0/3222404792 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7feb540a3cc0 0x7feb540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:43:30.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.911+0000 7feb69600700 1 -- 172.21.15.1:0/3222404792 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7feb600cf4c0 con 0x7feb6c102af0 2024-04-26T01:43:30.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.911+0000 7feb6be00700 1 --2- 172.21.15.1:0/3222404792 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7feb540a3cc0 0x7feb540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:43:30.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.912+0000 7feb6be00700 1 --2- 172.21.15.1:0/3222404792 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7feb540a3cc0 0x7feb540a6180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feb58011490 tx=0x7feb58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:43:30.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:30.914+0000 7feb69600700 1 -- 172.21.15.1:0/3222404792 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb60022020 con 0x7feb6c102af0 2024-04-26T01:43:31.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:30 smithi173 ceph-mon[32553]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T01:43:31.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:30 smithi001 ceph-mon[27822]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T01:43:31.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:31.158+0000 7feb73400700 1 -- 172.21.15.1:0/3222404792 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7feb6c062e80 con 0x7feb540a3cc0 2024-04-26T01:43:31.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:30 smithi097 ceph-mon[32853]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T01:43:32.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:31 smithi173 ceph-mon[32553]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:32.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:31 smithi173 ceph-mon[32553]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T01:43:32.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:31 smithi173 ceph-mon[32553]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:43:32.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:31 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:43:32.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:31 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:43:32.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:31 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:32.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:31 smithi001 ceph-mon[27822]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:31 smithi001 ceph-mon[27822]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T01:43:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:31 smithi001 ceph-mon[27822]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:43:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:31 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:43:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:31 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:43:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:31 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:32.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:31 smithi097 ceph-mon[32853]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:32.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:31 smithi097 ceph-mon[32853]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T01:43:32.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:31 smithi097 ceph-mon[32853]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:43:32.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:43:32.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:43:32.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:33.058 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:32 smithi001 ceph-mon[27822]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T01:43:33.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:32 smithi173 ceph-mon[32553]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T01:43:33.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:32 smithi097 ceph-mon[32853]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T01:43:34.054 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:33 smithi001 ceph-mon[27822]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:34.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:33 smithi173 ceph-mon[32553]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:34.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:33 smithi097 ceph-mon[32853]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:36.065 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:35 smithi173 ceph-mon[32553]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-26T01:43:36.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:35 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3350965466' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97cd2e88-a56c-4b1a-8104-65b23a77986a"}]: dispatch 2024-04-26T01:43:36.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:35 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3350965466' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97cd2e88-a56c-4b1a-8104-65b23a77986a"}]': finished 2024-04-26T01:43:36.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:35 smithi173 ceph-mon[32553]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T01:43:36.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:35 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:43:36.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:35 smithi001 ceph-mon[27822]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-26T01:43:36.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:35 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3350965466' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97cd2e88-a56c-4b1a-8104-65b23a77986a"}]: dispatch 2024-04-26T01:43:36.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:35 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3350965466' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97cd2e88-a56c-4b1a-8104-65b23a77986a"}]': finished 2024-04-26T01:43:36.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:35 smithi001 ceph-mon[27822]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T01:43:36.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:35 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:43:36.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:35 smithi097 ceph-mon[32853]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-26T01:43:36.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:35 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3350965466' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97cd2e88-a56c-4b1a-8104-65b23a77986a"}]: dispatch 2024-04-26T01:43:36.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:35 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3350965466' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97cd2e88-a56c-4b1a-8104-65b23a77986a"}]': finished 2024-04-26T01:43:36.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:35 smithi097 ceph-mon[32853]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T01:43:36.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:35 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:43:37.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:36 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3392063946' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:43:37.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:36 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3392063946' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:43:37.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:36 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3392063946' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:43:38.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:37 smithi001 ceph-mon[27822]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:43:38.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:37 smithi097 ceph-mon[32853]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:43:38.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:37 smithi173 ceph-mon[32553]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:43:40.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:39 smithi001 ceph-mon[27822]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-26T01:43:40.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:39 smithi097 ceph-mon[32853]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-26T01:43:40.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:39 smithi173 ceph-mon[32553]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-26T01:43:42.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:41 smithi001 ceph-mon[27822]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-26T01:43:42.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:41 smithi097 ceph-mon[32853]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-26T01:43:42.315 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:41 smithi173 ceph-mon[32553]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-26T01:43:44.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:43 smithi001 ceph-mon[27822]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:43:44.220 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:43 smithi097 ceph-mon[32853]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:43:44.315 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:43 smithi173 ceph-mon[32553]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:43:44.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:44 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T01:43:44.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:44 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:45.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:44 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T01:43:45.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:44 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:45.315 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:44 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T01:43:45.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:44 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:46.126 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:45 smithi001 ceph-mon[27822]: Deploying daemon osd.6 on smithi001 2024-04-26T01:43:46.126 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:45 smithi001 ceph-mon[27822]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:46.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:45 smithi097 ceph-mon[32853]: Deploying daemon osd.6 on smithi001 2024-04-26T01:43:46.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:45 smithi097 ceph-mon[32853]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:46.315 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:45 smithi173 ceph-mon[32553]: Deploying daemon osd.6 on smithi001 2024-04-26T01:43:46.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:45 smithi173 ceph-mon[32553]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:47.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:47 smithi001 ceph-mon[27822]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:48.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:47 smithi097 ceph-mon[32853]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:48.315 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:47 smithi173 ceph-mon[32553]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:50.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:49 smithi001 ceph-mon[27822]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:50.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:49 smithi097 ceph-mon[32853]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:50.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:49 smithi173 ceph-mon[32553]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:52.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:51 smithi001 ceph-mon[27822]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:52.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:51 smithi097 ceph-mon[32853]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:52.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:51 smithi173 ceph-mon[32553]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:54.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:54 smithi173 ceph-mon[32553]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:54.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:54 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:54.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:54 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:54.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:54 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:54.599 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:54 smithi001 ceph-mon[27822]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:54.599 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:54 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:54.599 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:54 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:54.599 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:54 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:54.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:54 smithi097 ceph-mon[32853]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:54.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:54 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:54.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:54 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:54.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:54 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:56.546 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:56 smithi001 ceph-mon[27822]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:56.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:56 smithi173 ceph-mon[32553]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:56.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:56 smithi097 ceph-mon[32853]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:58.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:58.306+0000 7feb69600700 1 -- 172.21.15.1:0/3222404792 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7feb6c062e80 con 0x7feb540a3cc0 2024-04-26T01:43:58.305 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 6 on host 'smithi001' 2024-04-26T01:43:58.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:58.310+0000 7feb73400700 1 -- 172.21.15.1:0/3222404792 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7feb540a3cc0 msgr2=0x7feb540a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:43:58.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:58.310+0000 7feb73400700 1 --2- 172.21.15.1:0/3222404792 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7feb540a3cc0 0x7feb540a6180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feb58011490 tx=0x7feb58009260 comp rx=0 tx=0).stop 2024-04-26T01:43:58.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:58.310+0000 7feb73400700 1 -- 172.21.15.1:0/3222404792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb6c102af0 msgr2=0x7feb6c0767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:43:58.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:58.310+0000 7feb73400700 1 --2- 172.21.15.1:0/3222404792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb6c102af0 0x7feb6c0767e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7feb6000e3f0 tx=0x7feb6000e700 comp rx=0 tx=0).stop 2024-04-26T01:43:58.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:58.310+0000 7feb73400700 1 -- 172.21.15.1:0/3222404792 shutdown_connections 2024-04-26T01:43:58.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:58.310+0000 7feb73400700 1 --2- 172.21.15.1:0/3222404792 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7feb540a3cc0 0x7feb540a6180 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:58.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:58.310+0000 7feb73400700 1 --2- 172.21.15.1:0/3222404792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb6c104690 0x7feb6c075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:58.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:58.310+0000 7feb73400700 1 --2- 172.21.15.1:0/3222404792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb6c102af0 0x7feb6c0767e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:58.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:58.310+0000 7feb73400700 1 --2- 172.21.15.1:0/3222404792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c103cf0 0x7feb6c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:43:58.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:58.310+0000 7feb73400700 1 -- 172.21.15.1:0/3222404792 >> 172.21.15.1:0/3222404792 conn(0x7feb6c0fe090 msgr2=0x7feb6c106ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:43:58.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:58.311+0000 7feb73400700 1 -- 172.21.15.1:0/3222404792 shutdown_connections 2024-04-26T01:43:58.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:43:58.311+0000 7feb73400700 1 -- 172.21.15.1:0/3222404792 wait complete. 2024-04-26T01:43:58.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:58 smithi097 ceph-mon[32853]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:58.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:58 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:58.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:58 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:58.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:58 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:58.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:58 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:58.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:58 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:58.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:58 smithi097 ceph-mon[32853]: from='osd.6 [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T01:43:58.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:58 smithi173 ceph-mon[32553]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:58.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:58 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:58.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:58 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:58.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:58 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:58.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:58 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:58.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:58 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:58.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:58 smithi173 ceph-mon[32553]: from='osd.6 [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T01:43:58.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:58 smithi001 ceph-mon[27822]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:43:58.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:58 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:58.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:58 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:58.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:58 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:43:58.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:58 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:43:58.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:58 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:58.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:58 smithi001 ceph-mon[27822]: from='osd.6 [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T01:43:59.166 DEBUG:teuthology.orchestra.run.smithi001:osd.6> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.6.service 2024-04-26T01:43:59.173 INFO:tasks.cephadm:Deploying osd.7 on smithi097 with /dev/vg_nvme/lv_2... 2024-04-26T01:43:59.174 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T01:43:59.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:59 smithi001 ceph-mon[27822]: from='osd.6 [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T01:43:59.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:59 smithi001 ceph-mon[27822]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T01:43:59.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:59 smithi001 ceph-mon[27822]: from='osd.6 [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-04-26T01:43:59.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:59 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:43:59.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:59 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:59.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:59 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:59.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:43:59 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:59.379 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 01:43:59 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T01:43:59.289+0000 7efed7e00700 -1 osd.6 0 waiting for initial osdmap 2024-04-26T01:43:59.379 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 01:43:59 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T01:43:59.315+0000 7efed2c00700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:43:59.419 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:43:59.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:59 smithi097 ceph-mon[32853]: from='osd.6 [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T01:43:59.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:59 smithi097 ceph-mon[32853]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T01:43:59.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:59 smithi097 ceph-mon[32853]: from='osd.6 [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-04-26T01:43:59.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:59 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:43:59.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:59 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:59.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:59 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:59.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:43:59 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:59.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:59 smithi173 ceph-mon[32553]: from='osd.6 [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T01:43:59.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:59 smithi173 ceph-mon[32553]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T01:43:59.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:59 smithi173 ceph-mon[32553]: from='osd.6 [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-04-26T01:43:59.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:59 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:43:59.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:59 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:43:59.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:59 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:43:59.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:43:59 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:00.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:00 smithi097 ceph-mon[32853]: purged_snaps scrub starts 2024-04-26T01:44:00.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:00 smithi097 ceph-mon[32853]: purged_snaps scrub ok 2024-04-26T01:44:00.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:00 smithi097 ceph-mon[32853]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:44:00.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:00 smithi097 ceph-mon[32853]: from='osd.6 [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-04-26T01:44:00.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:00 smithi097 ceph-mon[32853]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T01:44:00.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:00 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:44:00.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:00 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:44:00.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:00 smithi173 ceph-mon[32553]: purged_snaps scrub starts 2024-04-26T01:44:00.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:00 smithi173 ceph-mon[32553]: purged_snaps scrub ok 2024-04-26T01:44:00.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:00 smithi173 ceph-mon[32553]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:44:00.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:00 smithi173 ceph-mon[32553]: from='osd.6 [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-04-26T01:44:00.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:00 smithi173 ceph-mon[32553]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T01:44:00.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:00 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:44:00.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:00 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:44:00.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:00 smithi001 ceph-mon[27822]: purged_snaps scrub starts 2024-04-26T01:44:00.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:00 smithi001 ceph-mon[27822]: purged_snaps scrub ok 2024-04-26T01:44:00.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:00 smithi001 ceph-mon[27822]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:44:00.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:00 smithi001 ceph-mon[27822]: from='osd.6 [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-04-26T01:44:00.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:00 smithi001 ceph-mon[27822]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T01:44:00.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:00 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:44:00.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:00 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:44:01.347 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T01:44:01.347 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T01:44:01.347 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-26T01:44:01.347 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-26T01:44:01.347 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0132516 s, 791 MB/s 2024-04-26T01:44:01.348 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 2024-04-26T01:44:01.348 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-26T01:44:01.501 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:01 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:44:01.501 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:01 smithi097 ceph-mon[32853]: osd.6 [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587] boot 2024-04-26T01:44:01.501 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:01 smithi097 ceph-mon[32853]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T01:44:01.501 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:01 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:44:01.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:01 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:44:01.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:01 smithi173 ceph-mon[32553]: osd.6 [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587] boot 2024-04-26T01:44:01.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:01 smithi173 ceph-mon[32553]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T01:44:01.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:01 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:44:01.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:01 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:44:01.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:01 smithi001 ceph-mon[27822]: osd.6 [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587] boot 2024-04-26T01:44:01.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:01 smithi001 ceph-mon[27822]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T01:44:01.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:01 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:44:01.960 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_2 2024-04-26T01:44:02.202 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:44:02.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:02 smithi001 ceph-mon[27822]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:44:02.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:02 smithi001 ceph-mon[27822]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T01:44:02.545 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:02 smithi097 ceph-mon[32853]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:44:02.545 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:02 smithi097 ceph-mon[32853]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T01:44:02.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:02 smithi173 ceph-mon[32553]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:44:02.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:02 smithi173 ceph-mon[32553]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T01:44:03.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.648+0000 7f4bc01a0700 1 -- 172.21.15.97:0/1957096309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb8103ca0 msgr2=0x7f4bb8104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:03.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.648+0000 7f4bc01a0700 1 --2- 172.21.15.97:0/1957096309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb8103ca0 0x7f4bb8104100 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80099c0 tx=0x7f4ba8009cd0 comp rx=0 tx=0).stop 2024-04-26T01:44:03.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.648+0000 7f4bc01a0700 1 -- 172.21.15.97:0/1957096309 shutdown_connections 2024-04-26T01:44:03.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.649+0000 7f4bc01a0700 1 --2- 172.21.15.97:0/1957096309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bb8104640 0x7f4bb810aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:03.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.649+0000 7f4bc01a0700 1 --2- 172.21.15.97:0/1957096309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bb8102aa0 0x7f4bb8102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:03.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.649+0000 7f4bc01a0700 1 --2- 172.21.15.97:0/1957096309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb8103ca0 0x7f4bb8104100 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:03.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.649+0000 7f4bc01a0700 1 -- 172.21.15.97:0/1957096309 >> 172.21.15.97:0/1957096309 conn(0x7f4bb80fe040 msgr2=0x7f4bb8100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:03.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.649+0000 7f4bc01a0700 1 -- 172.21.15.97:0/1957096309 shutdown_connections 2024-04-26T01:44:03.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.649+0000 7f4bc01a0700 1 -- 172.21.15.97:0/1957096309 wait complete. 2024-04-26T01:44:03.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.649+0000 7f4bc01a0700 1 Processor -- start 2024-04-26T01:44:03.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.649+0000 7f4bc01a0700 1 -- start start 2024-04-26T01:44:03.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.649+0000 7f4bc01a0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb8102aa0 0x7f4bb8198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:03.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.649+0000 7f4bc01a0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bb8103ca0 0x7f4bb8199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:03.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.649+0000 7f4bc01a0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bb8104640 0x7f4bb81a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:03.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.649+0000 7f4bc01a0700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb810e680 con 0x7f4bb8103ca0 2024-04-26T01:44:03.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.649+0000 7f4bc01a0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb810e500 con 0x7f4bb8102aa0 2024-04-26T01:44:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.649+0000 7f4bc01a0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb810e800 con 0x7f4bb8104640 2024-04-26T01:44:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.650+0000 7f4bbdf3c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb8102aa0 0x7f4bb8198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.650+0000 7f4bbdf3c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb8102aa0 0x7f4bb8198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53718/0 (socket says 172.21.15.97:53718) 2024-04-26T01:44:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.650+0000 7f4bbdf3c700 1 -- 172.21.15.97:0/1850785897 learned_addr learned my addr 172.21.15.97:0/1850785897 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T01:44:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.650+0000 7f4bbe73d700 1 --2- 172.21.15.97:0/1850785897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bb8104640 0x7f4bb81a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.650+0000 7f4bbdf3c700 1 -- 172.21.15.97:0/1850785897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bb8104640 msgr2=0x7f4bb81a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.650+0000 7f4bbdf3c700 1 --2- 172.21.15.97:0/1850785897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bb8104640 0x7f4bb81a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.650+0000 7f4bbd73b700 1 --2- 172.21.15.97:0/1850785897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bb8103ca0 0x7f4bb8199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.650+0000 7f4bbdf3c700 1 -- 172.21.15.97:0/1850785897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bb8103ca0 msgr2=0x7f4bb8199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.650+0000 7f4bbdf3c700 1 --2- 172.21.15.97:0/1850785897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bb8103ca0 0x7f4bb8199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.650+0000 7f4bbdf3c700 1 -- 172.21.15.97:0/1850785897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb81a0990 con 0x7f4bb8102aa0 2024-04-26T01:44:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.650+0000 7f4bbe73d700 1 --2- 172.21.15.97:0/1850785897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bb8104640 0x7f4bb81a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:44:03.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.650+0000 7f4bbdf3c700 1 --2- 172.21.15.97:0/1850785897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb8102aa0 0x7f4bb8198d30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400b500 tx=0x7f4bb400b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:03.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.650+0000 7f4baeffd700 1 -- 172.21.15.97:0/1850785897 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4015020 con 0x7f4bb8102aa0 2024-04-26T01:44:03.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.650+0000 7f4baeffd700 1 -- 172.21.15.97:0/1850785897 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bb4011ce0 con 0x7f4bb8102aa0 2024-04-26T01:44:03.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.650+0000 7f4bc01a0700 1 -- 172.21.15.97:0/1850785897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb81a0c80 con 0x7f4bb8102aa0 2024-04-26T01:44:03.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.650+0000 7f4baeffd700 1 -- 172.21.15.97:0/1850785897 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4010d50 con 0x7f4bb8102aa0 2024-04-26T01:44:03.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.650+0000 7f4bc01a0700 1 -- 172.21.15.97:0/1850785897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb81a11c0 con 0x7f4bb8102aa0 2024-04-26T01:44:03.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.651+0000 7f4bc01a0700 1 -- 172.21.15.97:0/1850785897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb8069730 con 0x7f4bb8102aa0 2024-04-26T01:44:03.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.653+0000 7f4baeffd700 1 -- 172.21.15.97:0/1850785897 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f4bb4018020 con 0x7f4bb8102aa0 2024-04-26T01:44:03.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.653+0000 7f4baeffd700 1 --2- 172.21.15.97:0/1850785897 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f4ba40a3cc0 0x7f4ba40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:03.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.654+0000 7f4baeffd700 1 -- 172.21.15.97:0/1850785897 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f4bb40cf420 con 0x7f4bb8102aa0 2024-04-26T01:44:03.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.654+0000 7f4bbd73b700 1 --2- 172.21.15.97:0/1850785897 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f4ba40a3cc0 0x7f4ba40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:03.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.655+0000 7f4bbd73b700 1 --2- 172.21.15.97:0/1850785897 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f4ba40a3cc0 0x7f4ba40a6180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80098e0 tx=0x7f4ba8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:03.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.657+0000 7f4baeffd700 1 -- 172.21.15.97:0/1850785897 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4bb401f020 con 0x7f4bb8102aa0 2024-04-26T01:44:03.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:03.865+0000 7f4bc01a0700 1 -- 172.21.15.97:0/1850785897 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4bb8062e80 con 0x7f4ba40a3cc0 2024-04-26T01:44:04.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:04 smithi173 ceph-mon[32553]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:04.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:04 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:44:04.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:04 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:44:04.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:04 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:04.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:04 smithi001 ceph-mon[27822]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:04.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:04 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:44:04.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:04 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:44:04.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:04 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:04.638 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:04 smithi097 ceph-mon[32853]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:04.638 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:04 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:44:04.638 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:04 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:44:04.639 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:04 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:05.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:05 smithi001 ceph-mon[27822]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:44:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:05 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:05 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:05 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:05 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:05 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:05 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:05 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:05 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:05.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:05 smithi097 ceph-mon[32853]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:44:05.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:05 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:05.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:05 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:05.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:05 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:05.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:05 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:05.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:05 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:05.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:05 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:05.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:05 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:05.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:05 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:05.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:05 smithi173 ceph-mon[32553]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:44:05.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:05 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:05.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:05 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:05.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:05 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:05.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:05 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:05.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:05 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:05.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:05 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:05.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:05 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:05.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:05 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:06.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:06 smithi173 ceph-mon[32553]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:06.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:06 smithi173 ceph-mon[32553]: Detected new or changed devices on smithi001 2024-04-26T01:44:06.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:06 smithi173 ceph-mon[32553]: Adjusting osd_memory_target on smithi001 to 5698M 2024-04-26T01:44:06.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:06 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:06.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:06 smithi001 ceph-mon[27822]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:06.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:06 smithi001 ceph-mon[27822]: Detected new or changed devices on smithi001 2024-04-26T01:44:06.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:06 smithi001 ceph-mon[27822]: Adjusting osd_memory_target on smithi001 to 5698M 2024-04-26T01:44:06.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:06 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:06.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:06 smithi097 ceph-mon[32853]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:06.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:06 smithi097 ceph-mon[32853]: Detected new or changed devices on smithi001 2024-04-26T01:44:06.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:06 smithi097 ceph-mon[32853]: Adjusting osd_memory_target on smithi001 to 5698M 2024-04-26T01:44:06.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:06 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:07.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:07 smithi173 ceph-mon[32553]: from='client.? 172.21.15.97:0/2488943555' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afc0c2dd-e112-4358-b3aa-944366d5d383"}]: dispatch 2024-04-26T01:44:07.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:07 smithi173 ceph-mon[32553]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afc0c2dd-e112-4358-b3aa-944366d5d383"}]: dispatch 2024-04-26T01:44:07.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:07 smithi173 ceph-mon[32553]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "afc0c2dd-e112-4358-b3aa-944366d5d383"}]': finished 2024-04-26T01:44:07.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:07 smithi173 ceph-mon[32553]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T01:44:07.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:07 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:44:07.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:07 smithi173 ceph-mon[32553]: from='client.? 172.21.15.97:0/2541137621' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:44:07.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:07 smithi001 ceph-mon[27822]: from='client.? 172.21.15.97:0/2488943555' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afc0c2dd-e112-4358-b3aa-944366d5d383"}]: dispatch 2024-04-26T01:44:07.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:07 smithi001 ceph-mon[27822]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afc0c2dd-e112-4358-b3aa-944366d5d383"}]: dispatch 2024-04-26T01:44:07.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:07 smithi001 ceph-mon[27822]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "afc0c2dd-e112-4358-b3aa-944366d5d383"}]': finished 2024-04-26T01:44:07.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:07 smithi001 ceph-mon[27822]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T01:44:07.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:07 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:44:07.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:07 smithi001 ceph-mon[27822]: from='client.? 172.21.15.97:0/2541137621' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:44:07.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:07 smithi097 ceph-mon[32853]: from='client.? 172.21.15.97:0/2488943555' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afc0c2dd-e112-4358-b3aa-944366d5d383"}]: dispatch 2024-04-26T01:44:07.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:07 smithi097 ceph-mon[32853]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "afc0c2dd-e112-4358-b3aa-944366d5d383"}]: dispatch 2024-04-26T01:44:07.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:07 smithi097 ceph-mon[32853]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "afc0c2dd-e112-4358-b3aa-944366d5d383"}]': finished 2024-04-26T01:44:07.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:07 smithi097 ceph-mon[32853]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T01:44:07.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:07 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:44:07.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:07 smithi097 ceph-mon[32853]: from='client.? 172.21.15.97:0/2541137621' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:44:08.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:08 smithi173 ceph-mon[32553]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:08.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:08 smithi001 ceph-mon[27822]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:08.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:08 smithi097 ceph-mon[32853]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:10.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:10 smithi001 ceph-mon[27822]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:10.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:10 smithi097 ceph-mon[32853]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:10.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:10 smithi173 ceph-mon[32553]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:12.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:12 smithi097 ceph-mon[32853]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:12.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:12 smithi001 ceph-mon[27822]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:12.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:12 smithi173 ceph-mon[32553]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:14.406 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:14 smithi097 ceph-mon[32853]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:14.406 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:14 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T01:44:14.407 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:14 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:14.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:14 smithi001 ceph-mon[27822]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:14.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:14 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T01:44:14.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:14 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:14.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:14 smithi173 ceph-mon[32553]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:14.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:14 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T01:44:14.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:14 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:15.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:15 smithi001 ceph-mon[27822]: Deploying daemon osd.7 on smithi097 2024-04-26T01:44:15.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:15 smithi097 ceph-mon[32853]: Deploying daemon osd.7 on smithi097 2024-04-26T01:44:15.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:15 smithi173 ceph-mon[32553]: Deploying daemon osd.7 on smithi097 2024-04-26T01:44:16.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:16 smithi097 ceph-mon[32853]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:16.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:16 smithi173 ceph-mon[32553]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:16.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:16 smithi001 ceph-mon[27822]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:18.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:18 smithi001 ceph-mon[27822]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:18.639 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:18 smithi097 ceph-mon[32853]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:18.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:18 smithi173 ceph-mon[32553]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:20.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:20 smithi001 ceph-mon[27822]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:20.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:20.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:20.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:20 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:20.633 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:20 smithi097 ceph-mon[32853]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:20.633 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:20 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:20.633 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:20 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:20.633 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:20 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:20.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:20 smithi173 ceph-mon[32553]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:20.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:20 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:20.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:20 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:20.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:20 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:22.532 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:22 smithi097 ceph-mon[32853]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:22.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:22 smithi001 ceph-mon[27822]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:22.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:22 smithi173 ceph-mon[32553]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:23.442 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 7 on host 'smithi097' 2024-04-26T01:44:23.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:23.445+0000 7f4baeffd700 1 -- 172.21.15.97:0/1850785897 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4bb8062e80 con 0x7f4ba40a3cc0 2024-04-26T01:44:23.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:23.447+0000 7f4bc01a0700 1 -- 172.21.15.97:0/1850785897 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f4ba40a3cc0 msgr2=0x7f4ba40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:23.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:23.447+0000 7f4bc01a0700 1 --2- 172.21.15.97:0/1850785897 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f4ba40a3cc0 0x7f4ba40a6180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80098e0 tx=0x7f4ba8000bc0 comp rx=0 tx=0).stop 2024-04-26T01:44:23.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:23.447+0000 7f4bc01a0700 1 -- 172.21.15.97:0/1850785897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb8102aa0 msgr2=0x7f4bb8198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:23.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:23.447+0000 7f4bc01a0700 1 --2- 172.21.15.97:0/1850785897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb8102aa0 0x7f4bb8198d30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400b500 tx=0x7f4bb400b8c0 comp rx=0 tx=0).stop 2024-04-26T01:44:23.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:23.447+0000 7f4bc01a0700 1 -- 172.21.15.97:0/1850785897 shutdown_connections 2024-04-26T01:44:23.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:23.447+0000 7f4bc01a0700 1 --2- 172.21.15.97:0/1850785897 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f4ba40a3cc0 0x7f4ba40a6180 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:23.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:23.447+0000 7f4bc01a0700 1 --2- 172.21.15.97:0/1850785897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bb8104640 0x7f4bb81a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:23.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:23.447+0000 7f4bc01a0700 1 --2- 172.21.15.97:0/1850785897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb8102aa0 0x7f4bb8198d30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:23.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:23.447+0000 7f4bc01a0700 1 --2- 172.21.15.97:0/1850785897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bb8103ca0 0x7f4bb8199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:23.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:23.447+0000 7f4bc01a0700 1 -- 172.21.15.97:0/1850785897 >> 172.21.15.97:0/1850785897 conn(0x7f4bb80fe040 msgr2=0x7f4bb81070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:23.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:23.447+0000 7f4bc01a0700 1 -- 172.21.15.97:0/1850785897 shutdown_connections 2024-04-26T01:44:23.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:44:23.447+0000 7f4bc01a0700 1 -- 172.21.15.97:0/1850785897 wait complete. 2024-04-26T01:44:24.050 DEBUG:teuthology.orchestra.run.smithi097:osd.7> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.7.service 2024-04-26T01:44:24.052 INFO:tasks.cephadm:Deploying osd.8 on smithi173 with /dev/vg_nvme/lv_2... 2024-04-26T01:44:24.053 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T01:44:24.300 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:44:24.732 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:24 smithi173 ceph-mon[32553]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:24.732 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:24 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:24.732 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:24 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:24.732 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:24 smithi173 ceph-mon[32553]: from='osd.7 [v2:172.21.15.97:6816/2600022531,v1:172.21.15.97:6817/2600022531]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T01:44:24.732 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:24 smithi173 ceph-mon[32553]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T01:44:24.733 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:24 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:24.733 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:24 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:24.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:24 smithi097 ceph-mon[32853]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:24.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:24.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:24.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:24 smithi097 ceph-mon[32853]: from='osd.7 [v2:172.21.15.97:6816/2600022531,v1:172.21.15.97:6817/2600022531]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T01:44:24.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:24 smithi097 ceph-mon[32853]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T01:44:24.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:24.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:24.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:24 smithi001 ceph-mon[27822]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:24.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:24 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:24.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:24 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:24.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:24 smithi001 ceph-mon[27822]: from='osd.7 [v2:172.21.15.97:6816/2600022531,v1:172.21.15.97:6817/2600022531]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T01:44:24.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:24 smithi001 ceph-mon[27822]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T01:44:24.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:24 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:24.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:24 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:25.697 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:25 smithi173 ceph-mon[32553]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T01:44:25.698 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:25 smithi173 ceph-mon[32553]: from='osd.7 [v2:172.21.15.97:6816/2600022531,v1:172.21.15.97:6817/2600022531]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:44:25.698 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:25 smithi173 ceph-mon[32553]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T01:44:25.698 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:25 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:44:25.698 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:25 smithi173 ceph-mon[32553]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:44:25.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:25 smithi097 ceph-mon[32853]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T01:44:25.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:25 smithi097 ceph-mon[32853]: from='osd.7 [v2:172.21.15.97:6816/2600022531,v1:172.21.15.97:6817/2600022531]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:44:25.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:25 smithi097 ceph-mon[32853]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T01:44:25.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:25 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:44:25.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:25 smithi097 ceph-mon[32853]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:44:25.733 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 01:44:25 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T01:44:25.415+0000 7f5f200ec700 -1 osd.7 0 waiting for initial osdmap 2024-04-26T01:44:25.733 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 01:44:25 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T01:44:25.426+0000 7f5f1bedf700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:44:25.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:25 smithi001 ceph-mon[27822]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T01:44:25.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:25 smithi001 ceph-mon[27822]: from='osd.7 [v2:172.21.15.97:6816/2600022531,v1:172.21.15.97:6817/2600022531]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:44:25.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:25 smithi001 ceph-mon[27822]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T01:44:25.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:25 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:44:25.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:25 smithi001 ceph-mon[27822]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:44:26.070 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T01:44:26.070 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T01:44:26.070 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-04-26T01:44:26.070 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-04-26T01:44:26.070 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112812 s, 929 MB/s 2024-04-26T01:44:26.070 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-04-26T01:44:26.644 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:26 smithi173 ceph-mon[32553]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:26.644 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:26 smithi173 ceph-mon[32553]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T01:44:26.644 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:26 smithi173 ceph-mon[32553]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T01:44:26.644 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:26 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:44:26.673 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_2 2024-04-26T01:44:26.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:26 smithi097 ceph-mon[32853]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:26.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:26 smithi097 ceph-mon[32853]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T01:44:26.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:26 smithi097 ceph-mon[32853]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T01:44:26.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:26 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:44:26.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:26 smithi001 ceph-mon[27822]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:44:26.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:26 smithi001 ceph-mon[27822]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T01:44:26.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:26 smithi001 ceph-mon[27822]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T01:44:26.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:26 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:44:26.920 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:44:27.708 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:27 smithi173 ceph-mon[32553]: purged_snaps scrub starts 2024-04-26T01:44:27.709 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:27 smithi173 ceph-mon[32553]: purged_snaps scrub ok 2024-04-26T01:44:27.709 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:27 smithi173 ceph-mon[32553]: osd.7 [v2:172.21.15.97:6816/2600022531,v1:172.21.15.97:6817/2600022531] boot 2024-04-26T01:44:27.709 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:27 smithi173 ceph-mon[32553]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T01:44:27.709 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:27 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:44:27.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:27 smithi097 ceph-mon[32853]: purged_snaps scrub starts 2024-04-26T01:44:27.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:27 smithi097 ceph-mon[32853]: purged_snaps scrub ok 2024-04-26T01:44:27.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:27 smithi097 ceph-mon[32853]: osd.7 [v2:172.21.15.97:6816/2600022531,v1:172.21.15.97:6817/2600022531] boot 2024-04-26T01:44:27.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:27 smithi097 ceph-mon[32853]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T01:44:27.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:27 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:44:27.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:27 smithi001 ceph-mon[27822]: purged_snaps scrub starts 2024-04-26T01:44:27.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:27 smithi001 ceph-mon[27822]: purged_snaps scrub ok 2024-04-26T01:44:27.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:27 smithi001 ceph-mon[27822]: osd.7 [v2:172.21.15.97:6816/2600022531,v1:172.21.15.97:6817/2600022531] boot 2024-04-26T01:44:27.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:27 smithi001 ceph-mon[27822]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T01:44:27.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:27 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:44:28.486 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.486+0000 7eff3bbe4700 1 -- 172.21.15.173:0/3942673285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff34104650 msgr2=0x7eff3410af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:28.486 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.486+0000 7eff3bbe4700 1 --2- 172.21.15.173:0/3942673285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff34104650 0x7eff3410af00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009990 tx=0x7eff2c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:44:28.486 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.486+0000 7eff3bbe4700 1 -- 172.21.15.173:0/3942673285 shutdown_connections 2024-04-26T01:44:28.486 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.486+0000 7eff3bbe4700 1 --2- 172.21.15.173:0/3942673285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff34104650 0x7eff3410af00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:28.486 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.486+0000 7eff3bbe4700 1 --2- 172.21.15.173:0/3942673285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff34102ab0 0x7eff34102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:28.486 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.486+0000 7eff3bbe4700 1 --2- 172.21.15.173:0/3942673285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff34103cb0 0x7eff34104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:28.486 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.486+0000 7eff3bbe4700 1 -- 172.21.15.173:0/3942673285 >> 172.21.15.173:0/3942673285 conn(0x7eff340fe010 msgr2=0x7eff34100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:28.486 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.486+0000 7eff3bbe4700 1 -- 172.21.15.173:0/3942673285 shutdown_connections 2024-04-26T01:44:28.487 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.486+0000 7eff3bbe4700 1 -- 172.21.15.173:0/3942673285 wait complete. 2024-04-26T01:44:28.487 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.487+0000 7eff3bbe4700 1 Processor -- start 2024-04-26T01:44:28.487 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.487+0000 7eff3bbe4700 1 -- start start 2024-04-26T01:44:28.487 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.487+0000 7eff3bbe4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff34102ab0 0x7eff34198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:28.487 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.487+0000 7eff3bbe4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff34103cb0 0x7eff34199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:28.487 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.487+0000 7eff3bbe4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff34104650 0x7eff341a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:28.487 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.487+0000 7eff3bbe4700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7eff3410e660 con 0x7eff34104650 2024-04-26T01:44:28.487 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.487+0000 7eff3bbe4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff3410e4e0 con 0x7eff34102ab0 2024-04-26T01:44:28.487 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.487+0000 7eff3bbe4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7eff3410e7e0 con 0x7eff34103cb0 2024-04-26T01:44:28.487 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.487+0000 7eff3917f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff34103cb0 0x7eff34199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:28.487 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.487+0000 7eff3917f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff34103cb0 0x7eff34199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:42182/0 (socket says 172.21.15.173:42182) 2024-04-26T01:44:28.488 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.487+0000 7eff3917f700 1 -- 172.21.15.173:0/1048176542 learned_addr learned my addr 172.21.15.173:0/1048176542 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-26T01:44:28.488 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.487+0000 7eff3917f700 1 -- 172.21.15.173:0/1048176542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff34102ab0 msgr2=0x7eff34198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:28.488 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.487+0000 7eff3917f700 1 --2- 172.21.15.173:0/1048176542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff34102ab0 0x7eff34198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:28.488 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.487+0000 7eff3917f700 1 -- 172.21.15.173:0/1048176542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff34104650 msgr2=0x7eff341a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:28.488 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.487+0000 7eff3917f700 1 --2- 172.21.15.173:0/1048176542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff34104650 0x7eff341a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:28.488 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.487+0000 7eff3917f700 1 -- 172.21.15.173:0/1048176542 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff341a0950 con 0x7eff34103cb0 2024-04-26T01:44:28.488 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.487+0000 7eff3917f700 1 --2- 172.21.15.173:0/1048176542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff34103cb0 0x7eff34199290 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7eff2400b550 tx=0x7eff2400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:28.488 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.487+0000 7eff2affd700 1 -- 172.21.15.173:0/1048176542 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24015020 con 0x7eff34103cb0 2024-04-26T01:44:28.488 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.488+0000 7eff2affd700 1 -- 172.21.15.173:0/1048176542 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff24011ce0 con 0x7eff34103cb0 2024-04-26T01:44:28.488 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.488+0000 7eff3bbe4700 1 -- 172.21.15.173:0/1048176542 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff341a0c40 con 0x7eff34103cb0 2024-04-26T01:44:28.488 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.488+0000 7eff2affd700 1 -- 172.21.15.173:0/1048176542 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24010d20 con 0x7eff34103cb0 2024-04-26T01:44:28.489 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.488+0000 7eff3bbe4700 1 -- 172.21.15.173:0/1048176542 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff341a1200 con 0x7eff34103cb0 2024-04-26T01:44:28.489 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.489+0000 7eff3bbe4700 1 -- 172.21.15.173:0/1048176542 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff34069730 con 0x7eff34103cb0 2024-04-26T01:44:28.490 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.490+0000 7eff2affd700 1 -- 172.21.15.173:0/1048176542 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7eff24018020 con 0x7eff34103cb0 2024-04-26T01:44:28.490 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.491+0000 7eff2affd700 1 --2- 172.21.15.173:0/1048176542 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7eff200a3c40 0x7eff200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:28.490 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.491+0000 7eff2affd700 1 -- 172.21.15.173:0/1048176542 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7eff240cf620 con 0x7eff34103cb0 2024-04-26T01:44:28.490 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.492+0000 7eff39980700 1 --2- 172.21.15.173:0/1048176542 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7eff200a3c40 0x7eff200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:28.490 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.493+0000 7eff39980700 1 --2- 172.21.15.173:0/1048176542 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7eff200a3c40 0x7eff200a6100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7eff3419a1e0 tx=0x7eff30009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:28.492 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.495+0000 7eff2affd700 1 -- 172.21.15.173:0/1048176542 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff2401f020 con 0x7eff34103cb0 2024-04-26T01:44:28.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:28 smithi173 ceph-mon[32553]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:28.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:28 smithi173 ceph-mon[32553]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T01:44:28.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:28 smithi173 ceph-mon[32553]: Detected new or changed devices on smithi097 2024-04-26T01:44:28.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:28.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:28.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:28.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:28.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:28.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:28 smithi173 ceph-mon[32553]: Adjusting osd_memory_target on smithi097 to 5696M 2024-04-26T01:44:28.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:28.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:28.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:28.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:28.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:28.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:28.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:28.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:28.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:28 smithi173 ceph-mon[32553]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T01:44:28.700 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:28.702+0000 7eff3bbe4700 1 -- 172.21.15.173:0/1048176542 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7eff34062e80 con 0x7eff200a3c40 2024-04-26T01:44:28.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:28 smithi097 ceph-mon[32853]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:28.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:28 smithi097 ceph-mon[32853]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T01:44:28.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:28 smithi097 ceph-mon[32853]: Detected new or changed devices on smithi097 2024-04-26T01:44:28.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:28.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:28.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:28.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:28.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:28.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:28 smithi097 ceph-mon[32853]: Adjusting osd_memory_target on smithi097 to 5696M 2024-04-26T01:44:28.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:28.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:28.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:28.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:28.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:28.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:28.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:28.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:28.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:28 smithi097 ceph-mon[32853]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T01:44:28.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:28 smithi001 ceph-mon[27822]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:28.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:28 smithi001 ceph-mon[27822]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T01:44:28.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:28 smithi001 ceph-mon[27822]: Detected new or changed devices on smithi097 2024-04-26T01:44:28.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:28.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:28.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:28.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:28.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:28.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:28 smithi001 ceph-mon[27822]: Adjusting osd_memory_target on smithi097 to 5696M 2024-04-26T01:44:28.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:28.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:28.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:28.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:28.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:28.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:28.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:28.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:28.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:28 smithi001 ceph-mon[27822]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T01:44:29.666 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:44:29.666 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:44:29.666 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:29.666 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:29 smithi173 ceph-mon[32553]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T01:44:29.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:29 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:44:29.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:29 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:44:29.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:29 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:29.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:29 smithi097 ceph-mon[32853]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T01:44:29.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:44:29.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:44:29.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:29.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:29 smithi001 ceph-mon[27822]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T01:44:30.544 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:30 smithi173 ceph-mon[32553]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:44:30.544 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:30 smithi173 ceph-mon[32553]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:30.544 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:30 smithi173 ceph-mon[32553]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T01:44:30.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:30 smithi097 ceph-mon[32853]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:44:30.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:30 smithi097 ceph-mon[32853]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:30.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:30 smithi097 ceph-mon[32853]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T01:44:30.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:30 smithi001 ceph-mon[27822]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:44:30.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:30 smithi001 ceph-mon[27822]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:30.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:30 smithi001 ceph-mon[27822]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T01:44:31.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:31 smithi097 ceph-mon[32853]: from='client.? 172.21.15.173:0/1798715229' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41fa612b-793e-44da-af8b-44d17a9de303"}]: dispatch 2024-04-26T01:44:31.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:31 smithi097 ceph-mon[32853]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41fa612b-793e-44da-af8b-44d17a9de303"}]: dispatch 2024-04-26T01:44:31.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:31 smithi097 ceph-mon[32853]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41fa612b-793e-44da-af8b-44d17a9de303"}]': finished 2024-04-26T01:44:31.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:31 smithi097 ceph-mon[32853]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T01:44:31.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:44:31.734 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:31 smithi173 ceph-mon[32553]: from='client.? 172.21.15.173:0/1798715229' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41fa612b-793e-44da-af8b-44d17a9de303"}]: dispatch 2024-04-26T01:44:31.734 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:31 smithi173 ceph-mon[32553]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41fa612b-793e-44da-af8b-44d17a9de303"}]: dispatch 2024-04-26T01:44:31.734 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:31 smithi173 ceph-mon[32553]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41fa612b-793e-44da-af8b-44d17a9de303"}]': finished 2024-04-26T01:44:31.734 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:31 smithi173 ceph-mon[32553]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T01:44:31.734 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:31 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:44:31.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:31 smithi001 ceph-mon[27822]: from='client.? 172.21.15.173:0/1798715229' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41fa612b-793e-44da-af8b-44d17a9de303"}]: dispatch 2024-04-26T01:44:31.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:31 smithi001 ceph-mon[27822]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41fa612b-793e-44da-af8b-44d17a9de303"}]: dispatch 2024-04-26T01:44:31.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:31 smithi001 ceph-mon[27822]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41fa612b-793e-44da-af8b-44d17a9de303"}]': finished 2024-04-26T01:44:31.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:31 smithi001 ceph-mon[27822]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T01:44:31.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:31 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:44:32.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:32 smithi097 ceph-mon[32853]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:32.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:32 smithi097 ceph-mon[32853]: from='client.? 172.21.15.173:0/236822475' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:44:32.815 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:32 smithi173 ceph-mon[32553]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:32.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:32 smithi173 ceph-mon[32553]: from='client.? 172.21.15.173:0/236822475' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:44:32.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:32 smithi001 ceph-mon[27822]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:32.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:32 smithi001 ceph-mon[27822]: from='client.? 172.21.15.173:0/236822475' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:44:34.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:34 smithi001 ceph-mon[27822]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:34.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:34 smithi097 ceph-mon[32853]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:34.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:34 smithi173 ceph-mon[32553]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:36.718 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:36 smithi173 ceph-mon[32553]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:36.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:36 smithi097 ceph-mon[32853]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:36.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:36 smithi001 ceph-mon[27822]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:38.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:38 smithi097 ceph-mon[32853]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:38.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:38 smithi173 ceph-mon[32553]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:38.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:38 smithi001 ceph-mon[27822]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:39.654 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T01:44:39.655 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:39.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T01:44:39.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:39.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T01:44:39.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:40.756 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:40 smithi173 ceph-mon[32553]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:40.756 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:40 smithi173 ceph-mon[32553]: Deploying daemon osd.8 on smithi173 2024-04-26T01:44:40.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:40 smithi001 ceph-mon[27822]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:40.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:40 smithi001 ceph-mon[27822]: Deploying daemon osd.8 on smithi173 2024-04-26T01:44:40.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:40 smithi097 ceph-mon[32853]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:40.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:40 smithi097 ceph-mon[32853]: Deploying daemon osd.8 on smithi173 2024-04-26T01:44:42.731 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:42 smithi173 ceph-mon[32553]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:42.784 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:42 smithi097 ceph-mon[32853]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:42.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:42 smithi001 ceph-mon[27822]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:44.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:44 smithi173 ceph-mon[32553]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:44.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:44 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:44.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:44 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:44.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:44 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:44.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:44 smithi001 ceph-mon[27822]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:44.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:44 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:44.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:44 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:44.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:44 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:44.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:44 smithi097 ceph-mon[32853]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:44.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:44 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:44.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:44 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:44.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:44 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:46.789 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:46 smithi173 ceph-mon[32553]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:46.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:46 smithi001 ceph-mon[27822]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:46.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:46 smithi097 ceph-mon[32853]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:47.434 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:47.435+0000 7eff2affd700 1 -- 172.21.15.173:0/1048176542 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7eff34062e80 con 0x7eff200a3c40 2024-04-26T01:44:47.434 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 8 on host 'smithi173' 2024-04-26T01:44:47.435 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:47.438+0000 7eff3bbe4700 1 -- 172.21.15.173:0/1048176542 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7eff200a3c40 msgr2=0x7eff200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:47.436 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:47.438+0000 7eff3bbe4700 1 --2- 172.21.15.173:0/1048176542 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7eff200a3c40 0x7eff200a6100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7eff3419a1e0 tx=0x7eff30009250 comp rx=0 tx=0).stop 2024-04-26T01:44:47.436 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:47.438+0000 7eff3bbe4700 1 -- 172.21.15.173:0/1048176542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff34103cb0 msgr2=0x7eff34199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:47.436 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:47.438+0000 7eff3bbe4700 1 --2- 172.21.15.173:0/1048176542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff34103cb0 0x7eff34199290 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7eff2400b550 tx=0x7eff2400b910 comp rx=0 tx=0).stop 2024-04-26T01:44:47.436 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:47.438+0000 7eff3bbe4700 1 -- 172.21.15.173:0/1048176542 shutdown_connections 2024-04-26T01:44:47.436 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:47.438+0000 7eff3bbe4700 1 --2- 172.21.15.173:0/1048176542 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7eff200a3c40 0x7eff200a6100 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:47.436 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:47.438+0000 7eff3bbe4700 1 --2- 172.21.15.173:0/1048176542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff34104650 0x7eff341a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:47.436 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:47.438+0000 7eff3bbe4700 1 --2- 172.21.15.173:0/1048176542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff34102ab0 0x7eff34198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:47.436 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:47.438+0000 7eff3bbe4700 1 --2- 172.21.15.173:0/1048176542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff34103cb0 0x7eff34199290 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:47.436 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:47.438+0000 7eff3bbe4700 1 -- 172.21.15.173:0/1048176542 >> 172.21.15.173:0/1048176542 conn(0x7eff340fe010 msgr2=0x7eff34107080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:47.436 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:47.438+0000 7eff3bbe4700 1 -- 172.21.15.173:0/1048176542 shutdown_connections 2024-04-26T01:44:47.436 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:44:47.438+0000 7eff3bbe4700 1 -- 172.21.15.173:0/1048176542 wait complete. 2024-04-26T01:44:47.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:47 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:47.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:47 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:47.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:47 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:47.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:47 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:47.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:47 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:47.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:47 smithi097 ceph-mon[32853]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:47.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:47 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:47.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:47 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:47.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:47 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:48.016 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:47 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:48.016 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:47 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:48.016 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:47 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:48.016 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:47 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:48.016 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:47 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:48.016 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:47 smithi173 ceph-mon[32553]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:48.016 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:47 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:48.017 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:47 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:48.017 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:47 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:48.044 DEBUG:teuthology.orchestra.run.smithi173:osd.8> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.8.service 2024-04-26T01:44:48.046 INFO:tasks.cephadm:Deploying osd.9 on smithi001 with /dev/vg_nvme/lv_1... 2024-04-26T01:44:48.046 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T01:44:48.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:47 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:48.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:47 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:48.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:47 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:48.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:47 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:48.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:47 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:48.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:47 smithi001 ceph-mon[27822]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:48.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:47 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:48.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:47 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:48.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:47 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:44:48.497 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:44:48.950 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:48 smithi001 ceph-mon[27822]: from='osd.8 [v2:172.21.15.173:6816/1334205527,v1:172.21.15.173:6817/1334205527]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T01:44:48.950 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:48 smithi001 ceph-mon[27822]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T01:44:48.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:48 smithi097 ceph-mon[32853]: from='osd.8 [v2:172.21.15.173:6816/1334205527,v1:172.21.15.173:6817/1334205527]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T01:44:48.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:48 smithi097 ceph-mon[32853]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T01:44:49.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:48 smithi173 ceph-mon[32553]: from='osd.8 [v2:172.21.15.173:6816/1334205527,v1:172.21.15.173:6817/1334205527]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T01:44:49.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:48 smithi173 ceph-mon[32553]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T01:44:49.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:49 smithi001 ceph-mon[27822]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T01:44:49.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:49 smithi001 ceph-mon[27822]: from='osd.8 [v2:172.21.15.173:6816/1334205527,v1:172.21.15.173:6817/1334205527]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:44:49.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:49 smithi001 ceph-mon[27822]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T01:44:49.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:49 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:44:49.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:49 smithi001 ceph-mon[27822]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:44:49.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:49 smithi001 ceph-mon[27822]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:49.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:49 smithi097 ceph-mon[32853]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T01:44:49.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:49 smithi097 ceph-mon[32853]: from='osd.8 [v2:172.21.15.173:6816/1334205527,v1:172.21.15.173:6817/1334205527]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:44:49.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:49 smithi097 ceph-mon[32853]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T01:44:49.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:49 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:44:49.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:49 smithi097 ceph-mon[32853]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:44:49.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:49 smithi097 ceph-mon[32853]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:50.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:49 smithi173 ceph-mon[32553]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T01:44:50.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:49 smithi173 ceph-mon[32553]: from='osd.8 [v2:172.21.15.173:6816/1334205527,v1:172.21.15.173:6817/1334205527]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:44:50.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:49 smithi173 ceph-mon[32553]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T01:44:50.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:49 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:44:50.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:49 smithi173 ceph-mon[32553]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:44:50.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:49 smithi173 ceph-mon[32553]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:50.877 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:50 smithi001 ceph-mon[27822]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-26T01:44:50.877 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:50 smithi001 ceph-mon[27822]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T01:44:50.877 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:50 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:44:50.877 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:50 smithi001 ceph-mon[27822]: from='osd.8 ' entity='osd.8' 2024-04-26T01:44:50.956 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:50 smithi173 ceph-mon[32553]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-26T01:44:50.957 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:50 smithi173 ceph-mon[32553]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T01:44:50.957 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:50 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:44:50.957 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:50 smithi173 ceph-mon[32553]: from='osd.8 ' entity='osd.8' 2024-04-26T01:44:50.957 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 01:44:50 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T01:44:50.605+0000 7fc178cfe700 -1 osd.8 0 waiting for initial osdmap 2024-04-26T01:44:50.957 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 01:44:50 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T01:44:50.755+0000 7fc170ae9700 -1 osd.8 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:44:50.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:50 smithi097 ceph-mon[32853]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-26T01:44:50.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:50 smithi097 ceph-mon[32853]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T01:44:50.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:50 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:44:50.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:50 smithi097 ceph-mon[32853]: from='osd.8 ' entity='osd.8' 2024-04-26T01:44:51.496 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T01:44:51.496 INFO:teuthology.orchestra.run.smithi001.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T01:44:51.496 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10+0 records in 2024-04-26T01:44:51.496 INFO:teuthology.orchestra.run.smithi001.stderr:10+0 records out 2024-04-26T01:44:51.497 INFO:teuthology.orchestra.run.smithi001.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0362301 s, 289 MB/s 2024-04-26T01:44:51.497 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 2024-04-26T01:44:51.497 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping successful for: 2024-04-26T01:44:51.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:51 smithi001 ceph-mon[27822]: purged_snaps scrub starts 2024-04-26T01:44:51.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:51 smithi001 ceph-mon[27822]: purged_snaps scrub ok 2024-04-26T01:44:51.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:51 smithi001 ceph-mon[27822]: osdmap e60: 9 total, 8 up, 9 in 2024-04-26T01:44:51.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:51 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:44:51.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:51 smithi001 ceph-mon[27822]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:51.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:51 smithi001 ceph-mon[27822]: Detected new or changed devices on smithi173 2024-04-26T01:44:51.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:51 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:51.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:51 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:51.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:51 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:51.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:51 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:51.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:51 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:51.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:51 smithi001 ceph-mon[27822]: Adjusting osd_memory_target on smithi173 to 5696M 2024-04-26T01:44:51.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:51 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:51.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:51 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:51.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:51 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:51.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:51 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:51.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:51 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:44:51.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:51 smithi097 ceph-mon[32853]: purged_snaps scrub starts 2024-04-26T01:44:51.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:51 smithi097 ceph-mon[32853]: purged_snaps scrub ok 2024-04-26T01:44:51.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:51 smithi097 ceph-mon[32853]: osdmap e60: 9 total, 8 up, 9 in 2024-04-26T01:44:51.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:51 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:44:51.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:51 smithi097 ceph-mon[32853]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:51.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:51 smithi097 ceph-mon[32853]: Detected new or changed devices on smithi173 2024-04-26T01:44:51.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:51 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:51.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:51 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:51.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:51 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:51.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:51 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:51.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:51 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:51.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:51 smithi097 ceph-mon[32853]: Adjusting osd_memory_target on smithi173 to 5696M 2024-04-26T01:44:51.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:51 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:51.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:51 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:51.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:51 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:51.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:51 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:51.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:51 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:44:52.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:51 smithi173 ceph-mon[32553]: purged_snaps scrub starts 2024-04-26T01:44:52.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:51 smithi173 ceph-mon[32553]: purged_snaps scrub ok 2024-04-26T01:44:52.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:51 smithi173 ceph-mon[32553]: osdmap e60: 9 total, 8 up, 9 in 2024-04-26T01:44:52.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:51 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:44:52.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:51 smithi173 ceph-mon[32553]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:44:52.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:51 smithi173 ceph-mon[32553]: Detected new or changed devices on smithi173 2024-04-26T01:44:52.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:51 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:52.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:51 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:52.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:51 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:52.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:51 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:52.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:51 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:44:52.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:51 smithi173 ceph-mon[32553]: Adjusting osd_memory_target on smithi173 to 5696M 2024-04-26T01:44:52.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:51 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:52.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:51 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:52.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:51 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:44:52.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:51 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:44:52.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:51 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:44:52.735 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi001:vg_nvme/lv_1 2024-04-26T01:44:53.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:52 smithi173 ceph-mon[32553]: osd.8 [v2:172.21.15.173:6816/1334205527,v1:172.21.15.173:6817/1334205527] boot 2024-04-26T01:44:53.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:52 smithi173 ceph-mon[32553]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T01:44:53.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:44:53.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:52 smithi001 ceph-mon[27822]: osd.8 [v2:172.21.15.173:6816/1334205527,v1:172.21.15.173:6817/1334205527] boot 2024-04-26T01:44:53.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:52 smithi001 ceph-mon[27822]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T01:44:53.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:44:53.182 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:44:53.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:52 smithi097 ceph-mon[32853]: osd.8 [v2:172.21.15.173:6816/1334205527,v1:172.21.15.173:6817/1334205527] boot 2024-04-26T01:44:53.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:52 smithi097 ceph-mon[32853]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T01:44:53.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:44:53.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:53 smithi001 ceph-mon[27822]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T01:44:53.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:53 smithi001 ceph-mon[27822]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:44:53.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:53 smithi097 ceph-mon[32853]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T01:44:53.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:53 smithi097 ceph-mon[32853]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:44:54.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:53 smithi173 ceph-mon[32553]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T01:44:54.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:53 smithi173 ceph-mon[32553]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:44:54.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:54 smithi001 ceph-mon[27822]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T01:44:55.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:54 smithi173 ceph-mon[32553]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T01:44:55.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:54 smithi097 ceph-mon[32853]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T01:44:55.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.389+0000 7f75e0e00700 1 -- 172.21.15.1:0/779281731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75dc1005e0 msgr2=0x7f75dc100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:55.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.389+0000 7f75e0e00700 1 --2- 172.21.15.1:0/779281731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75dc1005e0 0x7f75dc100a00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0098f0 tx=0x7f75cc009c00 comp rx=0 tx=0).stop 2024-04-26T01:44:55.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.390+0000 7f75e0e00700 1 -- 172.21.15.1:0/779281731 shutdown_connections 2024-04-26T01:44:55.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.390+0000 7f75e0e00700 1 --2- 172.21.15.1:0/779281731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75dc1086f0 0x7f75dc10aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:55.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.390+0000 7f75e0e00700 1 --2- 172.21.15.1:0/779281731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75dc1005e0 0x7f75dc100a00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:55.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.390+0000 7f75e0e00700 1 --2- 172.21.15.1:0/779281731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75dc100ff0 0x7f75dc1081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:55.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.390+0000 7f75e0e00700 1 -- 172.21.15.1:0/779281731 >> 172.21.15.1:0/779281731 conn(0x7f75dc0fc1c0 msgr2=0x7f75dc0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:55.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.390+0000 7f75e0e00700 1 -- 172.21.15.1:0/779281731 shutdown_connections 2024-04-26T01:44:55.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.390+0000 7f75e0e00700 1 -- 172.21.15.1:0/779281731 wait complete. 2024-04-26T01:44:55.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.391+0000 7f75e0e00700 1 Processor -- start 2024-04-26T01:44:55.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.391+0000 7f75e0e00700 1 -- start start 2024-04-26T01:44:55.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.392+0000 7f75e0e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75dc1005e0 0x7f75dc19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:55.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.392+0000 7f75e0e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75dc100ff0 0x7f75dc19b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:55.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.392+0000 7f75e0e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75dc1086f0 0x7f75dc1a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:55.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.392+0000 7f75e0e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc10e440 con 0x7f75dc1005e0 2024-04-26T01:44:55.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.392+0000 7f75e0e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc10e2c0 con 0x7f75dc100ff0 2024-04-26T01:44:55.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.392+0000 7f75e0e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc10e5c0 con 0x7f75dc1086f0 2024-04-26T01:44:55.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.392+0000 7f75db400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75dc1005e0 0x7f75dc19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:55.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.392+0000 7f75db400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75dc1005e0 0x7f75dc19ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53610/0 (socket says 172.21.15.1:53610) 2024-04-26T01:44:55.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.392+0000 7f75db400700 1 -- 172.21.15.1:0/3553938563 learned_addr learned my addr 172.21.15.1:0/3553938563 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:44:55.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.392+0000 7f75daa00700 1 --2- 172.21.15.1:0/3553938563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75dc100ff0 0x7f75dc19b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:55.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.392+0000 7f75dbe00700 1 --2- 172.21.15.1:0/3553938563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75dc1086f0 0x7f75dc1a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:55.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.393+0000 7f75db400700 1 -- 172.21.15.1:0/3553938563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75dc1086f0 msgr2=0x7f75dc1a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:55.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.393+0000 7f75db400700 1 --2- 172.21.15.1:0/3553938563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75dc1086f0 0x7f75dc1a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:55.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.393+0000 7f75db400700 1 -- 172.21.15.1:0/3553938563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75dc100ff0 msgr2=0x7f75dc19b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:55.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.393+0000 7f75db400700 1 --2- 172.21.15.1:0/3553938563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75dc100ff0 0x7f75dc19b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:55.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.393+0000 7f75db400700 1 -- 172.21.15.1:0/3553938563 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75dc1a29b0 con 0x7f75dc1005e0 2024-04-26T01:44:55.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.393+0000 7f75db400700 1 --2- 172.21.15.1:0/3553938563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75dc1005e0 0x7f75dc19ad50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00fc20 tx=0x7f75cc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:55.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.393+0000 7f75d3e00700 1 -- 172.21.15.1:0/3553938563 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc025070 con 0x7f75dc1005e0 2024-04-26T01:44:55.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.393+0000 7f75e0e00700 1 -- 172.21.15.1:0/3553938563 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75dc1a2c40 con 0x7f75dc1005e0 2024-04-26T01:44:55.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.394+0000 7f75d3e00700 1 -- 172.21.15.1:0/3553938563 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75cc005670 con 0x7f75dc1005e0 2024-04-26T01:44:55.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.394+0000 7f75d3e00700 1 -- 172.21.15.1:0/3553938563 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc003c40 con 0x7f75dc1005e0 2024-04-26T01:44:55.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.394+0000 7f75e0e00700 1 -- 172.21.15.1:0/3553938563 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75dc1a30f0 con 0x7f75dc1005e0 2024-04-26T01:44:55.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.395+0000 7f75e0e00700 1 -- 172.21.15.1:0/3553938563 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75dc069730 con 0x7f75dc1005e0 2024-04-26T01:44:55.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.397+0000 7f75d3e00700 1 -- 172.21.15.1:0/3553938563 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f75cc028070 con 0x7f75dc1005e0 2024-04-26T01:44:55.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.398+0000 7f75d3e00700 1 --2- 172.21.15.1:0/3553938563 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f75c00a3cc0 0x7f75c00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:55.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.399+0000 7f75d3e00700 1 -- 172.21.15.1:0/3553938563 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5829+0+0 (secure 0 0 0) 0x7f75cc0d0650 con 0x7f75dc1005e0 2024-04-26T01:44:55.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.399+0000 7f75daa00700 1 --2- 172.21.15.1:0/3553938563 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f75c00a3cc0 0x7f75c00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:55.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.400+0000 7f75daa00700 1 --2- 172.21.15.1:0/3553938563 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f75c00a3cc0 0x7f75c00a6180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009c60 tx=0x7f75c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:55.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.402+0000 7f75d3e00700 1 -- 172.21.15.1:0/3553938563 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f75cc02f020 con 0x7f75dc1005e0 2024-04-26T01:44:55.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:44:55.645+0000 7f75e0e00700 1 -- 172.21.15.1:0/3553938563 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f75dc103290 con 0x7f75c00a3cc0 2024-04-26T01:44:56.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:55 smithi173 ceph-mon[32553]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:44:56.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:55 smithi173 ceph-mon[32553]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T01:44:56.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:55 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:44:56.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:55 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:44:56.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:55 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:56.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:55 smithi001 ceph-mon[27822]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:44:56.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:55 smithi001 ceph-mon[27822]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T01:44:56.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:55 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:44:56.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:55 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:44:56.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:55 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:56.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:55 smithi097 ceph-mon[32853]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:44:56.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:55 smithi097 ceph-mon[32853]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T01:44:56.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:55 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:44:56.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:55 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:44:56.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:55 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:56.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:56 smithi001 ceph-mon[27822]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:44:56.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:56 smithi001 ceph-mon[27822]: osdmap e65: 9 total, 9 up, 9 in 2024-04-26T01:44:57.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:56 smithi173 ceph-mon[32553]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:44:57.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:56 smithi173 ceph-mon[32553]: osdmap e65: 9 total, 9 up, 9 in 2024-04-26T01:44:57.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:56 smithi097 ceph-mon[32853]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:44:57.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:56 smithi097 ceph-mon[32853]: osdmap e65: 9 total, 9 up, 9 in 2024-04-26T01:44:57.976 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:57 smithi001 ceph-mon[27822]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T01:44:58.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:57 smithi173 ceph-mon[32553]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T01:44:58.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:57 smithi097 ceph-mon[32853]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T01:45:00.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:59 smithi173 ceph-mon[32553]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:45:00.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:59 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/2202806890' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbbb163b-f8c4-4be5-8f80-82445e8d407f"}]: dispatch 2024-04-26T01:45:00.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:59 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/2202806890' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbbb163b-f8c4-4be5-8f80-82445e8d407f"}]': finished 2024-04-26T01:45:00.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:59 smithi173 ceph-mon[32553]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T01:45:00.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:44:59 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:45:00.085 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:59 smithi001 ceph-mon[27822]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:45:00.085 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:59 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2202806890' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbbb163b-f8c4-4be5-8f80-82445e8d407f"}]: dispatch 2024-04-26T01:45:00.085 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:59 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2202806890' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbbb163b-f8c4-4be5-8f80-82445e8d407f"}]': finished 2024-04-26T01:45:00.085 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:59 smithi001 ceph-mon[27822]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T01:45:00.085 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:44:59 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:45:00.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:59 smithi097 ceph-mon[32853]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:45:00.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:59 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/2202806890' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbbb163b-f8c4-4be5-8f80-82445e8d407f"}]: dispatch 2024-04-26T01:45:00.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:59 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/2202806890' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbbb163b-f8c4-4be5-8f80-82445e8d407f"}]': finished 2024-04-26T01:45:00.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:59 smithi097 ceph-mon[32853]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T01:45:00.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:44:59 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:45:01.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:00 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/2238187485' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:45:01.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:00 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2238187485' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:45:01.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:00 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/2238187485' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:45:02.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:01 smithi173 ceph-mon[32553]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:45:02.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:01 smithi001 ceph-mon[27822]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:45:02.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:01 smithi097 ceph-mon[32853]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:45:03.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:03 smithi001 ceph-mon[27822]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:45:04.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:03 smithi173 ceph-mon[32553]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:45:04.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:03 smithi097 ceph-mon[32853]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:45:06.026 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:05 smithi001 ceph-mon[27822]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-26T01:45:06.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:05 smithi173 ceph-mon[32553]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-26T01:45:06.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:05 smithi097 ceph-mon[32853]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-26T01:45:07.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:07 smithi001 ceph-mon[27822]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:08.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:07 smithi173 ceph-mon[32553]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:08.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:07 smithi097 ceph-mon[32853]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:10.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:09 smithi173 ceph-mon[32553]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:10.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:09 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T01:45:10.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:09 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:10.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:09 smithi173 ceph-mon[32553]: Deploying daemon osd.9 on smithi001 2024-04-26T01:45:10.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:09 smithi001 ceph-mon[27822]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:10.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:09 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T01:45:10.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:09 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:10.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:09 smithi001 ceph-mon[27822]: Deploying daemon osd.9 on smithi001 2024-04-26T01:45:10.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:09 smithi097 ceph-mon[32853]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:10.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:09 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T01:45:10.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:09 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:10.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:09 smithi097 ceph-mon[32853]: Deploying daemon osd.9 on smithi001 2024-04-26T01:45:11.980 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:11 smithi001 ceph-mon[27822]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:12.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:11 smithi173 ceph-mon[32553]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:12.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:11 smithi097 ceph-mon[32853]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:14.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:13 smithi001 ceph-mon[27822]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:14.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:13 smithi097 ceph-mon[32853]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:14.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:13 smithi173 ceph-mon[32553]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:16.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:15 smithi001 ceph-mon[27822]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:16.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:15 smithi097 ceph-mon[32853]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:16.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:15 smithi173 ceph-mon[32553]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:18 smithi001 ceph-mon[27822]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:18 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:18 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:18 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:18.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:18 smithi097 ceph-mon[32853]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:18.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:18 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:18.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:18 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:18.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:18 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:19.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:18 smithi173 ceph-mon[32553]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:19.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:18 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:19.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:18 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:19.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:18 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:20.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:20 smithi001 ceph-mon[27822]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:20.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:20 smithi097 ceph-mon[32853]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:21.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:20 smithi173 ceph-mon[32553]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:22 smithi097 ceph-mon[32853]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:22 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:22 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:22 smithi097 ceph-mon[32853]: from='osd.9 [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T01:45:22.561 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:22 smithi001 ceph-mon[27822]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:22.561 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:22 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:22.561 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:22 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:22.561 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:22 smithi001 ceph-mon[27822]: from='osd.9 [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T01:45:22.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:22 smithi173 ceph-mon[32553]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:22.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:22 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:22.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:22 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:22.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:22 smithi173 ceph-mon[32553]: from='osd.9 [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T01:45:22.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:45:22.661+0000 7f75d3e00700 1 -- 172.21.15.1:0/3553938563 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f75dc103290 con 0x7f75c00a3cc0 2024-04-26T01:45:22.659 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 9 on host 'smithi001' 2024-04-26T01:45:22.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:45:22.664+0000 7f75e0e00700 1 -- 172.21.15.1:0/3553938563 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f75c00a3cc0 msgr2=0x7f75c00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:22.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:45:22.664+0000 7f75e0e00700 1 --2- 172.21.15.1:0/3553938563 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f75c00a3cc0 0x7f75c00a6180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009c60 tx=0x7f75c8009260 comp rx=0 tx=0).stop 2024-04-26T01:45:22.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:45:22.664+0000 7f75e0e00700 1 -- 172.21.15.1:0/3553938563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75dc1005e0 msgr2=0x7f75dc19ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:22.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:45:22.664+0000 7f75e0e00700 1 --2- 172.21.15.1:0/3553938563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75dc1005e0 0x7f75dc19ad50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00fc20 tx=0x7f75cc005270 comp rx=0 tx=0).stop 2024-04-26T01:45:22.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:45:22.664+0000 7f75e0e00700 1 -- 172.21.15.1:0/3553938563 shutdown_connections 2024-04-26T01:45:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:45:22.664+0000 7f75e0e00700 1 --2- 172.21.15.1:0/3553938563 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f75c00a3cc0 0x7f75c00a6180 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:45:22.664+0000 7f75e0e00700 1 --2- 172.21.15.1:0/3553938563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75dc1086f0 0x7f75dc1a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:45:22.664+0000 7f75e0e00700 1 --2- 172.21.15.1:0/3553938563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75dc1005e0 0x7f75dc19ad50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:45:22.664+0000 7f75e0e00700 1 --2- 172.21.15.1:0/3553938563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75dc100ff0 0x7f75dc19b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:45:22.665+0000 7f75e0e00700 1 -- 172.21.15.1:0/3553938563 >> 172.21.15.1:0/3553938563 conn(0x7f75dc0fc1c0 msgr2=0x7f75dc0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:45:22.665+0000 7f75e0e00700 1 -- 172.21.15.1:0/3553938563 shutdown_connections 2024-04-26T01:45:22.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:45:22.665+0000 7f75e0e00700 1 -- 172.21.15.1:0/3553938563 wait complete. 2024-04-26T01:45:23.242 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:23 smithi001 ceph-mon[27822]: from='osd.9 [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T01:45:23.243 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:23 smithi001 ceph-mon[27822]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T01:45:23.243 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:23 smithi001 ceph-mon[27822]: from='osd.9 [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-04-26T01:45:23.243 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:23 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:45:23.243 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:23 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:23.243 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:23 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:23.243 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:23 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:23.243 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:23 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:23.243 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:23 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:23.243 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:23 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:23.371 DEBUG:teuthology.orchestra.run.smithi001:osd.9> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.9.service 2024-04-26T01:45:23.379 INFO:tasks.cephadm:Deploying osd.10 on smithi097 with /dev/vg_nvme/lv_1... 2024-04-26T01:45:23.379 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T01:45:23.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:23 smithi097 ceph-mon[32853]: from='osd.9 [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T01:45:23.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:23 smithi097 ceph-mon[32853]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T01:45:23.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:23 smithi097 ceph-mon[32853]: from='osd.9 [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-04-26T01:45:23.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:23 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:45:23.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:23 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:23.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:23 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:23.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:23 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:23.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:23 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:23.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:23 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:23.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:23 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:23.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:23 smithi173 ceph-mon[32553]: from='osd.9 [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T01:45:23.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:23 smithi173 ceph-mon[32553]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T01:45:23.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:23 smithi173 ceph-mon[32553]: from='osd.9 [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-04-26T01:45:23.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:23 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:45:23.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:23 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:23.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:23 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:23.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:23 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:23.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:23 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:23.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:23 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:23.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:23 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:23.618 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:45:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:24 smithi097 ceph-mon[32853]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:24 smithi097 ceph-mon[32853]: from='osd.9 [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-04-26T01:45:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:24 smithi097 ceph-mon[32853]: osdmap e68: 10 total, 9 up, 10 in 2024-04-26T01:45:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:45:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:24 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:45:24.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:24 smithi173 ceph-mon[32553]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:24.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:24 smithi173 ceph-mon[32553]: from='osd.9 [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-04-26T01:45:24.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:24 smithi173 ceph-mon[32553]: osdmap e68: 10 total, 9 up, 10 in 2024-04-26T01:45:24.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:24 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:45:24.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:24 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:45:24.570 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:24 smithi001 ceph-mon[27822]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:45:24.570 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:24 smithi001 ceph-mon[27822]: from='osd.9 [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-04-26T01:45:24.570 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:24 smithi001 ceph-mon[27822]: osdmap e68: 10 total, 9 up, 10 in 2024-04-26T01:45:24.570 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:24 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:45:24.570 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:24 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:45:25.319 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:25 smithi097 ceph-mon[32853]: purged_snaps scrub starts 2024-04-26T01:45:25.320 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:25 smithi097 ceph-mon[32853]: purged_snaps scrub ok 2024-04-26T01:45:25.320 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:25 smithi097 ceph-mon[32853]: osd.9 [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839] boot 2024-04-26T01:45:25.320 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:25 smithi097 ceph-mon[32853]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T01:45:25.320 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:25 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:45:25.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:25 smithi001 ceph-mon[27822]: purged_snaps scrub starts 2024-04-26T01:45:25.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:25 smithi001 ceph-mon[27822]: purged_snaps scrub ok 2024-04-26T01:45:25.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:25 smithi001 ceph-mon[27822]: osd.9 [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839] boot 2024-04-26T01:45:25.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:25 smithi001 ceph-mon[27822]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T01:45:25.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:25 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:45:25.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:25 smithi173 ceph-mon[32553]: purged_snaps scrub starts 2024-04-26T01:45:25.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:25 smithi173 ceph-mon[32553]: purged_snaps scrub ok 2024-04-26T01:45:25.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:25 smithi173 ceph-mon[32553]: osd.9 [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839] boot 2024-04-26T01:45:25.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:25 smithi173 ceph-mon[32553]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T01:45:25.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:25 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:45:25.694 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T01:45:25.694 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T01:45:25.695 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-26T01:45:25.695 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-26T01:45:25.695 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.013447 s, 780 MB/s 2024-04-26T01:45:25.695 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-26T01:45:26.340 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:26 smithi097 ceph-mon[32853]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:26.340 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:26 smithi097 ceph-mon[32853]: osdmap e70: 10 total, 10 up, 10 in 2024-04-26T01:45:26.371 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_1 2024-04-26T01:45:26.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:26 smithi173 ceph-mon[32553]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:26.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:26 smithi173 ceph-mon[32553]: osdmap e70: 10 total, 10 up, 10 in 2024-04-26T01:45:26.608 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:45:26.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:26 smithi001 ceph-mon[27822]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:26.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:26 smithi001 ceph-mon[27822]: osdmap e70: 10 total, 10 up, 10 in 2024-04-26T01:45:28.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.239+0000 7fe746f69700 1 -- 172.21.15.97:0/542343745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7401019b0 msgr2=0x7fe740101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:28.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.239+0000 7fe746f69700 1 --2- 172.21.15.97:0/542343745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7401019b0 0x7fe740101dd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe7340099c0 tx=0x7fe734009cd0 comp rx=0 tx=0).stop 2024-04-26T01:45:28.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.239+0000 7fe746f69700 1 -- 172.21.15.97:0/542343745 shutdown_connections 2024-04-26T01:45:28.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.239+0000 7fe746f69700 1 --2- 172.21.15.97:0/542343745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe740103550 0x7fe740109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:28.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.239+0000 7fe746f69700 1 --2- 172.21.15.97:0/542343745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7401019b0 0x7fe740101dd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:28.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.239+0000 7fe746f69700 1 --2- 172.21.15.97:0/542343745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe740102bb0 0x7fe740103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:28.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.239+0000 7fe746f69700 1 -- 172.21.15.97:0/542343745 >> 172.21.15.97:0/542343745 conn(0x7fe7400fcf70 msgr2=0x7fe7400ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:28.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.239+0000 7fe746f69700 1 -- 172.21.15.97:0/542343745 shutdown_connections 2024-04-26T01:45:28.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.239+0000 7fe746f69700 1 -- 172.21.15.97:0/542343745 wait complete. 2024-04-26T01:45:28.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.240+0000 7fe746f69700 1 Processor -- start 2024-04-26T01:45:28.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.240+0000 7fe746f69700 1 -- start start 2024-04-26T01:45:28.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.240+0000 7fe746f69700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7401019b0 0x7fe74006bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:28.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.240+0000 7fe746f69700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe740102bb0 0x7fe74019fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:28.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.240+0000 7fe746f69700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe740103550 0x7fe7401a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:28.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.240+0000 7fe746f69700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe74010d4a0 con 0x7fe740103550 2024-04-26T01:45:28.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.240+0000 7fe746f69700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe74010d320 con 0x7fe740102bb0 2024-04-26T01:45:28.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.240+0000 7fe746f69700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe74010d620 con 0x7fe7401019b0 2024-04-26T01:45:28.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.240+0000 7fe73ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe740102bb0 0x7fe74019fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:28.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.240+0000 7fe73ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe740102bb0 0x7fe74019fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53436/0 (socket says 172.21.15.97:53436) 2024-04-26T01:45:28.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.240+0000 7fe73ffff700 1 -- 172.21.15.97:0/3538095296 learned_addr learned my addr 172.21.15.97:0/3538095296 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-26T01:45:28.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.240+0000 7fe744d05700 1 --2- 172.21.15.97:0/3538095296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7401019b0 0x7fe74006bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:28.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.240+0000 7fe73ffff700 1 -- 172.21.15.97:0/3538095296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7401019b0 msgr2=0x7fe74006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:28.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.240+0000 7fe73ffff700 1 --2- 172.21.15.97:0/3538095296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7401019b0 0x7fe74006bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:28.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.240+0000 7fe73ffff700 1 -- 172.21.15.97:0/3538095296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe740103550 msgr2=0x7fe7401a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:28.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.240+0000 7fe73ffff700 1 --2- 172.21.15.97:0/3538095296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe740103550 0x7fe7401a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:28.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.240+0000 7fe73ffff700 1 -- 172.21.15.97:0/3538095296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7401a2b60 con 0x7fe740102bb0 2024-04-26T01:45:28.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.240+0000 7fe73ffff700 1 --2- 172.21.15.97:0/3538095296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe740102bb0 0x7fe74019fec0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00b530 tx=0x7fe72c00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:28.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.241+0000 7fe73dffb700 1 -- 172.21.15.97:0/3538095296 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c015020 con 0x7fe740102bb0 2024-04-26T01:45:28.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.241+0000 7fe73dffb700 1 -- 172.21.15.97:0/3538095296 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe72c011ce0 con 0x7fe740102bb0 2024-04-26T01:45:28.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.241+0000 7fe73dffb700 1 -- 172.21.15.97:0/3538095296 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c010d20 con 0x7fe740102bb0 2024-04-26T01:45:28.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.241+0000 7fe746f69700 1 -- 172.21.15.97:0/3538095296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7401a2e50 con 0x7fe740102bb0 2024-04-26T01:45:28.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.241+0000 7fe746f69700 1 -- 172.21.15.97:0/3538095296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7401a3350 con 0x7fe740102bb0 2024-04-26T01:45:28.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.242+0000 7fe746f69700 1 -- 172.21.15.97:0/3538095296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe740069730 con 0x7fe740102bb0 2024-04-26T01:45:28.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.243+0000 7fe73dffb700 1 -- 172.21.15.97:0/3538095296 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fe72c018020 con 0x7fe740102bb0 2024-04-26T01:45:28.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.243+0000 7fe73dffb700 1 --2- 172.21.15.97:0/3538095296 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe7300a3cc0 0x7fe7300a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:28.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.243+0000 7fe73dffb700 1 -- 172.21.15.97:0/3538095296 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6261+0+0 (secure 0 0 0) 0x7fe72c0cf390 con 0x7fe740102bb0 2024-04-26T01:45:28.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.244+0000 7fe744d05700 1 --2- 172.21.15.97:0/3538095296 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe7300a3cc0 0x7fe7300a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:28.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.245+0000 7fe744d05700 1 --2- 172.21.15.97:0/3538095296 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe7300a3cc0 0x7fe7300a6180 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe734000c00 tx=0x7fe734005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:28.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.248+0000 7fe73dffb700 1 -- 172.21.15.97:0/3538095296 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe72c01f020 con 0x7fe740102bb0 2024-04-26T01:45:28.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:28.457+0000 7fe746f69700 1 -- 172.21.15.97:0/3538095296 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe740062e80 con 0x7fe7300a3cc0 2024-04-26T01:45:28.455 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:28 smithi097 ceph-mon[32853]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:28.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:28 smithi173 ceph-mon[32553]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:28.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:28 smithi001 ceph-mon[27822]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:29.377 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:29 smithi097 ceph-mon[32853]: Detected new or changed devices on smithi001 2024-04-26T01:45:29.378 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:29 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:29.378 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:29 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:29.378 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:29 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:29.378 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:29 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:29.378 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:29 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:29.378 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:29 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:29.378 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:29 smithi097 ceph-mon[32853]: Adjusting osd_memory_target on smithi001 to 4273M 2024-04-26T01:45:29.378 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:29 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:29.378 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:29 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:29.378 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:29 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:29.378 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:29 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:29.379 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:29 smithi097 ceph-mon[32853]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:45:29.379 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:29 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:45:29.379 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:29 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:45:29.379 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:29 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:29.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:29 smithi001 ceph-mon[27822]: Detected new or changed devices on smithi001 2024-04-26T01:45:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:29 smithi001 ceph-mon[27822]: Adjusting osd_memory_target on smithi001 to 4273M 2024-04-26T01:45:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:29 smithi001 ceph-mon[27822]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:45:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:45:29.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:45:29.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:29.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:29 smithi173 ceph-mon[32553]: Detected new or changed devices on smithi001 2024-04-26T01:45:29.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:29 smithi173 ceph-mon[32553]: Adjusting osd_memory_target on smithi001 to 4273M 2024-04-26T01:45:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:29 smithi173 ceph-mon[32553]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:45:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:45:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:45:29.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:30.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:30 smithi097 ceph-mon[32853]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:30.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:30 smithi173 ceph-mon[32553]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:30.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:30 smithi001 ceph-mon[27822]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:31.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:31 smithi097 ceph-mon[32853]: from='client.? 172.21.15.97:0/2857149463' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b447b34-0600-4727-826d-b3d68adfb18d"}]: dispatch 2024-04-26T01:45:31.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:31 smithi097 ceph-mon[32853]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b447b34-0600-4727-826d-b3d68adfb18d"}]: dispatch 2024-04-26T01:45:31.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:31 smithi097 ceph-mon[32853]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b447b34-0600-4727-826d-b3d68adfb18d"}]': finished 2024-04-26T01:45:31.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:31 smithi097 ceph-mon[32853]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T01:45:31.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:45:31.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:31 smithi173 ceph-mon[32553]: from='client.? 172.21.15.97:0/2857149463' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b447b34-0600-4727-826d-b3d68adfb18d"}]: dispatch 2024-04-26T01:45:31.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:31 smithi173 ceph-mon[32553]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b447b34-0600-4727-826d-b3d68adfb18d"}]: dispatch 2024-04-26T01:45:31.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:31 smithi173 ceph-mon[32553]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b447b34-0600-4727-826d-b3d68adfb18d"}]': finished 2024-04-26T01:45:31.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:31 smithi173 ceph-mon[32553]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T01:45:31.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:31 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:45:31.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:31 smithi001 ceph-mon[27822]: from='client.? 172.21.15.97:0/2857149463' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b447b34-0600-4727-826d-b3d68adfb18d"}]: dispatch 2024-04-26T01:45:31.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:31 smithi001 ceph-mon[27822]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b447b34-0600-4727-826d-b3d68adfb18d"}]: dispatch 2024-04-26T01:45:31.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:31 smithi001 ceph-mon[27822]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b447b34-0600-4727-826d-b3d68adfb18d"}]': finished 2024-04-26T01:45:31.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:31 smithi001 ceph-mon[27822]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T01:45:31.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:31 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:45:32.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:32 smithi097 ceph-mon[32853]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:32.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:32 smithi097 ceph-mon[32853]: from='client.? 172.21.15.97:0/1647202133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:45:32.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:32 smithi173 ceph-mon[32553]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:32.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:32 smithi173 ceph-mon[32553]: from='client.? 172.21.15.97:0/1647202133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:45:32.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:32 smithi001 ceph-mon[27822]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:32.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:32 smithi001 ceph-mon[27822]: from='client.? 172.21.15.97:0/1647202133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:45:34.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:34 smithi097 ceph-mon[32853]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:34.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:34 smithi173 ceph-mon[32553]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:34.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:34 smithi001 ceph-mon[27822]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:36.663 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:36 smithi097 ceph-mon[32853]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:36.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:36 smithi173 ceph-mon[32553]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:36.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:36 smithi001 ceph-mon[27822]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:38.618 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:38 smithi097 ceph-mon[32853]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:38.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:38 smithi173 ceph-mon[32553]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:38.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:38 smithi001 ceph-mon[27822]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:39.721 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T01:45:39.722 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:39.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T01:45:39.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:39.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T01:45:39.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:40.706 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:40 smithi097 ceph-mon[32853]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:40.706 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:40 smithi097 ceph-mon[32853]: Deploying daemon osd.10 on smithi097 2024-04-26T01:45:40.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:40 smithi173 ceph-mon[32553]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:40.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:40 smithi173 ceph-mon[32553]: Deploying daemon osd.10 on smithi097 2024-04-26T01:45:40.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:40 smithi001 ceph-mon[27822]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:40.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:40 smithi001 ceph-mon[27822]: Deploying daemon osd.10 on smithi097 2024-04-26T01:45:42.534 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:42 smithi097 ceph-mon[32853]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:42.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:42 smithi173 ceph-mon[32553]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:42.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:42 smithi001 ceph-mon[27822]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:44.561 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:44 smithi097 ceph-mon[32853]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:44.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:44 smithi173 ceph-mon[32553]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:44.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:44 smithi001 ceph-mon[27822]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:45.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:45 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:45.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:45 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:45.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:45 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:45.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:45 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:45.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:45 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:45.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:45 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:46.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:45 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:46.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:45 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:46.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:45 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:46.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:46 smithi001 ceph-mon[27822]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:46.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:46 smithi097 ceph-mon[32853]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:47.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:46 smithi173 ceph-mon[32553]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:48.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:48.399+0000 7fe73dffb700 1 -- 172.21.15.97:0/3538095296 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe740062e80 con 0x7fe7300a3cc0 2024-04-26T01:45:48.397 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 10 on host 'smithi097' 2024-04-26T01:45:48.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:48.401+0000 7fe746f69700 1 -- 172.21.15.97:0/3538095296 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe7300a3cc0 msgr2=0x7fe7300a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:48.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:48.401+0000 7fe746f69700 1 --2- 172.21.15.97:0/3538095296 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe7300a3cc0 0x7fe7300a6180 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe734000c00 tx=0x7fe734005fb0 comp rx=0 tx=0).stop 2024-04-26T01:45:48.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:48.401+0000 7fe746f69700 1 -- 172.21.15.97:0/3538095296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe740102bb0 msgr2=0x7fe74019fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:48.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:48.401+0000 7fe746f69700 1 --2- 172.21.15.97:0/3538095296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe740102bb0 0x7fe74019fec0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00b530 tx=0x7fe72c00b8f0 comp rx=0 tx=0).stop 2024-04-26T01:45:48.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:48.402+0000 7fe746f69700 1 -- 172.21.15.97:0/3538095296 shutdown_connections 2024-04-26T01:45:48.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:48.402+0000 7fe746f69700 1 --2- 172.21.15.97:0/3538095296 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe7300a3cc0 0x7fe7300a6180 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:48.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:48.402+0000 7fe746f69700 1 --2- 172.21.15.97:0/3538095296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe740103550 0x7fe7401a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:48.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:48.402+0000 7fe746f69700 1 --2- 172.21.15.97:0/3538095296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7401019b0 0x7fe74006bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:48.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:48.402+0000 7fe746f69700 1 --2- 172.21.15.97:0/3538095296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe740102bb0 0x7fe74019fec0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:48.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:48.402+0000 7fe746f69700 1 -- 172.21.15.97:0/3538095296 >> 172.21.15.97:0/3538095296 conn(0x7fe7400fcf70 msgr2=0x7fe740106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:48.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:48.402+0000 7fe746f69700 1 -- 172.21.15.97:0/3538095296 shutdown_connections 2024-04-26T01:45:48.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-26T01:45:48.402+0000 7fe746f69700 1 -- 172.21.15.97:0/3538095296 wait complete. 2024-04-26T01:45:48.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:48 smithi097 ceph-mon[32853]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:48.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:48 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:48.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:48 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:48.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:48 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:48.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:48 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:48.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:48 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:48.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:48 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:48.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:48 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:48.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:48 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:48.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:48 smithi097 ceph-mon[32853]: from='osd.10 [v2:172.21.15.97:6824/1693425268,v1:172.21.15.97:6825/1693425268]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T01:45:48.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:48 smithi097 ceph-mon[32853]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T01:45:49.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:48 smithi173 ceph-mon[32553]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:49.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:48 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:49.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:48 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:49.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:48 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:49.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:48 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:49.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:48 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:49.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:48 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:49.068 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:48 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:49.068 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:48 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:49.068 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:48 smithi173 ceph-mon[32553]: from='osd.10 [v2:172.21.15.97:6824/1693425268,v1:172.21.15.97:6825/1693425268]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T01:45:49.068 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:48 smithi173 ceph-mon[32553]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T01:45:49.076 DEBUG:teuthology.orchestra.run.smithi097:osd.10> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.10.service 2024-04-26T01:45:49.078 INFO:tasks.cephadm:Deploying osd.11 on smithi173 with /dev/vg_nvme/lv_1... 2024-04-26T01:45:49.078 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T01:45:49.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:48 smithi001 ceph-mon[27822]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:49.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:48 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:49.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:48 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:49.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:48 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:49.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:48 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:49.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:48 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:49.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:48 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:49.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:48 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:49.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:48 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:49.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:48 smithi001 ceph-mon[27822]: from='osd.10 [v2:172.21.15.97:6824/1693425268,v1:172.21.15.97:6825/1693425268]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T01:45:49.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:48 smithi001 ceph-mon[27822]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T01:45:49.325 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:45:50.555 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:50 smithi097 ceph-mon[32853]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:50.555 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:50 smithi097 ceph-mon[32853]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T01:45:50.555 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:50 smithi097 ceph-mon[32853]: from='osd.10 [v2:172.21.15.97:6824/1693425268,v1:172.21.15.97:6825/1693425268]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:45:50.555 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:50 smithi097 ceph-mon[32853]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T01:45:50.555 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:50 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:45:50.555 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:50 smithi097 ceph-mon[32853]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:45:50.555 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 01:45:50 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T01:45:50.401+0000 7fdc4deee700 -1 osd.10 0 waiting for initial osdmap 2024-04-26T01:45:50.555 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 01:45:50 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T01:45:50.419+0000 7fdc494e0700 -1 osd.10 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:45:50.597 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:50 smithi173 ceph-mon[32553]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:50.598 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:50 smithi173 ceph-mon[32553]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T01:45:50.598 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:50 smithi173 ceph-mon[32553]: from='osd.10 [v2:172.21.15.97:6824/1693425268,v1:172.21.15.97:6825/1693425268]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:45:50.598 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:50 smithi173 ceph-mon[32553]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T01:45:50.598 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:50 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:45:50.598 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:50 smithi173 ceph-mon[32553]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:45:50.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:50 smithi001 ceph-mon[27822]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:50.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:50 smithi001 ceph-mon[27822]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T01:45:50.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:50 smithi001 ceph-mon[27822]: from='osd.10 [v2:172.21.15.97:6824/1693425268,v1:172.21.15.97:6825/1693425268]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:45:50.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:50 smithi001 ceph-mon[27822]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T01:45:50.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:50 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:45:50.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:50 smithi001 ceph-mon[27822]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-26T01:45:51.196 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T01:45:51.196 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T01:45:51.197 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-04-26T01:45:51.197 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-04-26T01:45:51.197 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113435 s, 924 MB/s 2024-04-26T01:45:51.197 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-04-26T01:45:51.646 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:51 smithi097 ceph-mon[32853]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T01:45:51.646 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:51 smithi097 ceph-mon[32853]: osdmap e73: 11 total, 10 up, 11 in 2024-04-26T01:45:51.646 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:51 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:45:51.646 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:51 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:45:51.646 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:51 smithi097 ceph-mon[32853]: osd.10 [v2:172.21.15.97:6824/1693425268,v1:172.21.15.97:6825/1693425268] boot 2024-04-26T01:45:51.760 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:51 smithi173 ceph-mon[32553]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T01:45:51.761 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:51 smithi173 ceph-mon[32553]: osdmap e73: 11 total, 10 up, 11 in 2024-04-26T01:45:51.761 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:51 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:45:51.761 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:51 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:45:51.761 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:51 smithi173 ceph-mon[32553]: osd.10 [v2:172.21.15.97:6824/1693425268,v1:172.21.15.97:6825/1693425268] boot 2024-04-26T01:45:51.788 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_1 2024-04-26T01:45:51.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:51 smithi001 ceph-mon[27822]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-26T01:45:51.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:51 smithi001 ceph-mon[27822]: osdmap e73: 11 total, 10 up, 11 in 2024-04-26T01:45:51.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:51 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:45:51.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:51 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:45:51.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:51 smithi001 ceph-mon[27822]: osd.10 [v2:172.21.15.97:6824/1693425268,v1:172.21.15.97:6825/1693425268] boot 2024-04-26T01:45:52.029 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:45:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:52 smithi097 ceph-mon[32853]: purged_snaps scrub starts 2024-04-26T01:45:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:52 smithi097 ceph-mon[32853]: purged_snaps scrub ok 2024-04-26T01:45:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:52 smithi097 ceph-mon[32853]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:52 smithi097 ceph-mon[32853]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T01:45:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:45:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:52.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:52.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:52.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:52.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:52.782 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:52 smithi001 ceph-mon[27822]: purged_snaps scrub starts 2024-04-26T01:45:52.782 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:52 smithi001 ceph-mon[27822]: purged_snaps scrub ok 2024-04-26T01:45:52.782 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:52 smithi001 ceph-mon[27822]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:52.782 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:52 smithi001 ceph-mon[27822]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T01:45:52.782 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:45:52.782 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:52.782 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:52.782 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:52.782 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:52.782 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:52.782 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:52.782 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:52.782 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:52.783 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:52.783 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:52.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:52 smithi173 ceph-mon[32553]: purged_snaps scrub starts 2024-04-26T01:45:52.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:52 smithi173 ceph-mon[32553]: purged_snaps scrub ok 2024-04-26T01:45:52.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:52 smithi173 ceph-mon[32553]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 285 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:45:52.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:52 smithi173 ceph-mon[32553]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T01:45:52.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:45:52.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:52.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:52.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:52.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:52.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:52.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:45:52.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:52.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:52.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:45:52.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:45:53.652 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.652+0000 7f013999b700 1 -- 172.21.15.173:0/750913633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0134104640 msgr2=0x7f013410aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:53.652 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.652+0000 7f013999b700 1 --2- 172.21.15.173:0/750913633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0134104640 0x7f013410aef0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f012c00b0a0 tx=0x7f012c00b3b0 comp rx=0 tx=0).stop 2024-04-26T01:45:53.652 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.652+0000 7f013999b700 1 -- 172.21.15.173:0/750913633 shutdown_connections 2024-04-26T01:45:53.652 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.652+0000 7f013999b700 1 --2- 172.21.15.173:0/750913633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0134104640 0x7f013410aef0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:53.652 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.652+0000 7f013999b700 1 --2- 172.21.15.173:0/750913633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0134102aa0 0x7f0134102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:53.652 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.652+0000 7f013999b700 1 --2- 172.21.15.173:0/750913633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0134103ca0 0x7f0134104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:53.652 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.652+0000 7f013999b700 1 -- 172.21.15.173:0/750913633 >> 172.21.15.173:0/750913633 conn(0x7f01340fe040 msgr2=0x7f0134100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:53.652 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.652+0000 7f013999b700 1 -- 172.21.15.173:0/750913633 shutdown_connections 2024-04-26T01:45:53.652 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.652+0000 7f013999b700 1 -- 172.21.15.173:0/750913633 wait complete. 2024-04-26T01:45:53.652 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.652+0000 7f013999b700 1 Processor -- start 2024-04-26T01:45:53.652 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.652+0000 7f013999b700 1 -- start start 2024-04-26T01:45:53.652 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.653+0000 7f013999b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0134102aa0 0x7f0134198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:53.652 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.653+0000 7f013999b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0134103ca0 0x7f0134199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:53.653 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.653+0000 7f013999b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0134104640 0x7f01341a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:53.653 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.653+0000 7f013999b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f013410e590 con 0x7f0134102aa0 2024-04-26T01:45:53.653 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.653+0000 7f013999b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f013410e410 con 0x7f0134103ca0 2024-04-26T01:45:53.653 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.653+0000 7f013999b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f013410e710 con 0x7f0134104640 2024-04-26T01:45:53.653 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.653+0000 7f01337fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0134104640 0x7f01341a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:53.653 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.653+0000 7f01337fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0134104640 0x7f01341a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:48614/0 (socket says 172.21.15.173:48614) 2024-04-26T01:45:53.653 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.653+0000 7f01337fe700 1 -- 172.21.15.173:0/3872256705 learned_addr learned my addr 172.21.15.173:0/3872256705 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-26T01:45:53.653 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.653+0000 7f01327fc700 1 --2- 172.21.15.173:0/3872256705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0134103ca0 0x7f0134199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:53.653 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.653+0000 7f01337fe700 1 -- 172.21.15.173:0/3872256705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0134103ca0 msgr2=0x7f0134199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:53.653 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.653+0000 7f01337fe700 1 --2- 172.21.15.173:0/3872256705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0134103ca0 0x7f0134199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:53.653 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.653+0000 7f01337fe700 1 -- 172.21.15.173:0/3872256705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0134102aa0 msgr2=0x7f0134198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:53.653 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.653+0000 7f01337fe700 1 --2- 172.21.15.173:0/3872256705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0134102aa0 0x7f0134198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:53.653 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.653+0000 7f01337fe700 1 -- 172.21.15.173:0/3872256705 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01341a0870 con 0x7f0134104640 2024-04-26T01:45:53.653 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.653+0000 7f01337fe700 1 --2- 172.21.15.173:0/3872256705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0134104640 0x7f01341a01a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f012c00b070 tx=0x7f012c004ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:53.654 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.653+0000 7f0138999700 1 -- 172.21.15.173:0/3872256705 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c021030 con 0x7f0134104640 2024-04-26T01:45:53.654 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.653+0000 7f013999b700 1 -- 172.21.15.173:0/3872256705 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01341a0b00 con 0x7f0134104640 2024-04-26T01:45:53.654 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.653+0000 7f0138999700 1 -- 172.21.15.173:0/3872256705 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f012c007be0 con 0x7f0134104640 2024-04-26T01:45:53.654 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.653+0000 7f0138999700 1 -- 172.21.15.173:0/3872256705 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c004790 con 0x7f0134104640 2024-04-26T01:45:53.654 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.654+0000 7f013999b700 1 -- 172.21.15.173:0/3872256705 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01341a0fb0 con 0x7f0134104640 2024-04-26T01:45:53.655 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.654+0000 7f013999b700 1 -- 172.21.15.173:0/3872256705 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0134069730 con 0x7f0134104640 2024-04-26T01:45:53.655 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.655+0000 7f0138999700 1 -- 172.21.15.173:0/3872256705 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f012c024070 con 0x7f0134104640 2024-04-26T01:45:53.655 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.656+0000 7f0138999700 1 --2- 172.21.15.173:0/3872256705 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f011c0a3c40 0x7f011c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:53.655 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.656+0000 7f0138999700 1 -- 172.21.15.173:0/3872256705 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6723+0+0 (secure 0 0 0) 0x7f012c0d15f0 con 0x7f0134104640 2024-04-26T01:45:53.655 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.657+0000 7f0132ffd700 1 --2- 172.21.15.173:0/3872256705 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f011c0a3c40 0x7f011c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:53.656 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.658+0000 7f0132ffd700 1 --2- 172.21.15.173:0/3872256705 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f011c0a3c40 0x7f011c0a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f013419a100 tx=0x7f0124009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:53.658 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.660+0000 7f0138999700 1 -- 172.21.15.173:0/3872256705 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f012c09e690 con 0x7f0134104640 2024-04-26T01:45:53.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:53 smithi097 ceph-mon[32853]: Detected new or changed devices on smithi097 2024-04-26T01:45:53.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:53 smithi097 ceph-mon[32853]: Adjusting osd_memory_target on smithi097 to 4272M 2024-04-26T01:45:53.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:53 smithi097 ceph-mon[32853]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T01:45:53.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:53 smithi097 ceph-mon[32853]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T01:45:53.812 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:53 smithi001 ceph-mon[27822]: Detected new or changed devices on smithi097 2024-04-26T01:45:53.813 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:53 smithi001 ceph-mon[27822]: Adjusting osd_memory_target on smithi097 to 4272M 2024-04-26T01:45:53.813 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:53 smithi001 ceph-mon[27822]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T01:45:53.813 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:53 smithi001 ceph-mon[27822]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T01:45:53.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:53 smithi173 ceph-mon[32553]: Detected new or changed devices on smithi097 2024-04-26T01:45:53.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:53 smithi173 ceph-mon[32553]: Adjusting osd_memory_target on smithi097 to 4272M 2024-04-26T01:45:53.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:53 smithi173 ceph-mon[32553]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T01:45:53.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:53 smithi173 ceph-mon[32553]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T01:45:53.865 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:45:53.867+0000 7f013999b700 1 -- 172.21.15.173:0/3872256705 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0134062e80 con 0x7f011c0a3c40 2024-04-26T01:45:54.523 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:54 smithi001 ceph-mon[27822]: pgmap v261: 1 pgs: 1 remapped+peering; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:45:54.524 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:54 smithi001 ceph-mon[27822]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T01:45:54.524 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:54 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:45:54.524 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:54 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:45:54.524 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:54 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:54.725 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:54 smithi173 ceph-mon[32553]: pgmap v261: 1 pgs: 1 remapped+peering; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:45:54.725 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:54 smithi173 ceph-mon[32553]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T01:45:54.725 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:54 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:45:54.725 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:54 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:45:54.725 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:54 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:54.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:54 smithi097 ceph-mon[32853]: pgmap v261: 1 pgs: 1 remapped+peering; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:45:54.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:54 smithi097 ceph-mon[32853]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T01:45:54.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:54 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:45:54.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:54 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:45:54.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:54 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:45:55.680 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:55 smithi173 ceph-mon[32553]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:45:55.680 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:55 smithi173 ceph-mon[32553]: osdmap e78: 11 total, 11 up, 11 in 2024-04-26T01:45:55.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:55 smithi097 ceph-mon[32853]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:45:55.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:55 smithi097 ceph-mon[32853]: osdmap e78: 11 total, 11 up, 11 in 2024-04-26T01:45:55.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:55 smithi001 ceph-mon[27822]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:45:55.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:55 smithi001 ceph-mon[27822]: osdmap e78: 11 total, 11 up, 11 in 2024-04-26T01:45:56.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:56 smithi097 ceph-mon[32853]: pgmap v264: 1 pgs: 1 remapped+peering; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:45:56.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:56 smithi173 ceph-mon[32553]: pgmap v264: 1 pgs: 1 remapped+peering; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:45:56.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:56 smithi001 ceph-mon[27822]: pgmap v264: 1 pgs: 1 remapped+peering; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:45:57.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:57 smithi097 ceph-mon[32853]: from='client.? 172.21.15.173:0/3240319488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f439aae1-f50a-4176-889d-36fa5131541a"}]: dispatch 2024-04-26T01:45:57.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:57 smithi097 ceph-mon[32853]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f439aae1-f50a-4176-889d-36fa5131541a"}]: dispatch 2024-04-26T01:45:57.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:57 smithi097 ceph-mon[32853]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f439aae1-f50a-4176-889d-36fa5131541a"}]': finished 2024-04-26T01:45:57.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:57 smithi097 ceph-mon[32853]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T01:45:57.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:57 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:45:57.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:57 smithi097 ceph-mon[32853]: from='client.? 172.21.15.173:0/509608746' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:45:57.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:57 smithi173 ceph-mon[32553]: from='client.? 172.21.15.173:0/3240319488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f439aae1-f50a-4176-889d-36fa5131541a"}]: dispatch 2024-04-26T01:45:57.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:57 smithi173 ceph-mon[32553]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f439aae1-f50a-4176-889d-36fa5131541a"}]: dispatch 2024-04-26T01:45:57.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:57 smithi173 ceph-mon[32553]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f439aae1-f50a-4176-889d-36fa5131541a"}]': finished 2024-04-26T01:45:57.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:57 smithi173 ceph-mon[32553]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T01:45:57.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:57 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:45:57.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:57 smithi173 ceph-mon[32553]: from='client.? 172.21.15.173:0/509608746' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:45:57.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:57 smithi001 ceph-mon[27822]: from='client.? 172.21.15.173:0/3240319488' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f439aae1-f50a-4176-889d-36fa5131541a"}]: dispatch 2024-04-26T01:45:57.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:57 smithi001 ceph-mon[27822]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f439aae1-f50a-4176-889d-36fa5131541a"}]: dispatch 2024-04-26T01:45:57.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:57 smithi001 ceph-mon[27822]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f439aae1-f50a-4176-889d-36fa5131541a"}]': finished 2024-04-26T01:45:57.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:57 smithi001 ceph-mon[27822]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T01:45:57.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:57 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:45:57.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:57 smithi001 ceph-mon[27822]: from='client.? 172.21.15.173:0/509608746' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:45:58.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:45:58 smithi173 ceph-mon[32553]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T01:45:58.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:45:58 smithi001 ceph-mon[27822]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T01:45:58.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:45:58 smithi097 ceph-mon[32853]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T01:46:00.747 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:00 smithi173 ceph-mon[32553]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-26T01:46:00.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:00 smithi001 ceph-mon[27822]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-26T01:46:00.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:00 smithi097 ceph-mon[32853]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-26T01:46:02.629 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:02 smithi173 ceph-mon[32553]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-26T01:46:02.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:02 smithi001 ceph-mon[27822]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-26T01:46:02.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:02 smithi097 ceph-mon[32853]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-26T01:46:04.698 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:04 smithi173 ceph-mon[32553]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:46:04.698 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:04 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T01:46:04.698 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:04 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:04.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:04 smithi001 ceph-mon[27822]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:46:04.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:04 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T01:46:04.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:04 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:04.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:04 smithi097 ceph-mon[32853]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:46:04.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:04 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T01:46:04.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:04 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:05.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:05 smithi173 ceph-mon[32553]: Deploying daemon osd.11 on smithi173 2024-04-26T01:46:05.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:05 smithi001 ceph-mon[27822]: Deploying daemon osd.11 on smithi173 2024-04-26T01:46:05.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:05 smithi097 ceph-mon[32853]: Deploying daemon osd.11 on smithi173 2024-04-26T01:46:06.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:06 smithi001 ceph-mon[27822]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:46:06.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:06 smithi097 ceph-mon[32853]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:46:07.014 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:06 smithi173 ceph-mon[32553]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:46:08.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:08 smithi173 ceph-mon[32553]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:46:08.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:08 smithi001 ceph-mon[27822]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:46:08.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:08 smithi097 ceph-mon[32853]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:46:10.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:10 smithi097 ceph-mon[32853]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:46:10.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:10 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:10.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:10 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:10.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:10 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:10.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:10 smithi173 ceph-mon[32553]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:46:10.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:10 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:10.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:10 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:10.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:10 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:10.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:10 smithi001 ceph-mon[27822]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:46:10.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:10 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:10.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:10 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:10.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:10 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:12.383 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:12 smithi173 ceph-mon[32553]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:46:12.498 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 11 on host 'smithi173' 2024-04-26T01:46:12.498 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:12.499+0000 7f0138999700 1 -- 172.21.15.173:0/3872256705 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0134062e80 con 0x7f011c0a3c40 2024-04-26T01:46:12.499 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:12.501+0000 7f013999b700 1 -- 172.21.15.173:0/3872256705 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f011c0a3c40 msgr2=0x7f011c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:12.499 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:12.501+0000 7f013999b700 1 --2- 172.21.15.173:0/3872256705 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f011c0a3c40 0x7f011c0a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f013419a100 tx=0x7f0124009470 comp rx=0 tx=0).stop 2024-04-26T01:46:12.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:12.501+0000 7f013999b700 1 -- 172.21.15.173:0/3872256705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0134104640 msgr2=0x7f01341a01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:12.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:12.501+0000 7f013999b700 1 --2- 172.21.15.173:0/3872256705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0134104640 0x7f01341a01a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f012c00b070 tx=0x7f012c004ef0 comp rx=0 tx=0).stop 2024-04-26T01:46:12.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:12.501+0000 7f013999b700 1 -- 172.21.15.173:0/3872256705 shutdown_connections 2024-04-26T01:46:12.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:12.501+0000 7f013999b700 1 --2- 172.21.15.173:0/3872256705 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f011c0a3c40 0x7f011c0a6100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:12.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:12.501+0000 7f013999b700 1 --2- 172.21.15.173:0/3872256705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0134104640 0x7f01341a01a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:12.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:12.501+0000 7f013999b700 1 --2- 172.21.15.173:0/3872256705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0134102aa0 0x7f0134198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:12.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:12.501+0000 7f013999b700 1 --2- 172.21.15.173:0/3872256705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0134103ca0 0x7f0134199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:12.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:12.501+0000 7f013999b700 1 -- 172.21.15.173:0/3872256705 >> 172.21.15.173:0/3872256705 conn(0x7f01340fe040 msgr2=0x7f0134107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:12.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:12.502+0000 7f013999b700 1 -- 172.21.15.173:0/3872256705 shutdown_connections 2024-04-26T01:46:12.500 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:12.502+0000 7f013999b700 1 -- 172.21.15.173:0/3872256705 wait complete. 2024-04-26T01:46:12.565 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:12 smithi097 ceph-mon[32853]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:46:12.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:12 smithi001 ceph-mon[27822]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:46:13.072 DEBUG:teuthology.orchestra.run.smithi173:osd.11> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.11.service 2024-04-26T01:46:13.074 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-26T01:46:13.074 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-26T01:46:13.316 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 01:46:13 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T01:46:13.141+0000 7f7425cd37c0 -1 osd.11 0 log_to_monitors true 2024-04-26T01:46:13.546 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:46:13.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:13 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:13.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:13 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:13.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:13 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:13.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:13 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:13.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:13 smithi001 ceph-mon[27822]: from='osd.11 [v2:172.21.15.173:6824/1141524960,v1:172.21.15.173:6825/1141524960]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T01:46:13.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:13 smithi001 ceph-mon[27822]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T01:46:13.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:13 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:13.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:13 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:13.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:13 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:13.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:13 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:13.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:13 smithi097 ceph-mon[32853]: from='osd.11 [v2:172.21.15.173:6824/1141524960,v1:172.21.15.173:6825/1141524960]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T01:46:13.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:13 smithi097 ceph-mon[32853]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T01:46:13.771 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:13 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:13.771 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:13 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:13.771 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:13 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:13.772 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:13 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:13.772 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:13 smithi173 ceph-mon[32553]: from='osd.11 [v2:172.21.15.173:6824/1141524960,v1:172.21.15.173:6825/1141524960]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T01:46:13.772 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:13 smithi173 ceph-mon[32553]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T01:46:14.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:14 smithi097 ceph-mon[32853]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:46:14.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:14 smithi097 ceph-mon[32853]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T01:46:14.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:14 smithi097 ceph-mon[32853]: from='osd.11 [v2:172.21.15.173:6824/1141524960,v1:172.21.15.173:6825/1141524960]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:46:14.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:14 smithi097 ceph-mon[32853]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T01:46:14.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:14 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:46:14.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:14 smithi097 ceph-mon[32853]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:46:14.757 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:14 smithi001 ceph-mon[27822]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:46:14.758 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:14 smithi001 ceph-mon[27822]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T01:46:14.758 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:14 smithi001 ceph-mon[27822]: from='osd.11 [v2:172.21.15.173:6824/1141524960,v1:172.21.15.173:6825/1141524960]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:46:14.758 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:14 smithi001 ceph-mon[27822]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T01:46:14.758 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:14 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:46:14.758 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:14 smithi001 ceph-mon[27822]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:46:14.806 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:14 smithi173 ceph-mon[32553]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:46:14.806 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:14 smithi173 ceph-mon[32553]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T01:46:14.806 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:14 smithi173 ceph-mon[32553]: from='osd.11 [v2:172.21.15.173:6824/1141524960,v1:172.21.15.173:6825/1141524960]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:46:14.807 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:14 smithi173 ceph-mon[32553]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T01:46:14.807 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:14 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:46:14.807 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:14 smithi173 ceph-mon[32553]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-26T01:46:15.316 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 01:46:15 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T01:46:15.159+0000 7f741b338700 -1 osd.11 0 waiting for initial osdmap 2024-04-26T01:46:15.316 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 01:46:15 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T01:46:15.179+0000 7f741712b700 -1 osd.11 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:46:15.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.976+0000 7f3f2da00700 1 -- 172.21.15.1:0/1322284163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f28103bd0 msgr2=0x7f3f28104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:15.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.976+0000 7f3f2da00700 1 --2- 172.21.15.1:0/1322284163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f28103bd0 0x7f3f28104030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009990 tx=0x7f3f10009ca0 comp rx=0 tx=0).stop 2024-04-26T01:46:15.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.976+0000 7f3f2da00700 1 -- 172.21.15.1:0/1322284163 shutdown_connections 2024-04-26T01:46:15.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.976+0000 7f3f2da00700 1 --2- 172.21.15.1:0/1322284163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f28104570 0x7f3f2810ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:15.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.976+0000 7f3f2da00700 1 --2- 172.21.15.1:0/1322284163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f281029d0 0x7f3f28102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:15.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.976+0000 7f3f2da00700 1 --2- 172.21.15.1:0/1322284163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f28103bd0 0x7f3f28104030 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:15.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.976+0000 7f3f2da00700 1 -- 172.21.15.1:0/1322284163 >> 172.21.15.1:0/1322284163 conn(0x7f3f280fdf70 msgr2=0x7f3f281003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:15.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.977+0000 7f3f2da00700 1 -- 172.21.15.1:0/1322284163 shutdown_connections 2024-04-26T01:46:15.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.977+0000 7f3f2da00700 1 -- 172.21.15.1:0/1322284163 wait complete. 2024-04-26T01:46:15.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.978+0000 7f3f2da00700 1 Processor -- start 2024-04-26T01:46:15.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.978+0000 7f3f2da00700 1 -- start start 2024-04-26T01:46:15.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.978+0000 7f3f2da00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f281029d0 0x7f3f2819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:15.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.978+0000 7f3f2da00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f28103bd0 0x7f3f2819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:15.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.979+0000 7f3f2da00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f28104570 0x7f3f281a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:15.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.979+0000 7f3f2da00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2810e4c0 con 0x7f3f28103bd0 2024-04-26T01:46:15.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.979+0000 7f3f2da00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2810e340 con 0x7f3f281029d0 2024-04-26T01:46:15.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.979+0000 7f3f2da00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2810e640 con 0x7f3f28104570 2024-04-26T01:46:15.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.979+0000 7f3f27e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f281029d0 0x7f3f2819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:15.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.979+0000 7f3f1fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f28103bd0 0x7f3f2819b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:15.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.979+0000 7f3f27e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f281029d0 0x7f3f2819ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:42094/0 (socket says 172.21.15.1:42094) 2024-04-26T01:46:15.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.979+0000 7f3f27e00700 1 -- 172.21.15.1:0/1038035401 learned_addr learned my addr 172.21.15.1:0/1038035401 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:46:15.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.979+0000 7f3f2d000700 1 --2- 172.21.15.1:0/1038035401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f28104570 0x7f3f281a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:15.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.979+0000 7f3f27e00700 1 -- 172.21.15.1:0/1038035401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f28104570 msgr2=0x7f3f281a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:15.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.979+0000 7f3f27e00700 1 --2- 172.21.15.1:0/1038035401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f28104570 0x7f3f281a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:15.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.979+0000 7f3f27e00700 1 -- 172.21.15.1:0/1038035401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f28103bd0 msgr2=0x7f3f2819b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:15.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.979+0000 7f3f27e00700 1 --2- 172.21.15.1:0/1038035401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f28103bd0 0x7f3f2819b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:15.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.979+0000 7f3f27e00700 1 -- 172.21.15.1:0/1038035401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f281a2930 con 0x7f3f281029d0 2024-04-26T01:46:15.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.980+0000 7f3f27e00700 1 --2- 172.21.15.1:0/1038035401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f281029d0 0x7f3f2819ad30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800e970 tx=0x7f3f1800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:15.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.980+0000 7f3f25600700 1 -- 172.21.15.1:0/1038035401 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18004220 con 0x7f3f281029d0 2024-04-26T01:46:15.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.980+0000 7f3f2da00700 1 -- 172.21.15.1:0/1038035401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f281a2c20 con 0x7f3f281029d0 2024-04-26T01:46:15.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.980+0000 7f3f25600700 1 -- 172.21.15.1:0/1038035401 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f180043c0 con 0x7f3f281029d0 2024-04-26T01:46:15.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.980+0000 7f3f25600700 1 -- 172.21.15.1:0/1038035401 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18003ce0 con 0x7f3f281029d0 2024-04-26T01:46:15.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.980+0000 7f3f2da00700 1 -- 172.21.15.1:0/1038035401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f281a3160 con 0x7f3f281029d0 2024-04-26T01:46:15.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.981+0000 7f3f2da00700 1 -- 172.21.15.1:0/1038035401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f28069730 con 0x7f3f281029d0 2024-04-26T01:46:15.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.983+0000 7f3f25600700 1 -- 172.21.15.1:0/1038035401 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f3f1800c7e0 con 0x7f3f281029d0 2024-04-26T01:46:15.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.985+0000 7f3f25600700 1 --2- 172.21.15.1:0/1038035401 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f3f0c0a3c40 0x7f3f0c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:15.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.985+0000 7f3f25600700 1 -- 172.21.15.1:0/1038035401 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6847+0+0 (secure 0 0 0) 0x7f3f180d00f0 con 0x7f3f281029d0 2024-04-26T01:46:15.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.985+0000 7f3f1fe00700 1 --2- 172.21.15.1:0/1038035401 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f3f0c0a3c40 0x7f3f0c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:15.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.986+0000 7f3f1fe00700 1 --2- 172.21.15.1:0/1038035401 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f3f0c0a3c40 0x7f3f0c0a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000b2e0 tx=0x7f3f10005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:15.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:15.989+0000 7f3f25600700 1 -- 172.21.15.1:0/1038035401 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3f1809cff0 con 0x7f3f281029d0 2024-04-26T01:46:16.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:16.229+0000 7f3f2da00700 1 -- 172.21.15.1:0/1038035401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3f281a3a50 con 0x7f3f281029d0 2024-04-26T01:46:16.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:16.230+0000 7f3f25600700 1 -- 172.21.15.1:0/1038035401 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7f3f1809c8f0 con 0x7f3f281029d0 2024-04-26T01:46:16.228 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:46:16.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:16.233+0000 7f3f2da00700 1 -- 172.21.15.1:0/1038035401 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f3f0c0a3c40 msgr2=0x7f3f0c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:16.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:16.233+0000 7f3f2da00700 1 --2- 172.21.15.1:0/1038035401 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f3f0c0a3c40 0x7f3f0c0a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000b2e0 tx=0x7f3f10005fb0 comp rx=0 tx=0).stop 2024-04-26T01:46:16.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:16.233+0000 7f3f2da00700 1 -- 172.21.15.1:0/1038035401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f281029d0 msgr2=0x7f3f2819ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:16.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:16.233+0000 7f3f2da00700 1 --2- 172.21.15.1:0/1038035401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f281029d0 0x7f3f2819ad30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800e970 tx=0x7f3f1800ec80 comp rx=0 tx=0).stop 2024-04-26T01:46:16.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:16.233+0000 7f3f2da00700 1 -- 172.21.15.1:0/1038035401 shutdown_connections 2024-04-26T01:46:16.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:16.233+0000 7f3f2da00700 1 --2- 172.21.15.1:0/1038035401 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f3f0c0a3c40 0x7f3f0c0a6100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:16.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:16.233+0000 7f3f2da00700 1 --2- 172.21.15.1:0/1038035401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f28104570 0x7f3f281a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:16.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:16.233+0000 7f3f2da00700 1 --2- 172.21.15.1:0/1038035401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f281029d0 0x7f3f2819ad30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:16.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:16.233+0000 7f3f2da00700 1 --2- 172.21.15.1:0/1038035401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f28103bd0 0x7f3f2819b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:16.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:16.233+0000 7f3f2da00700 1 -- 172.21.15.1:0/1038035401 >> 172.21.15.1:0/1038035401 conn(0x7f3f280fdf70 msgr2=0x7f3f28074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:16.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:16.233+0000 7f3f2da00700 1 -- 172.21.15.1:0/1038035401 shutdown_connections 2024-04-26T01:46:16.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:16.233+0000 7f3f2da00700 1 -- 172.21.15.1:0/1038035401 wait complete. 2024-04-26T01:46:16.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:16 smithi173 ceph-mon[32553]: purged_snaps scrub starts 2024-04-26T01:46:16.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:16 smithi173 ceph-mon[32553]: purged_snaps scrub ok 2024-04-26T01:46:16.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:16 smithi173 ceph-mon[32553]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:46:16.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:16 smithi173 ceph-mon[32553]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-26T01:46:16.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:16 smithi173 ceph-mon[32553]: osdmap e81: 12 total, 11 up, 12 in 2024-04-26T01:46:16.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:16 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:46:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:16 smithi097 ceph-mon[32853]: purged_snaps scrub starts 2024-04-26T01:46:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:16 smithi097 ceph-mon[32853]: purged_snaps scrub ok 2024-04-26T01:46:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:16 smithi097 ceph-mon[32853]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:46:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:16 smithi097 ceph-mon[32853]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-26T01:46:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:16 smithi097 ceph-mon[32853]: osdmap e81: 12 total, 11 up, 12 in 2024-04-26T01:46:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:16 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:46:16.570 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:16 smithi001 ceph-mon[27822]: purged_snaps scrub starts 2024-04-26T01:46:16.571 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:16 smithi001 ceph-mon[27822]: purged_snaps scrub ok 2024-04-26T01:46:16.571 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:16 smithi001 ceph-mon[27822]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:46:16.571 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:16 smithi001 ceph-mon[27822]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-26T01:46:16.571 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:16 smithi001 ceph-mon[27822]: osdmap e81: 12 total, 11 up, 12 in 2024-04-26T01:46:16.571 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:16 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:46:16.967 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1714095976,"num_in_osds":12,"osd_in_since":1714095956,"num_remapped_pgs":0} 2024-04-26T01:46:16.967 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:46:17.416 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:46:17.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:17 smithi097 ceph-mon[32853]: osd.11 [v2:172.21.15.173:6824/1141524960,v1:172.21.15.173:6825/1141524960] boot 2024-04-26T01:46:17.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:17 smithi097 ceph-mon[32853]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T01:46:17.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:17 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:46:17.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:17 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1038035401' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T01:46:17.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:17 smithi097 ceph-mon[32853]: Detected new or changed devices on smithi173 2024-04-26T01:46:17.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:17 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:17.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:17 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:17.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:17 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:17.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:17 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:17.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:17 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:17.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:17 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:17.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:17 smithi097 ceph-mon[32853]: Adjusting osd_memory_target on smithi173 to 4272M 2024-04-26T01:46:17.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:17 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:17.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:17 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:17.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:17 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:17.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:17 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:17.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:17 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:17.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:17 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:17.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:17 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:17.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:17 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:17.565 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:17 smithi001 ceph-mon[27822]: osd.11 [v2:172.21.15.173:6824/1141524960,v1:172.21.15.173:6825/1141524960] boot 2024-04-26T01:46:17.565 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:17 smithi001 ceph-mon[27822]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T01:46:17.565 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:17 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:46:17.565 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:17 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1038035401' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T01:46:17.565 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:17 smithi001 ceph-mon[27822]: Detected new or changed devices on smithi173 2024-04-26T01:46:17.565 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:17 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:17.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:17 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:17.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:17 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:17.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:17 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:17.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:17 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:17.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:17 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:17.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:17 smithi001 ceph-mon[27822]: Adjusting osd_memory_target on smithi173 to 4272M 2024-04-26T01:46:17.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:17 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:17.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:17 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:17.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:17 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:17.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:17 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:17.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:17 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:17.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:17 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:17.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:17 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:17.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:17 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:17.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:17 smithi173 ceph-mon[32553]: osd.11 [v2:172.21.15.173:6824/1141524960,v1:172.21.15.173:6825/1141524960] boot 2024-04-26T01:46:17.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:17 smithi173 ceph-mon[32553]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T01:46:17.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:17 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:46:17.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:17 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1038035401' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T01:46:17.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:17 smithi173 ceph-mon[32553]: Detected new or changed devices on smithi173 2024-04-26T01:46:17.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:17 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:17.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:17 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:17.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:17 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:17.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:17 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:17.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:17 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:17.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:17 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:17.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:17 smithi173 ceph-mon[32553]: Adjusting osd_memory_target on smithi173 to 4272M 2024-04-26T01:46:17.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:17 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:17.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:17 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:17.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:17 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:17.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:17 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:17.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:17 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:17.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:17 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:17.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:17 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:17.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:17 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:18.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:18 smithi173 ceph-mon[32553]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:18.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:18 smithi173 ceph-mon[32553]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T01:46:18.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:18 smithi001 ceph-mon[27822]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:18.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:18 smithi001 ceph-mon[27822]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T01:46:18.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:18 smithi097 ceph-mon[32853]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:18.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:18 smithi097 ceph-mon[32853]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T01:46:19.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.620+0000 7f69a9c00700 1 -- 172.21.15.1:0/1304094926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69a410cf60 msgr2=0x7f69a410f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:19.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.620+0000 7f69a9c00700 1 --2- 172.21.15.1:0/1304094926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69a410cf60 0x7f69a410f420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f699c00b0a0 tx=0x7f699c00b3b0 comp rx=0 tx=0).stop 2024-04-26T01:46:19.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.621+0000 7f69a9c00700 1 -- 172.21.15.1:0/1304094926 shutdown_connections 2024-04-26T01:46:19.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.621+0000 7f69a9c00700 1 --2- 172.21.15.1:0/1304094926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69a410cf60 0x7f69a410f420 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:19.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.621+0000 7f69a9c00700 1 --2- 172.21.15.1:0/1304094926 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69a4103b30 0x7f69a4103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:19.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.621+0000 7f69a9c00700 1 --2- 172.21.15.1:0/1304094926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69a4104450 0x7f69a410c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:19.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.621+0000 7f69a9c00700 1 -- 172.21.15.1:0/1304094926 >> 172.21.15.1:0/1304094926 conn(0x7f69a40fd340 msgr2=0x7f69a40ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:19.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.621+0000 7f69a9c00700 1 -- 172.21.15.1:0/1304094926 shutdown_connections 2024-04-26T01:46:19.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.621+0000 7f69a9c00700 1 -- 172.21.15.1:0/1304094926 wait complete. 2024-04-26T01:46:19.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.622+0000 7f69a9c00700 1 Processor -- start 2024-04-26T01:46:19.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.622+0000 7f69a9c00700 1 -- start start 2024-04-26T01:46:19.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.623+0000 7f69a9c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69a4103b30 0x7f69a419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:19.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.623+0000 7f69a9c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69a4104450 0x7f69a419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:19.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.623+0000 7f69a9c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69a410cf60 0x7f69a419fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:19.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.623+0000 7f69a9c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f69a4112ba0 con 0x7f69a4103b30 2024-04-26T01:46:19.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.623+0000 7f69a9c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f69a4112a20 con 0x7f69a410cf60 2024-04-26T01:46:19.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.623+0000 7f69a9c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f69a4112d20 con 0x7f69a4104450 2024-04-26T01:46:19.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.623+0000 7f69a3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69a4103b30 0x7f69a419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:19.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.623+0000 7f69a3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69a4103b30 0x7f69a419b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56796/0 (socket says 172.21.15.1:56796) 2024-04-26T01:46:19.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.623+0000 7f69a3e00700 1 -- 172.21.15.1:0/332137346 learned_addr learned my addr 172.21.15.1:0/332137346 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:46:19.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.623+0000 7f69a3400700 1 --2- 172.21.15.1:0/332137346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69a4104450 0x7f69a419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:19.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.623+0000 7f69a9200700 1 --2- 172.21.15.1:0/332137346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69a410cf60 0x7f69a419fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:19.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.624+0000 7f69a3e00700 1 -- 172.21.15.1:0/332137346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69a4104450 msgr2=0x7f69a419b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:19.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.624+0000 7f69a3e00700 1 --2- 172.21.15.1:0/332137346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69a4104450 0x7f69a419b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:19.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.624+0000 7f69a3e00700 1 -- 172.21.15.1:0/332137346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69a410cf60 msgr2=0x7f69a419fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:19.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.624+0000 7f69a3e00700 1 --2- 172.21.15.1:0/332137346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69a410cf60 0x7f69a419fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:19.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.624+0000 7f69a3e00700 1 -- 172.21.15.1:0/332137346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a41a0220 con 0x7f69a4103b30 2024-04-26T01:46:19.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.624+0000 7f69a3400700 1 --2- 172.21.15.1:0/332137346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69a4104450 0x7f69a419b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:46:19.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.624+0000 7f69a9200700 1 --2- 172.21.15.1:0/332137346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69a410cf60 0x7f69a419fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:46:19.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.624+0000 7f69a3e00700 1 --2- 172.21.15.1:0/332137346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69a4103b30 0x7f69a419b210 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f699400d950 tx=0x7f699400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:19.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.624+0000 7f69a0c00700 1 -- 172.21.15.1:0/332137346 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994016070 con 0x7f69a4103b30 2024-04-26T01:46:19.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.624+0000 7f69a9c00700 1 -- 172.21.15.1:0/332137346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a41a0510 con 0x7f69a4103b30 2024-04-26T01:46:19.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.624+0000 7f69a0c00700 1 -- 172.21.15.1:0/332137346 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f699400fd40 con 0x7f69a4103b30 2024-04-26T01:46:19.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.624+0000 7f69a0c00700 1 -- 172.21.15.1:0/332137346 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699400e3b0 con 0x7f69a4103b30 2024-04-26T01:46:19.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.625+0000 7f69a9c00700 1 -- 172.21.15.1:0/332137346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a41a7d90 con 0x7f69a4103b30 2024-04-26T01:46:19.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.626+0000 7f69a9c00700 1 -- 172.21.15.1:0/332137346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a4069730 con 0x7f69a4103b30 2024-04-26T01:46:19.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.628+0000 7f69a0c00700 1 -- 172.21.15.1:0/332137346 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f6994019070 con 0x7f69a4103b30 2024-04-26T01:46:19.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.630+0000 7f69a0c00700 1 --2- 172.21.15.1:0/332137346 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f69880a3c40 0x7f69880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:19.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.630+0000 7f69a0c00700 1 -- 172.21.15.1:0/332137346 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f69940cfab0 con 0x7f69a4103b30 2024-04-26T01:46:19.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.630+0000 7f69a3400700 1 --2- 172.21.15.1:0/332137346 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f69880a3c40 0x7f69880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:19.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.631+0000 7f69a3400700 1 --2- 172.21.15.1:0/332137346 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f69880a3c40 0x7f69880a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f69a410e7b0 tx=0x7f6990009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:19.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.635+0000 7f69a0c00700 1 -- 172.21.15.1:0/332137346 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f699409c890 con 0x7f69a4103b30 2024-04-26T01:46:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.867+0000 7f69a9c00700 1 -- 172.21.15.1:0/332137346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69a41a7f60 con 0x7f69a4103b30 2024-04-26T01:46:19.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.870+0000 7f69a0c00700 1 -- 172.21.15.1:0/332137346 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18361 (secure 0 0 0) 0x7f699409c190 con 0x7f69a4103b30 2024-04-26T01:46:19.868 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:46:19.869 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":83,"fsid":"77cfcc80-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:38:03.531182+0000","modified":"2024-04-26T01:46:17.747424+0000","last_up_change":"2024-04-26T01:46:16.151275+0000","last_in_change":"2024-04-26T01:45:56.514228+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:42:04.805488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb10ae12-09ee-4d6b-bc78-8c6016613b81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6803","nonce":3176633965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6805","nonce":3176633965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6809","nonce":3176633965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6807","nonce":3176633965}]},"public_addr":"172.21.15.1:6803/3176633965","cluster_addr":"172.21.15.1:6805/3176633965","heartbeat_back_addr":"172.21.15.1:6809/3176633965","heartbeat_front_addr":"172.21.15.1:6807/3176633965","state":["exists","up"]},{"osd":1,"uuid":"acd12888-6ff6-40bc-99e7-e5c395e85b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6801","nonce":3755022621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6803","nonce":3755022621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6807","nonce":3755022621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6805","nonce":3755022621}]},"public_addr":"172.21.15.97:6801/3755022621","cluster_addr":"172.21.15.97:6803/3755022621","heartbeat_back_addr":"172.21.15.97:6807/3755022621","heartbeat_front_addr":"172.21.15.97:6805/3755022621","state":["exists","up"]},{"osd":2,"uuid":"79971508-e256-4145-8111-a75666e886fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6801","nonce":734861771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6803","nonce":734861771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6807","nonce":734861771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6805","nonce":734861771}]},"public_addr":"172.21.15.173:6801/734861771","cluster_addr":"172.21.15.173:6803/734861771","heartbeat_back_addr":"172.21.15.173:6807/734861771","heartbeat_front_addr":"172.21.15.173:6805/734861771","state":["exists","up"]},{"osd":3,"uuid":"a8ccc4a9-0f31-446a-8bcb-4691532397c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6811","nonce":3349961961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6813","nonce":3349961961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6817","nonce":3349961961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6815","nonce":3349961961}]},"public_addr":"172.21.15.1:6811/3349961961","cluster_addr":"172.21.15.1:6813/3349961961","heartbeat_back_addr":"172.21.15.1:6817/3349961961","heartbeat_front_addr":"172.21.15.1:6815/3349961961","state":["exists","up"]},{"osd":4,"uuid":"5b724648-4df7-40fc-89d5-55f170cdd806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6809","nonce":2494639397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6811","nonce":2494639397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6815","nonce":2494639397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6813","nonce":2494639397}]},"public_addr":"172.21.15.97:6809/2494639397","cluster_addr":"172.21.15.97:6811/2494639397","heartbeat_back_addr":"172.21.15.97:6815/2494639397","heartbeat_front_addr":"172.21.15.97:6813/2494639397","state":["exists","up"]},{"osd":5,"uuid":"a5a24a9c-3007-4b87-8562-40c98c41ef8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6809","nonce":1026507957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6811","nonce":1026507957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6815","nonce":1026507957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6813","nonce":1026507957}]},"public_addr":"172.21.15.173:6809/1026507957","cluster_addr":"172.21.15.173:6811/1026507957","heartbeat_back_addr":"172.21.15.173:6815/1026507957","heartbeat_front_addr":"172.21.15.173:6813/1026507957","state":["exists","up"]},{"osd":6,"uuid":"97cd2e88-a56c-4b1a-8104-65b23a77986a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6819","nonce":444030587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6821","nonce":444030587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6825","nonce":444030587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6823","nonce":444030587}]},"public_addr":"172.21.15.1:6819/444030587","cluster_addr":"172.21.15.1:6821/444030587","heartbeat_back_addr":"172.21.15.1:6825/444030587","heartbeat_front_addr":"172.21.15.1:6823/444030587","state":["exists","up"]},{"osd":7,"uuid":"afc0c2dd-e112-4358-b3aa-944366d5d383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6817","nonce":2600022531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6819","nonce":2600022531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6823","nonce":2600022531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6821","nonce":2600022531}]},"public_addr":"172.21.15.97:6817/2600022531","cluster_addr":"172.21.15.97:6819/2600022531","heartbeat_back_addr":"172.21.15.97:6823/2600022531","heartbeat_front_addr":"172.21.15.97:6821/2600022531","state":["exists","up"]},{"osd":8,"uuid":"41fa612b-793e-44da-af8b-44d17a9de303","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6817","nonce":1334205527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6819","nonce":1334205527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6823","nonce":1334205527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6821","nonce":1334205527}]},"public_addr":"172.21.15.173:6817/1334205527","cluster_addr":"172.21.15.173:6819/1334205527","heartbeat_back_addr":"172.21.15.173:6823/1334205527","heartbeat_front_addr":"172.21.15.173:6821/1334205527","state":["exists","up"]},{"osd":9,"uuid":"bbbb163b-f8c4-4be5-8f80-82445e8d407f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6827","nonce":2973486839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6829","nonce":2973486839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6833","nonce":2973486839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6831","nonce":2973486839}]},"public_addr":"172.21.15.1:6827/2973486839","cluster_addr":"172.21.15.1:6829/2973486839","heartbeat_back_addr":"172.21.15.1:6833/2973486839","heartbeat_front_addr":"172.21.15.1:6831/2973486839","state":["exists","up"]},{"osd":10,"uuid":"1b447b34-0600-4727-826d-b3d68adfb18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6825","nonce":1693425268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6827","nonce":1693425268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6831","nonce":1693425268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6829","nonce":1693425268}]},"public_addr":"172.21.15.97:6825/1693425268","cluster_addr":"172.21.15.97:6827/1693425268","heartbeat_back_addr":"172.21.15.97:6831/1693425268","heartbeat_front_addr":"172.21.15.97:6829/1693425268","state":["exists","up"]},{"osd":11,"uuid":"f439aae1-f50a-4176-889d-36fa5131541a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6825","nonce":1141524960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6827","nonce":1141524960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6831","nonce":1141524960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6829","nonce":1141524960}]},"public_addr":"172.21.15.173:6825/1141524960","cluster_addr":"172.21.15.173:6827/1141524960","heartbeat_back_addr":"172.21.15.173:6831/1141524960","heartbeat_front_addr":"172.21.15.173:6829/1141524960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:12.955643+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:38.846652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:02.556062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:34.900933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:01.170878+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:24.842573+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:58.304440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:24.868689+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:48.772003+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:22.611681+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:49.470779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:14.163984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6800/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/1000222947":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/457958597":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6801/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:6800/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/2715798962":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/1923058728":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/2489038777":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/2869502448":"2024-04-27T01:38:45.974031+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:46:19.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.873+0000 7f69a9c00700 1 -- 172.21.15.1:0/332137346 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f69880a3c40 msgr2=0x7f69880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:19.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.873+0000 7f69a9c00700 1 --2- 172.21.15.1:0/332137346 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f69880a3c40 0x7f69880a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f69a410e7b0 tx=0x7f6990009260 comp rx=0 tx=0).stop 2024-04-26T01:46:19.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.873+0000 7f69a9c00700 1 -- 172.21.15.1:0/332137346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69a4103b30 msgr2=0x7f69a419b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:19.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.873+0000 7f69a9c00700 1 --2- 172.21.15.1:0/332137346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69a4103b30 0x7f69a419b210 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f699400d950 tx=0x7f699400dc60 comp rx=0 tx=0).stop 2024-04-26T01:46:19.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.874+0000 7f69a9c00700 1 -- 172.21.15.1:0/332137346 shutdown_connections 2024-04-26T01:46:19.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.874+0000 7f69a9c00700 1 --2- 172.21.15.1:0/332137346 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f69880a3c40 0x7f69880a6100 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:19.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.874+0000 7f69a9c00700 1 --2- 172.21.15.1:0/332137346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69a410cf60 0x7f69a419fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:19.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.874+0000 7f69a9c00700 1 --2- 172.21.15.1:0/332137346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69a4103b30 0x7f69a419b210 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:19.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.874+0000 7f69a9c00700 1 --2- 172.21.15.1:0/332137346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69a4104450 0x7f69a419b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:19.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.874+0000 7f69a9c00700 1 -- 172.21.15.1:0/332137346 >> 172.21.15.1:0/332137346 conn(0x7f69a40fd340 msgr2=0x7f69a410f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:19.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.874+0000 7f69a9c00700 1 -- 172.21.15.1:0/332137346 shutdown_connections 2024-04-26T01:46:19.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:19.874+0000 7f69a9c00700 1 -- 172.21.15.1:0/332137346 wait complete. 2024-04-26T01:46:20.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:20 smithi097 ceph-mon[32853]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:20.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:20 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/332137346' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:46:20.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:20 smithi173 ceph-mon[32553]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:20.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:20 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/332137346' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:46:20.585 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:20 smithi001 ceph-mon[27822]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:20.586 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:20 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/332137346' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:46:20.664 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T01:42:04.805488+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T01:46:20.665 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-26T01:46:21.143 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:46:22.360 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:22 smithi001 ceph-mon[27822]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:22.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:22 smithi097 ceph-mon[32853]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:22.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:22 smithi173 ceph-mon[32553]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:23.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.189+0000 7f2285c00700 1 -- 172.21.15.1:0/2210851196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f228006b370 msgr2=0x7f228006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:23.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.189+0000 7f2285c00700 1 --2- 172.21.15.1:0/2210851196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f228006b370 0x7f228006b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f22700099c0 tx=0x7f2270009cd0 comp rx=0 tx=0).stop 2024-04-26T01:46:23.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.189+0000 7f2285c00700 1 -- 172.21.15.1:0/2210851196 shutdown_connections 2024-04-26T01:46:23.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.189+0000 7f2285c00700 1 --2- 172.21.15.1:0/2210851196 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2280107e10 0x7f228010f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:23.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.189+0000 7f2285c00700 1 --2- 172.21.15.1:0/2210851196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f228006b370 0x7f228006b750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:23.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.189+0000 7f2285c00700 1 --2- 172.21.15.1:0/2210851196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f228006bd20 0x7f22801076e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:23.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.189+0000 7f2285c00700 1 -- 172.21.15.1:0/2210851196 >> 172.21.15.1:0/2210851196 conn(0x7f22800788f0 msgr2=0x7f2280078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:23.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.189+0000 7f2285c00700 1 -- 172.21.15.1:0/2210851196 shutdown_connections 2024-04-26T01:46:23.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.189+0000 7f2285c00700 1 -- 172.21.15.1:0/2210851196 wait complete. 2024-04-26T01:46:23.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.190+0000 7f2285c00700 1 Processor -- start 2024-04-26T01:46:23.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.190+0000 7f2285c00700 1 -- start start 2024-04-26T01:46:23.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.191+0000 7f2285c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f228006b370 0x7f2280101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:23.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.191+0000 7f2285c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f228006bd20 0x7f2280102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:23.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.191+0000 7f2285c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2280107e10 0x7f2280102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:23.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.191+0000 7f2285c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2280112c40 con 0x7f228006b370 2024-04-26T01:46:23.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.191+0000 7f2285c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2280112ac0 con 0x7f228006bd20 2024-04-26T01:46:23.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.191+0000 7f2285c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2280112dc0 con 0x7f2280107e10 2024-04-26T01:46:23.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.191+0000 7f227ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f228006b370 0x7f2280101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:23.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.191+0000 7f227ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f228006b370 0x7f2280101df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43214/0 (socket says 172.21.15.1:43214) 2024-04-26T01:46:23.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.191+0000 7f227ea00700 1 -- 172.21.15.1:0/2077037621 learned_addr learned my addr 172.21.15.1:0/2077037621 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:46:23.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.191+0000 7f227e000700 1 --2- 172.21.15.1:0/2077037621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f228006bd20 0x7f2280102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:23.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.191+0000 7f227f400700 1 --2- 172.21.15.1:0/2077037621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2280107e10 0x7f2280102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:23.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.191+0000 7f227ea00700 1 -- 172.21.15.1:0/2077037621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2280107e10 msgr2=0x7f2280102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:23.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.191+0000 7f227ea00700 1 --2- 172.21.15.1:0/2077037621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2280107e10 0x7f2280102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:23.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.191+0000 7f227ea00700 1 -- 172.21.15.1:0/2077037621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f228006bd20 msgr2=0x7f2280102330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:23.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.192+0000 7f227ea00700 1 --2- 172.21.15.1:0/2077037621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f228006bd20 0x7f2280102330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:23.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.192+0000 7f227ea00700 1 -- 172.21.15.1:0/2077037621 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22801a7640 con 0x7f228006b370 2024-04-26T01:46:23.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.192+0000 7f227e000700 1 --2- 172.21.15.1:0/2077037621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f228006bd20 0x7f2280102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:46:23.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.192+0000 7f227ea00700 1 --2- 172.21.15.1:0/2077037621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f228006b370 0x7f2280101df0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2270005910 tx=0x7f2270004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:23.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.192+0000 7f2277400700 1 -- 172.21.15.1:0/2077037621 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2270026070 con 0x7f228006b370 2024-04-26T01:46:23.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.192+0000 7f2285c00700 1 -- 172.21.15.1:0/2077037621 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22801a78d0 con 0x7f228006b370 2024-04-26T01:46:23.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.192+0000 7f2277400700 1 -- 172.21.15.1:0/2077037621 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2270003c70 con 0x7f228006b370 2024-04-26T01:46:23.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.192+0000 7f2285c00700 1 -- 172.21.15.1:0/2077037621 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22801a7d80 con 0x7f228006b370 2024-04-26T01:46:23.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.192+0000 7f2277400700 1 -- 172.21.15.1:0/2077037621 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227000f9e0 con 0x7f228006b370 2024-04-26T01:46:23.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.194+0000 7f2285c00700 1 -- 172.21.15.1:0/2077037621 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f228010b050 con 0x7f228006b370 2024-04-26T01:46:23.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.196+0000 7f2277400700 1 -- 172.21.15.1:0/2077037621 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f2270029070 con 0x7f228006b370 2024-04-26T01:46:23.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.197+0000 7f2277400700 1 --2- 172.21.15.1:0/2077037621 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f22640a3c40 0x7f22640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:23.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.197+0000 7f2277400700 1 -- 172.21.15.1:0/2077037621 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f22700d0770 con 0x7f228006b370 2024-04-26T01:46:23.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.197+0000 7f227e000700 1 --2- 172.21.15.1:0/2077037621 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f22640a3c40 0x7f22640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:23.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.198+0000 7f227e000700 1 --2- 172.21.15.1:0/2077037621 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f22640a3c40 0x7f22640a6100 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f226c009680 tx=0x7f226c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:23.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.201+0000 7f2277400700 1 -- 172.21.15.1:0/2077037621 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f227005dff0 con 0x7f228006b370 2024-04-26T01:46:23.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.435+0000 7f2285c00700 1 -- 172.21.15.1:0/2077037621 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f228010aa30 con 0x7f228006b370 2024-04-26T01:46:23.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.436+0000 7f2277400700 1 -- 172.21.15.1:0/2077037621 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f2270013070 con 0x7f228006b370 2024-04-26T01:46:23.434 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-04-26T01:46:23.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.439+0000 7f2285c00700 1 -- 172.21.15.1:0/2077037621 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f22640a3c40 msgr2=0x7f22640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:23.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.439+0000 7f2285c00700 1 --2- 172.21.15.1:0/2077037621 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f22640a3c40 0x7f22640a6100 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f226c009680 tx=0x7f226c009260 comp rx=0 tx=0).stop 2024-04-26T01:46:23.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.439+0000 7f2285c00700 1 -- 172.21.15.1:0/2077037621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f228006b370 msgr2=0x7f2280101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:23.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.439+0000 7f2285c00700 1 --2- 172.21.15.1:0/2077037621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f228006b370 0x7f2280101df0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2270005910 tx=0x7f2270004270 comp rx=0 tx=0).stop 2024-04-26T01:46:23.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.439+0000 7f2285c00700 1 -- 172.21.15.1:0/2077037621 shutdown_connections 2024-04-26T01:46:23.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.439+0000 7f2285c00700 1 --2- 172.21.15.1:0/2077037621 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f22640a3c40 0x7f22640a6100 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:23.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.439+0000 7f2285c00700 1 --2- 172.21.15.1:0/2077037621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2280107e10 0x7f2280102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:23.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.439+0000 7f2285c00700 1 --2- 172.21.15.1:0/2077037621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f228006b370 0x7f2280101df0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:23.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.439+0000 7f2285c00700 1 --2- 172.21.15.1:0/2077037621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f228006bd20 0x7f2280102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:23.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.439+0000 7f2285c00700 1 -- 172.21.15.1:0/2077037621 >> 172.21.15.1:0/2077037621 conn(0x7f22800788f0 msgr2=0x7f22800fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:23.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.440+0000 7f2285c00700 1 -- 172.21.15.1:0/2077037621 shutdown_connections 2024-04-26T01:46:23.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:23.440+0000 7f2285c00700 1 -- 172.21.15.1:0/2077037621 wait complete. 2024-04-26T01:46:24.189 INFO:tasks.cephadm:Adding mds.a on smithi001 2024-04-26T01:46:24.189 INFO:tasks.cephadm:Adding mds.d on smithi001 2024-04-26T01:46:24.189 INFO:tasks.cephadm:Adding mds.g on smithi001 2024-04-26T01:46:24.189 INFO:tasks.cephadm:Adding mds.j on smithi001 2024-04-26T01:46:24.190 INFO:tasks.cephadm:Adding mds.b on smithi097 2024-04-26T01:46:24.190 INFO:tasks.cephadm:Adding mds.e on smithi097 2024-04-26T01:46:24.190 INFO:tasks.cephadm:Adding mds.h on smithi097 2024-04-26T01:46:24.190 INFO:tasks.cephadm:Adding mds.k on smithi097 2024-04-26T01:46:24.190 INFO:tasks.cephadm:Adding mds.c on smithi173 2024-04-26T01:46:24.190 INFO:tasks.cephadm:Adding mds.f on smithi173 2024-04-26T01:46:24.190 INFO:tasks.cephadm:Adding mds.i on smithi173 2024-04-26T01:46:24.190 INFO:tasks.cephadm:Adding mds.l on smithi173 2024-04-26T01:46:24.190 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l' 2024-04-26T01:46:24.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:24 smithi001 ceph-mon[27822]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:24.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:24 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2077037621' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T01:46:24.438 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:46:24.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:24 smithi097 ceph-mon[32853]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:24.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:24 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/2077037621' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T01:46:24.555 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:24 smithi173 ceph-mon[32553]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:24.555 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:24 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/2077037621' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T01:46:25.948 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.948+0000 7f96ac989700 1 -- 172.21.15.173:0/3844226699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96a8108830 msgr2=0x7f96a810ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:25.949 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.948+0000 7f96ac989700 1 --2- 172.21.15.173:0/3844226699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96a8108830 0x7f96a810ac40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f96a000b0a0 tx=0x7f96a000b3b0 comp rx=0 tx=0).stop 2024-04-26T01:46:25.949 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.949+0000 7f96ac989700 1 -- 172.21.15.173:0/3844226699 shutdown_connections 2024-04-26T01:46:25.949 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.949+0000 7f96ac989700 1 --2- 172.21.15.173:0/3844226699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96a8108830 0x7f96a810ac40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:25.949 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.949+0000 7f96ac989700 1 --2- 172.21.15.173:0/3844226699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96a81035d0 0x7f96a81059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:25.949 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.949+0000 7f96ac989700 1 --2- 172.21.15.173:0/3844226699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96a8105f00 0x7f96a81082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:25.949 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.949+0000 7f96ac989700 1 -- 172.21.15.173:0/3844226699 >> 172.21.15.173:0/3844226699 conn(0x7f96a80fcfa0 msgr2=0x7f96a80ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:25.949 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.949+0000 7f96ac989700 1 -- 172.21.15.173:0/3844226699 shutdown_connections 2024-04-26T01:46:25.949 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.949+0000 7f96ac989700 1 -- 172.21.15.173:0/3844226699 wait complete. 2024-04-26T01:46:25.949 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.949+0000 7f96ac989700 1 Processor -- start 2024-04-26T01:46:25.949 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.950+0000 7f96ac989700 1 -- start start 2024-04-26T01:46:25.949 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.950+0000 7f96ac989700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96a81035d0 0x7f96a8198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:25.949 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.950+0000 7f96ac989700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96a8105f00 0x7f96a8199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:25.950 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.950+0000 7f96ac989700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96a8108830 0x7f96a81a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:25.950 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.950+0000 7f96ac989700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f96a810e5e0 con 0x7f96a81035d0 2024-04-26T01:46:25.950 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.950+0000 7f96ac989700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f96a810e460 con 0x7f96a8108830 2024-04-26T01:46:25.950 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.950+0000 7f96ac989700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f96a810e760 con 0x7f96a8105f00 2024-04-26T01:46:25.950 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.950+0000 7f96a5d9b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96a8105f00 0x7f96a8199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:25.950 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.950+0000 7f96a5d9b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96a8105f00 0x7f96a8199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:38918/0 (socket says 172.21.15.173:38918) 2024-04-26T01:46:25.950 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.950+0000 7f96a5d9b700 1 -- 172.21.15.173:0/2355293745 learned_addr learned my addr 172.21.15.173:0/2355293745 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-26T01:46:25.950 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.950+0000 7f96a6d9d700 1 --2- 172.21.15.173:0/2355293745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96a8108830 0x7f96a81a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:25.950 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.950+0000 7f96a5d9b700 1 -- 172.21.15.173:0/2355293745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96a8108830 msgr2=0x7f96a81a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:25.951 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.950+0000 7f96a5d9b700 1 --2- 172.21.15.173:0/2355293745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96a8108830 0x7f96a81a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:25.951 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.950+0000 7f96a5d9b700 1 -- 172.21.15.173:0/2355293745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96a81035d0 msgr2=0x7f96a8198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:25.951 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.950+0000 7f96a5d9b700 1 --2- 172.21.15.173:0/2355293745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96a81035d0 0x7f96a8198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:25.951 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.950+0000 7f96a5d9b700 1 -- 172.21.15.173:0/2355293745 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a81a0860 con 0x7f96a8105f00 2024-04-26T01:46:25.951 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.950+0000 7f96a5d9b700 1 --2- 172.21.15.173:0/2355293745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96a8105f00 0x7f96a8199150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f969c00b530 tx=0x7f969c00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:25.951 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.950+0000 7f96977fe700 1 -- 172.21.15.173:0/2355293745 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c015020 con 0x7f96a8105f00 2024-04-26T01:46:25.951 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.950+0000 7f96ac989700 1 -- 172.21.15.173:0/2355293745 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a81a0b50 con 0x7f96a8105f00 2024-04-26T01:46:25.951 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.951+0000 7f96977fe700 1 -- 172.21.15.173:0/2355293745 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f969c011ce0 con 0x7f96a8105f00 2024-04-26T01:46:25.951 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.951+0000 7f96977fe700 1 -- 172.21.15.173:0/2355293745 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c010e40 con 0x7f96a8105f00 2024-04-26T01:46:25.951 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.951+0000 7f96ac989700 1 -- 172.21.15.173:0/2355293745 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a81a10e0 con 0x7f96a8105f00 2024-04-26T01:46:25.951 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.951+0000 7f96ac989700 1 -- 172.21.15.173:0/2355293745 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a8069730 con 0x7f96a8105f00 2024-04-26T01:46:25.952 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.953+0000 7f96977fe700 1 -- 172.21.15.173:0/2355293745 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f969c018020 con 0x7f96a8105f00 2024-04-26T01:46:25.952 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.953+0000 7f96977fe700 1 --2- 172.21.15.173:0/2355293745 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f96900a3cc0 0x7f96900a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:25.952 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.953+0000 7f96977fe700 1 -- 172.21.15.173:0/2355293745 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f969c0cfd80 con 0x7f96a8105f00 2024-04-26T01:46:25.952 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.954+0000 7f96a659c700 1 --2- 172.21.15.173:0/2355293745 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f96900a3cc0 0x7f96900a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:25.952 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.955+0000 7f96a659c700 1 --2- 172.21.15.173:0/2355293745 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f96900a3cc0 0x7f96900a6180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9698005950 tx=0x7f96980058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:25.954 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:25.957+0000 7f96977fe700 1 -- 172.21.15.173:0/2355293745 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f969c01f020 con 0x7f96a8105f00 2024-04-26T01:46:26.165 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:26.168+0000 7f96ac989700 1 -- 172.21.15.173:0/2355293745 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l", "target": ["mon-mgr", ""]}) -- 0x7f96a8063080 con 0x7f96900a3cc0 2024-04-26T01:46:26.182 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:26.185+0000 7f96977fe700 1 -- 172.21.15.173:0/2355293745 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f96a8063080 con 0x7f96900a3cc0 2024-04-26T01:46:26.182 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled mds.all update... 2024-04-26T01:46:26.184 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:26.187+0000 7f96ac989700 1 -- 172.21.15.173:0/2355293745 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f96900a3cc0 msgr2=0x7f96900a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:26.184 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:26.187+0000 7f96ac989700 1 --2- 172.21.15.173:0/2355293745 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f96900a3cc0 0x7f96900a6180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9698005950 tx=0x7f96980058e0 comp rx=0 tx=0).stop 2024-04-26T01:46:26.184 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:26.187+0000 7f96ac989700 1 -- 172.21.15.173:0/2355293745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96a8105f00 msgr2=0x7f96a8199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:26.184 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:26.187+0000 7f96ac989700 1 --2- 172.21.15.173:0/2355293745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96a8105f00 0x7f96a8199150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f969c00b530 tx=0x7f969c00b840 comp rx=0 tx=0).stop 2024-04-26T01:46:26.184 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:26.187+0000 7f96ac989700 1 -- 172.21.15.173:0/2355293745 shutdown_connections 2024-04-26T01:46:26.185 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:26.187+0000 7f96ac989700 1 --2- 172.21.15.173:0/2355293745 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f96900a3cc0 0x7f96900a6180 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:26.185 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:26.187+0000 7f96ac989700 1 --2- 172.21.15.173:0/2355293745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96a8108830 0x7f96a81a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:26.185 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:26.187+0000 7f96ac989700 1 --2- 172.21.15.173:0/2355293745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96a81035d0 0x7f96a8198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:26.185 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:26.187+0000 7f96ac989700 1 --2- 172.21.15.173:0/2355293745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96a8105f00 0x7f96a8199150 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:26.185 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:26.187+0000 7f96ac989700 1 -- 172.21.15.173:0/2355293745 >> 172.21.15.173:0/2355293745 conn(0x7f96a80fcfa0 msgr2=0x7f96a8103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:26.185 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:26.187+0000 7f96ac989700 1 -- 172.21.15.173:0/2355293745 shutdown_connections 2024-04-26T01:46:26.185 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-26T01:46:26.187+0000 7f96ac989700 1 -- 172.21.15.173:0/2355293745 wait complete. 2024-04-26T01:46:26.339 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:26 smithi173 ceph-mon[32553]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:26.339 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:26 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:26.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:26 smithi097 ceph-mon[32853]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:26.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:26 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:26.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:26 smithi001 ceph-mon[27822]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:26.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:26 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:26.779 DEBUG:teuthology.orchestra.run.smithi001:mds.a> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.a.service 2024-04-26T01:46:26.786 DEBUG:teuthology.orchestra.run.smithi001:mds.d> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.d.service 2024-04-26T01:46:26.831 DEBUG:teuthology.orchestra.run.smithi001:mds.g> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.g.service 2024-04-26T01:46:26.876 DEBUG:teuthology.orchestra.run.smithi001:mds.j> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.j.service 2024-04-26T01:46:26.921 DEBUG:teuthology.orchestra.run.smithi097:mds.b> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.b.service 2024-04-26T01:46:26.924 DEBUG:teuthology.orchestra.run.smithi097:mds.e> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.e.service 2024-04-26T01:46:26.967 DEBUG:teuthology.orchestra.run.smithi097:mds.h> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.h.service 2024-04-26T01:46:27.010 DEBUG:teuthology.orchestra.run.smithi097:mds.k> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.k.service 2024-04-26T01:46:27.053 DEBUG:teuthology.orchestra.run.smithi173:mds.c> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.c.service 2024-04-26T01:46:27.056 DEBUG:teuthology.orchestra.run.smithi173:mds.f> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.f.service 2024-04-26T01:46:27.098 DEBUG:teuthology.orchestra.run.smithi173:mds.i> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.i.service 2024-04-26T01:46:27.141 DEBUG:teuthology.orchestra.run.smithi173:mds.l> sudo journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.l.service 2024-04-26T01:46:27.185 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-26T01:46:27.185 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:46:27.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:27 smithi097 ceph-mon[32853]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:46:27.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:27 smithi097 ceph-mon[32853]: Saving service mds.all spec with placement smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l;count:12 2024-04-26T01:46:27.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:27 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:27.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:27 smithi173 ceph-mon[32553]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:46:27.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:27 smithi173 ceph-mon[32553]: Saving service mds.all spec with placement smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l;count:12 2024-04-26T01:46:27.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:27 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:27.626 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:46:27.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:27 smithi001 ceph-mon[27822]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:46:27.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:27 smithi001 ceph-mon[27822]: Saving service mds.all spec with placement smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l;count:12 2024-04-26T01:46:27.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:27 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:28.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:28 smithi173 ceph-mon[32553]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:28.327 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:28.327 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:28.327 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:28.327 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:28.327 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:28.327 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:28.327 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:28 smithi173 ceph-mon[32553]: Deploying daemon mds.l on smithi173 2024-04-26T01:46:28.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:28 smithi001 ceph-mon[27822]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:28.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:28.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:28.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:28.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:28.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:28.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:28.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:28 smithi001 ceph-mon[27822]: Deploying daemon mds.l on smithi173 2024-04-26T01:46:28.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:28 smithi097 ceph-mon[32853]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:28.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:28.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:28.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:28.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:28.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:28.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:28.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:28 smithi097 ceph-mon[32853]: Deploying daemon mds.l on smithi173 2024-04-26T01:46:29.566 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 26 01:46:29 smithi173 systemd[1]: Starting Ceph mds.l for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T01:46:29.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.630+0000 7f7981e00700 1 -- 172.21.15.1:0/3301422982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f797c06a750 msgr2=0x7f797c06abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:29.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.630+0000 7f7981e00700 1 --2- 172.21.15.1:0/3301422982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f797c06a750 0x7f797c06abb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7968009990 tx=0x7f7968009ca0 comp rx=0 tx=0).stop 2024-04-26T01:46:29.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.630+0000 7f7981e00700 1 -- 172.21.15.1:0/3301422982 shutdown_connections 2024-04-26T01:46:29.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.630+0000 7f7981e00700 1 --2- 172.21.15.1:0/3301422982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f797c06b0f0 0x7f797c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:29.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.630+0000 7f7981e00700 1 --2- 172.21.15.1:0/3301422982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f797c108a30 0x7f797c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:29.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.630+0000 7f7981e00700 1 --2- 172.21.15.1:0/3301422982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f797c06a750 0x7f797c06abb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:29.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.630+0000 7f7981e00700 1 -- 172.21.15.1:0/3301422982 >> 172.21.15.1:0/3301422982 conn(0x7f797c0788f0 msgr2=0x7f797c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:29.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.630+0000 7f7981e00700 1 -- 172.21.15.1:0/3301422982 shutdown_connections 2024-04-26T01:46:29.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.631+0000 7f7981e00700 1 -- 172.21.15.1:0/3301422982 wait complete. 2024-04-26T01:46:29.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.632+0000 7f7981e00700 1 Processor -- start 2024-04-26T01:46:29.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.632+0000 7f7981e00700 1 -- start start 2024-04-26T01:46:29.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.633+0000 7f7981e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f797c06a750 0x7f797c199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:29.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.633+0000 7f7981e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f797c06b0f0 0x7f797c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:29.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.633+0000 7f7981e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f797c108a30 0x7f797c19d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:29.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.633+0000 7f7981e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f797c112bd0 con 0x7f797c108a30 2024-04-26T01:46:29.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.633+0000 7f7981e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f797c112a50 con 0x7f797c06b0f0 2024-04-26T01:46:29.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.633+0000 7f7981e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f797c112d50 con 0x7f797c06a750 2024-04-26T01:46:29.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.633+0000 7f797aa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f797c06a750 0x7f797c199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:29.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.633+0000 7f797b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f797c108a30 0x7f797c19d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:29.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.633+0000 7f797a000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f797c06b0f0 0x7f797c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:29.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.633+0000 7f797aa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f797c06a750 0x7f797c199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:34108/0 (socket says 172.21.15.1:34108) 2024-04-26T01:46:29.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.633+0000 7f797aa00700 1 -- 172.21.15.1:0/988152073 learned_addr learned my addr 172.21.15.1:0/988152073 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:46:29.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.634+0000 7f797aa00700 1 -- 172.21.15.1:0/988152073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f797c06b0f0 msgr2=0x7f797c1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:29.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.634+0000 7f797aa00700 1 --2- 172.21.15.1:0/988152073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f797c06b0f0 0x7f797c1996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:29.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.634+0000 7f797aa00700 1 -- 172.21.15.1:0/988152073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f797c108a30 msgr2=0x7f797c19d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:29.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.634+0000 7f797aa00700 1 --2- 172.21.15.1:0/988152073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f797c108a30 0x7f797c19d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:29.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.634+0000 7f797aa00700 1 -- 172.21.15.1:0/988152073 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797c19e120 con 0x7f797c06a750 2024-04-26T01:46:29.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.634+0000 7f797b400700 1 --2- 172.21.15.1:0/988152073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f797c108a30 0x7f797c19d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:46:29.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.634+0000 7f797aa00700 1 --2- 172.21.15.1:0/988152073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f797c06a750 0x7f797c199160 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f796c00e9c0 tx=0x7f796c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:29.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.634+0000 7f7980a00700 1 -- 172.21.15.1:0/988152073 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c00cbd0 con 0x7f797c06a750 2024-04-26T01:46:29.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.634+0000 7f7980a00700 1 -- 172.21.15.1:0/988152073 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f796c00cd70 con 0x7f797c06a750 2024-04-26T01:46:29.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.634+0000 7f7980a00700 1 -- 172.21.15.1:0/988152073 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c005610 con 0x7f797c06a750 2024-04-26T01:46:29.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.634+0000 7f7981e00700 1 -- 172.21.15.1:0/988152073 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f797c19e410 con 0x7f797c06a750 2024-04-26T01:46:29.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.634+0000 7f7981e00700 1 -- 172.21.15.1:0/988152073 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f797c101940 con 0x7f797c06a750 2024-04-26T01:46:29.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.636+0000 7f7981e00700 1 -- 172.21.15.1:0/988152073 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f797c104170 con 0x7f797c06a750 2024-04-26T01:46:29.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.637+0000 7f7980a00700 1 -- 172.21.15.1:0/988152073 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f796c004750 con 0x7f797c06a750 2024-04-26T01:46:29.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.639+0000 7f7980a00700 1 --2- 172.21.15.1:0/988152073 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f79600a3cc0 0x7f79600a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:29.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.639+0000 7f7980a00700 1 -- 172.21.15.1:0/988152073 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f796c0d0200 con 0x7f797c06a750 2024-04-26T01:46:29.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.639+0000 7f797a000700 1 --2- 172.21.15.1:0/988152073 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f79600a3cc0 0x7f79600a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:29.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.640+0000 7f797a000700 1 --2- 172.21.15.1:0/988152073 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f79600a3cc0 0x7f79600a6180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f79680098e0 tx=0x7f7968005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:29.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.643+0000 7f7980a00700 1 -- 172.21.15.1:0/988152073 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f796c09cfe0 con 0x7f797c06a750 2024-04-26T01:46:29.863 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 26 01:46:29 smithi173 podman[62479]: 2024-04-26 01:46:29.505417778 +0000 UTC m=+0.018144992 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:46:29.863 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 26 01:46:29 smithi173 podman[62479]: 2024-04-26 01:46:29.62088491 +0000 UTC m=+0.133612147 container create 4d65dc01c89eb0fb95c08fb97efdbdc23fac9402ff42a2492d0ee1333cec3d0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-26T01:46:29.863 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 26 01:46:29 smithi173 podman[62479]: 2024-04-26 01:46:29.840059074 +0000 UTC m=+0.352786265 container init 4d65dc01c89eb0fb95c08fb97efdbdc23fac9402ff42a2492d0ee1333cec3d0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-26T01:46:29.863 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 26 01:46:29 smithi173 podman[62479]: 2024-04-26 01:46:29.843012364 +0000 UTC m=+0.355739565 container start 4d65dc01c89eb0fb95c08fb97efdbdc23fac9402ff42a2492d0ee1333cec3d0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-26T01:46:29.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.923+0000 7f7981e00700 1 -- 172.21.15.1:0/988152073 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f797c19a470 con 0x7f797c06a750 2024-04-26T01:46:29.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.924+0000 7f7980a00700 1 -- 172.21.15.1:0/988152073 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f796c09c8e0 con 0x7f797c06a750 2024-04-26T01:46:29.922 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:46:29.922 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T01:46:29.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.927+0000 7f7981e00700 1 -- 172.21.15.1:0/988152073 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f79600a3cc0 msgr2=0x7f79600a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:29.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.927+0000 7f7981e00700 1 --2- 172.21.15.1:0/988152073 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f79600a3cc0 0x7f79600a6180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f79680098e0 tx=0x7f7968005960 comp rx=0 tx=0).stop 2024-04-26T01:46:29.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.927+0000 7f7981e00700 1 -- 172.21.15.1:0/988152073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f797c06a750 msgr2=0x7f797c199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:29.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.927+0000 7f7981e00700 1 --2- 172.21.15.1:0/988152073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f797c06a750 0x7f797c199160 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f796c00e9c0 tx=0x7f796c00ecd0 comp rx=0 tx=0).stop 2024-04-26T01:46:29.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.927+0000 7f7981e00700 1 -- 172.21.15.1:0/988152073 shutdown_connections 2024-04-26T01:46:29.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.927+0000 7f7981e00700 1 --2- 172.21.15.1:0/988152073 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f79600a3cc0 0x7f79600a6180 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:29.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.927+0000 7f7981e00700 1 --2- 172.21.15.1:0/988152073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f797c108a30 0x7f797c19d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:29.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.927+0000 7f7981e00700 1 --2- 172.21.15.1:0/988152073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f797c06a750 0x7f797c199160 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:29.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.927+0000 7f7981e00700 1 --2- 172.21.15.1:0/988152073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f797c06b0f0 0x7f797c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:29.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.927+0000 7f7981e00700 1 -- 172.21.15.1:0/988152073 >> 172.21.15.1:0/988152073 conn(0x7f797c0788f0 msgr2=0x7f797c0ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:29.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.927+0000 7f7981e00700 1 -- 172.21.15.1:0/988152073 shutdown_connections 2024-04-26T01:46:29.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:29.928+0000 7f7981e00700 1 -- 172.21.15.1:0/988152073 wait complete. 2024-04-26T01:46:29.926 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 2 2024-04-26T01:46:30.205 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 26 01:46:29 smithi173 ceph-mds[62498]: -- 172.21.15.173:0/313592927 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558bb3ab9a40 con 0x558bb3a8c800 2024-04-26T01:46:30.206 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 26 01:46:29 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-l[62494]: starting mds.l at 2024-04-26T01:46:30.206 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 26 01:46:29 smithi173 bash[62479]: 4d65dc01c89eb0fb95c08fb97efdbdc23fac9402ff42a2492d0ee1333cec3d0d 2024-04-26T01:46:30.206 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 26 01:46:29 smithi173 systemd[1]: Started Ceph mds.l for 77cfcc80-036d-11ef-bc93-c7b262605968. 2024-04-26T01:46:30.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:30 smithi097 ceph-mon[32853]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:30 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.173:6832/188689332,v1:172.21.15.173:6833/188689332] up:boot 2024-04-26T01:46:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:30 smithi097 ceph-mon[32853]: fsmap 1 up:standby 2024-04-26T01:46:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:30 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T01:46:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:30 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/988152073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:46:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:30 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:30 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:30 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:30 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:30 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:30 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:30.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:30 smithi173 ceph-mon[32553]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:30.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:30 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.173:6832/188689332,v1:172.21.15.173:6833/188689332] up:boot 2024-04-26T01:46:30.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:30 smithi173 ceph-mon[32553]: fsmap 1 up:standby 2024-04-26T01:46:30.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:30 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T01:46:30.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:30 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/988152073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:46:30.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:30 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:30.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:30 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:30.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:30 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:30.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:30 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:30.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:30 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:30.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:30 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:30.608 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:30 smithi001 ceph-mon[27822]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:30.608 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:30 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.173:6832/188689332,v1:172.21.15.173:6833/188689332] up:boot 2024-04-26T01:46:30.608 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:30 smithi001 ceph-mon[27822]: fsmap 1 up:standby 2024-04-26T01:46:30.608 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:30 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T01:46:30.609 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:30 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/988152073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:46:30.609 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:30 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:30.609 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:30 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:30.609 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:30 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:30.609 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:30 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:30.609 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:30 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:30.609 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:30 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:31.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:31 smithi097 ceph-mon[32853]: Deploying daemon mds.k on smithi097 2024-04-26T01:46:31.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:31 smithi173 ceph-mon[32553]: Deploying daemon mds.k on smithi097 2024-04-26T01:46:31.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:31 smithi001 ceph-mon[27822]: Deploying daemon mds.k on smithi097 2024-04-26T01:46:32.218 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 26 01:46:32 smithi097 systemd[1]: Starting Ceph mds.k for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T01:46:32.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:32 smithi097 ceph-mon[32853]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:32.483 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 26 01:46:32 smithi097 podman[61533]: 2024-04-26 01:46:32.247697836 +0000 UTC m=+0.017950145 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:46:32.483 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 26 01:46:32 smithi097 podman[61533]: 2024-04-26 01:46:32.349336348 +0000 UTC m=+0.119588720 container create 4fb93c802074b43b52277bd6a8b5ac10c84706ef22d1ffe6c209a81e0d83e797 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T01:46:32.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:32 smithi173 ceph-mon[32553]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:32.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:32 smithi001 ceph-mon[27822]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:32.670 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:46:32.983 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 26 01:46:32 smithi097 podman[61533]: 2024-04-26 01:46:32.602713641 +0000 UTC m=+0.372965941 container init 4fb93c802074b43b52277bd6a8b5ac10c84706ef22d1ffe6c209a81e0d83e797 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T01:46:32.983 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 26 01:46:32 smithi097 podman[61533]: 2024-04-26 01:46:32.6051434 +0000 UTC m=+0.375395698 container start 4fb93c802074b43b52277bd6a8b5ac10c84706ef22d1ffe6c209a81e0d83e797 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-26T01:46:32.983 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 26 01:46:32 smithi097 ceph-mds[61552]: -- 172.21.15.97:0/2594596254 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c662b45a40 con 0x55c662b18c00 2024-04-26T01:46:32.983 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 26 01:46:32 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-k[61548]: starting mds.k at 2024-04-26T01:46:32.983 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 26 01:46:32 smithi097 bash[61533]: 4fb93c802074b43b52277bd6a8b5ac10c84706ef22d1ffe6c209a81e0d83e797 2024-04-26T01:46:32.983 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 26 01:46:32 smithi097 systemd[1]: Started Ceph mds.k for 77cfcc80-036d-11ef-bc93-c7b262605968. 2024-04-26T01:46:33.132 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:46:33.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:33 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:33.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:33 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:33.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:33 smithi001 ceph-mon[27822]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:33.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:33 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:33.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:33 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:33.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:33 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:33.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:33 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:33.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:33 smithi001 ceph-mon[27822]: Deploying daemon mds.j on smithi001 2024-04-26T01:46:33.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:33 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.97:6832/4061367802,v1:172.21.15.97:6833/4061367802] up:boot 2024-04-26T01:46:33.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:33 smithi001 ceph-mon[27822]: fsmap 2 up:standby 2024-04-26T01:46:33.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:33 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T01:46:34.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:33 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:34.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:33 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:34.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:33 smithi173 ceph-mon[32553]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:34.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:33 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:34.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:33 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:34.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:33 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:34.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:33 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:34.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:33 smithi173 ceph-mon[32553]: Deploying daemon mds.j on smithi001 2024-04-26T01:46:34.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:33 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.97:6832/4061367802,v1:172.21.15.97:6833/4061367802] up:boot 2024-04-26T01:46:34.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:33 smithi173 ceph-mon[32553]: fsmap 2 up:standby 2024-04-26T01:46:34.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:33 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T01:46:34.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:33 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:34.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:33 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:34.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:33 smithi097 ceph-mon[32853]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:34.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:33 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:34.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:33 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:34.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:33 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:34.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:33 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:34.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:33 smithi097 ceph-mon[32853]: Deploying daemon mds.j on smithi001 2024-04-26T01:46:34.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:33 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.97:6832/4061367802,v1:172.21.15.97:6833/4061367802] up:boot 2024-04-26T01:46:34.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:33 smithi097 ceph-mon[32853]: fsmap 2 up:standby 2024-04-26T01:46:34.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:33 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T01:46:35.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.505+0000 7fbbb7000700 1 -- 172.21.15.1:0/423657227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbb0077890 msgr2=0x7fbbb01133e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:35.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.505+0000 7fbbb7000700 1 --2- 172.21.15.1:0/423657227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbb0077890 0x7fbbb01133e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009990 tx=0x7fbb98009ca0 comp rx=0 tx=0).stop 2024-04-26T01:46:35.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.505+0000 7fbbb7000700 1 -- 172.21.15.1:0/423657227 shutdown_connections 2024-04-26T01:46:35.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.505+0000 7fbbb7000700 1 --2- 172.21.15.1:0/423657227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbb0113920 0x7fbbb0115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:35.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.505+0000 7fbbb7000700 1 --2- 172.21.15.1:0/423657227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbb0076f70 0x7fbbb0077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:35.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.505+0000 7fbbb7000700 1 --2- 172.21.15.1:0/423657227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbb0077890 0x7fbbb01133e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:35.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.505+0000 7fbbb7000700 1 -- 172.21.15.1:0/423657227 >> 172.21.15.1:0/423657227 conn(0x7fbbb0100770 msgr2=0x7fbbb0102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:35.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.506+0000 7fbbb7000700 1 -- 172.21.15.1:0/423657227 shutdown_connections 2024-04-26T01:46:35.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.506+0000 7fbbb7000700 1 -- 172.21.15.1:0/423657227 wait complete. 2024-04-26T01:46:35.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.507+0000 7fbbb7000700 1 Processor -- start 2024-04-26T01:46:35.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.507+0000 7fbbb7000700 1 -- start start 2024-04-26T01:46:35.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.507+0000 7fbbb7000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbb0076f70 0x7fbbb01a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:35.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.507+0000 7fbbb7000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbb0077890 0x7fbbb01a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:35.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.507+0000 7fbbb7000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbb0113920 0x7fbbb019f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:35.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.507+0000 7fbbb7000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb01192f0 con 0x7fbbb0076f70 2024-04-26T01:46:35.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.507+0000 7fbbb7000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0119170 con 0x7fbbb0113920 2024-04-26T01:46:35.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.507+0000 7fbbb7000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0119470 con 0x7fbbb0077890 2024-04-26T01:46:35.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.507+0000 7fbbafe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbb0076f70 0x7fbbb01a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:35.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.508+0000 7fbbafe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbb0076f70 0x7fbbb01a5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60118/0 (socket says 172.21.15.1:60118) 2024-04-26T01:46:35.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.508+0000 7fbbafe00700 1 -- 172.21.15.1:0/4239041237 learned_addr learned my addr 172.21.15.1:0/4239041237 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:46:35.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.508+0000 7fbba7400700 1 --2- 172.21.15.1:0/4239041237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbb0077890 0x7fbbb01a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:35.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.508+0000 7fbbb5000700 1 --2- 172.21.15.1:0/4239041237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbb0113920 0x7fbbb019f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:35.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.508+0000 7fbba7400700 1 -- 172.21.15.1:0/4239041237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbb0113920 msgr2=0x7fbbb019f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:35.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.508+0000 7fbba7400700 1 --2- 172.21.15.1:0/4239041237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbb0113920 0x7fbbb019f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:35.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.508+0000 7fbba7400700 1 -- 172.21.15.1:0/4239041237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbb0076f70 msgr2=0x7fbbb01a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:35.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.508+0000 7fbba7400700 1 --2- 172.21.15.1:0/4239041237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbb0076f70 0x7fbbb01a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:35.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.508+0000 7fbba7400700 1 -- 172.21.15.1:0/4239041237 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb019fd30 con 0x7fbbb0077890 2024-04-26T01:46:35.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.508+0000 7fbba7400700 1 --2- 172.21.15.1:0/4239041237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbb0077890 0x7fbbb01a5950 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbb980098e0 tx=0x7fbb980057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:35.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.509+0000 7fbbad600700 1 -- 172.21.15.1:0/4239041237 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98027070 con 0x7fbbb0077890 2024-04-26T01:46:35.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.509+0000 7fbbb7000700 1 -- 172.21.15.1:0/4239041237 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb019ffc0 con 0x7fbbb0077890 2024-04-26T01:46:35.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.509+0000 7fbbad600700 1 -- 172.21.15.1:0/4239041237 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb98022470 con 0x7fbbb0077890 2024-04-26T01:46:35.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.509+0000 7fbbad600700 1 -- 172.21.15.1:0/4239041237 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98017650 con 0x7fbbb0077890 2024-04-26T01:46:35.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.509+0000 7fbbb7000700 1 -- 172.21.15.1:0/4239041237 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb01ac2e0 con 0x7fbbb0077890 2024-04-26T01:46:35.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.510+0000 7fbbb7000700 1 -- 172.21.15.1:0/4239041237 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbb0069730 con 0x7fbbb0077890 2024-04-26T01:46:35.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.512+0000 7fbbad600700 1 -- 172.21.15.1:0/4239041237 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fbb9802a070 con 0x7fbbb0077890 2024-04-26T01:46:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.513+0000 7fbbad600700 1 --2- 172.21.15.1:0/4239041237 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbb940a3c40 0x7fbb940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.514+0000 7fbbad600700 1 -- 172.21.15.1:0/4239041237 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fbb980d0e90 con 0x7fbbb0077890 2024-04-26T01:46:35.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.514+0000 7fbbafe00700 1 --2- 172.21.15.1:0/4239041237 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbb940a3c40 0x7fbb940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:35.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.514+0000 7fbbafe00700 1 --2- 172.21.15.1:0/4239041237 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbb940a3c40 0x7fbb940a6100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009c10 tx=0x7fbba000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:35.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.516+0000 7fbbad600700 1 -- 172.21.15.1:0/4239041237 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbb98031080 con 0x7fbbb0077890 2024-04-26T01:46:35.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.799+0000 7fbbb7000700 1 -- 172.21.15.1:0/4239041237 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbb01a0b00 con 0x7fbbb0077890 2024-04-26T01:46:35.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.800+0000 7fbbad600700 1 -- 172.21.15.1:0/4239041237 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7fbb980d08c0 con 0x7fbbb0077890 2024-04-26T01:46:35.798 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:46:35.798 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T01:46:35.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.803+0000 7fbbb7000700 1 -- 172.21.15.1:0/4239041237 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbb940a3c40 msgr2=0x7fbb940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:35.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.803+0000 7fbbb7000700 1 --2- 172.21.15.1:0/4239041237 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbb940a3c40 0x7fbb940a6100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009c10 tx=0x7fbba000b040 comp rx=0 tx=0).stop 2024-04-26T01:46:35.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.803+0000 7fbbb7000700 1 -- 172.21.15.1:0/4239041237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbb0077890 msgr2=0x7fbbb01a5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:35.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.803+0000 7fbbb7000700 1 --2- 172.21.15.1:0/4239041237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbb0077890 0x7fbbb01a5950 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbb980098e0 tx=0x7fbb980057c0 comp rx=0 tx=0).stop 2024-04-26T01:46:35.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.803+0000 7fbbb7000700 1 -- 172.21.15.1:0/4239041237 shutdown_connections 2024-04-26T01:46:35.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.803+0000 7fbbb7000700 1 --2- 172.21.15.1:0/4239041237 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbb940a3c40 0x7fbb940a6100 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:35.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.803+0000 7fbbb7000700 1 --2- 172.21.15.1:0/4239041237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbb0113920 0x7fbbb019f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:35.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.803+0000 7fbbb7000700 1 --2- 172.21.15.1:0/4239041237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbb0076f70 0x7fbbb01a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:35.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.803+0000 7fbbb7000700 1 --2- 172.21.15.1:0/4239041237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbb0077890 0x7fbbb01a5950 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:35.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.803+0000 7fbbb7000700 1 -- 172.21.15.1:0/4239041237 >> 172.21.15.1:0/4239041237 conn(0x7fbbb0100770 msgr2=0x7fbbb0114590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:35.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.803+0000 7fbbb7000700 1 -- 172.21.15.1:0/4239041237 shutdown_connections 2024-04-26T01:46:35.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:35.803+0000 7fbbb7000700 1 -- 172.21.15.1:0/4239041237 wait complete. 2024-04-26T01:46:35.802 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 3 2024-04-26T01:46:36.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:35 smithi001 ceph-mon[27822]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:36.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:35 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/4239041237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:46:36.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:35 smithi097 ceph-mon[32853]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:36.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:35 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/4239041237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:46:36.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:35 smithi173 ceph-mon[32553]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:36.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:35 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/4239041237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:46:37.379 INFO:journalctl@ceph.mds.j.smithi001.stdout:Apr 26 01:46:37 smithi001 systemd[1]: Starting Ceph mds.j for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T01:46:37.836 INFO:journalctl@ceph.mds.j.smithi001.stdout:Apr 26 01:46:37 smithi001 podman[65709]: 2024-04-26 01:46:37.454124006 +0000 UTC m=+0.033248173 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:46:37.836 INFO:journalctl@ceph.mds.j.smithi001.stdout:Apr 26 01:46:37 smithi001 podman[65709]: 2024-04-26 01:46:37.566347605 +0000 UTC m=+0.145471806 container create caccc4506c76456b2ec2cbd0a9968e42a1e3883bb0949749343dd355b715bf67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-26T01:46:38.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:37 smithi173 ceph-mon[32553]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:38.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:37 smithi001 ceph-mon[27822]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:38.130 INFO:journalctl@ceph.mds.j.smithi001.stdout:Apr 26 01:46:37 smithi001 podman[65709]: 2024-04-26 01:46:37.862234611 +0000 UTC m=+0.441358815 container init caccc4506c76456b2ec2cbd0a9968e42a1e3883bb0949749343dd355b715bf67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-26T01:46:38.130 INFO:journalctl@ceph.mds.j.smithi001.stdout:Apr 26 01:46:37 smithi001 podman[65709]: 2024-04-26 01:46:37.873908773 +0000 UTC m=+0.453032978 container start caccc4506c76456b2ec2cbd0a9968e42a1e3883bb0949749343dd355b715bf67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-j, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-26T01:46:38.130 INFO:journalctl@ceph.mds.j.smithi001.stdout:Apr 26 01:46:37 smithi001 ceph-mds[65727]: -- 172.21.15.1:0/2857917608 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b9c2c07a40 con 0x55b9c2bdac00 2024-04-26T01:46:38.130 INFO:journalctl@ceph.mds.j.smithi001.stdout:Apr 26 01:46:37 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-j[65723]: starting mds.j at 2024-04-26T01:46:38.130 INFO:journalctl@ceph.mds.j.smithi001.stdout:Apr 26 01:46:37 smithi001 bash[65709]: caccc4506c76456b2ec2cbd0a9968e42a1e3883bb0949749343dd355b715bf67 2024-04-26T01:46:38.130 INFO:journalctl@ceph.mds.j.smithi001.stdout:Apr 26 01:46:37 smithi001 systemd[1]: Started Ceph mds.j for 77cfcc80-036d-11ef-bc93-c7b262605968. 2024-04-26T01:46:38.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:37 smithi097 ceph-mon[32853]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:38.958 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:46:39.360 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:39.360 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:39.360 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:39.360 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:39.361 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:39.361 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:39.361 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:39 smithi001 ceph-mon[27822]: Deploying daemon mds.d on smithi001 2024-04-26T01:46:39.361 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:39 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.1:6834/1477554934,v1:172.21.15.1:6835/1477554934] up:boot 2024-04-26T01:46:39.361 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:39 smithi001 ceph-mon[27822]: fsmap 3 up:standby 2024-04-26T01:46:39.361 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:39 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T01:46:39.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:39.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:39.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:39.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:39.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:39.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:39.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:39 smithi097 ceph-mon[32853]: Deploying daemon mds.d on smithi001 2024-04-26T01:46:39.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:39 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.1:6834/1477554934,v1:172.21.15.1:6835/1477554934] up:boot 2024-04-26T01:46:39.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:39 smithi097 ceph-mon[32853]: fsmap 3 up:standby 2024-04-26T01:46:39.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:39 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T01:46:39.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:39.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:39.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:39.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:39.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:39.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:39.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:39 smithi173 ceph-mon[32553]: Deploying daemon mds.d on smithi001 2024-04-26T01:46:39.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:39 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.1:6834/1477554934,v1:172.21.15.1:6835/1477554934] up:boot 2024-04-26T01:46:39.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:39 smithi173 ceph-mon[32553]: fsmap 3 up:standby 2024-04-26T01:46:39.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:39 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T01:46:40.357 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:40 smithi001 ceph-mon[27822]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:40.373 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:46:40.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:40 smithi097 ceph-mon[32853]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:40.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:40 smithi173 ceph-mon[32553]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:41.428 INFO:journalctl@ceph.mds.d.smithi001.stdout:Apr 26 01:46:41 smithi001 systemd[1]: Starting Ceph mds.d for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T01:46:42.327 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:42 smithi001 ceph-mon[27822]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:42.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:42 smithi097 ceph-mon[32853]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:42.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:42 smithi173 ceph-mon[32553]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:42.581 INFO:journalctl@ceph.mds.d.smithi001.stdout:Apr 26 01:46:42 smithi001 podman[66120]: 2024-04-26 01:46:42.235044167 +0000 UTC m=+0.039688641 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:46:42.581 INFO:journalctl@ceph.mds.d.smithi001.stdout:Apr 26 01:46:42 smithi001 podman[66120]: 2024-04-26 01:46:42.363106913 +0000 UTC m=+0.167751424 container create 49c38e895f49913864012419e786531b53e2f3fef010703f782157ba65c4c2b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T01:46:42.880 INFO:journalctl@ceph.mds.d.smithi001.stdout:Apr 26 01:46:42 smithi001 podman[66120]: 2024-04-26 01:46:42.686346993 +0000 UTC m=+0.490991511 container init 49c38e895f49913864012419e786531b53e2f3fef010703f782157ba65c4c2b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T01:46:42.880 INFO:journalctl@ceph.mds.d.smithi001.stdout:Apr 26 01:46:42 smithi001 podman[66120]: 2024-04-26 01:46:42.695859705 +0000 UTC m=+0.500504189 container start 49c38e895f49913864012419e786531b53e2f3fef010703f782157ba65c4c2b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-d, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T01:46:42.880 INFO:journalctl@ceph.mds.d.smithi001.stdout:Apr 26 01:46:42 smithi001 ceph-mds[66145]: -- 172.21.15.1:0/1124601029 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564946e9fa40 con 0x564946e73000 2024-04-26T01:46:42.880 INFO:journalctl@ceph.mds.d.smithi001.stdout:Apr 26 01:46:42 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-d[66139]: starting mds.d at 2024-04-26T01:46:42.880 INFO:journalctl@ceph.mds.d.smithi001.stdout:Apr 26 01:46:42 smithi001 bash[66120]: 49c38e895f49913864012419e786531b53e2f3fef010703f782157ba65c4c2b7 2024-04-26T01:46:42.880 INFO:journalctl@ceph.mds.d.smithi001.stdout:Apr 26 01:46:42 smithi001 systemd[1]: Started Ceph mds.d for 77cfcc80-036d-11ef-bc93-c7b262605968. 2024-04-26T01:46:42.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.756+0000 7f8868c00700 1 -- 172.21.15.1:0/1415120039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8864074e70 msgr2=0x7f8864075250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:42.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.756+0000 7f8868c00700 1 --2- 172.21.15.1:0/1415120039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8864074e70 0x7f8864075250 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8854009990 tx=0x7f8854009ca0 comp rx=0 tx=0).stop 2024-04-26T01:46:42.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.756+0000 7f8868c00700 1 -- 172.21.15.1:0/1415120039 shutdown_connections 2024-04-26T01:46:42.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.756+0000 7f8868c00700 1 --2- 172.21.15.1:0/1415120039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f886410f1b0 0x7f8864111620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:42.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.756+0000 7f8868c00700 1 --2- 172.21.15.1:0/1415120039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8864074e70 0x7f8864075250 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:42.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.757+0000 7f8868c00700 1 --2- 172.21.15.1:0/1415120039 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8864075790 0x7f886410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:42.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.757+0000 7f8868c00700 1 -- 172.21.15.1:0/1415120039 >> 172.21.15.1:0/1415120039 conn(0x7f88640fe4c0 msgr2=0x7f88641008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:42.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.757+0000 7f8868c00700 1 -- 172.21.15.1:0/1415120039 shutdown_connections 2024-04-26T01:46:42.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.757+0000 7f8868c00700 1 -- 172.21.15.1:0/1415120039 wait complete. 2024-04-26T01:46:42.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.758+0000 7f8868c00700 1 Processor -- start 2024-04-26T01:46:42.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.758+0000 7f8868c00700 1 -- start start 2024-04-26T01:46:42.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.758+0000 7f8868c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8864074e70 0x7f886419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:42.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.758+0000 7f8868c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8864075790 0x7f886419b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:42.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.758+0000 7f8868c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f886410f1b0 0x7f886419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:42.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.758+0000 7f8868c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8864114da0 con 0x7f886410f1b0 2024-04-26T01:46:42.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.758+0000 7f8868c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8864114c20 con 0x7f8864074e70 2024-04-26T01:46:42.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.758+0000 7f8868c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8864114f20 con 0x7f8864075790 2024-04-26T01:46:42.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.758+0000 7f8863e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f886410f1b0 0x7f886419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:42.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.759+0000 7f8863e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f886410f1b0 0x7f886419fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46756/0 (socket says 172.21.15.1:46756) 2024-04-26T01:46:42.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.759+0000 7f8863400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8864074e70 0x7f886419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:42.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.759+0000 7f8863e00700 1 -- 172.21.15.1:0/3177440555 learned_addr learned my addr 172.21.15.1:0/3177440555 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:46:42.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.759+0000 7f8862a00700 1 --2- 172.21.15.1:0/3177440555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8864075790 0x7f886419b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:42.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.759+0000 7f8863e00700 1 -- 172.21.15.1:0/3177440555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8864075790 msgr2=0x7f886419b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:42.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.759+0000 7f8863e00700 1 --2- 172.21.15.1:0/3177440555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8864075790 0x7f886419b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:42.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.759+0000 7f8863e00700 1 -- 172.21.15.1:0/3177440555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8864074e70 msgr2=0x7f886419b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:42.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.759+0000 7f8863e00700 1 --2- 172.21.15.1:0/3177440555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8864074e70 0x7f886419b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:42.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.759+0000 7f8863e00700 1 -- 172.21.15.1:0/3177440555 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88641a02a0 con 0x7f886410f1b0 2024-04-26T01:46:42.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.759+0000 7f8863e00700 1 --2- 172.21.15.1:0/3177440555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f886410f1b0 0x7f886419fb60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f885c00ef30 tx=0x7f885c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:42.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.760+0000 7f885be00700 1 -- 172.21.15.1:0/3177440555 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c0092e0 con 0x7f886410f1b0 2024-04-26T01:46:42.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.760+0000 7f885be00700 1 -- 172.21.15.1:0/3177440555 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f885c004bb0 con 0x7f886410f1b0 2024-04-26T01:46:42.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.760+0000 7f8868c00700 1 -- 172.21.15.1:0/3177440555 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88641a0530 con 0x7f886410f1b0 2024-04-26T01:46:42.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.760+0000 7f885be00700 1 -- 172.21.15.1:0/3177440555 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c004020 con 0x7f886410f1b0 2024-04-26T01:46:42.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.760+0000 7f8868c00700 1 -- 172.21.15.1:0/3177440555 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88641a7e10 con 0x7f886410f1b0 2024-04-26T01:46:42.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.761+0000 7f8868c00700 1 -- 172.21.15.1:0/3177440555 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8864069730 con 0x7f886410f1b0 2024-04-26T01:46:42.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.763+0000 7f885be00700 1 -- 172.21.15.1:0/3177440555 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f885c01b030 con 0x7f886410f1b0 2024-04-26T01:46:42.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.764+0000 7f885be00700 1 --2- 172.21.15.1:0/3177440555 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f88480a3cc0 0x7f88480a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:42.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.765+0000 7f885be00700 1 -- 172.21.15.1:0/3177440555 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f885c0cf9b0 con 0x7f886410f1b0 2024-04-26T01:46:42.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.765+0000 7f8863400700 1 --2- 172.21.15.1:0/3177440555 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f88480a3cc0 0x7f88480a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:42.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.765+0000 7f8863400700 1 --2- 172.21.15.1:0/3177440555 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f88480a3cc0 0x7f88480a6180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8854006010 tx=0x7f885400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:42.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:42.770+0000 7f885be00700 1 -- 172.21.15.1:0/3177440555 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f885c022020 con 0x7f886410f1b0 2024-04-26T01:46:43.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:43.063+0000 7f8868c00700 1 -- 172.21.15.1:0/3177440555 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f886402df80 con 0x7f886410f1b0 2024-04-26T01:46:43.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:43.064+0000 7f885be00700 1 -- 172.21.15.1:0/3177440555 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3040 (secure 0 0 0) 0x7f885c010040 con 0x7f886410f1b0 2024-04-26T01:46:43.062 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:46:43.062 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T01:46:43.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:43.067+0000 7f8868c00700 1 -- 172.21.15.1:0/3177440555 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f88480a3cc0 msgr2=0x7f88480a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:43.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:43.067+0000 7f8868c00700 1 --2- 172.21.15.1:0/3177440555 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f88480a3cc0 0x7f88480a6180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8854006010 tx=0x7f885400b260 comp rx=0 tx=0).stop 2024-04-26T01:46:43.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:43.067+0000 7f8868c00700 1 -- 172.21.15.1:0/3177440555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f886410f1b0 msgr2=0x7f886419fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:43.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:43.067+0000 7f8868c00700 1 --2- 172.21.15.1:0/3177440555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f886410f1b0 0x7f886419fb60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f885c00ef30 tx=0x7f885c00c3a0 comp rx=0 tx=0).stop 2024-04-26T01:46:43.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:43.068+0000 7f8868c00700 1 -- 172.21.15.1:0/3177440555 shutdown_connections 2024-04-26T01:46:43.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:43.068+0000 7f8868c00700 1 --2- 172.21.15.1:0/3177440555 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f88480a3cc0 0x7f88480a6180 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:43.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:43.068+0000 7f8868c00700 1 --2- 172.21.15.1:0/3177440555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f886410f1b0 0x7f886419fb60 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:43.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:43.068+0000 7f8868c00700 1 --2- 172.21.15.1:0/3177440555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8864074e70 0x7f886419b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:43.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:43.068+0000 7f8868c00700 1 --2- 172.21.15.1:0/3177440555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8864075790 0x7f886419b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:43.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:43.068+0000 7f8868c00700 1 -- 172.21.15.1:0/3177440555 >> 172.21.15.1:0/3177440555 conn(0x7f88640fe4c0 msgr2=0x7f88640ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:43.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:43.068+0000 7f8868c00700 1 -- 172.21.15.1:0/3177440555 shutdown_connections 2024-04-26T01:46:43.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:43.068+0000 7f8868c00700 1 -- 172.21.15.1:0/3177440555 wait complete. 2024-04-26T01:46:43.067 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 4 2024-04-26T01:46:44.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:43 smithi001 ceph-mon[27822]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:44.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:43 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:44.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:43 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:44.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:43 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:44.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:43 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:44.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:43 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:44.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:43 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:44.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:43 smithi001 ceph-mon[27822]: Deploying daemon mds.e on smithi097 2024-04-26T01:46:44.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:43 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3177440555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:46:44.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:43 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.1:6836/76325266,v1:172.21.15.1:6837/76325266] up:boot 2024-04-26T01:46:44.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:43 smithi001 ceph-mon[27822]: fsmap 4 up:standby 2024-04-26T01:46:44.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:43 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T01:46:44.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:43 smithi097 ceph-mon[32853]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:44.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:43 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:44.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:43 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:44.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:43 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:44.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:43 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:44.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:43 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:44.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:43 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:44.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:43 smithi097 ceph-mon[32853]: Deploying daemon mds.e on smithi097 2024-04-26T01:46:44.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:43 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3177440555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:46:44.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:43 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.1:6836/76325266,v1:172.21.15.1:6837/76325266] up:boot 2024-04-26T01:46:44.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:43 smithi097 ceph-mon[32853]: fsmap 4 up:standby 2024-04-26T01:46:44.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:43 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T01:46:44.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:43 smithi173 ceph-mon[32553]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:44.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:43 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:44.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:43 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:44.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:43 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:44.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:43 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:44.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:43 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:44.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:43 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:44.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:43 smithi173 ceph-mon[32553]: Deploying daemon mds.e on smithi097 2024-04-26T01:46:44.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:43 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3177440555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:46:44.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:43 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.1:6836/76325266,v1:172.21.15.1:6837/76325266] up:boot 2024-04-26T01:46:44.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:43 smithi173 ceph-mon[32553]: fsmap 4 up:standby 2024-04-26T01:46:44.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:43 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T01:46:45.233 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 26 01:46:45 smithi097 systemd[1]: Starting Ceph mds.e for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T01:46:45.628 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 26 01:46:45 smithi097 podman[61850]: 2024-04-26 01:46:45.271477288 +0000 UTC m=+0.017567266 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:46:45.628 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 26 01:46:45 smithi097 podman[61850]: 2024-04-26 01:46:45.369877277 +0000 UTC m=+0.115967232 container create 1ae8bb4ee485c3913513c06db8e8f45b4a90065e03da8c35ba51878ef6f15b3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-e, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T01:46:45.628 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 26 01:46:45 smithi097 podman[61850]: 2024-04-26 01:46:45.605581638 +0000 UTC m=+0.351671593 container init 1ae8bb4ee485c3913513c06db8e8f45b4a90065e03da8c35ba51878ef6f15b3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-26T01:46:45.628 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 26 01:46:45 smithi097 podman[61850]: 2024-04-26 01:46:45.608256123 +0000 UTC m=+0.354346079 container start 1ae8bb4ee485c3913513c06db8e8f45b4a90065e03da8c35ba51878ef6f15b3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-26T01:46:45.628 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 26 01:46:45 smithi097 ceph-mds[61870]: -- 172.21.15.97:0/2411594980 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a52bd5fa40 con 0x55a52bd32800 2024-04-26T01:46:45.783 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:46:45.982 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 26 01:46:45 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-e[61866]: starting mds.e at 2024-04-26T01:46:45.983 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 26 01:46:45 smithi097 bash[61850]: 1ae8bb4ee485c3913513c06db8e8f45b4a90065e03da8c35ba51878ef6f15b3d 2024-04-26T01:46:45.983 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 26 01:46:45 smithi097 systemd[1]: Started Ceph mds.e for 77cfcc80-036d-11ef-bc93-c7b262605968. 2024-04-26T01:46:45.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:45 smithi097 ceph-mon[32853]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:45.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:45 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:45.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:45 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:45.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:45 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:45.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:45 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:45.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:45 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:45.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:45 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:46.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:45 smithi001 ceph-mon[27822]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:46.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:45 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:46.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:45 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:46.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:45 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:46.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:45 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:46.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:45 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:46.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:45 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:46.274 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:46:46.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:45 smithi173 ceph-mon[32553]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:46.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:45 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:46.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:45 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:46.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:45 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:46.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:45 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:46.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:45 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:46.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:45 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:47.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:46 smithi173 ceph-mon[32553]: Deploying daemon mds.f on smithi173 2024-04-26T01:46:47.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:46 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.97:6834/763675408,v1:172.21.15.97:6835/763675408] up:boot 2024-04-26T01:46:47.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:46 smithi173 ceph-mon[32553]: fsmap 5 up:standby 2024-04-26T01:46:47.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:46 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T01:46:47.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:46 smithi001 ceph-mon[27822]: Deploying daemon mds.f on smithi173 2024-04-26T01:46:47.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:46 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.97:6834/763675408,v1:172.21.15.97:6835/763675408] up:boot 2024-04-26T01:46:47.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:46 smithi001 ceph-mon[27822]: fsmap 5 up:standby 2024-04-26T01:46:47.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:46 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T01:46:47.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:46 smithi097 ceph-mon[32853]: Deploying daemon mds.f on smithi173 2024-04-26T01:46:47.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:46 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.97:6834/763675408,v1:172.21.15.97:6835/763675408] up:boot 2024-04-26T01:46:47.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:46 smithi097 ceph-mon[32853]: fsmap 5 up:standby 2024-04-26T01:46:47.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:46 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T01:46:48.058 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:47 smithi173 ceph-mon[32553]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:48.059 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:46:47 smithi173 systemd[1]: Starting Ceph mds.f for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T01:46:48.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:47 smithi001 ceph-mon[27822]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:48.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:47 smithi097 ceph-mon[32853]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:48.316 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:46:48 smithi173 podman[62797]: 2024-04-26 01:46:48.037185113 +0000 UTC m=+0.018680159 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:46:48.316 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:46:48 smithi173 podman[62797]: 2024-04-26 01:46:48.148584436 +0000 UTC m=+0.130079458 container create a584bc72024dd8819b3d444a54554a3b74d0dd4e9fdf5ed1f6690e11fac600a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T01:46:48.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.515+0000 7fe28aa00700 1 -- 172.21.15.1:0/2779541791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe284109e90 msgr2=0x7fe284111a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:48.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.515+0000 7fe28aa00700 1 --2- 172.21.15.1:0/2779541791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe284109e90 0x7fe284111a60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00b0d0 tx=0x7fe27c00b3e0 comp rx=0 tx=0).stop 2024-04-26T01:46:48.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.515+0000 7fe28aa00700 1 -- 172.21.15.1:0/2779541791 shutdown_connections 2024-04-26T01:46:48.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.515+0000 7fe28aa00700 1 --2- 172.21.15.1:0/2779541791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe284109e90 0x7fe284111a60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:48.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.515+0000 7fe28aa00700 1 --2- 172.21.15.1:0/2779541791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe284104db0 0x7fe284105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:48.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.515+0000 7fe28aa00700 1 --2- 172.21.15.1:0/2779541791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe284105760 0x7fe2841097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:48.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.515+0000 7fe28aa00700 1 -- 172.21.15.1:0/2779541791 >> 172.21.15.1:0/2779541791 conn(0x7fe284100620 msgr2=0x7fe284102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:48.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.515+0000 7fe28aa00700 1 -- 172.21.15.1:0/2779541791 shutdown_connections 2024-04-26T01:46:48.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.515+0000 7fe28aa00700 1 -- 172.21.15.1:0/2779541791 wait complete. 2024-04-26T01:46:48.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.516+0000 7fe28aa00700 1 Processor -- start 2024-04-26T01:46:48.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.517+0000 7fe28aa00700 1 -- start start 2024-04-26T01:46:48.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.517+0000 7fe28aa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe284104db0 0x7fe28419b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:48.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.517+0000 7fe28aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe284105760 0x7fe28419b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:48.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.517+0000 7fe28aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe284109e90 0x7fe28419fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:48.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.517+0000 7fe28aa00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe284114d20 con 0x7fe284105760 2024-04-26T01:46:48.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.517+0000 7fe28aa00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe284114ba0 con 0x7fe284109e90 2024-04-26T01:46:48.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.517+0000 7fe28aa00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe284114ea0 con 0x7fe284104db0 2024-04-26T01:46:48.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.517+0000 7fe289600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe284104db0 0x7fe28419b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:48.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.517+0000 7fe288c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe284105760 0x7fe28419b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:48.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.518+0000 7fe289600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe284104db0 0x7fe28419b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:37184/0 (socket says 172.21.15.1:37184) 2024-04-26T01:46:48.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.518+0000 7fe289600700 1 -- 172.21.15.1:0/973732551 learned_addr learned my addr 172.21.15.1:0/973732551 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:46:48.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.518+0000 7fe28a000700 1 --2- 172.21.15.1:0/973732551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe284109e90 0x7fe28419fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:48.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.518+0000 7fe289600700 1 -- 172.21.15.1:0/973732551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe284109e90 msgr2=0x7fe28419fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:48.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.518+0000 7fe289600700 1 --2- 172.21.15.1:0/973732551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe284109e90 0x7fe28419fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:48.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.518+0000 7fe289600700 1 -- 172.21.15.1:0/973732551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe284105760 msgr2=0x7fe28419b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:48.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.518+0000 7fe289600700 1 --2- 172.21.15.1:0/973732551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe284105760 0x7fe28419b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:48.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.518+0000 7fe289600700 1 -- 172.21.15.1:0/973732551 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2841a01c0 con 0x7fe284104db0 2024-04-26T01:46:48.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.518+0000 7fe288c00700 1 --2- 172.21.15.1:0/973732551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe284105760 0x7fe28419b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:46:48.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.518+0000 7fe289600700 1 --2- 172.21.15.1:0/973732551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe284104db0 0x7fe28419b1b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe27800e9a0 tx=0x7fe27800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:48.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.519+0000 7fe282000700 1 -- 172.21.15.1:0/973732551 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27800cbd0 con 0x7fe284104db0 2024-04-26T01:46:48.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.519+0000 7fe282000700 1 -- 172.21.15.1:0/973732551 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe27800cd70 con 0x7fe284104db0 2024-04-26T01:46:48.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.519+0000 7fe28aa00700 1 -- 172.21.15.1:0/973732551 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2841a04b0 con 0x7fe284104db0 2024-04-26T01:46:48.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.519+0000 7fe282000700 1 -- 172.21.15.1:0/973732551 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe278018680 con 0x7fe284104db0 2024-04-26T01:46:48.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.519+0000 7fe28aa00700 1 -- 172.21.15.1:0/973732551 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2841a7d30 con 0x7fe284104db0 2024-04-26T01:46:48.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.521+0000 7fe28aa00700 1 -- 172.21.15.1:0/973732551 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe284069730 con 0x7fe284104db0 2024-04-26T01:46:48.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.522+0000 7fe282000700 1 -- 172.21.15.1:0/973732551 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fe278010430 con 0x7fe284104db0 2024-04-26T01:46:48.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.524+0000 7fe282000700 1 --2- 172.21.15.1:0/973732551 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe26c0a3cc0 0x7fe26c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:48.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.524+0000 7fe282000700 1 -- 172.21.15.1:0/973732551 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fe2780d02b0 con 0x7fe284104db0 2024-04-26T01:46:48.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.524+0000 7fe288c00700 1 --2- 172.21.15.1:0/973732551 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe26c0a3cc0 0x7fe26c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:48.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.525+0000 7fe288c00700 1 --2- 172.21.15.1:0/973732551 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe26c0a3cc0 0x7fe26c0a6180 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe284103d80 tx=0x7fe270009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:48.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.528+0000 7fe282000700 1 -- 172.21.15.1:0/973732551 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe27809d090 con 0x7fe284104db0 2024-04-26T01:46:48.630 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:46:48 smithi173 podman[62797]: 2024-04-26 01:46:48.367810579 +0000 UTC m=+0.349305609 container init a584bc72024dd8819b3d444a54554a3b74d0dd4e9fdf5ed1f6690e11fac600a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-26T01:46:48.630 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:46:48 smithi173 podman[62797]: 2024-04-26 01:46:48.370725109 +0000 UTC m=+0.352220140 container start a584bc72024dd8819b3d444a54554a3b74d0dd4e9fdf5ed1f6690e11fac600a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T01:46:48.630 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:46:48 smithi173 ceph-mds[62816]: -- 172.21.15.173:0/1398684346 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56511d19ba40 con 0x56511d16e800 2024-04-26T01:46:48.631 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:46:48 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: starting mds.f at 2024-04-26T01:46:48.631 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:46:48 smithi173 bash[62797]: a584bc72024dd8819b3d444a54554a3b74d0dd4e9fdf5ed1f6690e11fac600a1 2024-04-26T01:46:48.631 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:46:48 smithi173 systemd[1]: Started Ceph mds.f for 77cfcc80-036d-11ef-bc93-c7b262605968. 2024-04-26T01:46:48.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.825+0000 7fe28aa00700 1 -- 172.21.15.1:0/973732551 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe28402df80 con 0x7fe284104db0 2024-04-26T01:46:48.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.826+0000 7fe282000700 1 -- 172.21.15.1:0/973732551 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4678 (secure 0 0 0) 0x7fe27809c990 con 0x7fe284104db0 2024-04-26T01:46:48.824 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:46:48.824 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T01:46:48.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.830+0000 7fe28aa00700 1 -- 172.21.15.1:0/973732551 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe26c0a3cc0 msgr2=0x7fe26c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:48.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.830+0000 7fe28aa00700 1 --2- 172.21.15.1:0/973732551 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe26c0a3cc0 0x7fe26c0a6180 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe284103d80 tx=0x7fe270009260 comp rx=0 tx=0).stop 2024-04-26T01:46:48.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.830+0000 7fe28aa00700 1 -- 172.21.15.1:0/973732551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe284104db0 msgr2=0x7fe28419b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:48.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.830+0000 7fe28aa00700 1 --2- 172.21.15.1:0/973732551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe284104db0 0x7fe28419b1b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe27800e9a0 tx=0x7fe27800ed60 comp rx=0 tx=0).stop 2024-04-26T01:46:48.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.830+0000 7fe28aa00700 1 -- 172.21.15.1:0/973732551 shutdown_connections 2024-04-26T01:46:48.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.830+0000 7fe28aa00700 1 --2- 172.21.15.1:0/973732551 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe26c0a3cc0 0x7fe26c0a6180 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:48.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.830+0000 7fe28aa00700 1 --2- 172.21.15.1:0/973732551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe284109e90 0x7fe28419fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:48.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.830+0000 7fe28aa00700 1 --2- 172.21.15.1:0/973732551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe284104db0 0x7fe28419b1b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:48.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.830+0000 7fe28aa00700 1 --2- 172.21.15.1:0/973732551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe284105760 0x7fe28419b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:48.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.830+0000 7fe28aa00700 1 -- 172.21.15.1:0/973732551 >> 172.21.15.1:0/973732551 conn(0x7fe284100620 msgr2=0x7fe284101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:48.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.830+0000 7fe28aa00700 1 -- 172.21.15.1:0/973732551 shutdown_connections 2024-04-26T01:46:48.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:48.830+0000 7fe28aa00700 1 -- 172.21.15.1:0/973732551 wait complete. 2024-04-26T01:46:48.830 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 6 2024-04-26T01:46:49.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:49 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:49.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:49 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:49.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:49 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:49.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:49 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:49.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:49 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:49.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:49 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:49.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:49 smithi173 ceph-mon[32553]: Deploying daemon mds.c on smithi173 2024-04-26T01:46:49.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:49 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/973732551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:46:49.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:49 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.173:6834/2952462325,v1:172.21.15.173:6835/2952462325] up:boot 2024-04-26T01:46:49.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:49 smithi173 ceph-mon[32553]: fsmap 6 up:standby 2024-04-26T01:46:49.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:49 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:46:49.828 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:49 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:49.828 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:49 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:49.828 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:49 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:49.828 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:49 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:49.828 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:49 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:49.828 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:49 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:49.828 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:49 smithi001 ceph-mon[27822]: Deploying daemon mds.c on smithi173 2024-04-26T01:46:49.828 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:49 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/973732551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:46:49.828 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:49 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.173:6834/2952462325,v1:172.21.15.173:6835/2952462325] up:boot 2024-04-26T01:46:49.828 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:49 smithi001 ceph-mon[27822]: fsmap 6 up:standby 2024-04-26T01:46:49.828 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:49 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:46:49.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:49 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:49.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:49 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:49.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:49 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:49.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:49 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:49.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:49 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:49.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:49 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:49.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:49 smithi097 ceph-mon[32853]: Deploying daemon mds.c on smithi173 2024-04-26T01:46:49.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:49 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/973732551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:46:49.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:49 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.173:6834/2952462325,v1:172.21.15.173:6835/2952462325] up:boot 2024-04-26T01:46:49.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:49 smithi097 ceph-mon[32853]: fsmap 6 up:standby 2024-04-26T01:46:49.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:49 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:46:50.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:50 smithi173 ceph-mon[32553]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:50.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:50 smithi001 ceph-mon[27822]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:50.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:50 smithi097 ceph-mon[32853]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:51.566 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 26 01:46:51 smithi173 systemd[1]: Started Ceph mds.c for 77cfcc80-036d-11ef-bc93-c7b262605968. 2024-04-26T01:46:51.566 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:46:52.044 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:46:52.534 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:52 smithi001 ceph-mon[27822]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:52.534 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:52.535 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:52.535 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:52.535 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:52.535 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:52.535 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:52.535 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:52 smithi001 ceph-mon[27822]: Deploying daemon mds.h on smithi097 2024-04-26T01:46:52.535 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:52 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] up:boot 2024-04-26T01:46:52.535 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:52 smithi001 ceph-mon[27822]: fsmap 7 up:standby 2024-04-26T01:46:52.535 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:52 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T01:46:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:52 smithi097 ceph-mon[32853]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:52 smithi097 ceph-mon[32853]: Deploying daemon mds.h on smithi097 2024-04-26T01:46:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:52 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] up:boot 2024-04-26T01:46:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:52 smithi097 ceph-mon[32853]: fsmap 7 up:standby 2024-04-26T01:46:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:52 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T01:46:52.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:52 smithi173 ceph-mon[32553]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:52.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:52.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:52.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:52.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:52.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:52.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:52.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:52 smithi173 ceph-mon[32553]: Deploying daemon mds.h on smithi097 2024-04-26T01:46:52.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:52 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] up:boot 2024-04-26T01:46:52.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:52 smithi173 ceph-mon[32553]: fsmap 7 up:standby 2024-04-26T01:46:52.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:52 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T01:46:53.733 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 26 01:46:53 smithi097 systemd[1]: Starting Ceph mds.h for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T01:46:54.033 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 26 01:46:53 smithi097 podman[62163]: 2024-04-26 01:46:53.687547088 +0000 UTC m=+0.017985067 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:46:54.033 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 26 01:46:53 smithi097 podman[62163]: 2024-04-26 01:46:53.789185653 +0000 UTC m=+0.119623631 container create 529af91b6a60ae00c159c33e66feb891349bb7e71cf025f0023208d0355aacb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-26T01:46:54.033 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 26 01:46:54 smithi097 podman[62163]: 2024-04-26 01:46:54.033398467 +0000 UTC m=+0.363836436 container init 529af91b6a60ae00c159c33e66feb891349bb7e71cf025f0023208d0355aacb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True) 2024-04-26T01:46:54.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.245+0000 7fdd7f000700 1 -- 172.21.15.1:0/1496635969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd78107980 msgr2=0x7fdd78107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:54.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.245+0000 7fdd7f000700 1 --2- 172.21.15.1:0/1496635969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd78107980 0x7fdd78107d60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009990 tx=0x7fdd68009ca0 comp rx=0 tx=0).stop 2024-04-26T01:46:54.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.245+0000 7fdd7f000700 1 -- 172.21.15.1:0/1496635969 shutdown_connections 2024-04-26T01:46:54.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.245+0000 7fdd7f000700 1 --2- 172.21.15.1:0/1496635969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd781022f0 0x7fdd7810e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:54.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.245+0000 7fdd7f000700 1 --2- 172.21.15.1:0/1496635969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd78107980 0x7fdd78107d60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:54.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.245+0000 7fdd7f000700 1 --2- 172.21.15.1:0/1496635969 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd78101930 0x7fdd78101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:54.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.245+0000 7fdd7f000700 1 -- 172.21.15.1:0/1496635969 >> 172.21.15.1:0/1496635969 conn(0x7fdd780788b0 msgr2=0x7fdd78078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:54.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.245+0000 7fdd7f000700 1 -- 172.21.15.1:0/1496635969 shutdown_connections 2024-04-26T01:46:54.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.245+0000 7fdd7f000700 1 -- 172.21.15.1:0/1496635969 wait complete. 2024-04-26T01:46:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.247+0000 7fdd7f000700 1 Processor -- start 2024-04-26T01:46:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.247+0000 7fdd7f000700 1 -- start start 2024-04-26T01:46:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.248+0000 7fdd7f000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd78101930 0x7fdd7819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.248+0000 7fdd7f000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd781022f0 0x7fdd7819b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.248+0000 7fdd7f000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd78107980 0x7fdd7819fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.248+0000 7fdd7f000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdd78111b20 con 0x7fdd78101930 2024-04-26T01:46:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.248+0000 7fdd7f000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdd781119a0 con 0x7fdd781022f0 2024-04-26T01:46:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.248+0000 7fdd7f000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdd78111ca0 con 0x7fdd78107980 2024-04-26T01:46:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.248+0000 7fdd77e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd78101930 0x7fdd7819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.248+0000 7fdd77e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd78101930 0x7fdd7819b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49200/0 (socket says 172.21.15.1:49200) 2024-04-26T01:46:54.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.248+0000 7fdd77e00700 1 -- 172.21.15.1:0/3837325970 learned_addr learned my addr 172.21.15.1:0/3837325970 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:46:54.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.248+0000 7fdd6f400700 1 --2- 172.21.15.1:0/3837325970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd781022f0 0x7fdd7819b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:54.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.248+0000 7fdd7d000700 1 --2- 172.21.15.1:0/3837325970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd78107980 0x7fdd7819fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:54.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.249+0000 7fdd6f400700 1 -- 172.21.15.1:0/3837325970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd78107980 msgr2=0x7fdd7819fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:54.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.249+0000 7fdd6f400700 1 --2- 172.21.15.1:0/3837325970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd78107980 0x7fdd7819fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:54.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.249+0000 7fdd6f400700 1 -- 172.21.15.1:0/3837325970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd78101930 msgr2=0x7fdd7819b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:54.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.249+0000 7fdd6f400700 1 --2- 172.21.15.1:0/3837325970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd78101930 0x7fdd7819b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:54.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.249+0000 7fdd6f400700 1 -- 172.21.15.1:0/3837325970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd781a02b0 con 0x7fdd781022f0 2024-04-26T01:46:54.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.249+0000 7fdd6f400700 1 --2- 172.21.15.1:0/3837325970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd781022f0 0x7fdd7819b7e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdd6000ef10 tx=0x7fdd6000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:54.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.249+0000 7fdd75600700 1 -- 172.21.15.1:0/3837325970 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6000c9c0 con 0x7fdd781022f0 2024-04-26T01:46:54.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.249+0000 7fdd7f000700 1 -- 172.21.15.1:0/3837325970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd781a0540 con 0x7fdd781022f0 2024-04-26T01:46:54.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.249+0000 7fdd75600700 1 -- 172.21.15.1:0/3837325970 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd60019070 con 0x7fdd781022f0 2024-04-26T01:46:54.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.249+0000 7fdd75600700 1 -- 172.21.15.1:0/3837325970 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd60014480 con 0x7fdd781022f0 2024-04-26T01:46:54.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.249+0000 7fdd7f000700 1 -- 172.21.15.1:0/3837325970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd781a7e20 con 0x7fdd781022f0 2024-04-26T01:46:54.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.251+0000 7fdd7f000700 1 -- 172.21.15.1:0/3837325970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd78069730 con 0x7fdd781022f0 2024-04-26T01:46:54.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.253+0000 7fdd75600700 1 -- 172.21.15.1:0/3837325970 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fdd60012050 con 0x7fdd781022f0 2024-04-26T01:46:54.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.254+0000 7fdd75600700 1 --2- 172.21.15.1:0/3837325970 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fdd5c0a3c40 0x7fdd5c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:54.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.254+0000 7fdd75600700 1 -- 172.21.15.1:0/3837325970 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fdd600cff80 con 0x7fdd781022f0 2024-04-26T01:46:54.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.255+0000 7fdd77e00700 1 --2- 172.21.15.1:0/3837325970 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fdd5c0a3c40 0x7fdd5c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:54.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.255+0000 7fdd77e00700 1 --2- 172.21.15.1:0/3837325970 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fdd5c0a3c40 0x7fdd5c0a6100 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdd68006010 tx=0x7fdd6800b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:54.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.259+0000 7fdd75600700 1 -- 172.21.15.1:0/3837325970 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd60010040 con 0x7fdd781022f0 2024-04-26T01:46:54.438 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 26 01:46:54 smithi097 podman[62163]: 2024-04-26 01:46:54.035924355 +0000 UTC m=+0.366362326 container start 529af91b6a60ae00c159c33e66feb891349bb7e71cf025f0023208d0355aacb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T01:46:54.439 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 26 01:46:54 smithi097 ceph-mds[62182]: -- 172.21.15.97:0/1278538140 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ca9dd9ba40 con 0x55ca9dd6f000 2024-04-26T01:46:54.439 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 26 01:46:54 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-h[62178]: starting mds.h at 2024-04-26T01:46:54.439 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 26 01:46:54 smithi097 bash[62163]: 529af91b6a60ae00c159c33e66feb891349bb7e71cf025f0023208d0355aacb5 2024-04-26T01:46:54.439 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 26 01:46:54 smithi097 systemd[1]: Started Ceph mds.h for 77cfcc80-036d-11ef-bc93-c7b262605968. 2024-04-26T01:46:54.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.537+0000 7fdd7f000700 1 -- 172.21.15.1:0/3837325970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd781a7ff0 con 0x7fdd781022f0 2024-04-26T01:46:54.535 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:54 smithi001 ceph-mon[27822]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:54.535 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:54 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:54.536 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:54 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:54.536 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:54 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:54.536 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:54 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:54.536 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:54 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:54.536 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:54 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:54.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.538+0000 7fdd75600700 1 -- 172.21.15.1:0/3837325970 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7159 (secure 0 0 0) 0x7fdd6009cd60 con 0x7fdd781022f0 2024-04-26T01:46:54.537 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:46:54.537 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2952462325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2952462325},{"type":"v1","addr":"172.21.15.173:6835","nonce":2952462325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-26T01:46:54.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.542+0000 7fdd7f000700 1 -- 172.21.15.1:0/3837325970 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fdd5c0a3c40 msgr2=0x7fdd5c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:54.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.543+0000 7fdd7f000700 1 --2- 172.21.15.1:0/3837325970 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fdd5c0a3c40 0x7fdd5c0a6100 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdd68006010 tx=0x7fdd6800b280 comp rx=0 tx=0).stop 2024-04-26T01:46:54.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.543+0000 7fdd7f000700 1 -- 172.21.15.1:0/3837325970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd781022f0 msgr2=0x7fdd7819b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:54.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.543+0000 7fdd7f000700 1 --2- 172.21.15.1:0/3837325970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd781022f0 0x7fdd7819b7e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdd6000ef10 tx=0x7fdd6000c3a0 comp rx=0 tx=0).stop 2024-04-26T01:46:54.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.543+0000 7fdd7f000700 1 -- 172.21.15.1:0/3837325970 shutdown_connections 2024-04-26T01:46:54.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.543+0000 7fdd7f000700 1 --2- 172.21.15.1:0/3837325970 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fdd5c0a3c40 0x7fdd5c0a6100 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:54.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.543+0000 7fdd7f000700 1 --2- 172.21.15.1:0/3837325970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd78107980 0x7fdd7819fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:54.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.543+0000 7fdd7f000700 1 --2- 172.21.15.1:0/3837325970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd78101930 0x7fdd7819b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:54.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.543+0000 7fdd7f000700 1 --2- 172.21.15.1:0/3837325970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd781022f0 0x7fdd7819b7e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:54.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.543+0000 7fdd7f000700 1 -- 172.21.15.1:0/3837325970 >> 172.21.15.1:0/3837325970 conn(0x7fdd780788b0 msgr2=0x7fdd7806af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:54.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.543+0000 7fdd7f000700 1 -- 172.21.15.1:0/3837325970 shutdown_connections 2024-04-26T01:46:54.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:54.543+0000 7fdd7f000700 1 -- 172.21.15.1:0/3837325970 wait complete. 2024-04-26T01:46:54.542 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 9 2024-04-26T01:46:54.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:54 smithi097 ceph-mon[32853]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:54.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:54 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:54.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:54 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:54.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:54 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:54.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:54 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:54.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:54 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:54.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:54 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:54.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:54 smithi173 ceph-mon[32553]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:54.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:54 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:54.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:54 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:54.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:54 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:54.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:54 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:54.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:54 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:54.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:54 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:55.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:55 smithi173 ceph-mon[32553]: Deploying daemon mds.i on smithi173 2024-04-26T01:46:55.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:55 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.97:6836/3607142398,v1:172.21.15.97:6837/3607142398] up:boot 2024-04-26T01:46:55.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:55 smithi173 ceph-mon[32553]: fsmap 8 up:standby 2024-04-26T01:46:55.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:55 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T01:46:55.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:55 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3837325970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:46:55.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:55 smithi001 ceph-mon[27822]: Deploying daemon mds.i on smithi173 2024-04-26T01:46:55.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:55 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.97:6836/3607142398,v1:172.21.15.97:6837/3607142398] up:boot 2024-04-26T01:46:55.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:55 smithi001 ceph-mon[27822]: fsmap 8 up:standby 2024-04-26T01:46:55.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:55 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T01:46:55.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:55 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3837325970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:46:55.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:55 smithi097 ceph-mon[32853]: Deploying daemon mds.i on smithi173 2024-04-26T01:46:55.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:55 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.97:6836/3607142398,v1:172.21.15.97:6837/3607142398] up:boot 2024-04-26T01:46:55.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:55 smithi097 ceph-mon[32853]: fsmap 8 up:standby 2024-04-26T01:46:55.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:55 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T01:46:55.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:55 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3837325970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:46:56.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:56 smithi173 ceph-mon[32553]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:56.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:56 smithi097 ceph-mon[32853]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:56.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:56 smithi001 ceph-mon[27822]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:56.912 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 26 01:46:56 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-i[63434]: starting mds.i at 2024-04-26T01:46:57.316 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 26 01:46:56 smithi173 bash[63419]: 3fcc96c7635733abd0d7cf43945a7ac6328d4f4254664dc508050d9af513ac27 2024-04-26T01:46:57.316 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 26 01:46:56 smithi173 systemd[1]: Started Ceph mds.i for 77cfcc80-036d-11ef-bc93-c7b262605968. 2024-04-26T01:46:57.354 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:46:57.832 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:46:58.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:57 smithi097 ceph-mon[32853]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:58.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:57 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:58.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:57 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:58.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:57 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:58.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:57 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:58.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:57 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:58.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:57 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:58.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:57 smithi097 ceph-mon[32853]: Deploying daemon mds.g on smithi001 2024-04-26T01:46:58.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:57 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] up:boot 2024-04-26T01:46:58.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:57 smithi097 ceph-mon[32853]: fsmap 9 up:standby 2024-04-26T01:46:58.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:57 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T01:46:58.290 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:57 smithi001 ceph-mon[27822]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:58.291 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:57 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:58.291 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:57 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:58.291 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:57 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:58.291 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:57 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:58.291 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:57 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:58.291 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:57 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:58.291 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:57 smithi001 ceph-mon[27822]: Deploying daemon mds.g on smithi001 2024-04-26T01:46:58.291 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:57 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] up:boot 2024-04-26T01:46:58.291 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:57 smithi001 ceph-mon[27822]: fsmap 9 up:standby 2024-04-26T01:46:58.291 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:57 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T01:46:58.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:57 smithi173 ceph-mon[32553]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:46:58.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:57 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:58.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:57 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:58.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:57 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:46:58.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:57 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:46:58.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:57 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:46:58.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:57 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:58.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:57 smithi173 ceph-mon[32553]: Deploying daemon mds.g on smithi001 2024-04-26T01:46:58.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:57 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] up:boot 2024-04-26T01:46:58.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:57 smithi173 ceph-mon[32553]: fsmap 9 up:standby 2024-04-26T01:46:58.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:57 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T01:47:00.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:46:59 smithi097 ceph-mon[32853]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:00.261 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:46:59 smithi001 ceph-mon[27822]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:00.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:46:59 smithi173 ceph-mon[32553]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:00.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.973+0000 7f408c600700 1 -- 172.21.15.1:0/1538993579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4084101d10 msgr2=0x7f4084105d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:00.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.973+0000 7f408c600700 1 --2- 172.21.15.1:0/1538993579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4084101d10 0x7f4084105d10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f40700099c0 tx=0x7f4070009cd0 comp rx=0 tx=0).stop 2024-04-26T01:47:00.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.973+0000 7f408c600700 1 -- 172.21.15.1:0/1538993579 shutdown_connections 2024-04-26T01:47:00.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.973+0000 7f408c600700 1 --2- 172.21.15.1:0/1538993579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40841063f0 0x7f408410dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:00.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.973+0000 7f408c600700 1 --2- 172.21.15.1:0/1538993579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4084101360 0x7f4084101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:00.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.973+0000 7f408c600700 1 --2- 172.21.15.1:0/1538993579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4084101d10 0x7f4084105d10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:00.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.973+0000 7f408c600700 1 -- 172.21.15.1:0/1538993579 >> 172.21.15.1:0/1538993579 conn(0x7f4084077fb0 msgr2=0x7f40840ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:00.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.974+0000 7f408c600700 1 -- 172.21.15.1:0/1538993579 shutdown_connections 2024-04-26T01:47:00.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.974+0000 7f408c600700 1 -- 172.21.15.1:0/1538993579 wait complete. 2024-04-26T01:47:00.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.975+0000 7f408c600700 1 Processor -- start 2024-04-26T01:47:00.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.975+0000 7f408c600700 1 -- start start 2024-04-26T01:47:00.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.975+0000 7f408c600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4084101360 0x7f4084199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:00.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.975+0000 7f408c600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4084101d10 0x7f4084199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:00.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.975+0000 7f408c600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40841063f0 0x7f408419d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:00.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.975+0000 7f408c600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4084111b10 con 0x7f40841063f0 2024-04-26T01:47:00.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.975+0000 7f408c600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4084111990 con 0x7f4084101d10 2024-04-26T01:47:00.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.975+0000 7f408c600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4084111c90 con 0x7f4084101360 2024-04-26T01:47:00.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.976+0000 7f4089200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4084101d10 0x7f4084199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:00.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.976+0000 7f4089c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4084101360 0x7f4084199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:00.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.976+0000 7f4089200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4084101d10 0x7f4084199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:40048/0 (socket says 172.21.15.1:40048) 2024-04-26T01:47:00.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.976+0000 7f4089c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4084101360 0x7f4084199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:45720/0 (socket says 172.21.15.1:45720) 2024-04-26T01:47:00.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.976+0000 7f4089200700 1 -- 172.21.15.1:0/1155148244 learned_addr learned my addr 172.21.15.1:0/1155148244 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:47:00.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.976+0000 7f408a600700 1 --2- 172.21.15.1:0/1155148244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40841063f0 0x7f408419d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:00.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.976+0000 7f4089200700 1 -- 172.21.15.1:0/1155148244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4084101360 msgr2=0x7f4084199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:00.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.976+0000 7f4089200700 1 --2- 172.21.15.1:0/1155148244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4084101360 0x7f4084199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.976+0000 7f4089200700 1 -- 172.21.15.1:0/1155148244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40841063f0 msgr2=0x7f408419d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.976+0000 7f4089200700 1 --2- 172.21.15.1:0/1155148244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40841063f0 0x7f408419d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.976+0000 7f4089200700 1 -- 172.21.15.1:0/1155148244 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408419e110 con 0x7f4084101d10 2024-04-26T01:47:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.976+0000 7f4089200700 1 --2- 172.21.15.1:0/1155148244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4084101d10 0x7f4084199690 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4070006010 tx=0x7f4070005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.980+0000 7f4082000700 1 -- 172.21.15.1:0/1155148244 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070026070 con 0x7f4084101d10 2024-04-26T01:47:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.980+0000 7f4082000700 1 -- 172.21.15.1:0/1155148244 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4070003c70 con 0x7f4084101d10 2024-04-26T01:47:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.980+0000 7f4082000700 1 -- 172.21.15.1:0/1155148244 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407000fb60 con 0x7f4084101d10 2024-04-26T01:47:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.980+0000 7f408c600700 1 -- 172.21.15.1:0/1155148244 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408419e3a0 con 0x7f4084101d10 2024-04-26T01:47:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.980+0000 7f408c600700 1 -- 172.21.15.1:0/1155148244 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40840fd350 con 0x7f4084101d10 2024-04-26T01:47:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.981+0000 7f408c600700 1 -- 172.21.15.1:0/1155148244 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40841027f0 con 0x7f4084101d10 2024-04-26T01:47:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.983+0000 7f4082000700 1 -- 172.21.15.1:0/1155148244 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f4070029070 con 0x7f4084101d10 2024-04-26T01:47:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.984+0000 7f4082000700 1 --2- 172.21.15.1:0/1155148244 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f406c0a3cc0 0x7f406c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.985+0000 7f4082000700 1 -- 172.21.15.1:0/1155148244 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f40700d05c0 con 0x7f4084101d10 2024-04-26T01:47:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.985+0000 7f4089c00700 1 --2- 172.21.15.1:0/1155148244 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f406c0a3cc0 0x7f406c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:00.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.985+0000 7f4089c00700 1 --2- 172.21.15.1:0/1155148244 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f406c0a3cc0 0x7f406c0a6180 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4078010d80 tx=0x7f4078012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:00.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:46:59.989+0000 7f4082000700 1 -- 172.21.15.1:0/1155148244 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4070013070 con 0x7f4084101d10 2024-04-26T01:47:00.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:00.282+0000 7f408c600700 1 -- 172.21.15.1:0/1155148244 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f408419a490 con 0x7f4084101d10 2024-04-26T01:47:00.790 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:47:00.790 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2952462325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2952462325},{"type":"v1","addr":"172.21.15.173:6835","nonce":2952462325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-26T01:47:00.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:00.283+0000 7f4082000700 1 -- 172.21.15.1:0/1155148244 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7986 (secure 0 0 0) 0x7f408419a490 con 0x7f4084101d10 2024-04-26T01:47:00.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:00.286+0000 7f408c600700 1 -- 172.21.15.1:0/1155148244 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f406c0a3cc0 msgr2=0x7f406c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:00.286+0000 7f408c600700 1 --2- 172.21.15.1:0/1155148244 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f406c0a3cc0 0x7f406c0a6180 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4078010d80 tx=0x7f4078012040 comp rx=0 tx=0).stop 2024-04-26T01:47:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:00.286+0000 7f408c600700 1 -- 172.21.15.1:0/1155148244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4084101d10 msgr2=0x7f4084199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:00.286+0000 7f408c600700 1 --2- 172.21.15.1:0/1155148244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4084101d10 0x7f4084199690 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4070006010 tx=0x7f4070005280 comp rx=0 tx=0).stop 2024-04-26T01:47:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:00.286+0000 7f408c600700 1 -- 172.21.15.1:0/1155148244 shutdown_connections 2024-04-26T01:47:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:00.286+0000 7f408c600700 1 --2- 172.21.15.1:0/1155148244 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f406c0a3cc0 0x7f406c0a6180 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:00.286+0000 7f408c600700 1 --2- 172.21.15.1:0/1155148244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40841063f0 0x7f408419d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:00.286+0000 7f408c600700 1 --2- 172.21.15.1:0/1155148244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4084101360 0x7f4084199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:00.286+0000 7f408c600700 1 --2- 172.21.15.1:0/1155148244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4084101d10 0x7f4084199690 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:00.286+0000 7f408c600700 1 -- 172.21.15.1:0/1155148244 >> 172.21.15.1:0/1155148244 conn(0x7f4084077fb0 msgr2=0x7f408410eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:00.287+0000 7f408c600700 1 -- 172.21.15.1:0/1155148244 shutdown_connections 2024-04-26T01:47:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:00.287+0000 7f408c600700 1 -- 172.21.15.1:0/1155148244 wait complete. 2024-04-26T01:47:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 10 2024-04-26T01:47:01.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:00 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1155148244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:47:01.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:00 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1155148244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:47:01.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:00 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1155148244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:47:02.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:01 smithi001 ceph-mon[27822]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:02.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:01 smithi173 ceph-mon[32553]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:01 smithi097 ceph-mon[32853]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:02.629 INFO:journalctl@ceph.mds.g.smithi001.stdout:Apr 26 01:47:02 smithi001 systemd[1]: Started Ceph mds.g for 77cfcc80-036d-11ef-bc93-c7b262605968. 2024-04-26T01:47:03.596 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:47:03.675 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:03.675 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:03.675 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:03.675 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:47:03.675 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:47:03.675 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:03.676 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:03 smithi001 ceph-mon[27822]: Deploying daemon mds.a on smithi001 2024-04-26T01:47:03.676 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:03 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.1:6838/42404130,v1:172.21.15.1:6839/42404130] up:boot 2024-04-26T01:47:03.676 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:03 smithi001 ceph-mon[27822]: fsmap 10 up:standby 2024-04-26T01:47:03.676 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T01:47:03.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:03.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:03.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:03.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:47:03.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:47:03.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:03.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:03 smithi097 ceph-mon[32853]: Deploying daemon mds.a on smithi001 2024-04-26T01:47:03.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:03 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.1:6838/42404130,v1:172.21.15.1:6839/42404130] up:boot 2024-04-26T01:47:03.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:03 smithi097 ceph-mon[32853]: fsmap 10 up:standby 2024-04-26T01:47:03.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T01:47:03.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:03.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:03.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:03.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:47:03.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:47:03.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:03.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:03 smithi173 ceph-mon[32553]: Deploying daemon mds.a on smithi001 2024-04-26T01:47:03.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:03 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.1:6838/42404130,v1:172.21.15.1:6839/42404130] up:boot 2024-04-26T01:47:03.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:03 smithi173 ceph-mon[32553]: fsmap 10 up:standby 2024-04-26T01:47:03.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T01:47:04.531 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:04 smithi001 ceph-mon[27822]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:04.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:04 smithi097 ceph-mon[32853]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:47:04.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:04 smithi173 ceph-mon[32553]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:06.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:06 smithi001 ceph-mon[27822]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:06.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:06 smithi097 ceph-mon[32853]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:06.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:06 smithi173 ceph-mon[32553]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:07.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.428+0000 7f9701e00700 1 -- 172.21.15.1:0/60649871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96fc076f70 msgr2=0x7f96fc077350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:07.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.428+0000 7f9701e00700 1 --2- 172.21.15.1:0/60649871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96fc076f70 0x7f96fc077350 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009990 tx=0x7f96ec009ca0 comp rx=0 tx=0).stop 2024-04-26T01:47:07.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.428+0000 7f9701e00700 1 -- 172.21.15.1:0/60649871 shutdown_connections 2024-04-26T01:47:07.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.428+0000 7f9701e00700 1 --2- 172.21.15.1:0/60649871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96fc1138f0 0x7f96fc115ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:07.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.428+0000 7f9701e00700 1 --2- 172.21.15.1:0/60649871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96fc076f70 0x7f96fc077350 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:07.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.428+0000 7f9701e00700 1 --2- 172.21.15.1:0/60649871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96fc077890 0x7f96fc1133b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:07.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.428+0000 7f9701e00700 1 -- 172.21.15.1:0/60649871 >> 172.21.15.1:0/60649871 conn(0x7f96fc100720 msgr2=0x7f96fc102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:07.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.428+0000 7f9701e00700 1 -- 172.21.15.1:0/60649871 shutdown_connections 2024-04-26T01:47:07.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.428+0000 7f9701e00700 1 -- 172.21.15.1:0/60649871 wait complete. 2024-04-26T01:47:07.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.429+0000 7f9701e00700 1 Processor -- start 2024-04-26T01:47:07.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.430+0000 7f9701e00700 1 -- start start 2024-04-26T01:47:07.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.430+0000 7f9701e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96fc076f70 0x7f96fc19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:07.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.430+0000 7f9701e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96fc077890 0x7f96fc19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:07.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.430+0000 7f9701e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96fc1138f0 0x7f96fc1a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:07.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.430+0000 7f9701e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc1192b0 con 0x7f96fc077890 2024-04-26T01:47:07.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.430+0000 7f9701e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc119130 con 0x7f96fc076f70 2024-04-26T01:47:07.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.430+0000 7f9701e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc119430 con 0x7f96fc1138f0 2024-04-26T01:47:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.431+0000 7f96faa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96fc076f70 0x7f96fc19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.431+0000 7f96faa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96fc076f70 0x7f96fc19f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:38470/0 (socket says 172.21.15.1:38470) 2024-04-26T01:47:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.431+0000 7f96faa00700 1 -- 172.21.15.1:0/1920713818 learned_addr learned my addr 172.21.15.1:0/1920713818 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:47:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.431+0000 7f96fa000700 1 --2- 172.21.15.1:0/1920713818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96fc077890 0x7f96fc19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.431+0000 7f96fb400700 1 --2- 172.21.15.1:0/1920713818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96fc1138f0 0x7f96fc1a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.431+0000 7f96faa00700 1 -- 172.21.15.1:0/1920713818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96fc1138f0 msgr2=0x7f96fc1a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.431+0000 7f96faa00700 1 --2- 172.21.15.1:0/1920713818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96fc1138f0 0x7f96fc1a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.431+0000 7f96faa00700 1 -- 172.21.15.1:0/1920713818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96fc077890 msgr2=0x7f96fc19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.431+0000 7f96faa00700 1 --2- 172.21.15.1:0/1920713818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96fc077890 0x7f96fc19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.431+0000 7f96faa00700 1 -- 172.21.15.1:0/1920713818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96fc1a4760 con 0x7f96fc076f70 2024-04-26T01:47:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.431+0000 7f96fa000700 1 --2- 172.21.15.1:0/1920713818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96fc077890 0x7f96fc19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:47:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.432+0000 7f96faa00700 1 --2- 172.21.15.1:0/1920713818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96fc076f70 0x7f96fc19f7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f96ec000c00 tx=0x7f96ec0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:07.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.432+0000 7f9700a00700 1 -- 172.21.15.1:0/1920713818 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec027070 con 0x7f96fc076f70 2024-04-26T01:47:07.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.432+0000 7f9700a00700 1 -- 172.21.15.1:0/1920713818 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96ec022470 con 0x7f96fc076f70 2024-04-26T01:47:07.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.432+0000 7f9701e00700 1 -- 172.21.15.1:0/1920713818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96fc1a49f0 con 0x7f96fc076f70 2024-04-26T01:47:07.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.432+0000 7f9700a00700 1 -- 172.21.15.1:0/1920713818 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec017650 con 0x7f96fc076f70 2024-04-26T01:47:07.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.432+0000 7f9701e00700 1 -- 172.21.15.1:0/1920713818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96fc1ac2d0 con 0x7f96fc076f70 2024-04-26T01:47:07.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.433+0000 7f9701e00700 1 -- 172.21.15.1:0/1920713818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96fc069730 con 0x7f96fc076f70 2024-04-26T01:47:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.435+0000 7f9700a00700 1 -- 172.21.15.1:0/1920713818 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f96ec02a070 con 0x7f96fc076f70 2024-04-26T01:47:07.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.436+0000 7f9700a00700 1 --2- 172.21.15.1:0/1920713818 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f96e00a3cc0 0x7f96e00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:07.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.436+0000 7f9700a00700 1 -- 172.21.15.1:0/1920713818 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f96ec0d0d80 con 0x7f96fc076f70 2024-04-26T01:47:07.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.437+0000 7f96fa000700 1 --2- 172.21.15.1:0/1920713818 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f96e00a3cc0 0x7f96e00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:07.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.437+0000 7f96fa000700 1 --2- 172.21.15.1:0/1920713818 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f96e00a3cc0 0x7f96e00a6180 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f96fc104320 tx=0x7f96e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:07.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.441+0000 7f9700a00700 1 -- 172.21.15.1:0/1920713818 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f96ec013070 con 0x7f96fc076f70 2024-04-26T01:47:07.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.723+0000 7f9701e00700 1 -- 172.21.15.1:0/1920713818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96fc1a0ab0 con 0x7f96fc076f70 2024-04-26T01:47:07.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.724+0000 7f9700a00700 1 -- 172.21.15.1:0/1920713818 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9626 (secure 0 0 0) 0x7f96ec09dbf0 con 0x7f96fc076f70 2024-04-26T01:47:07.723 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:47:07.723 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2952462325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2952462325},{"type":"v1","addr":"172.21.15.173:6835","nonce":2952462325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-26T01:47:07.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.729+0000 7f9701e00700 1 -- 172.21.15.1:0/1920713818 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f96e00a3cc0 msgr2=0x7f96e00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:07.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.729+0000 7f9701e00700 1 --2- 172.21.15.1:0/1920713818 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f96e00a3cc0 0x7f96e00a6180 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f96fc104320 tx=0x7f96e8009260 comp rx=0 tx=0).stop 2024-04-26T01:47:07.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.729+0000 7f9701e00700 1 -- 172.21.15.1:0/1920713818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96fc076f70 msgr2=0x7f96fc19f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:07.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.729+0000 7f9701e00700 1 --2- 172.21.15.1:0/1920713818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96fc076f70 0x7f96fc19f7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f96ec000c00 tx=0x7f96ec0057c0 comp rx=0 tx=0).stop 2024-04-26T01:47:07.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.729+0000 7f9701e00700 1 -- 172.21.15.1:0/1920713818 shutdown_connections 2024-04-26T01:47:07.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.729+0000 7f9701e00700 1 --2- 172.21.15.1:0/1920713818 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f96e00a3cc0 0x7f96e00a6180 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:07.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.729+0000 7f9701e00700 1 --2- 172.21.15.1:0/1920713818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96fc1138f0 0x7f96fc1a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:07.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.729+0000 7f9701e00700 1 --2- 172.21.15.1:0/1920713818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96fc076f70 0x7f96fc19f7a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:07.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.729+0000 7f9701e00700 1 --2- 172.21.15.1:0/1920713818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96fc077890 0x7f96fc19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:07.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.729+0000 7f9701e00700 1 -- 172.21.15.1:0/1920713818 >> 172.21.15.1:0/1920713818 conn(0x7f96fc100720 msgr2=0x7f96fc102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:07.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.729+0000 7f9701e00700 1 -- 172.21.15.1:0/1920713818 shutdown_connections 2024-04-26T01:47:07.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:07.729+0000 7f9701e00700 1 -- 172.21.15.1:0/1920713818 wait complete. 2024-04-26T01:47:07.728 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 12 2024-04-26T01:47:08.112 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:07 smithi001 ceph-mon[27822]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:08.112 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:07 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:08.112 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:07 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:08.112 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:07 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:08.112 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:07 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:47:08.112 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:07 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:47:08.112 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:07 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:08.112 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:07 smithi001 ceph-mon[27822]: Deploying daemon mds.b on smithi097 2024-04-26T01:47:08.112 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:07 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.1:6840/3885053557,v1:172.21.15.1:6841/3885053557] up:boot 2024-04-26T01:47:08.112 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:07 smithi001 ceph-mon[27822]: fsmap 11 up:standby 2024-04-26T01:47:08.112 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:07 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T01:47:08.112 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:07 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1920713818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:47:08.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:07 smithi097 ceph-mon[32853]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:08.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:07 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:08.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:07 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:08.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:07 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:08.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:07 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:47:08.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:07 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:47:08.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:07 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:08.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:07 smithi097 ceph-mon[32853]: Deploying daemon mds.b on smithi097 2024-04-26T01:47:08.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:07 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.1:6840/3885053557,v1:172.21.15.1:6841/3885053557] up:boot 2024-04-26T01:47:08.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:07 smithi097 ceph-mon[32853]: fsmap 11 up:standby 2024-04-26T01:47:08.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:07 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T01:47:08.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:07 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1920713818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:47:08.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:07 smithi173 ceph-mon[32553]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:08.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:07 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:08.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:07 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:08.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:07 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:08.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:07 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:47:08.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:07 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:47:08.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:07 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:08.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:07 smithi173 ceph-mon[32553]: Deploying daemon mds.b on smithi097 2024-04-26T01:47:08.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:07 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.1:6840/3885053557,v1:172.21.15.1:6841/3885053557] up:boot 2024-04-26T01:47:08.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:07 smithi173 ceph-mon[32553]: fsmap 11 up:standby 2024-04-26T01:47:08.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:07 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T01:47:08.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:07 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1920713818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:47:09.233 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 01:47:09 smithi097 systemd[1]: Starting Ceph mds.b for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T01:47:09.625 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 01:47:09 smithi097 podman[62470]: 2024-04-26 01:47:09.262217875 +0000 UTC m=+0.018128821 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:47:09.625 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 01:47:09 smithi097 podman[62470]: 2024-04-26 01:47:09.383600762 +0000 UTC m=+0.139511698 container create 1ded51ec221f25975ae82448e9d2a8a49932017381671665f31e97f4780d2489 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-26T01:47:09.625 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 01:47:09 smithi097 podman[62470]: 2024-04-26 01:47:09.602384698 +0000 UTC m=+0.358295638 container init 1ded51ec221f25975ae82448e9d2a8a49932017381671665f31e97f4780d2489 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T01:47:09.625 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 01:47:09 smithi097 podman[62470]: 2024-04-26 01:47:09.605264061 +0000 UTC m=+0.361174997 container start 1ded51ec221f25975ae82448e9d2a8a49932017381671665f31e97f4780d2489 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T01:47:09.912 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 01:47:09 smithi097 ceph-mds[62488]: -- 172.21.15.97:0/3045463407 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c104241a40 con 0x55c104215000 2024-04-26T01:47:09.912 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 01:47:09 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-b[62484]: starting mds.b at 2024-04-26T01:47:09.912 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 01:47:09 smithi097 bash[62470]: 1ded51ec221f25975ae82448e9d2a8a49932017381671665f31e97f4780d2489 2024-04-26T01:47:09.912 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 01:47:09 smithi097 systemd[1]: Started Ceph mds.b for 77cfcc80-036d-11ef-bc93-c7b262605968. 2024-04-26T01:47:10.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:09 smithi097 ceph-mon[32853]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:10.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:09 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:10.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:09 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:10.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:09 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:10.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:09 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:10.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:09 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:10.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:09 smithi173 ceph-mon[32553]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:10.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:09 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:10.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:09 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:10.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:09 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:10.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:09 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:10.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:09 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:10.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:09 smithi001 ceph-mon[27822]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:10.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:09 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:10.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:09 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:10.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:09 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:10.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:09 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:10.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:09 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:10.692 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:47:11.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:10 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] up:boot 2024-04-26T01:47:11.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:10 smithi001 ceph-mon[27822]: fsmap 12 up:standby 2024-04-26T01:47:11.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:10 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T01:47:11.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:10 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:11.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:10 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:11.133 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:47:11.216 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:10 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] up:boot 2024-04-26T01:47:11.216 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:10 smithi097 ceph-mon[32853]: fsmap 12 up:standby 2024-04-26T01:47:11.216 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:10 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T01:47:11.216 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:10 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:11.216 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:10 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:11.221 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:10 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] up:boot 2024-04-26T01:47:11.222 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:10 smithi173 ceph-mon[32553]: fsmap 12 up:standby 2024-04-26T01:47:11.222 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:10 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T01:47:11.222 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:10 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:11.222 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:10 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:12.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:11 smithi097 ceph-mon[32853]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:12.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:11 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:12.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:11 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:12.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:11 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:12.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:11 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:12.288 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:11 smithi001 ceph-mon[27822]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:12.288 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:11 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:12.288 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:11 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:12.288 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:11 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:12.288 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:11 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:12.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:11 smithi173 ceph-mon[32553]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:12.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:11 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:12.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:11 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:12.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:11 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:12.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:11 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:13.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.761+0000 7fe339000700 1 -- 172.21.15.1:0/774996686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe33406bb10 msgr2=0x7fe33410f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:13.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.761+0000 7fe339000700 1 --2- 172.21.15.1:0/774996686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe33406bb10 0x7fe33410f900 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00b0a0 tx=0x7fe32c00b3b0 comp rx=0 tx=0).stop 2024-04-26T01:47:13.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.762+0000 7fe339000700 1 -- 172.21.15.1:0/774996686 shutdown_connections 2024-04-26T01:47:13.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.762+0000 7fe339000700 1 --2- 172.21.15.1:0/774996686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe33406bb10 0x7fe33410f900 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:13.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.762+0000 7fe339000700 1 --2- 172.21.15.1:0/774996686 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe334108a30 0x7fe334108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:13.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.762+0000 7fe339000700 1 --2- 172.21.15.1:0/774996686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe33406b170 0x7fe33406b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:13.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.762+0000 7fe339000700 1 -- 172.21.15.1:0/774996686 >> 172.21.15.1:0/774996686 conn(0x7fe3340788f0 msgr2=0x7fe334078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:13.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.762+0000 7fe339000700 1 -- 172.21.15.1:0/774996686 shutdown_connections 2024-04-26T01:47:13.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.762+0000 7fe339000700 1 -- 172.21.15.1:0/774996686 wait complete. 2024-04-26T01:47:13.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.763+0000 7fe339000700 1 Processor -- start 2024-04-26T01:47:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.763+0000 7fe339000700 1 -- start start 2024-04-26T01:47:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.764+0000 7fe339000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe33406b170 0x7fe33419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.764+0000 7fe339000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe33406bb10 0x7fe33419b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.764+0000 7fe339000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe334108a30 0x7fe33419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.764+0000 7fe339000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe334112c30 con 0x7fe334108a30 2024-04-26T01:47:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.764+0000 7fe339000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe334112ab0 con 0x7fe33406b170 2024-04-26T01:47:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.764+0000 7fe339000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe334112db0 con 0x7fe33406bb10 2024-04-26T01:47:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.764+0000 7fe331e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe33406b170 0x7fe33419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.764+0000 7fe331e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe33406b170 0x7fe33419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:55634/0 (socket says 172.21.15.1:55634) 2024-04-26T01:47:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.764+0000 7fe331e00700 1 -- 172.21.15.1:0/163496507 learned_addr learned my addr 172.21.15.1:0/163496507 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:47:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.764+0000 7fe331400700 1 --2- 172.21.15.1:0/163496507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe33406bb10 0x7fe33419b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:13.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.764+0000 7fe332800700 1 --2- 172.21.15.1:0/163496507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe334108a30 0x7fe33419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.764+0000 7fe331e00700 1 -- 172.21.15.1:0/163496507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe33406bb10 msgr2=0x7fe33419b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.764+0000 7fe331e00700 1 --2- 172.21.15.1:0/163496507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe33406bb10 0x7fe33419b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.764+0000 7fe331e00700 1 -- 172.21.15.1:0/163496507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe334108a30 msgr2=0x7fe33419fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.764+0000 7fe331e00700 1 --2- 172.21.15.1:0/163496507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe334108a30 0x7fe33419fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.764+0000 7fe331e00700 1 -- 172.21.15.1:0/163496507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3341a0340 con 0x7fe33406b170 2024-04-26T01:47:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.765+0000 7fe331e00700 1 --2- 172.21.15.1:0/163496507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe33406b170 0x7fe33419b380 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe32400e9c0 tx=0x7fe32400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.765+0000 7fe32aa00700 1 -- 172.21.15.1:0/163496507 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32400cbd0 con 0x7fe33406b170 2024-04-26T01:47:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.765+0000 7fe32aa00700 1 -- 172.21.15.1:0/163496507 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe32400cd70 con 0x7fe33406b170 2024-04-26T01:47:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.765+0000 7fe339000700 1 -- 172.21.15.1:0/163496507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3341a0630 con 0x7fe33406b170 2024-04-26T01:47:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.765+0000 7fe32aa00700 1 -- 172.21.15.1:0/163496507 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324005610 con 0x7fe33406b170 2024-04-26T01:47:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.765+0000 7fe339000700 1 -- 172.21.15.1:0/163496507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3341a7eb0 con 0x7fe33406b170 2024-04-26T01:47:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.766+0000 7fe339000700 1 -- 172.21.15.1:0/163496507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe334069730 con 0x7fe33406b170 2024-04-26T01:47:13.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.769+0000 7fe32aa00700 1 -- 172.21.15.1:0/163496507 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fe324004750 con 0x7fe33406b170 2024-04-26T01:47:13.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.770+0000 7fe32aa00700 1 --2- 172.21.15.1:0/163496507 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe3180a3cc0 0x7fe3180a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:13.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.770+0000 7fe32aa00700 1 -- 172.21.15.1:0/163496507 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fe3240d03f0 con 0x7fe33406b170 2024-04-26T01:47:13.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.770+0000 7fe331400700 1 --2- 172.21.15.1:0/163496507 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe3180a3cc0 0x7fe3180a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:13.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.771+0000 7fe331400700 1 --2- 172.21.15.1:0/163496507 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe3180a3cc0 0x7fe3180a6180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe334078ed0 tx=0x7fe320009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:13.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:13.772+0000 7fe32aa00700 1 -- 172.21.15.1:0/163496507 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe324053ff0 con 0x7fe33406b170 2024-04-26T01:47:14.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:14.069+0000 7fe339000700 1 -- 172.21.15.1:0/163496507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe33402df80 con 0x7fe33406b170 2024-04-26T01:47:14.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:14.070+0000 7fe32aa00700 1 -- 172.21.15.1:0/163496507 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10452 (secure 0 0 0) 0x7fe32409d1d0 con 0x7fe33406b170 2024-04-26T01:47:14.068 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:47:14.069 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2952462325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2952462325},{"type":"v1","addr":"172.21.15.173:6835","nonce":2952462325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24518,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-26T01:47:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:14.074+0000 7fe339000700 1 -- 172.21.15.1:0/163496507 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe3180a3cc0 msgr2=0x7fe3180a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:14.074+0000 7fe339000700 1 --2- 172.21.15.1:0/163496507 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe3180a3cc0 0x7fe3180a6180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe334078ed0 tx=0x7fe320009260 comp rx=0 tx=0).stop 2024-04-26T01:47:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:14.074+0000 7fe339000700 1 -- 172.21.15.1:0/163496507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe33406b170 msgr2=0x7fe33419b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:14.074+0000 7fe339000700 1 --2- 172.21.15.1:0/163496507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe33406b170 0x7fe33419b380 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe32400e9c0 tx=0x7fe32400ecd0 comp rx=0 tx=0).stop 2024-04-26T01:47:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:14.074+0000 7fe339000700 1 -- 172.21.15.1:0/163496507 shutdown_connections 2024-04-26T01:47:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:14.074+0000 7fe339000700 1 --2- 172.21.15.1:0/163496507 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe3180a3cc0 0x7fe3180a6180 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:14.074+0000 7fe339000700 1 --2- 172.21.15.1:0/163496507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe334108a30 0x7fe33419fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:14.074+0000 7fe339000700 1 --2- 172.21.15.1:0/163496507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe33406b170 0x7fe33419b380 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:14.074+0000 7fe339000700 1 --2- 172.21.15.1:0/163496507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe33406bb10 0x7fe33419b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:14.074+0000 7fe339000700 1 -- 172.21.15.1:0/163496507 >> 172.21.15.1:0/163496507 conn(0x7fe3340788f0 msgr2=0x7fe33410ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:14.074+0000 7fe339000700 1 -- 172.21.15.1:0/163496507 shutdown_connections 2024-04-26T01:47:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:14.074+0000 7fe339000700 1 -- 172.21.15.1:0/163496507 wait complete. 2024-04-26T01:47:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 13 2024-04-26T01:47:14.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:13 smithi097 ceph-mon[32853]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:14.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:13 smithi173 ceph-mon[32553]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:14.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:13 smithi001 ceph-mon[27822]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:14.908 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-04-26T01:47:15.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:14 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/163496507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:47:15.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:14 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/163496507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:47:15.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:14 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/163496507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:47:15.371 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:47:16.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:16 smithi001 ceph-mon[27822]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:16.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:16 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:16.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:16 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:16.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:16 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:16.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:16 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:16.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:16 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:16.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:16 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:16.308 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:16 smithi173 ceph-mon[32553]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:16.308 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:16 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:16.308 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:16 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:16.309 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:16 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:16.309 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:16 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:16.309 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:16 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:16.309 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:16 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:16 smithi097 ceph-mon[32853]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:16 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:16 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:16 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:16 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:16 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:16 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:17.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.467+0000 7f162ae00700 1 -- 172.21.15.1:0/2891922992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624109fa0 msgr2=0x7f1624111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:17.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.467+0000 7f162ae00700 1 --2- 172.21.15.1:0/2891922992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624109fa0 0x7f1624111b70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f161c00b0d0 tx=0x7f161c00b3e0 comp rx=0 tx=0).stop 2024-04-26T01:47:17.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.468+0000 7f162ae00700 1 -- 172.21.15.1:0/2891922992 shutdown_connections 2024-04-26T01:47:17.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.468+0000 7f162ae00700 1 --2- 172.21.15.1:0/2891922992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624109fa0 0x7f1624111b70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:17.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.468+0000 7f162ae00700 1 --2- 172.21.15.1:0/2891922992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1624104ec0 0x7f16241052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:17.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.468+0000 7f162ae00700 1 --2- 172.21.15.1:0/2891922992 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1624105870 0x7f16241098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:17.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.468+0000 7f162ae00700 1 -- 172.21.15.1:0/2891922992 >> 172.21.15.1:0/2891922992 conn(0x7f1624100750 msgr2=0x7f1624102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:17.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.468+0000 7f162ae00700 1 -- 172.21.15.1:0/2891922992 shutdown_connections 2024-04-26T01:47:17.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.468+0000 7f162ae00700 1 -- 172.21.15.1:0/2891922992 wait complete. 2024-04-26T01:47:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.469+0000 7f162ae00700 1 Processor -- start 2024-04-26T01:47:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.470+0000 7f162ae00700 1 -- start start 2024-04-26T01:47:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.470+0000 7f162ae00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1624104ec0 0x7f162419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.470+0000 7f162ae00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624105870 0x7f162419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.470+0000 7f162ae00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1624109fa0 0x7f162419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.470+0000 7f162ae00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1624114e30 con 0x7f1624105870 2024-04-26T01:47:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.470+0000 7f162ae00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1624114cb0 con 0x7f1624104ec0 2024-04-26T01:47:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.470+0000 7f162ae00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1624114fb0 con 0x7f1624109fa0 2024-04-26T01:47:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.470+0000 7f1628e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1624109fa0 0x7f162419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.470+0000 7f161b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624105870 0x7f162419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.470+0000 7f1628e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1624109fa0 0x7f162419fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:38106/0 (socket says 172.21.15.1:38106) 2024-04-26T01:47:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.470+0000 7f1628e00700 1 -- 172.21.15.1:0/93843634 learned_addr learned my addr 172.21.15.1:0/93843634 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:47:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.470+0000 7f1623e00700 1 --2- 172.21.15.1:0/93843634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1624104ec0 0x7f162419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.471+0000 7f1628e00700 1 -- 172.21.15.1:0/93843634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1624104ec0 msgr2=0x7f162419b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.471+0000 7f1628e00700 1 --2- 172.21.15.1:0/93843634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1624104ec0 0x7f162419b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:17.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.471+0000 7f1628e00700 1 -- 172.21.15.1:0/93843634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624105870 msgr2=0x7f162419b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:17.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.471+0000 7f1628e00700 1 --2- 172.21.15.1:0/93843634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624105870 0x7f162419b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:17.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.471+0000 7f1628e00700 1 -- 172.21.15.1:0/93843634 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16241a02a0 con 0x7f1624109fa0 2024-04-26T01:47:17.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.471+0000 7f1628e00700 1 --2- 172.21.15.1:0/93843634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1624109fa0 0x7f162419fb60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f161c007a00 tx=0x7f161c007a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:17.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.471+0000 7f1621600700 1 -- 172.21.15.1:0/93843634 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c01f070 con 0x7f1624109fa0 2024-04-26T01:47:17.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.471+0000 7f1621600700 1 -- 172.21.15.1:0/93843634 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f161c004c50 con 0x7f1624109fa0 2024-04-26T01:47:17.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.471+0000 7f1621600700 1 -- 172.21.15.1:0/93843634 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c01b690 con 0x7f1624109fa0 2024-04-26T01:47:17.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.471+0000 7f162ae00700 1 -- 172.21.15.1:0/93843634 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16241a0530 con 0x7f1624109fa0 2024-04-26T01:47:17.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.472+0000 7f162ae00700 1 -- 172.21.15.1:0/93843634 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16241a7e10 con 0x7f1624109fa0 2024-04-26T01:47:17.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.473+0000 7f162ae00700 1 -- 172.21.15.1:0/93843634 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1624069730 con 0x7f1624109fa0 2024-04-26T01:47:17.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.475+0000 7f1621600700 1 -- 172.21.15.1:0/93843634 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f161c022070 con 0x7f1624109fa0 2024-04-26T01:47:17.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.476+0000 7f1621600700 1 --2- 172.21.15.1:0/93843634 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f16080a3cc0 0x7f16080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:17.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.476+0000 7f1621600700 1 -- 172.21.15.1:0/93843634 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f161c00f020 con 0x7f1624109fa0 2024-04-26T01:47:17.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.476+0000 7f1623e00700 1 --2- 172.21.15.1:0/93843634 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f16080a3cc0 0x7f16080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:17.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.477+0000 7f1623e00700 1 --2- 172.21.15.1:0/93843634 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f16080a3cc0 0x7f16080a6180 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1614009c60 tx=0x7f1614009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.481+0000 7f1621600700 1 -- 172.21.15.1:0/93843634 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f161c029020 con 0x7f1624109fa0 2024-04-26T01:47:17.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.718+0000 7f162ae00700 1 -- 172.21.15.1:0/93843634 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f162402df80 con 0x7f1624109fa0 2024-04-26T01:47:17.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.719+0000 7f1621600700 1 -- 172.21.15.1:0/93843634 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) ==== 84+0+65 (secure 0 0 0) 0x7f161c09ddd0 con 0x7f1624109fa0 2024-04-26T01:47:17.717 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:47:17.717 INFO:teuthology.orchestra.run.smithi001.stdout:[ 2024-04-26T01:47:17.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T01:47:17.717 INFO:teuthology.orchestra.run.smithi001.stdout: "poolnum": 1, 2024-04-26T01:47:17.717 INFO:teuthology.orchestra.run.smithi001.stdout: "poolname": ".mgr" 2024-04-26T01:47:17.717 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:47:17.717 INFO:teuthology.orchestra.run.smithi001.stdout:] 2024-04-26T01:47:17.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.722+0000 7f162ae00700 1 -- 172.21.15.1:0/93843634 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f16080a3cc0 msgr2=0x7f16080a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:17.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.722+0000 7f162ae00700 1 --2- 172.21.15.1:0/93843634 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f16080a3cc0 0x7f16080a6180 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1614009c60 tx=0x7f1614009260 comp rx=0 tx=0).stop 2024-04-26T01:47:17.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.722+0000 7f162ae00700 1 -- 172.21.15.1:0/93843634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1624109fa0 msgr2=0x7f162419fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:17.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.722+0000 7f162ae00700 1 --2- 172.21.15.1:0/93843634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1624109fa0 0x7f162419fb60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f161c007a00 tx=0x7f161c007a30 comp rx=0 tx=0).stop 2024-04-26T01:47:17.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.722+0000 7f162ae00700 1 -- 172.21.15.1:0/93843634 shutdown_connections 2024-04-26T01:47:17.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.722+0000 7f162ae00700 1 --2- 172.21.15.1:0/93843634 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f16080a3cc0 0x7f16080a6180 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:17.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.722+0000 7f162ae00700 1 --2- 172.21.15.1:0/93843634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1624109fa0 0x7f162419fb60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:17.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.722+0000 7f162ae00700 1 --2- 172.21.15.1:0/93843634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1624104ec0 0x7f162419b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:17.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.722+0000 7f162ae00700 1 --2- 172.21.15.1:0/93843634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1624105870 0x7f162419b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:17.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.722+0000 7f162ae00700 1 -- 172.21.15.1:0/93843634 >> 172.21.15.1:0/93843634 conn(0x7f1624100750 msgr2=0x7f1624101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:17.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.722+0000 7f162ae00700 1 -- 172.21.15.1:0/93843634 shutdown_connections 2024-04-26T01:47:17.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:17.723+0000 7f162ae00700 1 -- 172.21.15.1:0/93843634 wait complete. 2024-04-26T01:47:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:17 smithi097 ceph-mon[32853]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:17 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:17 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:17 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:17 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:17 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/93843634' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T01:47:18.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:17 smithi173 ceph-mon[32553]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:18.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:17 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:18.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:17 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:18.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:17 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:18.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:17 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:18.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:17 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/93843634' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T01:47:18.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:17 smithi001 ceph-mon[27822]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:18.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:17 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:18.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:17 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:18.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:17 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:18.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:17 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:18.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:17 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/93843634' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T01:47:18.932 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T01:47:18.933 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T01:47:19.407 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:47:19.994 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:19 smithi001 ceph-mon[27822]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:20.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:19 smithi097 ceph-mon[32853]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:20.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:19 smithi173 ceph-mon[32553]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:21.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.697+0000 7f2aff600700 1 -- 172.21.15.1:0/1780338236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2af8108a30 msgr2=0x7f2af8108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:21.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.697+0000 7f2aff600700 1 --2- 172.21.15.1:0/1780338236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2af8108a30 0x7f2af8108e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2aec009990 tx=0x7f2aec009ca0 comp rx=0 tx=0).stop 2024-04-26T01:47:21.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.697+0000 7f2aff600700 1 -- 172.21.15.1:0/1780338236 shutdown_connections 2024-04-26T01:47:21.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.697+0000 7f2aff600700 1 --2- 172.21.15.1:0/1780338236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2af806b0f0 0x7f2af810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:21.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.697+0000 7f2aff600700 1 --2- 172.21.15.1:0/1780338236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2af8108a30 0x7f2af8108e10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:21.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.697+0000 7f2aff600700 1 --2- 172.21.15.1:0/1780338236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af806a750 0x7f2af806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:21.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.697+0000 7f2aff600700 1 -- 172.21.15.1:0/1780338236 >> 172.21.15.1:0/1780338236 conn(0x7f2af80788f0 msgr2=0x7f2af8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:21.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.697+0000 7f2aff600700 1 -- 172.21.15.1:0/1780338236 shutdown_connections 2024-04-26T01:47:21.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.697+0000 7f2aff600700 1 -- 172.21.15.1:0/1780338236 wait complete. 2024-04-26T01:47:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.699+0000 7f2aff600700 1 Processor -- start 2024-04-26T01:47:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.699+0000 7f2aff600700 1 -- start start 2024-04-26T01:47:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.699+0000 7f2aff600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2af806a750 0x7f2af819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.699+0000 7f2aff600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2af806b0f0 0x7f2af819b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:21.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.699+0000 7f2aff600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af8108a30 0x7f2af819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:21.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.699+0000 7f2aff600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2af8112c00 con 0x7f2af806b0f0 2024-04-26T01:47:21.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.699+0000 7f2aff600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2af8112a80 con 0x7f2af8108a30 2024-04-26T01:47:21.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.699+0000 7f2aff600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2af8112d80 con 0x7f2af806a750 2024-04-26T01:47:21.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.700+0000 7f2afcc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2af806a750 0x7f2af819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:21.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.700+0000 7f2afcc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2af806a750 0x7f2af819b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:38132/0 (socket says 172.21.15.1:38132) 2024-04-26T01:47:21.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.700+0000 7f2afcc00700 1 -- 172.21.15.1:0/1826311567 learned_addr learned my addr 172.21.15.1:0/1826311567 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:47:21.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.700+0000 7f2af7e00700 1 --2- 172.21.15.1:0/1826311567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2af806b0f0 0x7f2af819b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:21.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.700+0000 7f2afd600700 1 --2- 172.21.15.1:0/1826311567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af8108a30 0x7f2af819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:21.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.700+0000 7f2afcc00700 1 -- 172.21.15.1:0/1826311567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af8108a30 msgr2=0x7f2af819fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:21.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.700+0000 7f2afcc00700 1 --2- 172.21.15.1:0/1826311567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af8108a30 0x7f2af819fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:21.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.700+0000 7f2afcc00700 1 -- 172.21.15.1:0/1826311567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2af806b0f0 msgr2=0x7f2af819b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:21.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.700+0000 7f2afcc00700 1 --2- 172.21.15.1:0/1826311567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2af806b0f0 0x7f2af819b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:21.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.700+0000 7f2afcc00700 1 -- 172.21.15.1:0/1826311567 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af81a0360 con 0x7f2af806a750 2024-04-26T01:47:21.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.700+0000 7f2afcc00700 1 --2- 172.21.15.1:0/1826311567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2af806a750 0x7f2af819b350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2aec005910 tx=0x7f2aec00b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:21.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.701+0000 7f2af5600700 1 -- 172.21.15.1:0/1826311567 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec027070 con 0x7f2af806a750 2024-04-26T01:47:21.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.701+0000 7f2af5600700 1 -- 172.21.15.1:0/1826311567 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aec022470 con 0x7f2af806a750 2024-04-26T01:47:21.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.701+0000 7f2af5600700 1 -- 172.21.15.1:0/1826311567 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec017650 con 0x7f2af806a750 2024-04-26T01:47:21.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.701+0000 7f2aff600700 1 -- 172.21.15.1:0/1826311567 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af81a05f0 con 0x7f2af806a750 2024-04-26T01:47:21.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.701+0000 7f2aff600700 1 -- 172.21.15.1:0/1826311567 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af81a7ed0 con 0x7f2af806a750 2024-04-26T01:47:21.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.702+0000 7f2aff600700 1 -- 172.21.15.1:0/1826311567 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2af8104170 con 0x7f2af806a750 2024-04-26T01:47:21.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.704+0000 7f2af5600700 1 -- 172.21.15.1:0/1826311567 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f2aec02a070 con 0x7f2af806a750 2024-04-26T01:47:21.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.706+0000 7f2af5600700 1 --2- 172.21.15.1:0/1826311567 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2ae00a3cc0 0x7f2ae00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:21.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.706+0000 7f2af5600700 1 -- 172.21.15.1:0/1826311567 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f2aec0d0e20 con 0x7f2af806a750 2024-04-26T01:47:21.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.706+0000 7f2af7e00700 1 --2- 172.21.15.1:0/1826311567 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2ae00a3cc0 0x7f2ae00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:21.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.707+0000 7f2af7e00700 1 --2- 172.21.15.1:0/1826311567 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2ae00a3cc0 0x7f2ae00a6180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2af810fd40 tx=0x7f2ae4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:21.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.708+0000 7f2af5600700 1 -- 172.21.15.1:0/1826311567 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2aec013070 con 0x7f2af806a750 2024-04-26T01:47:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:21.943+0000 7f2aff600700 1 -- 172.21.15.1:0/1826311567 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2af81a80a0 con 0x7f2af806a750 2024-04-26T01:47:22.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:22.047+0000 7f2af5600700 1 -- 172.21.15.1:0/1826311567 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) ==== 138+0+0 (secure 0 0 0) 0x7f2aec09dc90 con 0x7f2af806a750 2024-04-26T01:47:22.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:22.058+0000 7f2aff600700 1 -- 172.21.15.1:0/1826311567 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2ae00a3cc0 msgr2=0x7f2ae00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:22.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:22.058+0000 7f2aff600700 1 --2- 172.21.15.1:0/1826311567 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2ae00a3cc0 0x7f2ae00a6180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2af810fd40 tx=0x7f2ae4009260 comp rx=0 tx=0).stop 2024-04-26T01:47:22.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:22.058+0000 7f2aff600700 1 -- 172.21.15.1:0/1826311567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2af806a750 msgr2=0x7f2af819b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:22.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:22.058+0000 7f2aff600700 1 --2- 172.21.15.1:0/1826311567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2af806a750 0x7f2af819b350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2aec005910 tx=0x7f2aec00b670 comp rx=0 tx=0).stop 2024-04-26T01:47:22.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:22.058+0000 7f2aff600700 1 -- 172.21.15.1:0/1826311567 shutdown_connections 2024-04-26T01:47:22.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:22.058+0000 7f2aff600700 1 --2- 172.21.15.1:0/1826311567 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2ae00a3cc0 0x7f2ae00a6180 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:22.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:22.058+0000 7f2aff600700 1 --2- 172.21.15.1:0/1826311567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af8108a30 0x7f2af819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:22.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:22.058+0000 7f2aff600700 1 --2- 172.21.15.1:0/1826311567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2af806a750 0x7f2af819b350 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:22.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:22.058+0000 7f2aff600700 1 --2- 172.21.15.1:0/1826311567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2af806b0f0 0x7f2af819b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:22.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:22.058+0000 7f2aff600700 1 -- 172.21.15.1:0/1826311567 >> 172.21.15.1:0/1826311567 conn(0x7f2af80788f0 msgr2=0x7f2af80ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:22.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:22.059+0000 7f2aff600700 1 -- 172.21.15.1:0/1826311567 shutdown_connections 2024-04-26T01:47:22.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:22.059+0000 7f2aff600700 1 -- 172.21.15.1:0/1826311567 wait complete. 2024-04-26T01:47:22.058 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' created 2024-04-26T01:47:22.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:22 smithi173 ceph-mon[32553]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:22.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:22 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:22.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:22 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:22.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:22 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:22.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:22 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:22.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:22 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:22.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:22 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1826311567' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T01:47:22.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:22 smithi173 ceph-mon[32553]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T01:47:22.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:22 smithi001 ceph-mon[27822]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:22.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:22 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:22.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:22 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:22.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:22 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:22.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:22 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:22.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:22 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:22.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:22 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1826311567' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T01:47:22.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:22 smithi001 ceph-mon[27822]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T01:47:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:22 smithi097 ceph-mon[32853]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:22 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:22 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:22 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:22 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:22 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:47:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:22 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1826311567' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T01:47:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:22 smithi097 ceph-mon[32853]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T01:47:22.976 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T01:47:23.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:23 smithi173 ceph-mon[32553]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T01:47:23.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:23 smithi173 ceph-mon[32553]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T01:47:23.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:23 smithi001 ceph-mon[27822]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T01:47:23.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:23 smithi001 ceph-mon[27822]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T01:47:23.442 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:47:23.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:23 smithi097 ceph-mon[32853]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T01:47:23.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:23 smithi097 ceph-mon[32853]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T01:47:24.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:24 smithi173 ceph-mon[32553]: pgmap v315: 33 pgs: 9 creating+peering, 1 active+clean, 23 unknown; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:24.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:24 smithi173 ceph-mon[32553]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T01:47:24.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:24 smithi173 ceph-mon[32553]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T01:47:24.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:24 smithi173 ceph-mon[32553]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T01:47:24.319 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:24 smithi001 ceph-mon[27822]: pgmap v315: 33 pgs: 9 creating+peering, 1 active+clean, 23 unknown; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:24.319 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:24 smithi001 ceph-mon[27822]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T01:47:24.319 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:24 smithi001 ceph-mon[27822]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T01:47:24.319 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:24 smithi001 ceph-mon[27822]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T01:47:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:24 smithi097 ceph-mon[32853]: pgmap v315: 33 pgs: 9 creating+peering, 1 active+clean, 23 unknown; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:24 smithi097 ceph-mon[32853]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T01:47:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:24 smithi097 ceph-mon[32853]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T01:47:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:24 smithi097 ceph-mon[32853]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T01:47:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.593+0000 7f883fa00700 1 -- 172.21.15.1:0/1579956914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8838105870 msgr2=0x7f88381098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.593+0000 7f883fa00700 1 --2- 172.21.15.1:0/1579956914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8838105870 0x7f88381098c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8824009990 tx=0x7f8824009ca0 comp rx=0 tx=0).stop 2024-04-26T01:47:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.593+0000 7f883fa00700 1 -- 172.21.15.1:0/1579956914 shutdown_connections 2024-04-26T01:47:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.593+0000 7f883fa00700 1 --2- 172.21.15.1:0/1579956914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8838109fa0 0x7f8838111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.593+0000 7f883fa00700 1 --2- 172.21.15.1:0/1579956914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8838104ec0 0x7f88381052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.593+0000 7f883fa00700 1 --2- 172.21.15.1:0/1579956914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8838105870 0x7f88381098c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.593+0000 7f883fa00700 1 -- 172.21.15.1:0/1579956914 >> 172.21.15.1:0/1579956914 conn(0x7f8838100710 msgr2=0x7f8838102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.593+0000 7f883fa00700 1 -- 172.21.15.1:0/1579956914 shutdown_connections 2024-04-26T01:47:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.593+0000 7f883fa00700 1 -- 172.21.15.1:0/1579956914 wait complete. 2024-04-26T01:47:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.595+0000 7f883fa00700 1 Processor -- start 2024-04-26T01:47:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.595+0000 7f883fa00700 1 -- start start 2024-04-26T01:47:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.596+0000 7f883fa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8838104ec0 0x7f883819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.596+0000 7f883fa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8838105870 0x7f883819b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.596+0000 7f883fa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8838109fa0 0x7f883819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.596+0000 7f883fa00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8838114e80 con 0x7f8838109fa0 2024-04-26T01:47:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.596+0000 7f883fa00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8838114d00 con 0x7f8838104ec0 2024-04-26T01:47:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.596+0000 7f883fa00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8838115000 con 0x7f8838105870 2024-04-26T01:47:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.596+0000 7f883da00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8838109fa0 0x7f883819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:25.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.596+0000 7f883da00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8838109fa0 0x7f883819fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60700/0 (socket says 172.21.15.1:60700) 2024-04-26T01:47:25.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.596+0000 7f883da00700 1 -- 172.21.15.1:0/3175625575 learned_addr learned my addr 172.21.15.1:0/3175625575 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:47:25.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.596+0000 7f883d000700 1 --2- 172.21.15.1:0/3175625575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8838104ec0 0x7f883819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:25.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.597+0000 7f8837e00700 1 --2- 172.21.15.1:0/3175625575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8838105870 0x7f883819b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:25.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.597+0000 7f883da00700 1 -- 172.21.15.1:0/3175625575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8838105870 msgr2=0x7f883819b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:25.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.597+0000 7f883da00700 1 --2- 172.21.15.1:0/3175625575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8838105870 0x7f883819b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:25.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.597+0000 7f883da00700 1 -- 172.21.15.1:0/3175625575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8838104ec0 msgr2=0x7f883819b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:25.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.597+0000 7f883da00700 1 --2- 172.21.15.1:0/3175625575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8838104ec0 0x7f883819b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:25.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.597+0000 7f883da00700 1 -- 172.21.15.1:0/3175625575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88381a03a0 con 0x7f8838109fa0 2024-04-26T01:47:25.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.597+0000 7f883d000700 1 --2- 172.21.15.1:0/3175625575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8838104ec0 0x7f883819b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:47:25.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.597+0000 7f883da00700 1 --2- 172.21.15.1:0/3175625575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8838109fa0 0x7f883819fc60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f883000ef30 tx=0x7f883000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:25.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.598+0000 7f8835600700 1 -- 172.21.15.1:0/3175625575 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88300092e0 con 0x7f8838109fa0 2024-04-26T01:47:25.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.598+0000 7f8835600700 1 -- 172.21.15.1:0/3175625575 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8830004bb0 con 0x7f8838109fa0 2024-04-26T01:47:25.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.598+0000 7f883fa00700 1 -- 172.21.15.1:0/3175625575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88381a0630 con 0x7f8838109fa0 2024-04-26T01:47:25.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.598+0000 7f8835600700 1 -- 172.21.15.1:0/3175625575 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830004020 con 0x7f8838109fa0 2024-04-26T01:47:25.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.598+0000 7f883fa00700 1 -- 172.21.15.1:0/3175625575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88381a7f10 con 0x7f8838109fa0 2024-04-26T01:47:25.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.599+0000 7f883fa00700 1 -- 172.21.15.1:0/3175625575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8838069730 con 0x7f8838109fa0 2024-04-26T01:47:25.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.602+0000 7f8835600700 1 -- 172.21.15.1:0/3175625575 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f883001b030 con 0x7f8838109fa0 2024-04-26T01:47:25.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.604+0000 7f8835600700 1 --2- 172.21.15.1:0/3175625575 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f88200a3cc0 0x7f88200a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:25.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.604+0000 7f8835600700 1 -- 172.21.15.1:0/3175625575 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7497+0+0 (secure 0 0 0) 0x7f88300d03d0 con 0x7f8838109fa0 2024-04-26T01:47:25.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.604+0000 7f883d000700 1 --2- 172.21.15.1:0/3175625575 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f88200a3cc0 0x7f88200a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:25.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.605+0000 7f883d000700 1 --2- 172.21.15.1:0/3175625575 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f88200a3cc0 0x7f88200a6180 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f882c009680 tx=0x7f882c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:25.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.607+0000 7f8835600700 1 -- 172.21.15.1:0/3175625575 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8830022020 con 0x7f8838109fa0 2024-04-26T01:47:25.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:25.840+0000 7f883fa00700 1 -- 172.21.15.1:0/3175625575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f883802df80 con 0x7f8838109fa0 2024-04-26T01:47:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:26.085+0000 7f8835600700 1 -- 172.21.15.1:0/3175625575 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v87) ==== 170+0+0 (secure 0 0 0) 0x7f8830010040 con 0x7f8838109fa0 2024-04-26T01:47:26.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:26.092+0000 7f883fa00700 1 -- 172.21.15.1:0/3175625575 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f88200a3cc0 msgr2=0x7f88200a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:26.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:26.092+0000 7f883fa00700 1 --2- 172.21.15.1:0/3175625575 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f88200a3cc0 0x7f88200a6180 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f882c009680 tx=0x7f882c009260 comp rx=0 tx=0).stop 2024-04-26T01:47:26.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:26.092+0000 7f883fa00700 1 -- 172.21.15.1:0/3175625575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8838109fa0 msgr2=0x7f883819fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:26.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:26.092+0000 7f883fa00700 1 --2- 172.21.15.1:0/3175625575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8838109fa0 0x7f883819fc60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f883000ef30 tx=0x7f883000c3a0 comp rx=0 tx=0).stop 2024-04-26T01:47:26.091 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:26 smithi001 ceph-mon[27822]: pgmap v318: 33 pgs: 9 creating+peering, 12 active+clean, 12 unknown; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:26.091 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:26 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3175625575' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T01:47:26.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:26.093+0000 7f883fa00700 1 -- 172.21.15.1:0/3175625575 shutdown_connections 2024-04-26T01:47:26.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:26.093+0000 7f883fa00700 1 --2- 172.21.15.1:0/3175625575 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f88200a3cc0 0x7f88200a6180 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:26.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:26.093+0000 7f883fa00700 1 --2- 172.21.15.1:0/3175625575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8838109fa0 0x7f883819fc60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:26.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:26.093+0000 7f883fa00700 1 --2- 172.21.15.1:0/3175625575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8838104ec0 0x7f883819b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:26.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:26.093+0000 7f883fa00700 1 --2- 172.21.15.1:0/3175625575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8838105870 0x7f883819b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:26.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:26.093+0000 7f883fa00700 1 -- 172.21.15.1:0/3175625575 >> 172.21.15.1:0/3175625575 conn(0x7f8838100710 msgr2=0x7f8838102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:26.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:26.095+0000 7f883fa00700 1 -- 172.21.15.1:0/3175625575 shutdown_connections 2024-04-26T01:47:26.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:26.096+0000 7f883fa00700 1 -- 172.21.15.1:0/3175625575 wait complete. 2024-04-26T01:47:26.098 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' created 2024-04-26T01:47:26.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:26 smithi097 ceph-mon[32853]: pgmap v318: 33 pgs: 9 creating+peering, 12 active+clean, 12 unknown; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:26.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:26 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3175625575' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T01:47:26.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:26 smithi173 ceph-mon[32553]: pgmap v318: 33 pgs: 9 creating+peering, 12 active+clean, 12 unknown; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:26.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:26 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3175625575' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T01:47:27.072 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T01:47:27.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:27 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3175625575' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T01:47:27.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:27 smithi001 ceph-mon[27822]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T01:47:27.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:27 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3175625575' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T01:47:27.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:27 smithi097 ceph-mon[32853]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T01:47:27.536 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:47:27.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:27 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3175625575' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T01:47:27.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:27 smithi173 ceph-mon[32553]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T01:47:28.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:28 smithi001 ceph-mon[27822]: pgmap v320: 97 pgs: 34 creating+peering, 24 active+clean, 39 unknown; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:28.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:28 smithi001 ceph-mon[27822]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T01:47:28.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:28 smithi097 ceph-mon[32853]: pgmap v320: 97 pgs: 34 creating+peering, 24 active+clean, 39 unknown; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:28.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:28 smithi097 ceph-mon[32853]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T01:47:28.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:28 smithi173 ceph-mon[32553]: pgmap v320: 97 pgs: 34 creating+peering, 24 active+clean, 39 unknown; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:28.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:28 smithi173 ceph-mon[32553]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T01:47:29.376 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:29 smithi001 ceph-mon[27822]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T01:47:29.376 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:29 smithi001 ceph-mon[27822]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T01:47:29.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:29 smithi097 ceph-mon[32853]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T01:47:29.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:29 smithi097 ceph-mon[32853]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T01:47:29.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:29 smithi173 ceph-mon[32553]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T01:47:29.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:29 smithi173 ceph-mon[32553]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T01:47:29.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.705+0000 7f9a6d200700 1 -- 172.21.15.1:0/2105157561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a68103450 msgr2=0x7f9a6810f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:29.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.705+0000 7f9a6d200700 1 --2- 172.21.15.1:0/2105157561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a68103450 0x7f9a6810f8b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9a60009f90 tx=0x7f9a6000b330 comp rx=0 tx=0).stop 2024-04-26T01:47:29.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.706+0000 7f9a6d200700 1 -- 172.21.15.1:0/2105157561 shutdown_connections 2024-04-26T01:47:29.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.706+0000 7f9a6d200700 1 --2- 172.21.15.1:0/2105157561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a68103450 0x7f9a6810f8b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:29.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.706+0000 7f9a6d200700 1 --2- 172.21.15.1:0/2105157561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a68108a50 0x7f9a68108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:29.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.706+0000 7f9a6d200700 1 --2- 172.21.15.1:0/2105157561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a68102a90 0x7f9a68102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:29.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.706+0000 7f9a6d200700 1 -- 172.21.15.1:0/2105157561 >> 172.21.15.1:0/2105157561 conn(0x7f9a68079ba0 msgr2=0x7f9a68079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:29.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.706+0000 7f9a6d200700 1 -- 172.21.15.1:0/2105157561 shutdown_connections 2024-04-26T01:47:29.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.706+0000 7f9a6d200700 1 -- 172.21.15.1:0/2105157561 wait complete. 2024-04-26T01:47:29.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.707+0000 7f9a6d200700 1 Processor -- start 2024-04-26T01:47:29.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.707+0000 7f9a6d200700 1 -- start start 2024-04-26T01:47:29.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.708+0000 7f9a6d200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a68102a90 0x7f9a68199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:29.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.708+0000 7f9a6d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a68103450 0x7f9a68199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:29.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.708+0000 7f9a6d200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a68108a50 0x7f9a6819d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:29.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.708+0000 7f9a6d200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9a68112c50 con 0x7f9a68103450 2024-04-26T01:47:29.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.708+0000 7f9a6d200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9a68112ad0 con 0x7f9a68102a90 2024-04-26T01:47:29.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.708+0000 7f9a6d200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9a68112dd0 con 0x7f9a68108a50 2024-04-26T01:47:29.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.708+0000 7f9a65e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a68102a90 0x7f9a68199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:29.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.708+0000 7f9a65400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a68103450 0x7f9a68199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:29.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.708+0000 7f9a66800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a68108a50 0x7f9a6819d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:29.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.708+0000 7f9a65e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a68102a90 0x7f9a68199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:51892/0 (socket says 172.21.15.1:51892) 2024-04-26T01:47:29.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.708+0000 7f9a65e00700 1 -- 172.21.15.1:0/3520809477 learned_addr learned my addr 172.21.15.1:0/3520809477 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:47:29.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.709+0000 7f9a66800700 1 -- 172.21.15.1:0/3520809477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a68102a90 msgr2=0x7f9a68199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:29.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.709+0000 7f9a66800700 1 --2- 172.21.15.1:0/3520809477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a68102a90 0x7f9a68199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:29.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.709+0000 7f9a66800700 1 -- 172.21.15.1:0/3520809477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a68103450 msgr2=0x7f9a68199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:29.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.709+0000 7f9a66800700 1 --2- 172.21.15.1:0/3520809477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a68103450 0x7f9a68199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:29.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.709+0000 7f9a66800700 1 -- 172.21.15.1:0/3520809477 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a6819e100 con 0x7f9a68108a50 2024-04-26T01:47:29.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.709+0000 7f9a66800700 1 --2- 172.21.15.1:0/3520809477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a68108a50 0x7f9a6819d9c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000b6c0 tx=0x7f9a6000cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:29.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.709+0000 7f9a5ea00700 1 -- 172.21.15.1:0/3520809477 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a60021070 con 0x7f9a68108a50 2024-04-26T01:47:29.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.709+0000 7f9a5ea00700 1 -- 172.21.15.1:0/3520809477 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a60007d60 con 0x7f9a68108a50 2024-04-26T01:47:29.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.709+0000 7f9a5ea00700 1 -- 172.21.15.1:0/3520809477 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a60005040 con 0x7f9a68108a50 2024-04-26T01:47:29.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.709+0000 7f9a6d200700 1 -- 172.21.15.1:0/3520809477 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a6819e390 con 0x7f9a68108a50 2024-04-26T01:47:29.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.710+0000 7f9a6d200700 1 -- 172.21.15.1:0/3520809477 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a681a5c70 con 0x7f9a68108a50 2024-04-26T01:47:29.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.711+0000 7f9a6d200700 1 -- 172.21.15.1:0/3520809477 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a68069730 con 0x7f9a68108a50 2024-04-26T01:47:29.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.713+0000 7f9a5ea00700 1 -- 172.21.15.1:0/3520809477 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f9a60024070 con 0x7f9a68108a50 2024-04-26T01:47:29.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.714+0000 7f9a5ea00700 1 --2- 172.21.15.1:0/3520809477 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f9a4c0a3c40 0x7f9a4c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:29.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.714+0000 7f9a5ea00700 1 -- 172.21.15.1:0/3520809477 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7879+0+0 (secure 0 0 0) 0x7f9a6000f020 con 0x7f9a68108a50 2024-04-26T01:47:29.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.714+0000 7f9a65e00700 1 --2- 172.21.15.1:0/3520809477 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f9a4c0a3c40 0x7f9a4c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:29.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.715+0000 7f9a65e00700 1 --2- 172.21.15.1:0/3520809477 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f9a4c0a3c40 0x7f9a4c0a6100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009750 tx=0x7f9a58009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:29.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.718+0000 7f9a5ea00700 1 -- 172.21.15.1:0/3520809477 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a6002b020 con 0x7f9a68108a50 2024-04-26T01:47:29.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:29.997+0000 7f9a6d200700 1 -- 172.21.15.1:0/3520809477 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9a6802df80 con 0x7f9a68108a50 2024-04-26T01:47:30.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:30.123+0000 7f9a5ea00700 1 -- 172.21.15.1:0/3520809477 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f9a60007420 con 0x7f9a68108a50 2024-04-26T01:47:30.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:30.129+0000 7f9a6d200700 1 -- 172.21.15.1:0/3520809477 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f9a4c0a3c40 msgr2=0x7f9a4c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:30.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:30.129+0000 7f9a6d200700 1 --2- 172.21.15.1:0/3520809477 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f9a4c0a3c40 0x7f9a4c0a6100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009750 tx=0x7f9a58009310 comp rx=0 tx=0).stop 2024-04-26T01:47:30.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:30.129+0000 7f9a6d200700 1 -- 172.21.15.1:0/3520809477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a68108a50 msgr2=0x7f9a6819d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:30.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:30.129+0000 7f9a6d200700 1 --2- 172.21.15.1:0/3520809477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a68108a50 0x7f9a6819d9c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000b6c0 tx=0x7f9a6000cc30 comp rx=0 tx=0).stop 2024-04-26T01:47:30.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:30.129+0000 7f9a6d200700 1 -- 172.21.15.1:0/3520809477 shutdown_connections 2024-04-26T01:47:30.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:30.129+0000 7f9a6d200700 1 --2- 172.21.15.1:0/3520809477 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f9a4c0a3c40 0x7f9a4c0a6100 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:30.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:30.129+0000 7f9a6d200700 1 --2- 172.21.15.1:0/3520809477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a68108a50 0x7f9a6819d9c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:30.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:30.129+0000 7f9a6d200700 1 --2- 172.21.15.1:0/3520809477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a68102a90 0x7f9a68199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:30.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:30.129+0000 7f9a6d200700 1 --2- 172.21.15.1:0/3520809477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a68103450 0x7f9a68199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:30.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:30.129+0000 7f9a6d200700 1 -- 172.21.15.1:0/3520809477 >> 172.21.15.1:0/3520809477 conn(0x7f9a68079ba0 msgr2=0x7f9a681017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:30.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:30.130+0000 7f9a6d200700 1 -- 172.21.15.1:0/3520809477 shutdown_connections 2024-04-26T01:47:30.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:30.131+0000 7f9a6d200700 1 -- 172.21.15.1:0/3520809477 wait complete. 2024-04-26T01:47:30.134 INFO:teuthology.orchestra.run.smithi001.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T01:47:30.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:30 smithi001 ceph-mon[27822]: pgmap v323: 97 pgs: 25 creating+peering, 53 active+clean, 19 unknown; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:30.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:30 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3520809477' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T01:47:30.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:30 smithi001 ceph-mon[27822]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T01:47:30.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:30 smithi001 ceph-mon[27822]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T01:47:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:30 smithi097 ceph-mon[32853]: pgmap v323: 97 pgs: 25 creating+peering, 53 active+clean, 19 unknown; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:30 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3520809477' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T01:47:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:30 smithi097 ceph-mon[32853]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T01:47:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:30 smithi097 ceph-mon[32853]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T01:47:30.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:30 smithi173 ceph-mon[32553]: pgmap v323: 97 pgs: 25 creating+peering, 53 active+clean, 19 unknown; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:47:30.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:30 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3520809477' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T01:47:30.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:30 smithi173 ceph-mon[32553]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T01:47:30.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:30 smithi173 ceph-mon[32553]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T01:47:30.963 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:47:31.431 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:47:31.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:31 smithi097 ceph-mon[32853]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T01:47:31.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:31 smithi097 ceph-mon[32853]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T01:47:31.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:31 smithi097 ceph-mon[32853]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T01:47:31.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:31 smithi097 ceph-mon[32853]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T01:47:31.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:31 smithi173 ceph-mon[32553]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T01:47:31.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:31 smithi173 ceph-mon[32553]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T01:47:31.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:31 smithi173 ceph-mon[32553]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T01:47:31.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:31 smithi173 ceph-mon[32553]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T01:47:31.587 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:31 smithi001 ceph-mon[27822]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T01:47:31.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:31 smithi001 ceph-mon[27822]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T01:47:31.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:31 smithi001 ceph-mon[27822]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T01:47:31.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:31 smithi001 ceph-mon[27822]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T01:47:32.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:32 smithi001 ceph-mon[27822]: pgmap v325: 97 pgs: 25 creating+peering, 72 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 0 op/s 2024-04-26T01:47:32.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:32 smithi001 ceph-mon[27822]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T01:47:32.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:32 smithi001 ceph-mon[27822]: Cluster is now healthy 2024-04-26T01:47:32.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:32 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] up:active 2024-04-26T01:47:32.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:32 smithi001 ceph-mon[27822]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:47:32.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:32 smithi001 ceph-mon[27822]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:47:32.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:32 smithi097 ceph-mon[32853]: pgmap v325: 97 pgs: 25 creating+peering, 72 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 0 op/s 2024-04-26T01:47:32.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:32 smithi097 ceph-mon[32853]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T01:47:32.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:32 smithi097 ceph-mon[32853]: Cluster is now healthy 2024-04-26T01:47:32.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:32 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] up:active 2024-04-26T01:47:32.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:32 smithi097 ceph-mon[32853]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:47:32.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:32 smithi097 ceph-mon[32853]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:47:32.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:32 smithi173 ceph-mon[32553]: pgmap v325: 97 pgs: 25 creating+peering, 72 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 0 op/s 2024-04-26T01:47:32.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:32 smithi173 ceph-mon[32553]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T01:47:32.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:32 smithi173 ceph-mon[32553]: Cluster is now healthy 2024-04-26T01:47:32.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:32 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] up:active 2024-04-26T01:47:32.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:32 smithi173 ceph-mon[32553]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:47:32.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:32 smithi173 ceph-mon[32553]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:47:33.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.817+0000 7f792ae00700 1 -- 172.21.15.1:0/817512794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7924076f70 msgr2=0x7f7924077350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:33.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.817+0000 7f792ae00700 1 --2- 172.21.15.1:0/817512794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7924076f70 0x7f7924077350 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f791400b0a0 tx=0x7f791400b3b0 comp rx=0 tx=0).stop 2024-04-26T01:47:33.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.817+0000 7f792ae00700 1 -- 172.21.15.1:0/817512794 shutdown_connections 2024-04-26T01:47:33.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.817+0000 7f792ae00700 1 --2- 172.21.15.1:0/817512794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79241138f0 0x7f7924115ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:33.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.817+0000 7f792ae00700 1 --2- 172.21.15.1:0/817512794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7924076f70 0x7f7924077350 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:33.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.817+0000 7f792ae00700 1 --2- 172.21.15.1:0/817512794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7924077890 0x7f79241133b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:33.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.817+0000 7f792ae00700 1 -- 172.21.15.1:0/817512794 >> 172.21.15.1:0/817512794 conn(0x7f7924100740 msgr2=0x7f7924102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:33.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.817+0000 7f792ae00700 1 -- 172.21.15.1:0/817512794 shutdown_connections 2024-04-26T01:47:33.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.817+0000 7f792ae00700 1 -- 172.21.15.1:0/817512794 wait complete. 2024-04-26T01:47:33.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.819+0000 7f792ae00700 1 Processor -- start 2024-04-26T01:47:33.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.819+0000 7f792ae00700 1 -- start start 2024-04-26T01:47:33.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.819+0000 7f792ae00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7924076f70 0x7f79241a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:33.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.819+0000 7f792ae00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7924077890 0x7f79241a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:33.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.819+0000 7f792ae00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79241138f0 0x7f792419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:33.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.819+0000 7f792ae00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7924119260 con 0x7f7924077890 2024-04-26T01:47:33.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.820+0000 7f792ae00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f79241190e0 con 0x7f79241138f0 2024-04-26T01:47:33.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.820+0000 7f792ae00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f79241193e0 con 0x7f7924076f70 2024-04-26T01:47:33.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.820+0000 7f7923e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7924076f70 0x7f79241a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:33.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.820+0000 7f7923e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7924076f70 0x7f79241a53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:34674/0 (socket says 172.21.15.1:34674) 2024-04-26T01:47:33.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.820+0000 7f7923e00700 1 -- 172.21.15.1:0/3859614991 learned_addr learned my addr 172.21.15.1:0/3859614991 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:47:33.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.820+0000 7f791b400700 1 --2- 172.21.15.1:0/3859614991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7924077890 0x7f79241a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:33.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.820+0000 7f7928e00700 1 --2- 172.21.15.1:0/3859614991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79241138f0 0x7f792419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:33.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.820+0000 7f7923e00700 1 -- 172.21.15.1:0/3859614991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79241138f0 msgr2=0x7f792419f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:33.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.820+0000 7f7923e00700 1 --2- 172.21.15.1:0/3859614991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79241138f0 0x7f792419f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:33.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.820+0000 7f7923e00700 1 -- 172.21.15.1:0/3859614991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7924077890 msgr2=0x7f79241a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:33.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.820+0000 7f7923e00700 1 --2- 172.21.15.1:0/3859614991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7924077890 0x7f79241a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:33.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.820+0000 7f7923e00700 1 -- 172.21.15.1:0/3859614991 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792419fd40 con 0x7f7924076f70 2024-04-26T01:47:33.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.821+0000 7f7923e00700 1 --2- 172.21.15.1:0/3859614991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7924076f70 0x7f79241a53d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f791400b070 tx=0x7f7914004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:33.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.821+0000 7f7921600700 1 -- 172.21.15.1:0/3859614991 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7914021070 con 0x7f7924076f70 2024-04-26T01:47:33.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.821+0000 7f792ae00700 1 -- 172.21.15.1:0/3859614991 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792419ffd0 con 0x7f7924076f70 2024-04-26T01:47:33.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.821+0000 7f792ae00700 1 -- 172.21.15.1:0/3859614991 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79241ac2a0 con 0x7f7924076f70 2024-04-26T01:47:33.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.821+0000 7f7921600700 1 -- 172.21.15.1:0/3859614991 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7914007e30 con 0x7f7924076f70 2024-04-26T01:47:33.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.821+0000 7f7921600700 1 -- 172.21.15.1:0/3859614991 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79140044a0 con 0x7f7924076f70 2024-04-26T01:47:33.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.822+0000 7f792ae00700 1 -- 172.21.15.1:0/3859614991 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7924069730 con 0x7f7924076f70 2024-04-26T01:47:33.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.824+0000 7f7921600700 1 -- 172.21.15.1:0/3859614991 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f7914003c00 con 0x7f7924076f70 2024-04-26T01:47:33.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.826+0000 7f7921600700 1 --2- 172.21.15.1:0/3859614991 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f79080a3cc0 0x7f79080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:33.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.826+0000 7f7921600700 1 -- 172.21.15.1:0/3859614991 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f79140d0f20 con 0x7f7924076f70 2024-04-26T01:47:33.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.826+0000 7f791b400700 1 --2- 172.21.15.1:0/3859614991 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f79080a3cc0 0x7f79080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:33.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.827+0000 7f791b400700 1 --2- 172.21.15.1:0/3859614991 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f79080a3cc0 0x7f79080a6180 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7924103e90 tx=0x7f790c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:33.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:33.828+0000 7f7921600700 1 -- 172.21.15.1:0/3859614991 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f791402a030 con 0x7f7924076f70 2024-04-26T01:47:34.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:34.059+0000 7f792ae00700 1 -- 172.21.15.1:0/3859614991 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f792402df80 con 0x7f7924076f70 2024-04-26T01:47:34.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:34.061+0000 7f7921600700 1 -- 172.21.15.1:0/3859614991 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22124 (secure 0 0 0) 0x7f791409da40 con 0x7f7924076f70 2024-04-26T01:47:34.059 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:47:34.059 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":90,"fsid":"77cfcc80-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:38:03.531182+0000","modified":"2024-04-26T01:47:29.999575+0000","last_up_change":"2024-04-26T01:46:16.151275+0000","last_in_change":"2024-04-26T01:45:56.514228+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:42:04.805488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:47:21.957421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:47:25.860260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb10ae12-09ee-4d6b-bc78-8c6016613b81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6803","nonce":3176633965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6805","nonce":3176633965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6809","nonce":3176633965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6807","nonce":3176633965}]},"public_addr":"172.21.15.1:6803/3176633965","cluster_addr":"172.21.15.1:6805/3176633965","heartbeat_back_addr":"172.21.15.1:6809/3176633965","heartbeat_front_addr":"172.21.15.1:6807/3176633965","state":["exists","up"]},{"osd":1,"uuid":"acd12888-6ff6-40bc-99e7-e5c395e85b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6801","nonce":3755022621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6803","nonce":3755022621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6807","nonce":3755022621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6805","nonce":3755022621}]},"public_addr":"172.21.15.97:6801/3755022621","cluster_addr":"172.21.15.97:6803/3755022621","heartbeat_back_addr":"172.21.15.97:6807/3755022621","heartbeat_front_addr":"172.21.15.97:6805/3755022621","state":["exists","up"]},{"osd":2,"uuid":"79971508-e256-4145-8111-a75666e886fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6801","nonce":734861771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6803","nonce":734861771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6807","nonce":734861771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6805","nonce":734861771}]},"public_addr":"172.21.15.173:6801/734861771","cluster_addr":"172.21.15.173:6803/734861771","heartbeat_back_addr":"172.21.15.173:6807/734861771","heartbeat_front_addr":"172.21.15.173:6805/734861771","state":["exists","up"]},{"osd":3,"uuid":"a8ccc4a9-0f31-446a-8bcb-4691532397c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6811","nonce":3349961961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6813","nonce":3349961961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6817","nonce":3349961961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6815","nonce":3349961961}]},"public_addr":"172.21.15.1:6811/3349961961","cluster_addr":"172.21.15.1:6813/3349961961","heartbeat_back_addr":"172.21.15.1:6817/3349961961","heartbeat_front_addr":"172.21.15.1:6815/3349961961","state":["exists","up"]},{"osd":4,"uuid":"5b724648-4df7-40fc-89d5-55f170cdd806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6809","nonce":2494639397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6811","nonce":2494639397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6815","nonce":2494639397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6813","nonce":2494639397}]},"public_addr":"172.21.15.97:6809/2494639397","cluster_addr":"172.21.15.97:6811/2494639397","heartbeat_back_addr":"172.21.15.97:6815/2494639397","heartbeat_front_addr":"172.21.15.97:6813/2494639397","state":["exists","up"]},{"osd":5,"uuid":"a5a24a9c-3007-4b87-8562-40c98c41ef8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6809","nonce":1026507957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6811","nonce":1026507957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6815","nonce":1026507957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6813","nonce":1026507957}]},"public_addr":"172.21.15.173:6809/1026507957","cluster_addr":"172.21.15.173:6811/1026507957","heartbeat_back_addr":"172.21.15.173:6815/1026507957","heartbeat_front_addr":"172.21.15.173:6813/1026507957","state":["exists","up"]},{"osd":6,"uuid":"97cd2e88-a56c-4b1a-8104-65b23a77986a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6819","nonce":444030587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6821","nonce":444030587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6825","nonce":444030587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6823","nonce":444030587}]},"public_addr":"172.21.15.1:6819/444030587","cluster_addr":"172.21.15.1:6821/444030587","heartbeat_back_addr":"172.21.15.1:6825/444030587","heartbeat_front_addr":"172.21.15.1:6823/444030587","state":["exists","up"]},{"osd":7,"uuid":"afc0c2dd-e112-4358-b3aa-944366d5d383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6817","nonce":2600022531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6819","nonce":2600022531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6823","nonce":2600022531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6821","nonce":2600022531}]},"public_addr":"172.21.15.97:6817/2600022531","cluster_addr":"172.21.15.97:6819/2600022531","heartbeat_back_addr":"172.21.15.97:6823/2600022531","heartbeat_front_addr":"172.21.15.97:6821/2600022531","state":["exists","up"]},{"osd":8,"uuid":"41fa612b-793e-44da-af8b-44d17a9de303","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6817","nonce":1334205527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6819","nonce":1334205527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6823","nonce":1334205527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6821","nonce":1334205527}]},"public_addr":"172.21.15.173:6817/1334205527","cluster_addr":"172.21.15.173:6819/1334205527","heartbeat_back_addr":"172.21.15.173:6823/1334205527","heartbeat_front_addr":"172.21.15.173:6821/1334205527","state":["exists","up"]},{"osd":9,"uuid":"bbbb163b-f8c4-4be5-8f80-82445e8d407f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6827","nonce":2973486839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6829","nonce":2973486839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6833","nonce":2973486839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6831","nonce":2973486839}]},"public_addr":"172.21.15.1:6827/2973486839","cluster_addr":"172.21.15.1:6829/2973486839","heartbeat_back_addr":"172.21.15.1:6833/2973486839","heartbeat_front_addr":"172.21.15.1:6831/2973486839","state":["exists","up"]},{"osd":10,"uuid":"1b447b34-0600-4727-826d-b3d68adfb18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6825","nonce":1693425268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6827","nonce":1693425268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6831","nonce":1693425268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6829","nonce":1693425268}]},"public_addr":"172.21.15.97:6825/1693425268","cluster_addr":"172.21.15.97:6827/1693425268","heartbeat_back_addr":"172.21.15.97:6831/1693425268","heartbeat_front_addr":"172.21.15.97:6829/1693425268","state":["exists","up"]},{"osd":11,"uuid":"f439aae1-f50a-4176-889d-36fa5131541a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6825","nonce":1141524960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6827","nonce":1141524960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6831","nonce":1141524960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6829","nonce":1141524960}]},"public_addr":"172.21.15.173:6825/1141524960","cluster_addr":"172.21.15.173:6827/1141524960","heartbeat_back_addr":"172.21.15.173:6831/1141524960","heartbeat_front_addr":"172.21.15.173:6829/1141524960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:12.955643+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:38.846652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:02.556062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:34.900933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:01.170878+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:24.842573+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:58.304440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:24.868689+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:48.772003+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:22.611681+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:49.470779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:14.163984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6800/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/1000222947":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/457958597":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6801/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:6800/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/2715798962":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/1923058728":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/2489038777":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/2869502448":"2024-04-27T01:38:45.974031+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:47:34.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:34.064+0000 7f792ae00700 1 -- 172.21.15.1:0/3859614991 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f79080a3cc0 msgr2=0x7f79080a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:34.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:34.064+0000 7f792ae00700 1 --2- 172.21.15.1:0/3859614991 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f79080a3cc0 0x7f79080a6180 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7924103e90 tx=0x7f790c009260 comp rx=0 tx=0).stop 2024-04-26T01:47:34.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:34.064+0000 7f792ae00700 1 -- 172.21.15.1:0/3859614991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7924076f70 msgr2=0x7f79241a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:34.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:34.064+0000 7f792ae00700 1 --2- 172.21.15.1:0/3859614991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7924076f70 0x7f79241a53d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f791400b070 tx=0x7f7914004d00 comp rx=0 tx=0).stop 2024-04-26T01:47:34.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:34.065+0000 7f792ae00700 1 -- 172.21.15.1:0/3859614991 shutdown_connections 2024-04-26T01:47:34.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:34.065+0000 7f792ae00700 1 --2- 172.21.15.1:0/3859614991 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f79080a3cc0 0x7f79080a6180 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:34.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:34.065+0000 7f792ae00700 1 --2- 172.21.15.1:0/3859614991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79241138f0 0x7f792419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:34.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:34.065+0000 7f792ae00700 1 --2- 172.21.15.1:0/3859614991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7924076f70 0x7f79241a53d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:34.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:34.065+0000 7f792ae00700 1 --2- 172.21.15.1:0/3859614991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7924077890 0x7f79241a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:34.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:34.065+0000 7f792ae00700 1 -- 172.21.15.1:0/3859614991 >> 172.21.15.1:0/3859614991 conn(0x7f7924100740 msgr2=0x7f7924102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:34.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:34.065+0000 7f792ae00700 1 -- 172.21.15.1:0/3859614991 shutdown_connections 2024-04-26T01:47:34.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:34.065+0000 7f792ae00700 1 -- 172.21.15.1:0/3859614991 wait complete. 2024-04-26T01:47:34.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:34 smithi001 ceph-mon[27822]: pgmap v326: 97 pgs: 97 active+clean; 578 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-26T01:47:34.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:34 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3859614991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:47:34.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:34 smithi097 ceph-mon[32853]: pgmap v326: 97 pgs: 97 active+clean; 578 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-26T01:47:34.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:34 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3859614991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:47:34.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:34 smithi173 ceph-mon[32553]: pgmap v326: 97 pgs: 97 active+clean; 578 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-26T01:47:34.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:34 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3859614991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:47:34.883 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:47:35.233 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 01:47:34 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-b[62484]: 2024-04-26T01:47:34.928+0000 7f2b66035700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T01:47:35.381 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:47:36.421 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:36 smithi001 ceph-mon[27822]: pgmap v327: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-26T01:47:36.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:36 smithi097 ceph-mon[32853]: pgmap v327: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-26T01:47:36.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:36 smithi173 ceph-mon[32553]: pgmap v327: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-26T01:47:37.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.725+0000 7f8375000700 1 -- 172.21.15.1:0/3834364790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8370108a30 msgr2=0x7f8370108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:37.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.725+0000 7f8375000700 1 --2- 172.21.15.1:0/3834364790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8370108a30 0x7f8370108e10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f83600099c0 tx=0x7f8360009cd0 comp rx=0 tx=0).stop 2024-04-26T01:47:37.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.725+0000 7f8375000700 1 -- 172.21.15.1:0/3834364790 shutdown_connections 2024-04-26T01:47:37.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.725+0000 7f8375000700 1 --2- 172.21.15.1:0/3834364790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f837006bb10 0x7f837010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:37.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.725+0000 7f8375000700 1 --2- 172.21.15.1:0/3834364790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8370108a30 0x7f8370108e10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:37.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.725+0000 7f8375000700 1 --2- 172.21.15.1:0/3834364790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f837006b170 0x7f837006b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:37.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.725+0000 7f8375000700 1 -- 172.21.15.1:0/3834364790 >> 172.21.15.1:0/3834364790 conn(0x7f83700788e0 msgr2=0x7f8370078cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:37.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.725+0000 7f8375000700 1 -- 172.21.15.1:0/3834364790 shutdown_connections 2024-04-26T01:47:37.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.725+0000 7f8375000700 1 -- 172.21.15.1:0/3834364790 wait complete. 2024-04-26T01:47:37.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.726+0000 7f8375000700 1 Processor -- start 2024-04-26T01:47:37.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.727+0000 7f8375000700 1 -- start start 2024-04-26T01:47:37.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.727+0000 7f8375000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f837006b170 0x7f8370101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:37.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.727+0000 7f8375000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f837006bb10 0x7f8370101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:37.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.727+0000 7f8375000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8370108a30 0x7f83701024c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:37.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.727+0000 7f8375000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8370112c00 con 0x7f837006b170 2024-04-26T01:47:37.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.727+0000 7f8375000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8370112a80 con 0x7f8370108a30 2024-04-26T01:47:37.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.727+0000 7f8375000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8370112d80 con 0x7f837006bb10 2024-04-26T01:47:37.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.727+0000 7f836de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f837006b170 0x7f8370101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:37.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.727+0000 7f8367e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f837006bb10 0x7f8370101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:37.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.728+0000 7f8367e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f837006bb10 0x7f8370101f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:34692/0 (socket says 172.21.15.1:34692) 2024-04-26T01:47:37.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.728+0000 7f8367e00700 1 -- 172.21.15.1:0/3208291305 learned_addr learned my addr 172.21.15.1:0/3208291305 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:47:37.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.728+0000 7f836e800700 1 --2- 172.21.15.1:0/3208291305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8370108a30 0x7f83701024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:37.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.728+0000 7f8367e00700 1 -- 172.21.15.1:0/3208291305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8370108a30 msgr2=0x7f83701024c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:37.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.728+0000 7f8367e00700 1 --2- 172.21.15.1:0/3208291305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8370108a30 0x7f83701024c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:37.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.728+0000 7f8367e00700 1 -- 172.21.15.1:0/3208291305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f837006b170 msgr2=0x7f8370101a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:37.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.728+0000 7f8367e00700 1 --2- 172.21.15.1:0/3208291305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f837006b170 0x7f8370101a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:37.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.728+0000 7f8367e00700 1 -- 172.21.15.1:0/3208291305 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83701a76a0 con 0x7f837006bb10 2024-04-26T01:47:37.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.728+0000 7f836de00700 1 --2- 172.21.15.1:0/3208291305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f837006b170 0x7f8370101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:47:37.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.728+0000 7f8367e00700 1 --2- 172.21.15.1:0/3208291305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f837006bb10 0x7f8370101f80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f835800e9a0 tx=0x7f835800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:37.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.729+0000 7f8366a00700 1 -- 172.21.15.1:0/3208291305 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835800ca10 con 0x7f837006bb10 2024-04-26T01:47:37.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.729+0000 7f8375000700 1 -- 172.21.15.1:0/3208291305 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83701a7990 con 0x7f837006bb10 2024-04-26T01:47:37.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.729+0000 7f8366a00700 1 -- 172.21.15.1:0/3208291305 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f835800cbb0 con 0x7f837006bb10 2024-04-26T01:47:37.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.729+0000 7f8366a00700 1 -- 172.21.15.1:0/3208291305 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358018680 con 0x7f837006bb10 2024-04-26T01:47:37.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.729+0000 7f8375000700 1 -- 172.21.15.1:0/3208291305 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83701a7f20 con 0x7f837006bb10 2024-04-26T01:47:37.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.730+0000 7f8375000700 1 -- 172.21.15.1:0/3208291305 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8370104170 con 0x7f837006bb10 2024-04-26T01:47:37.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.732+0000 7f8366a00700 1 -- 172.21.15.1:0/3208291305 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f8358010430 con 0x7f837006bb10 2024-04-26T01:47:37.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.733+0000 7f8366a00700 1 --2- 172.21.15.1:0/3208291305 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f83540a3cc0 0x7f83540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:37.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.734+0000 7f8366a00700 1 -- 172.21.15.1:0/3208291305 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f83580d0440 con 0x7f837006bb10 2024-04-26T01:47:37.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.734+0000 7f836de00700 1 --2- 172.21.15.1:0/3208291305 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f83540a3cc0 0x7f83540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:37.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.734+0000 7f836de00700 1 --2- 172.21.15.1:0/3208291305 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f83540a3cc0 0x7f83540a6180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8360003760 tx=0x7f8360005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:37.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.738+0000 7f8366a00700 1 -- 172.21.15.1:0/3208291305 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f835809ced0 con 0x7f837006bb10 2024-04-26T01:47:37.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.968+0000 7f8375000700 1 -- 172.21.15.1:0/3208291305 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f837002e140 con 0x7f837006bb10 2024-04-26T01:47:37.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.970+0000 7f8366a00700 1 -- 172.21.15.1:0/3208291305 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22124 (secure 0 0 0) 0x7f835809c7d0 con 0x7f837006bb10 2024-04-26T01:47:37.969 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:47:37.969 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":90,"fsid":"77cfcc80-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:38:03.531182+0000","modified":"2024-04-26T01:47:29.999575+0000","last_up_change":"2024-04-26T01:46:16.151275+0000","last_in_change":"2024-04-26T01:45:56.514228+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:42:04.805488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:47:21.957421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:47:25.860260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb10ae12-09ee-4d6b-bc78-8c6016613b81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6803","nonce":3176633965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6805","nonce":3176633965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6809","nonce":3176633965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6807","nonce":3176633965}]},"public_addr":"172.21.15.1:6803/3176633965","cluster_addr":"172.21.15.1:6805/3176633965","heartbeat_back_addr":"172.21.15.1:6809/3176633965","heartbeat_front_addr":"172.21.15.1:6807/3176633965","state":["exists","up"]},{"osd":1,"uuid":"acd12888-6ff6-40bc-99e7-e5c395e85b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6801","nonce":3755022621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6803","nonce":3755022621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6807","nonce":3755022621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6805","nonce":3755022621}]},"public_addr":"172.21.15.97:6801/3755022621","cluster_addr":"172.21.15.97:6803/3755022621","heartbeat_back_addr":"172.21.15.97:6807/3755022621","heartbeat_front_addr":"172.21.15.97:6805/3755022621","state":["exists","up"]},{"osd":2,"uuid":"79971508-e256-4145-8111-a75666e886fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6801","nonce":734861771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6803","nonce":734861771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6807","nonce":734861771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6805","nonce":734861771}]},"public_addr":"172.21.15.173:6801/734861771","cluster_addr":"172.21.15.173:6803/734861771","heartbeat_back_addr":"172.21.15.173:6807/734861771","heartbeat_front_addr":"172.21.15.173:6805/734861771","state":["exists","up"]},{"osd":3,"uuid":"a8ccc4a9-0f31-446a-8bcb-4691532397c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6811","nonce":3349961961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6813","nonce":3349961961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6817","nonce":3349961961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6815","nonce":3349961961}]},"public_addr":"172.21.15.1:6811/3349961961","cluster_addr":"172.21.15.1:6813/3349961961","heartbeat_back_addr":"172.21.15.1:6817/3349961961","heartbeat_front_addr":"172.21.15.1:6815/3349961961","state":["exists","up"]},{"osd":4,"uuid":"5b724648-4df7-40fc-89d5-55f170cdd806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6809","nonce":2494639397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6811","nonce":2494639397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6815","nonce":2494639397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6813","nonce":2494639397}]},"public_addr":"172.21.15.97:6809/2494639397","cluster_addr":"172.21.15.97:6811/2494639397","heartbeat_back_addr":"172.21.15.97:6815/2494639397","heartbeat_front_addr":"172.21.15.97:6813/2494639397","state":["exists","up"]},{"osd":5,"uuid":"a5a24a9c-3007-4b87-8562-40c98c41ef8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6809","nonce":1026507957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6811","nonce":1026507957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6815","nonce":1026507957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6813","nonce":1026507957}]},"public_addr":"172.21.15.173:6809/1026507957","cluster_addr":"172.21.15.173:6811/1026507957","heartbeat_back_addr":"172.21.15.173:6815/1026507957","heartbeat_front_addr":"172.21.15.173:6813/1026507957","state":["exists","up"]},{"osd":6,"uuid":"97cd2e88-a56c-4b1a-8104-65b23a77986a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6819","nonce":444030587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6821","nonce":444030587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6825","nonce":444030587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6823","nonce":444030587}]},"public_addr":"172.21.15.1:6819/444030587","cluster_addr":"172.21.15.1:6821/444030587","heartbeat_back_addr":"172.21.15.1:6825/444030587","heartbeat_front_addr":"172.21.15.1:6823/444030587","state":["exists","up"]},{"osd":7,"uuid":"afc0c2dd-e112-4358-b3aa-944366d5d383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6817","nonce":2600022531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6819","nonce":2600022531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6823","nonce":2600022531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6821","nonce":2600022531}]},"public_addr":"172.21.15.97:6817/2600022531","cluster_addr":"172.21.15.97:6819/2600022531","heartbeat_back_addr":"172.21.15.97:6823/2600022531","heartbeat_front_addr":"172.21.15.97:6821/2600022531","state":["exists","up"]},{"osd":8,"uuid":"41fa612b-793e-44da-af8b-44d17a9de303","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6817","nonce":1334205527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6819","nonce":1334205527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6823","nonce":1334205527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6821","nonce":1334205527}]},"public_addr":"172.21.15.173:6817/1334205527","cluster_addr":"172.21.15.173:6819/1334205527","heartbeat_back_addr":"172.21.15.173:6823/1334205527","heartbeat_front_addr":"172.21.15.173:6821/1334205527","state":["exists","up"]},{"osd":9,"uuid":"bbbb163b-f8c4-4be5-8f80-82445e8d407f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6827","nonce":2973486839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6829","nonce":2973486839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6833","nonce":2973486839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6831","nonce":2973486839}]},"public_addr":"172.21.15.1:6827/2973486839","cluster_addr":"172.21.15.1:6829/2973486839","heartbeat_back_addr":"172.21.15.1:6833/2973486839","heartbeat_front_addr":"172.21.15.1:6831/2973486839","state":["exists","up"]},{"osd":10,"uuid":"1b447b34-0600-4727-826d-b3d68adfb18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6825","nonce":1693425268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6827","nonce":1693425268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6831","nonce":1693425268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6829","nonce":1693425268}]},"public_addr":"172.21.15.97:6825/1693425268","cluster_addr":"172.21.15.97:6827/1693425268","heartbeat_back_addr":"172.21.15.97:6831/1693425268","heartbeat_front_addr":"172.21.15.97:6829/1693425268","state":["exists","up"]},{"osd":11,"uuid":"f439aae1-f50a-4176-889d-36fa5131541a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6825","nonce":1141524960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6827","nonce":1141524960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6831","nonce":1141524960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6829","nonce":1141524960}]},"public_addr":"172.21.15.173:6825/1141524960","cluster_addr":"172.21.15.173:6827/1141524960","heartbeat_back_addr":"172.21.15.173:6831/1141524960","heartbeat_front_addr":"172.21.15.173:6829/1141524960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:12.955643+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:38.846652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:02.556062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:34.900933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:01.170878+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:24.842573+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:58.304440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:24.868689+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:48.772003+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:22.611681+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:49.470779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:14.163984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6800/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/1000222947":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/457958597":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6801/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:6800/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/2715798962":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/1923058728":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/2489038777":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/2869502448":"2024-04-27T01:38:45.974031+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:47:37.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.973+0000 7f8375000700 1 -- 172.21.15.1:0/3208291305 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f83540a3cc0 msgr2=0x7f83540a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:37.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.973+0000 7f8375000700 1 --2- 172.21.15.1:0/3208291305 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f83540a3cc0 0x7f83540a6180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8360003760 tx=0x7f8360005fb0 comp rx=0 tx=0).stop 2024-04-26T01:47:37.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.973+0000 7f8375000700 1 -- 172.21.15.1:0/3208291305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f837006bb10 msgr2=0x7f8370101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:37.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.973+0000 7f8375000700 1 --2- 172.21.15.1:0/3208291305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f837006bb10 0x7f8370101f80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f835800e9a0 tx=0x7f835800ecb0 comp rx=0 tx=0).stop 2024-04-26T01:47:37.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.973+0000 7f8375000700 1 -- 172.21.15.1:0/3208291305 shutdown_connections 2024-04-26T01:47:37.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.973+0000 7f8375000700 1 --2- 172.21.15.1:0/3208291305 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f83540a3cc0 0x7f83540a6180 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:37.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.973+0000 7f8375000700 1 --2- 172.21.15.1:0/3208291305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8370108a30 0x7f83701024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:37.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.973+0000 7f8375000700 1 --2- 172.21.15.1:0/3208291305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f837006b170 0x7f8370101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:37.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.973+0000 7f8375000700 1 --2- 172.21.15.1:0/3208291305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f837006bb10 0x7f8370101f80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:37.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.973+0000 7f8375000700 1 -- 172.21.15.1:0/3208291305 >> 172.21.15.1:0/3208291305 conn(0x7f83700788e0 msgr2=0x7f83700ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:37.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.974+0000 7f8375000700 1 -- 172.21.15.1:0/3208291305 shutdown_connections 2024-04-26T01:47:37.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:37.974+0000 7f8375000700 1 -- 172.21.15.1:0/3208291305 wait complete. 2024-04-26T01:47:38.353 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:38 smithi001 ceph-mon[27822]: pgmap v328: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-04-26T01:47:38.353 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:38 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3208291305' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:47:38.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:38 smithi097 ceph-mon[32853]: pgmap v328: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-04-26T01:47:38.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:38 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3208291305' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:47:38.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:38 smithi173 ceph-mon[32553]: pgmap v328: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-04-26T01:47:38.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:38 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3208291305' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:47:38.739 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-04-26T01:47:39.204 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:47:40.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:40 smithi097 ceph-mon[32853]: pgmap v329: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T01:47:40.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:40 smithi173 ceph-mon[32553]: pgmap v329: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T01:47:40.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:40 smithi001 ceph-mon[27822]: pgmap v329: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T01:47:41.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.557+0000 7f2d9d600700 1 -- 172.21.15.1:0/349092353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d98102a90 msgr2=0x7f2d98102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:41.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.557+0000 7f2d9d600700 1 --2- 172.21.15.1:0/349092353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d98102a90 0x7f2d98102f10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2d80009990 tx=0x7f2d80009ca0 comp rx=0 tx=0).stop 2024-04-26T01:47:41.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.557+0000 7f2d9d600700 1 -- 172.21.15.1:0/349092353 shutdown_connections 2024-04-26T01:47:41.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.557+0000 7f2d9d600700 1 --2- 172.21.15.1:0/349092353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d98103450 0x7f2d9810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:41.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.557+0000 7f2d9d600700 1 --2- 172.21.15.1:0/349092353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d98108a50 0x7f2d98108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:41.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.557+0000 7f2d9d600700 1 --2- 172.21.15.1:0/349092353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d98102a90 0x7f2d98102f10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:41.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.557+0000 7f2d9d600700 1 -- 172.21.15.1:0/349092353 >> 172.21.15.1:0/349092353 conn(0x7f2d98079ba0 msgr2=0x7f2d98079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:41.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.557+0000 7f2d9d600700 1 -- 172.21.15.1:0/349092353 shutdown_connections 2024-04-26T01:47:41.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.558+0000 7f2d9d600700 1 -- 172.21.15.1:0/349092353 wait complete. 2024-04-26T01:47:41.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.559+0000 7f2d9d600700 1 Processor -- start 2024-04-26T01:47:41.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.559+0000 7f2d9d600700 1 -- start start 2024-04-26T01:47:41.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.560+0000 7f2d9d600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d98102a90 0x7f2d98074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:41.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.560+0000 7f2d9d600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d98103450 0x7f2d98075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:41.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.560+0000 7f2d9d600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d98108a50 0x7f2d980757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:41.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.560+0000 7f2d9d600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2d98112be0 con 0x7f2d98108a50 2024-04-26T01:47:41.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.560+0000 7f2d9d600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2d98112a60 con 0x7f2d98102a90 2024-04-26T01:47:41.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.560+0000 7f2d9d600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2d98112d60 con 0x7f2d98103450 2024-04-26T01:47:41.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.560+0000 7f2d96a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d98102a90 0x7f2d98074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:41.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.560+0000 7f2d96a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d98102a90 0x7f2d98074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:52142/0 (socket says 172.21.15.1:52142) 2024-04-26T01:47:41.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.560+0000 7f2d96a00700 1 -- 172.21.15.1:0/2284792419 learned_addr learned my addr 172.21.15.1:0/2284792419 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:47:41.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.560+0000 7f2d8e000700 1 --2- 172.21.15.1:0/2284792419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d98103450 0x7f2d98075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:41.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.560+0000 7f2d97400700 1 --2- 172.21.15.1:0/2284792419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d98108a50 0x7f2d980757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:41.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.560+0000 7f2d96a00700 1 -- 172.21.15.1:0/2284792419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d98103450 msgr2=0x7f2d98075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:41.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.560+0000 7f2d96a00700 1 --2- 172.21.15.1:0/2284792419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d98103450 0x7f2d98075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:41.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.560+0000 7f2d96a00700 1 -- 172.21.15.1:0/2284792419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d98108a50 msgr2=0x7f2d980757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:41.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.560+0000 7f2d96a00700 1 --2- 172.21.15.1:0/2284792419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d98108a50 0x7f2d980757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:41.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.560+0000 7f2d96a00700 1 -- 172.21.15.1:0/2284792419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d981a7660 con 0x7f2d98102a90 2024-04-26T01:47:41.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.560+0000 7f2d97400700 1 --2- 172.21.15.1:0/2284792419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d98108a50 0x7f2d980757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:47:41.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.561+0000 7f2d96a00700 1 --2- 172.21.15.1:0/2284792419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d98102a90 0x7f2d98074d40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800c370 tx=0x7f2d8800c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:41.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.561+0000 7f2d8fe00700 1 -- 172.21.15.1:0/2284792419 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8800cea0 con 0x7f2d98102a90 2024-04-26T01:47:41.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.561+0000 7f2d8fe00700 1 -- 172.21.15.1:0/2284792419 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d880090d0 con 0x7f2d98102a90 2024-04-26T01:47:41.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.561+0000 7f2d8fe00700 1 -- 172.21.15.1:0/2284792419 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88004770 con 0x7f2d98102a90 2024-04-26T01:47:41.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.561+0000 7f2d9d600700 1 -- 172.21.15.1:0/2284792419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d981a78f0 con 0x7f2d98102a90 2024-04-26T01:47:41.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.561+0000 7f2d9d600700 1 -- 172.21.15.1:0/2284792419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d981a7e50 con 0x7f2d98102a90 2024-04-26T01:47:41.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.562+0000 7f2d9d600700 1 -- 172.21.15.1:0/2284792419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d98069730 con 0x7f2d98102a90 2024-04-26T01:47:41.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.564+0000 7f2d8fe00700 1 -- 172.21.15.1:0/2284792419 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f2d8801a020 con 0x7f2d98102a90 2024-04-26T01:47:41.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.565+0000 7f2d8fe00700 1 --2- 172.21.15.1:0/2284792419 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2d7c0a3cc0 0x7f2d7c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:41.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.566+0000 7f2d8fe00700 1 -- 172.21.15.1:0/2284792419 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f2d880cfc60 con 0x7f2d98102a90 2024-04-26T01:47:41.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.566+0000 7f2d8e000700 1 --2- 172.21.15.1:0/2284792419 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2d7c0a3cc0 0x7f2d7c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:41.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.566+0000 7f2d8e000700 1 --2- 172.21.15.1:0/2284792419 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2d7c0a3cc0 0x7f2d7c0a6180 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2d8000b2e0 tx=0x7f2d80005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:41.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.570+0000 7f2d8fe00700 1 -- 172.21.15.1:0/2284792419 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d88010040 con 0x7f2d98102a90 2024-04-26T01:47:41.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:41.851+0000 7f2d9d600700 1 -- 172.21.15.1:0/2284792419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2d9802e1a0 con 0x7f2d98102a90 2024-04-26T01:47:42.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:42.164+0000 7f2d8fe00700 1 -- 172.21.15.1:0/2284792419 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f2d880bc020 con 0x7f2d98102a90 2024-04-26T01:47:42.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:42.169+0000 7f2d9d600700 1 -- 172.21.15.1:0/2284792419 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2d7c0a3cc0 msgr2=0x7f2d7c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:42.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:42.169+0000 7f2d9d600700 1 --2- 172.21.15.1:0/2284792419 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2d7c0a3cc0 0x7f2d7c0a6180 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2d8000b2e0 tx=0x7f2d80005960 comp rx=0 tx=0).stop 2024-04-26T01:47:42.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:42.169+0000 7f2d9d600700 1 -- 172.21.15.1:0/2284792419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d98102a90 msgr2=0x7f2d98074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:42.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:42.169+0000 7f2d9d600700 1 --2- 172.21.15.1:0/2284792419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d98102a90 0x7f2d98074d40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800c370 tx=0x7f2d8800c680 comp rx=0 tx=0).stop 2024-04-26T01:47:42.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:42.169+0000 7f2d9d600700 1 -- 172.21.15.1:0/2284792419 shutdown_connections 2024-04-26T01:47:42.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:42.169+0000 7f2d9d600700 1 --2- 172.21.15.1:0/2284792419 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2d7c0a3cc0 0x7f2d7c0a6180 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:42.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:42.169+0000 7f2d9d600700 1 --2- 172.21.15.1:0/2284792419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d98108a50 0x7f2d980757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:42.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:42.169+0000 7f2d9d600700 1 --2- 172.21.15.1:0/2284792419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d98102a90 0x7f2d98074d40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:42.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:42.169+0000 7f2d9d600700 1 --2- 172.21.15.1:0/2284792419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d98103450 0x7f2d98075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:42.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:42.169+0000 7f2d9d600700 1 -- 172.21.15.1:0/2284792419 >> 172.21.15.1:0/2284792419 conn(0x7f2d98079ba0 msgr2=0x7f2d98101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:42.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:42.170+0000 7f2d9d600700 1 -- 172.21.15.1:0/2284792419 shutdown_connections 2024-04-26T01:47:42.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:42.170+0000 7f2d9d600700 1 -- 172.21.15.1:0/2284792419 wait complete. 2024-04-26T01:47:42.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:42 smithi097 ceph-mon[32853]: pgmap v330: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-26T01:47:42.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:42 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/2284792419' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T01:47:42.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:42 smithi097 ceph-mon[32853]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T01:47:42.560 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:42 smithi001 ceph-mon[27822]: pgmap v330: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-26T01:47:42.560 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:42 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2284792419' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T01:47:42.560 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:42 smithi001 ceph-mon[27822]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T01:47:42.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:42 smithi173 ceph-mon[32553]: pgmap v330: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-26T01:47:42.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:42 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/2284792419' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T01:47:42.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:42 smithi173 ceph-mon[32553]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T01:47:42.968 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-26T01:47:42.968 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-04-26T01:47:43.454 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:47:43.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:43 smithi097 ceph-mon[32853]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T01:47:43.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:43 smithi097 ceph-mon[32853]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:47:43.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:43 smithi173 ceph-mon[32553]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T01:47:43.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:43 smithi173 ceph-mon[32553]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:47:43.611 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:43 smithi001 ceph-mon[27822]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T01:47:43.611 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:43 smithi001 ceph-mon[27822]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:47:44.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:44 smithi001 ceph-mon[27822]: pgmap v331: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 3 op/s 2024-04-26T01:47:44.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:44 smithi097 ceph-mon[32853]: pgmap v331: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 3 op/s 2024-04-26T01:47:44.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:44 smithi173 ceph-mon[32553]: pgmap v331: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 3 op/s 2024-04-26T01:47:45.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.783+0000 7f94ace00700 1 -- 172.21.15.1:0/3594354368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94a8109c80 msgr2=0x7f94a810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:45.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.783+0000 7f94ace00700 1 --2- 172.21.15.1:0/3594354368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94a8109c80 0x7f94a810dcd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f94900099c0 tx=0x7f9490009cd0 comp rx=0 tx=0).stop 2024-04-26T01:47:45.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.783+0000 7f94ace00700 1 -- 172.21.15.1:0/3594354368 shutdown_connections 2024-04-26T01:47:45.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.783+0000 7f94ace00700 1 --2- 172.21.15.1:0/3594354368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94a810e3b0 0x7f94a8115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:45.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.783+0000 7f94ace00700 1 --2- 172.21.15.1:0/3594354368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94a81092d0 0x7f94a81096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:45.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.783+0000 7f94ace00700 1 --2- 172.21.15.1:0/3594354368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94a8109c80 0x7f94a810dcd0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:45.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.783+0000 7f94ace00700 1 -- 172.21.15.1:0/3594354368 >> 172.21.15.1:0/3594354368 conn(0x7f94a8076210 msgr2=0x7f94a8107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:45.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.783+0000 7f94ace00700 1 -- 172.21.15.1:0/3594354368 shutdown_connections 2024-04-26T01:47:45.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.783+0000 7f94ace00700 1 -- 172.21.15.1:0/3594354368 wait complete. 2024-04-26T01:47:45.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.784+0000 7f94ace00700 1 Processor -- start 2024-04-26T01:47:45.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.785+0000 7f94ace00700 1 -- start start 2024-04-26T01:47:45.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.785+0000 7f94ace00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94a81092d0 0x7f94a819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:45.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.785+0000 7f94ace00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94a8109c80 0x7f94a819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:45.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.785+0000 7f94ace00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94a810e3b0 0x7f94a81a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:45.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.785+0000 7f94ace00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8119290 con 0x7f94a810e3b0 2024-04-26T01:47:45.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.785+0000 7f94ace00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8119110 con 0x7f94a8109c80 2024-04-26T01:47:45.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.785+0000 7f94ace00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8119410 con 0x7f94a81092d0 2024-04-26T01:47:45.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.785+0000 7f94a5e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94a81092d0 0x7f94a819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:45.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.785+0000 7f94a6800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94a810e3b0 0x7f94a81a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:45.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.785+0000 7f94a5e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94a81092d0 0x7f94a819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:46210/0 (socket says 172.21.15.1:46210) 2024-04-26T01:47:45.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.785+0000 7f94a5e00700 1 -- 172.21.15.1:0/1130685803 learned_addr learned my addr 172.21.15.1:0/1130685803 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:47:45.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.785+0000 7f949fe00700 1 --2- 172.21.15.1:0/1130685803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94a8109c80 0x7f94a819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:45.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.786+0000 7f949fe00700 1 -- 172.21.15.1:0/1130685803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94a81092d0 msgr2=0x7f94a819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:45.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.786+0000 7f949fe00700 1 --2- 172.21.15.1:0/1130685803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94a81092d0 0x7f94a819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:45.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.786+0000 7f949fe00700 1 -- 172.21.15.1:0/1130685803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94a810e3b0 msgr2=0x7f94a81a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:45.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.786+0000 7f949fe00700 1 --2- 172.21.15.1:0/1130685803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94a810e3b0 0x7f94a81a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:45.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.786+0000 7f949fe00700 1 -- 172.21.15.1:0/1130685803 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a81a4730 con 0x7f94a8109c80 2024-04-26T01:47:45.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.786+0000 7f94a6800700 1 --2- 172.21.15.1:0/1130685803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94a810e3b0 0x7f94a81a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:47:45.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.786+0000 7f94a5e00700 1 --2- 172.21.15.1:0/1130685803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94a81092d0 0x7f94a819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:47:45.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.786+0000 7f949fe00700 1 --2- 172.21.15.1:0/1130685803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94a8109c80 0x7f94a819fc60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f94900098e0 tx=0x7f9490004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:45.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.787+0000 7f949ea00700 1 -- 172.21.15.1:0/1130685803 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9490026070 con 0x7f94a8109c80 2024-04-26T01:47:45.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.787+0000 7f94ace00700 1 -- 172.21.15.1:0/1130685803 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a81a49c0 con 0x7f94a8109c80 2024-04-26T01:47:45.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.787+0000 7f949ea00700 1 -- 172.21.15.1:0/1130685803 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9490003c70 con 0x7f94a8109c80 2024-04-26T01:47:45.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.787+0000 7f949ea00700 1 -- 172.21.15.1:0/1130685803 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949000f9c0 con 0x7f94a8109c80 2024-04-26T01:47:45.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.787+0000 7f94ace00700 1 -- 172.21.15.1:0/1130685803 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a81ac2a0 con 0x7f94a8109c80 2024-04-26T01:47:45.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.788+0000 7f94ace00700 1 -- 172.21.15.1:0/1130685803 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a8069730 con 0x7f94a8109c80 2024-04-26T01:47:45.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.790+0000 7f949ea00700 1 -- 172.21.15.1:0/1130685803 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f9490029070 con 0x7f94a8109c80 2024-04-26T01:47:45.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.792+0000 7f949ea00700 1 --2- 172.21.15.1:0/1130685803 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f948c0a3c40 0x7f948c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:45.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.792+0000 7f949ea00700 1 -- 172.21.15.1:0/1130685803 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f94900d0ba0 con 0x7f94a8109c80 2024-04-26T01:47:45.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.792+0000 7f94a5e00700 1 --2- 172.21.15.1:0/1130685803 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f948c0a3c40 0x7f948c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:45.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.792+0000 7f94a5e00700 1 --2- 172.21.15.1:0/1130685803 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f948c0a3c40 0x7f948c0a6100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9498011550 tx=0x7f9498009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:45.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:45.794+0000 7f949ea00700 1 -- 172.21.15.1:0/1130685803 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f94900d0e20 con 0x7f94a8109c80 2024-04-26T01:47:46.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:46.076+0000 7f94ace00700 1 -- 172.21.15.1:0/1130685803 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f94a81a0a80 con 0x7f94a8109c80 2024-04-26T01:47:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:46.194+0000 7f949ea00700 1 -- 172.21.15.1:0/1130685803 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f9490013070 con 0x7f94a8109c80 2024-04-26T01:47:46.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:46.199+0000 7f94ace00700 1 -- 172.21.15.1:0/1130685803 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f948c0a3c40 msgr2=0x7f948c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:46.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:46.199+0000 7f94ace00700 1 --2- 172.21.15.1:0/1130685803 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f948c0a3c40 0x7f948c0a6100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9498011550 tx=0x7f9498009260 comp rx=0 tx=0).stop 2024-04-26T01:47:46.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:46.199+0000 7f94ace00700 1 -- 172.21.15.1:0/1130685803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94a8109c80 msgr2=0x7f94a819fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:46.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:46.199+0000 7f94ace00700 1 --2- 172.21.15.1:0/1130685803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94a8109c80 0x7f94a819fc60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f94900098e0 tx=0x7f9490004270 comp rx=0 tx=0).stop 2024-04-26T01:47:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:46.199+0000 7f94ace00700 1 -- 172.21.15.1:0/1130685803 shutdown_connections 2024-04-26T01:47:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:46.199+0000 7f94ace00700 1 --2- 172.21.15.1:0/1130685803 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f948c0a3c40 0x7f948c0a6100 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:46.199+0000 7f94ace00700 1 --2- 172.21.15.1:0/1130685803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94a810e3b0 0x7f94a81a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:46.199+0000 7f94ace00700 1 --2- 172.21.15.1:0/1130685803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94a81092d0 0x7f94a819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:46.199+0000 7f94ace00700 1 --2- 172.21.15.1:0/1130685803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94a8109c80 0x7f94a819fc60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:46.199+0000 7f94ace00700 1 -- 172.21.15.1:0/1130685803 >> 172.21.15.1:0/1130685803 conn(0x7f94a8076210 msgr2=0x7f94a8076830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:46.200+0000 7f94ace00700 1 -- 172.21.15.1:0/1130685803 shutdown_connections 2024-04-26T01:47:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:46.200+0000 7f94ace00700 1 -- 172.21.15.1:0/1130685803 wait complete. 2024-04-26T01:47:46.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:46 smithi001 ceph-mon[27822]: pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-04-26T01:47:46.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:46 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1130685803' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T01:47:46.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:46 smithi001 ceph-mon[27822]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T01:47:46.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:46 smithi097 ceph-mon[32853]: pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-04-26T01:47:46.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:46 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1130685803' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T01:47:46.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:46 smithi097 ceph-mon[32853]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T01:47:46.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:46 smithi173 ceph-mon[32553]: pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-04-26T01:47:46.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:46 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1130685803' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T01:47:46.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:46 smithi173 ceph-mon[32553]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T01:47:46.989 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-04-26T01:47:47.460 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:47:47.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:47 smithi097 ceph-mon[32853]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T01:47:47.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:47 smithi097 ceph-mon[32853]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T01:47:47.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:47 smithi097 ceph-mon[32853]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T01:47:47.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:47 smithi097 ceph-mon[32853]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:47:47.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:47 smithi097 ceph-mon[32853]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-26T01:47:47.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:47 smithi097 ceph-mon[32853]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T01:47:47.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:47 smithi173 ceph-mon[32553]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T01:47:47.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:47 smithi173 ceph-mon[32553]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T01:47:47.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:47 smithi173 ceph-mon[32553]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T01:47:47.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:47 smithi173 ceph-mon[32553]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:47:47.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:47 smithi173 ceph-mon[32553]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-26T01:47:47.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:47 smithi173 ceph-mon[32553]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T01:47:47.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:47 smithi001 ceph-mon[27822]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T01:47:47.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:47 smithi001 ceph-mon[27822]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T01:47:47.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:47 smithi001 ceph-mon[27822]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T01:47:47.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:47 smithi001 ceph-mon[27822]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T01:47:47.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:47 smithi001 ceph-mon[27822]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-26T01:47:47.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:47 smithi001 ceph-mon[27822]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T01:47:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:48 smithi097 ceph-mon[32853]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-26T01:47:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:48 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] up:active 2024-04-26T01:47:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:48 smithi097 ceph-mon[32853]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T01:47:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:48 smithi097 ceph-mon[32853]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T01:47:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:48 smithi097 ceph-mon[32853]: Cluster is now healthy 2024-04-26T01:47:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:48 smithi097 ceph-mon[32853]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-26T01:47:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:48 smithi097 ceph-mon[32853]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-26T01:47:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:48 smithi097 ceph-mon[32853]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-26T01:47:48.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:48 smithi173 ceph-mon[32553]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-26T01:47:48.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:48 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] up:active 2024-04-26T01:47:48.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:48 smithi173 ceph-mon[32553]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T01:47:48.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:48 smithi173 ceph-mon[32553]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T01:47:48.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:48 smithi173 ceph-mon[32553]: Cluster is now healthy 2024-04-26T01:47:48.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:48 smithi173 ceph-mon[32553]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-26T01:47:48.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:48 smithi173 ceph-mon[32553]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-26T01:47:48.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:48 smithi173 ceph-mon[32553]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-26T01:47:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:48 smithi001 ceph-mon[27822]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-26T01:47:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:48 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] up:active 2024-04-26T01:47:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:48 smithi001 ceph-mon[27822]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T01:47:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:48 smithi001 ceph-mon[27822]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T01:47:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:48 smithi001 ceph-mon[27822]: Cluster is now healthy 2024-04-26T01:47:48.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:48 smithi001 ceph-mon[27822]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-26T01:47:48.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:48 smithi001 ceph-mon[27822]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-26T01:47:48.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:48 smithi001 ceph-mon[27822]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-26T01:47:49.364 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:49 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] up:active 2024-04-26T01:47:49.364 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:49 smithi001 ceph-mon[27822]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-26T01:47:49.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.539+0000 7f8584a00700 1 -- 172.21.15.1:0/699257836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8580109c80 msgr2=0x7f858010dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:49.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.539+0000 7f8584a00700 1 --2- 172.21.15.1:0/699257836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8580109c80 0x7f858010dcd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8568009990 tx=0x7f8568009ca0 comp rx=0 tx=0).stop 2024-04-26T01:47:49.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.539+0000 7f8584a00700 1 -- 172.21.15.1:0/699257836 shutdown_connections 2024-04-26T01:47:49.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.539+0000 7f8584a00700 1 --2- 172.21.15.1:0/699257836 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f858010e3b0 0x7f8580115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:49.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.539+0000 7f8584a00700 1 --2- 172.21.15.1:0/699257836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85801092d0 0x7f85801096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:49.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.539+0000 7f8584a00700 1 --2- 172.21.15.1:0/699257836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8580109c80 0x7f858010dcd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:49.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.539+0000 7f8584a00700 1 -- 172.21.15.1:0/699257836 >> 172.21.15.1:0/699257836 conn(0x7f8580076210 msgr2=0x7f8580107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:49.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.539+0000 7f8584a00700 1 -- 172.21.15.1:0/699257836 shutdown_connections 2024-04-26T01:47:49.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.539+0000 7f8584a00700 1 -- 172.21.15.1:0/699257836 wait complete. 2024-04-26T01:47:49.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.540+0000 7f8584a00700 1 Processor -- start 2024-04-26T01:47:49.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.540+0000 7f8584a00700 1 -- start start 2024-04-26T01:47:49.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.541+0000 7f8584a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85801092d0 0x7f85801a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:49.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.541+0000 7f8584a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8580109c80 0x7f85801a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:49.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.541+0000 7f8584a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f858010e3b0 0x7f858019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:49.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.541+0000 7f8584a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8580119290 con 0x7f85801092d0 2024-04-26T01:47:49.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.541+0000 7f8584a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8580119110 con 0x7f8580109c80 2024-04-26T01:47:49.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.541+0000 7f8584a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8580119410 con 0x7f858010e3b0 2024-04-26T01:47:49.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.541+0000 7f857de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85801092d0 0x7f85801a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:49.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.541+0000 7f857de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85801092d0 0x7f85801a53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50870/0 (socket says 172.21.15.1:50870) 2024-04-26T01:47:49.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.541+0000 7f857de00700 1 -- 172.21.15.1:0/2531763928 learned_addr learned my addr 172.21.15.1:0/2531763928 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:47:49.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.541+0000 7f8575400700 1 --2- 172.21.15.1:0/2531763928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8580109c80 0x7f85801a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:49.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.541+0000 7f857e800700 1 --2- 172.21.15.1:0/2531763928 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f858010e3b0 0x7f858019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:49.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.541+0000 7f857de00700 1 -- 172.21.15.1:0/2531763928 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f858010e3b0 msgr2=0x7f858019f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:49.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.541+0000 7f857de00700 1 --2- 172.21.15.1:0/2531763928 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f858010e3b0 0x7f858019f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:49.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.542+0000 7f857de00700 1 -- 172.21.15.1:0/2531763928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8580109c80 msgr2=0x7f85801a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:49.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.542+0000 7f857de00700 1 --2- 172.21.15.1:0/2531763928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8580109c80 0x7f85801a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:49.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.542+0000 7f857de00700 1 -- 172.21.15.1:0/2531763928 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858019fd50 con 0x7f85801092d0 2024-04-26T01:47:49.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.542+0000 7f857de00700 1 --2- 172.21.15.1:0/2531763928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85801092d0 0x7f85801a53e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f857000eb30 tx=0x7f857000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:49.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.542+0000 7f8577400700 1 -- 172.21.15.1:0/2531763928 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857000cc90 con 0x7f85801092d0 2024-04-26T01:47:49.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.542+0000 7f8577400700 1 -- 172.21.15.1:0/2531763928 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85700044d0 con 0x7f85801092d0 2024-04-26T01:47:49.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.542+0000 7f8577400700 1 -- 172.21.15.1:0/2531763928 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8570010430 con 0x7f85801092d0 2024-04-26T01:47:49.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.542+0000 7f8584a00700 1 -- 172.21.15.1:0/2531763928 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85801a0040 con 0x7f85801092d0 2024-04-26T01:47:49.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.542+0000 7f8584a00700 1 -- 172.21.15.1:0/2531763928 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85801ac2b0 con 0x7f85801092d0 2024-04-26T01:47:49.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.543+0000 7f8584a00700 1 -- 172.21.15.1:0/2531763928 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8580069730 con 0x7f85801092d0 2024-04-26T01:47:49.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.547+0000 7f8577400700 1 -- 172.21.15.1:0/2531763928 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f8570003ea0 con 0x7f85801092d0 2024-04-26T01:47:49.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.549+0000 7f8577400700 1 --2- 172.21.15.1:0/2531763928 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f85640a3cc0 0x7f85640a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:49.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.549+0000 7f8577400700 1 -- 172.21.15.1:0/2531763928 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f85700d05b0 con 0x7f85801092d0 2024-04-26T01:47:49.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.549+0000 7f8575400700 1 --2- 172.21.15.1:0/2531763928 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f85640a3cc0 0x7f85640a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:49.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.550+0000 7f8575400700 1 --2- 172.21.15.1:0/2531763928 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f85640a3cc0 0x7f85640a6180 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f858010f780 tx=0x7f856800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:49.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.552+0000 7f8577400700 1 -- 172.21.15.1:0/2531763928 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f857009d0e0 con 0x7f85801092d0 2024-04-26T01:47:49.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:49 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] up:active 2024-04-26T01:47:49.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:49 smithi173 ceph-mon[32553]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-26T01:47:49.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:49 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] up:active 2024-04-26T01:47:49.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:49 smithi097 ceph-mon[32853]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-26T01:47:49.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:49.840+0000 7f8584a00700 1 -- 172.21.15.1:0/2531763928 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f858002df80 con 0x7f85801092d0 2024-04-26T01:47:50.232 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 01:47:49 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-b[62484]: 2024-04-26T01:47:49.929+0000 7f2b66035700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T01:47:50.233 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 01:47:49 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-b[62484]: 2024-04-26T01:47:49.929+0000 7f2b66035700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-26T01:47:50.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:50.240+0000 7f8577400700 1 -- 172.21.15.1:0/2531763928 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f857009c9e0 con 0x7f85801092d0 2024-04-26T01:47:50.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:50.244+0000 7f8584a00700 1 -- 172.21.15.1:0/2531763928 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f85640a3cc0 msgr2=0x7f85640a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:50.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:50.244+0000 7f8584a00700 1 --2- 172.21.15.1:0/2531763928 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f85640a3cc0 0x7f85640a6180 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f858010f780 tx=0x7f856800b260 comp rx=0 tx=0).stop 2024-04-26T01:47:50.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:50.244+0000 7f8584a00700 1 -- 172.21.15.1:0/2531763928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85801092d0 msgr2=0x7f85801a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:50.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:50.244+0000 7f8584a00700 1 --2- 172.21.15.1:0/2531763928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85801092d0 0x7f85801a53e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f857000eb30 tx=0x7f857000ee40 comp rx=0 tx=0).stop 2024-04-26T01:47:50.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:50.244+0000 7f8584a00700 1 -- 172.21.15.1:0/2531763928 shutdown_connections 2024-04-26T01:47:50.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:50.244+0000 7f8584a00700 1 --2- 172.21.15.1:0/2531763928 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f85640a3cc0 0x7f85640a6180 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:50.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:50.244+0000 7f8584a00700 1 --2- 172.21.15.1:0/2531763928 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f858010e3b0 0x7f858019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:50.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:50.244+0000 7f8584a00700 1 --2- 172.21.15.1:0/2531763928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85801092d0 0x7f85801a53e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:50.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:50.244+0000 7f8584a00700 1 --2- 172.21.15.1:0/2531763928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8580109c80 0x7f85801a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:50.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:50.244+0000 7f8584a00700 1 -- 172.21.15.1:0/2531763928 >> 172.21.15.1:0/2531763928 conn(0x7f8580076210 msgr2=0x7f85800767d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:50.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:50.245+0000 7f8584a00700 1 -- 172.21.15.1:0/2531763928 shutdown_connections 2024-04-26T01:47:50.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:50.245+0000 7f8584a00700 1 -- 172.21.15.1:0/2531763928 wait complete. 2024-04-26T01:47:50.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:50 smithi173 ceph-mon[32553]: pgmap v334: 97 pgs: 97 active+clean; 580 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 3 op/s 2024-04-26T01:47:50.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:50 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/2531763928' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T01:47:50.610 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:50 smithi001 ceph-mon[27822]: pgmap v334: 97 pgs: 97 active+clean; 580 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 3 op/s 2024-04-26T01:47:50.611 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:50 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2531763928' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T01:47:50.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:50 smithi097 ceph-mon[32853]: pgmap v334: 97 pgs: 97 active+clean; 580 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 3 op/s 2024-04-26T01:47:50.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:50 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/2531763928' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T01:47:51.157 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-04-26T01:47:51.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:51 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/2531763928' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T01:47:51.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:51 smithi173 ceph-mon[32553]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-26T01:47:51.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:51 smithi173 ceph-mon[32553]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:47:51.583 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:47:51.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:51 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2531763928' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T01:47:51.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:51 smithi001 ceph-mon[27822]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-26T01:47:51.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:51 smithi001 ceph-mon[27822]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:47:51.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:51 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/2531763928' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T01:47:51.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:51 smithi097 ceph-mon[32853]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-26T01:47:51.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:51 smithi097 ceph-mon[32853]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:47:52.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:52 smithi001 ceph-mon[27822]: pgmap v335: 97 pgs: 97 active+clean; 582 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-04-26T01:47:52.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:52 smithi173 ceph-mon[32553]: pgmap v335: 97 pgs: 97 active+clean; 582 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-04-26T01:47:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:52 smithi097 ceph-mon[32853]: pgmap v335: 97 pgs: 97 active+clean; 582 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-04-26T01:47:53.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.779+0000 7ff815600700 1 -- 172.21.15.1:0/2415302091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff810113c10 msgr2=0x7ff810116000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:53.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.779+0000 7ff815600700 1 --2- 172.21.15.1:0/2415302091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff810113c10 0x7ff810116000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff808009990 tx=0x7ff808009ca0 comp rx=0 tx=0).stop 2024-04-26T01:47:53.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.779+0000 7ff815600700 1 -- 172.21.15.1:0/2415302091 shutdown_connections 2024-04-26T01:47:53.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.779+0000 7ff815600700 1 --2- 172.21.15.1:0/2415302091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff810113c10 0x7ff810116000 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:53.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.779+0000 7ff815600700 1 --2- 172.21.15.1:0/2415302091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff810074c70 0x7ff810075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:53.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.779+0000 7ff815600700 1 --2- 172.21.15.1:0/2415302091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff810075590 0x7ff8101136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:53.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.779+0000 7ff815600700 1 -- 172.21.15.1:0/2415302091 >> 172.21.15.1:0/2415302091 conn(0x7ff810079ba0 msgr2=0x7ff810079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:53.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.779+0000 7ff815600700 1 -- 172.21.15.1:0/2415302091 shutdown_connections 2024-04-26T01:47:53.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.779+0000 7ff815600700 1 -- 172.21.15.1:0/2415302091 wait complete. 2024-04-26T01:47:53.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.780+0000 7ff815600700 1 Processor -- start 2024-04-26T01:47:53.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.781+0000 7ff815600700 1 -- start start 2024-04-26T01:47:53.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.781+0000 7ff815600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff810074c70 0x7ff81019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:53.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.781+0000 7ff815600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff810075590 0x7ff81019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:53.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.781+0000 7ff815600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff810113c10 0x7ff8101a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:53.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.781+0000 7ff815600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff810119220 con 0x7ff810113c10 2024-04-26T01:47:53.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.781+0000 7ff815600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff8101190a0 con 0x7ff810075590 2024-04-26T01:47:53.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.781+0000 7ff815600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff8101193a0 con 0x7ff810074c70 2024-04-26T01:47:53.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.781+0000 7ff80ea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff810074c70 0x7ff81019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:53.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.781+0000 7ff80f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff810113c10 0x7ff8101a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.781+0000 7ff80ea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff810074c70 0x7ff81019f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:43514/0 (socket says 172.21.15.1:43514) 2024-04-26T01:47:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.781+0000 7ff80ea00700 1 -- 172.21.15.1:0/2485029036 learned_addr learned my addr 172.21.15.1:0/2485029036 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:47:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.781+0000 7ff806000700 1 --2- 172.21.15.1:0/2485029036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff810075590 0x7ff81019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.782+0000 7ff80f400700 1 -- 172.21.15.1:0/2485029036 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff810074c70 msgr2=0x7ff81019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.782+0000 7ff80f400700 1 --2- 172.21.15.1:0/2485029036 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff810074c70 0x7ff81019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.782+0000 7ff80f400700 1 -- 172.21.15.1:0/2485029036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff810075590 msgr2=0x7ff81019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.782+0000 7ff80f400700 1 --2- 172.21.15.1:0/2485029036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff810075590 0x7ff81019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.782+0000 7ff80f400700 1 -- 172.21.15.1:0/2485029036 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8101a46e0 con 0x7ff810113c10 2024-04-26T01:47:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.782+0000 7ff806000700 1 --2- 172.21.15.1:0/2485029036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff810075590 0x7ff81019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:47:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.782+0000 7ff80f400700 1 --2- 172.21.15.1:0/2485029036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff810113c10 0x7ff8101a3fa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff8080098e0 tx=0x7ff8080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.783+0000 7ff807e00700 1 -- 172.21.15.1:0/2485029036 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff808027070 con 0x7ff810113c10 2024-04-26T01:47:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.783+0000 7ff807e00700 1 -- 172.21.15.1:0/2485029036 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff808022470 con 0x7ff810113c10 2024-04-26T01:47:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.783+0000 7ff815600700 1 -- 172.21.15.1:0/2485029036 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8101a4970 con 0x7ff810113c10 2024-04-26T01:47:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.783+0000 7ff807e00700 1 -- 172.21.15.1:0/2485029036 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff808017650 con 0x7ff810113c10 2024-04-26T01:47:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.783+0000 7ff815600700 1 -- 172.21.15.1:0/2485029036 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8101ac250 con 0x7ff810113c10 2024-04-26T01:47:53.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.785+0000 7ff815600700 1 -- 172.21.15.1:0/2485029036 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff810076360 con 0x7ff810113c10 2024-04-26T01:47:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.786+0000 7ff807e00700 1 -- 172.21.15.1:0/2485029036 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7ff80802a070 con 0x7ff810113c10 2024-04-26T01:47:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.788+0000 7ff807e00700 1 --2- 172.21.15.1:0/2485029036 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ff7f40a3c40 0x7ff7f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.788+0000 7ff807e00700 1 -- 172.21.15.1:0/2485029036 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7ff8080d0fe0 con 0x7ff810113c10 2024-04-26T01:47:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.788+0000 7ff80ea00700 1 --2- 172.21.15.1:0/2485029036 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ff7f40a3c40 0x7ff7f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.789+0000 7ff80ea00700 1 --2- 172.21.15.1:0/2485029036 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ff7f40a3c40 0x7ff7f40a6100 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff800009c10 tx=0x7ff80000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:53.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:53.792+0000 7ff807e00700 1 -- 172.21.15.1:0/2485029036 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff80805eff0 con 0x7ff810113c10 2024-04-26T01:47:54.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:54.076+0000 7ff815600700 1 -- 172.21.15.1:0/2485029036 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7ff8101a0a30 con 0x7ff810113c10 2024-04-26T01:47:54.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:54.269+0000 7ff807e00700 1 -- 172.21.15.1:0/2485029036 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7ff808013070 con 0x7ff810113c10 2024-04-26T01:47:54.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:54.273+0000 7ff815600700 1 -- 172.21.15.1:0/2485029036 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ff7f40a3c40 msgr2=0x7ff7f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:54.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:54.273+0000 7ff815600700 1 --2- 172.21.15.1:0/2485029036 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ff7f40a3c40 0x7ff7f40a6100 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff800009c10 tx=0x7ff80000b040 comp rx=0 tx=0).stop 2024-04-26T01:47:54.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:54.273+0000 7ff815600700 1 -- 172.21.15.1:0/2485029036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff810113c10 msgr2=0x7ff8101a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:54.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:54.273+0000 7ff815600700 1 --2- 172.21.15.1:0/2485029036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff810113c10 0x7ff8101a3fa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff8080098e0 tx=0x7ff8080047b0 comp rx=0 tx=0).stop 2024-04-26T01:47:54.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:54.274+0000 7ff815600700 1 -- 172.21.15.1:0/2485029036 shutdown_connections 2024-04-26T01:47:54.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:54.274+0000 7ff815600700 1 --2- 172.21.15.1:0/2485029036 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ff7f40a3c40 0x7ff7f40a6100 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:54.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:54.274+0000 7ff815600700 1 --2- 172.21.15.1:0/2485029036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff810113c10 0x7ff8101a3fa0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:54.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:54.274+0000 7ff815600700 1 --2- 172.21.15.1:0/2485029036 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff810074c70 0x7ff81019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:54.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:54.274+0000 7ff815600700 1 --2- 172.21.15.1:0/2485029036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff810075590 0x7ff81019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:54.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:54.274+0000 7ff815600700 1 -- 172.21.15.1:0/2485029036 >> 172.21.15.1:0/2485029036 conn(0x7ff810079ba0 msgr2=0x7ff81010f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:54.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:54.274+0000 7ff815600700 1 -- 172.21.15.1:0/2485029036 shutdown_connections 2024-04-26T01:47:54.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:54.274+0000 7ff815600700 1 -- 172.21.15.1:0/2485029036 wait complete. 2024-04-26T01:47:54.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:54 smithi001 ceph-mon[27822]: pgmap v336: 97 pgs: 97 active+clean; 582 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-26T01:47:54.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:54 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2485029036' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T01:47:54.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:54 smithi173 ceph-mon[32553]: pgmap v336: 97 pgs: 97 active+clean; 582 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-26T01:47:54.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:54 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/2485029036' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T01:47:54.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:54 smithi097 ceph-mon[32853]: pgmap v336: 97 pgs: 97 active+clean; 582 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-26T01:47:54.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:54 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/2485029036' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T01:47:55.069 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-26T01:47:55.070 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-04-26T01:47:55.529 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:47:55.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:55 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/2485029036' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T01:47:55.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:55 smithi173 ceph-mon[32553]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:47:55.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:55 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2485029036' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T01:47:55.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:55 smithi001 ceph-mon[27822]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:47:55.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:55 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/2485029036' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T01:47:55.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:55 smithi097 ceph-mon[32853]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:47:56.552 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:56 smithi001 ceph-mon[27822]: pgmap v337: 97 pgs: 97 active+clean; 582 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T01:47:56.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:56 smithi173 ceph-mon[32553]: pgmap v337: 97 pgs: 97 active+clean; 582 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T01:47:56.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:56 smithi097 ceph-mon[32853]: pgmap v337: 97 pgs: 97 active+clean; 582 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T01:47:57.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.840+0000 7fda7d200700 1 -- 172.21.15.1:0/3788577253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda78076f70 msgr2=0x7fda78077350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:57.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.840+0000 7fda7d200700 1 --2- 172.21.15.1:0/3788577253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda78076f70 0x7fda78077350 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fda6800b0a0 tx=0x7fda6800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:47:57.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.840+0000 7fda7d200700 1 -- 172.21.15.1:0/3788577253 shutdown_connections 2024-04-26T01:47:57.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.840+0000 7fda7d200700 1 --2- 172.21.15.1:0/3788577253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda78113920 0x7fda78115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:57.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.840+0000 7fda7d200700 1 --2- 172.21.15.1:0/3788577253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda78076f70 0x7fda78077350 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:57.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.840+0000 7fda7d200700 1 --2- 172.21.15.1:0/3788577253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda78077890 0x7fda781133e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:57.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.840+0000 7fda7d200700 1 -- 172.21.15.1:0/3788577253 >> 172.21.15.1:0/3788577253 conn(0x7fda78100770 msgr2=0x7fda78102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:57.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.841+0000 7fda7d200700 1 -- 172.21.15.1:0/3788577253 shutdown_connections 2024-04-26T01:47:57.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.841+0000 7fda7d200700 1 -- 172.21.15.1:0/3788577253 wait complete. 2024-04-26T01:47:57.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.842+0000 7fda7d200700 1 Processor -- start 2024-04-26T01:47:57.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.842+0000 7fda7d200700 1 -- start start 2024-04-26T01:47:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.842+0000 7fda7d200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda78076f70 0x7fda781a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.842+0000 7fda7d200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda78077890 0x7fda781a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.842+0000 7fda7d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda78113920 0x7fda7819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.842+0000 7fda7d200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fda78119310 con 0x7fda78113920 2024-04-26T01:47:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.842+0000 7fda7d200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fda78119190 con 0x7fda78077890 2024-04-26T01:47:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.842+0000 7fda7d200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fda78119490 con 0x7fda78076f70 2024-04-26T01:47:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.843+0000 7fda76800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda78113920 0x7fda7819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.843+0000 7fda76800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda78113920 0x7fda7819f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49760/0 (socket says 172.21.15.1:49760) 2024-04-26T01:47:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.843+0000 7fda76800700 1 -- 172.21.15.1:0/3028355783 learned_addr learned my addr 172.21.15.1:0/3028355783 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:47:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.843+0000 7fda75e00700 1 --2- 172.21.15.1:0/3028355783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda78076f70 0x7fda781a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.843+0000 7fda6fe00700 1 --2- 172.21.15.1:0/3028355783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda78077890 0x7fda781a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:57.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.843+0000 7fda76800700 1 -- 172.21.15.1:0/3028355783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda78076f70 msgr2=0x7fda781a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:57.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.843+0000 7fda76800700 1 --2- 172.21.15.1:0/3028355783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda78076f70 0x7fda781a53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:57.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.843+0000 7fda76800700 1 -- 172.21.15.1:0/3028355783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda78077890 msgr2=0x7fda781a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:57.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.843+0000 7fda76800700 1 --2- 172.21.15.1:0/3028355783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda78077890 0x7fda781a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:57.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.843+0000 7fda76800700 1 -- 172.21.15.1:0/3028355783 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7819fd10 con 0x7fda78113920 2024-04-26T01:47:57.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.844+0000 7fda76800700 1 --2- 172.21.15.1:0/3028355783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda78113920 0x7fda7819f5b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fda7000e940 tx=0x7fda7000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:57.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.844+0000 7fda6ea00700 1 -- 172.21.15.1:0/3028355783 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70004a30 con 0x7fda78113920 2024-04-26T01:47:57.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.844+0000 7fda6ea00700 1 -- 172.21.15.1:0/3028355783 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda700054d0 con 0x7fda78113920 2024-04-26T01:47:57.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.844+0000 7fda6ea00700 1 -- 172.21.15.1:0/3028355783 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7000cc70 con 0x7fda78113920 2024-04-26T01:47:57.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.844+0000 7fda7d200700 1 -- 172.21.15.1:0/3028355783 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda781a0000 con 0x7fda78113920 2024-04-26T01:47:57.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.844+0000 7fda7d200700 1 -- 172.21.15.1:0/3028355783 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda781ac2c0 con 0x7fda78113920 2024-04-26T01:47:57.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.845+0000 7fda7d200700 1 -- 172.21.15.1:0/3028355783 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda78069730 con 0x7fda78113920 2024-04-26T01:47:57.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.848+0000 7fda6ea00700 1 -- 172.21.15.1:0/3028355783 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fda70005000 con 0x7fda78113920 2024-04-26T01:47:57.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.849+0000 7fda6ea00700 1 --2- 172.21.15.1:0/3028355783 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fda5c0a3cc0 0x7fda5c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:57.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.849+0000 7fda6ea00700 1 -- 172.21.15.1:0/3028355783 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7fda700d0540 con 0x7fda78113920 2024-04-26T01:47:57.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.849+0000 7fda75e00700 1 --2- 172.21.15.1:0/3028355783 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fda5c0a3cc0 0x7fda5c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:57.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.850+0000 7fda75e00700 1 --2- 172.21.15.1:0/3028355783 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fda5c0a3cc0 0x7fda5c0a6180 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fda680062a0 tx=0x7fda68006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:57.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:57.852+0000 7fda6ea00700 1 -- 172.21.15.1:0/3028355783 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda700052f0 con 0x7fda78113920 2024-04-26T01:47:58.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:58.115+0000 7fda7d200700 1 -- 172.21.15.1:0/3028355783 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fda78062e80 con 0x7fda5c0a3cc0 2024-04-26T01:47:58.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:58.155+0000 7fda6ea00700 1 -- 172.21.15.1:0/3028355783 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fda78062e80 con 0x7fda5c0a3cc0 2024-04-26T01:47:58.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:58.158+0000 7fda7d200700 1 -- 172.21.15.1:0/3028355783 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fda5c0a3cc0 msgr2=0x7fda5c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:58.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:58.158+0000 7fda7d200700 1 --2- 172.21.15.1:0/3028355783 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fda5c0a3cc0 0x7fda5c0a6180 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fda680062a0 tx=0x7fda68006210 comp rx=0 tx=0).stop 2024-04-26T01:47:58.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:58.158+0000 7fda7d200700 1 -- 172.21.15.1:0/3028355783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda78113920 msgr2=0x7fda7819f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:58.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:58.158+0000 7fda7d200700 1 --2- 172.21.15.1:0/3028355783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda78113920 0x7fda7819f5b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fda7000e940 tx=0x7fda7000ec50 comp rx=0 tx=0).stop 2024-04-26T01:47:58.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:58.158+0000 7fda7d200700 1 -- 172.21.15.1:0/3028355783 shutdown_connections 2024-04-26T01:47:58.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:58.158+0000 7fda7d200700 1 --2- 172.21.15.1:0/3028355783 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fda5c0a3cc0 0x7fda5c0a6180 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:58.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:58.158+0000 7fda7d200700 1 --2- 172.21.15.1:0/3028355783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda78113920 0x7fda7819f5b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:58.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:58.158+0000 7fda7d200700 1 --2- 172.21.15.1:0/3028355783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda78076f70 0x7fda781a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:58.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:58.158+0000 7fda7d200700 1 --2- 172.21.15.1:0/3028355783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda78077890 0x7fda781a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:58.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:58.158+0000 7fda7d200700 1 -- 172.21.15.1:0/3028355783 >> 172.21.15.1:0/3028355783 conn(0x7fda78100770 msgr2=0x7fda78102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:58.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:58.158+0000 7fda7d200700 1 -- 172.21.15.1:0/3028355783 shutdown_connections 2024-04-26T01:47:58.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:47:58.158+0000 7fda7d200700 1 -- 172.21.15.1:0/3028355783 wait complete. 2024-04-26T01:47:58.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:58 smithi001 ceph-mon[27822]: pgmap v338: 97 pgs: 97 active+clean; 582 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-26T01:47:58.379 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:47:58 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:47:58.129+0000 7f80da000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:47:58.380 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:47:58 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:47:58.129+0000 7f80da000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:47:58.380 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:47:58 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:47:58.129+0000 7f80da000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:47:58.380 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:47:58 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:47:58.129+0000 7f80da000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:47:58.380 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:47:58 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:47:58.129+0000 7f80da000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:47:58.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:58 smithi173 ceph-mon[32553]: pgmap v338: 97 pgs: 97 active+clean; 582 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-26T01:47:58.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:58 smithi097 ceph-mon[32853]: pgmap v338: 97 pgs: 97 active+clean; 582 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-26T01:47:58.920 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-26T01:47:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:47:59.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:47:59 smithi173 ceph-mon[32553]: from='client.14484 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:47:59.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:47:59 smithi001 ceph-mon[27822]: from='client.14484 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:47:59.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:47:59 smithi097 ceph-mon[32853]: from='client.14484 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:00.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:00 smithi173 ceph-mon[32553]: pgmap v339: 97 pgs: 97 active+clean; 582 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-26T01:48:00.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:00 smithi173 ceph-mon[32553]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-26T01:48:00.571 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:00 smithi001 ceph-mon[27822]: pgmap v339: 97 pgs: 97 active+clean; 582 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-26T01:48:00.572 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:00 smithi001 ceph-mon[27822]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-26T01:48:00.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:00 smithi097 ceph-mon[32853]: pgmap v339: 97 pgs: 97 active+clean; 582 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-26T01:48:00.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:00 smithi097 ceph-mon[32853]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-26T01:48:01.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.574+0000 7fda55000700 1 -- 172.21.15.1:0/2179516123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda5010f220 msgr2=0x7fda50111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:01.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.574+0000 7fda55000700 1 --2- 172.21.15.1:0/2179516123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda5010f220 0x7fda50111610 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fda4800b0a0 tx=0x7fda4800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:48:01.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.574+0000 7fda55000700 1 -- 172.21.15.1:0/2179516123 shutdown_connections 2024-04-26T01:48:01.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.574+0000 7fda55000700 1 --2- 172.21.15.1:0/2179516123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda5010f220 0x7fda50111610 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:01.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.574+0000 7fda55000700 1 --2- 172.21.15.1:0/2179516123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda50074d60 0x7fda50075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:01.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.574+0000 7fda55000700 1 --2- 172.21.15.1:0/2179516123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda50075680 0x7fda5010ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:01.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.574+0000 7fda55000700 1 -- 172.21.15.1:0/2179516123 >> 172.21.15.1:0/2179516123 conn(0x7fda500fe490 msgr2=0x7fda501008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:01.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.574+0000 7fda55000700 1 -- 172.21.15.1:0/2179516123 shutdown_connections 2024-04-26T01:48:01.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.574+0000 7fda55000700 1 -- 172.21.15.1:0/2179516123 wait complete. 2024-04-26T01:48:01.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.575+0000 7fda55000700 1 Processor -- start 2024-04-26T01:48:01.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.576+0000 7fda55000700 1 -- start start 2024-04-26T01:48:01.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.576+0000 7fda55000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda50074d60 0x7fda5019b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:01.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.576+0000 7fda55000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda50075680 0x7fda5019b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:01.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.576+0000 7fda55000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda5010f220 0x7fda5019fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:01.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.576+0000 7fda55000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fda50114d10 con 0x7fda50075680 2024-04-26T01:48:01.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.576+0000 7fda55000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fda50114b90 con 0x7fda5010f220 2024-04-26T01:48:01.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.576+0000 7fda55000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fda50114e90 con 0x7fda50074d60 2024-04-26T01:48:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.576+0000 7fda4f400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda50074d60 0x7fda5019b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.577+0000 7fda4ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda50075680 0x7fda5019b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.577+0000 7fda4f400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda50074d60 0x7fda5019b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:43580/0 (socket says 172.21.15.1:43580) 2024-04-26T01:48:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.577+0000 7fda4f400700 1 -- 172.21.15.1:0/2275146003 learned_addr learned my addr 172.21.15.1:0/2275146003 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:48:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.577+0000 7fda4fe00700 1 --2- 172.21.15.1:0/2275146003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda5010f220 0x7fda5019fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.577+0000 7fda4f400700 1 -- 172.21.15.1:0/2275146003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda5010f220 msgr2=0x7fda5019fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:01.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.577+0000 7fda4f400700 1 --2- 172.21.15.1:0/2275146003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda5010f220 0x7fda5019fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:01.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.577+0000 7fda4f400700 1 -- 172.21.15.1:0/2275146003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda50075680 msgr2=0x7fda5019b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:01.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.577+0000 7fda4f400700 1 --2- 172.21.15.1:0/2275146003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda50075680 0x7fda5019b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:01.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.577+0000 7fda4f400700 1 -- 172.21.15.1:0/2275146003 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda501a01c0 con 0x7fda50074d60 2024-04-26T01:48:01.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.577+0000 7fda4ea00700 1 --2- 172.21.15.1:0/2275146003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda50075680 0x7fda5019b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:48:01.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.577+0000 7fda4f400700 1 --2- 172.21.15.1:0/2275146003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda50074d60 0x7fda5019b200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fda4000d950 tx=0x7fda4000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:01.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.578+0000 7fda47e00700 1 -- 172.21.15.1:0/2275146003 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40016070 con 0x7fda50074d60 2024-04-26T01:48:01.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.578+0000 7fda47e00700 1 -- 172.21.15.1:0/2275146003 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda4000fd40 con 0x7fda50074d60 2024-04-26T01:48:01.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.578+0000 7fda55000700 1 -- 172.21.15.1:0/2275146003 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda501a04b0 con 0x7fda50074d60 2024-04-26T01:48:01.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.578+0000 7fda47e00700 1 -- 172.21.15.1:0/2275146003 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4000e3b0 con 0x7fda50074d60 2024-04-26T01:48:01.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.578+0000 7fda55000700 1 -- 172.21.15.1:0/2275146003 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda501a7d30 con 0x7fda50074d60 2024-04-26T01:48:01.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.579+0000 7fda55000700 1 -- 172.21.15.1:0/2275146003 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda50069730 con 0x7fda50074d60 2024-04-26T01:48:01.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.581+0000 7fda47e00700 1 -- 172.21.15.1:0/2275146003 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fda40019070 con 0x7fda50074d60 2024-04-26T01:48:01.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.582+0000 7fda47e00700 1 --2- 172.21.15.1:0/2275146003 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fda340a3cd0 0x7fda340a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:01.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.583+0000 7fda47e00700 1 -- 172.21.15.1:0/2275146003 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7fda400cff90 con 0x7fda50074d60 2024-04-26T01:48:01.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.583+0000 7fda4ea00700 1 --2- 172.21.15.1:0/2275146003 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fda340a3cd0 0x7fda340a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:01.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.583+0000 7fda4ea00700 1 --2- 172.21.15.1:0/2275146003 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fda340a3cd0 0x7fda340a6190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fda501105d0 tx=0x7fda3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:01.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.587+0000 7fda47e00700 1 -- 172.21.15.1:0/2275146003 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda4009c8c0 con 0x7fda50074d60 2024-04-26T01:48:01.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.854+0000 7fda55000700 1 -- 172.21.15.1:0/2275146003 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fda50062e80 con 0x7fda340a3cd0 2024-04-26T01:48:01.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.896+0000 7fda47e00700 1 -- 172.21.15.1:0/2275146003 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fda50062e80 con 0x7fda340a3cd0 2024-04-26T01:48:01.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.899+0000 7fda55000700 1 -- 172.21.15.1:0/2275146003 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fda340a3cd0 msgr2=0x7fda340a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:01.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.899+0000 7fda55000700 1 --2- 172.21.15.1:0/2275146003 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fda340a3cd0 0x7fda340a6190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fda501105d0 tx=0x7fda3c009260 comp rx=0 tx=0).stop 2024-04-26T01:48:01.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.899+0000 7fda55000700 1 -- 172.21.15.1:0/2275146003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda50074d60 msgr2=0x7fda5019b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:01.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.899+0000 7fda55000700 1 --2- 172.21.15.1:0/2275146003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda50074d60 0x7fda5019b200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fda4000d950 tx=0x7fda4000dc60 comp rx=0 tx=0).stop 2024-04-26T01:48:01.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.900+0000 7fda55000700 1 -- 172.21.15.1:0/2275146003 shutdown_connections 2024-04-26T01:48:01.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.900+0000 7fda55000700 1 --2- 172.21.15.1:0/2275146003 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fda340a3cd0 0x7fda340a6190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:01.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.900+0000 7fda55000700 1 --2- 172.21.15.1:0/2275146003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda5010f220 0x7fda5019fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:01.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.900+0000 7fda55000700 1 --2- 172.21.15.1:0/2275146003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda50074d60 0x7fda5019b200 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:01.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.900+0000 7fda55000700 1 --2- 172.21.15.1:0/2275146003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda50075680 0x7fda5019b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:01.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.900+0000 7fda55000700 1 -- 172.21.15.1:0/2275146003 >> 172.21.15.1:0/2275146003 conn(0x7fda500fe490 msgr2=0x7fda500ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:01.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.900+0000 7fda55000700 1 -- 172.21.15.1:0/2275146003 shutdown_connections 2024-04-26T01:48:01.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:01.900+0000 7fda55000700 1 -- 172.21.15.1:0/2275146003 wait complete. 2024-04-26T01:48:02.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:02 smithi173 ceph-mon[32553]: pgmap v340: 97 pgs: 97 active+clean; 582 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-26T01:48:02.585 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:02 smithi001 ceph-mon[27822]: pgmap v340: 97 pgs: 97 active+clean; 582 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-26T01:48:02.646 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-26T01:48:02.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:02 smithi097 ceph-mon[32853]: pgmap v340: 97 pgs: 97 active+clean; 582 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-26T01:48:03.101 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:03.555 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:03 smithi001 ceph-mon[27822]: from='client.24625 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:03.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:03 smithi097 ceph-mon[32853]: from='client.24625 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:03.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:03 smithi173 ceph-mon[32553]: from='client.24625 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:04.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:04 smithi001 ceph-mon[27822]: pgmap v341: 97 pgs: 97 active+clean; 588 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-26T01:48:04.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:04 smithi097 ceph-mon[32853]: pgmap v341: 97 pgs: 97 active+clean; 588 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-26T01:48:04.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:04 smithi173 ceph-mon[32553]: pgmap v341: 97 pgs: 97 active+clean; 588 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-26T01:48:05.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.296+0000 7f48a3e00700 1 -- 172.21.15.1:0/1182520642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f489c06b070 msgr2=0x7f489c10efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:05.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.296+0000 7f48a3e00700 1 --2- 172.21.15.1:0/1182520642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f489c06b070 0x7f489c10efb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f48880098f0 tx=0x7f4888009c00 comp rx=0 tx=0).stop 2024-04-26T01:48:05.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.297+0000 7f48a3e00700 1 -- 172.21.15.1:0/1182520642 shutdown_connections 2024-04-26T01:48:05.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.297+0000 7f48a3e00700 1 --2- 172.21.15.1:0/1182520642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f489c10f4f0 0x7f489c1118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:05.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.297+0000 7f48a3e00700 1 --2- 172.21.15.1:0/1182520642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f489c06a750 0x7f489c06ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:05.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.297+0000 7f48a3e00700 1 --2- 172.21.15.1:0/1182520642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f489c06b070 0x7f489c10efb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:05.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.297+0000 7f48a3e00700 1 -- 172.21.15.1:0/1182520642 >> 172.21.15.1:0/1182520642 conn(0x7f489c0788f0 msgr2=0x7f489c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:05.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.297+0000 7f48a3e00700 1 -- 172.21.15.1:0/1182520642 shutdown_connections 2024-04-26T01:48:05.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.297+0000 7f48a3e00700 1 -- 172.21.15.1:0/1182520642 wait complete. 2024-04-26T01:48:05.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.299+0000 7f48a3e00700 1 Processor -- start 2024-04-26T01:48:05.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.299+0000 7f48a3e00700 1 -- start start 2024-04-26T01:48:05.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.299+0000 7f48a3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f489c06a750 0x7f489c19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:05.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.300+0000 7f48a3e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f489c06b070 0x7f489c19b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:05.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.300+0000 7f48a3e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f489c10f4f0 0x7f489c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:05.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.300+0000 7f48a3e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f489c114e00 con 0x7f489c06a750 2024-04-26T01:48:05.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.300+0000 7f48a3e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f489c114c80 con 0x7f489c10f4f0 2024-04-26T01:48:05.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.300+0000 7f48a3e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f489c114f80 con 0x7f489c06b070 2024-04-26T01:48:05.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.300+0000 7f48a1400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f489c06a750 0x7f489c19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:05.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.300+0000 7f48a1400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f489c06a750 0x7f489c19b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55016/0 (socket says 172.21.15.1:55016) 2024-04-26T01:48:05.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.300+0000 7f48a0a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f489c06b070 0x7f489c19b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:05.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.300+0000 7f48a1400700 1 -- 172.21.15.1:0/1981399274 learned_addr learned my addr 172.21.15.1:0/1981399274 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:48:05.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.300+0000 7f48a1e00700 1 --2- 172.21.15.1:0/1981399274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f489c10f4f0 0x7f489c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:05.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.300+0000 7f48a0a00700 1 -- 172.21.15.1:0/1981399274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f489c10f4f0 msgr2=0x7f489c19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:05.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.300+0000 7f48a0a00700 1 --2- 172.21.15.1:0/1981399274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f489c10f4f0 0x7f489c19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:05.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.300+0000 7f48a0a00700 1 -- 172.21.15.1:0/1981399274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f489c06a750 msgr2=0x7f489c19b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:05.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.300+0000 7f48a0a00700 1 --2- 172.21.15.1:0/1981399274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f489c06a750 0x7f489c19b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:05.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.300+0000 7f48a0a00700 1 -- 172.21.15.1:0/1981399274 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489c1a0380 con 0x7f489c06b070 2024-04-26T01:48:05.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.301+0000 7f48a0a00700 1 --2- 172.21.15.1:0/1981399274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f489c06b070 0x7f489c19b8b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4888009230 tx=0x7f488800fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:05.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.301+0000 7f489a000700 1 -- 172.21.15.1:0/1981399274 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888005140 con 0x7f489c06b070 2024-04-26T01:48:05.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.301+0000 7f48a3e00700 1 -- 172.21.15.1:0/1981399274 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489c1a0610 con 0x7f489c06b070 2024-04-26T01:48:05.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.301+0000 7f489a000700 1 -- 172.21.15.1:0/1981399274 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f488800b560 con 0x7f489c06b070 2024-04-26T01:48:05.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.301+0000 7f489a000700 1 -- 172.21.15.1:0/1981399274 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888003a40 con 0x7f489c06b070 2024-04-26T01:48:05.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.301+0000 7f48a3e00700 1 -- 172.21.15.1:0/1981399274 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f489c1a7ef0 con 0x7f489c06b070 2024-04-26T01:48:05.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.303+0000 7f48a3e00700 1 -- 172.21.15.1:0/1981399274 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f489c069730 con 0x7f489c06b070 2024-04-26T01:48:05.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.304+0000 7f489a000700 1 -- 172.21.15.1:0/1981399274 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f488800ba10 con 0x7f489c06b070 2024-04-26T01:48:05.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.306+0000 7f489a000700 1 --2- 172.21.15.1:0/1981399274 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f48840a3cd0 0x7f48840a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:05.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.306+0000 7f489a000700 1 -- 172.21.15.1:0/1981399274 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f48880d13d0 con 0x7f489c06b070 2024-04-26T01:48:05.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.306+0000 7f48a1400700 1 --2- 172.21.15.1:0/1981399274 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f48840a3cd0 0x7f48840a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:05.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.307+0000 7f48a1400700 1 --2- 172.21.15.1:0/1981399274 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f48840a3cd0 0x7f48840a6190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4890005d10 tx=0x7f4890009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:05.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.310+0000 7f489a000700 1 -- 172.21.15.1:0/1981399274 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4888013030 con 0x7f489c06b070 2024-04-26T01:48:05.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.585+0000 7f48a3e00700 1 -- 172.21.15.1:0/1981399274 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f489c062e80 con 0x7f48840a3cd0 2024-04-26T01:48:05.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.598+0000 7f489a000700 1 -- 172.21.15.1:0/1981399274 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f489c062e80 con 0x7f48840a3cd0 2024-04-26T01:48:05.596 INFO:teuthology.orchestra.run.smithi001.stdout:/volumes/qa/sv_0/3c9c1226-a639-45b5-a272-62db259d838f 2024-04-26T01:48:05.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.601+0000 7f48a3e00700 1 -- 172.21.15.1:0/1981399274 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f48840a3cd0 msgr2=0x7f48840a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:05.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.601+0000 7f48a3e00700 1 --2- 172.21.15.1:0/1981399274 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f48840a3cd0 0x7f48840a6190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4890005d10 tx=0x7f4890009310 comp rx=0 tx=0).stop 2024-04-26T01:48:05.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.601+0000 7f48a3e00700 1 -- 172.21.15.1:0/1981399274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f489c06b070 msgr2=0x7f489c19b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:05.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.601+0000 7f48a3e00700 1 --2- 172.21.15.1:0/1981399274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f489c06b070 0x7f489c19b8b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4888009230 tx=0x7f488800fa70 comp rx=0 tx=0).stop 2024-04-26T01:48:05.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.601+0000 7f48a3e00700 1 -- 172.21.15.1:0/1981399274 shutdown_connections 2024-04-26T01:48:05.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.601+0000 7f48a3e00700 1 --2- 172.21.15.1:0/1981399274 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f48840a3cd0 0x7f48840a6190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:05.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.601+0000 7f48a3e00700 1 --2- 172.21.15.1:0/1981399274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f489c10f4f0 0x7f489c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:05.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.601+0000 7f48a3e00700 1 --2- 172.21.15.1:0/1981399274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f489c06a750 0x7f489c19b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:05.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.601+0000 7f48a3e00700 1 --2- 172.21.15.1:0/1981399274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f489c06b070 0x7f489c19b8b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:05.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.601+0000 7f48a3e00700 1 -- 172.21.15.1:0/1981399274 >> 172.21.15.1:0/1981399274 conn(0x7f489c0788f0 msgr2=0x7f489c110a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:05.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.601+0000 7f48a3e00700 1 -- 172.21.15.1:0/1981399274 shutdown_connections 2024-04-26T01:48:05.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:05.601+0000 7f48a3e00700 1 -- 172.21.15.1:0/1981399274 wait complete. 2024-04-26T01:48:06.389 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-26T01:48:06.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:06 smithi001 ceph-mon[27822]: pgmap v342: 97 pgs: 97 active+clean; 588 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-26T01:48:06.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:06 smithi001 ceph-mon[27822]: from='client.24631 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:06.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:06 smithi097 ceph-mon[32853]: pgmap v342: 97 pgs: 97 active+clean; 588 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-26T01:48:06.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:06 smithi097 ceph-mon[32853]: from='client.24631 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:06.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:06 smithi173 ceph-mon[32553]: pgmap v342: 97 pgs: 97 active+clean; 588 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-26T01:48:06.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:06 smithi173 ceph-mon[32553]: from='client.24631 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:06.856 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:08.601 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:08 smithi001 ceph-mon[27822]: pgmap v343: 97 pgs: 97 active+clean; 608 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-04-26T01:48:08.618 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:08 smithi173 ceph-mon[32553]: pgmap v343: 97 pgs: 97 active+clean; 608 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-04-26T01:48:08.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:08 smithi097 ceph-mon[32853]: pgmap v343: 97 pgs: 97 active+clean; 608 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-04-26T01:48:08.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.926+0000 7fedf4c00700 1 -- 172.21.15.1:0/1222246193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedf0107dc0 msgr2=0x7fedf010f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.926+0000 7fedf4c00700 1 --2- 172.21.15.1:0/1222246193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedf0107dc0 0x7fedf010f990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fede8009fc0 tx=0x7fede800b330 comp rx=0 tx=0).stop 2024-04-26T01:48:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.927+0000 7fedf4c00700 1 -- 172.21.15.1:0/1222246193 shutdown_connections 2024-04-26T01:48:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.927+0000 7fedf4c00700 1 --2- 172.21.15.1:0/1222246193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedf0107dc0 0x7fedf010f990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.927+0000 7fedf4c00700 1 --2- 172.21.15.1:0/1222246193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf01011f0 0x7fedf01015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.927+0000 7fedf4c00700 1 --2- 172.21.15.1:0/1222246193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf0101ba0 0x7fedf01076e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.927+0000 7fedf4c00700 1 -- 172.21.15.1:0/1222246193 >> 172.21.15.1:0/1222246193 conn(0x7fedf0077f40 msgr2=0x7fedf00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.927+0000 7fedf4c00700 1 -- 172.21.15.1:0/1222246193 shutdown_connections 2024-04-26T01:48:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.927+0000 7fedf4c00700 1 -- 172.21.15.1:0/1222246193 wait complete. 2024-04-26T01:48:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.928+0000 7fedf4c00700 1 Processor -- start 2024-04-26T01:48:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.928+0000 7fedf4c00700 1 -- start start 2024-04-26T01:48:08.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.929+0000 7fedf4c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf01011f0 0x7fedf019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:08.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.929+0000 7fedf4c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedf0101ba0 0x7fedf019b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:08.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.929+0000 7fedf4c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf0107dc0 0x7fedf019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:08.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.929+0000 7fedf4c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fedf0112cb0 con 0x7fedf01011f0 2024-04-26T01:48:08.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.929+0000 7fedf4c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fedf0112b30 con 0x7fedf0107dc0 2024-04-26T01:48:08.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.929+0000 7fedf4c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fedf0112e30 con 0x7fedf0101ba0 2024-04-26T01:48:08.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.929+0000 7fedede00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf01011f0 0x7fedf019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:08.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.929+0000 7fedee800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf0107dc0 0x7fedf019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:08.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.929+0000 7fedede00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf01011f0 0x7fedf019b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55042/0 (socket says 172.21.15.1:55042) 2024-04-26T01:48:08.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.929+0000 7fedee800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf0107dc0 0x7fedf019fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:43798/0 (socket says 172.21.15.1:43798) 2024-04-26T01:48:08.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.929+0000 7fedede00700 1 -- 172.21.15.1:0/4144183524 learned_addr learned my addr 172.21.15.1:0/4144183524 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:48:08.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.929+0000 7feded400700 1 --2- 172.21.15.1:0/4144183524 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedf0101ba0 0x7fedf019b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:08.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.930+0000 7fedee800700 1 -- 172.21.15.1:0/4144183524 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedf0101ba0 msgr2=0x7fedf019b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:08.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.930+0000 7fedee800700 1 --2- 172.21.15.1:0/4144183524 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedf0101ba0 0x7fedf019b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:08.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.930+0000 7fedee800700 1 -- 172.21.15.1:0/4144183524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf01011f0 msgr2=0x7fedf019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:08.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.930+0000 7fedee800700 1 --2- 172.21.15.1:0/4144183524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf01011f0 0x7fedf019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:08.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.930+0000 7fedee800700 1 -- 172.21.15.1:0/4144183524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf01a0370 con 0x7fedf0107dc0 2024-04-26T01:48:08.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.930+0000 7fedee800700 1 --2- 172.21.15.1:0/4144183524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf0107dc0 0x7fedf019fc30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fede800b6c0 tx=0x7fede800cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:08.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.930+0000 7fede6a00700 1 -- 172.21.15.1:0/4144183524 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8021070 con 0x7fedf0107dc0 2024-04-26T01:48:08.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.931+0000 7fede6a00700 1 -- 172.21.15.1:0/4144183524 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede8007ac0 con 0x7fedf0107dc0 2024-04-26T01:48:08.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.931+0000 7fedf4c00700 1 -- 172.21.15.1:0/4144183524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf01a0600 con 0x7fedf0107dc0 2024-04-26T01:48:08.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.931+0000 7fede6a00700 1 -- 172.21.15.1:0/4144183524 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8005040 con 0x7fedf0107dc0 2024-04-26T01:48:08.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.931+0000 7fedf4c00700 1 -- 172.21.15.1:0/4144183524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf01a7ee0 con 0x7fedf0107dc0 2024-04-26T01:48:08.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.932+0000 7fedf4c00700 1 -- 172.21.15.1:0/4144183524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedf0069730 con 0x7fedf0107dc0 2024-04-26T01:48:08.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.934+0000 7fede6a00700 1 -- 172.21.15.1:0/4144183524 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fede8024070 con 0x7fedf0107dc0 2024-04-26T01:48:08.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.936+0000 7fede6a00700 1 --2- 172.21.15.1:0/4144183524 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fedd40a3cd0 0x7fedd40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:08.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.936+0000 7fede6a00700 1 -- 172.21.15.1:0/4144183524 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7fede800f020 con 0x7fedf0107dc0 2024-04-26T01:48:08.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.936+0000 7fedede00700 1 --2- 172.21.15.1:0/4144183524 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fedd40a3cd0 0x7fedd40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:08.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.937+0000 7fedede00700 1 --2- 172.21.15.1:0/4144183524 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fedd40a3cd0 0x7fedd40a6190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fede0009680 tx=0x7fede0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:08.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:08.940+0000 7fede6a00700 1 -- 172.21.15.1:0/4144183524 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fede802b020 con 0x7fedf0107dc0 2024-04-26T01:48:09.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:09.207+0000 7fedf4c00700 1 -- 172.21.15.1:0/4144183524 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fedf010ed60 con 0x7fedd40a3cd0 2024-04-26T01:48:09.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:09.246+0000 7fede6a00700 1 -- 172.21.15.1:0/4144183524 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fedf010ed60 con 0x7fedd40a3cd0 2024-04-26T01:48:09.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:09.249+0000 7fedf4c00700 1 -- 172.21.15.1:0/4144183524 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fedd40a3cd0 msgr2=0x7fedd40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:09.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:09.249+0000 7fedf4c00700 1 --2- 172.21.15.1:0/4144183524 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fedd40a3cd0 0x7fedd40a6190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fede0009680 tx=0x7fede0009260 comp rx=0 tx=0).stop 2024-04-26T01:48:09.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:09.249+0000 7fedf4c00700 1 -- 172.21.15.1:0/4144183524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf0107dc0 msgr2=0x7fedf019fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:09.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:09.249+0000 7fedf4c00700 1 --2- 172.21.15.1:0/4144183524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf0107dc0 0x7fedf019fc30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fede800b6c0 tx=0x7fede800cc30 comp rx=0 tx=0).stop 2024-04-26T01:48:09.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:09.249+0000 7fedf4c00700 1 -- 172.21.15.1:0/4144183524 shutdown_connections 2024-04-26T01:48:09.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:09.249+0000 7fedf4c00700 1 --2- 172.21.15.1:0/4144183524 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fedd40a3cd0 0x7fedd40a6190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:09.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:09.249+0000 7fedf4c00700 1 --2- 172.21.15.1:0/4144183524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf0107dc0 0x7fedf019fc30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:09.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:09.249+0000 7fedf4c00700 1 --2- 172.21.15.1:0/4144183524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf01011f0 0x7fedf019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:09.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:09.249+0000 7fedf4c00700 1 --2- 172.21.15.1:0/4144183524 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedf0101ba0 0x7fedf019b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:09.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:09.249+0000 7fedf4c00700 1 -- 172.21.15.1:0/4144183524 >> 172.21.15.1:0/4144183524 conn(0x7fedf0077f40 msgr2=0x7fedf0106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:09.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:09.250+0000 7fedf4c00700 1 -- 172.21.15.1:0/4144183524 shutdown_connections 2024-04-26T01:48:09.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:09.250+0000 7fedf4c00700 1 -- 172.21.15.1:0/4144183524 wait complete. 2024-04-26T01:48:10.047 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-26T01:48:10.515 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:10.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:10 smithi001 ceph-mon[27822]: pgmap v344: 97 pgs: 97 active+clean; 608 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-04-26T01:48:10.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:10 smithi001 ceph-mon[27822]: from='client.24614 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:10.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:10 smithi097 ceph-mon[32853]: pgmap v344: 97 pgs: 97 active+clean; 608 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-04-26T01:48:10.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:10 smithi097 ceph-mon[32853]: from='client.24614 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:10.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:10 smithi173 ceph-mon[32553]: pgmap v344: 97 pgs: 97 active+clean; 608 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-04-26T01:48:10.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:10 smithi173 ceph-mon[32553]: from='client.24614 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:12.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:12 smithi001 ceph-mon[27822]: pgmap v345: 97 pgs: 97 active+clean; 608 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-04-26T01:48:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.637+0000 7f035da00700 1 -- 172.21.15.1:0/2291275784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03581034b0 msgr2=0x7f035810f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.637+0000 7f035da00700 1 --2- 172.21.15.1:0/2291275784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03581034b0 0x7f035810f920 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f035000b0d0 tx=0x7f035000b3e0 comp rx=0 tx=0).stop 2024-04-26T01:48:12.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.637+0000 7f035da00700 1 -- 172.21.15.1:0/2291275784 shutdown_connections 2024-04-26T01:48:12.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.637+0000 7f035da00700 1 --2- 172.21.15.1:0/2291275784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03581034b0 0x7f035810f920 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:12.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.637+0000 7f035da00700 1 --2- 172.21.15.1:0/2291275784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0358107940 0x7f0358107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:12.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.637+0000 7f035da00700 1 --2- 172.21.15.1:0/2291275784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0358102af0 0x7f0358102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:12.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.637+0000 7f035da00700 1 -- 172.21.15.1:0/2291275784 >> 172.21.15.1:0/2291275784 conn(0x7f0358079ba0 msgr2=0x7f0358079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:12.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.637+0000 7f035da00700 1 -- 172.21.15.1:0/2291275784 shutdown_connections 2024-04-26T01:48:12.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.637+0000 7f035da00700 1 -- 172.21.15.1:0/2291275784 wait complete. 2024-04-26T01:48:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.639+0000 7f035da00700 1 Processor -- start 2024-04-26T01:48:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.639+0000 7f035da00700 1 -- start start 2024-04-26T01:48:12.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.639+0000 7f035da00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0358102af0 0x7f0358074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:12.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.639+0000 7f035da00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03581034b0 0x7f0358075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:12.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.639+0000 7f035da00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0358107940 0x7f03580757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:12.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.639+0000 7f035da00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0358112c90 con 0x7f0358107940 2024-04-26T01:48:12.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.639+0000 7f035da00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0358112b10 con 0x7f03581034b0 2024-04-26T01:48:12.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.639+0000 7f035da00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0358112e10 con 0x7f0358102af0 2024-04-26T01:48:12.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.640+0000 7f0356a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0358102af0 0x7f0358074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:12.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.640+0000 7f0356a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0358102af0 0x7f0358074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:47080/0 (socket says 172.21.15.1:47080) 2024-04-26T01:48:12.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.640+0000 7f0356a00700 1 -- 172.21.15.1:0/3592563041 learned_addr learned my addr 172.21.15.1:0/3592563041 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:48:12.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.640+0000 7f0357400700 1 --2- 172.21.15.1:0/3592563041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0358107940 0x7f03580757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:12.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.640+0000 7f034e000700 1 --2- 172.21.15.1:0/3592563041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03581034b0 0x7f0358075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:12.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.640+0000 7f0356a00700 1 -- 172.21.15.1:0/3592563041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03581034b0 msgr2=0x7f0358075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:12.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.640+0000 7f0356a00700 1 --2- 172.21.15.1:0/3592563041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03581034b0 0x7f0358075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:12.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.640+0000 7f0356a00700 1 -- 172.21.15.1:0/3592563041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0358107940 msgr2=0x7f03580757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:12.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.640+0000 7f0356a00700 1 --2- 172.21.15.1:0/3592563041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0358107940 0x7f03580757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:12.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.640+0000 7f0356a00700 1 -- 172.21.15.1:0/3592563041 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03581a75c0 con 0x7f0358102af0 2024-04-26T01:48:12.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.640+0000 7f0356a00700 1 --2- 172.21.15.1:0/3592563041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0358102af0 0x7f0358074d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f034800e9a0 tx=0x7f034800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:12.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.641+0000 7f034fe00700 1 -- 172.21.15.1:0/3592563041 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034800ca10 con 0x7f0358102af0 2024-04-26T01:48:12.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.641+0000 7f034fe00700 1 -- 172.21.15.1:0/3592563041 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f034800cbb0 con 0x7f0358102af0 2024-04-26T01:48:12.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.641+0000 7f034fe00700 1 -- 172.21.15.1:0/3592563041 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0348018680 con 0x7f0358102af0 2024-04-26T01:48:12.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.641+0000 7f035da00700 1 -- 172.21.15.1:0/3592563041 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03581a78b0 con 0x7f0358102af0 2024-04-26T01:48:12.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.641+0000 7f035da00700 1 -- 172.21.15.1:0/3592563041 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03581a7df0 con 0x7f0358102af0 2024-04-26T01:48:12.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.642+0000 7f035da00700 1 -- 172.21.15.1:0/3592563041 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0358069730 con 0x7f0358102af0 2024-04-26T01:48:12.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.644+0000 7f034fe00700 1 -- 172.21.15.1:0/3592563041 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f0348010430 con 0x7f0358102af0 2024-04-26T01:48:12.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.645+0000 7f034fe00700 1 --2- 172.21.15.1:0/3592563041 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f033c0a3cd0 0x7f033c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:12.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.646+0000 7f034fe00700 1 -- 172.21.15.1:0/3592563041 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f03480d0680 con 0x7f0358102af0 2024-04-26T01:48:12.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.646+0000 7f034e000700 1 --2- 172.21.15.1:0/3592563041 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f033c0a3cd0 0x7f033c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:12.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.646+0000 7f034e000700 1 --2- 172.21.15.1:0/3592563041 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f033c0a3cd0 0x7f033c0a6190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f035807a130 tx=0x7f0340009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:12.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.648+0000 7f034fe00700 1 -- 172.21.15.1:0/3592563041 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f034809cfb0 con 0x7f0358102af0 2024-04-26T01:48:12.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:12 smithi097 ceph-mon[32853]: pgmap v345: 97 pgs: 97 active+clean; 608 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-04-26T01:48:12.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:12 smithi173 ceph-mon[32553]: pgmap v345: 97 pgs: 97 active+clean; 608 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-04-26T01:48:12.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.919+0000 7f035da00700 1 -- 172.21.15.1:0/3592563041 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f0358062e80 con 0x7f033c0a3cd0 2024-04-26T01:48:12.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.931+0000 7f034fe00700 1 -- 172.21.15.1:0/3592563041 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f0358062e80 con 0x7f033c0a3cd0 2024-04-26T01:48:12.929 INFO:teuthology.orchestra.run.smithi001.stdout:/volumes/qa/sv_1/ef5e77dd-fbc9-41b0-9f37-faa021010076 2024-04-26T01:48:12.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.934+0000 7f035da00700 1 -- 172.21.15.1:0/3592563041 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f033c0a3cd0 msgr2=0x7f033c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:12.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.934+0000 7f035da00700 1 --2- 172.21.15.1:0/3592563041 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f033c0a3cd0 0x7f033c0a6190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f035807a130 tx=0x7f0340009260 comp rx=0 tx=0).stop 2024-04-26T01:48:12.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.934+0000 7f035da00700 1 -- 172.21.15.1:0/3592563041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0358102af0 msgr2=0x7f0358074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:12.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.934+0000 7f035da00700 1 --2- 172.21.15.1:0/3592563041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0358102af0 0x7f0358074d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f034800e9a0 tx=0x7f034800ecb0 comp rx=0 tx=0).stop 2024-04-26T01:48:12.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.934+0000 7f035da00700 1 -- 172.21.15.1:0/3592563041 shutdown_connections 2024-04-26T01:48:12.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.934+0000 7f035da00700 1 --2- 172.21.15.1:0/3592563041 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f033c0a3cd0 0x7f033c0a6190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:12.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.934+0000 7f035da00700 1 --2- 172.21.15.1:0/3592563041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0358107940 0x7f03580757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:12.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.934+0000 7f035da00700 1 --2- 172.21.15.1:0/3592563041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0358102af0 0x7f0358074d40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:12.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.934+0000 7f035da00700 1 --2- 172.21.15.1:0/3592563041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03581034b0 0x7f0358075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:12.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.934+0000 7f035da00700 1 -- 172.21.15.1:0/3592563041 >> 172.21.15.1:0/3592563041 conn(0x7f0358079ba0 msgr2=0x7f0358100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:12.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.934+0000 7f035da00700 1 -- 172.21.15.1:0/3592563041 shutdown_connections 2024-04-26T01:48:12.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:12.934+0000 7f035da00700 1 -- 172.21.15.1:0/3592563041 wait complete. 2024-04-26T01:48:13.693 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:48:14.158 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:14 smithi001 ceph-mon[27822]: pgmap v346: 97 pgs: 97 active+clean; 630 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-04-26T01:48:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:14 smithi001 ceph-mon[27822]: from='client.24643 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:14.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:14 smithi097 ceph-mon[32853]: pgmap v346: 97 pgs: 97 active+clean; 630 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-04-26T01:48:14.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:14 smithi097 ceph-mon[32853]: from='client.24643 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:14.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:14 smithi173 ceph-mon[32553]: pgmap v346: 97 pgs: 97 active+clean; 630 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-04-26T01:48:14.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:14 smithi173 ceph-mon[32553]: from='client.24643 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:16.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.253+0000 7f6317800700 1 -- 172.21.15.1:0/1895443226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63101092d0 msgr2=0x7f63101096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:16.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.253+0000 7f6317800700 1 --2- 172.21.15.1:0/1895443226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63101092d0 0x7f63101096b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f63040099c0 tx=0x7f6304009cd0 comp rx=0 tx=0).stop 2024-04-26T01:48:16.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.253+0000 7f6317800700 1 -- 172.21.15.1:0/1895443226 shutdown_connections 2024-04-26T01:48:16.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.253+0000 7f6317800700 1 --2- 172.21.15.1:0/1895443226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f631010e3b0 0x7f6310115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:16.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.253+0000 7f6317800700 1 --2- 172.21.15.1:0/1895443226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63101092d0 0x7f63101096b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:16.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.253+0000 7f6317800700 1 --2- 172.21.15.1:0/1895443226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6310109c80 0x7f631010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:16.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.253+0000 7f6317800700 1 -- 172.21.15.1:0/1895443226 >> 172.21.15.1:0/1895443226 conn(0x7f6310076210 msgr2=0x7f6310107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:16.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.253+0000 7f6317800700 1 -- 172.21.15.1:0/1895443226 shutdown_connections 2024-04-26T01:48:16.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.253+0000 7f6317800700 1 -- 172.21.15.1:0/1895443226 wait complete. 2024-04-26T01:48:16.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.254+0000 7f6317800700 1 Processor -- start 2024-04-26T01:48:16.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.255+0000 7f6317800700 1 -- start start 2024-04-26T01:48:16.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.255+0000 7f6317800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63101092d0 0x7f63101a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:16.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.255+0000 7f6317800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6310109c80 0x7f63101a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:16.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.255+0000 7f6317800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f631010e3b0 0x7f631019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:16.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.255+0000 7f6317800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6310119290 con 0x7f6310109c80 2024-04-26T01:48:16.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.255+0000 7f6317800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6310119110 con 0x7f631010e3b0 2024-04-26T01:48:16.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.255+0000 7f6317800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6310119410 con 0x7f63101092d0 2024-04-26T01:48:16.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.255+0000 7f630fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6310109c80 0x7f63101a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:16.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.255+0000 7f6314e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63101092d0 0x7f63101a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:16.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.255+0000 7f630fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6310109c80 0x7f63101a5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56598/0 (socket says 172.21.15.1:56598) 2024-04-26T01:48:16.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.255+0000 7f630fe00700 1 -- 172.21.15.1:0/4236797413 learned_addr learned my addr 172.21.15.1:0/4236797413 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:48:16.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.256+0000 7f6315800700 1 --2- 172.21.15.1:0/4236797413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f631010e3b0 0x7f631019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:16.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.256+0000 7f630fe00700 1 -- 172.21.15.1:0/4236797413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63101092d0 msgr2=0x7f63101a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:16.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.256+0000 7f630fe00700 1 --2- 172.21.15.1:0/4236797413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63101092d0 0x7f63101a53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:16.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.256+0000 7f630fe00700 1 -- 172.21.15.1:0/4236797413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f631010e3b0 msgr2=0x7f631019f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:16.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.256+0000 7f630fe00700 1 --2- 172.21.15.1:0/4236797413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f631010e3b0 0x7f631019f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:16.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.256+0000 7f630fe00700 1 -- 172.21.15.1:0/4236797413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631019fd10 con 0x7f6310109c80 2024-04-26T01:48:16.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.256+0000 7f630fe00700 1 --2- 172.21.15.1:0/4236797413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6310109c80 0x7f63101a5930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00e970 tx=0x7f62fc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:16.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.257+0000 7f630d600700 1 -- 172.21.15.1:0/4236797413 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc004a30 con 0x7f6310109c80 2024-04-26T01:48:16.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.257+0000 7f6317800700 1 -- 172.21.15.1:0/4236797413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63101a0000 con 0x7f6310109c80 2024-04-26T01:48:16.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.257+0000 7f630d600700 1 -- 172.21.15.1:0/4236797413 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62fc00cc90 con 0x7f6310109c80 2024-04-26T01:48:16.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.257+0000 7f630d600700 1 -- 172.21.15.1:0/4236797413 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc005610 con 0x7f6310109c80 2024-04-26T01:48:16.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.257+0000 7f6317800700 1 -- 172.21.15.1:0/4236797413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63101ac2c0 con 0x7f6310109c80 2024-04-26T01:48:16.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.258+0000 7f6317800700 1 -- 172.21.15.1:0/4236797413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6310069730 con 0x7f6310109c80 2024-04-26T01:48:16.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.260+0000 7f630d600700 1 -- 172.21.15.1:0/4236797413 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f62fc00c7e0 con 0x7f6310109c80 2024-04-26T01:48:16.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.261+0000 7f630d600700 1 --2- 172.21.15.1:0/4236797413 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f62f80a3cd0 0x7f62f80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:16.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.262+0000 7f630d600700 1 -- 172.21.15.1:0/4236797413 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f62fc0d0670 con 0x7f6310109c80 2024-04-26T01:48:16.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.262+0000 7f6314e00700 1 --2- 172.21.15.1:0/4236797413 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f62f80a3cd0 0x7f62f80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:16.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.262+0000 7f6314e00700 1 --2- 172.21.15.1:0/4236797413 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f62f80a3cd0 0x7f62f80a6190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6304000c00 tx=0x7f6304005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:16.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.266+0000 7f630d600700 1 -- 172.21.15.1:0/4236797413 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62fc09cfa0 con 0x7f6310109c80 2024-04-26T01:48:16.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.550+0000 7f6317800700 1 -- 172.21.15.1:0/4236797413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f631002df80 con 0x7f6310109c80 2024-04-26T01:48:16.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.551+0000 7f630d600700 1 -- 172.21.15.1:0/4236797413 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11841 (secure 0 0 0) 0x7f62fc09c8a0 con 0x7f6310109c80 2024-04-26T01:48:16.550 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:48:16.550 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:47:54.077766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2952462325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2952462325},{"type":"v1","addr":"172.21.15.173:6835","nonce":2952462325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24514,24518,24499]},"id":1}]} 2024-04-26T01:48:16.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.554+0000 7f6317800700 1 -- 172.21.15.1:0/4236797413 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f62f80a3cd0 msgr2=0x7f62f80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:16.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.554+0000 7f6317800700 1 --2- 172.21.15.1:0/4236797413 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f62f80a3cd0 0x7f62f80a6190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6304000c00 tx=0x7f6304005fb0 comp rx=0 tx=0).stop 2024-04-26T01:48:16.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.554+0000 7f6317800700 1 -- 172.21.15.1:0/4236797413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6310109c80 msgr2=0x7f63101a5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:16.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.554+0000 7f6317800700 1 --2- 172.21.15.1:0/4236797413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6310109c80 0x7f63101a5930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00e970 tx=0x7f62fc00ec80 comp rx=0 tx=0).stop 2024-04-26T01:48:16.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.554+0000 7f6317800700 1 -- 172.21.15.1:0/4236797413 shutdown_connections 2024-04-26T01:48:16.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.554+0000 7f6317800700 1 --2- 172.21.15.1:0/4236797413 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f62f80a3cd0 0x7f62f80a6190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:16.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.554+0000 7f6317800700 1 --2- 172.21.15.1:0/4236797413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f631010e3b0 0x7f631019f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:16.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.554+0000 7f6317800700 1 --2- 172.21.15.1:0/4236797413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63101092d0 0x7f63101a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:16.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.554+0000 7f6317800700 1 --2- 172.21.15.1:0/4236797413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6310109c80 0x7f63101a5930 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:16.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.554+0000 7f6317800700 1 -- 172.21.15.1:0/4236797413 >> 172.21.15.1:0/4236797413 conn(0x7f6310076210 msgr2=0x7f6310077e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:16.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.555+0000 7f6317800700 1 -- 172.21.15.1:0/4236797413 shutdown_connections 2024-04-26T01:48:16.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:16.555+0000 7f6317800700 1 -- 172.21.15.1:0/4236797413 wait complete. 2024-04-26T01:48:16.554 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-04-26T01:48:16.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:16 smithi001 ceph-mon[27822]: pgmap v347: 97 pgs: 97 active+clean; 630 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-26T01:48:16.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:16 smithi097 ceph-mon[32853]: pgmap v347: 97 pgs: 97 active+clean; 630 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-26T01:48:16.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:16 smithi173 ceph-mon[32553]: pgmap v347: 97 pgs: 97 active+clean; 630 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-26T01:48:17.349 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:48:17.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:17 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/4236797413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:48:17.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:17 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/4236797413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:48:17.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:17 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/4236797413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:48:17.822 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:18.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:18 smithi001 ceph-mon[27822]: pgmap v348: 97 pgs: 97 active+clean; 635 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-04-26T01:48:18.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:18 smithi097 ceph-mon[32853]: pgmap v348: 97 pgs: 97 active+clean; 635 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-04-26T01:48:18.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:18 smithi173 ceph-mon[32553]: pgmap v348: 97 pgs: 97 active+clean; 635 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-04-26T01:48:19.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.861+0000 7f0bb1800700 1 -- 172.21.15.1:0/3962377254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bac10e3b0 msgr2=0x7f0bac115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:19.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.861+0000 7f0bb1800700 1 --2- 172.21.15.1:0/3962377254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bac10e3b0 0x7f0bac115f80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0ba400b0d0 tx=0x7f0ba400b3e0 comp rx=0 tx=0).stop 2024-04-26T01:48:19.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.861+0000 7f0bb1800700 1 -- 172.21.15.1:0/3962377254 shutdown_connections 2024-04-26T01:48:19.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.861+0000 7f0bb1800700 1 --2- 172.21.15.1:0/3962377254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bac10e3b0 0x7f0bac115f80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:19.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.861+0000 7f0bb1800700 1 --2- 172.21.15.1:0/3962377254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bac1092d0 0x7f0bac1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:19.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.861+0000 7f0bb1800700 1 --2- 172.21.15.1:0/3962377254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bac109c80 0x7f0bac10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:19.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.861+0000 7f0bb1800700 1 -- 172.21.15.1:0/3962377254 >> 172.21.15.1:0/3962377254 conn(0x7f0bac076210 msgr2=0x7f0bac107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:19.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.861+0000 7f0bb1800700 1 -- 172.21.15.1:0/3962377254 shutdown_connections 2024-04-26T01:48:19.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.861+0000 7f0bb1800700 1 -- 172.21.15.1:0/3962377254 wait complete. 2024-04-26T01:48:19.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.862+0000 7f0bb1800700 1 Processor -- start 2024-04-26T01:48:19.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.862+0000 7f0bb1800700 1 -- start start 2024-04-26T01:48:19.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.863+0000 7f0bb1800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bac1092d0 0x7f0bac19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:19.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.863+0000 7f0bb1800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bac109c80 0x7f0bac19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:19.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.863+0000 7f0bb1800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bac10e3b0 0x7f0bac1a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:19.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.863+0000 7f0bb1800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0bac1192a0 con 0x7f0bac10e3b0 2024-04-26T01:48:19.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.863+0000 7f0bb1800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0bac119120 con 0x7f0bac1092d0 2024-04-26T01:48:19.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.863+0000 7f0bb1800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0bac119420 con 0x7f0bac109c80 2024-04-26T01:48:19.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.863+0000 7f0baaa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bac1092d0 0x7f0bac19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:19.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.863+0000 7f0baaa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bac1092d0 0x7f0bac19f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:40914/0 (socket says 172.21.15.1:40914) 2024-04-26T01:48:19.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.863+0000 7f0ba3e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bac109c80 0x7f0bac19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:19.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.863+0000 7f0baaa00700 1 -- 172.21.15.1:0/1384095574 learned_addr learned my addr 172.21.15.1:0/1384095574 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:48:19.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.863+0000 7f0bab400700 1 --2- 172.21.15.1:0/1384095574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bac10e3b0 0x7f0bac1a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:19.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.863+0000 7f0baaa00700 1 -- 172.21.15.1:0/1384095574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bac109c80 msgr2=0x7f0bac19fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:19.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.863+0000 7f0baaa00700 1 --2- 172.21.15.1:0/1384095574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bac109c80 0x7f0bac19fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:19.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.863+0000 7f0baaa00700 1 -- 172.21.15.1:0/1384095574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bac10e3b0 msgr2=0x7f0bac1a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:19.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.863+0000 7f0baaa00700 1 --2- 172.21.15.1:0/1384095574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bac10e3b0 0x7f0bac1a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:19.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.863+0000 7f0baaa00700 1 -- 172.21.15.1:0/1384095574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bac1a4740 con 0x7f0bac1092d0 2024-04-26T01:48:19.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.863+0000 7f0ba3e00700 1 --2- 172.21.15.1:0/1384095574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bac109c80 0x7f0bac19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:48:19.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.864+0000 7f0baaa00700 1 --2- 172.21.15.1:0/1384095574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bac1092d0 0x7f0bac19f730 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00e9a0 tx=0x7f0b9c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:19.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.864+0000 7f0ba3400700 1 -- 172.21.15.1:0/1384095574 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c00ca10 con 0x7f0bac1092d0 2024-04-26T01:48:19.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.864+0000 7f0bb1800700 1 -- 172.21.15.1:0/1384095574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bac1a4a30 con 0x7f0bac1092d0 2024-04-26T01:48:19.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.864+0000 7f0ba3400700 1 -- 172.21.15.1:0/1384095574 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b9c00cbb0 con 0x7f0bac1092d0 2024-04-26T01:48:19.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.864+0000 7f0ba3400700 1 -- 172.21.15.1:0/1384095574 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c018680 con 0x7f0bac1092d0 2024-04-26T01:48:19.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.864+0000 7f0bb1800700 1 -- 172.21.15.1:0/1384095574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bac1ac2b0 con 0x7f0bac1092d0 2024-04-26T01:48:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.865+0000 7f0bb1800700 1 -- 172.21.15.1:0/1384095574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bac069730 con 0x7f0bac1092d0 2024-04-26T01:48:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.867+0000 7f0ba3400700 1 -- 172.21.15.1:0/1384095574 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f0b9c010430 con 0x7f0bac1092d0 2024-04-26T01:48:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.869+0000 7f0ba3400700 1 --2- 172.21.15.1:0/1384095574 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0b900a3cd0 0x7f0b900a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.869+0000 7f0ba3400700 1 -- 172.21.15.1:0/1384095574 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f0b9c0d0640 con 0x7f0bac1092d0 2024-04-26T01:48:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.869+0000 7f0ba3e00700 1 --2- 172.21.15.1:0/1384095574 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0b900a3cd0 0x7f0b900a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.870+0000 7f0ba3e00700 1 --2- 172.21.15.1:0/1384095574 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0b900a3cd0 0x7f0b900a6190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0bac108830 tx=0x7f0b94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:19.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:19.873+0000 7f0ba3400700 1 -- 172.21.15.1:0/1384095574 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0b9c09cf70 con 0x7f0bac1092d0 2024-04-26T01:48:20.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:20.103+0000 7f0bb1800700 1 -- 172.21.15.1:0/1384095574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0bac02df80 con 0x7f0bac1092d0 2024-04-26T01:48:20.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:20.105+0000 7f0ba3400700 1 -- 172.21.15.1:0/1384095574 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22124 (secure 0 0 0) 0x7f0b9c09c870 con 0x7f0bac1092d0 2024-04-26T01:48:20.104 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:48:20.104 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":90,"fsid":"77cfcc80-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:38:03.531182+0000","modified":"2024-04-26T01:47:29.999575+0000","last_up_change":"2024-04-26T01:46:16.151275+0000","last_in_change":"2024-04-26T01:45:56.514228+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:42:04.805488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:47:21.957421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:47:25.860260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb10ae12-09ee-4d6b-bc78-8c6016613b81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6803","nonce":3176633965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6805","nonce":3176633965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6809","nonce":3176633965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6807","nonce":3176633965}]},"public_addr":"172.21.15.1:6803/3176633965","cluster_addr":"172.21.15.1:6805/3176633965","heartbeat_back_addr":"172.21.15.1:6809/3176633965","heartbeat_front_addr":"172.21.15.1:6807/3176633965","state":["exists","up"]},{"osd":1,"uuid":"acd12888-6ff6-40bc-99e7-e5c395e85b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6801","nonce":3755022621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6803","nonce":3755022621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6807","nonce":3755022621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6805","nonce":3755022621}]},"public_addr":"172.21.15.97:6801/3755022621","cluster_addr":"172.21.15.97:6803/3755022621","heartbeat_back_addr":"172.21.15.97:6807/3755022621","heartbeat_front_addr":"172.21.15.97:6805/3755022621","state":["exists","up"]},{"osd":2,"uuid":"79971508-e256-4145-8111-a75666e886fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6801","nonce":734861771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6803","nonce":734861771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6807","nonce":734861771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6805","nonce":734861771}]},"public_addr":"172.21.15.173:6801/734861771","cluster_addr":"172.21.15.173:6803/734861771","heartbeat_back_addr":"172.21.15.173:6807/734861771","heartbeat_front_addr":"172.21.15.173:6805/734861771","state":["exists","up"]},{"osd":3,"uuid":"a8ccc4a9-0f31-446a-8bcb-4691532397c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6811","nonce":3349961961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6813","nonce":3349961961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6817","nonce":3349961961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6815","nonce":3349961961}]},"public_addr":"172.21.15.1:6811/3349961961","cluster_addr":"172.21.15.1:6813/3349961961","heartbeat_back_addr":"172.21.15.1:6817/3349961961","heartbeat_front_addr":"172.21.15.1:6815/3349961961","state":["exists","up"]},{"osd":4,"uuid":"5b724648-4df7-40fc-89d5-55f170cdd806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6809","nonce":2494639397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6811","nonce":2494639397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6815","nonce":2494639397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6813","nonce":2494639397}]},"public_addr":"172.21.15.97:6809/2494639397","cluster_addr":"172.21.15.97:6811/2494639397","heartbeat_back_addr":"172.21.15.97:6815/2494639397","heartbeat_front_addr":"172.21.15.97:6813/2494639397","state":["exists","up"]},{"osd":5,"uuid":"a5a24a9c-3007-4b87-8562-40c98c41ef8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6809","nonce":1026507957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6811","nonce":1026507957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6815","nonce":1026507957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6813","nonce":1026507957}]},"public_addr":"172.21.15.173:6809/1026507957","cluster_addr":"172.21.15.173:6811/1026507957","heartbeat_back_addr":"172.21.15.173:6815/1026507957","heartbeat_front_addr":"172.21.15.173:6813/1026507957","state":["exists","up"]},{"osd":6,"uuid":"97cd2e88-a56c-4b1a-8104-65b23a77986a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6819","nonce":444030587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6821","nonce":444030587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6825","nonce":444030587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6823","nonce":444030587}]},"public_addr":"172.21.15.1:6819/444030587","cluster_addr":"172.21.15.1:6821/444030587","heartbeat_back_addr":"172.21.15.1:6825/444030587","heartbeat_front_addr":"172.21.15.1:6823/444030587","state":["exists","up"]},{"osd":7,"uuid":"afc0c2dd-e112-4358-b3aa-944366d5d383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6817","nonce":2600022531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6819","nonce":2600022531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6823","nonce":2600022531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6821","nonce":2600022531}]},"public_addr":"172.21.15.97:6817/2600022531","cluster_addr":"172.21.15.97:6819/2600022531","heartbeat_back_addr":"172.21.15.97:6823/2600022531","heartbeat_front_addr":"172.21.15.97:6821/2600022531","state":["exists","up"]},{"osd":8,"uuid":"41fa612b-793e-44da-af8b-44d17a9de303","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6817","nonce":1334205527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6819","nonce":1334205527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6823","nonce":1334205527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6821","nonce":1334205527}]},"public_addr":"172.21.15.173:6817/1334205527","cluster_addr":"172.21.15.173:6819/1334205527","heartbeat_back_addr":"172.21.15.173:6823/1334205527","heartbeat_front_addr":"172.21.15.173:6821/1334205527","state":["exists","up"]},{"osd":9,"uuid":"bbbb163b-f8c4-4be5-8f80-82445e8d407f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6827","nonce":2973486839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6829","nonce":2973486839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6833","nonce":2973486839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6831","nonce":2973486839}]},"public_addr":"172.21.15.1:6827/2973486839","cluster_addr":"172.21.15.1:6829/2973486839","heartbeat_back_addr":"172.21.15.1:6833/2973486839","heartbeat_front_addr":"172.21.15.1:6831/2973486839","state":["exists","up"]},{"osd":10,"uuid":"1b447b34-0600-4727-826d-b3d68adfb18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6825","nonce":1693425268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6827","nonce":1693425268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6831","nonce":1693425268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6829","nonce":1693425268}]},"public_addr":"172.21.15.97:6825/1693425268","cluster_addr":"172.21.15.97:6827/1693425268","heartbeat_back_addr":"172.21.15.97:6831/1693425268","heartbeat_front_addr":"172.21.15.97:6829/1693425268","state":["exists","up"]},{"osd":11,"uuid":"f439aae1-f50a-4176-889d-36fa5131541a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6825","nonce":1141524960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6827","nonce":1141524960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6831","nonce":1141524960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6829","nonce":1141524960}]},"public_addr":"172.21.15.173:6825/1141524960","cluster_addr":"172.21.15.173:6827/1141524960","heartbeat_back_addr":"172.21.15.173:6831/1141524960","heartbeat_front_addr":"172.21.15.173:6829/1141524960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:12.955643+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:38.846652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:02.556062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:34.900933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:01.170878+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:24.842573+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:58.304440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:24.868689+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:48.772003+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:22.611681+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:49.470779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:14.163984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6800/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/1000222947":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/457958597":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6801/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:6800/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/2715798962":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/1923058728":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/2489038777":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/2869502448":"2024-04-27T01:38:45.974031+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:48:20.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:20.108+0000 7f0bb1800700 1 -- 172.21.15.1:0/1384095574 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0b900a3cd0 msgr2=0x7f0b900a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:20.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:20.108+0000 7f0bb1800700 1 --2- 172.21.15.1:0/1384095574 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0b900a3cd0 0x7f0b900a6190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0bac108830 tx=0x7f0b94009260 comp rx=0 tx=0).stop 2024-04-26T01:48:20.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:20.108+0000 7f0bb1800700 1 -- 172.21.15.1:0/1384095574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bac1092d0 msgr2=0x7f0bac19f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:20.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:20.108+0000 7f0bb1800700 1 --2- 172.21.15.1:0/1384095574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bac1092d0 0x7f0bac19f730 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00e9a0 tx=0x7f0b9c00ecb0 comp rx=0 tx=0).stop 2024-04-26T01:48:20.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:20.108+0000 7f0bb1800700 1 -- 172.21.15.1:0/1384095574 shutdown_connections 2024-04-26T01:48:20.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:20.108+0000 7f0bb1800700 1 --2- 172.21.15.1:0/1384095574 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0b900a3cd0 0x7f0b900a6190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:20.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:20.108+0000 7f0bb1800700 1 --2- 172.21.15.1:0/1384095574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bac10e3b0 0x7f0bac1a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:20.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:20.108+0000 7f0bb1800700 1 --2- 172.21.15.1:0/1384095574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bac1092d0 0x7f0bac19f730 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:20.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:20.108+0000 7f0bb1800700 1 --2- 172.21.15.1:0/1384095574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bac109c80 0x7f0bac19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:20.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:20.108+0000 7f0bb1800700 1 -- 172.21.15.1:0/1384095574 >> 172.21.15.1:0/1384095574 conn(0x7f0bac076210 msgr2=0x7f0bac076790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:20.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:20.109+0000 7f0bb1800700 1 -- 172.21.15.1:0/1384095574 shutdown_connections 2024-04-26T01:48:20.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:20.109+0000 7f0bb1800700 1 -- 172.21.15.1:0/1384095574 wait complete. 2024-04-26T01:48:20.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:20 smithi001 ceph-mon[27822]: pgmap v349: 97 pgs: 97 active+clean; 635 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-04-26T01:48:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:20 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1384095574' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:48:20.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:20 smithi097 ceph-mon[32853]: pgmap v349: 97 pgs: 97 active+clean; 635 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-04-26T01:48:20.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:20 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1384095574' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:48:20.799 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T01:48:20.799 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:48:20.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:20 smithi173 ceph-mon[32553]: pgmap v349: 97 pgs: 97 active+clean; 635 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-04-26T01:48:20.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:20 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1384095574' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:48:21.227 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:21.716 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:21 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-26T01:48:21.716 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:21 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-26T01:48:21.716 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:21 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-26T01:48:21.716 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:21 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-26T01:48:21.716 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:21 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]: dispatch 2024-04-26T01:48:21.716 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:21 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-26T01:48:21.717 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:21 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-04-26T01:48:21.717 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:21 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-26T01:48:21.717 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:21 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-26T01:48:21.717 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:21 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-04-26T01:48:21.717 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:21 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:21.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:21 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-26T01:48:21.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:21 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-26T01:48:21.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:21 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-26T01:48:21.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:21 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-26T01:48:21.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:21 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]: dispatch 2024-04-26T01:48:21.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:21 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-26T01:48:21.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:21 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-04-26T01:48:21.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:21 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-26T01:48:21.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:21 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-26T01:48:21.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:21 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-04-26T01:48:21.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:21 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:21.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:21 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-26T01:48:21.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:21 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-26T01:48:21.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:21 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-26T01:48:21.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:21 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-26T01:48:21.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:21 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]: dispatch 2024-04-26T01:48:21.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:21 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-26T01:48:21.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:21 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-04-26T01:48:21.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:21 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-26T01:48:21.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:21 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-26T01:48:21.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:21 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-04-26T01:48:21.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:21 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:22 smithi097 ceph-mon[32853]: pgmap v350: 97 pgs: 97 active+clean; 635 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-04-26T01:48:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:22 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-26T01:48:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:22 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-26T01:48:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:22 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-26T01:48:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:22 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-26T01:48:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:22 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]': finished 2024-04-26T01:48:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:22 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-26T01:48:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:22 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-04-26T01:48:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:22 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-26T01:48:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:22 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-26T01:48:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:22 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-04-26T01:48:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:22 smithi097 ceph-mon[32853]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T01:48:22.785 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:22 smithi001 ceph-mon[27822]: pgmap v350: 97 pgs: 97 active+clean; 635 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-04-26T01:48:22.785 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:22 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-26T01:48:22.785 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:22 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-26T01:48:22.785 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:22 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-26T01:48:22.785 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:22 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-26T01:48:22.785 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:22 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]': finished 2024-04-26T01:48:22.785 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:22 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-26T01:48:22.785 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:22 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-04-26T01:48:22.785 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:22 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-26T01:48:22.786 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:22 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-26T01:48:22.786 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:22 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-04-26T01:48:22.786 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:22 smithi001 ceph-mon[27822]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T01:48:22.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:22 smithi173 ceph-mon[32553]: pgmap v350: 97 pgs: 97 active+clean; 635 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-04-26T01:48:22.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:22 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-26T01:48:22.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:22 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-26T01:48:22.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:22 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-26T01:48:22.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:22 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-26T01:48:22.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:22 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]': finished 2024-04-26T01:48:22.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:22 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-26T01:48:22.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:22 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-04-26T01:48:22.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:22 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-26T01:48:22.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:22 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-26T01:48:22.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:22 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-04-26T01:48:22.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:22 smithi173 ceph-mon[32553]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T01:48:23.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.338+0000 7f3c5a600700 1 -- 172.21.15.1:0/514609623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c5406a750 msgr2=0x7f3c5406abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:23.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.338+0000 7f3c5a600700 1 --2- 172.21.15.1:0/514609623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c5406a750 0x7f3c5406abb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009990 tx=0x7f3c3c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:48:23.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.339+0000 7f3c5a600700 1 -- 172.21.15.1:0/514609623 shutdown_connections 2024-04-26T01:48:23.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.339+0000 7f3c5a600700 1 --2- 172.21.15.1:0/514609623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c5406b0f0 0x7f3c5410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:23.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.339+0000 7f3c5a600700 1 --2- 172.21.15.1:0/514609623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c54108a30 0x7f3c54108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:23.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.339+0000 7f3c5a600700 1 --2- 172.21.15.1:0/514609623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c5406a750 0x7f3c5406abb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:23.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.339+0000 7f3c5a600700 1 -- 172.21.15.1:0/514609623 >> 172.21.15.1:0/514609623 conn(0x7f3c540788f0 msgr2=0x7f3c54078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:23.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.339+0000 7f3c5a600700 1 -- 172.21.15.1:0/514609623 shutdown_connections 2024-04-26T01:48:23.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.339+0000 7f3c5a600700 1 -- 172.21.15.1:0/514609623 wait complete. 2024-04-26T01:48:23.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.340+0000 7f3c5a600700 1 Processor -- start 2024-04-26T01:48:23.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.340+0000 7f3c5a600700 1 -- start start 2024-04-26T01:48:23.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.341+0000 7f3c5a600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c5406a750 0x7f3c54199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:23.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.341+0000 7f3c5a600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c5406b0f0 0x7f3c54199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:23.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.341+0000 7f3c5a600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c54108a30 0x7f3c5419d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:23.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.341+0000 7f3c5a600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3c54112c20 con 0x7f3c5406b0f0 2024-04-26T01:48:23.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.341+0000 7f3c5a600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3c54112aa0 con 0x7f3c54108a30 2024-04-26T01:48:23.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.341+0000 7f3c5a600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3c54112da0 con 0x7f3c5406a750 2024-04-26T01:48:23.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.341+0000 7f3c53400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c5406a750 0x7f3c54199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:23.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.341+0000 7f3c53400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c5406a750 0x7f3c54199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:46758/0 (socket says 172.21.15.1:46758) 2024-04-26T01:48:23.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.341+0000 7f3c53400700 1 -- 172.21.15.1:0/2606085169 learned_addr learned my addr 172.21.15.1:0/2606085169 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:48:23.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.341+0000 7f3c4be00700 1 --2- 172.21.15.1:0/2606085169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c5406b0f0 0x7f3c54199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:23.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.341+0000 7f3c53e00700 1 --2- 172.21.15.1:0/2606085169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c54108a30 0x7f3c5419d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:23.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.342+0000 7f3c53400700 1 -- 172.21.15.1:0/2606085169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c54108a30 msgr2=0x7f3c5419d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:23.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.342+0000 7f3c53400700 1 --2- 172.21.15.1:0/2606085169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c54108a30 0x7f3c5419d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:23.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.342+0000 7f3c53400700 1 -- 172.21.15.1:0/2606085169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c5406b0f0 msgr2=0x7f3c54199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:23.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.342+0000 7f3c53400700 1 --2- 172.21.15.1:0/2606085169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c5406b0f0 0x7f3c54199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:23.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.342+0000 7f3c53400700 1 -- 172.21.15.1:0/2606085169 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5419e110 con 0x7f3c5406a750 2024-04-26T01:48:23.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.342+0000 7f3c53400700 1 --2- 172.21.15.1:0/2606085169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c5406a750 0x7f3c54199150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400e9a0 tx=0x7f3c4400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:23.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.342+0000 7f3c50c00700 1 -- 172.21.15.1:0/2606085169 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4400ca10 con 0x7f3c5406a750 2024-04-26T01:48:23.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.342+0000 7f3c50c00700 1 -- 172.21.15.1:0/2606085169 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c4400cbb0 con 0x7f3c5406a750 2024-04-26T01:48:23.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.342+0000 7f3c5a600700 1 -- 172.21.15.1:0/2606085169 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c5419e400 con 0x7f3c5406a750 2024-04-26T01:48:23.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.342+0000 7f3c50c00700 1 -- 172.21.15.1:0/2606085169 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44018680 con 0x7f3c5406a750 2024-04-26T01:48:23.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.342+0000 7f3c5a600700 1 -- 172.21.15.1:0/2606085169 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c541a5c80 con 0x7f3c5406a750 2024-04-26T01:48:23.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.343+0000 7f3c5a600700 1 -- 172.21.15.1:0/2606085169 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c54069730 con 0x7f3c5406a750 2024-04-26T01:48:23.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.345+0000 7f3c50c00700 1 -- 172.21.15.1:0/2606085169 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f3c44010430 con 0x7f3c5406a750 2024-04-26T01:48:23.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.347+0000 7f3c50c00700 1 --2- 172.21.15.1:0/2606085169 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f3c380a3d50 0x7f3c380a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:23.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.347+0000 7f3c50c00700 1 -- 172.21.15.1:0/2606085169 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f3c440d06a0 con 0x7f3c5406a750 2024-04-26T01:48:23.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.347+0000 7f3c4be00700 1 --2- 172.21.15.1:0/2606085169 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f3c380a3d50 0x7f3c380a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:23.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.348+0000 7f3c4be00700 1 --2- 172.21.15.1:0/2606085169 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f3c380a3d50 0x7f3c380a6210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3c54101780 tx=0x7f3c3c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:23.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.351+0000 7f3c50c00700 1 -- 172.21.15.1:0/2606085169 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c4409cea0 con 0x7f3c5406a750 2024-04-26T01:48:23.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.582+0000 7f3c5a600700 1 -- 172.21.15.1:0/2606085169 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3c54062e80 con 0x7f3c380a3d50 2024-04-26T01:48:23.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.600+0000 7f3c50c00700 1 -- 172.21.15.1:0/2606085169 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258664 (secure 0 0 0) 0x7f3c3c067280 con 0x7f3c380a3d50 2024-04-26T01:48:23.599 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:23 smithi001 ceph-mon[27822]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T01:48:23.599 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:23 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:48:23.599 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:23 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:48:23.599 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:23 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:48:23.599 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:23 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:48:23.600 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:48:23.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.609+0000 7f3c5a600700 1 -- 172.21.15.1:0/2606085169 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f3c380a3d50 msgr2=0x7f3c380a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:23.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.609+0000 7f3c5a600700 1 --2- 172.21.15.1:0/2606085169 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f3c380a3d50 0x7f3c380a6210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3c54101780 tx=0x7f3c3c005fd0 comp rx=0 tx=0).stop 2024-04-26T01:48:23.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.609+0000 7f3c5a600700 1 -- 172.21.15.1:0/2606085169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c5406a750 msgr2=0x7f3c54199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:23.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.609+0000 7f3c5a600700 1 --2- 172.21.15.1:0/2606085169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c5406a750 0x7f3c54199150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400e9a0 tx=0x7f3c4400ecb0 comp rx=0 tx=0).stop 2024-04-26T01:48:23.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.609+0000 7f3c5a600700 1 -- 172.21.15.1:0/2606085169 shutdown_connections 2024-04-26T01:48:23.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.609+0000 7f3c5a600700 1 --2- 172.21.15.1:0/2606085169 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f3c380a3d50 0x7f3c380a6210 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:23.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.609+0000 7f3c5a600700 1 --2- 172.21.15.1:0/2606085169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c54108a30 0x7f3c5419d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:23.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.609+0000 7f3c5a600700 1 --2- 172.21.15.1:0/2606085169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c5406a750 0x7f3c54199150 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:23.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.609+0000 7f3c5a600700 1 --2- 172.21.15.1:0/2606085169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c5406b0f0 0x7f3c54199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:23.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.609+0000 7f3c5a600700 1 -- 172.21.15.1:0/2606085169 >> 172.21.15.1:0/2606085169 conn(0x7f3c540788f0 msgr2=0x7f3c540ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:23.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.610+0000 7f3c5a600700 1 -- 172.21.15.1:0/2606085169 shutdown_connections 2024-04-26T01:48:23.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:23.610+0000 7f3c5a600700 1 -- 172.21.15.1:0/2606085169 wait complete. 2024-04-26T01:48:23.610 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-04-26T01:48:23.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:23 smithi097 ceph-mon[32853]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T01:48:23.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:23 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:48:23.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:23 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:48:23.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:23 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:48:23.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:23 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:48:23.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:23 smithi173 ceph-mon[32553]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T01:48:23.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:23 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:48:23.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:23 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:48:23.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:23 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:48:23.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:23 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:48:24.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:24 smithi001 ceph-mon[27822]: pgmap v353: 97 pgs: 1 activating+degraded, 2 peering, 94 active+clean; 640 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 7 op/s; 8/192 objects degraded (4.167%); 0 B/s, 0 objects/s recovering 2024-04-26T01:48:24.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:24 smithi001 ceph-mon[27822]: Health check failed: Degraded data redundancy: 8/192 objects degraded (4.167%), 1 pg degraded (PG_DEGRADED) 2024-04-26T01:48:24.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:24 smithi001 ceph-mon[27822]: from='client.24661 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:48:24.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:24 smithi097 ceph-mon[32853]: pgmap v353: 97 pgs: 1 activating+degraded, 2 peering, 94 active+clean; 640 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 7 op/s; 8/192 objects degraded (4.167%); 0 B/s, 0 objects/s recovering 2024-04-26T01:48:24.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:24 smithi097 ceph-mon[32853]: Health check failed: Degraded data redundancy: 8/192 objects degraded (4.167%), 1 pg degraded (PG_DEGRADED) 2024-04-26T01:48:24.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:24 smithi097 ceph-mon[32853]: from='client.24661 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:48:24.745 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-04-26T01:48:22.818950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654950,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":349,"num_read_kb":296,"num_write":346,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358684,"kb_used_data":13672,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583140,"statfs":{"total":1151940427776,"available":1151573135360,"internally_reserved":0,"allocated":14000128,"data_stored":7674014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":40,"num_write":8,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003224"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.131645+0000","last_change":"2024-04-26T01:47:27.140418+0000","last_active":"2024-04-26T01:47:30.131645+0000","last_peered":"2024-04-26T01:47:30.131645+0000","last_clean":"2024-04-26T01:47:30.131645+0000","last_became_active":"2024-04-26T01:47:27.140017+0000","last_became_peered":"2024-04-26T01:47:27.140017+0000","last_unstale":"2024-04-26T01:47:30.131645+0000","last_undegraded":"2024-04-26T01:47:30.131645+0000","last_fullsized":"2024-04-26T01:47:30.131645+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:46:08.621222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.136104+0000","last_change":"2024-04-26T01:47:27.138746+0000","last_active":"2024-04-26T01:47:30.136104+0000","last_peered":"2024-04-26T01:47:30.136104+0000","last_clean":"2024-04-26T01:47:30.136104+0000","last_became_active":"2024-04-26T01:47:27.136163+0000","last_became_peered":"2024-04-26T01:47:27.136163+0000","last_unstale":"2024-04-26T01:47:30.136104+0000","last_undegraded":"2024-04-26T01:47:30.136104+0000","last_fullsized":"2024-04-26T01:47:30.136104+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:01:36.195332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.135031+0000","last_change":"2024-04-26T01:47:27.127017+0000","last_active":"2024-04-26T01:47:30.135031+0000","last_peered":"2024-04-26T01:47:30.135031+0000","last_clean":"2024-04-26T01:47:30.135031+0000","last_became_active":"2024-04-26T01:47:27.126610+0000","last_became_peered":"2024-04-26T01:47:27.126610+0000","last_unstale":"2024-04-26T01:47:30.135031+0000","last_undegraded":"2024-04-26T01:47:30.135031+0000","last_fullsized":"2024-04-26T01:47:30.135031+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:05:12.784486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.130541+0000","last_change":"2024-04-26T01:47:27.117640+0000","last_active":"2024-04-26T01:47:30.130541+0000","last_peered":"2024-04-26T01:47:30.130541+0000","last_clean":"2024-04-26T01:47:30.130541+0000","last_became_active":"2024-04-26T01:47:27.117266+0000","last_became_peered":"2024-04-26T01:47:27.117266+0000","last_unstale":"2024-04-26T01:47:30.130541+0000","last_undegraded":"2024-04-26T01:47:30.130541+0000","last_fullsized":"2024-04-26T01:47:30.130541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:26:57.824560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396755+0000","last_change":"2024-04-26T01:47:27.128185+0000","last_active":"2024-04-26T01:48:22.396755+0000","last_peered":"2024-04-26T01:48:22.396755+0000","last_clean":"2024-04-26T01:48:22.396755+0000","last_became_active":"2024-04-26T01:47:27.127794+0000","last_became_peered":"2024-04-26T01:47:27.127794+0000","last_unstale":"2024-04-26T01:48:22.396755+0000","last_undegraded":"2024-04-26T01:48:22.396755+0000","last_fullsized":"2024-04-26T01:48:22.396755+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:09:52.774007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"peering","last_fresh":"2024-04-26T01:47:30.125739+0000","last_change":"2024-04-26T01:48:21.383800+0000","last_active":"2024-04-26T01:48:21.383800+0000","last_peered":"2024-04-26T01:47:30.125739+0000","last_clean":"2024-04-26T01:47:30.125739+0000","last_became_active":"2024-04-26T01:47:27.121390+0000","last_became_peered":"2024-04-26T01:47:27.121390+0000","last_unstale":"2024-04-26T01:47:30.125739+0000","last_undegraded":"2024-04-26T01:47:30.125739+0000","last_fullsized":"2024-04-26T01:47:30.125739+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,7,9],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:48:21.386686+0000","last_change":"2024-04-26T01:47:27.123786+0000","last_active":"2024-04-26T01:48:21.386686+0000","last_peered":"2024-04-26T01:48:21.386686+0000","last_clean":"2024-04-26T01:48:21.386686+0000","last_became_active":"2024-04-26T01:47:27.123473+0000","last_became_peered":"2024-04-26T01:47:27.123473+0000","last_unstale":"2024-04-26T01:48:21.386686+0000","last_undegraded":"2024-04-26T01:48:21.386686+0000","last_fullsized":"2024-04-26T01:48:21.386686+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:48:55.031147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404450+0000","last_change":"2024-04-26T01:47:27.118756+0000","last_active":"2024-04-26T01:48:22.404450+0000","last_peered":"2024-04-26T01:48:22.404450+0000","last_clean":"2024-04-26T01:48:22.404450+0000","last_became_active":"2024-04-26T01:47:27.118406+0000","last_became_peered":"2024-04-26T01:47:27.118406+0000","last_unstale":"2024-04-26T01:48:22.404450+0000","last_undegraded":"2024-04-26T01:48:22.404450+0000","last_fullsized":"2024-04-26T01:48:22.404450+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:05.793682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398754+0000","last_change":"2024-04-26T01:47:27.117622+0000","last_active":"2024-04-26T01:48:22.398754+0000","last_peered":"2024-04-26T01:48:22.398754+0000","last_clean":"2024-04-26T01:48:22.398754+0000","last_became_active":"2024-04-26T01:47:27.115206+0000","last_became_peered":"2024-04-26T01:47:27.115206+0000","last_unstale":"2024-04-26T01:48:22.398754+0000","last_undegraded":"2024-04-26T01:48:22.398754+0000","last_fullsized":"2024-04-26T01:48:22.398754+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:13:17.866630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:48:21.383393+0000","last_change":"2024-04-26T01:47:27.119524+0000","last_active":"2024-04-26T01:48:21.383393+0000","last_peered":"2024-04-26T01:48:21.383393+0000","last_clean":"2024-04-26T01:48:21.383393+0000","last_became_active":"2024-04-26T01:47:27.119008+0000","last_became_peered":"2024-04-26T01:47:27.119008+0000","last_unstale":"2024-04-26T01:48:21.383393+0000","last_undegraded":"2024-04-26T01:48:21.383393+0000","last_fullsized":"2024-04-26T01:48:21.383393+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:37:31.894486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:48:21.385833+0000","last_change":"2024-04-26T01:47:27.122361+0000","last_active":"2024-04-26T01:48:21.385833+0000","last_peered":"2024-04-26T01:48:21.385833+0000","last_clean":"2024-04-26T01:48:21.385833+0000","last_became_active":"2024-04-26T01:47:27.121748+0000","last_became_peered":"2024-04-26T01:47:27.121748+0000","last_unstale":"2024-04-26T01:48:21.385833+0000","last_undegraded":"2024-04-26T01:48:21.385833+0000","last_fullsized":"2024-04-26T01:48:21.385833+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:18.680909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.130338+0000","last_change":"2024-04-26T01:47:27.116833+0000","last_active":"2024-04-26T01:47:30.130338+0000","last_peered":"2024-04-26T01:47:30.130338+0000","last_clean":"2024-04-26T01:47:30.130338+0000","last_became_active":"2024-04-26T01:47:27.116508+0000","last_became_peered":"2024-04-26T01:47:27.116508+0000","last_unstale":"2024-04-26T01:47:30.130338+0000","last_undegraded":"2024-04-26T01:47:30.130338+0000","last_fullsized":"2024-04-26T01:47:30.130338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:27:45.864521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.138627+0000","last_change":"2024-04-26T01:47:27.125342+0000","last_active":"2024-04-26T01:47:30.138627+0000","last_peered":"2024-04-26T01:47:30.138627+0000","last_clean":"2024-04-26T01:47:30.138627+0000","last_became_active":"2024-04-26T01:47:27.124862+0000","last_became_peered":"2024-04-26T01:47:27.124862+0000","last_unstale":"2024-04-26T01:47:30.138627+0000","last_undegraded":"2024-04-26T01:47:30.138627+0000","last_fullsized":"2024-04-26T01:47:30.138627+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:48.654711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.135645+0000","last_change":"2024-04-26T01:47:27.127365+0000","last_active":"2024-04-26T01:47:30.135645+0000","last_peered":"2024-04-26T01:47:30.135645+0000","last_clean":"2024-04-26T01:47:30.135645+0000","last_became_active":"2024-04-26T01:47:27.126868+0000","last_became_peered":"2024-04-26T01:47:27.126868+0000","last_unstale":"2024-04-26T01:47:30.135645+0000","last_undegraded":"2024-04-26T01:47:30.135645+0000","last_fullsized":"2024-04-26T01:47:30.135645+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:07:46.400333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.136629+0000","last_change":"2024-04-26T01:47:27.135584+0000","last_active":"2024-04-26T01:47:30.136629+0000","last_peered":"2024-04-26T01:47:30.136629+0000","last_clean":"2024-04-26T01:47:30.136629+0000","last_became_active":"2024-04-26T01:47:27.134849+0000","last_became_peered":"2024-04-26T01:47:27.134849+0000","last_unstale":"2024-04-26T01:47:30.136629+0000","last_undegraded":"2024-04-26T01:47:30.136629+0000","last_fullsized":"2024-04-26T01:47:30.136629+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:57:48.821595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398274+0000","last_change":"2024-04-26T01:47:27.136182+0000","last_active":"2024-04-26T01:48:22.398274+0000","last_peered":"2024-04-26T01:48:22.398274+0000","last_clean":"2024-04-26T01:48:22.398274+0000","last_became_active":"2024-04-26T01:47:27.135787+0000","last_became_peered":"2024-04-26T01:47:27.135787+0000","last_unstale":"2024-04-26T01:48:22.398274+0000","last_undegraded":"2024-04-26T01:48:22.398274+0000","last_fullsized":"2024-04-26T01:48:22.398274+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:38:14.275898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.131894+0000","last_change":"2024-04-26T01:47:27.140700+0000","last_active":"2024-04-26T01:47:30.131894+0000","last_peered":"2024-04-26T01:47:30.131894+0000","last_clean":"2024-04-26T01:47:30.131894+0000","last_became_active":"2024-04-26T01:47:27.140025+0000","last_became_peered":"2024-04-26T01:47:27.140025+0000","last_unstale":"2024-04-26T01:47:30.131894+0000","last_undegraded":"2024-04-26T01:47:30.131894+0000","last_fullsized":"2024-04-26T01:47:30.131894+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:50.465236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.127910+0000","last_change":"2024-04-26T01:47:27.117256+0000","last_active":"2024-04-26T01:47:30.127910+0000","last_peered":"2024-04-26T01:47:30.127910+0000","last_clean":"2024-04-26T01:47:30.127910+0000","last_became_active":"2024-04-26T01:47:27.116825+0000","last_became_peered":"2024-04-26T01:47:27.116825+0000","last_unstale":"2024-04-26T01:47:30.127910+0000","last_undegraded":"2024-04-26T01:47:30.127910+0000","last_fullsized":"2024-04-26T01:47:30.127910+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:23.313712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.127058+0000","last_change":"2024-04-26T01:47:27.130233+0000","last_active":"2024-04-26T01:47:30.127058+0000","last_peered":"2024-04-26T01:47:30.127058+0000","last_clean":"2024-04-26T01:47:30.127058+0000","last_became_active":"2024-04-26T01:47:27.129852+0000","last_became_peered":"2024-04-26T01:47:27.129852+0000","last_unstale":"2024-04-26T01:47:30.127058+0000","last_undegraded":"2024-04-26T01:47:30.127058+0000","last_fullsized":"2024-04-26T01:47:30.127058+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:49:01.407264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:47.238133+0000","last_change":"2024-04-26T01:47:23.068477+0000","last_active":"2024-04-26T01:47:47.238133+0000","last_peered":"2024-04-26T01:47:47.238133+0000","last_clean":"2024-04-26T01:47:47.238133+0000","last_became_active":"2024-04-26T01:47:23.068061+0000","last_became_peered":"2024-04-26T01:47:23.068061+0000","last_unstale":"2024-04-26T01:47:47.238133+0000","last_undegraded":"2024-04-26T01:47:47.238133+0000","last_fullsized":"2024-04-26T01:47:47.238133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:02.153724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.406110+0000","last_change":"2024-04-26T01:47:27.130385+0000","last_active":"2024-04-26T01:48:22.406110+0000","last_peered":"2024-04-26T01:48:22.406110+0000","last_clean":"2024-04-26T01:48:22.406110+0000","last_became_active":"2024-04-26T01:47:27.129912+0000","last_became_peered":"2024-04-26T01:47:27.129912+0000","last_unstale":"2024-04-26T01:48:22.406110+0000","last_undegraded":"2024-04-26T01:48:22.406110+0000","last_fullsized":"2024-04-26T01:48:22.406110+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:26:03.010544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.407897+0000","last_change":"2024-04-26T01:47:23.110806+0000","last_active":"2024-04-26T01:48:22.407897+0000","last_peered":"2024-04-26T01:48:22.407897+0000","last_clean":"2024-04-26T01:48:22.407897+0000","last_became_active":"2024-04-26T01:47:23.110441+0000","last_became_peered":"2024-04-26T01:47:23.110441+0000","last_unstale":"2024-04-26T01:48:22.407897+0000","last_undegraded":"2024-04-26T01:48:22.407897+0000","last_fullsized":"2024-04-26T01:48:22.407897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:39.239172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.136800+0000","last_change":"2024-04-26T01:47:27.138628+0000","last_active":"2024-04-26T01:47:30.136800+0000","last_peered":"2024-04-26T01:47:30.136800+0000","last_clean":"2024-04-26T01:47:30.136800+0000","last_became_active":"2024-04-26T01:47:27.130483+0000","last_became_peered":"2024-04-26T01:47:27.130483+0000","last_unstale":"2024-04-26T01:47:30.136800+0000","last_undegraded":"2024-04-26T01:47:30.136800+0000","last_fullsized":"2024-04-26T01:47:30.136800+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:40:41.242856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.397138+0000","last_change":"2024-04-26T01:47:23.060066+0000","last_active":"2024-04-26T01:48:22.397138+0000","last_peered":"2024-04-26T01:48:22.397138+0000","last_clean":"2024-04-26T01:48:22.397138+0000","last_became_active":"2024-04-26T01:47:23.059526+0000","last_became_peered":"2024-04-26T01:47:23.059526+0000","last_unstale":"2024-04-26T01:48:22.397138+0000","last_undegraded":"2024-04-26T01:48:22.397138+0000","last_fullsized":"2024-04-26T01:48:22.397138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:28:22.623695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398426+0000","last_change":"2024-04-26T01:47:27.128343+0000","last_active":"2024-04-26T01:48:22.398426+0000","last_peered":"2024-04-26T01:48:22.398426+0000","last_clean":"2024-04-26T01:48:22.398426+0000","last_became_active":"2024-04-26T01:47:27.127920+0000","last_became_peered":"2024-04-26T01:47:27.127920+0000","last_unstale":"2024-04-26T01:48:22.398426+0000","last_undegraded":"2024-04-26T01:48:22.398426+0000","last_fullsized":"2024-04-26T01:48:22.398426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:06.725033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396937+0000","last_change":"2024-04-26T01:47:23.057783+0000","last_active":"2024-04-26T01:48:22.396937+0000","last_peered":"2024-04-26T01:48:22.396937+0000","last_clean":"2024-04-26T01:48:22.396937+0000","last_became_active":"2024-04-26T01:47:23.057391+0000","last_became_peered":"2024-04-26T01:47:23.057391+0000","last_unstale":"2024-04-26T01:48:22.396937+0000","last_undegraded":"2024-04-26T01:48:22.396937+0000","last_fullsized":"2024-04-26T01:48:22.396937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:10:10.050787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.122666+0000","last_change":"2024-04-26T01:47:27.116458+0000","last_active":"2024-04-26T01:47:30.122666+0000","last_peered":"2024-04-26T01:47:30.122666+0000","last_clean":"2024-04-26T01:47:30.122666+0000","last_became_active":"2024-04-26T01:47:27.116084+0000","last_became_peered":"2024-04-26T01:47:27.116084+0000","last_unstale":"2024-04-26T01:47:30.122666+0000","last_undegraded":"2024-04-26T01:47:30.122666+0000","last_fullsized":"2024-04-26T01:47:30.122666+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:06:34.517478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:48:17.298494+0000","last_change":"2024-04-26T01:47:23.060657+0000","last_active":"2024-04-26T01:48:17.298494+0000","last_peered":"2024-04-26T01:48:17.298494+0000","last_clean":"2024-04-26T01:48:17.298494+0000","last_became_active":"2024-04-26T01:47:23.060294+0000","last_became_peered":"2024-04-26T01:47:23.060294+0000","last_unstale":"2024-04-26T01:48:17.298494+0000","last_undegraded":"2024-04-26T01:48:17.298494+0000","last_fullsized":"2024-04-26T01:48:17.298494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:33.979717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396644+0000","last_change":"2024-04-26T01:47:27.120587+0000","last_active":"2024-04-26T01:48:22.396644+0000","last_peered":"2024-04-26T01:48:22.396644+0000","last_clean":"2024-04-26T01:48:22.396644+0000","last_became_active":"2024-04-26T01:47:27.120236+0000","last_became_peered":"2024-04-26T01:47:27.120236+0000","last_unstale":"2024-04-26T01:48:22.396644+0000","last_undegraded":"2024-04-26T01:48:22.396644+0000","last_fullsized":"2024-04-26T01:48:22.396644+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:30:38.488741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:48:21.383114+0000","last_change":"2024-04-26T01:47:23.069395+0000","last_active":"2024-04-26T01:48:21.383114+0000","last_peered":"2024-04-26T01:48:21.383114+0000","last_clean":"2024-04-26T01:48:21.383114+0000","last_became_active":"2024-04-26T01:47:23.069102+0000","last_became_peered":"2024-04-26T01:47:23.069102+0000","last_unstale":"2024-04-26T01:48:21.383114+0000","last_undegraded":"2024-04-26T01:48:21.383114+0000","last_fullsized":"2024-04-26T01:48:21.383114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:45.162604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.122613+0000","last_change":"2024-04-26T01:47:27.123859+0000","last_active":"2024-04-26T01:47:30.122613+0000","last_peered":"2024-04-26T01:47:30.122613+0000","last_clean":"2024-04-26T01:47:30.122613+0000","last_became_active":"2024-04-26T01:47:27.123438+0000","last_became_peered":"2024-04-26T01:47:27.123438+0000","last_unstale":"2024-04-26T01:47:30.122613+0000","last_undegraded":"2024-04-26T01:47:30.122613+0000","last_fullsized":"2024-04-26T01:47:30.122613+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:33:53.542809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:47.236058+0000","last_change":"2024-04-26T01:47:23.071544+0000","last_active":"2024-04-26T01:47:47.236058+0000","last_peered":"2024-04-26T01:47:47.236058+0000","last_clean":"2024-04-26T01:47:47.236058+0000","last_became_active":"2024-04-26T01:47:23.071191+0000","last_became_peered":"2024-04-26T01:47:23.071191+0000","last_unstale":"2024-04-26T01:47:47.236058+0000","last_undegraded":"2024-04-26T01:47:47.236058+0000","last_fullsized":"2024-04-26T01:47:47.236058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:58.889561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404118+0000","last_change":"2024-04-26T01:47:27.131887+0000","last_active":"2024-04-26T01:48:22.404118+0000","last_peered":"2024-04-26T01:48:22.404118+0000","last_clean":"2024-04-26T01:48:22.404118+0000","last_became_active":"2024-04-26T01:47:27.127973+0000","last_became_peered":"2024-04-26T01:47:27.127973+0000","last_unstale":"2024-04-26T01:48:22.404118+0000","last_undegraded":"2024-04-26T01:48:22.404118+0000","last_fullsized":"2024-04-26T01:48:22.404118+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:54:35.305779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.123079+0000","last_change":"2024-04-26T01:47:23.061353+0000","last_active":"2024-04-26T01:47:30.123079+0000","last_peered":"2024-04-26T01:47:30.123079+0000","last_clean":"2024-04-26T01:47:30.123079+0000","last_became_active":"2024-04-26T01:47:23.061037+0000","last_became_peered":"2024-04-26T01:47:23.061037+0000","last_unstale":"2024-04-26T01:47:30.123079+0000","last_undegraded":"2024-04-26T01:47:30.123079+0000","last_fullsized":"2024-04-26T01:47:30.123079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:13:59.617969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.124106+0000","last_change":"2024-04-26T01:47:27.122303+0000","last_active":"2024-04-26T01:47:30.124106+0000","last_peered":"2024-04-26T01:47:30.124106+0000","last_clean":"2024-04-26T01:47:30.124106+0000","last_became_active":"2024-04-26T01:47:27.121919+0000","last_became_peered":"2024-04-26T01:47:27.121919+0000","last_unstale":"2024-04-26T01:47:30.124106+0000","last_undegraded":"2024-04-26T01:47:30.124106+0000","last_fullsized":"2024-04-26T01:47:30.124106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:21.676663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:48:21.383404+0000","last_change":"2024-04-26T01:47:23.057961+0000","last_active":"2024-04-26T01:48:21.383404+0000","last_peered":"2024-04-26T01:48:21.383404+0000","last_clean":"2024-04-26T01:48:21.383404+0000","last_became_active":"2024-04-26T01:47:23.057529+0000","last_became_peered":"2024-04-26T01:47:23.057529+0000","last_unstale":"2024-04-26T01:48:21.383404+0000","last_undegraded":"2024-04-26T01:48:21.383404+0000","last_fullsized":"2024-04-26T01:48:21.383404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:14.420265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398667+0000","last_change":"2024-04-26T01:47:27.120083+0000","last_active":"2024-04-26T01:48:22.398667+0000","last_peered":"2024-04-26T01:48:22.398667+0000","last_clean":"2024-04-26T01:48:22.398667+0000","last_became_active":"2024-04-26T01:47:27.118410+0000","last_became_peered":"2024-04-26T01:47:27.118410+0000","last_unstale":"2024-04-26T01:48:22.398667+0000","last_undegraded":"2024-04-26T01:48:22.398667+0000","last_fullsized":"2024-04-26T01:48:22.398667+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:28.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:48:21.386848+0000","last_change":"2024-04-26T01:47:23.059221+0000","last_active":"2024-04-26T01:48:21.386848+0000","last_peered":"2024-04-26T01:48:21.386848+0000","last_clean":"2024-04-26T01:48:21.386848+0000","last_became_active":"2024-04-26T01:47:23.058881+0000","last_became_peered":"2024-04-26T01:47:23.058881+0000","last_unstale":"2024-04-26T01:48:21.386848+0000","last_undegraded":"2024-04-26T01:48:21.386848+0000","last_fullsized":"2024-04-26T01:48:21.386848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:30.433696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.139281+0000","last_change":"2024-04-26T01:47:27.136763+0000","last_active":"2024-04-26T01:47:30.139281+0000","last_peered":"2024-04-26T01:47:30.139281+0000","last_clean":"2024-04-26T01:47:30.139281+0000","last_became_active":"2024-04-26T01:47:27.133642+0000","last_became_peered":"2024-04-26T01:47:27.133642+0000","last_unstale":"2024-04-26T01:47:30.139281+0000","last_undegraded":"2024-04-26T01:47:30.139281+0000","last_fullsized":"2024-04-26T01:47:30.139281+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:05.901139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:46.214375+0000","last_change":"2024-04-26T01:47:23.108883+0000","last_active":"2024-04-26T01:47:46.214375+0000","last_peered":"2024-04-26T01:47:46.214375+0000","last_clean":"2024-04-26T01:47:46.214375+0000","last_became_active":"2024-04-26T01:47:23.108499+0000","last_became_peered":"2024-04-26T01:47:23.108499+0000","last_unstale":"2024-04-26T01:47:46.214375+0000","last_undegraded":"2024-04-26T01:47:46.214375+0000","last_fullsized":"2024-04-26T01:47:46.214375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:14:17.555416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.410651+0000","last_change":"2024-04-26T01:48:22.410651+0000","last_active":"2024-04-26T01:48:22.410651+0000","last_peered":"2024-04-26T01:48:22.410651+0000","last_clean":"2024-04-26T01:48:22.410651+0000","last_became_active":"2024-04-26T01:48:22.410359+0000","last_became_peered":"2024-04-26T01:48:22.410359+0000","last_unstale":"2024-04-26T01:48:22.410651+0000","last_undegraded":"2024-04-26T01:48:22.410651+0000","last_fullsized":"2024-04-26T01:48:22.410651+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:53:20.686915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'29","reported_seq":126,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:48:21.383210+0000","last_change":"2024-04-26T01:47:23.054893+0000","last_active":"2024-04-26T01:48:21.383210+0000","last_peered":"2024-04-26T01:48:21.383210+0000","last_clean":"2024-04-26T01:48:21.383210+0000","last_became_active":"2024-04-26T01:47:23.054558+0000","last_became_peered":"2024-04-26T01:47:23.054558+0000","last_unstale":"2024-04-26T01:48:21.383210+0000","last_undegraded":"2024-04-26T01:48:21.383210+0000","last_fullsized":"2024-04-26T01:48:21.383210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:59.356841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60347,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":73,"num_read_kb":95,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396887+0000","last_change":"2024-04-26T01:47:27.116934+0000","last_active":"2024-04-26T01:48:22.396887+0000","last_peered":"2024-04-26T01:48:22.396887+0000","last_clean":"2024-04-26T01:48:22.396887+0000","last_became_active":"2024-04-26T01:47:27.116527+0000","last_became_peered":"2024-04-26T01:47:27.116527+0000","last_unstale":"2024-04-26T01:48:22.396887+0000","last_undegraded":"2024-04-26T01:48:22.396887+0000","last_fullsized":"2024-04-26T01:48:22.396887+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:09:45.820533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:47.231312+0000","last_change":"2024-04-26T01:47:23.059032+0000","last_active":"2024-04-26T01:47:47.231312+0000","last_peered":"2024-04-26T01:47:47.231312+0000","last_clean":"2024-04-26T01:47:47.231312+0000","last_became_active":"2024-04-26T01:47:23.057992+0000","last_became_peered":"2024-04-26T01:47:23.057992+0000","last_unstale":"2024-04-26T01:47:47.231312+0000","last_undegraded":"2024-04-26T01:47:47.231312+0000","last_fullsized":"2024-04-26T01:47:47.231312+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:46:53.164925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.138112+0000","last_change":"2024-04-26T01:47:27.134574+0000","last_active":"2024-04-26T01:47:30.138112+0000","last_peered":"2024-04-26T01:47:30.138112+0000","last_clean":"2024-04-26T01:47:30.138112+0000","last_became_active":"2024-04-26T01:47:27.134184+0000","last_became_peered":"2024-04-26T01:47:27.134184+0000","last_unstale":"2024-04-26T01:47:30.138112+0000","last_undegraded":"2024-04-26T01:47:30.138112+0000","last_fullsized":"2024-04-26T01:47:30.138112+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:42:26.428014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:46.214438+0000","last_change":"2024-04-26T01:47:23.062649+0000","last_active":"2024-04-26T01:47:46.214438+0000","last_peered":"2024-04-26T01:47:46.214438+0000","last_clean":"2024-04-26T01:47:46.214438+0000","last_became_active":"2024-04-26T01:47:23.062282+0000","last_became_peered":"2024-04-26T01:47:23.062282+0000","last_unstale":"2024-04-26T01:47:46.214438+0000","last_undegraded":"2024-04-26T01:47:46.214438+0000","last_fullsized":"2024-04-26T01:47:46.214438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:38.699105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:48:21.387142+0000","last_change":"2024-04-26T01:47:27.124720+0000","last_active":"2024-04-26T01:48:21.387142+0000","last_peered":"2024-04-26T01:48:21.387142+0000","last_clean":"2024-04-26T01:48:21.387142+0000","last_became_active":"2024-04-26T01:47:27.124378+0000","last_became_peered":"2024-04-26T01:47:27.124378+0000","last_unstale":"2024-04-26T01:48:21.387142+0000","last_undegraded":"2024-04-26T01:48:21.387142+0000","last_fullsized":"2024-04-26T01:48:21.387142+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:45.462741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:48:17.314304+0000","last_change":"2024-04-26T01:47:23.056305+0000","last_active":"2024-04-26T01:48:17.314304+0000","last_peered":"2024-04-26T01:48:17.314304+0000","last_clean":"2024-04-26T01:48:17.314304+0000","last_became_active":"2024-04-26T01:47:23.055905+0000","last_became_peered":"2024-04-26T01:47:23.055905+0000","last_unstale":"2024-04-26T01:48:17.314304+0000","last_undegraded":"2024-04-26T01:48:17.314304+0000","last_fullsized":"2024-04-26T01:48:17.314304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:00:14.609673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:48:21.386180+0000","last_change":"2024-04-26T01:47:27.119880+0000","last_active":"2024-04-26T01:48:21.386180+0000","last_peered":"2024-04-26T01:48:21.386180+0000","last_clean":"2024-04-26T01:48:21.386180+0000","last_became_active":"2024-04-26T01:47:27.119483+0000","last_became_peered":"2024-04-26T01:47:27.119483+0000","last_unstale":"2024-04-26T01:48:21.386180+0000","last_undegraded":"2024-04-26T01:48:21.386180+0000","last_fullsized":"2024-04-26T01:48:21.386180+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:15.523150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":42,"reported_epoch":92,"state":"activating+degraded","last_fresh":"2024-04-26T01:48:22.397136+0000","last_change":"2024-04-26T01:48:22.410077+0000","last_active":"2024-04-26T01:48:21.388233+0000","last_peered":"2024-04-26T01:47:50.255243+0000","last_clean":"2024-04-26T01:47:50.255243+0000","last_became_active":"2024-04-26T01:47:23.109192+0000","last_became_peered":"2024-04-26T01:47:23.109192+0000","last_unstale":"2024-04-26T01:48:22.397136+0000","last_undegraded":"2024-04-26T01:48:22.397136+0000","last_fullsized":"2024-04-26T01:48:22.397136+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":["8","0","2"],"object_location_counts":[{"shards":"0,2,8","objects":8}],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.122583+0000","last_change":"2024-04-26T01:47:27.113508+0000","last_active":"2024-04-26T01:47:30.122583+0000","last_peered":"2024-04-26T01:47:30.122583+0000","last_clean":"2024-04-26T01:47:30.122583+0000","last_became_active":"2024-04-26T01:47:27.113066+0000","last_became_peered":"2024-04-26T01:47:27.113066+0000","last_unstale":"2024-04-26T01:47:30.122583+0000","last_undegraded":"2024-04-26T01:47:30.122583+0000","last_fullsized":"2024-04-26T01:47:30.122583+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:37.525486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.397062+0000","last_change":"2024-04-26T01:47:23.059644+0000","last_active":"2024-04-26T01:48:22.397062+0000","last_peered":"2024-04-26T01:48:22.397062+0000","last_clean":"2024-04-26T01:48:22.397062+0000","last_became_active":"2024-04-26T01:47:23.058966+0000","last_became_peered":"2024-04-26T01:47:23.058966+0000","last_unstale":"2024-04-26T01:48:22.397062+0000","last_undegraded":"2024-04-26T01:48:22.397062+0000","last_fullsized":"2024-04-26T01:48:22.397062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:57:36.475935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.131234+0000","last_change":"2024-04-26T01:47:27.112759+0000","last_active":"2024-04-26T01:47:30.131234+0000","last_peered":"2024-04-26T01:47:30.131234+0000","last_clean":"2024-04-26T01:47:30.131234+0000","last_became_active":"2024-04-26T01:47:27.112324+0000","last_became_peered":"2024-04-26T01:47:27.112324+0000","last_unstale":"2024-04-26T01:47:30.131234+0000","last_undegraded":"2024-04-26T01:47:30.131234+0000","last_fullsized":"2024-04-26T01:47:30.131234+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:51:23.092304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.125233+0000","last_change":"2024-04-26T01:47:23.110360+0000","last_active":"2024-04-26T01:47:30.125233+0000","last_peered":"2024-04-26T01:47:30.125233+0000","last_clean":"2024-04-26T01:47:30.125233+0000","last_became_active":"2024-04-26T01:47:23.109744+0000","last_became_peered":"2024-04-26T01:47:23.109744+0000","last_unstale":"2024-04-26T01:47:30.125233+0000","last_undegraded":"2024-04-26T01:47:30.125233+0000","last_fullsized":"2024-04-26T01:47:30.125233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:40:13.658396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:48:21.383218+0000","last_change":"2024-04-26T01:47:27.125399+0000","last_active":"2024-04-26T01:48:21.383218+0000","last_peered":"2024-04-26T01:48:21.383218+0000","last_clean":"2024-04-26T01:48:21.383218+0000","last_became_active":"2024-04-26T01:47:27.124871+0000","last_became_peered":"2024-04-26T01:47:27.124871+0000","last_unstale":"2024-04-26T01:48:21.383218+0000","last_undegraded":"2024-04-26T01:48:21.383218+0000","last_fullsized":"2024-04-26T01:48:21.383218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:28:48.285206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:48:21.385866+0000","last_change":"2024-04-26T01:47:23.067862+0000","last_active":"2024-04-26T01:48:21.385866+0000","last_peered":"2024-04-26T01:48:21.385866+0000","last_clean":"2024-04-26T01:48:21.385866+0000","last_became_active":"2024-04-26T01:47:23.067526+0000","last_became_peered":"2024-04-26T01:47:23.067526+0000","last_unstale":"2024-04-26T01:48:21.385866+0000","last_undegraded":"2024-04-26T01:48:21.385866+0000","last_fullsized":"2024-04-26T01:48:21.385866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:22.668405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"peering","last_fresh":"2024-04-26T01:47:30.125610+0000","last_change":"2024-04-26T01:48:21.383647+0000","last_active":"2024-04-26T01:48:21.383647+0000","last_peered":"2024-04-26T01:47:30.125610+0000","last_clean":"2024-04-26T01:47:30.125610+0000","last_became_active":"2024-04-26T01:47:27.126084+0000","last_became_peered":"2024-04-26T01:47:27.126084+0000","last_unstale":"2024-04-26T01:47:30.125610+0000","last_undegraded":"2024-04-26T01:47:30.125610+0000","last_fullsized":"2024-04-26T01:47:30.125610+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4,9],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.123294+0000","last_change":"2024-04-26T01:47:27.118307+0000","last_active":"2024-04-26T01:47:30.123294+0000","last_peered":"2024-04-26T01:47:30.123294+0000","last_clean":"2024-04-26T01:47:30.123294+0000","last_became_active":"2024-04-26T01:47:27.117986+0000","last_became_peered":"2024-04-26T01:47:27.117986+0000","last_unstale":"2024-04-26T01:47:30.123294+0000","last_undegraded":"2024-04-26T01:47:30.123294+0000","last_fullsized":"2024-04-26T01:47:30.123294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:40.667143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:48:19.316729+0000","last_change":"2024-04-26T01:47:23.109973+0000","last_active":"2024-04-26T01:48:19.316729+0000","last_peered":"2024-04-26T01:48:19.316729+0000","last_clean":"2024-04-26T01:48:19.316729+0000","last_became_active":"2024-04-26T01:47:23.109446+0000","last_became_peered":"2024-04-26T01:47:23.109446+0000","last_unstale":"2024-04-26T01:48:19.316729+0000","last_undegraded":"2024-04-26T01:48:19.316729+0000","last_fullsized":"2024-04-26T01:48:19.316729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:49:00.684164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":31,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.408237+0000","last_change":"2024-04-26T01:47:27.131502+0000","last_active":"2024-04-26T01:48:22.408237+0000","last_peered":"2024-04-26T01:48:22.408237+0000","last_clean":"2024-04-26T01:48:22.408237+0000","last_became_active":"2024-04-26T01:47:27.127697+0000","last_became_peered":"2024-04-26T01:47:27.127697+0000","last_unstale":"2024-04-26T01:48:22.408237+0000","last_undegraded":"2024-04-26T01:48:22.408237+0000","last_fullsized":"2024-04-26T01:48:22.408237+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:39:15.458189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:50.254761+0000","last_change":"2024-04-26T01:47:23.066596+0000","last_active":"2024-04-26T01:47:50.254761+0000","last_peered":"2024-04-26T01:47:50.254761+0000","last_clean":"2024-04-26T01:47:50.254761+0000","last_became_active":"2024-04-26T01:47:23.066216+0000","last_became_peered":"2024-04-26T01:47:23.066216+0000","last_unstale":"2024-04-26T01:47:50.254761+0000","last_undegraded":"2024-04-26T01:47:50.254761+0000","last_fullsized":"2024-04-26T01:47:50.254761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:08:31.544831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":357,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:48:21.386363+0000","last_change":"2024-04-26T01:45:54.834501+0000","last_active":"2024-04-26T01:48:21.386363+0000","last_peered":"2024-04-26T01:48:21.386363+0000","last_clean":"2024-04-26T01:48:21.386363+0000","last_became_active":"2024-04-26T01:45:54.833868+0000","last_became_peered":"2024-04-26T01:45:54.833868+0000","last_unstale":"2024-04-26T01:48:21.386363+0000","last_undegraded":"2024-04-26T01:48:21.386363+0000","last_fullsized":"2024-04-26T01:48:21.386363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:42:05.228956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:42:05.228956+0000","last_clean_scrub_stamp":"2024-04-26T01:42:05.228956+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:06:58.383315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.137915+0000","last_change":"2024-04-26T01:47:27.126475+0000","last_active":"2024-04-26T01:47:30.137915+0000","last_peered":"2024-04-26T01:47:30.137915+0000","last_clean":"2024-04-26T01:47:30.137915+0000","last_became_active":"2024-04-26T01:47:27.123760+0000","last_became_peered":"2024-04-26T01:47:27.123760+0000","last_unstale":"2024-04-26T01:47:30.137915+0000","last_undegraded":"2024-04-26T01:47:30.137915+0000","last_fullsized":"2024-04-26T01:47:30.137915+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:38:27.473909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:48:21.386548+0000","last_change":"2024-04-26T01:47:23.059810+0000","last_active":"2024-04-26T01:48:21.386548+0000","last_peered":"2024-04-26T01:48:21.386548+0000","last_clean":"2024-04-26T01:48:21.386548+0000","last_became_active":"2024-04-26T01:47:23.059504+0000","last_became_peered":"2024-04-26T01:47:23.059504+0000","last_unstale":"2024-04-26T01:48:21.386548+0000","last_undegraded":"2024-04-26T01:48:21.386548+0000","last_fullsized":"2024-04-26T01:48:21.386548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:50:51.220951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396987+0000","last_change":"2024-04-26T01:47:23.109460+0000","last_active":"2024-04-26T01:48:22.396987+0000","last_peered":"2024-04-26T01:48:22.396987+0000","last_clean":"2024-04-26T01:48:22.396987+0000","last_became_active":"2024-04-26T01:47:23.109090+0000","last_became_peered":"2024-04-26T01:47:23.109090+0000","last_unstale":"2024-04-26T01:48:22.396987+0000","last_undegraded":"2024-04-26T01:48:22.396987+0000","last_fullsized":"2024-04-26T01:48:22.396987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:56.821312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.139227+0000","last_change":"2024-04-26T01:47:27.129537+0000","last_active":"2024-04-26T01:47:30.139227+0000","last_peered":"2024-04-26T01:47:30.139227+0000","last_clean":"2024-04-26T01:47:30.139227+0000","last_became_active":"2024-04-26T01:47:27.128124+0000","last_became_peered":"2024-04-26T01:47:27.128124+0000","last_unstale":"2024-04-26T01:47:30.139227+0000","last_undegraded":"2024-04-26T01:47:30.139227+0000","last_fullsized":"2024-04-26T01:47:30.139227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:14:26.262197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'11","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:48:16.131977+0000","last_change":"2024-04-26T01:47:23.064511+0000","last_active":"2024-04-26T01:48:16.131977+0000","last_peered":"2024-04-26T01:48:16.131977+0000","last_clean":"2024-04-26T01:48:16.131977+0000","last_became_active":"2024-04-26T01:47:23.064080+0000","last_became_peered":"2024-04-26T01:47:23.064080+0000","last_unstale":"2024-04-26T01:48:16.131977+0000","last_undegraded":"2024-04-26T01:48:16.131977+0000","last_fullsized":"2024-04-26T01:48:16.131977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:37.562644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404336+0000","last_change":"2024-04-26T01:47:27.124057+0000","last_active":"2024-04-26T01:48:22.404336+0000","last_peered":"2024-04-26T01:48:22.404336+0000","last_clean":"2024-04-26T01:48:22.404336+0000","last_became_active":"2024-04-26T01:47:27.123636+0000","last_became_peered":"2024-04-26T01:47:27.123636+0000","last_unstale":"2024-04-26T01:48:22.404336+0000","last_undegraded":"2024-04-26T01:48:22.404336+0000","last_fullsized":"2024-04-26T01:48:22.404336+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:56:55.691697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:46.212817+0000","last_change":"2024-04-26T01:47:23.136571+0000","last_active":"2024-04-26T01:47:46.212817+0000","last_peered":"2024-04-26T01:47:46.212817+0000","last_clean":"2024-04-26T01:47:46.212817+0000","last_became_active":"2024-04-26T01:47:23.136205+0000","last_became_peered":"2024-04-26T01:47:23.136205+0000","last_unstale":"2024-04-26T01:47:46.212817+0000","last_undegraded":"2024-04-26T01:47:46.212817+0000","last_fullsized":"2024-04-26T01:47:46.212817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:24.418797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.136592+0000","last_change":"2024-04-26T01:47:27.139078+0000","last_active":"2024-04-26T01:47:30.136592+0000","last_peered":"2024-04-26T01:47:30.136592+0000","last_clean":"2024-04-26T01:47:30.136592+0000","last_became_active":"2024-04-26T01:47:27.136278+0000","last_became_peered":"2024-04-26T01:47:27.136278+0000","last_unstale":"2024-04-26T01:47:30.136592+0000","last_undegraded":"2024-04-26T01:47:30.136592+0000","last_fullsized":"2024-04-26T01:47:30.136592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:06:57.194991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.414610+0000","last_change":"2024-04-26T01:48:22.414610+0000","last_active":"2024-04-26T01:48:22.414610+0000","last_peered":"2024-04-26T01:48:22.414610+0000","last_clean":"2024-04-26T01:48:22.414610+0000","last_became_active":"2024-04-26T01:48:22.410275+0000","last_became_peered":"2024-04-26T01:48:22.410275+0000","last_unstale":"2024-04-26T01:48:22.414610+0000","last_undegraded":"2024-04-26T01:48:22.414610+0000","last_fullsized":"2024-04-26T01:48:22.414610+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:20:48.131988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396626+0000","last_change":"2024-04-26T01:47:27.115498+0000","last_active":"2024-04-26T01:48:22.396626+0000","last_peered":"2024-04-26T01:48:22.396626+0000","last_clean":"2024-04-26T01:48:22.396626+0000","last_became_active":"2024-04-26T01:47:27.115134+0000","last_became_peered":"2024-04-26T01:47:27.115134+0000","last_unstale":"2024-04-26T01:48:22.396626+0000","last_undegraded":"2024-04-26T01:48:22.396626+0000","last_fullsized":"2024-04-26T01:48:22.396626+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:20.795224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405265+0000","last_change":"2024-04-26T01:47:23.065366+0000","last_active":"2024-04-26T01:48:22.405265+0000","last_peered":"2024-04-26T01:48:22.405265+0000","last_clean":"2024-04-26T01:48:22.405265+0000","last_became_active":"2024-04-26T01:47:23.063216+0000","last_became_peered":"2024-04-26T01:47:23.063216+0000","last_unstale":"2024-04-26T01:48:22.405265+0000","last_undegraded":"2024-04-26T01:48:22.405265+0000","last_fullsized":"2024-04-26T01:48:22.405265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:06:05.261907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.130710+0000","last_change":"2024-04-26T01:47:27.127938+0000","last_active":"2024-04-26T01:47:30.130710+0000","last_peered":"2024-04-26T01:47:30.130710+0000","last_clean":"2024-04-26T01:47:30.130710+0000","last_became_active":"2024-04-26T01:47:27.127553+0000","last_became_peered":"2024-04-26T01:47:27.127553+0000","last_unstale":"2024-04-26T01:47:30.130710+0000","last_undegraded":"2024-04-26T01:47:30.130710+0000","last_fullsized":"2024-04-26T01:47:30.130710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:17:51.115349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.127002+0000","last_change":"2024-04-26T01:47:23.060475+0000","last_active":"2024-04-26T01:47:30.127002+0000","last_peered":"2024-04-26T01:47:30.127002+0000","last_clean":"2024-04-26T01:47:30.127002+0000","last_became_active":"2024-04-26T01:47:23.060098+0000","last_became_peered":"2024-04-26T01:47:23.060098+0000","last_unstale":"2024-04-26T01:47:30.127002+0000","last_undegraded":"2024-04-26T01:47:30.127002+0000","last_fullsized":"2024-04-26T01:47:30.127002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:49.009455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.407796+0000","last_change":"2024-04-26T01:47:27.137491+0000","last_active":"2024-04-26T01:48:22.407796+0000","last_peered":"2024-04-26T01:48:22.407796+0000","last_clean":"2024-04-26T01:48:22.407796+0000","last_became_active":"2024-04-26T01:47:27.137059+0000","last_became_peered":"2024-04-26T01:47:27.137059+0000","last_unstale":"2024-04-26T01:48:22.407796+0000","last_undegraded":"2024-04-26T01:48:22.407796+0000","last_fullsized":"2024-04-26T01:48:22.407796+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:20:25.004074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:46.212212+0000","last_change":"2024-04-26T01:47:23.062410+0000","last_active":"2024-04-26T01:47:46.212212+0000","last_peered":"2024-04-26T01:47:46.212212+0000","last_clean":"2024-04-26T01:47:46.212212+0000","last_became_active":"2024-04-26T01:47:23.062028+0000","last_became_peered":"2024-04-26T01:47:23.062028+0000","last_unstale":"2024-04-26T01:47:46.212212+0000","last_undegraded":"2024-04-26T01:47:46.212212+0000","last_fullsized":"2024-04-26T01:47:46.212212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:40:25.950729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:48:21.385992+0000","last_change":"2024-04-26T01:47:27.128253+0000","last_active":"2024-04-26T01:48:21.385992+0000","last_peered":"2024-04-26T01:48:21.385992+0000","last_clean":"2024-04-26T01:48:21.385992+0000","last_became_active":"2024-04-26T01:47:27.127869+0000","last_became_peered":"2024-04-26T01:47:27.127869+0000","last_unstale":"2024-04-26T01:48:21.385992+0000","last_undegraded":"2024-04-26T01:48:21.385992+0000","last_fullsized":"2024-04-26T01:48:21.385992+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:11:23.476938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:47.231341+0000","last_change":"2024-04-26T01:47:23.136862+0000","last_active":"2024-04-26T01:47:47.231341+0000","last_peered":"2024-04-26T01:47:47.231341+0000","last_clean":"2024-04-26T01:47:47.231341+0000","last_became_active":"2024-04-26T01:47:23.136380+0000","last_became_peered":"2024-04-26T01:47:23.136380+0000","last_unstale":"2024-04-26T01:47:47.231341+0000","last_undegraded":"2024-04-26T01:47:47.231341+0000","last_fullsized":"2024-04-26T01:47:47.231341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:39:18.103559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:48:21.386023+0000","last_change":"2024-04-26T01:47:27.113363+0000","last_active":"2024-04-26T01:48:21.386023+0000","last_peered":"2024-04-26T01:48:21.386023+0000","last_clean":"2024-04-26T01:48:21.386023+0000","last_became_active":"2024-04-26T01:47:27.113040+0000","last_became_peered":"2024-04-26T01:47:27.113040+0000","last_unstale":"2024-04-26T01:48:21.386023+0000","last_undegraded":"2024-04-26T01:48:21.386023+0000","last_fullsized":"2024-04-26T01:48:21.386023+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:42:16.428697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.133856+0000","last_change":"2024-04-26T01:47:23.067257+0000","last_active":"2024-04-26T01:47:30.133856+0000","last_peered":"2024-04-26T01:47:30.133856+0000","last_clean":"2024-04-26T01:47:30.133856+0000","last_became_active":"2024-04-26T01:47:23.066858+0000","last_became_peered":"2024-04-26T01:47:23.066858+0000","last_unstale":"2024-04-26T01:47:30.133856+0000","last_undegraded":"2024-04-26T01:47:30.133856+0000","last_fullsized":"2024-04-26T01:47:30.133856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:22:07.848074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:48:21.385567+0000","last_change":"2024-04-26T01:47:27.125885+0000","last_active":"2024-04-26T01:48:21.385567+0000","last_peered":"2024-04-26T01:48:21.385567+0000","last_clean":"2024-04-26T01:48:21.385567+0000","last_became_active":"2024-04-26T01:47:27.125524+0000","last_became_peered":"2024-04-26T01:47:27.125524+0000","last_unstale":"2024-04-26T01:48:21.385567+0000","last_undegraded":"2024-04-26T01:48:21.385567+0000","last_fullsized":"2024-04-26T01:48:21.385567+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:39.282023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.142439+0000","last_change":"2024-04-26T01:47:23.071051+0000","last_active":"2024-04-26T01:47:30.142439+0000","last_peered":"2024-04-26T01:47:30.142439+0000","last_clean":"2024-04-26T01:47:30.142439+0000","last_became_active":"2024-04-26T01:47:23.069524+0000","last_became_peered":"2024-04-26T01:47:23.069524+0000","last_unstale":"2024-04-26T01:47:30.142439+0000","last_undegraded":"2024-04-26T01:47:30.142439+0000","last_fullsized":"2024-04-26T01:47:30.142439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:24:57.643100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:48:21.387120+0000","last_change":"2024-04-26T01:47:27.114325+0000","last_active":"2024-04-26T01:48:21.387120+0000","last_peered":"2024-04-26T01:48:21.387120+0000","last_clean":"2024-04-26T01:48:21.387120+0000","last_became_active":"2024-04-26T01:47:27.113952+0000","last_became_peered":"2024-04-26T01:47:27.113952+0000","last_unstale":"2024-04-26T01:48:21.387120+0000","last_undegraded":"2024-04-26T01:48:21.387120+0000","last_fullsized":"2024-04-26T01:48:21.387120+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:39:19.167175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.130887+0000","last_change":"2024-04-26T01:47:27.118709+0000","last_active":"2024-04-26T01:47:30.130887+0000","last_peered":"2024-04-26T01:47:30.130887+0000","last_clean":"2024-04-26T01:47:30.130887+0000","last_became_active":"2024-04-26T01:47:27.118341+0000","last_became_peered":"2024-04-26T01:47:27.118341+0000","last_unstale":"2024-04-26T01:47:30.130887+0000","last_undegraded":"2024-04-26T01:47:30.130887+0000","last_fullsized":"2024-04-26T01:47:30.130887+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:13.736180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.126673+0000","last_change":"2024-04-26T01:47:27.119239+0000","last_active":"2024-04-26T01:47:30.126673+0000","last_peered":"2024-04-26T01:47:30.126673+0000","last_clean":"2024-04-26T01:47:30.126673+0000","last_became_active":"2024-04-26T01:47:27.118877+0000","last_became_peered":"2024-04-26T01:47:27.118877+0000","last_unstale":"2024-04-26T01:47:30.126673+0000","last_undegraded":"2024-04-26T01:47:30.126673+0000","last_fullsized":"2024-04-26T01:47:30.126673+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:04:10.033517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.126810+0000","last_change":"2024-04-26T01:47:27.142848+0000","last_active":"2024-04-26T01:47:30.126810+0000","last_peered":"2024-04-26T01:47:30.126810+0000","last_clean":"2024-04-26T01:47:30.126810+0000","last_became_active":"2024-04-26T01:47:27.142580+0000","last_became_peered":"2024-04-26T01:47:27.142580+0000","last_unstale":"2024-04-26T01:47:30.126810+0000","last_undegraded":"2024-04-26T01:47:30.126810+0000","last_fullsized":"2024-04-26T01:47:30.126810+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:15:46.923922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.135646+0000","last_change":"2024-04-26T01:47:27.134817+0000","last_active":"2024-04-26T01:47:30.135646+0000","last_peered":"2024-04-26T01:47:30.135646+0000","last_clean":"2024-04-26T01:47:30.135646+0000","last_became_active":"2024-04-26T01:47:27.134024+0000","last_became_peered":"2024-04-26T01:47:27.134024+0000","last_unstale":"2024-04-26T01:47:30.135646+0000","last_undegraded":"2024-04-26T01:47:30.135646+0000","last_fullsized":"2024-04-26T01:47:30.135646+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:56.746427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.138830+0000","last_change":"2024-04-26T01:47:27.143137+0000","last_active":"2024-04-26T01:47:30.138830+0000","last_peered":"2024-04-26T01:47:30.138830+0000","last_clean":"2024-04-26T01:47:30.138830+0000","last_became_active":"2024-04-26T01:47:27.142625+0000","last_became_peered":"2024-04-26T01:47:27.142625+0000","last_unstale":"2024-04-26T01:47:30.138830+0000","last_undegraded":"2024-04-26T01:47:30.138830+0000","last_fullsized":"2024-04-26T01:47:30.138830+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:42:50.468382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.131179+0000","last_change":"2024-04-26T01:47:27.119684+0000","last_active":"2024-04-26T01:47:30.131179+0000","last_peered":"2024-04-26T01:47:30.131179+0000","last_clean":"2024-04-26T01:47:30.131179+0000","last_became_active":"2024-04-26T01:47:27.119221+0000","last_became_peered":"2024-04-26T01:47:27.119221+0000","last_unstale":"2024-04-26T01:47:30.131179+0000","last_undegraded":"2024-04-26T01:47:30.131179+0000","last_fullsized":"2024-04-26T01:47:30.131179+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:45:09.451277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.128415+0000","last_change":"2024-04-26T01:47:27.123964+0000","last_active":"2024-04-26T01:47:30.128415+0000","last_peered":"2024-04-26T01:47:30.128415+0000","last_clean":"2024-04-26T01:47:30.128415+0000","last_became_active":"2024-04-26T01:47:27.123552+0000","last_became_peered":"2024-04-26T01:47:27.123552+0000","last_unstale":"2024-04-26T01:47:30.128415+0000","last_undegraded":"2024-04-26T01:47:30.128415+0000","last_fullsized":"2024-04-26T01:47:30.128415+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:33:12.566105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.412621+0000","last_change":"2024-04-26T01:48:22.412621+0000","last_active":"2024-04-26T01:48:22.412621+0000","last_peered":"2024-04-26T01:48:22.412621+0000","last_clean":"2024-04-26T01:48:22.412621+0000","last_became_active":"2024-04-26T01:48:22.411430+0000","last_became_peered":"2024-04-26T01:48:22.411430+0000","last_unstale":"2024-04-26T01:48:22.412621+0000","last_undegraded":"2024-04-26T01:48:22.412621+0000","last_fullsized":"2024-04-26T01:48:22.412621+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:34.152338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.122992+0000","last_change":"2024-04-26T01:47:27.131437+0000","last_active":"2024-04-26T01:47:30.122992+0000","last_peered":"2024-04-26T01:47:30.122992+0000","last_clean":"2024-04-26T01:47:30.122992+0000","last_became_active":"2024-04-26T01:47:27.131138+0000","last_became_peered":"2024-04-26T01:47:27.131138+0000","last_unstale":"2024-04-26T01:47:30.122992+0000","last_undegraded":"2024-04-26T01:47:30.122992+0000","last_fullsized":"2024-04-26T01:47:30.122992+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:55:39.132972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404958+0000","last_change":"2024-04-26T01:47:27.120928+0000","last_active":"2024-04-26T01:48:22.404958+0000","last_peered":"2024-04-26T01:48:22.404958+0000","last_clean":"2024-04-26T01:48:22.404958+0000","last_became_active":"2024-04-26T01:47:27.120012+0000","last_became_peered":"2024-04-26T01:47:27.120012+0000","last_unstale":"2024-04-26T01:48:22.404958+0000","last_undegraded":"2024-04-26T01:48:22.404958+0000","last_fullsized":"2024-04-26T01:48:22.404958+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:19:44.209876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398493+0000","last_change":"2024-04-26T01:47:27.138766+0000","last_active":"2024-04-26T01:48:22.398493+0000","last_peered":"2024-04-26T01:48:22.398493+0000","last_clean":"2024-04-26T01:48:22.398493+0000","last_became_active":"2024-04-26T01:47:27.138413+0000","last_became_peered":"2024-04-26T01:47:27.138413+0000","last_unstale":"2024-04-26T01:48:22.398493+0000","last_undegraded":"2024-04-26T01:48:22.398493+0000","last_fullsized":"2024-04-26T01:48:22.398493+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:51:13.117307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.138660+0000","last_change":"2024-04-26T01:47:27.141338+0000","last_active":"2024-04-26T01:47:30.138660+0000","last_peered":"2024-04-26T01:47:30.138660+0000","last_clean":"2024-04-26T01:47:30.138660+0000","last_became_active":"2024-04-26T01:47:27.140825+0000","last_became_peered":"2024-04-26T01:47:27.140825+0000","last_unstale":"2024-04-26T01:47:30.138660+0000","last_undegraded":"2024-04-26T01:47:30.138660+0000","last_fullsized":"2024-04-26T01:47:30.138660+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:12:08.680876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T01:47:30.130710+0000","last_change":"2024-04-26T01:47:27.114919+0000","last_active":"2024-04-26T01:47:30.130710+0000","last_peered":"2024-04-26T01:47:30.130710+0000","last_clean":"2024-04-26T01:47:30.130710+0000","last_became_active":"2024-04-26T01:47:27.114528+0000","last_became_peered":"2024-04-26T01:47:27.114528+0000","last_unstale":"2024-04-26T01:47:30.130710+0000","last_undegraded":"2024-04-26T01:47:30.130710+0000","last_fullsized":"2024-04-26T01:47:30.130710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:15:25.620566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64348,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":213,"num_read_kb":177,"num_write":163,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318298,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":473071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579935,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1675264,"data_stored":1125290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743461,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":473307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005101,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1011712,"data_stored":481986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1605632,"data_stored":1073916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462966,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":472696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1060864,"data_stored":541595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":473967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":529929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":490283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920832,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":473524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724605,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32740,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712412,"statfs":{"total":95995035648,"available":95961509888,"internally_reserved":0,"allocated":1601536,"data_stored":1064450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:48:24.750 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:48:24.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:24 smithi173 ceph-mon[32553]: pgmap v353: 97 pgs: 1 activating+degraded, 2 peering, 94 active+clean; 640 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 7 op/s; 8/192 objects degraded (4.167%); 0 B/s, 0 objects/s recovering 2024-04-26T01:48:24.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:24 smithi173 ceph-mon[32553]: Health check failed: Degraded data redundancy: 8/192 objects degraded (4.167%), 1 pg degraded (PG_DEGRADED) 2024-04-26T01:48:24.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:24 smithi173 ceph-mon[32553]: from='client.24661 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:48:25.248 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:26.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:26 smithi097 ceph-mon[32853]: pgmap v354: 97 pgs: 1 activating+degraded, 2 peering, 94 active+clean; 640 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 639 B/s wr, 5 op/s; 8/192 objects degraded (4.167%); 0 B/s, 0 objects/s recovering 2024-04-26T01:48:26.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:26 smithi173 ceph-mon[32553]: pgmap v354: 97 pgs: 1 activating+degraded, 2 peering, 94 active+clean; 640 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 639 B/s wr, 5 op/s; 8/192 objects degraded (4.167%); 0 B/s, 0 objects/s recovering 2024-04-26T01:48:26.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:26 smithi001 ceph-mon[27822]: pgmap v354: 97 pgs: 1 activating+degraded, 2 peering, 94 active+clean; 640 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 639 B/s wr, 5 op/s; 8/192 objects degraded (4.167%); 0 B/s, 0 objects/s recovering 2024-04-26T01:48:27.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.445+0000 7fb6f6a00700 1 -- 172.21.15.1:0/1236483524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f0102a90 msgr2=0x7fb6f0102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:27.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.445+0000 7fb6f6a00700 1 --2- 172.21.15.1:0/1236483524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f0102a90 0x7fb6f0102f10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009990 tx=0x7fb6dc009ca0 comp rx=0 tx=0).stop 2024-04-26T01:48:27.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.445+0000 7fb6f6a00700 1 -- 172.21.15.1:0/1236483524 shutdown_connections 2024-04-26T01:48:27.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.445+0000 7fb6f6a00700 1 --2- 172.21.15.1:0/1236483524 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6f0103450 0x7fb6f010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:27.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.445+0000 7fb6f6a00700 1 --2- 172.21.15.1:0/1236483524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6f0108a80 0x7fb6f0108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:27.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.445+0000 7fb6f6a00700 1 --2- 172.21.15.1:0/1236483524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f0102a90 0x7fb6f0102f10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:27.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.445+0000 7fb6f6a00700 1 -- 172.21.15.1:0/1236483524 >> 172.21.15.1:0/1236483524 conn(0x7fb6f0079bf0 msgr2=0x7fb6f007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:27.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.446+0000 7fb6f6a00700 1 -- 172.21.15.1:0/1236483524 shutdown_connections 2024-04-26T01:48:27.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.446+0000 7fb6f6a00700 1 -- 172.21.15.1:0/1236483524 wait complete. 2024-04-26T01:48:27.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.447+0000 7fb6f6a00700 1 Processor -- start 2024-04-26T01:48:27.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.447+0000 7fb6f6a00700 1 -- start start 2024-04-26T01:48:27.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.447+0000 7fb6f6a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6f0102a90 0x7fb6f019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:27.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.447+0000 7fb6f6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f0103450 0x7fb6f019b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:27.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.447+0000 7fb6f6a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6f0108a80 0x7fb6f019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:27.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.447+0000 7fb6f6a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f0112ca0 con 0x7fb6f0103450 2024-04-26T01:48:27.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.447+0000 7fb6f6a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f0112b20 con 0x7fb6f0108a80 2024-04-26T01:48:27.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.447+0000 7fb6f6a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f0112e20 con 0x7fb6f0102a90 2024-04-26T01:48:27.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.447+0000 7fb6efe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6f0102a90 0x7fb6f019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:27.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.448+0000 7fb6efe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6f0102a90 0x7fb6f019b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:46774/0 (socket says 172.21.15.1:46774) 2024-04-26T01:48:27.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.448+0000 7fb6efe00700 1 -- 172.21.15.1:0/4262361796 learned_addr learned my addr 172.21.15.1:0/4262361796 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:48:27.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.448+0000 7fb6ef400700 1 --2- 172.21.15.1:0/4262361796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f0103450 0x7fb6f019b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:27.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.448+0000 7fb6f4a00700 1 --2- 172.21.15.1:0/4262361796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6f0108a80 0x7fb6f019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:27.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.448+0000 7fb6efe00700 1 -- 172.21.15.1:0/4262361796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6f0108a80 msgr2=0x7fb6f019fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:27.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.448+0000 7fb6efe00700 1 --2- 172.21.15.1:0/4262361796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6f0108a80 0x7fb6f019fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:27.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.448+0000 7fb6efe00700 1 -- 172.21.15.1:0/4262361796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f0103450 msgr2=0x7fb6f019b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:27.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.448+0000 7fb6efe00700 1 --2- 172.21.15.1:0/4262361796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f0103450 0x7fb6f019b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:27.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.448+0000 7fb6efe00700 1 -- 172.21.15.1:0/4262361796 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f01a0330 con 0x7fb6f0102a90 2024-04-26T01:48:27.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.448+0000 7fb6efe00700 1 --2- 172.21.15.1:0/4262361796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6f0102a90 0x7fb6f019b320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000e9c0 tx=0x7fb6e000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:27.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.449+0000 7fb6ecc00700 1 -- 172.21.15.1:0/4262361796 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e000cbd0 con 0x7fb6f0102a90 2024-04-26T01:48:27.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.449+0000 7fb6f6a00700 1 -- 172.21.15.1:0/4262361796 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f01a0620 con 0x7fb6f0102a90 2024-04-26T01:48:27.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.449+0000 7fb6ecc00700 1 -- 172.21.15.1:0/4262361796 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6e000ee60 con 0x7fb6f0102a90 2024-04-26T01:48:27.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.449+0000 7fb6ecc00700 1 -- 172.21.15.1:0/4262361796 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e00055c0 con 0x7fb6f0102a90 2024-04-26T01:48:27.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.449+0000 7fb6f6a00700 1 -- 172.21.15.1:0/4262361796 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f01a7ea0 con 0x7fb6f0102a90 2024-04-26T01:48:27.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.450+0000 7fb6f6a00700 1 -- 172.21.15.1:0/4262361796 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f0069730 con 0x7fb6f0102a90 2024-04-26T01:48:27.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.452+0000 7fb6ecc00700 1 -- 172.21.15.1:0/4262361796 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fb6e0004750 con 0x7fb6f0102a90 2024-04-26T01:48:27.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.454+0000 7fb6ecc00700 1 --2- 172.21.15.1:0/4262361796 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb6d40a3cd0 0x7fb6d40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:27.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.454+0000 7fb6ecc00700 1 -- 172.21.15.1:0/4262361796 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fb6e00d07e0 con 0x7fb6f0102a90 2024-04-26T01:48:27.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.454+0000 7fb6ef400700 1 --2- 172.21.15.1:0/4262361796 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb6d40a3cd0 0x7fb6d40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:27.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.455+0000 7fb6ef400700 1 --2- 172.21.15.1:0/4262361796 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb6d40a3cd0 0x7fb6d40a6190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb6f010fd20 tx=0x7fb6dc00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:27.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.458+0000 7fb6ecc00700 1 -- 172.21.15.1:0/4262361796 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6e009cfe0 con 0x7fb6f0102a90 2024-04-26T01:48:27.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.685+0000 7fb6f6a00700 1 -- 172.21.15.1:0/4262361796 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb6f0101c10 con 0x7fb6d40a3cd0 2024-04-26T01:48:27.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.701+0000 7fb6ecc00700 1 -- 172.21.15.1:0/4262361796 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258775 (secure 0 0 0) 0x7fb6dc0672f0 con 0x7fb6d40a3cd0 2024-04-26T01:48:27.704 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:48:27.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.707+0000 7fb6f6a00700 1 -- 172.21.15.1:0/4262361796 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb6d40a3cd0 msgr2=0x7fb6d40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:27.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.707+0000 7fb6f6a00700 1 --2- 172.21.15.1:0/4262361796 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb6d40a3cd0 0x7fb6d40a6190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb6f010fd20 tx=0x7fb6dc00b260 comp rx=0 tx=0).stop 2024-04-26T01:48:27.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.707+0000 7fb6f6a00700 1 -- 172.21.15.1:0/4262361796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6f0102a90 msgr2=0x7fb6f019b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:27.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.707+0000 7fb6f6a00700 1 --2- 172.21.15.1:0/4262361796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6f0102a90 0x7fb6f019b320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000e9c0 tx=0x7fb6e000ecd0 comp rx=0 tx=0).stop 2024-04-26T01:48:27.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.707+0000 7fb6f6a00700 1 -- 172.21.15.1:0/4262361796 shutdown_connections 2024-04-26T01:48:27.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.707+0000 7fb6f6a00700 1 --2- 172.21.15.1:0/4262361796 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb6d40a3cd0 0x7fb6d40a6190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:27.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.707+0000 7fb6f6a00700 1 --2- 172.21.15.1:0/4262361796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6f0108a80 0x7fb6f019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:27.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.707+0000 7fb6f6a00700 1 --2- 172.21.15.1:0/4262361796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6f0102a90 0x7fb6f019b320 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:27.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.707+0000 7fb6f6a00700 1 --2- 172.21.15.1:0/4262361796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f0103450 0x7fb6f019b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:27.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.707+0000 7fb6f6a00700 1 -- 172.21.15.1:0/4262361796 >> 172.21.15.1:0/4262361796 conn(0x7fb6f0079bf0 msgr2=0x7fb6f0101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:27.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.707+0000 7fb6f6a00700 1 -- 172.21.15.1:0/4262361796 shutdown_connections 2024-04-26T01:48:27.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:27.707+0000 7fb6f6a00700 1 -- 172.21.15.1:0/4262361796 wait complete. 2024-04-26T01:48:27.711 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-04-26T01:48:28.039 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:27 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:48:28.039 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:27 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:48:28.039 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:27 smithi001 ceph-mon[27822]: pgmap v355: 97 pgs: 1 activating+degraded, 96 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s; 8/192 objects degraded (4.167%); 0 B/s, 0 objects/s recovering 2024-04-26T01:48:28.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:27 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:48:28.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:27 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:48:28.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:27 smithi173 ceph-mon[32553]: pgmap v355: 97 pgs: 1 activating+degraded, 96 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s; 8/192 objects degraded (4.167%); 0 B/s, 0 objects/s recovering 2024-04-26T01:48:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:27 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:48:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:27 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:48:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:27 smithi097 ceph-mon[32853]: pgmap v355: 97 pgs: 1 activating+degraded, 96 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s; 8/192 objects degraded (4.167%); 0 B/s, 0 objects/s recovering 2024-04-26T01:48:28.632 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-04-26T01:48:26.820714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654950,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":377,"num_read_kb":309,"num_write":346,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":294,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358996,"kb_used_data":13948,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582828,"statfs":{"total":1151940427776,"available":1151572815872,"internally_reserved":0,"allocated":14282752,"data_stored":7824996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4257,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":32,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003228"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398825+0000","last_change":"2024-04-26T01:47:27.140418+0000","last_active":"2024-04-26T01:48:22.398825+0000","last_peered":"2024-04-26T01:48:22.398825+0000","last_clean":"2024-04-26T01:48:22.398825+0000","last_became_active":"2024-04-26T01:47:27.140017+0000","last_became_peered":"2024-04-26T01:47:27.140017+0000","last_unstale":"2024-04-26T01:48:22.398825+0000","last_undegraded":"2024-04-26T01:48:22.398825+0000","last_fullsized":"2024-04-26T01:48:22.398825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:46:08.621222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404830+0000","last_change":"2024-04-26T01:47:27.138746+0000","last_active":"2024-04-26T01:48:22.404830+0000","last_peered":"2024-04-26T01:48:22.404830+0000","last_clean":"2024-04-26T01:48:22.404830+0000","last_became_active":"2024-04-26T01:47:27.136163+0000","last_became_peered":"2024-04-26T01:47:27.136163+0000","last_unstale":"2024-04-26T01:48:22.404830+0000","last_undegraded":"2024-04-26T01:48:22.404830+0000","last_fullsized":"2024-04-26T01:48:22.404830+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:01:36.195332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.403704+0000","last_change":"2024-04-26T01:47:27.127017+0000","last_active":"2024-04-26T01:48:22.403704+0000","last_peered":"2024-04-26T01:48:22.403704+0000","last_clean":"2024-04-26T01:48:22.403704+0000","last_became_active":"2024-04-26T01:47:27.126610+0000","last_became_peered":"2024-04-26T01:47:27.126610+0000","last_unstale":"2024-04-26T01:48:22.403704+0000","last_undegraded":"2024-04-26T01:48:22.403704+0000","last_fullsized":"2024-04-26T01:48:22.403704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:05:12.784486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746210+0000","last_change":"2024-04-26T01:47:27.117640+0000","last_active":"2024-04-26T01:48:22.746210+0000","last_peered":"2024-04-26T01:48:22.746210+0000","last_clean":"2024-04-26T01:48:22.746210+0000","last_became_active":"2024-04-26T01:47:27.117266+0000","last_became_peered":"2024-04-26T01:47:27.117266+0000","last_unstale":"2024-04-26T01:48:22.746210+0000","last_undegraded":"2024-04-26T01:48:22.746210+0000","last_fullsized":"2024-04-26T01:48:22.746210+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:26:57.824560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396755+0000","last_change":"2024-04-26T01:47:27.128185+0000","last_active":"2024-04-26T01:48:22.396755+0000","last_peered":"2024-04-26T01:48:22.396755+0000","last_clean":"2024-04-26T01:48:22.396755+0000","last_became_active":"2024-04-26T01:47:27.127794+0000","last_became_peered":"2024-04-26T01:47:27.127794+0000","last_unstale":"2024-04-26T01:48:22.396755+0000","last_undegraded":"2024-04-26T01:48:22.396755+0000","last_fullsized":"2024-04-26T01:48:22.396755+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:09:52.774007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.748601+0000","last_change":"2024-04-26T01:48:22.748601+0000","last_active":"2024-04-26T01:48:22.748601+0000","last_peered":"2024-04-26T01:48:22.748601+0000","last_clean":"2024-04-26T01:48:22.748601+0000","last_became_active":"2024-04-26T01:48:22.748229+0000","last_became_peered":"2024-04-26T01:48:22.748229+0000","last_unstale":"2024-04-26T01:48:22.748601+0000","last_undegraded":"2024-04-26T01:48:22.748601+0000","last_fullsized":"2024-04-26T01:48:22.748601+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:46:30.433063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400077+0000","last_change":"2024-04-26T01:47:27.123786+0000","last_active":"2024-04-26T01:48:22.400077+0000","last_peered":"2024-04-26T01:48:22.400077+0000","last_clean":"2024-04-26T01:48:22.400077+0000","last_became_active":"2024-04-26T01:47:27.123473+0000","last_became_peered":"2024-04-26T01:47:27.123473+0000","last_unstale":"2024-04-26T01:48:22.400077+0000","last_undegraded":"2024-04-26T01:48:22.400077+0000","last_fullsized":"2024-04-26T01:48:22.400077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:48:55.031147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404450+0000","last_change":"2024-04-26T01:47:27.118756+0000","last_active":"2024-04-26T01:48:22.404450+0000","last_peered":"2024-04-26T01:48:22.404450+0000","last_clean":"2024-04-26T01:48:22.404450+0000","last_became_active":"2024-04-26T01:47:27.118406+0000","last_became_peered":"2024-04-26T01:47:27.118406+0000","last_unstale":"2024-04-26T01:48:22.404450+0000","last_undegraded":"2024-04-26T01:48:22.404450+0000","last_fullsized":"2024-04-26T01:48:22.404450+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:05.793682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398754+0000","last_change":"2024-04-26T01:47:27.117622+0000","last_active":"2024-04-26T01:48:22.398754+0000","last_peered":"2024-04-26T01:48:22.398754+0000","last_clean":"2024-04-26T01:48:22.398754+0000","last_became_active":"2024-04-26T01:47:27.115206+0000","last_became_peered":"2024-04-26T01:47:27.115206+0000","last_unstale":"2024-04-26T01:48:22.398754+0000","last_undegraded":"2024-04-26T01:48:22.398754+0000","last_fullsized":"2024-04-26T01:48:22.398754+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:13:17.866630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405408+0000","last_change":"2024-04-26T01:47:27.119524+0000","last_active":"2024-04-26T01:48:22.405408+0000","last_peered":"2024-04-26T01:48:22.405408+0000","last_clean":"2024-04-26T01:48:22.405408+0000","last_became_active":"2024-04-26T01:47:27.119008+0000","last_became_peered":"2024-04-26T01:47:27.119008+0000","last_unstale":"2024-04-26T01:48:22.405408+0000","last_undegraded":"2024-04-26T01:48:22.405408+0000","last_fullsized":"2024-04-26T01:48:22.405408+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:37:31.894486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400411+0000","last_change":"2024-04-26T01:47:27.122361+0000","last_active":"2024-04-26T01:48:22.400411+0000","last_peered":"2024-04-26T01:48:22.400411+0000","last_clean":"2024-04-26T01:48:22.400411+0000","last_became_active":"2024-04-26T01:47:27.121748+0000","last_became_peered":"2024-04-26T01:47:27.121748+0000","last_unstale":"2024-04-26T01:48:22.400411+0000","last_undegraded":"2024-04-26T01:48:22.400411+0000","last_fullsized":"2024-04-26T01:48:22.400411+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:18.680909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746160+0000","last_change":"2024-04-26T01:47:27.116833+0000","last_active":"2024-04-26T01:48:22.746160+0000","last_peered":"2024-04-26T01:48:22.746160+0000","last_clean":"2024-04-26T01:48:22.746160+0000","last_became_active":"2024-04-26T01:47:27.116508+0000","last_became_peered":"2024-04-26T01:47:27.116508+0000","last_unstale":"2024-04-26T01:48:22.746160+0000","last_undegraded":"2024-04-26T01:48:22.746160+0000","last_fullsized":"2024-04-26T01:48:22.746160+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:27:45.864521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405977+0000","last_change":"2024-04-26T01:47:27.125342+0000","last_active":"2024-04-26T01:48:22.405977+0000","last_peered":"2024-04-26T01:48:22.405977+0000","last_clean":"2024-04-26T01:48:22.405977+0000","last_became_active":"2024-04-26T01:47:27.124862+0000","last_became_peered":"2024-04-26T01:47:27.124862+0000","last_unstale":"2024-04-26T01:48:22.405977+0000","last_undegraded":"2024-04-26T01:48:22.405977+0000","last_fullsized":"2024-04-26T01:48:22.405977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:48.654711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.403399+0000","last_change":"2024-04-26T01:47:27.127365+0000","last_active":"2024-04-26T01:48:22.403399+0000","last_peered":"2024-04-26T01:48:22.403399+0000","last_clean":"2024-04-26T01:48:22.403399+0000","last_became_active":"2024-04-26T01:47:27.126868+0000","last_became_peered":"2024-04-26T01:47:27.126868+0000","last_unstale":"2024-04-26T01:48:22.403399+0000","last_undegraded":"2024-04-26T01:48:22.403399+0000","last_fullsized":"2024-04-26T01:48:22.403399+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:07:46.400333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.401775+0000","last_change":"2024-04-26T01:47:27.135584+0000","last_active":"2024-04-26T01:48:22.401775+0000","last_peered":"2024-04-26T01:48:22.401775+0000","last_clean":"2024-04-26T01:48:22.401775+0000","last_became_active":"2024-04-26T01:47:27.134849+0000","last_became_peered":"2024-04-26T01:47:27.134849+0000","last_unstale":"2024-04-26T01:48:22.401775+0000","last_undegraded":"2024-04-26T01:48:22.401775+0000","last_fullsized":"2024-04-26T01:48:22.401775+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:57:48.821595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398274+0000","last_change":"2024-04-26T01:47:27.136182+0000","last_active":"2024-04-26T01:48:22.398274+0000","last_peered":"2024-04-26T01:48:22.398274+0000","last_clean":"2024-04-26T01:48:22.398274+0000","last_became_active":"2024-04-26T01:47:27.135787+0000","last_became_peered":"2024-04-26T01:47:27.135787+0000","last_unstale":"2024-04-26T01:48:22.398274+0000","last_undegraded":"2024-04-26T01:48:22.398274+0000","last_fullsized":"2024-04-26T01:48:22.398274+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:38:14.275898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399072+0000","last_change":"2024-04-26T01:47:27.140700+0000","last_active":"2024-04-26T01:48:22.399072+0000","last_peered":"2024-04-26T01:48:22.399072+0000","last_clean":"2024-04-26T01:48:22.399072+0000","last_became_active":"2024-04-26T01:47:27.140025+0000","last_became_peered":"2024-04-26T01:47:27.140025+0000","last_unstale":"2024-04-26T01:48:22.399072+0000","last_undegraded":"2024-04-26T01:48:22.399072+0000","last_fullsized":"2024-04-26T01:48:22.399072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:50.465236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.747360+0000","last_change":"2024-04-26T01:47:27.117256+0000","last_active":"2024-04-26T01:48:22.747360+0000","last_peered":"2024-04-26T01:48:22.747360+0000","last_clean":"2024-04-26T01:48:22.747360+0000","last_became_active":"2024-04-26T01:47:27.116825+0000","last_became_peered":"2024-04-26T01:47:27.116825+0000","last_unstale":"2024-04-26T01:48:22.747360+0000","last_undegraded":"2024-04-26T01:48:22.747360+0000","last_fullsized":"2024-04-26T01:48:22.747360+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:23.313712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399301+0000","last_change":"2024-04-26T01:47:27.130233+0000","last_active":"2024-04-26T01:48:22.399301+0000","last_peered":"2024-04-26T01:48:22.399301+0000","last_clean":"2024-04-26T01:48:22.399301+0000","last_became_active":"2024-04-26T01:47:27.129852+0000","last_became_peered":"2024-04-26T01:47:27.129852+0000","last_unstale":"2024-04-26T01:48:22.399301+0000","last_undegraded":"2024-04-26T01:48:22.399301+0000","last_fullsized":"2024-04-26T01:48:22.399301+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:49:01.407264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.749092+0000","last_change":"2024-04-26T01:47:23.068477+0000","last_active":"2024-04-26T01:48:22.749092+0000","last_peered":"2024-04-26T01:48:22.749092+0000","last_clean":"2024-04-26T01:48:22.749092+0000","last_became_active":"2024-04-26T01:47:23.068061+0000","last_became_peered":"2024-04-26T01:47:23.068061+0000","last_unstale":"2024-04-26T01:48:22.749092+0000","last_undegraded":"2024-04-26T01:48:22.749092+0000","last_fullsized":"2024-04-26T01:48:22.749092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:02.153724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.406110+0000","last_change":"2024-04-26T01:47:27.130385+0000","last_active":"2024-04-26T01:48:22.406110+0000","last_peered":"2024-04-26T01:48:22.406110+0000","last_clean":"2024-04-26T01:48:22.406110+0000","last_became_active":"2024-04-26T01:47:27.129912+0000","last_became_peered":"2024-04-26T01:47:27.129912+0000","last_unstale":"2024-04-26T01:48:22.406110+0000","last_undegraded":"2024-04-26T01:48:22.406110+0000","last_fullsized":"2024-04-26T01:48:22.406110+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:26:03.010544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.407897+0000","last_change":"2024-04-26T01:47:23.110806+0000","last_active":"2024-04-26T01:48:22.407897+0000","last_peered":"2024-04-26T01:48:22.407897+0000","last_clean":"2024-04-26T01:48:22.407897+0000","last_became_active":"2024-04-26T01:47:23.110441+0000","last_became_peered":"2024-04-26T01:47:23.110441+0000","last_unstale":"2024-04-26T01:48:22.407897+0000","last_undegraded":"2024-04-26T01:48:22.407897+0000","last_fullsized":"2024-04-26T01:48:22.407897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:39.239172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.406491+0000","last_change":"2024-04-26T01:47:27.138628+0000","last_active":"2024-04-26T01:48:22.406491+0000","last_peered":"2024-04-26T01:48:22.406491+0000","last_clean":"2024-04-26T01:48:22.406491+0000","last_became_active":"2024-04-26T01:47:27.130483+0000","last_became_peered":"2024-04-26T01:47:27.130483+0000","last_unstale":"2024-04-26T01:48:22.406491+0000","last_undegraded":"2024-04-26T01:48:22.406491+0000","last_fullsized":"2024-04-26T01:48:22.406491+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:40:41.242856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.397138+0000","last_change":"2024-04-26T01:47:23.060066+0000","last_active":"2024-04-26T01:48:22.397138+0000","last_peered":"2024-04-26T01:48:22.397138+0000","last_clean":"2024-04-26T01:48:22.397138+0000","last_became_active":"2024-04-26T01:47:23.059526+0000","last_became_peered":"2024-04-26T01:47:23.059526+0000","last_unstale":"2024-04-26T01:48:22.397138+0000","last_undegraded":"2024-04-26T01:48:22.397138+0000","last_fullsized":"2024-04-26T01:48:22.397138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:28:22.623695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398426+0000","last_change":"2024-04-26T01:47:27.128343+0000","last_active":"2024-04-26T01:48:22.398426+0000","last_peered":"2024-04-26T01:48:22.398426+0000","last_clean":"2024-04-26T01:48:22.398426+0000","last_became_active":"2024-04-26T01:47:27.127920+0000","last_became_peered":"2024-04-26T01:47:27.127920+0000","last_unstale":"2024-04-26T01:48:22.398426+0000","last_undegraded":"2024-04-26T01:48:22.398426+0000","last_fullsized":"2024-04-26T01:48:22.398426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:06.725033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396937+0000","last_change":"2024-04-26T01:47:23.057783+0000","last_active":"2024-04-26T01:48:22.396937+0000","last_peered":"2024-04-26T01:48:22.396937+0000","last_clean":"2024-04-26T01:48:22.396937+0000","last_became_active":"2024-04-26T01:47:23.057391+0000","last_became_peered":"2024-04-26T01:47:23.057391+0000","last_unstale":"2024-04-26T01:48:22.396937+0000","last_undegraded":"2024-04-26T01:48:22.396937+0000","last_fullsized":"2024-04-26T01:48:22.396937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:10:10.050787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746112+0000","last_change":"2024-04-26T01:47:27.116458+0000","last_active":"2024-04-26T01:48:22.746112+0000","last_peered":"2024-04-26T01:48:22.746112+0000","last_clean":"2024-04-26T01:48:22.746112+0000","last_became_active":"2024-04-26T01:47:27.116084+0000","last_became_peered":"2024-04-26T01:47:27.116084+0000","last_unstale":"2024-04-26T01:48:22.746112+0000","last_undegraded":"2024-04-26T01:48:22.746112+0000","last_fullsized":"2024-04-26T01:48:22.746112+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:06:34.517478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":67,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745854+0000","last_change":"2024-04-26T01:47:23.060657+0000","last_active":"2024-04-26T01:48:22.745854+0000","last_peered":"2024-04-26T01:48:22.745854+0000","last_clean":"2024-04-26T01:48:22.745854+0000","last_became_active":"2024-04-26T01:47:23.060294+0000","last_became_peered":"2024-04-26T01:47:23.060294+0000","last_unstale":"2024-04-26T01:48:22.745854+0000","last_undegraded":"2024-04-26T01:48:22.745854+0000","last_fullsized":"2024-04-26T01:48:22.745854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:33.979717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396644+0000","last_change":"2024-04-26T01:47:27.120587+0000","last_active":"2024-04-26T01:48:22.396644+0000","last_peered":"2024-04-26T01:48:22.396644+0000","last_clean":"2024-04-26T01:48:22.396644+0000","last_became_active":"2024-04-26T01:47:27.120236+0000","last_became_peered":"2024-04-26T01:47:27.120236+0000","last_unstale":"2024-04-26T01:48:22.396644+0000","last_undegraded":"2024-04-26T01:48:22.396644+0000","last_fullsized":"2024-04-26T01:48:22.396644+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:30:38.488741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405079+0000","last_change":"2024-04-26T01:47:23.069395+0000","last_active":"2024-04-26T01:48:22.405079+0000","last_peered":"2024-04-26T01:48:22.405079+0000","last_clean":"2024-04-26T01:48:22.405079+0000","last_became_active":"2024-04-26T01:47:23.069102+0000","last_became_peered":"2024-04-26T01:47:23.069102+0000","last_unstale":"2024-04-26T01:48:22.405079+0000","last_undegraded":"2024-04-26T01:48:22.405079+0000","last_fullsized":"2024-04-26T01:48:22.405079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:45.162604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745788+0000","last_change":"2024-04-26T01:47:27.123859+0000","last_active":"2024-04-26T01:48:22.745788+0000","last_peered":"2024-04-26T01:48:22.745788+0000","last_clean":"2024-04-26T01:48:22.745788+0000","last_became_active":"2024-04-26T01:47:27.123438+0000","last_became_peered":"2024-04-26T01:47:27.123438+0000","last_unstale":"2024-04-26T01:48:22.745788+0000","last_undegraded":"2024-04-26T01:48:22.745788+0000","last_fullsized":"2024-04-26T01:48:22.745788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:33:53.542809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.401049+0000","last_change":"2024-04-26T01:47:23.071544+0000","last_active":"2024-04-26T01:48:22.401049+0000","last_peered":"2024-04-26T01:48:22.401049+0000","last_clean":"2024-04-26T01:48:22.401049+0000","last_became_active":"2024-04-26T01:47:23.071191+0000","last_became_peered":"2024-04-26T01:47:23.071191+0000","last_unstale":"2024-04-26T01:48:22.401049+0000","last_undegraded":"2024-04-26T01:48:22.401049+0000","last_fullsized":"2024-04-26T01:48:22.401049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:58.889561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404118+0000","last_change":"2024-04-26T01:47:27.131887+0000","last_active":"2024-04-26T01:48:22.404118+0000","last_peered":"2024-04-26T01:48:22.404118+0000","last_clean":"2024-04-26T01:48:22.404118+0000","last_became_active":"2024-04-26T01:47:27.127973+0000","last_became_peered":"2024-04-26T01:47:27.127973+0000","last_unstale":"2024-04-26T01:48:22.404118+0000","last_undegraded":"2024-04-26T01:48:22.404118+0000","last_fullsized":"2024-04-26T01:48:22.404118+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:54:35.305779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745599+0000","last_change":"2024-04-26T01:47:23.061353+0000","last_active":"2024-04-26T01:48:22.745599+0000","last_peered":"2024-04-26T01:48:22.745599+0000","last_clean":"2024-04-26T01:48:22.745599+0000","last_became_active":"2024-04-26T01:47:23.061037+0000","last_became_peered":"2024-04-26T01:47:23.061037+0000","last_unstale":"2024-04-26T01:48:22.745599+0000","last_undegraded":"2024-04-26T01:48:22.745599+0000","last_fullsized":"2024-04-26T01:48:22.745599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:13:59.617969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746265+0000","last_change":"2024-04-26T01:47:27.122303+0000","last_active":"2024-04-26T01:48:22.746265+0000","last_peered":"2024-04-26T01:48:22.746265+0000","last_clean":"2024-04-26T01:48:22.746265+0000","last_became_active":"2024-04-26T01:47:27.121919+0000","last_became_peered":"2024-04-26T01:47:27.121919+0000","last_unstale":"2024-04-26T01:48:22.746265+0000","last_undegraded":"2024-04-26T01:48:22.746265+0000","last_fullsized":"2024-04-26T01:48:22.746265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:21.676663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":66,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:25.309636+0000","last_change":"2024-04-26T01:47:23.057961+0000","last_active":"2024-04-26T01:48:25.309636+0000","last_peered":"2024-04-26T01:48:25.309636+0000","last_clean":"2024-04-26T01:48:25.309636+0000","last_became_active":"2024-04-26T01:47:23.057529+0000","last_became_peered":"2024-04-26T01:47:23.057529+0000","last_unstale":"2024-04-26T01:48:25.309636+0000","last_undegraded":"2024-04-26T01:48:25.309636+0000","last_fullsized":"2024-04-26T01:48:25.309636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:14.420265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398667+0000","last_change":"2024-04-26T01:47:27.120083+0000","last_active":"2024-04-26T01:48:22.398667+0000","last_peered":"2024-04-26T01:48:22.398667+0000","last_clean":"2024-04-26T01:48:22.398667+0000","last_became_active":"2024-04-26T01:47:27.118410+0000","last_became_peered":"2024-04-26T01:47:27.118410+0000","last_unstale":"2024-04-26T01:48:22.398667+0000","last_undegraded":"2024-04-26T01:48:22.398667+0000","last_fullsized":"2024-04-26T01:48:22.398667+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:28.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400220+0000","last_change":"2024-04-26T01:47:23.059221+0000","last_active":"2024-04-26T01:48:22.400220+0000","last_peered":"2024-04-26T01:48:22.400220+0000","last_clean":"2024-04-26T01:48:22.400220+0000","last_became_active":"2024-04-26T01:47:23.058881+0000","last_became_peered":"2024-04-26T01:47:23.058881+0000","last_unstale":"2024-04-26T01:48:22.400220+0000","last_undegraded":"2024-04-26T01:48:22.400220+0000","last_fullsized":"2024-04-26T01:48:22.400220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:30.433696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.406173+0000","last_change":"2024-04-26T01:47:27.136763+0000","last_active":"2024-04-26T01:48:22.406173+0000","last_peered":"2024-04-26T01:48:22.406173+0000","last_clean":"2024-04-26T01:48:22.406173+0000","last_became_active":"2024-04-26T01:47:27.133642+0000","last_became_peered":"2024-04-26T01:47:27.133642+0000","last_unstale":"2024-04-26T01:48:22.406173+0000","last_undegraded":"2024-04-26T01:48:22.406173+0000","last_fullsized":"2024-04-26T01:48:22.406173+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:05.901139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746094+0000","last_change":"2024-04-26T01:47:23.108883+0000","last_active":"2024-04-26T01:48:22.746094+0000","last_peered":"2024-04-26T01:48:22.746094+0000","last_clean":"2024-04-26T01:48:22.746094+0000","last_became_active":"2024-04-26T01:47:23.108499+0000","last_became_peered":"2024-04-26T01:47:23.108499+0000","last_unstale":"2024-04-26T01:48:22.746094+0000","last_undegraded":"2024-04-26T01:48:22.746094+0000","last_fullsized":"2024-04-26T01:48:22.746094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:14:17.555416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.410651+0000","last_change":"2024-04-26T01:48:22.410651+0000","last_active":"2024-04-26T01:48:22.410651+0000","last_peered":"2024-04-26T01:48:22.410651+0000","last_clean":"2024-04-26T01:48:22.410651+0000","last_became_active":"2024-04-26T01:48:22.410359+0000","last_became_peered":"2024-04-26T01:48:22.410359+0000","last_unstale":"2024-04-26T01:48:22.410651+0000","last_undegraded":"2024-04-26T01:48:22.410651+0000","last_fullsized":"2024-04-26T01:48:22.410651+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:53:20.686915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'29","reported_seq":138,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:26.316864+0000","last_change":"2024-04-26T01:47:23.054893+0000","last_active":"2024-04-26T01:48:26.316864+0000","last_peered":"2024-04-26T01:48:26.316864+0000","last_clean":"2024-04-26T01:48:26.316864+0000","last_became_active":"2024-04-26T01:47:23.054558+0000","last_became_peered":"2024-04-26T01:47:23.054558+0000","last_unstale":"2024-04-26T01:48:26.316864+0000","last_undegraded":"2024-04-26T01:48:26.316864+0000","last_fullsized":"2024-04-26T01:48:26.316864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:59.356841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60347,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":83,"num_read_kb":100,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396887+0000","last_change":"2024-04-26T01:47:27.116934+0000","last_active":"2024-04-26T01:48:22.396887+0000","last_peered":"2024-04-26T01:48:22.396887+0000","last_clean":"2024-04-26T01:48:22.396887+0000","last_became_active":"2024-04-26T01:47:27.116527+0000","last_became_peered":"2024-04-26T01:47:27.116527+0000","last_unstale":"2024-04-26T01:48:22.396887+0000","last_undegraded":"2024-04-26T01:48:22.396887+0000","last_fullsized":"2024-04-26T01:48:22.396887+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:09:45.820533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399072+0000","last_change":"2024-04-26T01:47:23.059032+0000","last_active":"2024-04-26T01:48:22.399072+0000","last_peered":"2024-04-26T01:48:22.399072+0000","last_clean":"2024-04-26T01:48:22.399072+0000","last_became_active":"2024-04-26T01:47:23.057992+0000","last_became_peered":"2024-04-26T01:47:23.057992+0000","last_unstale":"2024-04-26T01:48:22.399072+0000","last_undegraded":"2024-04-26T01:48:22.399072+0000","last_fullsized":"2024-04-26T01:48:22.399072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:46:53.164925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.402515+0000","last_change":"2024-04-26T01:47:27.134574+0000","last_active":"2024-04-26T01:48:22.402515+0000","last_peered":"2024-04-26T01:48:22.402515+0000","last_clean":"2024-04-26T01:48:22.402515+0000","last_became_active":"2024-04-26T01:47:27.134184+0000","last_became_peered":"2024-04-26T01:47:27.134184+0000","last_unstale":"2024-04-26T01:48:22.402515+0000","last_undegraded":"2024-04-26T01:48:22.402515+0000","last_fullsized":"2024-04-26T01:48:22.402515+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:42:26.428014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745665+0000","last_change":"2024-04-26T01:47:23.062649+0000","last_active":"2024-04-26T01:48:22.745665+0000","last_peered":"2024-04-26T01:48:22.745665+0000","last_clean":"2024-04-26T01:48:22.745665+0000","last_became_active":"2024-04-26T01:47:23.062282+0000","last_became_peered":"2024-04-26T01:47:23.062282+0000","last_unstale":"2024-04-26T01:48:22.745665+0000","last_undegraded":"2024-04-26T01:48:22.745665+0000","last_fullsized":"2024-04-26T01:48:22.745665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:38.699105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400362+0000","last_change":"2024-04-26T01:47:27.124720+0000","last_active":"2024-04-26T01:48:22.400362+0000","last_peered":"2024-04-26T01:48:22.400362+0000","last_clean":"2024-04-26T01:48:22.400362+0000","last_became_active":"2024-04-26T01:47:27.124378+0000","last_became_peered":"2024-04-26T01:47:27.124378+0000","last_unstale":"2024-04-26T01:48:22.400362+0000","last_undegraded":"2024-04-26T01:48:22.400362+0000","last_fullsized":"2024-04-26T01:48:22.400362+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:45.462741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":67,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746010+0000","last_change":"2024-04-26T01:47:23.056305+0000","last_active":"2024-04-26T01:48:22.746010+0000","last_peered":"2024-04-26T01:48:22.746010+0000","last_clean":"2024-04-26T01:48:22.746010+0000","last_became_active":"2024-04-26T01:47:23.055905+0000","last_became_peered":"2024-04-26T01:47:23.055905+0000","last_unstale":"2024-04-26T01:48:22.746010+0000","last_undegraded":"2024-04-26T01:48:22.746010+0000","last_fullsized":"2024-04-26T01:48:22.746010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:00:14.609673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400015+0000","last_change":"2024-04-26T01:47:27.119880+0000","last_active":"2024-04-26T01:48:22.400015+0000","last_peered":"2024-04-26T01:48:22.400015+0000","last_clean":"2024-04-26T01:48:22.400015+0000","last_became_active":"2024-04-26T01:47:27.119483+0000","last_became_peered":"2024-04-26T01:47:27.119483+0000","last_unstale":"2024-04-26T01:48:22.400015+0000","last_undegraded":"2024-04-26T01:48:22.400015+0000","last_fullsized":"2024-04-26T01:48:22.400015+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:15.523150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":42,"reported_epoch":92,"state":"activating+degraded","last_fresh":"2024-04-26T01:48:22.397136+0000","last_change":"2024-04-26T01:48:22.410077+0000","last_active":"2024-04-26T01:48:21.388233+0000","last_peered":"2024-04-26T01:47:50.255243+0000","last_clean":"2024-04-26T01:47:50.255243+0000","last_became_active":"2024-04-26T01:47:23.109192+0000","last_became_peered":"2024-04-26T01:47:23.109192+0000","last_unstale":"2024-04-26T01:48:22.397136+0000","last_undegraded":"2024-04-26T01:48:22.397136+0000","last_fullsized":"2024-04-26T01:48:22.397136+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":["8","0","2"],"object_location_counts":[{"shards":"0,2,8","objects":8}],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745649+0000","last_change":"2024-04-26T01:47:27.113508+0000","last_active":"2024-04-26T01:48:22.745649+0000","last_peered":"2024-04-26T01:48:22.745649+0000","last_clean":"2024-04-26T01:48:22.745649+0000","last_became_active":"2024-04-26T01:47:27.113066+0000","last_became_peered":"2024-04-26T01:47:27.113066+0000","last_unstale":"2024-04-26T01:48:22.745649+0000","last_undegraded":"2024-04-26T01:48:22.745649+0000","last_fullsized":"2024-04-26T01:48:22.745649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:37.525486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.397062+0000","last_change":"2024-04-26T01:47:23.059644+0000","last_active":"2024-04-26T01:48:22.397062+0000","last_peered":"2024-04-26T01:48:22.397062+0000","last_clean":"2024-04-26T01:48:22.397062+0000","last_became_active":"2024-04-26T01:47:23.058966+0000","last_became_peered":"2024-04-26T01:47:23.058966+0000","last_unstale":"2024-04-26T01:48:22.397062+0000","last_undegraded":"2024-04-26T01:48:22.397062+0000","last_fullsized":"2024-04-26T01:48:22.397062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:57:36.475935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.749354+0000","last_change":"2024-04-26T01:48:22.749354+0000","last_active":"2024-04-26T01:48:22.749354+0000","last_peered":"2024-04-26T01:48:22.749354+0000","last_clean":"2024-04-26T01:48:22.749354+0000","last_became_active":"2024-04-26T01:48:22.748978+0000","last_became_peered":"2024-04-26T01:48:22.748978+0000","last_unstale":"2024-04-26T01:48:22.749354+0000","last_undegraded":"2024-04-26T01:48:22.749354+0000","last_fullsized":"2024-04-26T01:48:22.749354+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:51:16.087091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746667+0000","last_change":"2024-04-26T01:47:23.110360+0000","last_active":"2024-04-26T01:48:22.746667+0000","last_peered":"2024-04-26T01:48:22.746667+0000","last_clean":"2024-04-26T01:48:22.746667+0000","last_became_active":"2024-04-26T01:47:23.109744+0000","last_became_peered":"2024-04-26T01:47:23.109744+0000","last_unstale":"2024-04-26T01:48:22.746667+0000","last_undegraded":"2024-04-26T01:48:22.746667+0000","last_fullsized":"2024-04-26T01:48:22.746667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:40:13.658396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405212+0000","last_change":"2024-04-26T01:47:27.125399+0000","last_active":"2024-04-26T01:48:22.405212+0000","last_peered":"2024-04-26T01:48:22.405212+0000","last_clean":"2024-04-26T01:48:22.405212+0000","last_became_active":"2024-04-26T01:47:27.124871+0000","last_became_peered":"2024-04-26T01:47:27.124871+0000","last_unstale":"2024-04-26T01:48:22.405212+0000","last_undegraded":"2024-04-26T01:48:22.405212+0000","last_fullsized":"2024-04-26T01:48:22.405212+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:28:48.285206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400178+0000","last_change":"2024-04-26T01:47:23.067862+0000","last_active":"2024-04-26T01:48:22.400178+0000","last_peered":"2024-04-26T01:48:22.400178+0000","last_clean":"2024-04-26T01:48:22.400178+0000","last_became_active":"2024-04-26T01:47:23.067526+0000","last_became_peered":"2024-04-26T01:47:23.067526+0000","last_unstale":"2024-04-26T01:48:22.400178+0000","last_undegraded":"2024-04-26T01:48:22.400178+0000","last_fullsized":"2024-04-26T01:48:22.400178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:22.668405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.748548+0000","last_change":"2024-04-26T01:48:22.748548+0000","last_active":"2024-04-26T01:48:22.748548+0000","last_peered":"2024-04-26T01:48:22.748548+0000","last_clean":"2024-04-26T01:48:22.748548+0000","last_became_active":"2024-04-26T01:48:22.748182+0000","last_became_peered":"2024-04-26T01:48:22.748182+0000","last_unstale":"2024-04-26T01:48:22.748548+0000","last_undegraded":"2024-04-26T01:48:22.748548+0000","last_fullsized":"2024-04-26T01:48:22.748548+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:20:27.596827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745783+0000","last_change":"2024-04-26T01:47:27.118307+0000","last_active":"2024-04-26T01:48:22.745783+0000","last_peered":"2024-04-26T01:48:22.745783+0000","last_clean":"2024-04-26T01:48:22.745783+0000","last_became_active":"2024-04-26T01:47:27.117986+0000","last_became_peered":"2024-04-26T01:47:27.117986+0000","last_unstale":"2024-04-26T01:48:22.745783+0000","last_undegraded":"2024-04-26T01:48:22.745783+0000","last_fullsized":"2024-04-26T01:48:22.745783+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:40.667143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":66,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:24.324415+0000","last_change":"2024-04-26T01:47:23.109973+0000","last_active":"2024-04-26T01:48:24.324415+0000","last_peered":"2024-04-26T01:48:24.324415+0000","last_clean":"2024-04-26T01:48:24.324415+0000","last_became_active":"2024-04-26T01:47:23.109446+0000","last_became_peered":"2024-04-26T01:47:23.109446+0000","last_unstale":"2024-04-26T01:48:24.324415+0000","last_undegraded":"2024-04-26T01:48:24.324415+0000","last_fullsized":"2024-04-26T01:48:24.324415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:49:00.684164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.408237+0000","last_change":"2024-04-26T01:47:27.131502+0000","last_active":"2024-04-26T01:48:22.408237+0000","last_peered":"2024-04-26T01:48:22.408237+0000","last_clean":"2024-04-26T01:48:22.408237+0000","last_became_active":"2024-04-26T01:47:27.127697+0000","last_became_peered":"2024-04-26T01:47:27.127697+0000","last_unstale":"2024-04-26T01:48:22.408237+0000","last_undegraded":"2024-04-26T01:48:22.408237+0000","last_fullsized":"2024-04-26T01:48:22.408237+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:39:15.458189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746705+0000","last_change":"2024-04-26T01:47:23.066596+0000","last_active":"2024-04-26T01:48:22.746705+0000","last_peered":"2024-04-26T01:48:22.746705+0000","last_clean":"2024-04-26T01:48:22.746705+0000","last_became_active":"2024-04-26T01:47:23.066216+0000","last_became_peered":"2024-04-26T01:47:23.066216+0000","last_unstale":"2024-04-26T01:48:22.746705+0000","last_undegraded":"2024-04-26T01:48:22.746705+0000","last_fullsized":"2024-04-26T01:48:22.746705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:08:31.544831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":359,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400108+0000","last_change":"2024-04-26T01:45:54.834501+0000","last_active":"2024-04-26T01:48:22.400108+0000","last_peered":"2024-04-26T01:48:22.400108+0000","last_clean":"2024-04-26T01:48:22.400108+0000","last_became_active":"2024-04-26T01:45:54.833868+0000","last_became_peered":"2024-04-26T01:45:54.833868+0000","last_unstale":"2024-04-26T01:48:22.400108+0000","last_undegraded":"2024-04-26T01:48:22.400108+0000","last_fullsized":"2024-04-26T01:48:22.400108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:42:05.228956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:42:05.228956+0000","last_clean_scrub_stamp":"2024-04-26T01:42:05.228956+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:06:58.383315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400398+0000","last_change":"2024-04-26T01:47:27.126475+0000","last_active":"2024-04-26T01:48:22.400398+0000","last_peered":"2024-04-26T01:48:22.400398+0000","last_clean":"2024-04-26T01:48:22.400398+0000","last_became_active":"2024-04-26T01:47:27.123760+0000","last_became_peered":"2024-04-26T01:47:27.123760+0000","last_unstale":"2024-04-26T01:48:22.400398+0000","last_undegraded":"2024-04-26T01:48:22.400398+0000","last_fullsized":"2024-04-26T01:48:22.400398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:38:27.473909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400221+0000","last_change":"2024-04-26T01:47:23.059810+0000","last_active":"2024-04-26T01:48:22.400221+0000","last_peered":"2024-04-26T01:48:22.400221+0000","last_clean":"2024-04-26T01:48:22.400221+0000","last_became_active":"2024-04-26T01:47:23.059504+0000","last_became_peered":"2024-04-26T01:47:23.059504+0000","last_unstale":"2024-04-26T01:48:22.400221+0000","last_undegraded":"2024-04-26T01:48:22.400221+0000","last_fullsized":"2024-04-26T01:48:22.400221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:50:51.220951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396987+0000","last_change":"2024-04-26T01:47:23.109460+0000","last_active":"2024-04-26T01:48:22.396987+0000","last_peered":"2024-04-26T01:48:22.396987+0000","last_clean":"2024-04-26T01:48:22.396987+0000","last_became_active":"2024-04-26T01:47:23.109090+0000","last_became_peered":"2024-04-26T01:47:23.109090+0000","last_unstale":"2024-04-26T01:48:22.396987+0000","last_undegraded":"2024-04-26T01:48:22.396987+0000","last_fullsized":"2024-04-26T01:48:22.396987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:56.821312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.407355+0000","last_change":"2024-04-26T01:47:27.129537+0000","last_active":"2024-04-26T01:48:22.407355+0000","last_peered":"2024-04-26T01:48:22.407355+0000","last_clean":"2024-04-26T01:48:22.407355+0000","last_became_active":"2024-04-26T01:47:27.128124+0000","last_became_peered":"2024-04-26T01:47:27.128124+0000","last_unstale":"2024-04-26T01:48:22.407355+0000","last_undegraded":"2024-04-26T01:48:22.407355+0000","last_fullsized":"2024-04-26T01:48:22.407355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:14:26.262197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'11","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.747472+0000","last_change":"2024-04-26T01:47:23.064511+0000","last_active":"2024-04-26T01:48:22.747472+0000","last_peered":"2024-04-26T01:48:22.747472+0000","last_clean":"2024-04-26T01:48:22.747472+0000","last_became_active":"2024-04-26T01:47:23.064080+0000","last_became_peered":"2024-04-26T01:47:23.064080+0000","last_unstale":"2024-04-26T01:48:22.747472+0000","last_undegraded":"2024-04-26T01:48:22.747472+0000","last_fullsized":"2024-04-26T01:48:22.747472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:37.562644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404336+0000","last_change":"2024-04-26T01:47:27.124057+0000","last_active":"2024-04-26T01:48:22.404336+0000","last_peered":"2024-04-26T01:48:22.404336+0000","last_clean":"2024-04-26T01:48:22.404336+0000","last_became_active":"2024-04-26T01:47:27.123636+0000","last_became_peered":"2024-04-26T01:47:27.123636+0000","last_unstale":"2024-04-26T01:48:22.404336+0000","last_undegraded":"2024-04-26T01:48:22.404336+0000","last_fullsized":"2024-04-26T01:48:22.404336+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:56:55.691697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746155+0000","last_change":"2024-04-26T01:47:23.136571+0000","last_active":"2024-04-26T01:48:22.746155+0000","last_peered":"2024-04-26T01:48:22.746155+0000","last_clean":"2024-04-26T01:48:22.746155+0000","last_became_active":"2024-04-26T01:47:23.136205+0000","last_became_peered":"2024-04-26T01:47:23.136205+0000","last_unstale":"2024-04-26T01:48:22.746155+0000","last_undegraded":"2024-04-26T01:48:22.746155+0000","last_fullsized":"2024-04-26T01:48:22.746155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:24.418797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.747570+0000","last_change":"2024-04-26T01:48:22.747570+0000","last_active":"2024-04-26T01:48:22.747570+0000","last_peered":"2024-04-26T01:48:22.747570+0000","last_clean":"2024-04-26T01:48:22.747570+0000","last_became_active":"2024-04-26T01:48:22.747147+0000","last_became_peered":"2024-04-26T01:48:22.747147+0000","last_unstale":"2024-04-26T01:48:22.747570+0000","last_undegraded":"2024-04-26T01:48:22.747570+0000","last_fullsized":"2024-04-26T01:48:22.747570+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:37.034388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.414610+0000","last_change":"2024-04-26T01:48:22.414610+0000","last_active":"2024-04-26T01:48:22.414610+0000","last_peered":"2024-04-26T01:48:22.414610+0000","last_clean":"2024-04-26T01:48:22.414610+0000","last_became_active":"2024-04-26T01:48:22.410275+0000","last_became_peered":"2024-04-26T01:48:22.410275+0000","last_unstale":"2024-04-26T01:48:22.414610+0000","last_undegraded":"2024-04-26T01:48:22.414610+0000","last_fullsized":"2024-04-26T01:48:22.414610+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:20:48.131988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396626+0000","last_change":"2024-04-26T01:47:27.115498+0000","last_active":"2024-04-26T01:48:22.396626+0000","last_peered":"2024-04-26T01:48:22.396626+0000","last_clean":"2024-04-26T01:48:22.396626+0000","last_became_active":"2024-04-26T01:47:27.115134+0000","last_became_peered":"2024-04-26T01:47:27.115134+0000","last_unstale":"2024-04-26T01:48:22.396626+0000","last_undegraded":"2024-04-26T01:48:22.396626+0000","last_fullsized":"2024-04-26T01:48:22.396626+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:20.795224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405265+0000","last_change":"2024-04-26T01:47:23.065366+0000","last_active":"2024-04-26T01:48:22.405265+0000","last_peered":"2024-04-26T01:48:22.405265+0000","last_clean":"2024-04-26T01:48:22.405265+0000","last_became_active":"2024-04-26T01:47:23.063216+0000","last_became_peered":"2024-04-26T01:47:23.063216+0000","last_unstale":"2024-04-26T01:48:22.405265+0000","last_undegraded":"2024-04-26T01:48:22.405265+0000","last_fullsized":"2024-04-26T01:48:22.405265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:06:05.261907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398405+0000","last_change":"2024-04-26T01:47:27.127938+0000","last_active":"2024-04-26T01:48:22.398405+0000","last_peered":"2024-04-26T01:48:22.398405+0000","last_clean":"2024-04-26T01:48:22.398405+0000","last_became_active":"2024-04-26T01:47:27.127553+0000","last_became_peered":"2024-04-26T01:47:27.127553+0000","last_unstale":"2024-04-26T01:48:22.398405+0000","last_undegraded":"2024-04-26T01:48:22.398405+0000","last_fullsized":"2024-04-26T01:48:22.398405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:17:51.115349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398731+0000","last_change":"2024-04-26T01:47:23.060475+0000","last_active":"2024-04-26T01:48:22.398731+0000","last_peered":"2024-04-26T01:48:22.398731+0000","last_clean":"2024-04-26T01:48:22.398731+0000","last_became_active":"2024-04-26T01:47:23.060098+0000","last_became_peered":"2024-04-26T01:47:23.060098+0000","last_unstale":"2024-04-26T01:48:22.398731+0000","last_undegraded":"2024-04-26T01:48:22.398731+0000","last_fullsized":"2024-04-26T01:48:22.398731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:49.009455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.407796+0000","last_change":"2024-04-26T01:47:27.137491+0000","last_active":"2024-04-26T01:48:22.407796+0000","last_peered":"2024-04-26T01:48:22.407796+0000","last_clean":"2024-04-26T01:48:22.407796+0000","last_became_active":"2024-04-26T01:47:27.137059+0000","last_became_peered":"2024-04-26T01:47:27.137059+0000","last_unstale":"2024-04-26T01:48:22.407796+0000","last_undegraded":"2024-04-26T01:48:22.407796+0000","last_fullsized":"2024-04-26T01:48:22.407796+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:20:25.004074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405524+0000","last_change":"2024-04-26T01:48:22.405071+0000","last_active":"2024-04-26T01:48:22.405524+0000","last_peered":"2024-04-26T01:48:22.405524+0000","last_clean":"2024-04-26T01:48:22.405524+0000","last_became_active":"2024-04-26T01:48:22.401282+0000","last_became_peered":"2024-04-26T01:48:22.401282+0000","last_unstale":"2024-04-26T01:48:22.405524+0000","last_undegraded":"2024-04-26T01:48:22.405524+0000","last_fullsized":"2024-04-26T01:48:22.405524+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:24:53.628014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400294+0000","last_change":"2024-04-26T01:47:27.128253+0000","last_active":"2024-04-26T01:48:22.400294+0000","last_peered":"2024-04-26T01:48:22.400294+0000","last_clean":"2024-04-26T01:48:22.400294+0000","last_became_active":"2024-04-26T01:47:27.127869+0000","last_became_peered":"2024-04-26T01:47:27.127869+0000","last_unstale":"2024-04-26T01:48:22.400294+0000","last_undegraded":"2024-04-26T01:48:22.400294+0000","last_fullsized":"2024-04-26T01:48:22.400294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:11:23.476938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399412+0000","last_change":"2024-04-26T01:47:23.136862+0000","last_active":"2024-04-26T01:48:22.399412+0000","last_peered":"2024-04-26T01:48:22.399412+0000","last_clean":"2024-04-26T01:48:22.399412+0000","last_became_active":"2024-04-26T01:47:23.136380+0000","last_became_peered":"2024-04-26T01:47:23.136380+0000","last_unstale":"2024-04-26T01:48:22.399412+0000","last_undegraded":"2024-04-26T01:48:22.399412+0000","last_fullsized":"2024-04-26T01:48:22.399412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:39:18.103559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400366+0000","last_change":"2024-04-26T01:47:27.113363+0000","last_active":"2024-04-26T01:48:22.400366+0000","last_peered":"2024-04-26T01:48:22.400366+0000","last_clean":"2024-04-26T01:48:22.400366+0000","last_became_active":"2024-04-26T01:47:27.113040+0000","last_became_peered":"2024-04-26T01:47:27.113040+0000","last_unstale":"2024-04-26T01:48:22.400366+0000","last_undegraded":"2024-04-26T01:48:22.400366+0000","last_fullsized":"2024-04-26T01:48:22.400366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:42:16.428697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745984+0000","last_change":"2024-04-26T01:47:23.067257+0000","last_active":"2024-04-26T01:48:22.745984+0000","last_peered":"2024-04-26T01:48:22.745984+0000","last_clean":"2024-04-26T01:48:22.745984+0000","last_became_active":"2024-04-26T01:47:23.066858+0000","last_became_peered":"2024-04-26T01:47:23.066858+0000","last_unstale":"2024-04-26T01:48:22.745984+0000","last_undegraded":"2024-04-26T01:48:22.745984+0000","last_fullsized":"2024-04-26T01:48:22.745984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:22:07.848074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400154+0000","last_change":"2024-04-26T01:47:27.125885+0000","last_active":"2024-04-26T01:48:22.400154+0000","last_peered":"2024-04-26T01:48:22.400154+0000","last_clean":"2024-04-26T01:48:22.400154+0000","last_became_active":"2024-04-26T01:47:27.125524+0000","last_became_peered":"2024-04-26T01:47:27.125524+0000","last_unstale":"2024-04-26T01:48:22.400154+0000","last_undegraded":"2024-04-26T01:48:22.400154+0000","last_fullsized":"2024-04-26T01:48:22.400154+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:39.282023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.407608+0000","last_change":"2024-04-26T01:47:23.071051+0000","last_active":"2024-04-26T01:48:22.407608+0000","last_peered":"2024-04-26T01:48:22.407608+0000","last_clean":"2024-04-26T01:48:22.407608+0000","last_became_active":"2024-04-26T01:47:23.069524+0000","last_became_peered":"2024-04-26T01:47:23.069524+0000","last_unstale":"2024-04-26T01:48:22.407608+0000","last_undegraded":"2024-04-26T01:48:22.407608+0000","last_fullsized":"2024-04-26T01:48:22.407608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:24:57.643100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400370+0000","last_change":"2024-04-26T01:47:27.114325+0000","last_active":"2024-04-26T01:48:22.400370+0000","last_peered":"2024-04-26T01:48:22.400370+0000","last_clean":"2024-04-26T01:48:22.400370+0000","last_became_active":"2024-04-26T01:47:27.113952+0000","last_became_peered":"2024-04-26T01:47:27.113952+0000","last_unstale":"2024-04-26T01:48:22.400370+0000","last_undegraded":"2024-04-26T01:48:22.400370+0000","last_fullsized":"2024-04-26T01:48:22.400370+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:39:19.167175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398411+0000","last_change":"2024-04-26T01:47:27.118709+0000","last_active":"2024-04-26T01:48:22.398411+0000","last_peered":"2024-04-26T01:48:22.398411+0000","last_clean":"2024-04-26T01:48:22.398411+0000","last_became_active":"2024-04-26T01:47:27.118341+0000","last_became_peered":"2024-04-26T01:47:27.118341+0000","last_unstale":"2024-04-26T01:48:22.398411+0000","last_undegraded":"2024-04-26T01:48:22.398411+0000","last_fullsized":"2024-04-26T01:48:22.398411+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:13.736180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399082+0000","last_change":"2024-04-26T01:47:27.119239+0000","last_active":"2024-04-26T01:48:22.399082+0000","last_peered":"2024-04-26T01:48:22.399082+0000","last_clean":"2024-04-26T01:48:22.399082+0000","last_became_active":"2024-04-26T01:47:27.118877+0000","last_became_peered":"2024-04-26T01:47:27.118877+0000","last_unstale":"2024-04-26T01:48:22.399082+0000","last_undegraded":"2024-04-26T01:48:22.399082+0000","last_fullsized":"2024-04-26T01:48:22.399082+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:04:10.033517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398635+0000","last_change":"2024-04-26T01:47:27.142848+0000","last_active":"2024-04-26T01:48:22.398635+0000","last_peered":"2024-04-26T01:48:22.398635+0000","last_clean":"2024-04-26T01:48:22.398635+0000","last_became_active":"2024-04-26T01:47:27.142580+0000","last_became_peered":"2024-04-26T01:47:27.142580+0000","last_unstale":"2024-04-26T01:48:22.398635+0000","last_undegraded":"2024-04-26T01:48:22.398635+0000","last_fullsized":"2024-04-26T01:48:22.398635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:15:46.923922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404760+0000","last_change":"2024-04-26T01:47:27.134817+0000","last_active":"2024-04-26T01:48:22.404760+0000","last_peered":"2024-04-26T01:48:22.404760+0000","last_clean":"2024-04-26T01:48:22.404760+0000","last_became_active":"2024-04-26T01:47:27.134024+0000","last_became_peered":"2024-04-26T01:47:27.134024+0000","last_unstale":"2024-04-26T01:48:22.404760+0000","last_undegraded":"2024-04-26T01:48:22.404760+0000","last_fullsized":"2024-04-26T01:48:22.404760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:56.746427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405975+0000","last_change":"2024-04-26T01:47:27.143137+0000","last_active":"2024-04-26T01:48:22.405975+0000","last_peered":"2024-04-26T01:48:22.405975+0000","last_clean":"2024-04-26T01:48:22.405975+0000","last_became_active":"2024-04-26T01:47:27.142625+0000","last_became_peered":"2024-04-26T01:47:27.142625+0000","last_unstale":"2024-04-26T01:48:22.405975+0000","last_undegraded":"2024-04-26T01:48:22.405975+0000","last_fullsized":"2024-04-26T01:48:22.405975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:42:50.468382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.749582+0000","last_change":"2024-04-26T01:48:22.749582+0000","last_active":"2024-04-26T01:48:22.749582+0000","last_peered":"2024-04-26T01:48:22.749582+0000","last_clean":"2024-04-26T01:48:22.749582+0000","last_became_active":"2024-04-26T01:48:22.749191+0000","last_became_peered":"2024-04-26T01:48:22.749191+0000","last_unstale":"2024-04-26T01:48:22.749582+0000","last_undegraded":"2024-04-26T01:48:22.749582+0000","last_fullsized":"2024-04-26T01:48:22.749582+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:14.614842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399646+0000","last_change":"2024-04-26T01:47:27.123964+0000","last_active":"2024-04-26T01:48:22.399646+0000","last_peered":"2024-04-26T01:48:22.399646+0000","last_clean":"2024-04-26T01:48:22.399646+0000","last_became_active":"2024-04-26T01:47:27.123552+0000","last_became_peered":"2024-04-26T01:47:27.123552+0000","last_unstale":"2024-04-26T01:48:22.399646+0000","last_undegraded":"2024-04-26T01:48:22.399646+0000","last_fullsized":"2024-04-26T01:48:22.399646+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:33:12.566105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.412621+0000","last_change":"2024-04-26T01:48:22.412621+0000","last_active":"2024-04-26T01:48:22.412621+0000","last_peered":"2024-04-26T01:48:22.412621+0000","last_clean":"2024-04-26T01:48:22.412621+0000","last_became_active":"2024-04-26T01:48:22.411430+0000","last_became_peered":"2024-04-26T01:48:22.411430+0000","last_unstale":"2024-04-26T01:48:22.412621+0000","last_undegraded":"2024-04-26T01:48:22.412621+0000","last_fullsized":"2024-04-26T01:48:22.412621+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:34.152338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745667+0000","last_change":"2024-04-26T01:47:27.131437+0000","last_active":"2024-04-26T01:48:22.745667+0000","last_peered":"2024-04-26T01:48:22.745667+0000","last_clean":"2024-04-26T01:48:22.745667+0000","last_became_active":"2024-04-26T01:47:27.131138+0000","last_became_peered":"2024-04-26T01:47:27.131138+0000","last_unstale":"2024-04-26T01:48:22.745667+0000","last_undegraded":"2024-04-26T01:48:22.745667+0000","last_fullsized":"2024-04-26T01:48:22.745667+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:55:39.132972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404958+0000","last_change":"2024-04-26T01:47:27.120928+0000","last_active":"2024-04-26T01:48:22.404958+0000","last_peered":"2024-04-26T01:48:22.404958+0000","last_clean":"2024-04-26T01:48:22.404958+0000","last_became_active":"2024-04-26T01:47:27.120012+0000","last_became_peered":"2024-04-26T01:47:27.120012+0000","last_unstale":"2024-04-26T01:48:22.404958+0000","last_undegraded":"2024-04-26T01:48:22.404958+0000","last_fullsized":"2024-04-26T01:48:22.404958+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:19:44.209876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398493+0000","last_change":"2024-04-26T01:47:27.138766+0000","last_active":"2024-04-26T01:48:22.398493+0000","last_peered":"2024-04-26T01:48:22.398493+0000","last_clean":"2024-04-26T01:48:22.398493+0000","last_became_active":"2024-04-26T01:47:27.138413+0000","last_became_peered":"2024-04-26T01:47:27.138413+0000","last_unstale":"2024-04-26T01:48:22.398493+0000","last_undegraded":"2024-04-26T01:48:22.398493+0000","last_fullsized":"2024-04-26T01:48:22.398493+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:51:13.117307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.406315+0000","last_change":"2024-04-26T01:47:27.141338+0000","last_active":"2024-04-26T01:48:22.406315+0000","last_peered":"2024-04-26T01:48:22.406315+0000","last_clean":"2024-04-26T01:48:22.406315+0000","last_became_active":"2024-04-26T01:47:27.140825+0000","last_became_peered":"2024-04-26T01:47:27.140825+0000","last_unstale":"2024-04-26T01:48:22.406315+0000","last_undegraded":"2024-04-26T01:48:22.406315+0000","last_fullsized":"2024-04-26T01:48:22.406315+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:12:08.680876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398625+0000","last_change":"2024-04-26T01:47:27.114919+0000","last_active":"2024-04-26T01:48:22.398625+0000","last_peered":"2024-04-26T01:48:22.398625+0000","last_clean":"2024-04-26T01:48:22.398625+0000","last_became_active":"2024-04-26T01:47:27.114528+0000","last_became_peered":"2024-04-26T01:47:27.114528+0000","last_unstale":"2024-04-26T01:48:22.398625+0000","last_undegraded":"2024-04-26T01:48:22.398625+0000","last_fullsized":"2024-04-26T01:48:22.398625+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:15:25.620566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64348,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":241,"num_read_kb":190,"num_write":163,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318299,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":490347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579936,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28688,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716464,"statfs":{"total":95995035648,"available":95965659136,"internally_reserved":0,"allocated":1708032,"data_stored":1142566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743462,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":490583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005101,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1011712,"data_stored":481986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299441,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1622016,"data_stored":1082433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":489972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1060864,"data_stored":541595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":491243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":551350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":490283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":490912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724606,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32776,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712376,"statfs":{"total":95995035648,"available":95961473024,"internally_reserved":0,"allocated":1634304,"data_stored":1081726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:48:28.638 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:48:28.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:28 smithi001 ceph-mon[27822]: from='client.24667 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:48:28.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:28.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:28.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:28 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:48:29.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:28 smithi173 ceph-mon[32553]: from='client.24667 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:48:29.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:29.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:29.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:28 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:48:29.145 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:29.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:28 smithi097 ceph-mon[32853]: from='client.24667 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:48:29.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:29.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:29.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:28 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:48:29.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:29 smithi001 ceph-mon[27822]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 6 op/s; 13 B/s, 1 objects/s recovering 2024-04-26T01:48:30.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:29 smithi173 ceph-mon[32553]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 6 op/s; 13 B/s, 1 objects/s recovering 2024-04-26T01:48:30.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:29 smithi097 ceph-mon[32853]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 6 op/s; 13 B/s, 1 objects/s recovering 2024-04-26T01:48:31.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:30 smithi173 ceph-mon[32553]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 8/192 objects degraded (4.167%), 1 pg degraded) 2024-04-26T01:48:31.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:30 smithi173 ceph-mon[32553]: Cluster is now healthy 2024-04-26T01:48:31.096 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:30 smithi001 ceph-mon[27822]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 8/192 objects degraded (4.167%), 1 pg degraded) 2024-04-26T01:48:31.097 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:30 smithi001 ceph-mon[27822]: Cluster is now healthy 2024-04-26T01:48:31.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:30 smithi097 ceph-mon[32853]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 8/192 objects degraded (4.167%), 1 pg degraded) 2024-04-26T01:48:31.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:30 smithi097 ceph-mon[32853]: Cluster is now healthy 2024-04-26T01:48:31.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.277+0000 7f88ba800700 1 -- 172.21.15.1:0/409631243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88b410f4f0 msgr2=0x7f88b41118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:31.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.277+0000 7f88ba800700 1 --2- 172.21.15.1:0/409631243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88b410f4f0 0x7f88b41118e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00b0a0 tx=0x7f88ac00b3b0 comp rx=0 tx=0).stop 2024-04-26T01:48:31.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.277+0000 7f88ba800700 1 -- 172.21.15.1:0/409631243 shutdown_connections 2024-04-26T01:48:31.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.277+0000 7f88ba800700 1 --2- 172.21.15.1:0/409631243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88b410f4f0 0x7f88b41118e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:31.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.277+0000 7f88ba800700 1 --2- 172.21.15.1:0/409631243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88b406a750 0x7f88b406ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:31.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.277+0000 7f88ba800700 1 --2- 172.21.15.1:0/409631243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88b406b070 0x7f88b410efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:31.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.277+0000 7f88ba800700 1 -- 172.21.15.1:0/409631243 >> 172.21.15.1:0/409631243 conn(0x7f88b40788f0 msgr2=0x7f88b4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:31.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.277+0000 7f88ba800700 1 -- 172.21.15.1:0/409631243 shutdown_connections 2024-04-26T01:48:31.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.277+0000 7f88ba800700 1 -- 172.21.15.1:0/409631243 wait complete. 2024-04-26T01:48:31.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.278+0000 7f88ba800700 1 Processor -- start 2024-04-26T01:48:31.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.278+0000 7f88ba800700 1 -- start start 2024-04-26T01:48:31.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.279+0000 7f88ba800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88b406a750 0x7f88b419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:31.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.279+0000 7f88ba800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88b406b070 0x7f88b419b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:31.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.279+0000 7f88ba800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88b410f4f0 0x7f88b419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:31.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.279+0000 7f88ba800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f88b4114e10 con 0x7f88b410f4f0 2024-04-26T01:48:31.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.279+0000 7f88ba800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f88b4114c90 con 0x7f88b406b070 2024-04-26T01:48:31.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.279+0000 7f88ba800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f88b4114f90 con 0x7f88b406a750 2024-04-26T01:48:31.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.279+0000 7f88b3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88b410f4f0 0x7f88b419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:31.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.279+0000 7f88b3400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88b406a750 0x7f88b419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:31.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.279+0000 7f88b3400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88b406a750 0x7f88b419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:46790/0 (socket says 172.21.15.1:46790) 2024-04-26T01:48:31.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.279+0000 7f88b3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88b410f4f0 0x7f88b419fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39918/0 (socket says 172.21.15.1:39918) 2024-04-26T01:48:31.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.279+0000 7f88b3400700 1 -- 172.21.15.1:0/2225884852 learned_addr learned my addr 172.21.15.1:0/2225884852 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:48:31.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.279+0000 7f88aaa00700 1 --2- 172.21.15.1:0/2225884852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88b406b070 0x7f88b419b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:31.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.280+0000 7f88b3400700 1 -- 172.21.15.1:0/2225884852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88b406b070 msgr2=0x7f88b419b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:31.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.280+0000 7f88b3400700 1 --2- 172.21.15.1:0/2225884852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88b406b070 0x7f88b419b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:31.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.280+0000 7f88b3400700 1 -- 172.21.15.1:0/2225884852 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88b410f4f0 msgr2=0x7f88b419fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:31.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.280+0000 7f88b3400700 1 --2- 172.21.15.1:0/2225884852 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88b410f4f0 0x7f88b419fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:31.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.280+0000 7f88b3400700 1 -- 172.21.15.1:0/2225884852 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b41a0320 con 0x7f88b406a750 2024-04-26T01:48:31.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.280+0000 7f88b3400700 1 --2- 172.21.15.1:0/2225884852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88b406a750 0x7f88b419b360 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f88a400e9a0 tx=0x7f88a400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:31.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.280+0000 7f88b0c00700 1 -- 172.21.15.1:0/2225884852 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a400ca10 con 0x7f88b406a750 2024-04-26T01:48:31.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.280+0000 7f88b0c00700 1 -- 172.21.15.1:0/2225884852 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a400cbb0 con 0x7f88b406a750 2024-04-26T01:48:31.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.280+0000 7f88ba800700 1 -- 172.21.15.1:0/2225884852 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b41a0610 con 0x7f88b406a750 2024-04-26T01:48:31.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.280+0000 7f88b0c00700 1 -- 172.21.15.1:0/2225884852 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a4018680 con 0x7f88b406a750 2024-04-26T01:48:31.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.280+0000 7f88ba800700 1 -- 172.21.15.1:0/2225884852 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b41a7e90 con 0x7f88b406a750 2024-04-26T01:48:31.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.281+0000 7f88ba800700 1 -- 172.21.15.1:0/2225884852 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88b4069730 con 0x7f88b406a750 2024-04-26T01:48:31.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.284+0000 7f88b0c00700 1 -- 172.21.15.1:0/2225884852 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f88a4010430 con 0x7f88b406a750 2024-04-26T01:48:31.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.285+0000 7f88b0c00700 1 --2- 172.21.15.1:0/2225884852 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f88940a3cd0 0x7f88940a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:31.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.285+0000 7f88b0c00700 1 -- 172.21.15.1:0/2225884852 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f88a40d0790 con 0x7f88b406a750 2024-04-26T01:48:31.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.285+0000 7f88aaa00700 1 --2- 172.21.15.1:0/2225884852 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f88940a3cd0 0x7f88940a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:31.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.286+0000 7f88aaa00700 1 --2- 172.21.15.1:0/2225884852 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f88940a3cd0 0x7f88940a6190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f889c009680 tx=0x7f889c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:31.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.289+0000 7f88b0c00700 1 -- 172.21.15.1:0/2225884852 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88a409cf90 con 0x7f88b406a750 2024-04-26T01:48:31.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.516+0000 7f88ba800700 1 -- 172.21.15.1:0/2225884852 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f88b4062e80 con 0x7f88940a3cd0 2024-04-26T01:48:31.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.528+0000 7f88b0c00700 1 -- 172.21.15.1:0/2225884852 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258750 (secure 0 0 0) 0x7f88b4062e80 con 0x7f88940a3cd0 2024-04-26T01:48:31.530 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:48:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.534+0000 7f88ba800700 1 -- 172.21.15.1:0/2225884852 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f88940a3cd0 msgr2=0x7f88940a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.534+0000 7f88ba800700 1 --2- 172.21.15.1:0/2225884852 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f88940a3cd0 0x7f88940a6190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f889c009680 tx=0x7f889c009260 comp rx=0 tx=0).stop 2024-04-26T01:48:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.534+0000 7f88ba800700 1 -- 172.21.15.1:0/2225884852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88b406a750 msgr2=0x7f88b419b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.534+0000 7f88ba800700 1 --2- 172.21.15.1:0/2225884852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88b406a750 0x7f88b419b360 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f88a400e9a0 tx=0x7f88a400ecb0 comp rx=0 tx=0).stop 2024-04-26T01:48:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.534+0000 7f88ba800700 1 -- 172.21.15.1:0/2225884852 shutdown_connections 2024-04-26T01:48:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.534+0000 7f88ba800700 1 --2- 172.21.15.1:0/2225884852 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f88940a3cd0 0x7f88940a6190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.534+0000 7f88ba800700 1 --2- 172.21.15.1:0/2225884852 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88b410f4f0 0x7f88b419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.534+0000 7f88ba800700 1 --2- 172.21.15.1:0/2225884852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88b406a750 0x7f88b419b360 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.534+0000 7f88ba800700 1 --2- 172.21.15.1:0/2225884852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88b406b070 0x7f88b419b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.534+0000 7f88ba800700 1 -- 172.21.15.1:0/2225884852 >> 172.21.15.1:0/2225884852 conn(0x7f88b40788f0 msgr2=0x7f88b41109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.534+0000 7f88ba800700 1 -- 172.21.15.1:0/2225884852 shutdown_connections 2024-04-26T01:48:31.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:31.534+0000 7f88ba800700 1 -- 172.21.15.1:0/2225884852 wait complete. 2024-04-26T01:48:31.534 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-04-26T01:48:32.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:31 smithi173 ceph-mon[32553]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 542 B/s wr, 5 op/s; 11 B/s, 1 objects/s recovering 2024-04-26T01:48:32.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:31 smithi173 ceph-mon[32553]: from='client.24673 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:48:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:31 smithi001 ceph-mon[27822]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 542 B/s wr, 5 op/s; 11 B/s, 1 objects/s recovering 2024-04-26T01:48:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:31 smithi001 ceph-mon[27822]: from='client.24673 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:48:32.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:31 smithi097 ceph-mon[32853]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 542 B/s wr, 5 op/s; 11 B/s, 1 objects/s recovering 2024-04-26T01:48:32.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:31 smithi097 ceph-mon[32853]: from='client.24673 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:48:32.366 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-04-26T01:48:30.822224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654950,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":387,"num_read_kb":309,"num_write":346,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":290,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359036,"kb_used_data":13980,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582788,"statfs":{"total":1151940427776,"available":1151572774912,"internally_reserved":0,"allocated":14315520,"data_stored":7842030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4257,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":28,"num_write":1,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.439967"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398825+0000","last_change":"2024-04-26T01:47:27.140418+0000","last_active":"2024-04-26T01:48:22.398825+0000","last_peered":"2024-04-26T01:48:22.398825+0000","last_clean":"2024-04-26T01:48:22.398825+0000","last_became_active":"2024-04-26T01:47:27.140017+0000","last_became_peered":"2024-04-26T01:47:27.140017+0000","last_unstale":"2024-04-26T01:48:22.398825+0000","last_undegraded":"2024-04-26T01:48:22.398825+0000","last_fullsized":"2024-04-26T01:48:22.398825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:46:08.621222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404830+0000","last_change":"2024-04-26T01:47:27.138746+0000","last_active":"2024-04-26T01:48:22.404830+0000","last_peered":"2024-04-26T01:48:22.404830+0000","last_clean":"2024-04-26T01:48:22.404830+0000","last_became_active":"2024-04-26T01:47:27.136163+0000","last_became_peered":"2024-04-26T01:47:27.136163+0000","last_unstale":"2024-04-26T01:48:22.404830+0000","last_undegraded":"2024-04-26T01:48:22.404830+0000","last_fullsized":"2024-04-26T01:48:22.404830+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:01:36.195332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.403704+0000","last_change":"2024-04-26T01:47:27.127017+0000","last_active":"2024-04-26T01:48:22.403704+0000","last_peered":"2024-04-26T01:48:22.403704+0000","last_clean":"2024-04-26T01:48:22.403704+0000","last_became_active":"2024-04-26T01:47:27.126610+0000","last_became_peered":"2024-04-26T01:47:27.126610+0000","last_unstale":"2024-04-26T01:48:22.403704+0000","last_undegraded":"2024-04-26T01:48:22.403704+0000","last_fullsized":"2024-04-26T01:48:22.403704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:05:12.784486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746210+0000","last_change":"2024-04-26T01:47:27.117640+0000","last_active":"2024-04-26T01:48:22.746210+0000","last_peered":"2024-04-26T01:48:22.746210+0000","last_clean":"2024-04-26T01:48:22.746210+0000","last_became_active":"2024-04-26T01:47:27.117266+0000","last_became_peered":"2024-04-26T01:47:27.117266+0000","last_unstale":"2024-04-26T01:48:22.746210+0000","last_undegraded":"2024-04-26T01:48:22.746210+0000","last_fullsized":"2024-04-26T01:48:22.746210+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:26:57.824560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396755+0000","last_change":"2024-04-26T01:47:27.128185+0000","last_active":"2024-04-26T01:48:22.396755+0000","last_peered":"2024-04-26T01:48:22.396755+0000","last_clean":"2024-04-26T01:48:22.396755+0000","last_became_active":"2024-04-26T01:47:27.127794+0000","last_became_peered":"2024-04-26T01:47:27.127794+0000","last_unstale":"2024-04-26T01:48:22.396755+0000","last_undegraded":"2024-04-26T01:48:22.396755+0000","last_fullsized":"2024-04-26T01:48:22.396755+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:09:52.774007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.748601+0000","last_change":"2024-04-26T01:48:22.748601+0000","last_active":"2024-04-26T01:48:22.748601+0000","last_peered":"2024-04-26T01:48:22.748601+0000","last_clean":"2024-04-26T01:48:22.748601+0000","last_became_active":"2024-04-26T01:48:22.748229+0000","last_became_peered":"2024-04-26T01:48:22.748229+0000","last_unstale":"2024-04-26T01:48:22.748601+0000","last_undegraded":"2024-04-26T01:48:22.748601+0000","last_fullsized":"2024-04-26T01:48:22.748601+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:46:30.433063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400077+0000","last_change":"2024-04-26T01:47:27.123786+0000","last_active":"2024-04-26T01:48:22.400077+0000","last_peered":"2024-04-26T01:48:22.400077+0000","last_clean":"2024-04-26T01:48:22.400077+0000","last_became_active":"2024-04-26T01:47:27.123473+0000","last_became_peered":"2024-04-26T01:47:27.123473+0000","last_unstale":"2024-04-26T01:48:22.400077+0000","last_undegraded":"2024-04-26T01:48:22.400077+0000","last_fullsized":"2024-04-26T01:48:22.400077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:48:55.031147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404450+0000","last_change":"2024-04-26T01:47:27.118756+0000","last_active":"2024-04-26T01:48:22.404450+0000","last_peered":"2024-04-26T01:48:22.404450+0000","last_clean":"2024-04-26T01:48:22.404450+0000","last_became_active":"2024-04-26T01:47:27.118406+0000","last_became_peered":"2024-04-26T01:47:27.118406+0000","last_unstale":"2024-04-26T01:48:22.404450+0000","last_undegraded":"2024-04-26T01:48:22.404450+0000","last_fullsized":"2024-04-26T01:48:22.404450+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:05.793682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398754+0000","last_change":"2024-04-26T01:47:27.117622+0000","last_active":"2024-04-26T01:48:22.398754+0000","last_peered":"2024-04-26T01:48:22.398754+0000","last_clean":"2024-04-26T01:48:22.398754+0000","last_became_active":"2024-04-26T01:47:27.115206+0000","last_became_peered":"2024-04-26T01:47:27.115206+0000","last_unstale":"2024-04-26T01:48:22.398754+0000","last_undegraded":"2024-04-26T01:48:22.398754+0000","last_fullsized":"2024-04-26T01:48:22.398754+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:13:17.866630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405408+0000","last_change":"2024-04-26T01:47:27.119524+0000","last_active":"2024-04-26T01:48:22.405408+0000","last_peered":"2024-04-26T01:48:22.405408+0000","last_clean":"2024-04-26T01:48:22.405408+0000","last_became_active":"2024-04-26T01:47:27.119008+0000","last_became_peered":"2024-04-26T01:47:27.119008+0000","last_unstale":"2024-04-26T01:48:22.405408+0000","last_undegraded":"2024-04-26T01:48:22.405408+0000","last_fullsized":"2024-04-26T01:48:22.405408+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:37:31.894486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400411+0000","last_change":"2024-04-26T01:47:27.122361+0000","last_active":"2024-04-26T01:48:22.400411+0000","last_peered":"2024-04-26T01:48:22.400411+0000","last_clean":"2024-04-26T01:48:22.400411+0000","last_became_active":"2024-04-26T01:47:27.121748+0000","last_became_peered":"2024-04-26T01:47:27.121748+0000","last_unstale":"2024-04-26T01:48:22.400411+0000","last_undegraded":"2024-04-26T01:48:22.400411+0000","last_fullsized":"2024-04-26T01:48:22.400411+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:18.680909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746160+0000","last_change":"2024-04-26T01:47:27.116833+0000","last_active":"2024-04-26T01:48:22.746160+0000","last_peered":"2024-04-26T01:48:22.746160+0000","last_clean":"2024-04-26T01:48:22.746160+0000","last_became_active":"2024-04-26T01:47:27.116508+0000","last_became_peered":"2024-04-26T01:47:27.116508+0000","last_unstale":"2024-04-26T01:48:22.746160+0000","last_undegraded":"2024-04-26T01:48:22.746160+0000","last_fullsized":"2024-04-26T01:48:22.746160+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:27:45.864521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405977+0000","last_change":"2024-04-26T01:47:27.125342+0000","last_active":"2024-04-26T01:48:22.405977+0000","last_peered":"2024-04-26T01:48:22.405977+0000","last_clean":"2024-04-26T01:48:22.405977+0000","last_became_active":"2024-04-26T01:47:27.124862+0000","last_became_peered":"2024-04-26T01:47:27.124862+0000","last_unstale":"2024-04-26T01:48:22.405977+0000","last_undegraded":"2024-04-26T01:48:22.405977+0000","last_fullsized":"2024-04-26T01:48:22.405977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:48.654711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.403399+0000","last_change":"2024-04-26T01:47:27.127365+0000","last_active":"2024-04-26T01:48:22.403399+0000","last_peered":"2024-04-26T01:48:22.403399+0000","last_clean":"2024-04-26T01:48:22.403399+0000","last_became_active":"2024-04-26T01:47:27.126868+0000","last_became_peered":"2024-04-26T01:47:27.126868+0000","last_unstale":"2024-04-26T01:48:22.403399+0000","last_undegraded":"2024-04-26T01:48:22.403399+0000","last_fullsized":"2024-04-26T01:48:22.403399+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:07:46.400333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.401775+0000","last_change":"2024-04-26T01:47:27.135584+0000","last_active":"2024-04-26T01:48:22.401775+0000","last_peered":"2024-04-26T01:48:22.401775+0000","last_clean":"2024-04-26T01:48:22.401775+0000","last_became_active":"2024-04-26T01:47:27.134849+0000","last_became_peered":"2024-04-26T01:47:27.134849+0000","last_unstale":"2024-04-26T01:48:22.401775+0000","last_undegraded":"2024-04-26T01:48:22.401775+0000","last_fullsized":"2024-04-26T01:48:22.401775+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:57:48.821595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398274+0000","last_change":"2024-04-26T01:47:27.136182+0000","last_active":"2024-04-26T01:48:22.398274+0000","last_peered":"2024-04-26T01:48:22.398274+0000","last_clean":"2024-04-26T01:48:22.398274+0000","last_became_active":"2024-04-26T01:47:27.135787+0000","last_became_peered":"2024-04-26T01:47:27.135787+0000","last_unstale":"2024-04-26T01:48:22.398274+0000","last_undegraded":"2024-04-26T01:48:22.398274+0000","last_fullsized":"2024-04-26T01:48:22.398274+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:38:14.275898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399072+0000","last_change":"2024-04-26T01:47:27.140700+0000","last_active":"2024-04-26T01:48:22.399072+0000","last_peered":"2024-04-26T01:48:22.399072+0000","last_clean":"2024-04-26T01:48:22.399072+0000","last_became_active":"2024-04-26T01:47:27.140025+0000","last_became_peered":"2024-04-26T01:47:27.140025+0000","last_unstale":"2024-04-26T01:48:22.399072+0000","last_undegraded":"2024-04-26T01:48:22.399072+0000","last_fullsized":"2024-04-26T01:48:22.399072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:50.465236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.747360+0000","last_change":"2024-04-26T01:47:27.117256+0000","last_active":"2024-04-26T01:48:22.747360+0000","last_peered":"2024-04-26T01:48:22.747360+0000","last_clean":"2024-04-26T01:48:22.747360+0000","last_became_active":"2024-04-26T01:47:27.116825+0000","last_became_peered":"2024-04-26T01:47:27.116825+0000","last_unstale":"2024-04-26T01:48:22.747360+0000","last_undegraded":"2024-04-26T01:48:22.747360+0000","last_fullsized":"2024-04-26T01:48:22.747360+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:23.313712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399301+0000","last_change":"2024-04-26T01:47:27.130233+0000","last_active":"2024-04-26T01:48:22.399301+0000","last_peered":"2024-04-26T01:48:22.399301+0000","last_clean":"2024-04-26T01:48:22.399301+0000","last_became_active":"2024-04-26T01:47:27.129852+0000","last_became_peered":"2024-04-26T01:47:27.129852+0000","last_unstale":"2024-04-26T01:48:22.399301+0000","last_undegraded":"2024-04-26T01:48:22.399301+0000","last_fullsized":"2024-04-26T01:48:22.399301+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:49:01.407264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.749092+0000","last_change":"2024-04-26T01:47:23.068477+0000","last_active":"2024-04-26T01:48:22.749092+0000","last_peered":"2024-04-26T01:48:22.749092+0000","last_clean":"2024-04-26T01:48:22.749092+0000","last_became_active":"2024-04-26T01:47:23.068061+0000","last_became_peered":"2024-04-26T01:47:23.068061+0000","last_unstale":"2024-04-26T01:48:22.749092+0000","last_undegraded":"2024-04-26T01:48:22.749092+0000","last_fullsized":"2024-04-26T01:48:22.749092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:02.153724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.406110+0000","last_change":"2024-04-26T01:47:27.130385+0000","last_active":"2024-04-26T01:48:22.406110+0000","last_peered":"2024-04-26T01:48:22.406110+0000","last_clean":"2024-04-26T01:48:22.406110+0000","last_became_active":"2024-04-26T01:47:27.129912+0000","last_became_peered":"2024-04-26T01:47:27.129912+0000","last_unstale":"2024-04-26T01:48:22.406110+0000","last_undegraded":"2024-04-26T01:48:22.406110+0000","last_fullsized":"2024-04-26T01:48:22.406110+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:26:03.010544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.407897+0000","last_change":"2024-04-26T01:47:23.110806+0000","last_active":"2024-04-26T01:48:22.407897+0000","last_peered":"2024-04-26T01:48:22.407897+0000","last_clean":"2024-04-26T01:48:22.407897+0000","last_became_active":"2024-04-26T01:47:23.110441+0000","last_became_peered":"2024-04-26T01:47:23.110441+0000","last_unstale":"2024-04-26T01:48:22.407897+0000","last_undegraded":"2024-04-26T01:48:22.407897+0000","last_fullsized":"2024-04-26T01:48:22.407897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:39.239172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.406491+0000","last_change":"2024-04-26T01:47:27.138628+0000","last_active":"2024-04-26T01:48:22.406491+0000","last_peered":"2024-04-26T01:48:22.406491+0000","last_clean":"2024-04-26T01:48:22.406491+0000","last_became_active":"2024-04-26T01:47:27.130483+0000","last_became_peered":"2024-04-26T01:47:27.130483+0000","last_unstale":"2024-04-26T01:48:22.406491+0000","last_undegraded":"2024-04-26T01:48:22.406491+0000","last_fullsized":"2024-04-26T01:48:22.406491+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:40:41.242856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.397138+0000","last_change":"2024-04-26T01:47:23.060066+0000","last_active":"2024-04-26T01:48:22.397138+0000","last_peered":"2024-04-26T01:48:22.397138+0000","last_clean":"2024-04-26T01:48:22.397138+0000","last_became_active":"2024-04-26T01:47:23.059526+0000","last_became_peered":"2024-04-26T01:47:23.059526+0000","last_unstale":"2024-04-26T01:48:22.397138+0000","last_undegraded":"2024-04-26T01:48:22.397138+0000","last_fullsized":"2024-04-26T01:48:22.397138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:28:22.623695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398426+0000","last_change":"2024-04-26T01:47:27.128343+0000","last_active":"2024-04-26T01:48:22.398426+0000","last_peered":"2024-04-26T01:48:22.398426+0000","last_clean":"2024-04-26T01:48:22.398426+0000","last_became_active":"2024-04-26T01:47:27.127920+0000","last_became_peered":"2024-04-26T01:47:27.127920+0000","last_unstale":"2024-04-26T01:48:22.398426+0000","last_undegraded":"2024-04-26T01:48:22.398426+0000","last_fullsized":"2024-04-26T01:48:22.398426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:06.725033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396937+0000","last_change":"2024-04-26T01:47:23.057783+0000","last_active":"2024-04-26T01:48:22.396937+0000","last_peered":"2024-04-26T01:48:22.396937+0000","last_clean":"2024-04-26T01:48:22.396937+0000","last_became_active":"2024-04-26T01:47:23.057391+0000","last_became_peered":"2024-04-26T01:47:23.057391+0000","last_unstale":"2024-04-26T01:48:22.396937+0000","last_undegraded":"2024-04-26T01:48:22.396937+0000","last_fullsized":"2024-04-26T01:48:22.396937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:10:10.050787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746112+0000","last_change":"2024-04-26T01:47:27.116458+0000","last_active":"2024-04-26T01:48:22.746112+0000","last_peered":"2024-04-26T01:48:22.746112+0000","last_clean":"2024-04-26T01:48:22.746112+0000","last_became_active":"2024-04-26T01:47:27.116084+0000","last_became_peered":"2024-04-26T01:47:27.116084+0000","last_unstale":"2024-04-26T01:48:22.746112+0000","last_undegraded":"2024-04-26T01:48:22.746112+0000","last_fullsized":"2024-04-26T01:48:22.746112+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:06:34.517478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":72,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:27.312844+0000","last_change":"2024-04-26T01:47:23.060657+0000","last_active":"2024-04-26T01:48:27.312844+0000","last_peered":"2024-04-26T01:48:27.312844+0000","last_clean":"2024-04-26T01:48:27.312844+0000","last_became_active":"2024-04-26T01:47:23.060294+0000","last_became_peered":"2024-04-26T01:47:23.060294+0000","last_unstale":"2024-04-26T01:48:27.312844+0000","last_undegraded":"2024-04-26T01:48:27.312844+0000","last_fullsized":"2024-04-26T01:48:27.312844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:33.979717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396644+0000","last_change":"2024-04-26T01:47:27.120587+0000","last_active":"2024-04-26T01:48:22.396644+0000","last_peered":"2024-04-26T01:48:22.396644+0000","last_clean":"2024-04-26T01:48:22.396644+0000","last_became_active":"2024-04-26T01:47:27.120236+0000","last_became_peered":"2024-04-26T01:47:27.120236+0000","last_unstale":"2024-04-26T01:48:22.396644+0000","last_undegraded":"2024-04-26T01:48:22.396644+0000","last_fullsized":"2024-04-26T01:48:22.396644+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:30:38.488741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405079+0000","last_change":"2024-04-26T01:47:23.069395+0000","last_active":"2024-04-26T01:48:22.405079+0000","last_peered":"2024-04-26T01:48:22.405079+0000","last_clean":"2024-04-26T01:48:22.405079+0000","last_became_active":"2024-04-26T01:47:23.069102+0000","last_became_peered":"2024-04-26T01:47:23.069102+0000","last_unstale":"2024-04-26T01:48:22.405079+0000","last_undegraded":"2024-04-26T01:48:22.405079+0000","last_fullsized":"2024-04-26T01:48:22.405079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:45.162604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745788+0000","last_change":"2024-04-26T01:47:27.123859+0000","last_active":"2024-04-26T01:48:22.745788+0000","last_peered":"2024-04-26T01:48:22.745788+0000","last_clean":"2024-04-26T01:48:22.745788+0000","last_became_active":"2024-04-26T01:47:27.123438+0000","last_became_peered":"2024-04-26T01:47:27.123438+0000","last_unstale":"2024-04-26T01:48:22.745788+0000","last_undegraded":"2024-04-26T01:48:22.745788+0000","last_fullsized":"2024-04-26T01:48:22.745788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:33:53.542809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.401049+0000","last_change":"2024-04-26T01:47:23.071544+0000","last_active":"2024-04-26T01:48:22.401049+0000","last_peered":"2024-04-26T01:48:22.401049+0000","last_clean":"2024-04-26T01:48:22.401049+0000","last_became_active":"2024-04-26T01:47:23.071191+0000","last_became_peered":"2024-04-26T01:47:23.071191+0000","last_unstale":"2024-04-26T01:48:22.401049+0000","last_undegraded":"2024-04-26T01:48:22.401049+0000","last_fullsized":"2024-04-26T01:48:22.401049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:58.889561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404118+0000","last_change":"2024-04-26T01:47:27.131887+0000","last_active":"2024-04-26T01:48:22.404118+0000","last_peered":"2024-04-26T01:48:22.404118+0000","last_clean":"2024-04-26T01:48:22.404118+0000","last_became_active":"2024-04-26T01:47:27.127973+0000","last_became_peered":"2024-04-26T01:47:27.127973+0000","last_unstale":"2024-04-26T01:48:22.404118+0000","last_undegraded":"2024-04-26T01:48:22.404118+0000","last_fullsized":"2024-04-26T01:48:22.404118+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:54:35.305779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745599+0000","last_change":"2024-04-26T01:47:23.061353+0000","last_active":"2024-04-26T01:48:22.745599+0000","last_peered":"2024-04-26T01:48:22.745599+0000","last_clean":"2024-04-26T01:48:22.745599+0000","last_became_active":"2024-04-26T01:47:23.061037+0000","last_became_peered":"2024-04-26T01:47:23.061037+0000","last_unstale":"2024-04-26T01:48:22.745599+0000","last_undegraded":"2024-04-26T01:48:22.745599+0000","last_fullsized":"2024-04-26T01:48:22.745599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:13:59.617969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746265+0000","last_change":"2024-04-26T01:47:27.122303+0000","last_active":"2024-04-26T01:48:22.746265+0000","last_peered":"2024-04-26T01:48:22.746265+0000","last_clean":"2024-04-26T01:48:22.746265+0000","last_became_active":"2024-04-26T01:47:27.121919+0000","last_became_peered":"2024-04-26T01:47:27.121919+0000","last_unstale":"2024-04-26T01:48:22.746265+0000","last_undegraded":"2024-04-26T01:48:22.746265+0000","last_fullsized":"2024-04-26T01:48:22.746265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:21.676663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":66,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:25.309636+0000","last_change":"2024-04-26T01:47:23.057961+0000","last_active":"2024-04-26T01:48:25.309636+0000","last_peered":"2024-04-26T01:48:25.309636+0000","last_clean":"2024-04-26T01:48:25.309636+0000","last_became_active":"2024-04-26T01:47:23.057529+0000","last_became_peered":"2024-04-26T01:47:23.057529+0000","last_unstale":"2024-04-26T01:48:25.309636+0000","last_undegraded":"2024-04-26T01:48:25.309636+0000","last_fullsized":"2024-04-26T01:48:25.309636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:14.420265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398667+0000","last_change":"2024-04-26T01:47:27.120083+0000","last_active":"2024-04-26T01:48:22.398667+0000","last_peered":"2024-04-26T01:48:22.398667+0000","last_clean":"2024-04-26T01:48:22.398667+0000","last_became_active":"2024-04-26T01:47:27.118410+0000","last_became_peered":"2024-04-26T01:47:27.118410+0000","last_unstale":"2024-04-26T01:48:22.398667+0000","last_undegraded":"2024-04-26T01:48:22.398667+0000","last_fullsized":"2024-04-26T01:48:22.398667+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:28.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400220+0000","last_change":"2024-04-26T01:47:23.059221+0000","last_active":"2024-04-26T01:48:22.400220+0000","last_peered":"2024-04-26T01:48:22.400220+0000","last_clean":"2024-04-26T01:48:22.400220+0000","last_became_active":"2024-04-26T01:47:23.058881+0000","last_became_peered":"2024-04-26T01:47:23.058881+0000","last_unstale":"2024-04-26T01:48:22.400220+0000","last_undegraded":"2024-04-26T01:48:22.400220+0000","last_fullsized":"2024-04-26T01:48:22.400220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:30.433696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.406173+0000","last_change":"2024-04-26T01:47:27.136763+0000","last_active":"2024-04-26T01:48:22.406173+0000","last_peered":"2024-04-26T01:48:22.406173+0000","last_clean":"2024-04-26T01:48:22.406173+0000","last_became_active":"2024-04-26T01:47:27.133642+0000","last_became_peered":"2024-04-26T01:47:27.133642+0000","last_unstale":"2024-04-26T01:48:22.406173+0000","last_undegraded":"2024-04-26T01:48:22.406173+0000","last_fullsized":"2024-04-26T01:48:22.406173+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:05.901139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746094+0000","last_change":"2024-04-26T01:47:23.108883+0000","last_active":"2024-04-26T01:48:22.746094+0000","last_peered":"2024-04-26T01:48:22.746094+0000","last_clean":"2024-04-26T01:48:22.746094+0000","last_became_active":"2024-04-26T01:47:23.108499+0000","last_became_peered":"2024-04-26T01:47:23.108499+0000","last_unstale":"2024-04-26T01:48:22.746094+0000","last_undegraded":"2024-04-26T01:48:22.746094+0000","last_fullsized":"2024-04-26T01:48:22.746094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:14:17.555416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.410651+0000","last_change":"2024-04-26T01:48:22.410651+0000","last_active":"2024-04-26T01:48:22.410651+0000","last_peered":"2024-04-26T01:48:22.410651+0000","last_clean":"2024-04-26T01:48:22.410651+0000","last_became_active":"2024-04-26T01:48:22.410359+0000","last_became_peered":"2024-04-26T01:48:22.410359+0000","last_unstale":"2024-04-26T01:48:22.410651+0000","last_undegraded":"2024-04-26T01:48:22.410651+0000","last_fullsized":"2024-04-26T01:48:22.410651+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:53:20.686915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'29","reported_seq":138,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:26.316864+0000","last_change":"2024-04-26T01:47:23.054893+0000","last_active":"2024-04-26T01:48:26.316864+0000","last_peered":"2024-04-26T01:48:26.316864+0000","last_clean":"2024-04-26T01:48:26.316864+0000","last_became_active":"2024-04-26T01:47:23.054558+0000","last_became_peered":"2024-04-26T01:47:23.054558+0000","last_unstale":"2024-04-26T01:48:26.316864+0000","last_undegraded":"2024-04-26T01:48:26.316864+0000","last_fullsized":"2024-04-26T01:48:26.316864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:59.356841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60347,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":83,"num_read_kb":100,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396887+0000","last_change":"2024-04-26T01:47:27.116934+0000","last_active":"2024-04-26T01:48:22.396887+0000","last_peered":"2024-04-26T01:48:22.396887+0000","last_clean":"2024-04-26T01:48:22.396887+0000","last_became_active":"2024-04-26T01:47:27.116527+0000","last_became_peered":"2024-04-26T01:47:27.116527+0000","last_unstale":"2024-04-26T01:48:22.396887+0000","last_undegraded":"2024-04-26T01:48:22.396887+0000","last_fullsized":"2024-04-26T01:48:22.396887+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:09:45.820533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399072+0000","last_change":"2024-04-26T01:47:23.059032+0000","last_active":"2024-04-26T01:48:22.399072+0000","last_peered":"2024-04-26T01:48:22.399072+0000","last_clean":"2024-04-26T01:48:22.399072+0000","last_became_active":"2024-04-26T01:47:23.057992+0000","last_became_peered":"2024-04-26T01:47:23.057992+0000","last_unstale":"2024-04-26T01:48:22.399072+0000","last_undegraded":"2024-04-26T01:48:22.399072+0000","last_fullsized":"2024-04-26T01:48:22.399072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:46:53.164925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.402515+0000","last_change":"2024-04-26T01:47:27.134574+0000","last_active":"2024-04-26T01:48:22.402515+0000","last_peered":"2024-04-26T01:48:22.402515+0000","last_clean":"2024-04-26T01:48:22.402515+0000","last_became_active":"2024-04-26T01:47:27.134184+0000","last_became_peered":"2024-04-26T01:47:27.134184+0000","last_unstale":"2024-04-26T01:48:22.402515+0000","last_undegraded":"2024-04-26T01:48:22.402515+0000","last_fullsized":"2024-04-26T01:48:22.402515+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:42:26.428014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745665+0000","last_change":"2024-04-26T01:47:23.062649+0000","last_active":"2024-04-26T01:48:22.745665+0000","last_peered":"2024-04-26T01:48:22.745665+0000","last_clean":"2024-04-26T01:48:22.745665+0000","last_became_active":"2024-04-26T01:47:23.062282+0000","last_became_peered":"2024-04-26T01:47:23.062282+0000","last_unstale":"2024-04-26T01:48:22.745665+0000","last_undegraded":"2024-04-26T01:48:22.745665+0000","last_fullsized":"2024-04-26T01:48:22.745665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:38.699105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400362+0000","last_change":"2024-04-26T01:47:27.124720+0000","last_active":"2024-04-26T01:48:22.400362+0000","last_peered":"2024-04-26T01:48:22.400362+0000","last_clean":"2024-04-26T01:48:22.400362+0000","last_became_active":"2024-04-26T01:47:27.124378+0000","last_became_peered":"2024-04-26T01:47:27.124378+0000","last_unstale":"2024-04-26T01:48:22.400362+0000","last_undegraded":"2024-04-26T01:48:22.400362+0000","last_fullsized":"2024-04-26T01:48:22.400362+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:45.462741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":72,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:27.331051+0000","last_change":"2024-04-26T01:47:23.056305+0000","last_active":"2024-04-26T01:48:27.331051+0000","last_peered":"2024-04-26T01:48:27.331051+0000","last_clean":"2024-04-26T01:48:27.331051+0000","last_became_active":"2024-04-26T01:47:23.055905+0000","last_became_peered":"2024-04-26T01:47:23.055905+0000","last_unstale":"2024-04-26T01:48:27.331051+0000","last_undegraded":"2024-04-26T01:48:27.331051+0000","last_fullsized":"2024-04-26T01:48:27.331051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:00:14.609673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400015+0000","last_change":"2024-04-26T01:47:27.119880+0000","last_active":"2024-04-26T01:48:22.400015+0000","last_peered":"2024-04-26T01:48:22.400015+0000","last_clean":"2024-04-26T01:48:22.400015+0000","last_became_active":"2024-04-26T01:47:27.119483+0000","last_became_peered":"2024-04-26T01:47:27.119483+0000","last_unstale":"2024-04-26T01:48:22.400015+0000","last_undegraded":"2024-04-26T01:48:22.400015+0000","last_fullsized":"2024-04-26T01:48:22.400015+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:15.523150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":66,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.758774+0000","last_change":"2024-04-26T01:48:22.758774+0000","last_active":"2024-04-26T01:48:22.758774+0000","last_peered":"2024-04-26T01:48:22.758774+0000","last_clean":"2024-04-26T01:48:22.758774+0000","last_became_active":"2024-04-26T01:48:22.747354+0000","last_became_peered":"2024-04-26T01:48:22.747354+0000","last_unstale":"2024-04-26T01:48:22.758774+0000","last_undegraded":"2024-04-26T01:48:22.758774+0000","last_fullsized":"2024-04-26T01:48:22.758774+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:23:15.886774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745649+0000","last_change":"2024-04-26T01:47:27.113508+0000","last_active":"2024-04-26T01:48:22.745649+0000","last_peered":"2024-04-26T01:48:22.745649+0000","last_clean":"2024-04-26T01:48:22.745649+0000","last_became_active":"2024-04-26T01:47:27.113066+0000","last_became_peered":"2024-04-26T01:47:27.113066+0000","last_unstale":"2024-04-26T01:48:22.745649+0000","last_undegraded":"2024-04-26T01:48:22.745649+0000","last_fullsized":"2024-04-26T01:48:22.745649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:37.525486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.397062+0000","last_change":"2024-04-26T01:47:23.059644+0000","last_active":"2024-04-26T01:48:22.397062+0000","last_peered":"2024-04-26T01:48:22.397062+0000","last_clean":"2024-04-26T01:48:22.397062+0000","last_became_active":"2024-04-26T01:47:23.058966+0000","last_became_peered":"2024-04-26T01:47:23.058966+0000","last_unstale":"2024-04-26T01:48:22.397062+0000","last_undegraded":"2024-04-26T01:48:22.397062+0000","last_fullsized":"2024-04-26T01:48:22.397062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:57:36.475935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.749354+0000","last_change":"2024-04-26T01:48:22.749354+0000","last_active":"2024-04-26T01:48:22.749354+0000","last_peered":"2024-04-26T01:48:22.749354+0000","last_clean":"2024-04-26T01:48:22.749354+0000","last_became_active":"2024-04-26T01:48:22.748978+0000","last_became_peered":"2024-04-26T01:48:22.748978+0000","last_unstale":"2024-04-26T01:48:22.749354+0000","last_undegraded":"2024-04-26T01:48:22.749354+0000","last_fullsized":"2024-04-26T01:48:22.749354+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:51:16.087091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746667+0000","last_change":"2024-04-26T01:47:23.110360+0000","last_active":"2024-04-26T01:48:22.746667+0000","last_peered":"2024-04-26T01:48:22.746667+0000","last_clean":"2024-04-26T01:48:22.746667+0000","last_became_active":"2024-04-26T01:47:23.109744+0000","last_became_peered":"2024-04-26T01:47:23.109744+0000","last_unstale":"2024-04-26T01:48:22.746667+0000","last_undegraded":"2024-04-26T01:48:22.746667+0000","last_fullsized":"2024-04-26T01:48:22.746667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:40:13.658396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405212+0000","last_change":"2024-04-26T01:47:27.125399+0000","last_active":"2024-04-26T01:48:22.405212+0000","last_peered":"2024-04-26T01:48:22.405212+0000","last_clean":"2024-04-26T01:48:22.405212+0000","last_became_active":"2024-04-26T01:47:27.124871+0000","last_became_peered":"2024-04-26T01:47:27.124871+0000","last_unstale":"2024-04-26T01:48:22.405212+0000","last_undegraded":"2024-04-26T01:48:22.405212+0000","last_fullsized":"2024-04-26T01:48:22.405212+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:28:48.285206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400178+0000","last_change":"2024-04-26T01:47:23.067862+0000","last_active":"2024-04-26T01:48:22.400178+0000","last_peered":"2024-04-26T01:48:22.400178+0000","last_clean":"2024-04-26T01:48:22.400178+0000","last_became_active":"2024-04-26T01:47:23.067526+0000","last_became_peered":"2024-04-26T01:47:23.067526+0000","last_unstale":"2024-04-26T01:48:22.400178+0000","last_undegraded":"2024-04-26T01:48:22.400178+0000","last_fullsized":"2024-04-26T01:48:22.400178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:22.668405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.748548+0000","last_change":"2024-04-26T01:48:22.748548+0000","last_active":"2024-04-26T01:48:22.748548+0000","last_peered":"2024-04-26T01:48:22.748548+0000","last_clean":"2024-04-26T01:48:22.748548+0000","last_became_active":"2024-04-26T01:48:22.748182+0000","last_became_peered":"2024-04-26T01:48:22.748182+0000","last_unstale":"2024-04-26T01:48:22.748548+0000","last_undegraded":"2024-04-26T01:48:22.748548+0000","last_fullsized":"2024-04-26T01:48:22.748548+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:20:27.596827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745783+0000","last_change":"2024-04-26T01:47:27.118307+0000","last_active":"2024-04-26T01:48:22.745783+0000","last_peered":"2024-04-26T01:48:22.745783+0000","last_clean":"2024-04-26T01:48:22.745783+0000","last_became_active":"2024-04-26T01:47:27.117986+0000","last_became_peered":"2024-04-26T01:47:27.117986+0000","last_unstale":"2024-04-26T01:48:22.745783+0000","last_undegraded":"2024-04-26T01:48:22.745783+0000","last_fullsized":"2024-04-26T01:48:22.745783+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:40.667143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":66,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:24.324415+0000","last_change":"2024-04-26T01:47:23.109973+0000","last_active":"2024-04-26T01:48:24.324415+0000","last_peered":"2024-04-26T01:48:24.324415+0000","last_clean":"2024-04-26T01:48:24.324415+0000","last_became_active":"2024-04-26T01:47:23.109446+0000","last_became_peered":"2024-04-26T01:47:23.109446+0000","last_unstale":"2024-04-26T01:48:24.324415+0000","last_undegraded":"2024-04-26T01:48:24.324415+0000","last_fullsized":"2024-04-26T01:48:24.324415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:49:00.684164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.408237+0000","last_change":"2024-04-26T01:47:27.131502+0000","last_active":"2024-04-26T01:48:22.408237+0000","last_peered":"2024-04-26T01:48:22.408237+0000","last_clean":"2024-04-26T01:48:22.408237+0000","last_became_active":"2024-04-26T01:47:27.127697+0000","last_became_peered":"2024-04-26T01:47:27.127697+0000","last_unstale":"2024-04-26T01:48:22.408237+0000","last_undegraded":"2024-04-26T01:48:22.408237+0000","last_fullsized":"2024-04-26T01:48:22.408237+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:39:15.458189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746705+0000","last_change":"2024-04-26T01:47:23.066596+0000","last_active":"2024-04-26T01:48:22.746705+0000","last_peered":"2024-04-26T01:48:22.746705+0000","last_clean":"2024-04-26T01:48:22.746705+0000","last_became_active":"2024-04-26T01:47:23.066216+0000","last_became_peered":"2024-04-26T01:47:23.066216+0000","last_unstale":"2024-04-26T01:48:22.746705+0000","last_undegraded":"2024-04-26T01:48:22.746705+0000","last_fullsized":"2024-04-26T01:48:22.746705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:08:31.544831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":359,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400108+0000","last_change":"2024-04-26T01:45:54.834501+0000","last_active":"2024-04-26T01:48:22.400108+0000","last_peered":"2024-04-26T01:48:22.400108+0000","last_clean":"2024-04-26T01:48:22.400108+0000","last_became_active":"2024-04-26T01:45:54.833868+0000","last_became_peered":"2024-04-26T01:45:54.833868+0000","last_unstale":"2024-04-26T01:48:22.400108+0000","last_undegraded":"2024-04-26T01:48:22.400108+0000","last_fullsized":"2024-04-26T01:48:22.400108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:42:05.228956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:42:05.228956+0000","last_clean_scrub_stamp":"2024-04-26T01:42:05.228956+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:06:58.383315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400398+0000","last_change":"2024-04-26T01:47:27.126475+0000","last_active":"2024-04-26T01:48:22.400398+0000","last_peered":"2024-04-26T01:48:22.400398+0000","last_clean":"2024-04-26T01:48:22.400398+0000","last_became_active":"2024-04-26T01:47:27.123760+0000","last_became_peered":"2024-04-26T01:47:27.123760+0000","last_unstale":"2024-04-26T01:48:22.400398+0000","last_undegraded":"2024-04-26T01:48:22.400398+0000","last_fullsized":"2024-04-26T01:48:22.400398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:38:27.473909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400221+0000","last_change":"2024-04-26T01:47:23.059810+0000","last_active":"2024-04-26T01:48:22.400221+0000","last_peered":"2024-04-26T01:48:22.400221+0000","last_clean":"2024-04-26T01:48:22.400221+0000","last_became_active":"2024-04-26T01:47:23.059504+0000","last_became_peered":"2024-04-26T01:47:23.059504+0000","last_unstale":"2024-04-26T01:48:22.400221+0000","last_undegraded":"2024-04-26T01:48:22.400221+0000","last_fullsized":"2024-04-26T01:48:22.400221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:50:51.220951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396987+0000","last_change":"2024-04-26T01:47:23.109460+0000","last_active":"2024-04-26T01:48:22.396987+0000","last_peered":"2024-04-26T01:48:22.396987+0000","last_clean":"2024-04-26T01:48:22.396987+0000","last_became_active":"2024-04-26T01:47:23.109090+0000","last_became_peered":"2024-04-26T01:47:23.109090+0000","last_unstale":"2024-04-26T01:48:22.396987+0000","last_undegraded":"2024-04-26T01:48:22.396987+0000","last_fullsized":"2024-04-26T01:48:22.396987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:56.821312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.407355+0000","last_change":"2024-04-26T01:47:27.129537+0000","last_active":"2024-04-26T01:48:22.407355+0000","last_peered":"2024-04-26T01:48:22.407355+0000","last_clean":"2024-04-26T01:48:22.407355+0000","last_became_active":"2024-04-26T01:47:27.128124+0000","last_became_peered":"2024-04-26T01:47:27.128124+0000","last_unstale":"2024-04-26T01:48:22.407355+0000","last_undegraded":"2024-04-26T01:48:22.407355+0000","last_fullsized":"2024-04-26T01:48:22.407355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:14:26.262197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'11","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.747472+0000","last_change":"2024-04-26T01:47:23.064511+0000","last_active":"2024-04-26T01:48:22.747472+0000","last_peered":"2024-04-26T01:48:22.747472+0000","last_clean":"2024-04-26T01:48:22.747472+0000","last_became_active":"2024-04-26T01:47:23.064080+0000","last_became_peered":"2024-04-26T01:47:23.064080+0000","last_unstale":"2024-04-26T01:48:22.747472+0000","last_undegraded":"2024-04-26T01:48:22.747472+0000","last_fullsized":"2024-04-26T01:48:22.747472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:37.562644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404336+0000","last_change":"2024-04-26T01:47:27.124057+0000","last_active":"2024-04-26T01:48:22.404336+0000","last_peered":"2024-04-26T01:48:22.404336+0000","last_clean":"2024-04-26T01:48:22.404336+0000","last_became_active":"2024-04-26T01:47:27.123636+0000","last_became_peered":"2024-04-26T01:47:27.123636+0000","last_unstale":"2024-04-26T01:48:22.404336+0000","last_undegraded":"2024-04-26T01:48:22.404336+0000","last_fullsized":"2024-04-26T01:48:22.404336+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:56:55.691697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746155+0000","last_change":"2024-04-26T01:47:23.136571+0000","last_active":"2024-04-26T01:48:22.746155+0000","last_peered":"2024-04-26T01:48:22.746155+0000","last_clean":"2024-04-26T01:48:22.746155+0000","last_became_active":"2024-04-26T01:47:23.136205+0000","last_became_peered":"2024-04-26T01:47:23.136205+0000","last_unstale":"2024-04-26T01:48:22.746155+0000","last_undegraded":"2024-04-26T01:48:22.746155+0000","last_fullsized":"2024-04-26T01:48:22.746155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:24.418797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.747570+0000","last_change":"2024-04-26T01:48:22.747570+0000","last_active":"2024-04-26T01:48:22.747570+0000","last_peered":"2024-04-26T01:48:22.747570+0000","last_clean":"2024-04-26T01:48:22.747570+0000","last_became_active":"2024-04-26T01:48:22.747147+0000","last_became_peered":"2024-04-26T01:48:22.747147+0000","last_unstale":"2024-04-26T01:48:22.747570+0000","last_undegraded":"2024-04-26T01:48:22.747570+0000","last_fullsized":"2024-04-26T01:48:22.747570+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:37.034388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.414610+0000","last_change":"2024-04-26T01:48:22.414610+0000","last_active":"2024-04-26T01:48:22.414610+0000","last_peered":"2024-04-26T01:48:22.414610+0000","last_clean":"2024-04-26T01:48:22.414610+0000","last_became_active":"2024-04-26T01:48:22.410275+0000","last_became_peered":"2024-04-26T01:48:22.410275+0000","last_unstale":"2024-04-26T01:48:22.414610+0000","last_undegraded":"2024-04-26T01:48:22.414610+0000","last_fullsized":"2024-04-26T01:48:22.414610+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:20:48.131988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396626+0000","last_change":"2024-04-26T01:47:27.115498+0000","last_active":"2024-04-26T01:48:22.396626+0000","last_peered":"2024-04-26T01:48:22.396626+0000","last_clean":"2024-04-26T01:48:22.396626+0000","last_became_active":"2024-04-26T01:47:27.115134+0000","last_became_peered":"2024-04-26T01:47:27.115134+0000","last_unstale":"2024-04-26T01:48:22.396626+0000","last_undegraded":"2024-04-26T01:48:22.396626+0000","last_fullsized":"2024-04-26T01:48:22.396626+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:20.795224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405265+0000","last_change":"2024-04-26T01:47:23.065366+0000","last_active":"2024-04-26T01:48:22.405265+0000","last_peered":"2024-04-26T01:48:22.405265+0000","last_clean":"2024-04-26T01:48:22.405265+0000","last_became_active":"2024-04-26T01:47:23.063216+0000","last_became_peered":"2024-04-26T01:47:23.063216+0000","last_unstale":"2024-04-26T01:48:22.405265+0000","last_undegraded":"2024-04-26T01:48:22.405265+0000","last_fullsized":"2024-04-26T01:48:22.405265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:06:05.261907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398405+0000","last_change":"2024-04-26T01:47:27.127938+0000","last_active":"2024-04-26T01:48:22.398405+0000","last_peered":"2024-04-26T01:48:22.398405+0000","last_clean":"2024-04-26T01:48:22.398405+0000","last_became_active":"2024-04-26T01:47:27.127553+0000","last_became_peered":"2024-04-26T01:47:27.127553+0000","last_unstale":"2024-04-26T01:48:22.398405+0000","last_undegraded":"2024-04-26T01:48:22.398405+0000","last_fullsized":"2024-04-26T01:48:22.398405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:17:51.115349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398731+0000","last_change":"2024-04-26T01:47:23.060475+0000","last_active":"2024-04-26T01:48:22.398731+0000","last_peered":"2024-04-26T01:48:22.398731+0000","last_clean":"2024-04-26T01:48:22.398731+0000","last_became_active":"2024-04-26T01:47:23.060098+0000","last_became_peered":"2024-04-26T01:47:23.060098+0000","last_unstale":"2024-04-26T01:48:22.398731+0000","last_undegraded":"2024-04-26T01:48:22.398731+0000","last_fullsized":"2024-04-26T01:48:22.398731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:49.009455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.407796+0000","last_change":"2024-04-26T01:47:27.137491+0000","last_active":"2024-04-26T01:48:22.407796+0000","last_peered":"2024-04-26T01:48:22.407796+0000","last_clean":"2024-04-26T01:48:22.407796+0000","last_became_active":"2024-04-26T01:47:27.137059+0000","last_became_peered":"2024-04-26T01:47:27.137059+0000","last_unstale":"2024-04-26T01:48:22.407796+0000","last_undegraded":"2024-04-26T01:48:22.407796+0000","last_fullsized":"2024-04-26T01:48:22.407796+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:20:25.004074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405524+0000","last_change":"2024-04-26T01:48:22.405071+0000","last_active":"2024-04-26T01:48:22.405524+0000","last_peered":"2024-04-26T01:48:22.405524+0000","last_clean":"2024-04-26T01:48:22.405524+0000","last_became_active":"2024-04-26T01:48:22.401282+0000","last_became_peered":"2024-04-26T01:48:22.401282+0000","last_unstale":"2024-04-26T01:48:22.405524+0000","last_undegraded":"2024-04-26T01:48:22.405524+0000","last_fullsized":"2024-04-26T01:48:22.405524+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:24:53.628014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400294+0000","last_change":"2024-04-26T01:47:27.128253+0000","last_active":"2024-04-26T01:48:22.400294+0000","last_peered":"2024-04-26T01:48:22.400294+0000","last_clean":"2024-04-26T01:48:22.400294+0000","last_became_active":"2024-04-26T01:47:27.127869+0000","last_became_peered":"2024-04-26T01:47:27.127869+0000","last_unstale":"2024-04-26T01:48:22.400294+0000","last_undegraded":"2024-04-26T01:48:22.400294+0000","last_fullsized":"2024-04-26T01:48:22.400294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:11:23.476938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399412+0000","last_change":"2024-04-26T01:47:23.136862+0000","last_active":"2024-04-26T01:48:22.399412+0000","last_peered":"2024-04-26T01:48:22.399412+0000","last_clean":"2024-04-26T01:48:22.399412+0000","last_became_active":"2024-04-26T01:47:23.136380+0000","last_became_peered":"2024-04-26T01:47:23.136380+0000","last_unstale":"2024-04-26T01:48:22.399412+0000","last_undegraded":"2024-04-26T01:48:22.399412+0000","last_fullsized":"2024-04-26T01:48:22.399412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:39:18.103559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400366+0000","last_change":"2024-04-26T01:47:27.113363+0000","last_active":"2024-04-26T01:48:22.400366+0000","last_peered":"2024-04-26T01:48:22.400366+0000","last_clean":"2024-04-26T01:48:22.400366+0000","last_became_active":"2024-04-26T01:47:27.113040+0000","last_became_peered":"2024-04-26T01:47:27.113040+0000","last_unstale":"2024-04-26T01:48:22.400366+0000","last_undegraded":"2024-04-26T01:48:22.400366+0000","last_fullsized":"2024-04-26T01:48:22.400366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:42:16.428697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745984+0000","last_change":"2024-04-26T01:47:23.067257+0000","last_active":"2024-04-26T01:48:22.745984+0000","last_peered":"2024-04-26T01:48:22.745984+0000","last_clean":"2024-04-26T01:48:22.745984+0000","last_became_active":"2024-04-26T01:47:23.066858+0000","last_became_peered":"2024-04-26T01:47:23.066858+0000","last_unstale":"2024-04-26T01:48:22.745984+0000","last_undegraded":"2024-04-26T01:48:22.745984+0000","last_fullsized":"2024-04-26T01:48:22.745984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:22:07.848074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400154+0000","last_change":"2024-04-26T01:47:27.125885+0000","last_active":"2024-04-26T01:48:22.400154+0000","last_peered":"2024-04-26T01:48:22.400154+0000","last_clean":"2024-04-26T01:48:22.400154+0000","last_became_active":"2024-04-26T01:47:27.125524+0000","last_became_peered":"2024-04-26T01:47:27.125524+0000","last_unstale":"2024-04-26T01:48:22.400154+0000","last_undegraded":"2024-04-26T01:48:22.400154+0000","last_fullsized":"2024-04-26T01:48:22.400154+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:39.282023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.407608+0000","last_change":"2024-04-26T01:47:23.071051+0000","last_active":"2024-04-26T01:48:22.407608+0000","last_peered":"2024-04-26T01:48:22.407608+0000","last_clean":"2024-04-26T01:48:22.407608+0000","last_became_active":"2024-04-26T01:47:23.069524+0000","last_became_peered":"2024-04-26T01:47:23.069524+0000","last_unstale":"2024-04-26T01:48:22.407608+0000","last_undegraded":"2024-04-26T01:48:22.407608+0000","last_fullsized":"2024-04-26T01:48:22.407608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:24:57.643100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400370+0000","last_change":"2024-04-26T01:47:27.114325+0000","last_active":"2024-04-26T01:48:22.400370+0000","last_peered":"2024-04-26T01:48:22.400370+0000","last_clean":"2024-04-26T01:48:22.400370+0000","last_became_active":"2024-04-26T01:47:27.113952+0000","last_became_peered":"2024-04-26T01:47:27.113952+0000","last_unstale":"2024-04-26T01:48:22.400370+0000","last_undegraded":"2024-04-26T01:48:22.400370+0000","last_fullsized":"2024-04-26T01:48:22.400370+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:39:19.167175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398411+0000","last_change":"2024-04-26T01:47:27.118709+0000","last_active":"2024-04-26T01:48:22.398411+0000","last_peered":"2024-04-26T01:48:22.398411+0000","last_clean":"2024-04-26T01:48:22.398411+0000","last_became_active":"2024-04-26T01:47:27.118341+0000","last_became_peered":"2024-04-26T01:47:27.118341+0000","last_unstale":"2024-04-26T01:48:22.398411+0000","last_undegraded":"2024-04-26T01:48:22.398411+0000","last_fullsized":"2024-04-26T01:48:22.398411+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:13.736180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399082+0000","last_change":"2024-04-26T01:47:27.119239+0000","last_active":"2024-04-26T01:48:22.399082+0000","last_peered":"2024-04-26T01:48:22.399082+0000","last_clean":"2024-04-26T01:48:22.399082+0000","last_became_active":"2024-04-26T01:47:27.118877+0000","last_became_peered":"2024-04-26T01:47:27.118877+0000","last_unstale":"2024-04-26T01:48:22.399082+0000","last_undegraded":"2024-04-26T01:48:22.399082+0000","last_fullsized":"2024-04-26T01:48:22.399082+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:04:10.033517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398635+0000","last_change":"2024-04-26T01:47:27.142848+0000","last_active":"2024-04-26T01:48:22.398635+0000","last_peered":"2024-04-26T01:48:22.398635+0000","last_clean":"2024-04-26T01:48:22.398635+0000","last_became_active":"2024-04-26T01:47:27.142580+0000","last_became_peered":"2024-04-26T01:47:27.142580+0000","last_unstale":"2024-04-26T01:48:22.398635+0000","last_undegraded":"2024-04-26T01:48:22.398635+0000","last_fullsized":"2024-04-26T01:48:22.398635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:15:46.923922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404760+0000","last_change":"2024-04-26T01:47:27.134817+0000","last_active":"2024-04-26T01:48:22.404760+0000","last_peered":"2024-04-26T01:48:22.404760+0000","last_clean":"2024-04-26T01:48:22.404760+0000","last_became_active":"2024-04-26T01:47:27.134024+0000","last_became_peered":"2024-04-26T01:47:27.134024+0000","last_unstale":"2024-04-26T01:48:22.404760+0000","last_undegraded":"2024-04-26T01:48:22.404760+0000","last_fullsized":"2024-04-26T01:48:22.404760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:56.746427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405975+0000","last_change":"2024-04-26T01:47:27.143137+0000","last_active":"2024-04-26T01:48:22.405975+0000","last_peered":"2024-04-26T01:48:22.405975+0000","last_clean":"2024-04-26T01:48:22.405975+0000","last_became_active":"2024-04-26T01:47:27.142625+0000","last_became_peered":"2024-04-26T01:47:27.142625+0000","last_unstale":"2024-04-26T01:48:22.405975+0000","last_undegraded":"2024-04-26T01:48:22.405975+0000","last_fullsized":"2024-04-26T01:48:22.405975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:42:50.468382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.749582+0000","last_change":"2024-04-26T01:48:22.749582+0000","last_active":"2024-04-26T01:48:22.749582+0000","last_peered":"2024-04-26T01:48:22.749582+0000","last_clean":"2024-04-26T01:48:22.749582+0000","last_became_active":"2024-04-26T01:48:22.749191+0000","last_became_peered":"2024-04-26T01:48:22.749191+0000","last_unstale":"2024-04-26T01:48:22.749582+0000","last_undegraded":"2024-04-26T01:48:22.749582+0000","last_fullsized":"2024-04-26T01:48:22.749582+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:14.614842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399646+0000","last_change":"2024-04-26T01:47:27.123964+0000","last_active":"2024-04-26T01:48:22.399646+0000","last_peered":"2024-04-26T01:48:22.399646+0000","last_clean":"2024-04-26T01:48:22.399646+0000","last_became_active":"2024-04-26T01:47:27.123552+0000","last_became_peered":"2024-04-26T01:47:27.123552+0000","last_unstale":"2024-04-26T01:48:22.399646+0000","last_undegraded":"2024-04-26T01:48:22.399646+0000","last_fullsized":"2024-04-26T01:48:22.399646+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:33:12.566105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.412621+0000","last_change":"2024-04-26T01:48:22.412621+0000","last_active":"2024-04-26T01:48:22.412621+0000","last_peered":"2024-04-26T01:48:22.412621+0000","last_clean":"2024-04-26T01:48:22.412621+0000","last_became_active":"2024-04-26T01:48:22.411430+0000","last_became_peered":"2024-04-26T01:48:22.411430+0000","last_unstale":"2024-04-26T01:48:22.412621+0000","last_undegraded":"2024-04-26T01:48:22.412621+0000","last_fullsized":"2024-04-26T01:48:22.412621+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:34.152338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745667+0000","last_change":"2024-04-26T01:47:27.131437+0000","last_active":"2024-04-26T01:48:22.745667+0000","last_peered":"2024-04-26T01:48:22.745667+0000","last_clean":"2024-04-26T01:48:22.745667+0000","last_became_active":"2024-04-26T01:47:27.131138+0000","last_became_peered":"2024-04-26T01:47:27.131138+0000","last_unstale":"2024-04-26T01:48:22.745667+0000","last_undegraded":"2024-04-26T01:48:22.745667+0000","last_fullsized":"2024-04-26T01:48:22.745667+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:55:39.132972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404958+0000","last_change":"2024-04-26T01:47:27.120928+0000","last_active":"2024-04-26T01:48:22.404958+0000","last_peered":"2024-04-26T01:48:22.404958+0000","last_clean":"2024-04-26T01:48:22.404958+0000","last_became_active":"2024-04-26T01:47:27.120012+0000","last_became_peered":"2024-04-26T01:47:27.120012+0000","last_unstale":"2024-04-26T01:48:22.404958+0000","last_undegraded":"2024-04-26T01:48:22.404958+0000","last_fullsized":"2024-04-26T01:48:22.404958+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:19:44.209876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398493+0000","last_change":"2024-04-26T01:47:27.138766+0000","last_active":"2024-04-26T01:48:22.398493+0000","last_peered":"2024-04-26T01:48:22.398493+0000","last_clean":"2024-04-26T01:48:22.398493+0000","last_became_active":"2024-04-26T01:47:27.138413+0000","last_became_peered":"2024-04-26T01:47:27.138413+0000","last_unstale":"2024-04-26T01:48:22.398493+0000","last_undegraded":"2024-04-26T01:48:22.398493+0000","last_fullsized":"2024-04-26T01:48:22.398493+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:51:13.117307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.406315+0000","last_change":"2024-04-26T01:47:27.141338+0000","last_active":"2024-04-26T01:48:22.406315+0000","last_peered":"2024-04-26T01:48:22.406315+0000","last_clean":"2024-04-26T01:48:22.406315+0000","last_became_active":"2024-04-26T01:47:27.140825+0000","last_became_peered":"2024-04-26T01:47:27.140825+0000","last_unstale":"2024-04-26T01:48:22.406315+0000","last_undegraded":"2024-04-26T01:48:22.406315+0000","last_fullsized":"2024-04-26T01:48:22.406315+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:12:08.680876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398625+0000","last_change":"2024-04-26T01:47:27.114919+0000","last_active":"2024-04-26T01:48:22.398625+0000","last_peered":"2024-04-26T01:48:22.398625+0000","last_clean":"2024-04-26T01:48:22.398625+0000","last_became_active":"2024-04-26T01:47:27.114528+0000","last_became_peered":"2024-04-26T01:47:27.114528+0000","last_unstale":"2024-04-26T01:48:22.398625+0000","last_undegraded":"2024-04-26T01:48:22.398625+0000","last_fullsized":"2024-04-26T01:48:22.398625+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:15:25.620566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64348,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":251,"num_read_kb":190,"num_write":163,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318299,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":490347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579936,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28688,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716464,"statfs":{"total":95995035648,"available":95965659136,"internally_reserved":0,"allocated":1708032,"data_stored":1142566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743463,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":490583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005102,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":490503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299441,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1622016,"data_stored":1082433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":489972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":550112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":491243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":551350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182472,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":490283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":490912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724607,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32776,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712376,"statfs":{"total":95995035648,"available":95961473024,"internally_reserved":0,"allocated":1634304,"data_stored":1081726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:48:32.371 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph status --format=json 2024-04-26T01:48:32.876 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:34.062 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:33 smithi001 ceph-mon[27822]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 10 B/s, 1 objects/s recovering 2024-04-26T01:48:34.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:33 smithi097 ceph-mon[32853]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 10 B/s, 1 objects/s recovering 2024-04-26T01:48:34.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:33 smithi173 ceph-mon[32553]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 10 B/s, 1 objects/s recovering 2024-04-26T01:48:34.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.934+0000 7f29c6600700 1 -- 172.21.15.1:0/3543035242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29c010e3b0 msgr2=0x7f29c0115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:34.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.934+0000 7f29c6600700 1 --2- 172.21.15.1:0/3543035242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29c010e3b0 0x7f29c0115f80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f29b80099c0 tx=0x7f29b8009cd0 comp rx=0 tx=0).stop 2024-04-26T01:48:34.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.934+0000 7f29c6600700 1 -- 172.21.15.1:0/3543035242 shutdown_connections 2024-04-26T01:48:34.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.934+0000 7f29c6600700 1 --2- 172.21.15.1:0/3543035242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29c010e3b0 0x7f29c0115f80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:34.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.934+0000 7f29c6600700 1 --2- 172.21.15.1:0/3543035242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c01092d0 0x7f29c01096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:34.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.934+0000 7f29c6600700 1 --2- 172.21.15.1:0/3543035242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29c0109c80 0x7f29c010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:34.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.934+0000 7f29c6600700 1 -- 172.21.15.1:0/3543035242 >> 172.21.15.1:0/3543035242 conn(0x7f29c0076210 msgr2=0x7f29c0107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:34.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.934+0000 7f29c6600700 1 -- 172.21.15.1:0/3543035242 shutdown_connections 2024-04-26T01:48:34.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.935+0000 7f29c6600700 1 -- 172.21.15.1:0/3543035242 wait complete. 2024-04-26T01:48:34.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.936+0000 7f29c6600700 1 Processor -- start 2024-04-26T01:48:34.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.936+0000 7f29c6600700 1 -- start start 2024-04-26T01:48:34.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.936+0000 7f29c6600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c01092d0 0x7f29c019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:34.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.936+0000 7f29c6600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29c0109c80 0x7f29c019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:34.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.936+0000 7f29c6600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29c010e3b0 0x7f29c01a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:34.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.936+0000 7f29c6600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f29c01192a0 con 0x7f29c0109c80 2024-04-26T01:48:34.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.936+0000 7f29c6600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f29c0119120 con 0x7f29c01092d0 2024-04-26T01:48:34.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.936+0000 7f29c6600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f29c0119420 con 0x7f29c010e3b0 2024-04-26T01:48:34.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.937+0000 7f29b6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29c0109c80 0x7f29c019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:34.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.937+0000 7f29bf400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c01092d0 0x7f29c019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:34.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.937+0000 7f29bfe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29c010e3b0 0x7f29c01a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:34.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.937+0000 7f29bf400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c01092d0 0x7f29c019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:49268/0 (socket says 172.21.15.1:49268) 2024-04-26T01:48:34.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.937+0000 7f29bf400700 1 -- 172.21.15.1:0/672058359 learned_addr learned my addr 172.21.15.1:0/672058359 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:48:34.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.937+0000 7f29bfe00700 1 -- 172.21.15.1:0/672058359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c01092d0 msgr2=0x7f29c019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:34.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.937+0000 7f29bfe00700 1 --2- 172.21.15.1:0/672058359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c01092d0 0x7f29c019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:34.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.937+0000 7f29bfe00700 1 -- 172.21.15.1:0/672058359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29c0109c80 msgr2=0x7f29c019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:34.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.937+0000 7f29bfe00700 1 --2- 172.21.15.1:0/672058359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29c0109c80 0x7f29c019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:34.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.937+0000 7f29bfe00700 1 -- 172.21.15.1:0/672058359 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c01a4730 con 0x7f29c010e3b0 2024-04-26T01:48:34.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.938+0000 7f29bfe00700 1 --2- 172.21.15.1:0/672058359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29c010e3b0 0x7f29c01a3ff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f29b80098e0 tx=0x7f29b80057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:34.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.938+0000 7f29bcc00700 1 -- 172.21.15.1:0/672058359 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8026070 con 0x7f29c010e3b0 2024-04-26T01:48:34.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.938+0000 7f29bcc00700 1 -- 172.21.15.1:0/672058359 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29b8003680 con 0x7f29c010e3b0 2024-04-26T01:48:34.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.938+0000 7f29c6600700 1 -- 172.21.15.1:0/672058359 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29c01a49c0 con 0x7f29c010e3b0 2024-04-26T01:48:34.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.938+0000 7f29bcc00700 1 -- 172.21.15.1:0/672058359 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b800fb40 con 0x7f29c010e3b0 2024-04-26T01:48:34.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.938+0000 7f29c6600700 1 -- 172.21.15.1:0/672058359 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29c01ac2a0 con 0x7f29c010e3b0 2024-04-26T01:48:34.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.939+0000 7f29c6600700 1 -- 172.21.15.1:0/672058359 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29c010a7b0 con 0x7f29c010e3b0 2024-04-26T01:48:34.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.941+0000 7f29bcc00700 1 -- 172.21.15.1:0/672058359 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f29b8029070 con 0x7f29c010e3b0 2024-04-26T01:48:34.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.943+0000 7f29bcc00700 1 --2- 172.21.15.1:0/672058359 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f29a40a3cd0 0x7f29a40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:34.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.943+0000 7f29bcc00700 1 -- 172.21.15.1:0/672058359 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f29b80d0df0 con 0x7f29c010e3b0 2024-04-26T01:48:34.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.943+0000 7f29bf400700 1 --2- 172.21.15.1:0/672058359 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f29a40a3cd0 0x7f29a40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:34.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.944+0000 7f29bf400700 1 --2- 172.21.15.1:0/672058359 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f29a40a3cd0 0x7f29a40a6190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f29b0005950 tx=0x7f29b000b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:34.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:34.947+0000 7f29bcc00700 1 -- 172.21.15.1:0/672058359 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f29b805dff0 con 0x7f29c010e3b0 2024-04-26T01:48:35.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:35.274+0000 7f29c6600700 1 -- 172.21.15.1:0/672058359 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f29c002df80 con 0x7f29c010e3b0 2024-04-26T01:48:35.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:35.275+0000 7f29bcc00700 1 -- 172.21.15.1:0/672058359 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2440 (secure 0 0 0) 0x7f29b8013070 con 0x7f29c010e3b0 2024-04-26T01:48:35.273 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:48:35.273 INFO:teuthology.orchestra.run.smithi001.stdout:{"fsid":"77cfcc80-036d-11ef-bc93-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":483,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":92,"num_osds":12,"num_up_osds":12,"osd_up_since":1714095976,"num_in_osds":12,"osd_in_since":1714095956,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":64,"data_bytes":654950,"bytes_used":367652864,"bytes_avail":1151572774912,"bytes_total":1151940427776,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":9,"recovering_keys_per_sec":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"read_bytes_sec":2388,"read_op_per_sec":5,"write_op_per_sec":0},"fsmap":{"epoch":25,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"e","status":"up:standby-replay","gid":24464},{"filesystem_id":1,"rank":1,"name":"h","status":"up:standby-replay","gid":24479},{"filesystem_id":1,"rank":0,"name":"f","status":"up:standby-replay","gid":24487},{"filesystem_id":1,"rank":2,"name":"c","status":"up:active","gid":24499},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24514},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24518}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.1:8443/"}},"servicemap":{"epoch":26,"modified":"2024-04-26T01:48:08.813167+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T01:48:35.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:35.278+0000 7f29c6600700 1 -- 172.21.15.1:0/672058359 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f29a40a3cd0 msgr2=0x7f29a40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:35.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:35.278+0000 7f29c6600700 1 --2- 172.21.15.1:0/672058359 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f29a40a3cd0 0x7f29a40a6190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f29b0005950 tx=0x7f29b000b500 comp rx=0 tx=0).stop 2024-04-26T01:48:35.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:35.278+0000 7f29c6600700 1 -- 172.21.15.1:0/672058359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29c010e3b0 msgr2=0x7f29c01a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:35.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:35.278+0000 7f29c6600700 1 --2- 172.21.15.1:0/672058359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29c010e3b0 0x7f29c01a3ff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f29b80098e0 tx=0x7f29b80057c0 comp rx=0 tx=0).stop 2024-04-26T01:48:35.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:35.278+0000 7f29c6600700 1 -- 172.21.15.1:0/672058359 shutdown_connections 2024-04-26T01:48:35.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:35.278+0000 7f29c6600700 1 --2- 172.21.15.1:0/672058359 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f29a40a3cd0 0x7f29a40a6190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:35.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:35.278+0000 7f29c6600700 1 --2- 172.21.15.1:0/672058359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29c010e3b0 0x7f29c01a3ff0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:35.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:35.278+0000 7f29c6600700 1 --2- 172.21.15.1:0/672058359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c01092d0 0x7f29c019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:35.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:35.278+0000 7f29c6600700 1 --2- 172.21.15.1:0/672058359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29c0109c80 0x7f29c019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:35.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:35.278+0000 7f29c6600700 1 -- 172.21.15.1:0/672058359 >> 172.21.15.1:0/672058359 conn(0x7f29c0076210 msgr2=0x7f29c00767d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:35.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:35.278+0000 7f29c6600700 1 -- 172.21.15.1:0/672058359 shutdown_connections 2024-04-26T01:48:35.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:35.278+0000 7f29c6600700 1 -- 172.21.15.1:0/672058359 wait complete. 2024-04-26T01:48:35.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:35 smithi001 ceph-mon[27822]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-04-26T01:48:35.931 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:35 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/672058359' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-26T01:48:36.010 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-04-26T01:48:36.010 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:48:36.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:35 smithi097 ceph-mon[32853]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-04-26T01:48:36.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:35 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/672058359' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-26T01:48:36.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:35 smithi173 ceph-mon[32553]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-04-26T01:48:36.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:35 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/672058359' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-26T01:48:36.502 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:38.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:37 smithi001 ceph-mon[27822]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-04-26T01:48:38.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:37 smithi097 ceph-mon[32853]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-04-26T01:48:38.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:37 smithi173 ceph-mon[32553]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-04-26T01:48:38.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.612+0000 7f08b0a00700 1 -- 172.21.15.1:0/3469671786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08ac108960 msgr2=0x7f08ac108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:38.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.612+0000 7f08b0a00700 1 --2- 172.21.15.1:0/3469671786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08ac108960 0x7f08ac108d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f089c009990 tx=0x7f089c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:48:38.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.612+0000 7f08b0a00700 1 -- 172.21.15.1:0/3469671786 shutdown_connections 2024-04-26T01:48:38.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.612+0000 7f08b0a00700 1 --2- 172.21.15.1:0/3469671786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08ac103320 0x7f08ac10f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:38.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.612+0000 7f08b0a00700 1 --2- 172.21.15.1:0/3469671786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08ac108960 0x7f08ac108d40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:38.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.612+0000 7f08b0a00700 1 --2- 172.21.15.1:0/3469671786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08ac102960 0x7f08ac102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:38.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.612+0000 7f08b0a00700 1 -- 172.21.15.1:0/3469671786 >> 172.21.15.1:0/3469671786 conn(0x7f08ac0fe490 msgr2=0x7f08ac1008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:38.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.612+0000 7f08b0a00700 1 -- 172.21.15.1:0/3469671786 shutdown_connections 2024-04-26T01:48:38.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.612+0000 7f08b0a00700 1 -- 172.21.15.1:0/3469671786 wait complete. 2024-04-26T01:48:38.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.613+0000 7f08b0a00700 1 Processor -- start 2024-04-26T01:48:38.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.613+0000 7f08b0a00700 1 -- start start 2024-04-26T01:48:38.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.614+0000 7f08b0a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08ac102960 0x7f08ac074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:38.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.614+0000 7f08b0a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08ac103320 0x7f08ac075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:38.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.614+0000 7f08b0a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08ac108960 0x7f08ac0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:38.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.614+0000 7f08b0a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f08ac112b70 con 0x7f08ac108960 2024-04-26T01:48:38.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.614+0000 7f08b0a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f08ac1129f0 con 0x7f08ac103320 2024-04-26T01:48:38.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.614+0000 7f08b0a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f08ac112cf0 con 0x7f08ac102960 2024-04-26T01:48:38.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.614+0000 7f08ab200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08ac108960 0x7f08ac0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:38.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.614+0000 7f08a3e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08ac103320 0x7f08ac075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:38.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.614+0000 7f08aa800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08ac102960 0x7f08ac074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:38.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.614+0000 7f08a3e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08ac103320 0x7f08ac075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:49286/0 (socket says 172.21.15.1:49286) 2024-04-26T01:48:38.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.614+0000 7f08a3e00700 1 -- 172.21.15.1:0/987601605 learned_addr learned my addr 172.21.15.1:0/987601605 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:48:38.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.615+0000 7f08a3e00700 1 -- 172.21.15.1:0/987601605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08ac102960 msgr2=0x7f08ac074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:38.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.615+0000 7f08a3e00700 1 --2- 172.21.15.1:0/987601605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08ac102960 0x7f08ac074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:38.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.615+0000 7f08a3e00700 1 -- 172.21.15.1:0/987601605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08ac108960 msgr2=0x7f08ac0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:38.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.615+0000 7f08a3e00700 1 --2- 172.21.15.1:0/987601605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08ac108960 0x7f08ac0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:38.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.615+0000 7f08a3e00700 1 -- 172.21.15.1:0/987601605 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ac1a7590 con 0x7f08ac103320 2024-04-26T01:48:38.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.615+0000 7f08aa800700 1 --2- 172.21.15.1:0/987601605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08ac102960 0x7f08ac074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:48:38.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.615+0000 7f08a3e00700 1 --2- 172.21.15.1:0/987601605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08ac103320 0x7f08ac075280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f089400e9c0 tx=0x7f089400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:38.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.615+0000 7f08a3400700 1 -- 172.21.15.1:0/987601605 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089400cbd0 con 0x7f08ac103320 2024-04-26T01:48:38.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.616+0000 7f08a3400700 1 -- 172.21.15.1:0/987601605 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f089400ee50 con 0x7f08ac103320 2024-04-26T01:48:38.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.616+0000 7f08b0a00700 1 -- 172.21.15.1:0/987601605 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08ac1a7880 con 0x7f08ac103320 2024-04-26T01:48:38.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.616+0000 7f08a3400700 1 -- 172.21.15.1:0/987601605 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894005550 con 0x7f08ac103320 2024-04-26T01:48:38.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.616+0000 7f08b0a00700 1 -- 172.21.15.1:0/987601605 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08ac1a7d70 con 0x7f08ac103320 2024-04-26T01:48:38.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.617+0000 7f08b0a00700 1 -- 172.21.15.1:0/987601605 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08ac10aef0 con 0x7f08ac103320 2024-04-26T01:48:38.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.619+0000 7f08a3400700 1 -- 172.21.15.1:0/987601605 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f0894004750 con 0x7f08ac103320 2024-04-26T01:48:38.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.620+0000 7f08a3400700 1 --2- 172.21.15.1:0/987601605 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f08900a3cd0 0x7f08900a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:38.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.620+0000 7f08a3400700 1 -- 172.21.15.1:0/987601605 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f08940d08a0 con 0x7f08ac103320 2024-04-26T01:48:38.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.621+0000 7f08aa800700 1 --2- 172.21.15.1:0/987601605 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f08900a3cd0 0x7f08900a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:38.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.621+0000 7f08aa800700 1 --2- 172.21.15.1:0/987601605 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f08900a3cd0 0x7f08900a6190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f089c0098e0 tx=0x7f089c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:38.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.624+0000 7f08a3400700 1 -- 172.21.15.1:0/987601605 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f089409d0a0 con 0x7f08ac103320 2024-04-26T01:48:38.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.854+0000 7f08b0a00700 1 -- 172.21.15.1:0/987601605 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f08ac062e80 con 0x7f08900a3cd0 2024-04-26T01:48:38.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.864+0000 7f08a3400700 1 -- 172.21.15.1:0/987601605 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258725 (secure 0 0 0) 0x7f089c0652c0 con 0x7f08900a3cd0 2024-04-26T01:48:38.867 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:48:38.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.870+0000 7f08b0a00700 1 -- 172.21.15.1:0/987601605 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f08900a3cd0 msgr2=0x7f08900a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:38.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.870+0000 7f08b0a00700 1 --2- 172.21.15.1:0/987601605 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f08900a3cd0 0x7f08900a6190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f089c0098e0 tx=0x7f089c005960 comp rx=0 tx=0).stop 2024-04-26T01:48:38.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.870+0000 7f08b0a00700 1 -- 172.21.15.1:0/987601605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08ac103320 msgr2=0x7f08ac075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:38.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.870+0000 7f08b0a00700 1 --2- 172.21.15.1:0/987601605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08ac103320 0x7f08ac075280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f089400e9c0 tx=0x7f089400ecd0 comp rx=0 tx=0).stop 2024-04-26T01:48:38.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.871+0000 7f08b0a00700 1 -- 172.21.15.1:0/987601605 shutdown_connections 2024-04-26T01:48:38.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.871+0000 7f08b0a00700 1 --2- 172.21.15.1:0/987601605 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f08900a3cd0 0x7f08900a6190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:38.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.871+0000 7f08b0a00700 1 --2- 172.21.15.1:0/987601605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08ac108960 0x7f08ac0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:38.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.871+0000 7f08b0a00700 1 --2- 172.21.15.1:0/987601605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08ac102960 0x7f08ac074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:38.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.871+0000 7f08b0a00700 1 --2- 172.21.15.1:0/987601605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08ac103320 0x7f08ac075280 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:38.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.871+0000 7f08b0a00700 1 -- 172.21.15.1:0/987601605 >> 172.21.15.1:0/987601605 conn(0x7f08ac0fe490 msgr2=0x7f08ac0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:38.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.871+0000 7f08b0a00700 1 -- 172.21.15.1:0/987601605 shutdown_connections 2024-04-26T01:48:38.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:38.871+0000 7f08b0a00700 1 -- 172.21.15.1:0/987601605 wait complete. 2024-04-26T01:48:38.870 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-04-26T01:48:39.774 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-04-26T01:48:38.825860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654950,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":442,"num_read_kb":339,"num_write":346,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359036,"kb_used_data":13980,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582788,"statfs":{"total":1151940427776,"available":1151572774912,"internally_reserved":0,"allocated":14315520,"data_stored":7842030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005145"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398825+0000","last_change":"2024-04-26T01:47:27.140418+0000","last_active":"2024-04-26T01:48:22.398825+0000","last_peered":"2024-04-26T01:48:22.398825+0000","last_clean":"2024-04-26T01:48:22.398825+0000","last_became_active":"2024-04-26T01:47:27.140017+0000","last_became_peered":"2024-04-26T01:47:27.140017+0000","last_unstale":"2024-04-26T01:48:22.398825+0000","last_undegraded":"2024-04-26T01:48:22.398825+0000","last_fullsized":"2024-04-26T01:48:22.398825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:46:08.621222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404830+0000","last_change":"2024-04-26T01:47:27.138746+0000","last_active":"2024-04-26T01:48:22.404830+0000","last_peered":"2024-04-26T01:48:22.404830+0000","last_clean":"2024-04-26T01:48:22.404830+0000","last_became_active":"2024-04-26T01:47:27.136163+0000","last_became_peered":"2024-04-26T01:47:27.136163+0000","last_unstale":"2024-04-26T01:48:22.404830+0000","last_undegraded":"2024-04-26T01:48:22.404830+0000","last_fullsized":"2024-04-26T01:48:22.404830+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:01:36.195332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.403704+0000","last_change":"2024-04-26T01:47:27.127017+0000","last_active":"2024-04-26T01:48:22.403704+0000","last_peered":"2024-04-26T01:48:22.403704+0000","last_clean":"2024-04-26T01:48:22.403704+0000","last_became_active":"2024-04-26T01:47:27.126610+0000","last_became_peered":"2024-04-26T01:47:27.126610+0000","last_unstale":"2024-04-26T01:48:22.403704+0000","last_undegraded":"2024-04-26T01:48:22.403704+0000","last_fullsized":"2024-04-26T01:48:22.403704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:05:12.784486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746210+0000","last_change":"2024-04-26T01:47:27.117640+0000","last_active":"2024-04-26T01:48:22.746210+0000","last_peered":"2024-04-26T01:48:22.746210+0000","last_clean":"2024-04-26T01:48:22.746210+0000","last_became_active":"2024-04-26T01:47:27.117266+0000","last_became_peered":"2024-04-26T01:47:27.117266+0000","last_unstale":"2024-04-26T01:48:22.746210+0000","last_undegraded":"2024-04-26T01:48:22.746210+0000","last_fullsized":"2024-04-26T01:48:22.746210+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:26:57.824560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396755+0000","last_change":"2024-04-26T01:47:27.128185+0000","last_active":"2024-04-26T01:48:22.396755+0000","last_peered":"2024-04-26T01:48:22.396755+0000","last_clean":"2024-04-26T01:48:22.396755+0000","last_became_active":"2024-04-26T01:47:27.127794+0000","last_became_peered":"2024-04-26T01:47:27.127794+0000","last_unstale":"2024-04-26T01:48:22.396755+0000","last_undegraded":"2024-04-26T01:48:22.396755+0000","last_fullsized":"2024-04-26T01:48:22.396755+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:09:52.774007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.748601+0000","last_change":"2024-04-26T01:48:22.748601+0000","last_active":"2024-04-26T01:48:22.748601+0000","last_peered":"2024-04-26T01:48:22.748601+0000","last_clean":"2024-04-26T01:48:22.748601+0000","last_became_active":"2024-04-26T01:48:22.748229+0000","last_became_peered":"2024-04-26T01:48:22.748229+0000","last_unstale":"2024-04-26T01:48:22.748601+0000","last_undegraded":"2024-04-26T01:48:22.748601+0000","last_fullsized":"2024-04-26T01:48:22.748601+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:46:30.433063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400077+0000","last_change":"2024-04-26T01:47:27.123786+0000","last_active":"2024-04-26T01:48:22.400077+0000","last_peered":"2024-04-26T01:48:22.400077+0000","last_clean":"2024-04-26T01:48:22.400077+0000","last_became_active":"2024-04-26T01:47:27.123473+0000","last_became_peered":"2024-04-26T01:47:27.123473+0000","last_unstale":"2024-04-26T01:48:22.400077+0000","last_undegraded":"2024-04-26T01:48:22.400077+0000","last_fullsized":"2024-04-26T01:48:22.400077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:48:55.031147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404450+0000","last_change":"2024-04-26T01:47:27.118756+0000","last_active":"2024-04-26T01:48:22.404450+0000","last_peered":"2024-04-26T01:48:22.404450+0000","last_clean":"2024-04-26T01:48:22.404450+0000","last_became_active":"2024-04-26T01:47:27.118406+0000","last_became_peered":"2024-04-26T01:47:27.118406+0000","last_unstale":"2024-04-26T01:48:22.404450+0000","last_undegraded":"2024-04-26T01:48:22.404450+0000","last_fullsized":"2024-04-26T01:48:22.404450+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:05.793682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398754+0000","last_change":"2024-04-26T01:47:27.117622+0000","last_active":"2024-04-26T01:48:22.398754+0000","last_peered":"2024-04-26T01:48:22.398754+0000","last_clean":"2024-04-26T01:48:22.398754+0000","last_became_active":"2024-04-26T01:47:27.115206+0000","last_became_peered":"2024-04-26T01:47:27.115206+0000","last_unstale":"2024-04-26T01:48:22.398754+0000","last_undegraded":"2024-04-26T01:48:22.398754+0000","last_fullsized":"2024-04-26T01:48:22.398754+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:13:17.866630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405408+0000","last_change":"2024-04-26T01:47:27.119524+0000","last_active":"2024-04-26T01:48:22.405408+0000","last_peered":"2024-04-26T01:48:22.405408+0000","last_clean":"2024-04-26T01:48:22.405408+0000","last_became_active":"2024-04-26T01:47:27.119008+0000","last_became_peered":"2024-04-26T01:47:27.119008+0000","last_unstale":"2024-04-26T01:48:22.405408+0000","last_undegraded":"2024-04-26T01:48:22.405408+0000","last_fullsized":"2024-04-26T01:48:22.405408+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:37:31.894486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400411+0000","last_change":"2024-04-26T01:47:27.122361+0000","last_active":"2024-04-26T01:48:22.400411+0000","last_peered":"2024-04-26T01:48:22.400411+0000","last_clean":"2024-04-26T01:48:22.400411+0000","last_became_active":"2024-04-26T01:47:27.121748+0000","last_became_peered":"2024-04-26T01:47:27.121748+0000","last_unstale":"2024-04-26T01:48:22.400411+0000","last_undegraded":"2024-04-26T01:48:22.400411+0000","last_fullsized":"2024-04-26T01:48:22.400411+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:18.680909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746160+0000","last_change":"2024-04-26T01:47:27.116833+0000","last_active":"2024-04-26T01:48:22.746160+0000","last_peered":"2024-04-26T01:48:22.746160+0000","last_clean":"2024-04-26T01:48:22.746160+0000","last_became_active":"2024-04-26T01:47:27.116508+0000","last_became_peered":"2024-04-26T01:47:27.116508+0000","last_unstale":"2024-04-26T01:48:22.746160+0000","last_undegraded":"2024-04-26T01:48:22.746160+0000","last_fullsized":"2024-04-26T01:48:22.746160+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:27:45.864521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405977+0000","last_change":"2024-04-26T01:47:27.125342+0000","last_active":"2024-04-26T01:48:22.405977+0000","last_peered":"2024-04-26T01:48:22.405977+0000","last_clean":"2024-04-26T01:48:22.405977+0000","last_became_active":"2024-04-26T01:47:27.124862+0000","last_became_peered":"2024-04-26T01:47:27.124862+0000","last_unstale":"2024-04-26T01:48:22.405977+0000","last_undegraded":"2024-04-26T01:48:22.405977+0000","last_fullsized":"2024-04-26T01:48:22.405977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:48.654711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.403399+0000","last_change":"2024-04-26T01:47:27.127365+0000","last_active":"2024-04-26T01:48:22.403399+0000","last_peered":"2024-04-26T01:48:22.403399+0000","last_clean":"2024-04-26T01:48:22.403399+0000","last_became_active":"2024-04-26T01:47:27.126868+0000","last_became_peered":"2024-04-26T01:47:27.126868+0000","last_unstale":"2024-04-26T01:48:22.403399+0000","last_undegraded":"2024-04-26T01:48:22.403399+0000","last_fullsized":"2024-04-26T01:48:22.403399+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:07:46.400333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.401775+0000","last_change":"2024-04-26T01:47:27.135584+0000","last_active":"2024-04-26T01:48:22.401775+0000","last_peered":"2024-04-26T01:48:22.401775+0000","last_clean":"2024-04-26T01:48:22.401775+0000","last_became_active":"2024-04-26T01:47:27.134849+0000","last_became_peered":"2024-04-26T01:47:27.134849+0000","last_unstale":"2024-04-26T01:48:22.401775+0000","last_undegraded":"2024-04-26T01:48:22.401775+0000","last_fullsized":"2024-04-26T01:48:22.401775+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:57:48.821595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398274+0000","last_change":"2024-04-26T01:47:27.136182+0000","last_active":"2024-04-26T01:48:22.398274+0000","last_peered":"2024-04-26T01:48:22.398274+0000","last_clean":"2024-04-26T01:48:22.398274+0000","last_became_active":"2024-04-26T01:47:27.135787+0000","last_became_peered":"2024-04-26T01:47:27.135787+0000","last_unstale":"2024-04-26T01:48:22.398274+0000","last_undegraded":"2024-04-26T01:48:22.398274+0000","last_fullsized":"2024-04-26T01:48:22.398274+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:38:14.275898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399072+0000","last_change":"2024-04-26T01:47:27.140700+0000","last_active":"2024-04-26T01:48:22.399072+0000","last_peered":"2024-04-26T01:48:22.399072+0000","last_clean":"2024-04-26T01:48:22.399072+0000","last_became_active":"2024-04-26T01:47:27.140025+0000","last_became_peered":"2024-04-26T01:47:27.140025+0000","last_unstale":"2024-04-26T01:48:22.399072+0000","last_undegraded":"2024-04-26T01:48:22.399072+0000","last_fullsized":"2024-04-26T01:48:22.399072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:50.465236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.747360+0000","last_change":"2024-04-26T01:47:27.117256+0000","last_active":"2024-04-26T01:48:22.747360+0000","last_peered":"2024-04-26T01:48:22.747360+0000","last_clean":"2024-04-26T01:48:22.747360+0000","last_became_active":"2024-04-26T01:47:27.116825+0000","last_became_peered":"2024-04-26T01:47:27.116825+0000","last_unstale":"2024-04-26T01:48:22.747360+0000","last_undegraded":"2024-04-26T01:48:22.747360+0000","last_fullsized":"2024-04-26T01:48:22.747360+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:23.313712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399301+0000","last_change":"2024-04-26T01:47:27.130233+0000","last_active":"2024-04-26T01:48:22.399301+0000","last_peered":"2024-04-26T01:48:22.399301+0000","last_clean":"2024-04-26T01:48:22.399301+0000","last_became_active":"2024-04-26T01:47:27.129852+0000","last_became_peered":"2024-04-26T01:47:27.129852+0000","last_unstale":"2024-04-26T01:48:22.399301+0000","last_undegraded":"2024-04-26T01:48:22.399301+0000","last_fullsized":"2024-04-26T01:48:22.399301+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:49:01.407264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.749092+0000","last_change":"2024-04-26T01:47:23.068477+0000","last_active":"2024-04-26T01:48:22.749092+0000","last_peered":"2024-04-26T01:48:22.749092+0000","last_clean":"2024-04-26T01:48:22.749092+0000","last_became_active":"2024-04-26T01:47:23.068061+0000","last_became_peered":"2024-04-26T01:47:23.068061+0000","last_unstale":"2024-04-26T01:48:22.749092+0000","last_undegraded":"2024-04-26T01:48:22.749092+0000","last_fullsized":"2024-04-26T01:48:22.749092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:02.153724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.406110+0000","last_change":"2024-04-26T01:47:27.130385+0000","last_active":"2024-04-26T01:48:22.406110+0000","last_peered":"2024-04-26T01:48:22.406110+0000","last_clean":"2024-04-26T01:48:22.406110+0000","last_became_active":"2024-04-26T01:47:27.129912+0000","last_became_peered":"2024-04-26T01:47:27.129912+0000","last_unstale":"2024-04-26T01:48:22.406110+0000","last_undegraded":"2024-04-26T01:48:22.406110+0000","last_fullsized":"2024-04-26T01:48:22.406110+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:26:03.010544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.407897+0000","last_change":"2024-04-26T01:47:23.110806+0000","last_active":"2024-04-26T01:48:22.407897+0000","last_peered":"2024-04-26T01:48:22.407897+0000","last_clean":"2024-04-26T01:48:22.407897+0000","last_became_active":"2024-04-26T01:47:23.110441+0000","last_became_peered":"2024-04-26T01:47:23.110441+0000","last_unstale":"2024-04-26T01:48:22.407897+0000","last_undegraded":"2024-04-26T01:48:22.407897+0000","last_fullsized":"2024-04-26T01:48:22.407897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:39.239172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.406491+0000","last_change":"2024-04-26T01:47:27.138628+0000","last_active":"2024-04-26T01:48:22.406491+0000","last_peered":"2024-04-26T01:48:22.406491+0000","last_clean":"2024-04-26T01:48:22.406491+0000","last_became_active":"2024-04-26T01:47:27.130483+0000","last_became_peered":"2024-04-26T01:47:27.130483+0000","last_unstale":"2024-04-26T01:48:22.406491+0000","last_undegraded":"2024-04-26T01:48:22.406491+0000","last_fullsized":"2024-04-26T01:48:22.406491+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:40:41.242856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.397138+0000","last_change":"2024-04-26T01:47:23.060066+0000","last_active":"2024-04-26T01:48:22.397138+0000","last_peered":"2024-04-26T01:48:22.397138+0000","last_clean":"2024-04-26T01:48:22.397138+0000","last_became_active":"2024-04-26T01:47:23.059526+0000","last_became_peered":"2024-04-26T01:47:23.059526+0000","last_unstale":"2024-04-26T01:48:22.397138+0000","last_undegraded":"2024-04-26T01:48:22.397138+0000","last_fullsized":"2024-04-26T01:48:22.397138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:28:22.623695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398426+0000","last_change":"2024-04-26T01:47:27.128343+0000","last_active":"2024-04-26T01:48:22.398426+0000","last_peered":"2024-04-26T01:48:22.398426+0000","last_clean":"2024-04-26T01:48:22.398426+0000","last_became_active":"2024-04-26T01:47:27.127920+0000","last_became_peered":"2024-04-26T01:47:27.127920+0000","last_unstale":"2024-04-26T01:48:22.398426+0000","last_undegraded":"2024-04-26T01:48:22.398426+0000","last_fullsized":"2024-04-26T01:48:22.398426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:06.725033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396937+0000","last_change":"2024-04-26T01:47:23.057783+0000","last_active":"2024-04-26T01:48:22.396937+0000","last_peered":"2024-04-26T01:48:22.396937+0000","last_clean":"2024-04-26T01:48:22.396937+0000","last_became_active":"2024-04-26T01:47:23.057391+0000","last_became_peered":"2024-04-26T01:47:23.057391+0000","last_unstale":"2024-04-26T01:48:22.396937+0000","last_undegraded":"2024-04-26T01:48:22.396937+0000","last_fullsized":"2024-04-26T01:48:22.396937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:10:10.050787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746112+0000","last_change":"2024-04-26T01:47:27.116458+0000","last_active":"2024-04-26T01:48:22.746112+0000","last_peered":"2024-04-26T01:48:22.746112+0000","last_clean":"2024-04-26T01:48:22.746112+0000","last_became_active":"2024-04-26T01:47:27.116084+0000","last_became_peered":"2024-04-26T01:47:27.116084+0000","last_unstale":"2024-04-26T01:48:22.746112+0000","last_undegraded":"2024-04-26T01:48:22.746112+0000","last_fullsized":"2024-04-26T01:48:22.746112+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:06:34.517478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":77,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:32.320529+0000","last_change":"2024-04-26T01:47:23.060657+0000","last_active":"2024-04-26T01:48:32.320529+0000","last_peered":"2024-04-26T01:48:32.320529+0000","last_clean":"2024-04-26T01:48:32.320529+0000","last_became_active":"2024-04-26T01:47:23.060294+0000","last_became_peered":"2024-04-26T01:47:23.060294+0000","last_unstale":"2024-04-26T01:48:32.320529+0000","last_undegraded":"2024-04-26T01:48:32.320529+0000","last_fullsized":"2024-04-26T01:48:32.320529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:33.979717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396644+0000","last_change":"2024-04-26T01:47:27.120587+0000","last_active":"2024-04-26T01:48:22.396644+0000","last_peered":"2024-04-26T01:48:22.396644+0000","last_clean":"2024-04-26T01:48:22.396644+0000","last_became_active":"2024-04-26T01:47:27.120236+0000","last_became_peered":"2024-04-26T01:47:27.120236+0000","last_unstale":"2024-04-26T01:48:22.396644+0000","last_undegraded":"2024-04-26T01:48:22.396644+0000","last_fullsized":"2024-04-26T01:48:22.396644+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:30:38.488741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405079+0000","last_change":"2024-04-26T01:47:23.069395+0000","last_active":"2024-04-26T01:48:22.405079+0000","last_peered":"2024-04-26T01:48:22.405079+0000","last_clean":"2024-04-26T01:48:22.405079+0000","last_became_active":"2024-04-26T01:47:23.069102+0000","last_became_peered":"2024-04-26T01:47:23.069102+0000","last_unstale":"2024-04-26T01:48:22.405079+0000","last_undegraded":"2024-04-26T01:48:22.405079+0000","last_fullsized":"2024-04-26T01:48:22.405079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:45.162604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745788+0000","last_change":"2024-04-26T01:47:27.123859+0000","last_active":"2024-04-26T01:48:22.745788+0000","last_peered":"2024-04-26T01:48:22.745788+0000","last_clean":"2024-04-26T01:48:22.745788+0000","last_became_active":"2024-04-26T01:47:27.123438+0000","last_became_peered":"2024-04-26T01:47:27.123438+0000","last_unstale":"2024-04-26T01:48:22.745788+0000","last_undegraded":"2024-04-26T01:48:22.745788+0000","last_fullsized":"2024-04-26T01:48:22.745788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:33:53.542809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.401049+0000","last_change":"2024-04-26T01:47:23.071544+0000","last_active":"2024-04-26T01:48:22.401049+0000","last_peered":"2024-04-26T01:48:22.401049+0000","last_clean":"2024-04-26T01:48:22.401049+0000","last_became_active":"2024-04-26T01:47:23.071191+0000","last_became_peered":"2024-04-26T01:47:23.071191+0000","last_unstale":"2024-04-26T01:48:22.401049+0000","last_undegraded":"2024-04-26T01:48:22.401049+0000","last_fullsized":"2024-04-26T01:48:22.401049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:58.889561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404118+0000","last_change":"2024-04-26T01:47:27.131887+0000","last_active":"2024-04-26T01:48:22.404118+0000","last_peered":"2024-04-26T01:48:22.404118+0000","last_clean":"2024-04-26T01:48:22.404118+0000","last_became_active":"2024-04-26T01:47:27.127973+0000","last_became_peered":"2024-04-26T01:47:27.127973+0000","last_unstale":"2024-04-26T01:48:22.404118+0000","last_undegraded":"2024-04-26T01:48:22.404118+0000","last_fullsized":"2024-04-26T01:48:22.404118+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:54:35.305779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745599+0000","last_change":"2024-04-26T01:47:23.061353+0000","last_active":"2024-04-26T01:48:22.745599+0000","last_peered":"2024-04-26T01:48:22.745599+0000","last_clean":"2024-04-26T01:48:22.745599+0000","last_became_active":"2024-04-26T01:47:23.061037+0000","last_became_peered":"2024-04-26T01:47:23.061037+0000","last_unstale":"2024-04-26T01:48:22.745599+0000","last_undegraded":"2024-04-26T01:48:22.745599+0000","last_fullsized":"2024-04-26T01:48:22.745599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:13:59.617969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746265+0000","last_change":"2024-04-26T01:47:27.122303+0000","last_active":"2024-04-26T01:48:22.746265+0000","last_peered":"2024-04-26T01:48:22.746265+0000","last_clean":"2024-04-26T01:48:22.746265+0000","last_became_active":"2024-04-26T01:47:27.121919+0000","last_became_peered":"2024-04-26T01:47:27.121919+0000","last_unstale":"2024-04-26T01:48:22.746265+0000","last_undegraded":"2024-04-26T01:48:22.746265+0000","last_fullsized":"2024-04-26T01:48:22.746265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:21.676663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":76,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:35.324442+0000","last_change":"2024-04-26T01:47:23.057961+0000","last_active":"2024-04-26T01:48:35.324442+0000","last_peered":"2024-04-26T01:48:35.324442+0000","last_clean":"2024-04-26T01:48:35.324442+0000","last_became_active":"2024-04-26T01:47:23.057529+0000","last_became_peered":"2024-04-26T01:47:23.057529+0000","last_unstale":"2024-04-26T01:48:35.324442+0000","last_undegraded":"2024-04-26T01:48:35.324442+0000","last_fullsized":"2024-04-26T01:48:35.324442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:14.420265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":47,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398667+0000","last_change":"2024-04-26T01:47:27.120083+0000","last_active":"2024-04-26T01:48:22.398667+0000","last_peered":"2024-04-26T01:48:22.398667+0000","last_clean":"2024-04-26T01:48:22.398667+0000","last_became_active":"2024-04-26T01:47:27.118410+0000","last_became_peered":"2024-04-26T01:47:27.118410+0000","last_unstale":"2024-04-26T01:48:22.398667+0000","last_undegraded":"2024-04-26T01:48:22.398667+0000","last_fullsized":"2024-04-26T01:48:22.398667+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:28.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400220+0000","last_change":"2024-04-26T01:47:23.059221+0000","last_active":"2024-04-26T01:48:22.400220+0000","last_peered":"2024-04-26T01:48:22.400220+0000","last_clean":"2024-04-26T01:48:22.400220+0000","last_became_active":"2024-04-26T01:47:23.058881+0000","last_became_peered":"2024-04-26T01:47:23.058881+0000","last_unstale":"2024-04-26T01:48:22.400220+0000","last_undegraded":"2024-04-26T01:48:22.400220+0000","last_fullsized":"2024-04-26T01:48:22.400220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:30.433696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.406173+0000","last_change":"2024-04-26T01:47:27.136763+0000","last_active":"2024-04-26T01:48:22.406173+0000","last_peered":"2024-04-26T01:48:22.406173+0000","last_clean":"2024-04-26T01:48:22.406173+0000","last_became_active":"2024-04-26T01:47:27.133642+0000","last_became_peered":"2024-04-26T01:47:27.133642+0000","last_unstale":"2024-04-26T01:48:22.406173+0000","last_undegraded":"2024-04-26T01:48:22.406173+0000","last_fullsized":"2024-04-26T01:48:22.406173+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:05.901139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746094+0000","last_change":"2024-04-26T01:47:23.108883+0000","last_active":"2024-04-26T01:48:22.746094+0000","last_peered":"2024-04-26T01:48:22.746094+0000","last_clean":"2024-04-26T01:48:22.746094+0000","last_became_active":"2024-04-26T01:47:23.108499+0000","last_became_peered":"2024-04-26T01:47:23.108499+0000","last_unstale":"2024-04-26T01:48:22.746094+0000","last_undegraded":"2024-04-26T01:48:22.746094+0000","last_fullsized":"2024-04-26T01:48:22.746094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:14:17.555416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.410651+0000","last_change":"2024-04-26T01:48:22.410651+0000","last_active":"2024-04-26T01:48:22.410651+0000","last_peered":"2024-04-26T01:48:22.410651+0000","last_clean":"2024-04-26T01:48:22.410651+0000","last_became_active":"2024-04-26T01:48:22.410359+0000","last_became_peered":"2024-04-26T01:48:22.410359+0000","last_unstale":"2024-04-26T01:48:22.410651+0000","last_undegraded":"2024-04-26T01:48:22.410651+0000","last_fullsized":"2024-04-26T01:48:22.410651+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:53:20.686915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'29","reported_seq":158,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:36.332119+0000","last_change":"2024-04-26T01:47:23.054893+0000","last_active":"2024-04-26T01:48:36.332119+0000","last_peered":"2024-04-26T01:48:36.332119+0000","last_clean":"2024-04-26T01:48:36.332119+0000","last_became_active":"2024-04-26T01:47:23.054558+0000","last_became_peered":"2024-04-26T01:47:23.054558+0000","last_unstale":"2024-04-26T01:48:36.332119+0000","last_undegraded":"2024-04-26T01:48:36.332119+0000","last_fullsized":"2024-04-26T01:48:36.332119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:59.356841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60347,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":103,"num_read_kb":110,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396887+0000","last_change":"2024-04-26T01:47:27.116934+0000","last_active":"2024-04-26T01:48:22.396887+0000","last_peered":"2024-04-26T01:48:22.396887+0000","last_clean":"2024-04-26T01:48:22.396887+0000","last_became_active":"2024-04-26T01:47:27.116527+0000","last_became_peered":"2024-04-26T01:47:27.116527+0000","last_unstale":"2024-04-26T01:48:22.396887+0000","last_undegraded":"2024-04-26T01:48:22.396887+0000","last_fullsized":"2024-04-26T01:48:22.396887+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:09:45.820533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399072+0000","last_change":"2024-04-26T01:47:23.059032+0000","last_active":"2024-04-26T01:48:22.399072+0000","last_peered":"2024-04-26T01:48:22.399072+0000","last_clean":"2024-04-26T01:48:22.399072+0000","last_became_active":"2024-04-26T01:47:23.057992+0000","last_became_peered":"2024-04-26T01:47:23.057992+0000","last_unstale":"2024-04-26T01:48:22.399072+0000","last_undegraded":"2024-04-26T01:48:22.399072+0000","last_fullsized":"2024-04-26T01:48:22.399072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:46:53.164925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.402515+0000","last_change":"2024-04-26T01:47:27.134574+0000","last_active":"2024-04-26T01:48:22.402515+0000","last_peered":"2024-04-26T01:48:22.402515+0000","last_clean":"2024-04-26T01:48:22.402515+0000","last_became_active":"2024-04-26T01:47:27.134184+0000","last_became_peered":"2024-04-26T01:47:27.134184+0000","last_unstale":"2024-04-26T01:48:22.402515+0000","last_undegraded":"2024-04-26T01:48:22.402515+0000","last_fullsized":"2024-04-26T01:48:22.402515+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:42:26.428014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745665+0000","last_change":"2024-04-26T01:47:23.062649+0000","last_active":"2024-04-26T01:48:22.745665+0000","last_peered":"2024-04-26T01:48:22.745665+0000","last_clean":"2024-04-26T01:48:22.745665+0000","last_became_active":"2024-04-26T01:47:23.062282+0000","last_became_peered":"2024-04-26T01:47:23.062282+0000","last_unstale":"2024-04-26T01:48:22.745665+0000","last_undegraded":"2024-04-26T01:48:22.745665+0000","last_fullsized":"2024-04-26T01:48:22.745665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:38.699105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400362+0000","last_change":"2024-04-26T01:47:27.124720+0000","last_active":"2024-04-26T01:48:22.400362+0000","last_peered":"2024-04-26T01:48:22.400362+0000","last_clean":"2024-04-26T01:48:22.400362+0000","last_became_active":"2024-04-26T01:47:27.124378+0000","last_became_peered":"2024-04-26T01:47:27.124378+0000","last_unstale":"2024-04-26T01:48:22.400362+0000","last_undegraded":"2024-04-26T01:48:22.400362+0000","last_fullsized":"2024-04-26T01:48:22.400362+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:45.462741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":82,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:37.350121+0000","last_change":"2024-04-26T01:47:23.056305+0000","last_active":"2024-04-26T01:48:37.350121+0000","last_peered":"2024-04-26T01:48:37.350121+0000","last_clean":"2024-04-26T01:48:37.350121+0000","last_became_active":"2024-04-26T01:47:23.055905+0000","last_became_peered":"2024-04-26T01:47:23.055905+0000","last_unstale":"2024-04-26T01:48:37.350121+0000","last_undegraded":"2024-04-26T01:48:37.350121+0000","last_fullsized":"2024-04-26T01:48:37.350121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:00:14.609673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400015+0000","last_change":"2024-04-26T01:47:27.119880+0000","last_active":"2024-04-26T01:48:22.400015+0000","last_peered":"2024-04-26T01:48:22.400015+0000","last_clean":"2024-04-26T01:48:22.400015+0000","last_became_active":"2024-04-26T01:47:27.119483+0000","last_became_peered":"2024-04-26T01:47:27.119483+0000","last_unstale":"2024-04-26T01:48:22.400015+0000","last_undegraded":"2024-04-26T01:48:22.400015+0000","last_fullsized":"2024-04-26T01:48:22.400015+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:15.523150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":66,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.758774+0000","last_change":"2024-04-26T01:48:22.758774+0000","last_active":"2024-04-26T01:48:22.758774+0000","last_peered":"2024-04-26T01:48:22.758774+0000","last_clean":"2024-04-26T01:48:22.758774+0000","last_became_active":"2024-04-26T01:48:22.747354+0000","last_became_peered":"2024-04-26T01:48:22.747354+0000","last_unstale":"2024-04-26T01:48:22.758774+0000","last_undegraded":"2024-04-26T01:48:22.758774+0000","last_fullsized":"2024-04-26T01:48:22.758774+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:23:15.886774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745649+0000","last_change":"2024-04-26T01:47:27.113508+0000","last_active":"2024-04-26T01:48:22.745649+0000","last_peered":"2024-04-26T01:48:22.745649+0000","last_clean":"2024-04-26T01:48:22.745649+0000","last_became_active":"2024-04-26T01:47:27.113066+0000","last_became_peered":"2024-04-26T01:47:27.113066+0000","last_unstale":"2024-04-26T01:48:22.745649+0000","last_undegraded":"2024-04-26T01:48:22.745649+0000","last_fullsized":"2024-04-26T01:48:22.745649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:37.525486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.397062+0000","last_change":"2024-04-26T01:47:23.059644+0000","last_active":"2024-04-26T01:48:22.397062+0000","last_peered":"2024-04-26T01:48:22.397062+0000","last_clean":"2024-04-26T01:48:22.397062+0000","last_became_active":"2024-04-26T01:47:23.058966+0000","last_became_peered":"2024-04-26T01:47:23.058966+0000","last_unstale":"2024-04-26T01:48:22.397062+0000","last_undegraded":"2024-04-26T01:48:22.397062+0000","last_fullsized":"2024-04-26T01:48:22.397062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:57:36.475935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.749354+0000","last_change":"2024-04-26T01:48:22.749354+0000","last_active":"2024-04-26T01:48:22.749354+0000","last_peered":"2024-04-26T01:48:22.749354+0000","last_clean":"2024-04-26T01:48:22.749354+0000","last_became_active":"2024-04-26T01:48:22.748978+0000","last_became_peered":"2024-04-26T01:48:22.748978+0000","last_unstale":"2024-04-26T01:48:22.749354+0000","last_undegraded":"2024-04-26T01:48:22.749354+0000","last_fullsized":"2024-04-26T01:48:22.749354+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:51:16.087091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746667+0000","last_change":"2024-04-26T01:47:23.110360+0000","last_active":"2024-04-26T01:48:22.746667+0000","last_peered":"2024-04-26T01:48:22.746667+0000","last_clean":"2024-04-26T01:48:22.746667+0000","last_became_active":"2024-04-26T01:47:23.109744+0000","last_became_peered":"2024-04-26T01:47:23.109744+0000","last_unstale":"2024-04-26T01:48:22.746667+0000","last_undegraded":"2024-04-26T01:48:22.746667+0000","last_fullsized":"2024-04-26T01:48:22.746667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:40:13.658396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405212+0000","last_change":"2024-04-26T01:47:27.125399+0000","last_active":"2024-04-26T01:48:22.405212+0000","last_peered":"2024-04-26T01:48:22.405212+0000","last_clean":"2024-04-26T01:48:22.405212+0000","last_became_active":"2024-04-26T01:47:27.124871+0000","last_became_peered":"2024-04-26T01:47:27.124871+0000","last_unstale":"2024-04-26T01:48:22.405212+0000","last_undegraded":"2024-04-26T01:48:22.405212+0000","last_fullsized":"2024-04-26T01:48:22.405212+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:28:48.285206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400178+0000","last_change":"2024-04-26T01:47:23.067862+0000","last_active":"2024-04-26T01:48:22.400178+0000","last_peered":"2024-04-26T01:48:22.400178+0000","last_clean":"2024-04-26T01:48:22.400178+0000","last_became_active":"2024-04-26T01:47:23.067526+0000","last_became_peered":"2024-04-26T01:47:23.067526+0000","last_unstale":"2024-04-26T01:48:22.400178+0000","last_undegraded":"2024-04-26T01:48:22.400178+0000","last_fullsized":"2024-04-26T01:48:22.400178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:22.668405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.748548+0000","last_change":"2024-04-26T01:48:22.748548+0000","last_active":"2024-04-26T01:48:22.748548+0000","last_peered":"2024-04-26T01:48:22.748548+0000","last_clean":"2024-04-26T01:48:22.748548+0000","last_became_active":"2024-04-26T01:48:22.748182+0000","last_became_peered":"2024-04-26T01:48:22.748182+0000","last_unstale":"2024-04-26T01:48:22.748548+0000","last_undegraded":"2024-04-26T01:48:22.748548+0000","last_fullsized":"2024-04-26T01:48:22.748548+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:20:27.596827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745783+0000","last_change":"2024-04-26T01:47:27.118307+0000","last_active":"2024-04-26T01:48:22.745783+0000","last_peered":"2024-04-26T01:48:22.745783+0000","last_clean":"2024-04-26T01:48:22.745783+0000","last_became_active":"2024-04-26T01:47:27.117986+0000","last_became_peered":"2024-04-26T01:47:27.117986+0000","last_unstale":"2024-04-26T01:48:22.745783+0000","last_undegraded":"2024-04-26T01:48:22.745783+0000","last_fullsized":"2024-04-26T01:48:22.745783+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:40.667143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":76,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:34.344575+0000","last_change":"2024-04-26T01:47:23.109973+0000","last_active":"2024-04-26T01:48:34.344575+0000","last_peered":"2024-04-26T01:48:34.344575+0000","last_clean":"2024-04-26T01:48:34.344575+0000","last_became_active":"2024-04-26T01:47:23.109446+0000","last_became_peered":"2024-04-26T01:47:23.109446+0000","last_unstale":"2024-04-26T01:48:34.344575+0000","last_undegraded":"2024-04-26T01:48:34.344575+0000","last_fullsized":"2024-04-26T01:48:34.344575+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:49:00.684164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.408237+0000","last_change":"2024-04-26T01:47:27.131502+0000","last_active":"2024-04-26T01:48:22.408237+0000","last_peered":"2024-04-26T01:48:22.408237+0000","last_clean":"2024-04-26T01:48:22.408237+0000","last_became_active":"2024-04-26T01:47:27.127697+0000","last_became_peered":"2024-04-26T01:47:27.127697+0000","last_unstale":"2024-04-26T01:48:22.408237+0000","last_undegraded":"2024-04-26T01:48:22.408237+0000","last_fullsized":"2024-04-26T01:48:22.408237+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:39:15.458189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746705+0000","last_change":"2024-04-26T01:47:23.066596+0000","last_active":"2024-04-26T01:48:22.746705+0000","last_peered":"2024-04-26T01:48:22.746705+0000","last_clean":"2024-04-26T01:48:22.746705+0000","last_became_active":"2024-04-26T01:47:23.066216+0000","last_became_peered":"2024-04-26T01:47:23.066216+0000","last_unstale":"2024-04-26T01:48:22.746705+0000","last_undegraded":"2024-04-26T01:48:22.746705+0000","last_fullsized":"2024-04-26T01:48:22.746705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:08:31.544831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400162+0000","last_change":"2024-04-26T01:45:54.834501+0000","last_active":"2024-04-26T01:48:22.400162+0000","last_peered":"2024-04-26T01:48:22.400162+0000","last_clean":"2024-04-26T01:48:22.400162+0000","last_became_active":"2024-04-26T01:45:54.833868+0000","last_became_peered":"2024-04-26T01:45:54.833868+0000","last_unstale":"2024-04-26T01:48:22.400162+0000","last_undegraded":"2024-04-26T01:48:22.400162+0000","last_fullsized":"2024-04-26T01:48:22.400162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:42:05.228956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:42:05.228956+0000","last_clean_scrub_stamp":"2024-04-26T01:42:05.228956+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:06:58.383315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400398+0000","last_change":"2024-04-26T01:47:27.126475+0000","last_active":"2024-04-26T01:48:22.400398+0000","last_peered":"2024-04-26T01:48:22.400398+0000","last_clean":"2024-04-26T01:48:22.400398+0000","last_became_active":"2024-04-26T01:47:27.123760+0000","last_became_peered":"2024-04-26T01:47:27.123760+0000","last_unstale":"2024-04-26T01:48:22.400398+0000","last_undegraded":"2024-04-26T01:48:22.400398+0000","last_fullsized":"2024-04-26T01:48:22.400398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:38:27.473909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400221+0000","last_change":"2024-04-26T01:47:23.059810+0000","last_active":"2024-04-26T01:48:22.400221+0000","last_peered":"2024-04-26T01:48:22.400221+0000","last_clean":"2024-04-26T01:48:22.400221+0000","last_became_active":"2024-04-26T01:47:23.059504+0000","last_became_peered":"2024-04-26T01:47:23.059504+0000","last_unstale":"2024-04-26T01:48:22.400221+0000","last_undegraded":"2024-04-26T01:48:22.400221+0000","last_fullsized":"2024-04-26T01:48:22.400221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:50:51.220951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396987+0000","last_change":"2024-04-26T01:47:23.109460+0000","last_active":"2024-04-26T01:48:22.396987+0000","last_peered":"2024-04-26T01:48:22.396987+0000","last_clean":"2024-04-26T01:48:22.396987+0000","last_became_active":"2024-04-26T01:47:23.109090+0000","last_became_peered":"2024-04-26T01:47:23.109090+0000","last_unstale":"2024-04-26T01:48:22.396987+0000","last_undegraded":"2024-04-26T01:48:22.396987+0000","last_fullsized":"2024-04-26T01:48:22.396987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:56.821312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.407355+0000","last_change":"2024-04-26T01:47:27.129537+0000","last_active":"2024-04-26T01:48:22.407355+0000","last_peered":"2024-04-26T01:48:22.407355+0000","last_clean":"2024-04-26T01:48:22.407355+0000","last_became_active":"2024-04-26T01:47:27.128124+0000","last_became_peered":"2024-04-26T01:47:27.128124+0000","last_unstale":"2024-04-26T01:48:22.407355+0000","last_undegraded":"2024-04-26T01:48:22.407355+0000","last_fullsized":"2024-04-26T01:48:22.407355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:14:26.262197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'11","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.747472+0000","last_change":"2024-04-26T01:47:23.064511+0000","last_active":"2024-04-26T01:48:22.747472+0000","last_peered":"2024-04-26T01:48:22.747472+0000","last_clean":"2024-04-26T01:48:22.747472+0000","last_became_active":"2024-04-26T01:47:23.064080+0000","last_became_peered":"2024-04-26T01:47:23.064080+0000","last_unstale":"2024-04-26T01:48:22.747472+0000","last_undegraded":"2024-04-26T01:48:22.747472+0000","last_fullsized":"2024-04-26T01:48:22.747472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:37.562644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404336+0000","last_change":"2024-04-26T01:47:27.124057+0000","last_active":"2024-04-26T01:48:22.404336+0000","last_peered":"2024-04-26T01:48:22.404336+0000","last_clean":"2024-04-26T01:48:22.404336+0000","last_became_active":"2024-04-26T01:47:27.123636+0000","last_became_peered":"2024-04-26T01:47:27.123636+0000","last_unstale":"2024-04-26T01:48:22.404336+0000","last_undegraded":"2024-04-26T01:48:22.404336+0000","last_fullsized":"2024-04-26T01:48:22.404336+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:56:55.691697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746155+0000","last_change":"2024-04-26T01:47:23.136571+0000","last_active":"2024-04-26T01:48:22.746155+0000","last_peered":"2024-04-26T01:48:22.746155+0000","last_clean":"2024-04-26T01:48:22.746155+0000","last_became_active":"2024-04-26T01:47:23.136205+0000","last_became_peered":"2024-04-26T01:47:23.136205+0000","last_unstale":"2024-04-26T01:48:22.746155+0000","last_undegraded":"2024-04-26T01:48:22.746155+0000","last_fullsized":"2024-04-26T01:48:22.746155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:24.418797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.747570+0000","last_change":"2024-04-26T01:48:22.747570+0000","last_active":"2024-04-26T01:48:22.747570+0000","last_peered":"2024-04-26T01:48:22.747570+0000","last_clean":"2024-04-26T01:48:22.747570+0000","last_became_active":"2024-04-26T01:48:22.747147+0000","last_became_peered":"2024-04-26T01:48:22.747147+0000","last_unstale":"2024-04-26T01:48:22.747570+0000","last_undegraded":"2024-04-26T01:48:22.747570+0000","last_fullsized":"2024-04-26T01:48:22.747570+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:37.034388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.414610+0000","last_change":"2024-04-26T01:48:22.414610+0000","last_active":"2024-04-26T01:48:22.414610+0000","last_peered":"2024-04-26T01:48:22.414610+0000","last_clean":"2024-04-26T01:48:22.414610+0000","last_became_active":"2024-04-26T01:48:22.410275+0000","last_became_peered":"2024-04-26T01:48:22.410275+0000","last_unstale":"2024-04-26T01:48:22.414610+0000","last_undegraded":"2024-04-26T01:48:22.414610+0000","last_fullsized":"2024-04-26T01:48:22.414610+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:20:48.131988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396626+0000","last_change":"2024-04-26T01:47:27.115498+0000","last_active":"2024-04-26T01:48:22.396626+0000","last_peered":"2024-04-26T01:48:22.396626+0000","last_clean":"2024-04-26T01:48:22.396626+0000","last_became_active":"2024-04-26T01:47:27.115134+0000","last_became_peered":"2024-04-26T01:47:27.115134+0000","last_unstale":"2024-04-26T01:48:22.396626+0000","last_undegraded":"2024-04-26T01:48:22.396626+0000","last_fullsized":"2024-04-26T01:48:22.396626+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:20.795224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405265+0000","last_change":"2024-04-26T01:47:23.065366+0000","last_active":"2024-04-26T01:48:22.405265+0000","last_peered":"2024-04-26T01:48:22.405265+0000","last_clean":"2024-04-26T01:48:22.405265+0000","last_became_active":"2024-04-26T01:47:23.063216+0000","last_became_peered":"2024-04-26T01:47:23.063216+0000","last_unstale":"2024-04-26T01:48:22.405265+0000","last_undegraded":"2024-04-26T01:48:22.405265+0000","last_fullsized":"2024-04-26T01:48:22.405265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:06:05.261907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398405+0000","last_change":"2024-04-26T01:47:27.127938+0000","last_active":"2024-04-26T01:48:22.398405+0000","last_peered":"2024-04-26T01:48:22.398405+0000","last_clean":"2024-04-26T01:48:22.398405+0000","last_became_active":"2024-04-26T01:47:27.127553+0000","last_became_peered":"2024-04-26T01:47:27.127553+0000","last_unstale":"2024-04-26T01:48:22.398405+0000","last_undegraded":"2024-04-26T01:48:22.398405+0000","last_fullsized":"2024-04-26T01:48:22.398405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:17:51.115349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398731+0000","last_change":"2024-04-26T01:47:23.060475+0000","last_active":"2024-04-26T01:48:22.398731+0000","last_peered":"2024-04-26T01:48:22.398731+0000","last_clean":"2024-04-26T01:48:22.398731+0000","last_became_active":"2024-04-26T01:47:23.060098+0000","last_became_peered":"2024-04-26T01:47:23.060098+0000","last_unstale":"2024-04-26T01:48:22.398731+0000","last_undegraded":"2024-04-26T01:48:22.398731+0000","last_fullsized":"2024-04-26T01:48:22.398731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:49.009455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.407796+0000","last_change":"2024-04-26T01:47:27.137491+0000","last_active":"2024-04-26T01:48:22.407796+0000","last_peered":"2024-04-26T01:48:22.407796+0000","last_clean":"2024-04-26T01:48:22.407796+0000","last_became_active":"2024-04-26T01:47:27.137059+0000","last_became_peered":"2024-04-26T01:47:27.137059+0000","last_unstale":"2024-04-26T01:48:22.407796+0000","last_undegraded":"2024-04-26T01:48:22.407796+0000","last_fullsized":"2024-04-26T01:48:22.407796+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:20:25.004074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405524+0000","last_change":"2024-04-26T01:48:22.405071+0000","last_active":"2024-04-26T01:48:22.405524+0000","last_peered":"2024-04-26T01:48:22.405524+0000","last_clean":"2024-04-26T01:48:22.405524+0000","last_became_active":"2024-04-26T01:48:22.401282+0000","last_became_peered":"2024-04-26T01:48:22.401282+0000","last_unstale":"2024-04-26T01:48:22.405524+0000","last_undegraded":"2024-04-26T01:48:22.405524+0000","last_fullsized":"2024-04-26T01:48:22.405524+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:24:53.628014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400294+0000","last_change":"2024-04-26T01:47:27.128253+0000","last_active":"2024-04-26T01:48:22.400294+0000","last_peered":"2024-04-26T01:48:22.400294+0000","last_clean":"2024-04-26T01:48:22.400294+0000","last_became_active":"2024-04-26T01:47:27.127869+0000","last_became_peered":"2024-04-26T01:47:27.127869+0000","last_unstale":"2024-04-26T01:48:22.400294+0000","last_undegraded":"2024-04-26T01:48:22.400294+0000","last_fullsized":"2024-04-26T01:48:22.400294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:11:23.476938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399412+0000","last_change":"2024-04-26T01:47:23.136862+0000","last_active":"2024-04-26T01:48:22.399412+0000","last_peered":"2024-04-26T01:48:22.399412+0000","last_clean":"2024-04-26T01:48:22.399412+0000","last_became_active":"2024-04-26T01:47:23.136380+0000","last_became_peered":"2024-04-26T01:47:23.136380+0000","last_unstale":"2024-04-26T01:48:22.399412+0000","last_undegraded":"2024-04-26T01:48:22.399412+0000","last_fullsized":"2024-04-26T01:48:22.399412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:39:18.103559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400366+0000","last_change":"2024-04-26T01:47:27.113363+0000","last_active":"2024-04-26T01:48:22.400366+0000","last_peered":"2024-04-26T01:48:22.400366+0000","last_clean":"2024-04-26T01:48:22.400366+0000","last_became_active":"2024-04-26T01:47:27.113040+0000","last_became_peered":"2024-04-26T01:47:27.113040+0000","last_unstale":"2024-04-26T01:48:22.400366+0000","last_undegraded":"2024-04-26T01:48:22.400366+0000","last_fullsized":"2024-04-26T01:48:22.400366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:42:16.428697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745984+0000","last_change":"2024-04-26T01:47:23.067257+0000","last_active":"2024-04-26T01:48:22.745984+0000","last_peered":"2024-04-26T01:48:22.745984+0000","last_clean":"2024-04-26T01:48:22.745984+0000","last_became_active":"2024-04-26T01:47:23.066858+0000","last_became_peered":"2024-04-26T01:47:23.066858+0000","last_unstale":"2024-04-26T01:48:22.745984+0000","last_undegraded":"2024-04-26T01:48:22.745984+0000","last_fullsized":"2024-04-26T01:48:22.745984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:22:07.848074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400154+0000","last_change":"2024-04-26T01:47:27.125885+0000","last_active":"2024-04-26T01:48:22.400154+0000","last_peered":"2024-04-26T01:48:22.400154+0000","last_clean":"2024-04-26T01:48:22.400154+0000","last_became_active":"2024-04-26T01:47:27.125524+0000","last_became_peered":"2024-04-26T01:47:27.125524+0000","last_unstale":"2024-04-26T01:48:22.400154+0000","last_undegraded":"2024-04-26T01:48:22.400154+0000","last_fullsized":"2024-04-26T01:48:22.400154+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:39.282023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.407608+0000","last_change":"2024-04-26T01:47:23.071051+0000","last_active":"2024-04-26T01:48:22.407608+0000","last_peered":"2024-04-26T01:48:22.407608+0000","last_clean":"2024-04-26T01:48:22.407608+0000","last_became_active":"2024-04-26T01:47:23.069524+0000","last_became_peered":"2024-04-26T01:47:23.069524+0000","last_unstale":"2024-04-26T01:48:22.407608+0000","last_undegraded":"2024-04-26T01:48:22.407608+0000","last_fullsized":"2024-04-26T01:48:22.407608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:24:57.643100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400370+0000","last_change":"2024-04-26T01:47:27.114325+0000","last_active":"2024-04-26T01:48:22.400370+0000","last_peered":"2024-04-26T01:48:22.400370+0000","last_clean":"2024-04-26T01:48:22.400370+0000","last_became_active":"2024-04-26T01:47:27.113952+0000","last_became_peered":"2024-04-26T01:47:27.113952+0000","last_unstale":"2024-04-26T01:48:22.400370+0000","last_undegraded":"2024-04-26T01:48:22.400370+0000","last_fullsized":"2024-04-26T01:48:22.400370+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:39:19.167175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398411+0000","last_change":"2024-04-26T01:47:27.118709+0000","last_active":"2024-04-26T01:48:22.398411+0000","last_peered":"2024-04-26T01:48:22.398411+0000","last_clean":"2024-04-26T01:48:22.398411+0000","last_became_active":"2024-04-26T01:47:27.118341+0000","last_became_peered":"2024-04-26T01:47:27.118341+0000","last_unstale":"2024-04-26T01:48:22.398411+0000","last_undegraded":"2024-04-26T01:48:22.398411+0000","last_fullsized":"2024-04-26T01:48:22.398411+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:13.736180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399082+0000","last_change":"2024-04-26T01:47:27.119239+0000","last_active":"2024-04-26T01:48:22.399082+0000","last_peered":"2024-04-26T01:48:22.399082+0000","last_clean":"2024-04-26T01:48:22.399082+0000","last_became_active":"2024-04-26T01:47:27.118877+0000","last_became_peered":"2024-04-26T01:47:27.118877+0000","last_unstale":"2024-04-26T01:48:22.399082+0000","last_undegraded":"2024-04-26T01:48:22.399082+0000","last_fullsized":"2024-04-26T01:48:22.399082+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:04:10.033517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398635+0000","last_change":"2024-04-26T01:47:27.142848+0000","last_active":"2024-04-26T01:48:22.398635+0000","last_peered":"2024-04-26T01:48:22.398635+0000","last_clean":"2024-04-26T01:48:22.398635+0000","last_became_active":"2024-04-26T01:47:27.142580+0000","last_became_peered":"2024-04-26T01:47:27.142580+0000","last_unstale":"2024-04-26T01:48:22.398635+0000","last_undegraded":"2024-04-26T01:48:22.398635+0000","last_fullsized":"2024-04-26T01:48:22.398635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:15:46.923922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404760+0000","last_change":"2024-04-26T01:47:27.134817+0000","last_active":"2024-04-26T01:48:22.404760+0000","last_peered":"2024-04-26T01:48:22.404760+0000","last_clean":"2024-04-26T01:48:22.404760+0000","last_became_active":"2024-04-26T01:47:27.134024+0000","last_became_peered":"2024-04-26T01:47:27.134024+0000","last_unstale":"2024-04-26T01:48:22.404760+0000","last_undegraded":"2024-04-26T01:48:22.404760+0000","last_fullsized":"2024-04-26T01:48:22.404760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:56.746427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405975+0000","last_change":"2024-04-26T01:47:27.143137+0000","last_active":"2024-04-26T01:48:22.405975+0000","last_peered":"2024-04-26T01:48:22.405975+0000","last_clean":"2024-04-26T01:48:22.405975+0000","last_became_active":"2024-04-26T01:47:27.142625+0000","last_became_peered":"2024-04-26T01:47:27.142625+0000","last_unstale":"2024-04-26T01:48:22.405975+0000","last_undegraded":"2024-04-26T01:48:22.405975+0000","last_fullsized":"2024-04-26T01:48:22.405975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:42:50.468382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.749582+0000","last_change":"2024-04-26T01:48:22.749582+0000","last_active":"2024-04-26T01:48:22.749582+0000","last_peered":"2024-04-26T01:48:22.749582+0000","last_clean":"2024-04-26T01:48:22.749582+0000","last_became_active":"2024-04-26T01:48:22.749191+0000","last_became_peered":"2024-04-26T01:48:22.749191+0000","last_unstale":"2024-04-26T01:48:22.749582+0000","last_undegraded":"2024-04-26T01:48:22.749582+0000","last_fullsized":"2024-04-26T01:48:22.749582+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:14.614842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399646+0000","last_change":"2024-04-26T01:47:27.123964+0000","last_active":"2024-04-26T01:48:22.399646+0000","last_peered":"2024-04-26T01:48:22.399646+0000","last_clean":"2024-04-26T01:48:22.399646+0000","last_became_active":"2024-04-26T01:47:27.123552+0000","last_became_peered":"2024-04-26T01:47:27.123552+0000","last_unstale":"2024-04-26T01:48:22.399646+0000","last_undegraded":"2024-04-26T01:48:22.399646+0000","last_fullsized":"2024-04-26T01:48:22.399646+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:33:12.566105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.412621+0000","last_change":"2024-04-26T01:48:22.412621+0000","last_active":"2024-04-26T01:48:22.412621+0000","last_peered":"2024-04-26T01:48:22.412621+0000","last_clean":"2024-04-26T01:48:22.412621+0000","last_became_active":"2024-04-26T01:48:22.411430+0000","last_became_peered":"2024-04-26T01:48:22.411430+0000","last_unstale":"2024-04-26T01:48:22.412621+0000","last_undegraded":"2024-04-26T01:48:22.412621+0000","last_fullsized":"2024-04-26T01:48:22.412621+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:34.152338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745667+0000","last_change":"2024-04-26T01:47:27.131437+0000","last_active":"2024-04-26T01:48:22.745667+0000","last_peered":"2024-04-26T01:48:22.745667+0000","last_clean":"2024-04-26T01:48:22.745667+0000","last_became_active":"2024-04-26T01:47:27.131138+0000","last_became_peered":"2024-04-26T01:47:27.131138+0000","last_unstale":"2024-04-26T01:48:22.745667+0000","last_undegraded":"2024-04-26T01:48:22.745667+0000","last_fullsized":"2024-04-26T01:48:22.745667+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:55:39.132972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404958+0000","last_change":"2024-04-26T01:47:27.120928+0000","last_active":"2024-04-26T01:48:22.404958+0000","last_peered":"2024-04-26T01:48:22.404958+0000","last_clean":"2024-04-26T01:48:22.404958+0000","last_became_active":"2024-04-26T01:47:27.120012+0000","last_became_peered":"2024-04-26T01:47:27.120012+0000","last_unstale":"2024-04-26T01:48:22.404958+0000","last_undegraded":"2024-04-26T01:48:22.404958+0000","last_fullsized":"2024-04-26T01:48:22.404958+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:19:44.209876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398493+0000","last_change":"2024-04-26T01:47:27.138766+0000","last_active":"2024-04-26T01:48:22.398493+0000","last_peered":"2024-04-26T01:48:22.398493+0000","last_clean":"2024-04-26T01:48:22.398493+0000","last_became_active":"2024-04-26T01:47:27.138413+0000","last_became_peered":"2024-04-26T01:47:27.138413+0000","last_unstale":"2024-04-26T01:48:22.398493+0000","last_undegraded":"2024-04-26T01:48:22.398493+0000","last_fullsized":"2024-04-26T01:48:22.398493+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:51:13.117307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.406315+0000","last_change":"2024-04-26T01:47:27.141338+0000","last_active":"2024-04-26T01:48:22.406315+0000","last_peered":"2024-04-26T01:48:22.406315+0000","last_clean":"2024-04-26T01:48:22.406315+0000","last_became_active":"2024-04-26T01:47:27.140825+0000","last_became_peered":"2024-04-26T01:47:27.140825+0000","last_unstale":"2024-04-26T01:48:22.406315+0000","last_undegraded":"2024-04-26T01:48:22.406315+0000","last_fullsized":"2024-04-26T01:48:22.406315+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:12:08.680876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398625+0000","last_change":"2024-04-26T01:47:27.114919+0000","last_active":"2024-04-26T01:48:22.398625+0000","last_peered":"2024-04-26T01:48:22.398625+0000","last_clean":"2024-04-26T01:48:22.398625+0000","last_became_active":"2024-04-26T01:47:27.114528+0000","last_became_peered":"2024-04-26T01:47:27.114528+0000","last_unstale":"2024-04-26T01:48:22.398625+0000","last_undegraded":"2024-04-26T01:48:22.398625+0000","last_fullsized":"2024-04-26T01:48:22.398625+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:15:25.620566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64348,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":306,"num_read_kb":220,"num_write":163,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318301,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":490347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579938,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28688,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716464,"statfs":{"total":95995035648,"available":95965659136,"internally_reserved":0,"allocated":1708032,"data_stored":1142566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743464,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":490583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005104,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":490503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299443,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1622016,"data_stored":1082433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462969,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":489972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":550112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":491243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":551350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":490283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920836,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":490912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32776,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712376,"statfs":{"total":95995035648,"available":95961473024,"internally_reserved":0,"allocated":1634304,"data_stored":1081726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:48:40.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:39 smithi001 ceph-mon[27822]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-04-26T01:48:40.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:39 smithi001 ceph-mon[27822]: from='client.24662 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:48:40.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:39 smithi097 ceph-mon[32853]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-04-26T01:48:40.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:39 smithi097 ceph-mon[32853]: from='client.24662 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:48:40.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:39 smithi173 ceph-mon[32553]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-04-26T01:48:40.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:39 smithi173 ceph-mon[32553]: from='client.24662 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:48:42.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:41 smithi097 ceph-mon[32853]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:48:42.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:41 smithi173 ceph-mon[32553]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:48:42.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:41 smithi001 ceph-mon[27822]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:48:42.779 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:48:43.266 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:44.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:43 smithi001 ceph-mon[27822]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:48:44.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:43 smithi097 ceph-mon[32853]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:48:44.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:43 smithi173 ceph-mon[32553]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:48:45.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.451+0000 7f1feaa00700 1 -- 172.21.15.1:0/3111627287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fe4105780 msgr2=0x7f1fe41097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:45.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.451+0000 7f1feaa00700 1 --2- 172.21.15.1:0/3111627287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fe4105780 0x7f1fe41097d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00099c0 tx=0x7f1fd0009cd0 comp rx=0 tx=0).stop 2024-04-26T01:48:45.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.451+0000 7f1feaa00700 1 -- 172.21.15.1:0/3111627287 shutdown_connections 2024-04-26T01:48:45.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.451+0000 7f1feaa00700 1 --2- 172.21.15.1:0/3111627287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fe4109eb0 0x7f1fe4111a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:45.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.451+0000 7f1feaa00700 1 --2- 172.21.15.1:0/3111627287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fe4104dd0 0x7f1fe41051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:45.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.451+0000 7f1feaa00700 1 --2- 172.21.15.1:0/3111627287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fe4105780 0x7f1fe41097d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:45.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.451+0000 7f1feaa00700 1 -- 172.21.15.1:0/3111627287 >> 172.21.15.1:0/3111627287 conn(0x7f1fe4100660 msgr2=0x7f1fe4102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:45.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.451+0000 7f1feaa00700 1 -- 172.21.15.1:0/3111627287 shutdown_connections 2024-04-26T01:48:45.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.451+0000 7f1feaa00700 1 -- 172.21.15.1:0/3111627287 wait complete. 2024-04-26T01:48:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.452+0000 7f1feaa00700 1 Processor -- start 2024-04-26T01:48:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.452+0000 7f1feaa00700 1 -- start start 2024-04-26T01:48:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.453+0000 7f1feaa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fe4104dd0 0x7f1fe419b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.453+0000 7f1feaa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fe4105780 0x7f1fe419b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.453+0000 7f1feaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fe4109eb0 0x7f1fe419fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.453+0000 7f1feaa00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe4114d50 con 0x7f1fe4109eb0 2024-04-26T01:48:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.453+0000 7f1feaa00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe4114bd0 con 0x7f1fe4105780 2024-04-26T01:48:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.453+0000 7f1feaa00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe4114ed0 con 0x7f1fe4104dd0 2024-04-26T01:48:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.453+0000 7f1fea000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fe4109eb0 0x7f1fe419fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.453+0000 7f1fe9600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fe4104dd0 0x7f1fe419b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.453+0000 7f1fe9600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fe4104dd0 0x7f1fe419b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:53062/0 (socket says 172.21.15.1:53062) 2024-04-26T01:48:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.453+0000 7f1fea000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fe4109eb0 0x7f1fe419fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38556/0 (socket says 172.21.15.1:38556) 2024-04-26T01:48:45.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.453+0000 7f1fe9600700 1 -- 172.21.15.1:0/1483639346 learned_addr learned my addr 172.21.15.1:0/1483639346 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:48:45.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.453+0000 7f1fe8c00700 1 --2- 172.21.15.1:0/1483639346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fe4105780 0x7f1fe419b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:45.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.454+0000 7f1fea000700 1 -- 172.21.15.1:0/1483639346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fe4104dd0 msgr2=0x7f1fe419b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:45.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.454+0000 7f1fea000700 1 --2- 172.21.15.1:0/1483639346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fe4104dd0 0x7f1fe419b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:45.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.454+0000 7f1fea000700 1 -- 172.21.15.1:0/1483639346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fe4105780 msgr2=0x7f1fe419b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:45.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.454+0000 7f1fea000700 1 --2- 172.21.15.1:0/1483639346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fe4105780 0x7f1fe419b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:45.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.454+0000 7f1fea000700 1 -- 172.21.15.1:0/1483639346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe41a0270 con 0x7f1fe4109eb0 2024-04-26T01:48:45.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.454+0000 7f1fe9600700 1 --2- 172.21.15.1:0/1483639346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fe4104dd0 0x7f1fe419b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:48:45.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.454+0000 7f1fe8c00700 1 --2- 172.21.15.1:0/1483639346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fe4105780 0x7f1fe419b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:48:45.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.454+0000 7f1fea000700 1 --2- 172.21.15.1:0/1483639346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fe4109eb0 0x7f1fe419fb30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00efb0 tx=0x7f1fdc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:45.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.454+0000 7f1fe2000700 1 -- 172.21.15.1:0/1483639346 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc003a00 con 0x7f1fe4109eb0 2024-04-26T01:48:45.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.454+0000 7f1fe2000700 1 -- 172.21.15.1:0/1483639346 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fdc00ce50 con 0x7f1fe4109eb0 2024-04-26T01:48:45.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.454+0000 7f1fe2000700 1 -- 172.21.15.1:0/1483639346 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc0047c0 con 0x7f1fe4109eb0 2024-04-26T01:48:45.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.454+0000 7f1feaa00700 1 -- 172.21.15.1:0/1483639346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe41a0560 con 0x7f1fe4109eb0 2024-04-26T01:48:45.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.455+0000 7f1feaa00700 1 -- 172.21.15.1:0/1483639346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe41a7de0 con 0x7f1fe4109eb0 2024-04-26T01:48:45.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.456+0000 7f1feaa00700 1 -- 172.21.15.1:0/1483639346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe4069730 con 0x7f1fe4109eb0 2024-04-26T01:48:45.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.458+0000 7f1fe2000700 1 -- 172.21.15.1:0/1483639346 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f1fdc012050 con 0x7f1fe4109eb0 2024-04-26T01:48:45.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.459+0000 7f1fe2000700 1 --2- 172.21.15.1:0/1483639346 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f1fcc0a3d50 0x7f1fcc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:45.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.459+0000 7f1fe2000700 1 -- 172.21.15.1:0/1483639346 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f1fdc0cfdd0 con 0x7f1fe4109eb0 2024-04-26T01:48:45.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.460+0000 7f1fe9600700 1 --2- 172.21.15.1:0/1483639346 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f1fcc0a3d50 0x7f1fcc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:45.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.460+0000 7f1fe9600700 1 --2- 172.21.15.1:0/1483639346 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f1fcc0a3d50 0x7f1fcc0a6210 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8011590 tx=0x7f1fd8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.464+0000 7f1fe2000700 1 -- 172.21.15.1:0/1483639346 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1fdc022060 con 0x7f1fe4109eb0 2024-04-26T01:48:45.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.692+0000 7f1feaa00700 1 -- 172.21.15.1:0/1483639346 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1fe419c5c0 con 0x7f1fcc0a3d50 2024-04-26T01:48:45.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.713+0000 7f1fe2000700 1 -- 172.21.15.1:0/1483639346 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258722 (secure 0 0 0) 0x7f1fe419c5c0 con 0x7f1fcc0a3d50 2024-04-26T01:48:45.716 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:48:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.719+0000 7f1feaa00700 1 -- 172.21.15.1:0/1483639346 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f1fcc0a3d50 msgr2=0x7f1fcc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.719+0000 7f1feaa00700 1 --2- 172.21.15.1:0/1483639346 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f1fcc0a3d50 0x7f1fcc0a6210 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8011590 tx=0x7f1fd8009260 comp rx=0 tx=0).stop 2024-04-26T01:48:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.719+0000 7f1feaa00700 1 -- 172.21.15.1:0/1483639346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fe4109eb0 msgr2=0x7f1fe419fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.719+0000 7f1feaa00700 1 --2- 172.21.15.1:0/1483639346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fe4109eb0 0x7f1fe419fb30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00efb0 tx=0x7f1fdc00c3a0 comp rx=0 tx=0).stop 2024-04-26T01:48:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.719+0000 7f1feaa00700 1 -- 172.21.15.1:0/1483639346 shutdown_connections 2024-04-26T01:48:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.719+0000 7f1feaa00700 1 --2- 172.21.15.1:0/1483639346 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f1fcc0a3d50 0x7f1fcc0a6210 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.719+0000 7f1feaa00700 1 --2- 172.21.15.1:0/1483639346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fe4109eb0 0x7f1fe419fb30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.719+0000 7f1feaa00700 1 --2- 172.21.15.1:0/1483639346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fe4104dd0 0x7f1fe419b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.719+0000 7f1feaa00700 1 --2- 172.21.15.1:0/1483639346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fe4105780 0x7f1fe419b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.719+0000 7f1feaa00700 1 -- 172.21.15.1:0/1483639346 >> 172.21.15.1:0/1483639346 conn(0x7f1fe4100660 msgr2=0x7f1fe4102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.719+0000 7f1feaa00700 1 -- 172.21.15.1:0/1483639346 shutdown_connections 2024-04-26T01:48:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:45.720+0000 7f1feaa00700 1 -- 172.21.15.1:0/1483639346 wait complete. 2024-04-26T01:48:45.719 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-04-26T01:48:46.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:45 smithi001 ceph-mon[27822]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:48:46.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:45 smithi097 ceph-mon[32853]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:48:46.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:45 smithi173 ceph-mon[32553]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:48:46.667 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-04-26T01:48:44.828479+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654950,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":477,"num_read_kb":354,"num_write":346,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359036,"kb_used_data":13980,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582788,"statfs":{"total":1151940427776,"available":1151572774912,"internally_reserved":0,"allocated":14315520,"data_stored":7842030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005376"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398825+0000","last_change":"2024-04-26T01:47:27.140418+0000","last_active":"2024-04-26T01:48:22.398825+0000","last_peered":"2024-04-26T01:48:22.398825+0000","last_clean":"2024-04-26T01:48:22.398825+0000","last_became_active":"2024-04-26T01:47:27.140017+0000","last_became_peered":"2024-04-26T01:47:27.140017+0000","last_unstale":"2024-04-26T01:48:22.398825+0000","last_undegraded":"2024-04-26T01:48:22.398825+0000","last_fullsized":"2024-04-26T01:48:22.398825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:46:08.621222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404830+0000","last_change":"2024-04-26T01:47:27.138746+0000","last_active":"2024-04-26T01:48:22.404830+0000","last_peered":"2024-04-26T01:48:22.404830+0000","last_clean":"2024-04-26T01:48:22.404830+0000","last_became_active":"2024-04-26T01:47:27.136163+0000","last_became_peered":"2024-04-26T01:47:27.136163+0000","last_unstale":"2024-04-26T01:48:22.404830+0000","last_undegraded":"2024-04-26T01:48:22.404830+0000","last_fullsized":"2024-04-26T01:48:22.404830+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:01:36.195332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.403704+0000","last_change":"2024-04-26T01:47:27.127017+0000","last_active":"2024-04-26T01:48:22.403704+0000","last_peered":"2024-04-26T01:48:22.403704+0000","last_clean":"2024-04-26T01:48:22.403704+0000","last_became_active":"2024-04-26T01:47:27.126610+0000","last_became_peered":"2024-04-26T01:47:27.126610+0000","last_unstale":"2024-04-26T01:48:22.403704+0000","last_undegraded":"2024-04-26T01:48:22.403704+0000","last_fullsized":"2024-04-26T01:48:22.403704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:05:12.784486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746210+0000","last_change":"2024-04-26T01:47:27.117640+0000","last_active":"2024-04-26T01:48:22.746210+0000","last_peered":"2024-04-26T01:48:22.746210+0000","last_clean":"2024-04-26T01:48:22.746210+0000","last_became_active":"2024-04-26T01:47:27.117266+0000","last_became_peered":"2024-04-26T01:47:27.117266+0000","last_unstale":"2024-04-26T01:48:22.746210+0000","last_undegraded":"2024-04-26T01:48:22.746210+0000","last_fullsized":"2024-04-26T01:48:22.746210+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:26:57.824560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396755+0000","last_change":"2024-04-26T01:47:27.128185+0000","last_active":"2024-04-26T01:48:22.396755+0000","last_peered":"2024-04-26T01:48:22.396755+0000","last_clean":"2024-04-26T01:48:22.396755+0000","last_became_active":"2024-04-26T01:47:27.127794+0000","last_became_peered":"2024-04-26T01:47:27.127794+0000","last_unstale":"2024-04-26T01:48:22.396755+0000","last_undegraded":"2024-04-26T01:48:22.396755+0000","last_fullsized":"2024-04-26T01:48:22.396755+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:09:52.774007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.748601+0000","last_change":"2024-04-26T01:48:22.748601+0000","last_active":"2024-04-26T01:48:22.748601+0000","last_peered":"2024-04-26T01:48:22.748601+0000","last_clean":"2024-04-26T01:48:22.748601+0000","last_became_active":"2024-04-26T01:48:22.748229+0000","last_became_peered":"2024-04-26T01:48:22.748229+0000","last_unstale":"2024-04-26T01:48:22.748601+0000","last_undegraded":"2024-04-26T01:48:22.748601+0000","last_fullsized":"2024-04-26T01:48:22.748601+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:46:30.433063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400077+0000","last_change":"2024-04-26T01:47:27.123786+0000","last_active":"2024-04-26T01:48:22.400077+0000","last_peered":"2024-04-26T01:48:22.400077+0000","last_clean":"2024-04-26T01:48:22.400077+0000","last_became_active":"2024-04-26T01:47:27.123473+0000","last_became_peered":"2024-04-26T01:47:27.123473+0000","last_unstale":"2024-04-26T01:48:22.400077+0000","last_undegraded":"2024-04-26T01:48:22.400077+0000","last_fullsized":"2024-04-26T01:48:22.400077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:48:55.031147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404450+0000","last_change":"2024-04-26T01:47:27.118756+0000","last_active":"2024-04-26T01:48:22.404450+0000","last_peered":"2024-04-26T01:48:22.404450+0000","last_clean":"2024-04-26T01:48:22.404450+0000","last_became_active":"2024-04-26T01:47:27.118406+0000","last_became_peered":"2024-04-26T01:47:27.118406+0000","last_unstale":"2024-04-26T01:48:22.404450+0000","last_undegraded":"2024-04-26T01:48:22.404450+0000","last_fullsized":"2024-04-26T01:48:22.404450+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:05.793682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398754+0000","last_change":"2024-04-26T01:47:27.117622+0000","last_active":"2024-04-26T01:48:22.398754+0000","last_peered":"2024-04-26T01:48:22.398754+0000","last_clean":"2024-04-26T01:48:22.398754+0000","last_became_active":"2024-04-26T01:47:27.115206+0000","last_became_peered":"2024-04-26T01:47:27.115206+0000","last_unstale":"2024-04-26T01:48:22.398754+0000","last_undegraded":"2024-04-26T01:48:22.398754+0000","last_fullsized":"2024-04-26T01:48:22.398754+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:13:17.866630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405408+0000","last_change":"2024-04-26T01:47:27.119524+0000","last_active":"2024-04-26T01:48:22.405408+0000","last_peered":"2024-04-26T01:48:22.405408+0000","last_clean":"2024-04-26T01:48:22.405408+0000","last_became_active":"2024-04-26T01:47:27.119008+0000","last_became_peered":"2024-04-26T01:47:27.119008+0000","last_unstale":"2024-04-26T01:48:22.405408+0000","last_undegraded":"2024-04-26T01:48:22.405408+0000","last_fullsized":"2024-04-26T01:48:22.405408+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:37:31.894486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400411+0000","last_change":"2024-04-26T01:47:27.122361+0000","last_active":"2024-04-26T01:48:22.400411+0000","last_peered":"2024-04-26T01:48:22.400411+0000","last_clean":"2024-04-26T01:48:22.400411+0000","last_became_active":"2024-04-26T01:47:27.121748+0000","last_became_peered":"2024-04-26T01:47:27.121748+0000","last_unstale":"2024-04-26T01:48:22.400411+0000","last_undegraded":"2024-04-26T01:48:22.400411+0000","last_fullsized":"2024-04-26T01:48:22.400411+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:18.680909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746160+0000","last_change":"2024-04-26T01:47:27.116833+0000","last_active":"2024-04-26T01:48:22.746160+0000","last_peered":"2024-04-26T01:48:22.746160+0000","last_clean":"2024-04-26T01:48:22.746160+0000","last_became_active":"2024-04-26T01:47:27.116508+0000","last_became_peered":"2024-04-26T01:47:27.116508+0000","last_unstale":"2024-04-26T01:48:22.746160+0000","last_undegraded":"2024-04-26T01:48:22.746160+0000","last_fullsized":"2024-04-26T01:48:22.746160+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:27:45.864521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405977+0000","last_change":"2024-04-26T01:47:27.125342+0000","last_active":"2024-04-26T01:48:22.405977+0000","last_peered":"2024-04-26T01:48:22.405977+0000","last_clean":"2024-04-26T01:48:22.405977+0000","last_became_active":"2024-04-26T01:47:27.124862+0000","last_became_peered":"2024-04-26T01:47:27.124862+0000","last_unstale":"2024-04-26T01:48:22.405977+0000","last_undegraded":"2024-04-26T01:48:22.405977+0000","last_fullsized":"2024-04-26T01:48:22.405977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:48.654711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.403399+0000","last_change":"2024-04-26T01:47:27.127365+0000","last_active":"2024-04-26T01:48:22.403399+0000","last_peered":"2024-04-26T01:48:22.403399+0000","last_clean":"2024-04-26T01:48:22.403399+0000","last_became_active":"2024-04-26T01:47:27.126868+0000","last_became_peered":"2024-04-26T01:47:27.126868+0000","last_unstale":"2024-04-26T01:48:22.403399+0000","last_undegraded":"2024-04-26T01:48:22.403399+0000","last_fullsized":"2024-04-26T01:48:22.403399+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:07:46.400333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.401775+0000","last_change":"2024-04-26T01:47:27.135584+0000","last_active":"2024-04-26T01:48:22.401775+0000","last_peered":"2024-04-26T01:48:22.401775+0000","last_clean":"2024-04-26T01:48:22.401775+0000","last_became_active":"2024-04-26T01:47:27.134849+0000","last_became_peered":"2024-04-26T01:47:27.134849+0000","last_unstale":"2024-04-26T01:48:22.401775+0000","last_undegraded":"2024-04-26T01:48:22.401775+0000","last_fullsized":"2024-04-26T01:48:22.401775+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:57:48.821595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398274+0000","last_change":"2024-04-26T01:47:27.136182+0000","last_active":"2024-04-26T01:48:22.398274+0000","last_peered":"2024-04-26T01:48:22.398274+0000","last_clean":"2024-04-26T01:48:22.398274+0000","last_became_active":"2024-04-26T01:47:27.135787+0000","last_became_peered":"2024-04-26T01:47:27.135787+0000","last_unstale":"2024-04-26T01:48:22.398274+0000","last_undegraded":"2024-04-26T01:48:22.398274+0000","last_fullsized":"2024-04-26T01:48:22.398274+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:38:14.275898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399072+0000","last_change":"2024-04-26T01:47:27.140700+0000","last_active":"2024-04-26T01:48:22.399072+0000","last_peered":"2024-04-26T01:48:22.399072+0000","last_clean":"2024-04-26T01:48:22.399072+0000","last_became_active":"2024-04-26T01:47:27.140025+0000","last_became_peered":"2024-04-26T01:47:27.140025+0000","last_unstale":"2024-04-26T01:48:22.399072+0000","last_undegraded":"2024-04-26T01:48:22.399072+0000","last_fullsized":"2024-04-26T01:48:22.399072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:50.465236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.747360+0000","last_change":"2024-04-26T01:47:27.117256+0000","last_active":"2024-04-26T01:48:22.747360+0000","last_peered":"2024-04-26T01:48:22.747360+0000","last_clean":"2024-04-26T01:48:22.747360+0000","last_became_active":"2024-04-26T01:47:27.116825+0000","last_became_peered":"2024-04-26T01:47:27.116825+0000","last_unstale":"2024-04-26T01:48:22.747360+0000","last_undegraded":"2024-04-26T01:48:22.747360+0000","last_fullsized":"2024-04-26T01:48:22.747360+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:23.313712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399301+0000","last_change":"2024-04-26T01:47:27.130233+0000","last_active":"2024-04-26T01:48:22.399301+0000","last_peered":"2024-04-26T01:48:22.399301+0000","last_clean":"2024-04-26T01:48:22.399301+0000","last_became_active":"2024-04-26T01:47:27.129852+0000","last_became_peered":"2024-04-26T01:47:27.129852+0000","last_unstale":"2024-04-26T01:48:22.399301+0000","last_undegraded":"2024-04-26T01:48:22.399301+0000","last_fullsized":"2024-04-26T01:48:22.399301+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:49:01.407264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.749092+0000","last_change":"2024-04-26T01:47:23.068477+0000","last_active":"2024-04-26T01:48:22.749092+0000","last_peered":"2024-04-26T01:48:22.749092+0000","last_clean":"2024-04-26T01:48:22.749092+0000","last_became_active":"2024-04-26T01:47:23.068061+0000","last_became_peered":"2024-04-26T01:47:23.068061+0000","last_unstale":"2024-04-26T01:48:22.749092+0000","last_undegraded":"2024-04-26T01:48:22.749092+0000","last_fullsized":"2024-04-26T01:48:22.749092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:02.153724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.406110+0000","last_change":"2024-04-26T01:47:27.130385+0000","last_active":"2024-04-26T01:48:22.406110+0000","last_peered":"2024-04-26T01:48:22.406110+0000","last_clean":"2024-04-26T01:48:22.406110+0000","last_became_active":"2024-04-26T01:47:27.129912+0000","last_became_peered":"2024-04-26T01:47:27.129912+0000","last_unstale":"2024-04-26T01:48:22.406110+0000","last_undegraded":"2024-04-26T01:48:22.406110+0000","last_fullsized":"2024-04-26T01:48:22.406110+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:26:03.010544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.407897+0000","last_change":"2024-04-26T01:47:23.110806+0000","last_active":"2024-04-26T01:48:22.407897+0000","last_peered":"2024-04-26T01:48:22.407897+0000","last_clean":"2024-04-26T01:48:22.407897+0000","last_became_active":"2024-04-26T01:47:23.110441+0000","last_became_peered":"2024-04-26T01:47:23.110441+0000","last_unstale":"2024-04-26T01:48:22.407897+0000","last_undegraded":"2024-04-26T01:48:22.407897+0000","last_fullsized":"2024-04-26T01:48:22.407897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:39.239172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.406491+0000","last_change":"2024-04-26T01:47:27.138628+0000","last_active":"2024-04-26T01:48:22.406491+0000","last_peered":"2024-04-26T01:48:22.406491+0000","last_clean":"2024-04-26T01:48:22.406491+0000","last_became_active":"2024-04-26T01:47:27.130483+0000","last_became_peered":"2024-04-26T01:47:27.130483+0000","last_unstale":"2024-04-26T01:48:22.406491+0000","last_undegraded":"2024-04-26T01:48:22.406491+0000","last_fullsized":"2024-04-26T01:48:22.406491+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:40:41.242856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.397138+0000","last_change":"2024-04-26T01:47:23.060066+0000","last_active":"2024-04-26T01:48:22.397138+0000","last_peered":"2024-04-26T01:48:22.397138+0000","last_clean":"2024-04-26T01:48:22.397138+0000","last_became_active":"2024-04-26T01:47:23.059526+0000","last_became_peered":"2024-04-26T01:47:23.059526+0000","last_unstale":"2024-04-26T01:48:22.397138+0000","last_undegraded":"2024-04-26T01:48:22.397138+0000","last_fullsized":"2024-04-26T01:48:22.397138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:28:22.623695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398426+0000","last_change":"2024-04-26T01:47:27.128343+0000","last_active":"2024-04-26T01:48:22.398426+0000","last_peered":"2024-04-26T01:48:22.398426+0000","last_clean":"2024-04-26T01:48:22.398426+0000","last_became_active":"2024-04-26T01:47:27.127920+0000","last_became_peered":"2024-04-26T01:47:27.127920+0000","last_unstale":"2024-04-26T01:48:22.398426+0000","last_undegraded":"2024-04-26T01:48:22.398426+0000","last_fullsized":"2024-04-26T01:48:22.398426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:06.725033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396937+0000","last_change":"2024-04-26T01:47:23.057783+0000","last_active":"2024-04-26T01:48:22.396937+0000","last_peered":"2024-04-26T01:48:22.396937+0000","last_clean":"2024-04-26T01:48:22.396937+0000","last_became_active":"2024-04-26T01:47:23.057391+0000","last_became_peered":"2024-04-26T01:47:23.057391+0000","last_unstale":"2024-04-26T01:48:22.396937+0000","last_undegraded":"2024-04-26T01:48:22.396937+0000","last_fullsized":"2024-04-26T01:48:22.396937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:10:10.050787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746112+0000","last_change":"2024-04-26T01:47:27.116458+0000","last_active":"2024-04-26T01:48:22.746112+0000","last_peered":"2024-04-26T01:48:22.746112+0000","last_clean":"2024-04-26T01:48:22.746112+0000","last_became_active":"2024-04-26T01:47:27.116084+0000","last_became_peered":"2024-04-26T01:47:27.116084+0000","last_unstale":"2024-04-26T01:48:22.746112+0000","last_undegraded":"2024-04-26T01:48:22.746112+0000","last_fullsized":"2024-04-26T01:48:22.746112+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:06:34.517478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":87,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:42.335667+0000","last_change":"2024-04-26T01:47:23.060657+0000","last_active":"2024-04-26T01:48:42.335667+0000","last_peered":"2024-04-26T01:48:42.335667+0000","last_clean":"2024-04-26T01:48:42.335667+0000","last_became_active":"2024-04-26T01:47:23.060294+0000","last_became_peered":"2024-04-26T01:47:23.060294+0000","last_unstale":"2024-04-26T01:48:42.335667+0000","last_undegraded":"2024-04-26T01:48:42.335667+0000","last_fullsized":"2024-04-26T01:48:42.335667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:33.979717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396644+0000","last_change":"2024-04-26T01:47:27.120587+0000","last_active":"2024-04-26T01:48:22.396644+0000","last_peered":"2024-04-26T01:48:22.396644+0000","last_clean":"2024-04-26T01:48:22.396644+0000","last_became_active":"2024-04-26T01:47:27.120236+0000","last_became_peered":"2024-04-26T01:47:27.120236+0000","last_unstale":"2024-04-26T01:48:22.396644+0000","last_undegraded":"2024-04-26T01:48:22.396644+0000","last_fullsized":"2024-04-26T01:48:22.396644+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:30:38.488741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405079+0000","last_change":"2024-04-26T01:47:23.069395+0000","last_active":"2024-04-26T01:48:22.405079+0000","last_peered":"2024-04-26T01:48:22.405079+0000","last_clean":"2024-04-26T01:48:22.405079+0000","last_became_active":"2024-04-26T01:47:23.069102+0000","last_became_peered":"2024-04-26T01:47:23.069102+0000","last_unstale":"2024-04-26T01:48:22.405079+0000","last_undegraded":"2024-04-26T01:48:22.405079+0000","last_fullsized":"2024-04-26T01:48:22.405079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:45.162604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745788+0000","last_change":"2024-04-26T01:47:27.123859+0000","last_active":"2024-04-26T01:48:22.745788+0000","last_peered":"2024-04-26T01:48:22.745788+0000","last_clean":"2024-04-26T01:48:22.745788+0000","last_became_active":"2024-04-26T01:47:27.123438+0000","last_became_peered":"2024-04-26T01:47:27.123438+0000","last_unstale":"2024-04-26T01:48:22.745788+0000","last_undegraded":"2024-04-26T01:48:22.745788+0000","last_fullsized":"2024-04-26T01:48:22.745788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:33:53.542809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.401049+0000","last_change":"2024-04-26T01:47:23.071544+0000","last_active":"2024-04-26T01:48:22.401049+0000","last_peered":"2024-04-26T01:48:22.401049+0000","last_clean":"2024-04-26T01:48:22.401049+0000","last_became_active":"2024-04-26T01:47:23.071191+0000","last_became_peered":"2024-04-26T01:47:23.071191+0000","last_unstale":"2024-04-26T01:48:22.401049+0000","last_undegraded":"2024-04-26T01:48:22.401049+0000","last_fullsized":"2024-04-26T01:48:22.401049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:58.889561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404118+0000","last_change":"2024-04-26T01:47:27.131887+0000","last_active":"2024-04-26T01:48:22.404118+0000","last_peered":"2024-04-26T01:48:22.404118+0000","last_clean":"2024-04-26T01:48:22.404118+0000","last_became_active":"2024-04-26T01:47:27.127973+0000","last_became_peered":"2024-04-26T01:47:27.127973+0000","last_unstale":"2024-04-26T01:48:22.404118+0000","last_undegraded":"2024-04-26T01:48:22.404118+0000","last_fullsized":"2024-04-26T01:48:22.404118+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:54:35.305779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745599+0000","last_change":"2024-04-26T01:47:23.061353+0000","last_active":"2024-04-26T01:48:22.745599+0000","last_peered":"2024-04-26T01:48:22.745599+0000","last_clean":"2024-04-26T01:48:22.745599+0000","last_became_active":"2024-04-26T01:47:23.061037+0000","last_became_peered":"2024-04-26T01:47:23.061037+0000","last_unstale":"2024-04-26T01:48:22.745599+0000","last_undegraded":"2024-04-26T01:48:22.745599+0000","last_fullsized":"2024-04-26T01:48:22.745599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:13:59.617969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746265+0000","last_change":"2024-04-26T01:47:27.122303+0000","last_active":"2024-04-26T01:48:22.746265+0000","last_peered":"2024-04-26T01:48:22.746265+0000","last_clean":"2024-04-26T01:48:22.746265+0000","last_became_active":"2024-04-26T01:47:27.121919+0000","last_became_peered":"2024-04-26T01:47:27.121919+0000","last_unstale":"2024-04-26T01:48:22.746265+0000","last_undegraded":"2024-04-26T01:48:22.746265+0000","last_fullsized":"2024-04-26T01:48:22.746265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:21.676663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":81,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:40.332063+0000","last_change":"2024-04-26T01:47:23.057961+0000","last_active":"2024-04-26T01:48:40.332063+0000","last_peered":"2024-04-26T01:48:40.332063+0000","last_clean":"2024-04-26T01:48:40.332063+0000","last_became_active":"2024-04-26T01:47:23.057529+0000","last_became_peered":"2024-04-26T01:47:23.057529+0000","last_unstale":"2024-04-26T01:48:40.332063+0000","last_undegraded":"2024-04-26T01:48:40.332063+0000","last_fullsized":"2024-04-26T01:48:40.332063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:14.420265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398667+0000","last_change":"2024-04-26T01:47:27.120083+0000","last_active":"2024-04-26T01:48:22.398667+0000","last_peered":"2024-04-26T01:48:22.398667+0000","last_clean":"2024-04-26T01:48:22.398667+0000","last_became_active":"2024-04-26T01:47:27.118410+0000","last_became_peered":"2024-04-26T01:47:27.118410+0000","last_unstale":"2024-04-26T01:48:22.398667+0000","last_undegraded":"2024-04-26T01:48:22.398667+0000","last_fullsized":"2024-04-26T01:48:22.398667+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:28.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400220+0000","last_change":"2024-04-26T01:47:23.059221+0000","last_active":"2024-04-26T01:48:22.400220+0000","last_peered":"2024-04-26T01:48:22.400220+0000","last_clean":"2024-04-26T01:48:22.400220+0000","last_became_active":"2024-04-26T01:47:23.058881+0000","last_became_peered":"2024-04-26T01:47:23.058881+0000","last_unstale":"2024-04-26T01:48:22.400220+0000","last_undegraded":"2024-04-26T01:48:22.400220+0000","last_fullsized":"2024-04-26T01:48:22.400220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:30.433696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.406173+0000","last_change":"2024-04-26T01:47:27.136763+0000","last_active":"2024-04-26T01:48:22.406173+0000","last_peered":"2024-04-26T01:48:22.406173+0000","last_clean":"2024-04-26T01:48:22.406173+0000","last_became_active":"2024-04-26T01:47:27.133642+0000","last_became_peered":"2024-04-26T01:47:27.133642+0000","last_unstale":"2024-04-26T01:48:22.406173+0000","last_undegraded":"2024-04-26T01:48:22.406173+0000","last_fullsized":"2024-04-26T01:48:22.406173+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:05.901139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746094+0000","last_change":"2024-04-26T01:47:23.108883+0000","last_active":"2024-04-26T01:48:22.746094+0000","last_peered":"2024-04-26T01:48:22.746094+0000","last_clean":"2024-04-26T01:48:22.746094+0000","last_became_active":"2024-04-26T01:47:23.108499+0000","last_became_peered":"2024-04-26T01:47:23.108499+0000","last_unstale":"2024-04-26T01:48:22.746094+0000","last_undegraded":"2024-04-26T01:48:22.746094+0000","last_fullsized":"2024-04-26T01:48:22.746094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:14:17.555416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.410651+0000","last_change":"2024-04-26T01:48:22.410651+0000","last_active":"2024-04-26T01:48:22.410651+0000","last_peered":"2024-04-26T01:48:22.410651+0000","last_clean":"2024-04-26T01:48:22.410651+0000","last_became_active":"2024-04-26T01:48:22.410359+0000","last_became_peered":"2024-04-26T01:48:22.410359+0000","last_unstale":"2024-04-26T01:48:22.410651+0000","last_undegraded":"2024-04-26T01:48:22.410651+0000","last_fullsized":"2024-04-26T01:48:22.410651+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:53:20.686915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'29","reported_seq":168,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:41.339480+0000","last_change":"2024-04-26T01:47:23.054893+0000","last_active":"2024-04-26T01:48:41.339480+0000","last_peered":"2024-04-26T01:48:41.339480+0000","last_clean":"2024-04-26T01:48:41.339480+0000","last_became_active":"2024-04-26T01:47:23.054558+0000","last_became_peered":"2024-04-26T01:47:23.054558+0000","last_unstale":"2024-04-26T01:48:41.339480+0000","last_undegraded":"2024-04-26T01:48:41.339480+0000","last_fullsized":"2024-04-26T01:48:41.339480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:59.356841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60347,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":113,"num_read_kb":115,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396887+0000","last_change":"2024-04-26T01:47:27.116934+0000","last_active":"2024-04-26T01:48:22.396887+0000","last_peered":"2024-04-26T01:48:22.396887+0000","last_clean":"2024-04-26T01:48:22.396887+0000","last_became_active":"2024-04-26T01:47:27.116527+0000","last_became_peered":"2024-04-26T01:47:27.116527+0000","last_unstale":"2024-04-26T01:48:22.396887+0000","last_undegraded":"2024-04-26T01:48:22.396887+0000","last_fullsized":"2024-04-26T01:48:22.396887+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:09:45.820533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399072+0000","last_change":"2024-04-26T01:47:23.059032+0000","last_active":"2024-04-26T01:48:22.399072+0000","last_peered":"2024-04-26T01:48:22.399072+0000","last_clean":"2024-04-26T01:48:22.399072+0000","last_became_active":"2024-04-26T01:47:23.057992+0000","last_became_peered":"2024-04-26T01:47:23.057992+0000","last_unstale":"2024-04-26T01:48:22.399072+0000","last_undegraded":"2024-04-26T01:48:22.399072+0000","last_fullsized":"2024-04-26T01:48:22.399072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:46:53.164925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.402515+0000","last_change":"2024-04-26T01:47:27.134574+0000","last_active":"2024-04-26T01:48:22.402515+0000","last_peered":"2024-04-26T01:48:22.402515+0000","last_clean":"2024-04-26T01:48:22.402515+0000","last_became_active":"2024-04-26T01:47:27.134184+0000","last_became_peered":"2024-04-26T01:47:27.134184+0000","last_unstale":"2024-04-26T01:48:22.402515+0000","last_undegraded":"2024-04-26T01:48:22.402515+0000","last_fullsized":"2024-04-26T01:48:22.402515+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:42:26.428014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745665+0000","last_change":"2024-04-26T01:47:23.062649+0000","last_active":"2024-04-26T01:48:22.745665+0000","last_peered":"2024-04-26T01:48:22.745665+0000","last_clean":"2024-04-26T01:48:22.745665+0000","last_became_active":"2024-04-26T01:47:23.062282+0000","last_became_peered":"2024-04-26T01:47:23.062282+0000","last_unstale":"2024-04-26T01:48:22.745665+0000","last_undegraded":"2024-04-26T01:48:22.745665+0000","last_fullsized":"2024-04-26T01:48:22.745665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:38.699105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400362+0000","last_change":"2024-04-26T01:47:27.124720+0000","last_active":"2024-04-26T01:48:22.400362+0000","last_peered":"2024-04-26T01:48:22.400362+0000","last_clean":"2024-04-26T01:48:22.400362+0000","last_became_active":"2024-04-26T01:47:27.124378+0000","last_became_peered":"2024-04-26T01:47:27.124378+0000","last_unstale":"2024-04-26T01:48:22.400362+0000","last_undegraded":"2024-04-26T01:48:22.400362+0000","last_fullsized":"2024-04-26T01:48:22.400362+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:45.462741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":87,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:42.359947+0000","last_change":"2024-04-26T01:47:23.056305+0000","last_active":"2024-04-26T01:48:42.359947+0000","last_peered":"2024-04-26T01:48:42.359947+0000","last_clean":"2024-04-26T01:48:42.359947+0000","last_became_active":"2024-04-26T01:47:23.055905+0000","last_became_peered":"2024-04-26T01:47:23.055905+0000","last_unstale":"2024-04-26T01:48:42.359947+0000","last_undegraded":"2024-04-26T01:48:42.359947+0000","last_fullsized":"2024-04-26T01:48:42.359947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:00:14.609673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400015+0000","last_change":"2024-04-26T01:47:27.119880+0000","last_active":"2024-04-26T01:48:22.400015+0000","last_peered":"2024-04-26T01:48:22.400015+0000","last_clean":"2024-04-26T01:48:22.400015+0000","last_became_active":"2024-04-26T01:47:27.119483+0000","last_became_peered":"2024-04-26T01:47:27.119483+0000","last_unstale":"2024-04-26T01:48:22.400015+0000","last_undegraded":"2024-04-26T01:48:22.400015+0000","last_fullsized":"2024-04-26T01:48:22.400015+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:15.523150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":66,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.758774+0000","last_change":"2024-04-26T01:48:22.758774+0000","last_active":"2024-04-26T01:48:22.758774+0000","last_peered":"2024-04-26T01:48:22.758774+0000","last_clean":"2024-04-26T01:48:22.758774+0000","last_became_active":"2024-04-26T01:48:22.747354+0000","last_became_peered":"2024-04-26T01:48:22.747354+0000","last_unstale":"2024-04-26T01:48:22.758774+0000","last_undegraded":"2024-04-26T01:48:22.758774+0000","last_fullsized":"2024-04-26T01:48:22.758774+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:23:15.886774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745649+0000","last_change":"2024-04-26T01:47:27.113508+0000","last_active":"2024-04-26T01:48:22.745649+0000","last_peered":"2024-04-26T01:48:22.745649+0000","last_clean":"2024-04-26T01:48:22.745649+0000","last_became_active":"2024-04-26T01:47:27.113066+0000","last_became_peered":"2024-04-26T01:47:27.113066+0000","last_unstale":"2024-04-26T01:48:22.745649+0000","last_undegraded":"2024-04-26T01:48:22.745649+0000","last_fullsized":"2024-04-26T01:48:22.745649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:37.525486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.397062+0000","last_change":"2024-04-26T01:47:23.059644+0000","last_active":"2024-04-26T01:48:22.397062+0000","last_peered":"2024-04-26T01:48:22.397062+0000","last_clean":"2024-04-26T01:48:22.397062+0000","last_became_active":"2024-04-26T01:47:23.058966+0000","last_became_peered":"2024-04-26T01:47:23.058966+0000","last_unstale":"2024-04-26T01:48:22.397062+0000","last_undegraded":"2024-04-26T01:48:22.397062+0000","last_fullsized":"2024-04-26T01:48:22.397062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:57:36.475935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.749354+0000","last_change":"2024-04-26T01:48:22.749354+0000","last_active":"2024-04-26T01:48:22.749354+0000","last_peered":"2024-04-26T01:48:22.749354+0000","last_clean":"2024-04-26T01:48:22.749354+0000","last_became_active":"2024-04-26T01:48:22.748978+0000","last_became_peered":"2024-04-26T01:48:22.748978+0000","last_unstale":"2024-04-26T01:48:22.749354+0000","last_undegraded":"2024-04-26T01:48:22.749354+0000","last_fullsized":"2024-04-26T01:48:22.749354+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:51:16.087091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746667+0000","last_change":"2024-04-26T01:47:23.110360+0000","last_active":"2024-04-26T01:48:22.746667+0000","last_peered":"2024-04-26T01:48:22.746667+0000","last_clean":"2024-04-26T01:48:22.746667+0000","last_became_active":"2024-04-26T01:47:23.109744+0000","last_became_peered":"2024-04-26T01:47:23.109744+0000","last_unstale":"2024-04-26T01:48:22.746667+0000","last_undegraded":"2024-04-26T01:48:22.746667+0000","last_fullsized":"2024-04-26T01:48:22.746667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:40:13.658396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405212+0000","last_change":"2024-04-26T01:47:27.125399+0000","last_active":"2024-04-26T01:48:22.405212+0000","last_peered":"2024-04-26T01:48:22.405212+0000","last_clean":"2024-04-26T01:48:22.405212+0000","last_became_active":"2024-04-26T01:47:27.124871+0000","last_became_peered":"2024-04-26T01:47:27.124871+0000","last_unstale":"2024-04-26T01:48:22.405212+0000","last_undegraded":"2024-04-26T01:48:22.405212+0000","last_fullsized":"2024-04-26T01:48:22.405212+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:28:48.285206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400178+0000","last_change":"2024-04-26T01:47:23.067862+0000","last_active":"2024-04-26T01:48:22.400178+0000","last_peered":"2024-04-26T01:48:22.400178+0000","last_clean":"2024-04-26T01:48:22.400178+0000","last_became_active":"2024-04-26T01:47:23.067526+0000","last_became_peered":"2024-04-26T01:47:23.067526+0000","last_unstale":"2024-04-26T01:48:22.400178+0000","last_undegraded":"2024-04-26T01:48:22.400178+0000","last_fullsized":"2024-04-26T01:48:22.400178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:22.668405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.748548+0000","last_change":"2024-04-26T01:48:22.748548+0000","last_active":"2024-04-26T01:48:22.748548+0000","last_peered":"2024-04-26T01:48:22.748548+0000","last_clean":"2024-04-26T01:48:22.748548+0000","last_became_active":"2024-04-26T01:48:22.748182+0000","last_became_peered":"2024-04-26T01:48:22.748182+0000","last_unstale":"2024-04-26T01:48:22.748548+0000","last_undegraded":"2024-04-26T01:48:22.748548+0000","last_fullsized":"2024-04-26T01:48:22.748548+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:20:27.596827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745783+0000","last_change":"2024-04-26T01:47:27.118307+0000","last_active":"2024-04-26T01:48:22.745783+0000","last_peered":"2024-04-26T01:48:22.745783+0000","last_clean":"2024-04-26T01:48:22.745783+0000","last_became_active":"2024-04-26T01:47:27.117986+0000","last_became_peered":"2024-04-26T01:47:27.117986+0000","last_unstale":"2024-04-26T01:48:22.745783+0000","last_undegraded":"2024-04-26T01:48:22.745783+0000","last_fullsized":"2024-04-26T01:48:22.745783+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:40.667143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":81,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:39.353277+0000","last_change":"2024-04-26T01:47:23.109973+0000","last_active":"2024-04-26T01:48:39.353277+0000","last_peered":"2024-04-26T01:48:39.353277+0000","last_clean":"2024-04-26T01:48:39.353277+0000","last_became_active":"2024-04-26T01:47:23.109446+0000","last_became_peered":"2024-04-26T01:47:23.109446+0000","last_unstale":"2024-04-26T01:48:39.353277+0000","last_undegraded":"2024-04-26T01:48:39.353277+0000","last_fullsized":"2024-04-26T01:48:39.353277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:49:00.684164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.408237+0000","last_change":"2024-04-26T01:47:27.131502+0000","last_active":"2024-04-26T01:48:22.408237+0000","last_peered":"2024-04-26T01:48:22.408237+0000","last_clean":"2024-04-26T01:48:22.408237+0000","last_became_active":"2024-04-26T01:47:27.127697+0000","last_became_peered":"2024-04-26T01:47:27.127697+0000","last_unstale":"2024-04-26T01:48:22.408237+0000","last_undegraded":"2024-04-26T01:48:22.408237+0000","last_fullsized":"2024-04-26T01:48:22.408237+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:39:15.458189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746705+0000","last_change":"2024-04-26T01:47:23.066596+0000","last_active":"2024-04-26T01:48:22.746705+0000","last_peered":"2024-04-26T01:48:22.746705+0000","last_clean":"2024-04-26T01:48:22.746705+0000","last_became_active":"2024-04-26T01:47:23.066216+0000","last_became_peered":"2024-04-26T01:47:23.066216+0000","last_unstale":"2024-04-26T01:48:22.746705+0000","last_undegraded":"2024-04-26T01:48:22.746705+0000","last_fullsized":"2024-04-26T01:48:22.746705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:08:31.544831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400162+0000","last_change":"2024-04-26T01:45:54.834501+0000","last_active":"2024-04-26T01:48:22.400162+0000","last_peered":"2024-04-26T01:48:22.400162+0000","last_clean":"2024-04-26T01:48:22.400162+0000","last_became_active":"2024-04-26T01:45:54.833868+0000","last_became_peered":"2024-04-26T01:45:54.833868+0000","last_unstale":"2024-04-26T01:48:22.400162+0000","last_undegraded":"2024-04-26T01:48:22.400162+0000","last_fullsized":"2024-04-26T01:48:22.400162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:42:05.228956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:42:05.228956+0000","last_clean_scrub_stamp":"2024-04-26T01:42:05.228956+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:06:58.383315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400398+0000","last_change":"2024-04-26T01:47:27.126475+0000","last_active":"2024-04-26T01:48:22.400398+0000","last_peered":"2024-04-26T01:48:22.400398+0000","last_clean":"2024-04-26T01:48:22.400398+0000","last_became_active":"2024-04-26T01:47:27.123760+0000","last_became_peered":"2024-04-26T01:47:27.123760+0000","last_unstale":"2024-04-26T01:48:22.400398+0000","last_undegraded":"2024-04-26T01:48:22.400398+0000","last_fullsized":"2024-04-26T01:48:22.400398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:38:27.473909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400221+0000","last_change":"2024-04-26T01:47:23.059810+0000","last_active":"2024-04-26T01:48:22.400221+0000","last_peered":"2024-04-26T01:48:22.400221+0000","last_clean":"2024-04-26T01:48:22.400221+0000","last_became_active":"2024-04-26T01:47:23.059504+0000","last_became_peered":"2024-04-26T01:47:23.059504+0000","last_unstale":"2024-04-26T01:48:22.400221+0000","last_undegraded":"2024-04-26T01:48:22.400221+0000","last_fullsized":"2024-04-26T01:48:22.400221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:50:51.220951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396987+0000","last_change":"2024-04-26T01:47:23.109460+0000","last_active":"2024-04-26T01:48:22.396987+0000","last_peered":"2024-04-26T01:48:22.396987+0000","last_clean":"2024-04-26T01:48:22.396987+0000","last_became_active":"2024-04-26T01:47:23.109090+0000","last_became_peered":"2024-04-26T01:47:23.109090+0000","last_unstale":"2024-04-26T01:48:22.396987+0000","last_undegraded":"2024-04-26T01:48:22.396987+0000","last_fullsized":"2024-04-26T01:48:22.396987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:56.821312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.407355+0000","last_change":"2024-04-26T01:47:27.129537+0000","last_active":"2024-04-26T01:48:22.407355+0000","last_peered":"2024-04-26T01:48:22.407355+0000","last_clean":"2024-04-26T01:48:22.407355+0000","last_became_active":"2024-04-26T01:47:27.128124+0000","last_became_peered":"2024-04-26T01:47:27.128124+0000","last_unstale":"2024-04-26T01:48:22.407355+0000","last_undegraded":"2024-04-26T01:48:22.407355+0000","last_fullsized":"2024-04-26T01:48:22.407355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:14:26.262197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'11","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.747472+0000","last_change":"2024-04-26T01:47:23.064511+0000","last_active":"2024-04-26T01:48:22.747472+0000","last_peered":"2024-04-26T01:48:22.747472+0000","last_clean":"2024-04-26T01:48:22.747472+0000","last_became_active":"2024-04-26T01:47:23.064080+0000","last_became_peered":"2024-04-26T01:47:23.064080+0000","last_unstale":"2024-04-26T01:48:22.747472+0000","last_undegraded":"2024-04-26T01:48:22.747472+0000","last_fullsized":"2024-04-26T01:48:22.747472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:37.562644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404336+0000","last_change":"2024-04-26T01:47:27.124057+0000","last_active":"2024-04-26T01:48:22.404336+0000","last_peered":"2024-04-26T01:48:22.404336+0000","last_clean":"2024-04-26T01:48:22.404336+0000","last_became_active":"2024-04-26T01:47:27.123636+0000","last_became_peered":"2024-04-26T01:47:27.123636+0000","last_unstale":"2024-04-26T01:48:22.404336+0000","last_undegraded":"2024-04-26T01:48:22.404336+0000","last_fullsized":"2024-04-26T01:48:22.404336+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:56:55.691697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746155+0000","last_change":"2024-04-26T01:47:23.136571+0000","last_active":"2024-04-26T01:48:22.746155+0000","last_peered":"2024-04-26T01:48:22.746155+0000","last_clean":"2024-04-26T01:48:22.746155+0000","last_became_active":"2024-04-26T01:47:23.136205+0000","last_became_peered":"2024-04-26T01:47:23.136205+0000","last_unstale":"2024-04-26T01:48:22.746155+0000","last_undegraded":"2024-04-26T01:48:22.746155+0000","last_fullsized":"2024-04-26T01:48:22.746155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:24.418797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.747570+0000","last_change":"2024-04-26T01:48:22.747570+0000","last_active":"2024-04-26T01:48:22.747570+0000","last_peered":"2024-04-26T01:48:22.747570+0000","last_clean":"2024-04-26T01:48:22.747570+0000","last_became_active":"2024-04-26T01:48:22.747147+0000","last_became_peered":"2024-04-26T01:48:22.747147+0000","last_unstale":"2024-04-26T01:48:22.747570+0000","last_undegraded":"2024-04-26T01:48:22.747570+0000","last_fullsized":"2024-04-26T01:48:22.747570+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:37.034388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.414610+0000","last_change":"2024-04-26T01:48:22.414610+0000","last_active":"2024-04-26T01:48:22.414610+0000","last_peered":"2024-04-26T01:48:22.414610+0000","last_clean":"2024-04-26T01:48:22.414610+0000","last_became_active":"2024-04-26T01:48:22.410275+0000","last_became_peered":"2024-04-26T01:48:22.410275+0000","last_unstale":"2024-04-26T01:48:22.414610+0000","last_undegraded":"2024-04-26T01:48:22.414610+0000","last_fullsized":"2024-04-26T01:48:22.414610+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:20:48.131988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396626+0000","last_change":"2024-04-26T01:47:27.115498+0000","last_active":"2024-04-26T01:48:22.396626+0000","last_peered":"2024-04-26T01:48:22.396626+0000","last_clean":"2024-04-26T01:48:22.396626+0000","last_became_active":"2024-04-26T01:47:27.115134+0000","last_became_peered":"2024-04-26T01:47:27.115134+0000","last_unstale":"2024-04-26T01:48:22.396626+0000","last_undegraded":"2024-04-26T01:48:22.396626+0000","last_fullsized":"2024-04-26T01:48:22.396626+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:20.795224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405265+0000","last_change":"2024-04-26T01:47:23.065366+0000","last_active":"2024-04-26T01:48:22.405265+0000","last_peered":"2024-04-26T01:48:22.405265+0000","last_clean":"2024-04-26T01:48:22.405265+0000","last_became_active":"2024-04-26T01:47:23.063216+0000","last_became_peered":"2024-04-26T01:47:23.063216+0000","last_unstale":"2024-04-26T01:48:22.405265+0000","last_undegraded":"2024-04-26T01:48:22.405265+0000","last_fullsized":"2024-04-26T01:48:22.405265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:06:05.261907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398405+0000","last_change":"2024-04-26T01:47:27.127938+0000","last_active":"2024-04-26T01:48:22.398405+0000","last_peered":"2024-04-26T01:48:22.398405+0000","last_clean":"2024-04-26T01:48:22.398405+0000","last_became_active":"2024-04-26T01:47:27.127553+0000","last_became_peered":"2024-04-26T01:47:27.127553+0000","last_unstale":"2024-04-26T01:48:22.398405+0000","last_undegraded":"2024-04-26T01:48:22.398405+0000","last_fullsized":"2024-04-26T01:48:22.398405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:17:51.115349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398731+0000","last_change":"2024-04-26T01:47:23.060475+0000","last_active":"2024-04-26T01:48:22.398731+0000","last_peered":"2024-04-26T01:48:22.398731+0000","last_clean":"2024-04-26T01:48:22.398731+0000","last_became_active":"2024-04-26T01:47:23.060098+0000","last_became_peered":"2024-04-26T01:47:23.060098+0000","last_unstale":"2024-04-26T01:48:22.398731+0000","last_undegraded":"2024-04-26T01:48:22.398731+0000","last_fullsized":"2024-04-26T01:48:22.398731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:49.009455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.407796+0000","last_change":"2024-04-26T01:47:27.137491+0000","last_active":"2024-04-26T01:48:22.407796+0000","last_peered":"2024-04-26T01:48:22.407796+0000","last_clean":"2024-04-26T01:48:22.407796+0000","last_became_active":"2024-04-26T01:47:27.137059+0000","last_became_peered":"2024-04-26T01:47:27.137059+0000","last_unstale":"2024-04-26T01:48:22.407796+0000","last_undegraded":"2024-04-26T01:48:22.407796+0000","last_fullsized":"2024-04-26T01:48:22.407796+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:20:25.004074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405524+0000","last_change":"2024-04-26T01:48:22.405071+0000","last_active":"2024-04-26T01:48:22.405524+0000","last_peered":"2024-04-26T01:48:22.405524+0000","last_clean":"2024-04-26T01:48:22.405524+0000","last_became_active":"2024-04-26T01:48:22.401282+0000","last_became_peered":"2024-04-26T01:48:22.401282+0000","last_unstale":"2024-04-26T01:48:22.405524+0000","last_undegraded":"2024-04-26T01:48:22.405524+0000","last_fullsized":"2024-04-26T01:48:22.405524+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:24:53.628014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400294+0000","last_change":"2024-04-26T01:47:27.128253+0000","last_active":"2024-04-26T01:48:22.400294+0000","last_peered":"2024-04-26T01:48:22.400294+0000","last_clean":"2024-04-26T01:48:22.400294+0000","last_became_active":"2024-04-26T01:47:27.127869+0000","last_became_peered":"2024-04-26T01:47:27.127869+0000","last_unstale":"2024-04-26T01:48:22.400294+0000","last_undegraded":"2024-04-26T01:48:22.400294+0000","last_fullsized":"2024-04-26T01:48:22.400294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:11:23.476938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399412+0000","last_change":"2024-04-26T01:47:23.136862+0000","last_active":"2024-04-26T01:48:22.399412+0000","last_peered":"2024-04-26T01:48:22.399412+0000","last_clean":"2024-04-26T01:48:22.399412+0000","last_became_active":"2024-04-26T01:47:23.136380+0000","last_became_peered":"2024-04-26T01:47:23.136380+0000","last_unstale":"2024-04-26T01:48:22.399412+0000","last_undegraded":"2024-04-26T01:48:22.399412+0000","last_fullsized":"2024-04-26T01:48:22.399412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:39:18.103559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400366+0000","last_change":"2024-04-26T01:47:27.113363+0000","last_active":"2024-04-26T01:48:22.400366+0000","last_peered":"2024-04-26T01:48:22.400366+0000","last_clean":"2024-04-26T01:48:22.400366+0000","last_became_active":"2024-04-26T01:47:27.113040+0000","last_became_peered":"2024-04-26T01:47:27.113040+0000","last_unstale":"2024-04-26T01:48:22.400366+0000","last_undegraded":"2024-04-26T01:48:22.400366+0000","last_fullsized":"2024-04-26T01:48:22.400366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:42:16.428697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745984+0000","last_change":"2024-04-26T01:47:23.067257+0000","last_active":"2024-04-26T01:48:22.745984+0000","last_peered":"2024-04-26T01:48:22.745984+0000","last_clean":"2024-04-26T01:48:22.745984+0000","last_became_active":"2024-04-26T01:47:23.066858+0000","last_became_peered":"2024-04-26T01:47:23.066858+0000","last_unstale":"2024-04-26T01:48:22.745984+0000","last_undegraded":"2024-04-26T01:48:22.745984+0000","last_fullsized":"2024-04-26T01:48:22.745984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:22:07.848074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'1","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400154+0000","last_change":"2024-04-26T01:47:27.125885+0000","last_active":"2024-04-26T01:48:22.400154+0000","last_peered":"2024-04-26T01:48:22.400154+0000","last_clean":"2024-04-26T01:48:22.400154+0000","last_became_active":"2024-04-26T01:47:27.125524+0000","last_became_peered":"2024-04-26T01:47:27.125524+0000","last_unstale":"2024-04-26T01:48:22.400154+0000","last_undegraded":"2024-04-26T01:48:22.400154+0000","last_fullsized":"2024-04-26T01:48:22.400154+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:39.282023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.407608+0000","last_change":"2024-04-26T01:47:23.071051+0000","last_active":"2024-04-26T01:48:22.407608+0000","last_peered":"2024-04-26T01:48:22.407608+0000","last_clean":"2024-04-26T01:48:22.407608+0000","last_became_active":"2024-04-26T01:47:23.069524+0000","last_became_peered":"2024-04-26T01:47:23.069524+0000","last_unstale":"2024-04-26T01:48:22.407608+0000","last_undegraded":"2024-04-26T01:48:22.407608+0000","last_fullsized":"2024-04-26T01:48:22.407608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:24:57.643100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400370+0000","last_change":"2024-04-26T01:47:27.114325+0000","last_active":"2024-04-26T01:48:22.400370+0000","last_peered":"2024-04-26T01:48:22.400370+0000","last_clean":"2024-04-26T01:48:22.400370+0000","last_became_active":"2024-04-26T01:47:27.113952+0000","last_became_peered":"2024-04-26T01:47:27.113952+0000","last_unstale":"2024-04-26T01:48:22.400370+0000","last_undegraded":"2024-04-26T01:48:22.400370+0000","last_fullsized":"2024-04-26T01:48:22.400370+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:39:19.167175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398411+0000","last_change":"2024-04-26T01:47:27.118709+0000","last_active":"2024-04-26T01:48:22.398411+0000","last_peered":"2024-04-26T01:48:22.398411+0000","last_clean":"2024-04-26T01:48:22.398411+0000","last_became_active":"2024-04-26T01:47:27.118341+0000","last_became_peered":"2024-04-26T01:47:27.118341+0000","last_unstale":"2024-04-26T01:48:22.398411+0000","last_undegraded":"2024-04-26T01:48:22.398411+0000","last_fullsized":"2024-04-26T01:48:22.398411+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:13.736180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399082+0000","last_change":"2024-04-26T01:47:27.119239+0000","last_active":"2024-04-26T01:48:22.399082+0000","last_peered":"2024-04-26T01:48:22.399082+0000","last_clean":"2024-04-26T01:48:22.399082+0000","last_became_active":"2024-04-26T01:47:27.118877+0000","last_became_peered":"2024-04-26T01:47:27.118877+0000","last_unstale":"2024-04-26T01:48:22.399082+0000","last_undegraded":"2024-04-26T01:48:22.399082+0000","last_fullsized":"2024-04-26T01:48:22.399082+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:04:10.033517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398635+0000","last_change":"2024-04-26T01:47:27.142848+0000","last_active":"2024-04-26T01:48:22.398635+0000","last_peered":"2024-04-26T01:48:22.398635+0000","last_clean":"2024-04-26T01:48:22.398635+0000","last_became_active":"2024-04-26T01:47:27.142580+0000","last_became_peered":"2024-04-26T01:47:27.142580+0000","last_unstale":"2024-04-26T01:48:22.398635+0000","last_undegraded":"2024-04-26T01:48:22.398635+0000","last_fullsized":"2024-04-26T01:48:22.398635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:15:46.923922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404760+0000","last_change":"2024-04-26T01:47:27.134817+0000","last_active":"2024-04-26T01:48:22.404760+0000","last_peered":"2024-04-26T01:48:22.404760+0000","last_clean":"2024-04-26T01:48:22.404760+0000","last_became_active":"2024-04-26T01:47:27.134024+0000","last_became_peered":"2024-04-26T01:47:27.134024+0000","last_unstale":"2024-04-26T01:48:22.404760+0000","last_undegraded":"2024-04-26T01:48:22.404760+0000","last_fullsized":"2024-04-26T01:48:22.404760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:56.746427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405975+0000","last_change":"2024-04-26T01:47:27.143137+0000","last_active":"2024-04-26T01:48:22.405975+0000","last_peered":"2024-04-26T01:48:22.405975+0000","last_clean":"2024-04-26T01:48:22.405975+0000","last_became_active":"2024-04-26T01:47:27.142625+0000","last_became_peered":"2024-04-26T01:47:27.142625+0000","last_unstale":"2024-04-26T01:48:22.405975+0000","last_undegraded":"2024-04-26T01:48:22.405975+0000","last_fullsized":"2024-04-26T01:48:22.405975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:42:50.468382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.749582+0000","last_change":"2024-04-26T01:48:22.749582+0000","last_active":"2024-04-26T01:48:22.749582+0000","last_peered":"2024-04-26T01:48:22.749582+0000","last_clean":"2024-04-26T01:48:22.749582+0000","last_became_active":"2024-04-26T01:48:22.749191+0000","last_became_peered":"2024-04-26T01:48:22.749191+0000","last_unstale":"2024-04-26T01:48:22.749582+0000","last_undegraded":"2024-04-26T01:48:22.749582+0000","last_fullsized":"2024-04-26T01:48:22.749582+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:14.614842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399646+0000","last_change":"2024-04-26T01:47:27.123964+0000","last_active":"2024-04-26T01:48:22.399646+0000","last_peered":"2024-04-26T01:48:22.399646+0000","last_clean":"2024-04-26T01:48:22.399646+0000","last_became_active":"2024-04-26T01:47:27.123552+0000","last_became_peered":"2024-04-26T01:47:27.123552+0000","last_unstale":"2024-04-26T01:48:22.399646+0000","last_undegraded":"2024-04-26T01:48:22.399646+0000","last_fullsized":"2024-04-26T01:48:22.399646+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:33:12.566105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.412621+0000","last_change":"2024-04-26T01:48:22.412621+0000","last_active":"2024-04-26T01:48:22.412621+0000","last_peered":"2024-04-26T01:48:22.412621+0000","last_clean":"2024-04-26T01:48:22.412621+0000","last_became_active":"2024-04-26T01:48:22.411430+0000","last_became_peered":"2024-04-26T01:48:22.411430+0000","last_unstale":"2024-04-26T01:48:22.412621+0000","last_undegraded":"2024-04-26T01:48:22.412621+0000","last_fullsized":"2024-04-26T01:48:22.412621+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:34.152338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745667+0000","last_change":"2024-04-26T01:47:27.131437+0000","last_active":"2024-04-26T01:48:22.745667+0000","last_peered":"2024-04-26T01:48:22.745667+0000","last_clean":"2024-04-26T01:48:22.745667+0000","last_became_active":"2024-04-26T01:47:27.131138+0000","last_became_peered":"2024-04-26T01:47:27.131138+0000","last_unstale":"2024-04-26T01:48:22.745667+0000","last_undegraded":"2024-04-26T01:48:22.745667+0000","last_fullsized":"2024-04-26T01:48:22.745667+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:55:39.132972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.404958+0000","last_change":"2024-04-26T01:47:27.120928+0000","last_active":"2024-04-26T01:48:22.404958+0000","last_peered":"2024-04-26T01:48:22.404958+0000","last_clean":"2024-04-26T01:48:22.404958+0000","last_became_active":"2024-04-26T01:47:27.120012+0000","last_became_peered":"2024-04-26T01:47:27.120012+0000","last_unstale":"2024-04-26T01:48:22.404958+0000","last_undegraded":"2024-04-26T01:48:22.404958+0000","last_fullsized":"2024-04-26T01:48:22.404958+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:19:44.209876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398493+0000","last_change":"2024-04-26T01:47:27.138766+0000","last_active":"2024-04-26T01:48:22.398493+0000","last_peered":"2024-04-26T01:48:22.398493+0000","last_clean":"2024-04-26T01:48:22.398493+0000","last_became_active":"2024-04-26T01:47:27.138413+0000","last_became_peered":"2024-04-26T01:47:27.138413+0000","last_unstale":"2024-04-26T01:48:22.398493+0000","last_undegraded":"2024-04-26T01:48:22.398493+0000","last_fullsized":"2024-04-26T01:48:22.398493+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:51:13.117307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.406315+0000","last_change":"2024-04-26T01:47:27.141338+0000","last_active":"2024-04-26T01:48:22.406315+0000","last_peered":"2024-04-26T01:48:22.406315+0000","last_clean":"2024-04-26T01:48:22.406315+0000","last_became_active":"2024-04-26T01:47:27.140825+0000","last_became_peered":"2024-04-26T01:47:27.140825+0000","last_unstale":"2024-04-26T01:48:22.406315+0000","last_undegraded":"2024-04-26T01:48:22.406315+0000","last_fullsized":"2024-04-26T01:48:22.406315+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:12:08.680876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398625+0000","last_change":"2024-04-26T01:47:27.114919+0000","last_active":"2024-04-26T01:48:22.398625+0000","last_peered":"2024-04-26T01:48:22.398625+0000","last_clean":"2024-04-26T01:48:22.398625+0000","last_became_active":"2024-04-26T01:47:27.114528+0000","last_became_peered":"2024-04-26T01:47:27.114528+0000","last_unstale":"2024-04-26T01:48:22.398625+0000","last_undegraded":"2024-04-26T01:48:22.398625+0000","last_fullsized":"2024-04-26T01:48:22.398625+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:15:25.620566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64348,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":341,"num_read_kb":235,"num_write":163,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318302,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":490347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579939,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28688,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716464,"statfs":{"total":95995035648,"available":95965659136,"internally_reserved":0,"allocated":1708032,"data_stored":1142566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743466,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":490583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005105,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":490503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299444,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1622016,"data_stored":1082433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462970,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":489972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":550112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":491243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":551350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":490283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920837,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":490912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724610,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32776,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712376,"statfs":{"total":95995035648,"available":95961473024,"internally_reserved":0,"allocated":1634304,"data_stored":1081726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:48:46.670 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T01:48:46.671 INFO:tasks.cephadm:Setting up client nodes... 2024-04-26T01:48:46.671 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-26T01:48:47.193 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:47.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:46 smithi097 ceph-mon[32853]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:48:47.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:46 smithi173 ceph-mon[32553]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:48:47.340 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:46 smithi001 ceph-mon[27822]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:48:48.177 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:47 smithi001 ceph-mon[27822]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:48:48.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:47 smithi097 ceph-mon[32853]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:48:48.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:47 smithi173 ceph-mon[32553]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:48:49.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.422+0000 7fe102600700 1 -- 172.21.15.1:0/2028141784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0fc102660 msgr2=0x7fe0fc102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:49.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.422+0000 7fe102600700 1 --2- 172.21.15.1:0/2028141784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0fc102660 0x7fe0fc102ae0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400b0a0 tx=0x7fe0f400b3b0 comp rx=0 tx=0).stop 2024-04-26T01:48:49.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.422+0000 7fe102600700 1 -- 172.21.15.1:0/2028141784 shutdown_connections 2024-04-26T01:48:49.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.422+0000 7fe102600700 1 --2- 172.21.15.1:0/2028141784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0fc102660 0x7fe0fc102ae0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:49.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.422+0000 7fe102600700 1 --2- 172.21.15.1:0/2028141784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0fc101340 0x7fe0fc101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:49.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.422+0000 7fe102600700 1 --2- 172.21.15.1:0/2028141784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe0fc101ca0 0x7fe0fc102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:49.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.422+0000 7fe102600700 1 -- 172.21.15.1:0/2028141784 >> 172.21.15.1:0/2028141784 conn(0x7fe0fc0fcf60 msgr2=0x7fe0fc0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:49.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.422+0000 7fe102600700 1 -- 172.21.15.1:0/2028141784 shutdown_connections 2024-04-26T01:48:49.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.422+0000 7fe102600700 1 -- 172.21.15.1:0/2028141784 wait complete. 2024-04-26T01:48:49.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.423+0000 7fe102600700 1 Processor -- start 2024-04-26T01:48:49.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.423+0000 7fe102600700 1 -- start start 2024-04-26T01:48:49.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.424+0000 7fe102600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0fc101340 0x7fe0fc19ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:49.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.424+0000 7fe102600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe0fc101ca0 0x7fe0fc19b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:49.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.424+0000 7fe102600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0fc102660 0x7fe0fc1a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:49.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.424+0000 7fe102600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc110770 con 0x7fe0fc101ca0 2024-04-26T01:48:49.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.424+0000 7fe102600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1105f0 con 0x7fe0fc101340 2024-04-26T01:48:49.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.424+0000 7fe102600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1108f0 con 0x7fe0fc102660 2024-04-26T01:48:49.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.424+0000 7fe0fb400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0fc101340 0x7fe0fc19ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:49.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.424+0000 7fe0fb400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0fc101340 0x7fe0fc19ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:52698/0 (socket says 172.21.15.1:52698) 2024-04-26T01:48:49.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.424+0000 7fe0fb400700 1 -- 172.21.15.1:0/3533295320 learned_addr learned my addr 172.21.15.1:0/3533295320 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:48:49.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.424+0000 7fe0faa00700 1 --2- 172.21.15.1:0/3533295320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe0fc101ca0 0x7fe0fc19b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:49.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.424+0000 7fe0fbe00700 1 --2- 172.21.15.1:0/3533295320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0fc102660 0x7fe0fc1a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:49.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.425+0000 7fe0fb400700 1 -- 172.21.15.1:0/3533295320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0fc102660 msgr2=0x7fe0fc1a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:49.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.425+0000 7fe0fb400700 1 --2- 172.21.15.1:0/3533295320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0fc102660 0x7fe0fc1a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:49.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.425+0000 7fe0fb400700 1 -- 172.21.15.1:0/3533295320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe0fc101ca0 msgr2=0x7fe0fc19b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:49.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.425+0000 7fe0fb400700 1 --2- 172.21.15.1:0/3533295320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe0fc101ca0 0x7fe0fc19b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:49.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.425+0000 7fe0fb400700 1 -- 172.21.15.1:0/3533295320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0fc1a2930 con 0x7fe0fc101340 2024-04-26T01:48:49.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.425+0000 7fe0fbe00700 1 --2- 172.21.15.1:0/3533295320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0fc102660 0x7fe0fc1a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:48:49.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.425+0000 7fe0faa00700 1 --2- 172.21.15.1:0/3533295320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe0fc101ca0 0x7fe0fc19b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:48:49.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.425+0000 7fe0fb400700 1 --2- 172.21.15.1:0/3533295320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0fc101340 0x7fe0fc19ace0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00e9a0 tx=0x7fe0ec00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:49.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.425+0000 7fe0f3e00700 1 -- 172.21.15.1:0/3533295320 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec00ca10 con 0x7fe0fc101340 2024-04-26T01:48:49.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.425+0000 7fe102600700 1 -- 172.21.15.1:0/3533295320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0fc1a2c20 con 0x7fe0fc101340 2024-04-26T01:48:49.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.425+0000 7fe0f3e00700 1 -- 172.21.15.1:0/3533295320 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0ec00cbb0 con 0x7fe0fc101340 2024-04-26T01:48:49.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.425+0000 7fe0f3e00700 1 -- 172.21.15.1:0/3533295320 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec018680 con 0x7fe0fc101340 2024-04-26T01:48:49.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.426+0000 7fe102600700 1 -- 172.21.15.1:0/3533295320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0fc1a3160 con 0x7fe0fc101340 2024-04-26T01:48:49.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.427+0000 7fe102600700 1 -- 172.21.15.1:0/3533295320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0fc050f80 con 0x7fe0fc101340 2024-04-26T01:48:49.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.429+0000 7fe0f3e00700 1 -- 172.21.15.1:0/3533295320 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fe0ec010430 con 0x7fe0fc101340 2024-04-26T01:48:49.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.430+0000 7fe0f3e00700 1 --2- 172.21.15.1:0/3533295320 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe0e00a3d80 0x7fe0e00a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:49.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.430+0000 7fe0f3e00700 1 -- 172.21.15.1:0/3533295320 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fe0ec0d0770 con 0x7fe0fc101340 2024-04-26T01:48:49.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.430+0000 7fe0faa00700 1 --2- 172.21.15.1:0/3533295320 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe0e00a3d80 0x7fe0e00a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:49.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.431+0000 7fe0faa00700 1 --2- 172.21.15.1:0/3533295320 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe0e00a3d80 0x7fe0e00a6240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009680 tx=0x7fe0e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:49.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.435+0000 7fe0f3e00700 1 -- 172.21.15.1:0/3533295320 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe0ec09cf70 con 0x7fe0fc101340 2024-04-26T01:48:49.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.771+0000 7fe102600700 1 -- 172.21.15.1:0/3533295320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fe0fc063650 con 0x7fe0fc101340 2024-04-26T01:48:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.778+0000 7fe0f3e00700 1 -- 172.21.15.1:0/3533295320 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fe0ec09c870 con 0x7fe0fc101340 2024-04-26T01:48:49.777 INFO:teuthology.orchestra.run.smithi001.stdout:[client.0] 2024-04-26T01:48:49.777 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQABCCtm00EcLhAA3ZgnR/G9DRGVviHOlx4Vaw== 2024-04-26T01:48:49.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.782+0000 7fe102600700 1 -- 172.21.15.1:0/3533295320 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe0e00a3d80 msgr2=0x7fe0e00a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:49.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.782+0000 7fe102600700 1 --2- 172.21.15.1:0/3533295320 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe0e00a3d80 0x7fe0e00a6240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009680 tx=0x7fe0e8009260 comp rx=0 tx=0).stop 2024-04-26T01:48:49.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.782+0000 7fe102600700 1 -- 172.21.15.1:0/3533295320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0fc101340 msgr2=0x7fe0fc19ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:49.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.782+0000 7fe102600700 1 --2- 172.21.15.1:0/3533295320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0fc101340 0x7fe0fc19ace0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00e9a0 tx=0x7fe0ec00ecb0 comp rx=0 tx=0).stop 2024-04-26T01:48:49.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.782+0000 7fe102600700 1 -- 172.21.15.1:0/3533295320 shutdown_connections 2024-04-26T01:48:49.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.782+0000 7fe102600700 1 --2- 172.21.15.1:0/3533295320 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe0e00a3d80 0x7fe0e00a6240 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:49.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.782+0000 7fe102600700 1 --2- 172.21.15.1:0/3533295320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0fc102660 0x7fe0fc1a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:49.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.782+0000 7fe102600700 1 --2- 172.21.15.1:0/3533295320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0fc101340 0x7fe0fc19ace0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:49.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.782+0000 7fe102600700 1 --2- 172.21.15.1:0/3533295320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe0fc101ca0 0x7fe0fc19b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:49.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.782+0000 7fe102600700 1 -- 172.21.15.1:0/3533295320 >> 172.21.15.1:0/3533295320 conn(0x7fe0fc0fcf60 msgr2=0x7fe0fc0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:49.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.782+0000 7fe102600700 1 -- 172.21.15.1:0/3533295320 shutdown_connections 2024-04-26T01:48:49.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:49.782+0000 7fe102600700 1 -- 172.21.15.1:0/3533295320 wait complete. 2024-04-26T01:48:50.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:49 smithi001 ceph-mon[27822]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:48:50.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:49 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3533295320' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T01:48:50.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:49 smithi001 ceph-mon[27822]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T01:48:50.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:49 smithi001 ceph-mon[27822]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T01:48:50.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:49 smithi097 ceph-mon[32853]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:48:50.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:49 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3533295320' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T01:48:50.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:49 smithi097 ceph-mon[32853]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T01:48:50.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:49 smithi097 ceph-mon[32853]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T01:48:50.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:49 smithi173 ceph-mon[32553]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:48:50.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:49 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3533295320' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T01:48:50.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:49 smithi173 ceph-mon[32553]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T01:48:50.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:49 smithi173 ceph-mon[32553]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T01:48:50.601 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-04-26T01:48:50.601 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-26T01:48:50.602 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T01:48:50.744 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T01:48:50.744 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-26T01:48:50.745 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-26T01:48:51.253 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:51.992 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:51 smithi001 ceph-mon[27822]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:48:52.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:51 smithi097 ceph-mon[32853]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:48:52.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:51 smithi173 ceph-mon[32553]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:48:53.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.418+0000 7faa87200700 1 -- 172.21.15.1:0/2510117895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa88109e90 msgr2=0x7faa88111a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:53.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.418+0000 7faa87200700 1 --2- 172.21.15.1:0/2510117895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa88109e90 0x7faa88111a60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faa8000b0d0 tx=0x7faa8000b3e0 comp rx=0 tx=0).stop 2024-04-26T01:48:53.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.418+0000 7faa87200700 1 -- 172.21.15.1:0/2510117895 shutdown_connections 2024-04-26T01:48:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.418+0000 7faa87200700 1 --2- 172.21.15.1:0/2510117895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa88109e90 0x7faa88111a60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.418+0000 7faa87200700 1 --2- 172.21.15.1:0/2510117895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa88104db0 0x7faa88105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.418+0000 7faa87200700 1 --2- 172.21.15.1:0/2510117895 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa88105760 0x7faa881097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.418+0000 7faa87200700 1 -- 172.21.15.1:0/2510117895 >> 172.21.15.1:0/2510117895 conn(0x7faa88100620 msgr2=0x7faa88102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.419+0000 7faa87200700 1 -- 172.21.15.1:0/2510117895 shutdown_connections 2024-04-26T01:48:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.419+0000 7faa87200700 1 -- 172.21.15.1:0/2510117895 wait complete. 2024-04-26T01:48:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.419+0000 7faa87200700 1 Processor -- start 2024-04-26T01:48:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.420+0000 7faa87200700 1 -- start start 2024-04-26T01:48:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.420+0000 7faa87200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa88104db0 0x7faa8819b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.420+0000 7faa87200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa88105760 0x7faa8819b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.420+0000 7faa87200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa88109e90 0x7faa8819fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.420+0000 7faa87200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faa88114d90 con 0x7faa88109e90 2024-04-26T01:48:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.420+0000 7faa87200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faa88114c10 con 0x7faa88105760 2024-04-26T01:48:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.420+0000 7faa87200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faa88114f10 con 0x7faa88104db0 2024-04-26T01:48:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.421+0000 7faa85e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa88104db0 0x7faa8819b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.421+0000 7faa86800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa88109e90 0x7faa8819fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.421+0000 7faa86800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa88109e90 0x7faa8819fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38302/0 (socket says 172.21.15.1:38302) 2024-04-26T01:48:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.421+0000 7faa85400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa88105760 0x7faa8819b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.421+0000 7faa86800700 1 -- 172.21.15.1:0/521710464 learned_addr learned my addr 172.21.15.1:0/521710464 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:48:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.421+0000 7faa85e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa88104db0 0x7faa8819b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:41932/0 (socket says 172.21.15.1:41932) 2024-04-26T01:48:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.421+0000 7faa86800700 1 -- 172.21.15.1:0/521710464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa88104db0 msgr2=0x7faa8819b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.421+0000 7faa86800700 1 --2- 172.21.15.1:0/521710464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa88104db0 0x7faa8819b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.421+0000 7faa86800700 1 -- 172.21.15.1:0/521710464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa88105760 msgr2=0x7faa8819b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.421+0000 7faa86800700 1 --2- 172.21.15.1:0/521710464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa88105760 0x7faa8819b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.421+0000 7faa86800700 1 -- 172.21.15.1:0/521710464 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa881a0250 con 0x7faa88109e90 2024-04-26T01:48:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.421+0000 7faa85400700 1 --2- 172.21.15.1:0/521710464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa88105760 0x7faa8819b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:48:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.421+0000 7faa86800700 1 --2- 172.21.15.1:0/521710464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa88109e90 0x7faa8819fb10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faa800079a0 tx=0x7faa800079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.422+0000 7faa7ea00700 1 -- 172.21.15.1:0/521710464 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8001f070 con 0x7faa88109e90 2024-04-26T01:48:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.422+0000 7faa7ea00700 1 -- 172.21.15.1:0/521710464 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa80003c00 con 0x7faa88109e90 2024-04-26T01:48:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.422+0000 7faa87200700 1 -- 172.21.15.1:0/521710464 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa881a04e0 con 0x7faa88109e90 2024-04-26T01:48:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.422+0000 7faa7ea00700 1 -- 172.21.15.1:0/521710464 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8001b430 con 0x7faa88109e90 2024-04-26T01:48:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.422+0000 7faa87200700 1 -- 172.21.15.1:0/521710464 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa881a7dc0 con 0x7faa88109e90 2024-04-26T01:48:53.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.423+0000 7faa87200700 1 -- 172.21.15.1:0/521710464 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa88069730 con 0x7faa88109e90 2024-04-26T01:48:53.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.425+0000 7faa7ea00700 1 -- 172.21.15.1:0/521710464 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7faa80022070 con 0x7faa88109e90 2024-04-26T01:48:53.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.426+0000 7faa7ea00700 1 --2- 172.21.15.1:0/521710464 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7faa6c0a3d50 0x7faa6c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:53.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.427+0000 7faa7ea00700 1 -- 172.21.15.1:0/521710464 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7faa8000f020 con 0x7faa88109e90 2024-04-26T01:48:53.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.427+0000 7faa85e00700 1 --2- 172.21.15.1:0/521710464 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7faa6c0a3d50 0x7faa6c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:53.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.427+0000 7faa85e00700 1 --2- 172.21.15.1:0/521710464 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7faa6c0a3d50 0x7faa6c0a6210 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faa78009c60 tx=0x7faa78009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:53.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.429+0000 7faa7ea00700 1 -- 172.21.15.1:0/521710464 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa80029020 con 0x7faa88109e90 2024-04-26T01:48:53.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.723+0000 7faa87200700 1 -- 172.21.15.1:0/521710464 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7faa8802df80 con 0x7faa88109e90 2024-04-26T01:48:53.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.731+0000 7faa7ea00700 1 -- 172.21.15.1:0/521710464 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264984 (secure 0 0 0) 0x7faa8009e220 con 0x7faa88109e90 2024-04-26T01:48:53.731 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:48:53.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.737+0000 7faa87200700 1 -- 172.21.15.1:0/521710464 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7faa6c0a3d50 msgr2=0x7faa6c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:53.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.737+0000 7faa87200700 1 --2- 172.21.15.1:0/521710464 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7faa6c0a3d50 0x7faa6c0a6210 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faa78009c60 tx=0x7faa78009260 comp rx=0 tx=0).stop 2024-04-26T01:48:53.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.737+0000 7faa87200700 1 -- 172.21.15.1:0/521710464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa88109e90 msgr2=0x7faa8819fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:53.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.737+0000 7faa87200700 1 --2- 172.21.15.1:0/521710464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa88109e90 0x7faa8819fb10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faa800079a0 tx=0x7faa800079d0 comp rx=0 tx=0).stop 2024-04-26T01:48:53.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.737+0000 7faa87200700 1 -- 172.21.15.1:0/521710464 shutdown_connections 2024-04-26T01:48:53.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.737+0000 7faa87200700 1 --2- 172.21.15.1:0/521710464 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7faa6c0a3d50 0x7faa6c0a6210 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:53.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.737+0000 7faa87200700 1 --2- 172.21.15.1:0/521710464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa88109e90 0x7faa8819fb10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:53.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.737+0000 7faa87200700 1 --2- 172.21.15.1:0/521710464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa88104db0 0x7faa8819b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:53.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.737+0000 7faa87200700 1 --2- 172.21.15.1:0/521710464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa88105760 0x7faa8819b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:53.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.737+0000 7faa87200700 1 -- 172.21.15.1:0/521710464 >> 172.21.15.1:0/521710464 conn(0x7faa88100620 msgr2=0x7faa88101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:53.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.737+0000 7faa87200700 1 -- 172.21.15.1:0/521710464 shutdown_connections 2024-04-26T01:48:53.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:53.737+0000 7faa87200700 1 -- 172.21.15.1:0/521710464 wait complete. 2024-04-26T01:48:54.089 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:53 smithi001 ceph-mon[27822]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:48:54.089 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:53 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/521710464' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T01:48:54.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:53 smithi097 ceph-mon[32853]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:48:54.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:53 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/521710464' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T01:48:54.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:53 smithi173 ceph-mon[32553]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:48:54.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:53 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/521710464' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T01:48:54.601 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2140737121},{"type":"v1","addr":"172.21.15.1:6801","nonce":2140737121}]},"active_addr":"172.21.15.1:6801/2140737121","active_change":"2024-04-26T01:39:20.646409+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.1:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":304803917}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":1607430337}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":25191068}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":941157221}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":900215463}]}]} 2024-04-26T01:48:54.606 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-26T01:48:54.606 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-26T01:48:54.606 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:48:55.114 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:56.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:55 smithi097 ceph-mon[32853]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:48:56.278 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:55 smithi001 ceph-mon[27822]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:48:56.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:55 smithi173 ceph-mon[32553]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:48:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.218+0000 7f3070400700 1 -- 172.21.15.1:0/3438600907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f306810e3b0 msgr2=0x7f3068115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.218+0000 7f3070400700 1 --2- 172.21.15.1:0/3438600907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f306810e3b0 0x7f3068115f80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3060009fc0 tx=0x7f306000b330 comp rx=0 tx=0).stop 2024-04-26T01:48:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.219+0000 7f3070400700 1 -- 172.21.15.1:0/3438600907 shutdown_connections 2024-04-26T01:48:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.219+0000 7f3070400700 1 --2- 172.21.15.1:0/3438600907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f306810e3b0 0x7f3068115f80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.219+0000 7f3070400700 1 --2- 172.21.15.1:0/3438600907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30681092d0 0x7f30681096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.219+0000 7f3070400700 1 --2- 172.21.15.1:0/3438600907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3068109c80 0x7f306810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.219+0000 7f3070400700 1 -- 172.21.15.1:0/3438600907 >> 172.21.15.1:0/3438600907 conn(0x7f3068076210 msgr2=0x7f3068107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.219+0000 7f3070400700 1 -- 172.21.15.1:0/3438600907 shutdown_connections 2024-04-26T01:48:57.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.219+0000 7f3070400700 1 -- 172.21.15.1:0/3438600907 wait complete. 2024-04-26T01:48:57.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.220+0000 7f3070400700 1 Processor -- start 2024-04-26T01:48:57.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.220+0000 7f3070400700 1 -- start start 2024-04-26T01:48:57.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.220+0000 7f3070400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30681092d0 0x7f306819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:57.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.220+0000 7f3070400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3068109c80 0x7f306819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:57.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.220+0000 7f3070400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f306810e3b0 0x7f30681a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:57.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.220+0000 7f3070400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f30681192c0 con 0x7f3068109c80 2024-04-26T01:48:57.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.220+0000 7f3070400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3068119140 con 0x7f30681092d0 2024-04-26T01:48:57.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.220+0000 7f3070400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3068119440 con 0x7f306810e3b0 2024-04-26T01:48:57.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.221+0000 7f306d000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3068109c80 0x7f306819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:57.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.221+0000 7f306da00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30681092d0 0x7f306819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:57.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.221+0000 7f306d000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3068109c80 0x7f306819fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38320/0 (socket says 172.21.15.1:38320) 2024-04-26T01:48:57.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.221+0000 7f306d000700 1 -- 172.21.15.1:0/2076729485 learned_addr learned my addr 172.21.15.1:0/2076729485 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:48:57.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.221+0000 7f306e400700 1 --2- 172.21.15.1:0/2076729485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f306810e3b0 0x7f30681a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:57.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.221+0000 7f306d000700 1 -- 172.21.15.1:0/2076729485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f306810e3b0 msgr2=0x7f30681a3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:57.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.221+0000 7f306d000700 1 --2- 172.21.15.1:0/2076729485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f306810e3b0 0x7f30681a3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:57.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.221+0000 7f306d000700 1 -- 172.21.15.1:0/2076729485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30681092d0 msgr2=0x7f306819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:57.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.221+0000 7f306d000700 1 --2- 172.21.15.1:0/2076729485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30681092d0 0x7f306819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:57.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.221+0000 7f306d000700 1 -- 172.21.15.1:0/2076729485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30681a46f0 con 0x7f3068109c80 2024-04-26T01:48:57.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.221+0000 7f306e400700 1 --2- 172.21.15.1:0/2076729485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f306810e3b0 0x7f30681a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:48:57.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.222+0000 7f306d000700 1 --2- 172.21.15.1:0/2076729485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3068109c80 0x7f306819fc70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f305400e9f0 tx=0x7f305400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.222+0000 7f3066000700 1 -- 172.21.15.1:0/2076729485 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305400c9c0 con 0x7f3068109c80 2024-04-26T01:48:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.222+0000 7f3066000700 1 -- 172.21.15.1:0/2076729485 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f305400cb60 con 0x7f3068109c80 2024-04-26T01:48:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.222+0000 7f3070400700 1 -- 172.21.15.1:0/2076729485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30681a49e0 con 0x7f3068109c80 2024-04-26T01:48:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.222+0000 7f3066000700 1 -- 172.21.15.1:0/2076729485 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054018680 con 0x7f3068109c80 2024-04-26T01:48:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.222+0000 7f3070400700 1 -- 172.21.15.1:0/2076729485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30681ac260 con 0x7f3068109c80 2024-04-26T01:48:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.223+0000 7f3070400700 1 -- 172.21.15.1:0/2076729485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3068069730 con 0x7f3068109c80 2024-04-26T01:48:57.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.226+0000 7f3066000700 1 -- 172.21.15.1:0/2076729485 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f3054010430 con 0x7f3068109c80 2024-04-26T01:48:57.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.228+0000 7f3066000700 1 --2- 172.21.15.1:0/2076729485 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f30500a3cd0 0x7f30500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.228+0000 7f3066000700 1 -- 172.21.15.1:0/2076729485 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f30540d0730 con 0x7f3068109c80 2024-04-26T01:48:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.228+0000 7f306da00700 1 --2- 172.21.15.1:0/2076729485 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f30500a3cd0 0x7f30500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.229+0000 7f306da00700 1 --2- 172.21.15.1:0/2076729485 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f30500a3cd0 0x7f30500a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f305c009c60 tx=0x7f305c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:57.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.233+0000 7f3066000700 1 -- 172.21.15.1:0/2076729485 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f305409cf30 con 0x7f3068109c80 2024-04-26T01:48:57.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.466+0000 7f3070400700 1 -- 172.21.15.1:0/2076729485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f306802df80 con 0x7f3068109c80 2024-04-26T01:48:57.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.471+0000 7f3066000700 1 -- 172.21.15.1:0/2076729485 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22649 (secure 0 0 0) 0x7f305409c830 con 0x7f3068109c80 2024-04-26T01:48:57.469 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:48:57.470 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":92,"fsid":"77cfcc80-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:38:03.531182+0000","modified":"2024-04-26T01:48:22.381867+0000","last_up_change":"2024-04-26T01:46:16.151275+0000","last_in_change":"2024-04-26T01:45:56.514228+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:42:04.805488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:47:21.957421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:47:25.860260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb10ae12-09ee-4d6b-bc78-8c6016613b81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6803","nonce":3176633965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6805","nonce":3176633965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6809","nonce":3176633965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6807","nonce":3176633965}]},"public_addr":"172.21.15.1:6803/3176633965","cluster_addr":"172.21.15.1:6805/3176633965","heartbeat_back_addr":"172.21.15.1:6809/3176633965","heartbeat_front_addr":"172.21.15.1:6807/3176633965","state":["exists","up"]},{"osd":1,"uuid":"acd12888-6ff6-40bc-99e7-e5c395e85b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6801","nonce":3755022621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6803","nonce":3755022621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6807","nonce":3755022621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6805","nonce":3755022621}]},"public_addr":"172.21.15.97:6801/3755022621","cluster_addr":"172.21.15.97:6803/3755022621","heartbeat_back_addr":"172.21.15.97:6807/3755022621","heartbeat_front_addr":"172.21.15.97:6805/3755022621","state":["exists","up"]},{"osd":2,"uuid":"79971508-e256-4145-8111-a75666e886fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6801","nonce":734861771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6803","nonce":734861771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6807","nonce":734861771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6805","nonce":734861771}]},"public_addr":"172.21.15.173:6801/734861771","cluster_addr":"172.21.15.173:6803/734861771","heartbeat_back_addr":"172.21.15.173:6807/734861771","heartbeat_front_addr":"172.21.15.173:6805/734861771","state":["exists","up"]},{"osd":3,"uuid":"a8ccc4a9-0f31-446a-8bcb-4691532397c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6811","nonce":3349961961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6813","nonce":3349961961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6817","nonce":3349961961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6815","nonce":3349961961}]},"public_addr":"172.21.15.1:6811/3349961961","cluster_addr":"172.21.15.1:6813/3349961961","heartbeat_back_addr":"172.21.15.1:6817/3349961961","heartbeat_front_addr":"172.21.15.1:6815/3349961961","state":["exists","up"]},{"osd":4,"uuid":"5b724648-4df7-40fc-89d5-55f170cdd806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6809","nonce":2494639397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6811","nonce":2494639397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6815","nonce":2494639397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6813","nonce":2494639397}]},"public_addr":"172.21.15.97:6809/2494639397","cluster_addr":"172.21.15.97:6811/2494639397","heartbeat_back_addr":"172.21.15.97:6815/2494639397","heartbeat_front_addr":"172.21.15.97:6813/2494639397","state":["exists","up"]},{"osd":5,"uuid":"a5a24a9c-3007-4b87-8562-40c98c41ef8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6809","nonce":1026507957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6811","nonce":1026507957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6815","nonce":1026507957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6813","nonce":1026507957}]},"public_addr":"172.21.15.173:6809/1026507957","cluster_addr":"172.21.15.173:6811/1026507957","heartbeat_back_addr":"172.21.15.173:6815/1026507957","heartbeat_front_addr":"172.21.15.173:6813/1026507957","state":["exists","up"]},{"osd":6,"uuid":"97cd2e88-a56c-4b1a-8104-65b23a77986a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6819","nonce":444030587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6821","nonce":444030587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6825","nonce":444030587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6823","nonce":444030587}]},"public_addr":"172.21.15.1:6819/444030587","cluster_addr":"172.21.15.1:6821/444030587","heartbeat_back_addr":"172.21.15.1:6825/444030587","heartbeat_front_addr":"172.21.15.1:6823/444030587","state":["exists","up"]},{"osd":7,"uuid":"afc0c2dd-e112-4358-b3aa-944366d5d383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6817","nonce":2600022531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6819","nonce":2600022531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6823","nonce":2600022531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6821","nonce":2600022531}]},"public_addr":"172.21.15.97:6817/2600022531","cluster_addr":"172.21.15.97:6819/2600022531","heartbeat_back_addr":"172.21.15.97:6823/2600022531","heartbeat_front_addr":"172.21.15.97:6821/2600022531","state":["exists","up"]},{"osd":8,"uuid":"41fa612b-793e-44da-af8b-44d17a9de303","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6817","nonce":1334205527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6819","nonce":1334205527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6823","nonce":1334205527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6821","nonce":1334205527}]},"public_addr":"172.21.15.173:6817/1334205527","cluster_addr":"172.21.15.173:6819/1334205527","heartbeat_back_addr":"172.21.15.173:6823/1334205527","heartbeat_front_addr":"172.21.15.173:6821/1334205527","state":["exists","up"]},{"osd":9,"uuid":"bbbb163b-f8c4-4be5-8f80-82445e8d407f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6827","nonce":2973486839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6829","nonce":2973486839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6833","nonce":2973486839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6831","nonce":2973486839}]},"public_addr":"172.21.15.1:6827/2973486839","cluster_addr":"172.21.15.1:6829/2973486839","heartbeat_back_addr":"172.21.15.1:6833/2973486839","heartbeat_front_addr":"172.21.15.1:6831/2973486839","state":["exists","up"]},{"osd":10,"uuid":"1b447b34-0600-4727-826d-b3d68adfb18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6825","nonce":1693425268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6827","nonce":1693425268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6831","nonce":1693425268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6829","nonce":1693425268}]},"public_addr":"172.21.15.97:6825/1693425268","cluster_addr":"172.21.15.97:6827/1693425268","heartbeat_back_addr":"172.21.15.97:6831/1693425268","heartbeat_front_addr":"172.21.15.97:6829/1693425268","state":["exists","up"]},{"osd":11,"uuid":"f439aae1-f50a-4176-889d-36fa5131541a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6825","nonce":1141524960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6827","nonce":1141524960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6831","nonce":1141524960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6829","nonce":1141524960}]},"public_addr":"172.21.15.173:6825/1141524960","cluster_addr":"172.21.15.173:6827/1141524960","heartbeat_back_addr":"172.21.15.173:6831/1141524960","heartbeat_front_addr":"172.21.15.173:6829/1141524960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:12.955643+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:38.846652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:02.556062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:34.900933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:01.170878+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:24.842573+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:58.304440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:24.868689+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:48.772003+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:22.611681+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:49.470779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:14.163984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6800/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/1000222947":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/457958597":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6801/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:6800/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/2715798962":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/1923058728":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/2489038777":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/2869502448":"2024-04-27T01:38:45.974031+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:48:57.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.474+0000 7f3070400700 1 -- 172.21.15.1:0/2076729485 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f30500a3cd0 msgr2=0x7f30500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:57.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.474+0000 7f3070400700 1 --2- 172.21.15.1:0/2076729485 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f30500a3cd0 0x7f30500a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f305c009c60 tx=0x7f305c009260 comp rx=0 tx=0).stop 2024-04-26T01:48:57.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.474+0000 7f3070400700 1 -- 172.21.15.1:0/2076729485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3068109c80 msgr2=0x7f306819fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:57.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.474+0000 7f3070400700 1 --2- 172.21.15.1:0/2076729485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3068109c80 0x7f306819fc70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f305400e9f0 tx=0x7f305400ed00 comp rx=0 tx=0).stop 2024-04-26T01:48:57.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.474+0000 7f3070400700 1 -- 172.21.15.1:0/2076729485 shutdown_connections 2024-04-26T01:48:57.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.474+0000 7f3070400700 1 --2- 172.21.15.1:0/2076729485 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f30500a3cd0 0x7f30500a6190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.474+0000 7f3070400700 1 --2- 172.21.15.1:0/2076729485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f306810e3b0 0x7f30681a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.474+0000 7f3070400700 1 --2- 172.21.15.1:0/2076729485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30681092d0 0x7f306819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.474+0000 7f3070400700 1 --2- 172.21.15.1:0/2076729485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3068109c80 0x7f306819fc70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.474+0000 7f3070400700 1 -- 172.21.15.1:0/2076729485 >> 172.21.15.1:0/2076729485 conn(0x7f3068076210 msgr2=0x7f30680767d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.474+0000 7f3070400700 1 -- 172.21.15.1:0/2076729485 shutdown_connections 2024-04-26T01:48:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:48:57.474+0000 7f3070400700 1 -- 172.21.15.1:0/2076729485 wait complete. 2024-04-26T01:48:58.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:57 smithi001 ceph-mon[27822]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:48:58.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:57 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2076729485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:48:58.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:57 smithi097 ceph-mon[32853]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:48:58.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:57 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/2076729485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:48:58.305 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-26T01:48:58.306 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:48:58.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:57 smithi173 ceph-mon[32553]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:48:58.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:57 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/2076729485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:48:58.760 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:00.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:48:59 smithi097 ceph-mon[32853]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:00.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:48:59 smithi173 ceph-mon[32553]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:00.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:48:59 smithi001 ceph-mon[27822]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:00.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.983+0000 7f6061a00700 1 -- 172.21.15.1:0/579837317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f605c113920 msgr2=0x7f605c115d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:00.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.983+0000 7f6061a00700 1 --2- 172.21.15.1:0/579837317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f605c113920 0x7f605c115d10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f605400b0a0 tx=0x7f605400b3b0 comp rx=0 tx=0).stop 2024-04-26T01:49:00.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.984+0000 7f6061a00700 1 -- 172.21.15.1:0/579837317 shutdown_connections 2024-04-26T01:49:00.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.984+0000 7f6061a00700 1 --2- 172.21.15.1:0/579837317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f605c113920 0x7f605c115d10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:00.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.984+0000 7f6061a00700 1 --2- 172.21.15.1:0/579837317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f605c076f70 0x7f605c077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:00.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.984+0000 7f6061a00700 1 --2- 172.21.15.1:0/579837317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f605c077890 0x7f605c1133e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:00.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.984+0000 7f6061a00700 1 -- 172.21.15.1:0/579837317 >> 172.21.15.1:0/579837317 conn(0x7f605c100770 msgr2=0x7f605c102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:00.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.984+0000 7f6061a00700 1 -- 172.21.15.1:0/579837317 shutdown_connections 2024-04-26T01:49:00.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.984+0000 7f6061a00700 1 -- 172.21.15.1:0/579837317 wait complete. 2024-04-26T01:49:00.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.985+0000 7f6061a00700 1 Processor -- start 2024-04-26T01:49:00.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.985+0000 7f6061a00700 1 -- start start 2024-04-26T01:49:00.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.985+0000 7f6061a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f605c076f70 0x7f605c19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:00.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.985+0000 7f6061a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f605c077890 0x7f605c19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:00.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.985+0000 7f6061a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f605c113920 0x7f605c1a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:00.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.985+0000 7f6061a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f605c1192a0 con 0x7f605c113920 2024-04-26T01:49:00.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.986+0000 7f6061a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f605c119120 con 0x7f605c076f70 2024-04-26T01:49:00.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.986+0000 7f6061a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f605c119420 con 0x7f605c077890 2024-04-26T01:49:00.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.986+0000 7f605aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f605c076f70 0x7f605c19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:00.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.986+0000 7f605aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f605c076f70 0x7f605c19f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:35428/0 (socket says 172.21.15.1:35428) 2024-04-26T01:49:00.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.986+0000 7f605aa00700 1 -- 172.21.15.1:0/427041479 learned_addr learned my addr 172.21.15.1:0/427041479 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:00.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.986+0000 7f605b400700 1 --2- 172.21.15.1:0/427041479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f605c113920 0x7f605c1a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:00.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.986+0000 7f605a000700 1 --2- 172.21.15.1:0/427041479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f605c077890 0x7f605c19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:00.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.986+0000 7f605aa00700 1 -- 172.21.15.1:0/427041479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f605c077890 msgr2=0x7f605c19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:00.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.986+0000 7f605aa00700 1 --2- 172.21.15.1:0/427041479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f605c077890 0x7f605c19fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:00.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.986+0000 7f605aa00700 1 -- 172.21.15.1:0/427041479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f605c113920 msgr2=0x7f605c1a3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:00.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.986+0000 7f605aa00700 1 --2- 172.21.15.1:0/427041479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f605c113920 0x7f605c1a3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:00.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.986+0000 7f605aa00700 1 -- 172.21.15.1:0/427041479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605c1a4720 con 0x7f605c076f70 2024-04-26T01:49:00.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.987+0000 7f605aa00700 1 --2- 172.21.15.1:0/427041479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f605c076f70 0x7f605c19f710 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f604c00d950 tx=0x7f604c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:00.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.987+0000 7f6053400700 1 -- 172.21.15.1:0/427041479 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c016070 con 0x7f605c076f70 2024-04-26T01:49:00.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.987+0000 7f6061a00700 1 -- 172.21.15.1:0/427041479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f605c1a4a10 con 0x7f605c076f70 2024-04-26T01:49:00.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.987+0000 7f6053400700 1 -- 172.21.15.1:0/427041479 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f604c00fd40 con 0x7f605c076f70 2024-04-26T01:49:00.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.987+0000 7f6053400700 1 -- 172.21.15.1:0/427041479 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c00e3b0 con 0x7f605c076f70 2024-04-26T01:49:00.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.987+0000 7f6061a00700 1 -- 172.21.15.1:0/427041479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f605c1ac290 con 0x7f605c076f70 2024-04-26T01:49:00.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.988+0000 7f6061a00700 1 -- 172.21.15.1:0/427041479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f605c069730 con 0x7f605c076f70 2024-04-26T01:49:00.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.990+0000 7f6053400700 1 -- 172.21.15.1:0/427041479 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f604c019070 con 0x7f605c076f70 2024-04-26T01:49:00.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.992+0000 7f6053400700 1 --2- 172.21.15.1:0/427041479 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f60400a3cd0 0x7f60400a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:00.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.992+0000 7f6053400700 1 -- 172.21.15.1:0/427041479 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f604c0d00c0 con 0x7f605c076f70 2024-04-26T01:49:00.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.992+0000 7f605a000700 1 --2- 172.21.15.1:0/427041479 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f60400a3cd0 0x7f60400a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:00.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.993+0000 7f605a000700 1 --2- 172.21.15.1:0/427041479 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f60400a3cd0 0x7f60400a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f605c104370 tx=0x7f6048009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:00.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:00.996+0000 7f6053400700 1 -- 172.21.15.1:0/427041479 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f604c09c8c0 con 0x7f605c076f70 2024-04-26T01:49:01.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:01.231+0000 7f6061a00700 1 -- 172.21.15.1:0/427041479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f605c02df80 con 0x7f605c076f70 2024-04-26T01:49:01.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:01.233+0000 7f6053400700 1 -- 172.21.15.1:0/427041479 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22649 (secure 0 0 0) 0x7f604c09c1c0 con 0x7f605c076f70 2024-04-26T01:49:01.232 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:49:01.232 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":92,"fsid":"77cfcc80-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:38:03.531182+0000","modified":"2024-04-26T01:48:22.381867+0000","last_up_change":"2024-04-26T01:46:16.151275+0000","last_in_change":"2024-04-26T01:45:56.514228+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:42:04.805488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:47:21.957421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:47:25.860260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb10ae12-09ee-4d6b-bc78-8c6016613b81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6803","nonce":3176633965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6805","nonce":3176633965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6809","nonce":3176633965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6807","nonce":3176633965}]},"public_addr":"172.21.15.1:6803/3176633965","cluster_addr":"172.21.15.1:6805/3176633965","heartbeat_back_addr":"172.21.15.1:6809/3176633965","heartbeat_front_addr":"172.21.15.1:6807/3176633965","state":["exists","up"]},{"osd":1,"uuid":"acd12888-6ff6-40bc-99e7-e5c395e85b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6801","nonce":3755022621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6803","nonce":3755022621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6807","nonce":3755022621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6805","nonce":3755022621}]},"public_addr":"172.21.15.97:6801/3755022621","cluster_addr":"172.21.15.97:6803/3755022621","heartbeat_back_addr":"172.21.15.97:6807/3755022621","heartbeat_front_addr":"172.21.15.97:6805/3755022621","state":["exists","up"]},{"osd":2,"uuid":"79971508-e256-4145-8111-a75666e886fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6801","nonce":734861771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6803","nonce":734861771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6807","nonce":734861771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6805","nonce":734861771}]},"public_addr":"172.21.15.173:6801/734861771","cluster_addr":"172.21.15.173:6803/734861771","heartbeat_back_addr":"172.21.15.173:6807/734861771","heartbeat_front_addr":"172.21.15.173:6805/734861771","state":["exists","up"]},{"osd":3,"uuid":"a8ccc4a9-0f31-446a-8bcb-4691532397c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6811","nonce":3349961961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6813","nonce":3349961961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6817","nonce":3349961961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6815","nonce":3349961961}]},"public_addr":"172.21.15.1:6811/3349961961","cluster_addr":"172.21.15.1:6813/3349961961","heartbeat_back_addr":"172.21.15.1:6817/3349961961","heartbeat_front_addr":"172.21.15.1:6815/3349961961","state":["exists","up"]},{"osd":4,"uuid":"5b724648-4df7-40fc-89d5-55f170cdd806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6809","nonce":2494639397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6811","nonce":2494639397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6815","nonce":2494639397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6813","nonce":2494639397}]},"public_addr":"172.21.15.97:6809/2494639397","cluster_addr":"172.21.15.97:6811/2494639397","heartbeat_back_addr":"172.21.15.97:6815/2494639397","heartbeat_front_addr":"172.21.15.97:6813/2494639397","state":["exists","up"]},{"osd":5,"uuid":"a5a24a9c-3007-4b87-8562-40c98c41ef8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6809","nonce":1026507957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6811","nonce":1026507957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6815","nonce":1026507957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6813","nonce":1026507957}]},"public_addr":"172.21.15.173:6809/1026507957","cluster_addr":"172.21.15.173:6811/1026507957","heartbeat_back_addr":"172.21.15.173:6815/1026507957","heartbeat_front_addr":"172.21.15.173:6813/1026507957","state":["exists","up"]},{"osd":6,"uuid":"97cd2e88-a56c-4b1a-8104-65b23a77986a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6819","nonce":444030587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6821","nonce":444030587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6825","nonce":444030587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6823","nonce":444030587}]},"public_addr":"172.21.15.1:6819/444030587","cluster_addr":"172.21.15.1:6821/444030587","heartbeat_back_addr":"172.21.15.1:6825/444030587","heartbeat_front_addr":"172.21.15.1:6823/444030587","state":["exists","up"]},{"osd":7,"uuid":"afc0c2dd-e112-4358-b3aa-944366d5d383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6817","nonce":2600022531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6819","nonce":2600022531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6823","nonce":2600022531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6821","nonce":2600022531}]},"public_addr":"172.21.15.97:6817/2600022531","cluster_addr":"172.21.15.97:6819/2600022531","heartbeat_back_addr":"172.21.15.97:6823/2600022531","heartbeat_front_addr":"172.21.15.97:6821/2600022531","state":["exists","up"]},{"osd":8,"uuid":"41fa612b-793e-44da-af8b-44d17a9de303","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6817","nonce":1334205527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6819","nonce":1334205527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6823","nonce":1334205527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6821","nonce":1334205527}]},"public_addr":"172.21.15.173:6817/1334205527","cluster_addr":"172.21.15.173:6819/1334205527","heartbeat_back_addr":"172.21.15.173:6823/1334205527","heartbeat_front_addr":"172.21.15.173:6821/1334205527","state":["exists","up"]},{"osd":9,"uuid":"bbbb163b-f8c4-4be5-8f80-82445e8d407f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6827","nonce":2973486839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6829","nonce":2973486839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6833","nonce":2973486839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6831","nonce":2973486839}]},"public_addr":"172.21.15.1:6827/2973486839","cluster_addr":"172.21.15.1:6829/2973486839","heartbeat_back_addr":"172.21.15.1:6833/2973486839","heartbeat_front_addr":"172.21.15.1:6831/2973486839","state":["exists","up"]},{"osd":10,"uuid":"1b447b34-0600-4727-826d-b3d68adfb18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6825","nonce":1693425268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6827","nonce":1693425268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6831","nonce":1693425268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6829","nonce":1693425268}]},"public_addr":"172.21.15.97:6825/1693425268","cluster_addr":"172.21.15.97:6827/1693425268","heartbeat_back_addr":"172.21.15.97:6831/1693425268","heartbeat_front_addr":"172.21.15.97:6829/1693425268","state":["exists","up"]},{"osd":11,"uuid":"f439aae1-f50a-4176-889d-36fa5131541a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6825","nonce":1141524960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6827","nonce":1141524960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6831","nonce":1141524960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6829","nonce":1141524960}]},"public_addr":"172.21.15.173:6825/1141524960","cluster_addr":"172.21.15.173:6827/1141524960","heartbeat_back_addr":"172.21.15.173:6831/1141524960","heartbeat_front_addr":"172.21.15.173:6829/1141524960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:12.955643+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:38.846652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:02.556062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:34.900933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:01.170878+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:24.842573+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:58.304440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:24.868689+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:48.772003+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:22.611681+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:49.470779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:14.163984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6800/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/1000222947":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/457958597":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6801/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:6800/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/2715798962":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/1923058728":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/2489038777":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/2869502448":"2024-04-27T01:38:45.974031+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:49:01.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:01.236+0000 7f6061a00700 1 -- 172.21.15.1:0/427041479 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f60400a3cd0 msgr2=0x7f60400a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:01.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:01.236+0000 7f6061a00700 1 --2- 172.21.15.1:0/427041479 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f60400a3cd0 0x7f60400a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f605c104370 tx=0x7f6048009260 comp rx=0 tx=0).stop 2024-04-26T01:49:01.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:01.236+0000 7f6061a00700 1 -- 172.21.15.1:0/427041479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f605c076f70 msgr2=0x7f605c19f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:01.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:01.236+0000 7f6061a00700 1 --2- 172.21.15.1:0/427041479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f605c076f70 0x7f605c19f710 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f604c00d950 tx=0x7f604c00dc60 comp rx=0 tx=0).stop 2024-04-26T01:49:01.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:01.236+0000 7f6061a00700 1 -- 172.21.15.1:0/427041479 shutdown_connections 2024-04-26T01:49:01.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:01.236+0000 7f6061a00700 1 --2- 172.21.15.1:0/427041479 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f60400a3cd0 0x7f60400a6190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:01.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:01.236+0000 7f6061a00700 1 --2- 172.21.15.1:0/427041479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f605c113920 0x7f605c1a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:01.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:01.236+0000 7f6061a00700 1 --2- 172.21.15.1:0/427041479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f605c076f70 0x7f605c19f710 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:01.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:01.236+0000 7f6061a00700 1 --2- 172.21.15.1:0/427041479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f605c077890 0x7f605c19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:01.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:01.236+0000 7f6061a00700 1 -- 172.21.15.1:0/427041479 >> 172.21.15.1:0/427041479 conn(0x7f605c100770 msgr2=0x7f605c102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:01.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:01.237+0000 7f6061a00700 1 -- 172.21.15.1:0/427041479 shutdown_connections 2024-04-26T01:49:01.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:01.237+0000 7f6061a00700 1 -- 172.21.15.1:0/427041479 wait complete. 2024-04-26T01:49:02.094 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-26T01:49:02.095 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-26T01:49:02.095 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-26T01:49:02.095 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-26T01:49:02.096 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-26T01:49:02.096 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-26T01:49:02.096 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-26T01:49:02.097 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-26T01:49:02.097 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-26T01:49:02.098 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-26T01:49:02.098 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-26T01:49:02.098 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-26T01:49:02.220 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:01 smithi001 ceph-mon[27822]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:02.220 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:01 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/427041479' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:49:02.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:01 smithi097 ceph-mon[32853]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:02.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:01 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/427041479' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:49:02.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:01 smithi173 ceph-mon[32553]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:02.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:01 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/427041479' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:49:03.162 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:03.186 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:03.208 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:03.208 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:03.212 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:03.226 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:04.170 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:03 smithi001 ceph-mon[27822]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:04.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:03 smithi097 ceph-mon[32853]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:04.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:03 smithi173 ceph-mon[32553]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:06.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:05 smithi097 ceph-mon[32853]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:06.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:05 smithi173 ceph-mon[32553]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:06.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:05 smithi001 ceph-mon[27822]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:08.084 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:07 smithi001 ceph-mon[27822]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:08.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:07 smithi097 ceph-mon[32853]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:08.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:07 smithi173 ceph-mon[32553]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:09.134 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:10.015 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:09 smithi001 ceph-mon[27822]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:10.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.758+0000 7ffb50e00700 1 -- 172.21.15.1:0/1314288156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb4c06ac90 msgr2=0x7ffb4c06b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:10.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.758+0000 7ffb50e00700 1 --2- 172.21.15.1:0/1314288156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb4c06ac90 0x7ffb4c06b0f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffb34009990 tx=0x7ffb34009ca0 comp rx=0 tx=0).stop 2024-04-26T01:49:10.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.758+0000 7ffb50e00700 1 -- 172.21.15.1:0/1314288156 shutdown_connections 2024-04-26T01:49:10.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.758+0000 7ffb50e00700 1 --2- 172.21.15.1:0/1314288156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb4c105480 0x7ffb4c105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:10.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.758+0000 7ffb50e00700 1 --2- 172.21.15.1:0/1314288156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb4c107970 0x7ffb4c06a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:10.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.758+0000 7ffb50e00700 1 --2- 172.21.15.1:0/1314288156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb4c06ac90 0x7ffb4c06b0f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:10.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.758+0000 7ffb50e00700 1 -- 172.21.15.1:0/1314288156 >> 172.21.15.1:0/1314288156 conn(0x7ffb4c0788f0 msgr2=0x7ffb4c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:10.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.758+0000 7ffb50e00700 1 -- 172.21.15.1:0/1314288156 shutdown_connections 2024-04-26T01:49:10.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.758+0000 7ffb50e00700 1 -- 172.21.15.1:0/1314288156 wait complete. 2024-04-26T01:49:10.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.759+0000 7ffb50e00700 1 Processor -- start 2024-04-26T01:49:10.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.760+0000 7ffb50e00700 1 -- start start 2024-04-26T01:49:10.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.760+0000 7ffb50e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb4c06ac90 0x7ffb4c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:10.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.760+0000 7ffb50e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb4c105480 0x7ffb4c19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:10.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.760+0000 7ffb50e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb4c107970 0x7ffb4c1a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:10.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.760+0000 7ffb50e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c1191c0 con 0x7ffb4c105480 2024-04-26T01:49:10.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.760+0000 7ffb50e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c119040 con 0x7ffb4c06ac90 2024-04-26T01:49:10.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.760+0000 7ffb50e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c119340 con 0x7ffb4c107970 2024-04-26T01:49:10.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.760+0000 7ffb41400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb4c105480 0x7ffb4c19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:10.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.760+0000 7ffb41400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb4c105480 0x7ffb4c19fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52314/0 (socket says 172.21.15.1:52314) 2024-04-26T01:49:10.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.760+0000 7ffb41400700 1 -- 172.21.15.1:0/81312665 learned_addr learned my addr 172.21.15.1:0/81312665 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:10.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.760+0000 7ffb49e00700 1 --2- 172.21.15.1:0/81312665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb4c06ac90 0x7ffb4c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:10.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.760+0000 7ffb4a800700 1 --2- 172.21.15.1:0/81312665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb4c107970 0x7ffb4c1a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:10.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.761+0000 7ffb41400700 1 -- 172.21.15.1:0/81312665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb4c107970 msgr2=0x7ffb4c1a3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:10.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.761+0000 7ffb41400700 1 --2- 172.21.15.1:0/81312665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb4c107970 0x7ffb4c1a3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:10.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.761+0000 7ffb41400700 1 -- 172.21.15.1:0/81312665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb4c06ac90 msgr2=0x7ffb4c19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:10.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.761+0000 7ffb41400700 1 --2- 172.21.15.1:0/81312665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb4c06ac90 0x7ffb4c19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:10.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.761+0000 7ffb41400700 1 -- 172.21.15.1:0/81312665 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4c1a46e0 con 0x7ffb4c105480 2024-04-26T01:49:10.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.761+0000 7ffb41400700 1 --2- 172.21.15.1:0/81312665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb4c105480 0x7ffb4c19fc10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffb340057f0 tx=0x7ffb3400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:10.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.761+0000 7ffb43400700 1 -- 172.21.15.1:0/81312665 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb34026070 con 0x7ffb4c105480 2024-04-26T01:49:10.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.762+0000 7ffb43400700 1 -- 172.21.15.1:0/81312665 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb34021410 con 0x7ffb4c105480 2024-04-26T01:49:10.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.762+0000 7ffb43400700 1 -- 172.21.15.1:0/81312665 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb34017650 con 0x7ffb4c105480 2024-04-26T01:49:10.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.762+0000 7ffb50e00700 1 -- 172.21.15.1:0/81312665 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb4c1a4970 con 0x7ffb4c105480 2024-04-26T01:49:10.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.762+0000 7ffb50e00700 1 -- 172.21.15.1:0/81312665 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb4c1ac250 con 0x7ffb4c105480 2024-04-26T01:49:10.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.763+0000 7ffb50e00700 1 -- 172.21.15.1:0/81312665 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffb4c104e80 con 0x7ffb4c105480 2024-04-26T01:49:10.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.765+0000 7ffb43400700 1 -- 172.21.15.1:0/81312665 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7ffb34029070 con 0x7ffb4c105480 2024-04-26T01:49:10.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.766+0000 7ffb43400700 1 --2- 172.21.15.1:0/81312665 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ffb300a3d50 0x7ffb300a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:10.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.766+0000 7ffb43400700 1 -- 172.21.15.1:0/81312665 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7ffb340d1580 con 0x7ffb4c105480 2024-04-26T01:49:10.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.766+0000 7ffb49e00700 1 --2- 172.21.15.1:0/81312665 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ffb300a3d50 0x7ffb300a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:10.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.767+0000 7ffb43400700 1 --2- 172.21.15.1:0/81312665 >> [v2:172.21.15.173:6816/1334205527,v1:172.21.15.173:6817/1334205527] conn(0x7ffb300ad9f0 0x7ffb300afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:10.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.767+0000 7ffb43400700 1 -- 172.21.15.1:0/81312665 --> [v2:172.21.15.173:6816/1334205527,v1:172.21.15.173:6817/1334205527] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffb3401fe80 con 0x7ffb300ad9f0 2024-04-26T01:49:10.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.767+0000 7ffb43400700 1 -- 172.21.15.1:0/81312665 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7ffb340d1930 con 0x7ffb4c105480 2024-04-26T01:49:10.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.767+0000 7ffb4a800700 1 --2- 172.21.15.1:0/81312665 >> [v2:172.21.15.173:6816/1334205527,v1:172.21.15.173:6817/1334205527] conn(0x7ffb300ad9f0 0x7ffb300afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:10.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.767+0000 7ffb49e00700 1 --2- 172.21.15.1:0/81312665 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ffb300a3d50 0x7ffb300a6210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c009be0 tx=0x7ffb3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:10.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.768+0000 7ffb4a800700 1 --2- 172.21.15.1:0/81312665 >> [v2:172.21.15.173:6816/1334205527,v1:172.21.15.173:6817/1334205527] conn(0x7ffb300ad9f0 0x7ffb300afe90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:10.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.769+0000 7ffb43400700 1 -- 172.21.15.1:0/81312665 <== osd.8 v2:172.21.15.173:6816/1334205527 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffb3401fe80 con 0x7ffb300ad9f0 2024-04-26T01:49:10.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.787+0000 7ffb50e00700 1 -- 172.21.15.1:0/81312665 --> [v2:172.21.15.173:6816/1334205527,v1:172.21.15.173:6817/1334205527] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffb4c101f60 con 0x7ffb300ad9f0 2024-04-26T01:49:10.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.788+0000 7ffb43400700 1 -- 172.21.15.1:0/81312665 <== osd.8 v2:172.21.15.173:6816/1334205527 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffb4c101f60 con 0x7ffb300ad9f0 2024-04-26T01:49:10.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.789+0000 7ffb50e00700 1 -- 172.21.15.1:0/81312665 >> [v2:172.21.15.173:6816/1334205527,v1:172.21.15.173:6817/1334205527] conn(0x7ffb300ad9f0 msgr2=0x7ffb300afe90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:10.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.789+0000 7ffb50e00700 1 --2- 172.21.15.1:0/81312665 >> [v2:172.21.15.173:6816/1334205527,v1:172.21.15.173:6817/1334205527] conn(0x7ffb300ad9f0 0x7ffb300afe90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:10.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.789+0000 7ffb50e00700 1 -- 172.21.15.1:0/81312665 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ffb300a3d50 msgr2=0x7ffb300a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:10.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.789+0000 7ffb50e00700 1 --2- 172.21.15.1:0/81312665 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ffb300a3d50 0x7ffb300a6210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c009be0 tx=0x7ffb3c009260 comp rx=0 tx=0).stop 2024-04-26T01:49:10.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.789+0000 7ffb50e00700 1 -- 172.21.15.1:0/81312665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb4c105480 msgr2=0x7ffb4c19fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:10.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.789+0000 7ffb50e00700 1 --2- 172.21.15.1:0/81312665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb4c105480 0x7ffb4c19fc10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffb340057f0 tx=0x7ffb3400bfd0 comp rx=0 tx=0).stop 2024-04-26T01:49:10.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.789+0000 7ffb4a800700 1 -- 172.21.15.1:0/81312665 reap_dead start 2024-04-26T01:49:10.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.789+0000 7ffb50e00700 1 -- 172.21.15.1:0/81312665 shutdown_connections 2024-04-26T01:49:10.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.789+0000 7ffb50e00700 1 -- 172.21.15.1:0/81312665 >> 172.21.15.1:0/81312665 conn(0x7ffb4c0788f0 msgr2=0x7ffb4c104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:10.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.789+0000 7ffb50e00700 1 -- 172.21.15.1:0/81312665 shutdown_connections 2024-04-26T01:49:10.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.789+0000 7ffb50e00700 1 -- 172.21.15.1:0/81312665 wait complete. 2024-04-26T01:49:10.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:09 smithi097 ceph-mon[32853]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:10.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:09 smithi173 ceph-mon[32553]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.170+0000 7fbf30600700 1 -- 172.21.15.1:0/3999649896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf2810f4f0 msgr2=0x7fbf281118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.170+0000 7fbf30600700 1 --2- 172.21.15.1:0/3999649896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf2810f4f0 0x7fbf281118e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000b0a0 tx=0x7fbf2000b3b0 comp rx=0 tx=0).stop 2024-04-26T01:49:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.170+0000 7fbf30600700 1 -- 172.21.15.1:0/3999649896 shutdown_connections 2024-04-26T01:49:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.170+0000 7fbf30600700 1 --2- 172.21.15.1:0/3999649896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf2810f4f0 0x7fbf281118e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.170+0000 7fbf30600700 1 --2- 172.21.15.1:0/3999649896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf2806a750 0x7fbf2806ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.170+0000 7fbf30600700 1 --2- 172.21.15.1:0/3999649896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf2806b070 0x7fbf2810efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.170+0000 7fbf30600700 1 -- 172.21.15.1:0/3999649896 >> 172.21.15.1:0/3999649896 conn(0x7fbf280788f0 msgr2=0x7fbf28078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.170+0000 7fbf30600700 1 -- 172.21.15.1:0/3999649896 shutdown_connections 2024-04-26T01:49:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.170+0000 7fbf30600700 1 -- 172.21.15.1:0/3999649896 wait complete. 2024-04-26T01:49:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.171+0000 7fbf30600700 1 Processor -- start 2024-04-26T01:49:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.172+0000 7fbf30600700 1 -- start start 2024-04-26T01:49:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.172+0000 7fbf30600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf2806a750 0x7fbf2819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.172+0000 7fbf30600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf2806b070 0x7fbf2819b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.172+0000 7fbf30600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf2810f4f0 0x7fbf2819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.172+0000 7fbf30600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbf28114e40 con 0x7fbf2806b070 2024-04-26T01:49:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.172+0000 7fbf30600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf28114cc0 con 0x7fbf2810f4f0 2024-04-26T01:49:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.172+0000 7fbf30600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbf28114fc0 con 0x7fbf2806a750 2024-04-26T01:49:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.173+0000 7fbf2dc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf2806a750 0x7fbf2819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.173+0000 7fbf2dc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf2806a750 0x7fbf2819b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:50072/0 (socket says 172.21.15.1:50072) 2024-04-26T01:49:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.173+0000 7fbf2dc00700 1 -- 172.21.15.1:0/2196344487 learned_addr learned my addr 172.21.15.1:0/2196344487 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.173+0000 7fbf2d200700 1 --2- 172.21.15.1:0/2196344487 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf2806b070 0x7fbf2819b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.173+0000 7fbf2e600700 1 --2- 172.21.15.1:0/2196344487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf2810f4f0 0x7fbf2819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.173+0000 7fbf2e600700 1 -- 172.21.15.1:0/2196344487 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf2806a750 msgr2=0x7fbf2819b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.173+0000 7fbf2e600700 1 --2- 172.21.15.1:0/2196344487 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf2806a750 0x7fbf2819b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.173+0000 7fbf2e600700 1 -- 172.21.15.1:0/2196344487 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf2806b070 msgr2=0x7fbf2819b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.173+0000 7fbf2e600700 1 --2- 172.21.15.1:0/2196344487 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf2806b070 0x7fbf2819b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.173+0000 7fbf2e600700 1 -- 172.21.15.1:0/2196344487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf281a0380 con 0x7fbf2810f4f0 2024-04-26T01:49:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.173+0000 7fbf2d200700 1 --2- 172.21.15.1:0/2196344487 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf2806b070 0x7fbf2819b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:49:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.174+0000 7fbf2e600700 1 --2- 172.21.15.1:0/2196344487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf2810f4f0 0x7fbf2819fc40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000b070 tx=0x7fbf20004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.174+0000 7fbf26000700 1 -- 172.21.15.1:0/2196344487 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf20021070 con 0x7fbf2810f4f0 2024-04-26T01:49:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.174+0000 7fbf30600700 1 -- 172.21.15.1:0/2196344487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf281a0610 con 0x7fbf2810f4f0 2024-04-26T01:49:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.174+0000 7fbf30600700 1 -- 172.21.15.1:0/2196344487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf281a7ef0 con 0x7fbf2810f4f0 2024-04-26T01:49:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.174+0000 7fbf26000700 1 -- 172.21.15.1:0/2196344487 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf20007e30 con 0x7fbf2810f4f0 2024-04-26T01:49:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.174+0000 7fbf26000700 1 -- 172.21.15.1:0/2196344487 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf200046f0 con 0x7fbf2810f4f0 2024-04-26T01:49:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.175+0000 7fbf30600700 1 -- 172.21.15.1:0/2196344487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbf08001030 con 0x7fbf2810f4f0 2024-04-26T01:49:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.178+0000 7fbf26000700 1 -- 172.21.15.1:0/2196344487 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fbf20003c00 con 0x7fbf2810f4f0 2024-04-26T01:49:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.180+0000 7fbf26000700 1 --2- 172.21.15.1:0/2196344487 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbf100a3d50 0x7fbf100a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.180+0000 7fbf2dc00700 1 --2- 172.21.15.1:0/2196344487 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbf100a3d50 0x7fbf100a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.181+0000 7fbf26000700 1 -- 172.21.15.1:0/2196344487 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fbf2000f020 con 0x7fbf2810f4f0 2024-04-26T01:49:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.181+0000 7fbf26000700 1 --2- 172.21.15.1:0/2196344487 >> [v2:172.21.15.97:6816/2600022531,v1:172.21.15.97:6817/2600022531] conn(0x7fbf100ad990 0x7fbf100afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.181+0000 7fbf26000700 1 -- 172.21.15.1:0/2196344487 --> [v2:172.21.15.97:6816/2600022531,v1:172.21.15.97:6817/2600022531] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbf20010ec0 con 0x7fbf100ad990 2024-04-26T01:49:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.181+0000 7fbf26000700 1 -- 172.21.15.1:0/2196344487 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fbf200d2080 con 0x7fbf2810f4f0 2024-04-26T01:49:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.181+0000 7fbf2dc00700 1 --2- 172.21.15.1:0/2196344487 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbf100a3d50 0x7fbf100a6210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009cb0 tx=0x7fbf1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.182+0000 7fbf2d200700 1 --2- 172.21.15.1:0/2196344487 >> [v2:172.21.15.97:6816/2600022531,v1:172.21.15.97:6817/2600022531] conn(0x7fbf100ad990 0x7fbf100afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.182+0000 7fbf2d200700 1 --2- 172.21.15.1:0/2196344487 >> [v2:172.21.15.97:6816/2600022531,v1:172.21.15.97:6817/2600022531] conn(0x7fbf100ad990 0x7fbf100afe30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.183+0000 7fbf26000700 1 -- 172.21.15.1:0/2196344487 <== osd.7 v2:172.21.15.97:6816/2600022531 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf20010ec0 con 0x7fbf100ad990 2024-04-26T01:49:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.202+0000 7fbf30600700 1 -- 172.21.15.1:0/2196344487 --> [v2:172.21.15.97:6816/2600022531,v1:172.21.15.97:6817/2600022531] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbf08002e60 con 0x7fbf100ad990 2024-04-26T01:49:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.203+0000 7fbf26000700 1 -- 172.21.15.1:0/2196344487 <== osd.7 v2:172.21.15.97:6816/2600022531 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbf08002e60 con 0x7fbf100ad990 2024-04-26T01:49:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.203+0000 7fbf30600700 1 -- 172.21.15.1:0/2196344487 >> [v2:172.21.15.97:6816/2600022531,v1:172.21.15.97:6817/2600022531] conn(0x7fbf100ad990 msgr2=0x7fbf100afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.203+0000 7fbf30600700 1 --2- 172.21.15.1:0/2196344487 >> [v2:172.21.15.97:6816/2600022531,v1:172.21.15.97:6817/2600022531] conn(0x7fbf100ad990 0x7fbf100afe30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.203+0000 7fbf30600700 1 -- 172.21.15.1:0/2196344487 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbf100a3d50 msgr2=0x7fbf100a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.203+0000 7fbf30600700 1 --2- 172.21.15.1:0/2196344487 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbf100a3d50 0x7fbf100a6210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009cb0 tx=0x7fbf1c009260 comp rx=0 tx=0).stop 2024-04-26T01:49:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.203+0000 7fbf30600700 1 -- 172.21.15.1:0/2196344487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf2810f4f0 msgr2=0x7fbf2819fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.203+0000 7fbf30600700 1 --2- 172.21.15.1:0/2196344487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf2810f4f0 0x7fbf2819fc40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000b070 tx=0x7fbf20004c80 comp rx=0 tx=0).stop 2024-04-26T01:49:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.203+0000 7fbf2e600700 1 -- 172.21.15.1:0/2196344487 reap_dead start 2024-04-26T01:49:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.203+0000 7fbf30600700 1 -- 172.21.15.1:0/2196344487 shutdown_connections 2024-04-26T01:49:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.203+0000 7fbf30600700 1 -- 172.21.15.1:0/2196344487 >> 172.21.15.1:0/2196344487 conn(0x7fbf280788f0 msgr2=0x7fbf28110940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.203+0000 7fbf30600700 1 -- 172.21.15.1:0/2196344487 shutdown_connections 2024-04-26T01:49:10.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:09.203+0000 7fbf30600700 1 -- 172.21.15.1:0/2196344487 wait complete. 2024-04-26T01:49:11.006 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:11.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.189+0000 7f7bd9a00700 1 -- 172.21.15.1:0/2244304726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bd4075590 msgr2=0x7f7bd410ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:11.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.189+0000 7f7bd9a00700 1 --2- 172.21.15.1:0/2244304726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bd4075590 0x7f7bd410ef00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009990 tx=0x7f7bc0009ca0 comp rx=0 tx=0).stop 2024-04-26T01:49:11.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.190+0000 7f7bd9a00700 1 -- 172.21.15.1:0/2244304726 shutdown_connections 2024-04-26T01:49:11.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.190+0000 7f7bd9a00700 1 --2- 172.21.15.1:0/2244304726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bd410f440 0x7f7bd4111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:11.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.190+0000 7f7bd9a00700 1 --2- 172.21.15.1:0/2244304726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd4074c70 0x7f7bd4075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:11.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.190+0000 7f7bd9a00700 1 --2- 172.21.15.1:0/2244304726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bd4075590 0x7f7bd410ef00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:11.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.190+0000 7f7bd9a00700 1 -- 172.21.15.1:0/2244304726 >> 172.21.15.1:0/2244304726 conn(0x7f7bd4079ba0 msgr2=0x7f7bd4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:11.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.190+0000 7f7bd9a00700 1 -- 172.21.15.1:0/2244304726 shutdown_connections 2024-04-26T01:49:11.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.190+0000 7f7bd9a00700 1 -- 172.21.15.1:0/2244304726 wait complete. 2024-04-26T01:49:11.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.191+0000 7f7bd9a00700 1 Processor -- start 2024-04-26T01:49:11.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.191+0000 7f7bd9a00700 1 -- start start 2024-04-26T01:49:11.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.191+0000 7f7bd9a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bd4074c70 0x7f7bd419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:11.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.191+0000 7f7bd9a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bd4075590 0x7f7bd419b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:11.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.191+0000 7f7bd9a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd410f440 0x7f7bd419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:11.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.191+0000 7f7bd9a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd4114e50 con 0x7f7bd410f440 2024-04-26T01:49:11.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.191+0000 7f7bd9a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd4114cd0 con 0x7f7bd4074c70 2024-04-26T01:49:11.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.191+0000 7f7bd9a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd4114fd0 con 0x7f7bd4075590 2024-04-26T01:49:11.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.192+0000 7f7bd2a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bd4074c70 0x7f7bd419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:11.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.192+0000 7f7bd2a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bd4074c70 0x7f7bd419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:42526/0 (socket says 172.21.15.1:42526) 2024-04-26T01:49:11.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.192+0000 7f7bd2a00700 1 -- 172.21.15.1:0/652466770 learned_addr learned my addr 172.21.15.1:0/652466770 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:11.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.192+0000 7f7bd2000700 1 --2- 172.21.15.1:0/652466770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bd4075590 0x7f7bd419b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:11.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.192+0000 7f7bd3400700 1 --2- 172.21.15.1:0/652466770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd410f440 0x7f7bd419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:11.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.192+0000 7f7bd2a00700 1 -- 172.21.15.1:0/652466770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bd4075590 msgr2=0x7f7bd419b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:11.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.192+0000 7f7bd2a00700 1 --2- 172.21.15.1:0/652466770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bd4075590 0x7f7bd419b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:11.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.192+0000 7f7bd2a00700 1 -- 172.21.15.1:0/652466770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd410f440 msgr2=0x7f7bd419fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:11.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.192+0000 7f7bd2a00700 1 --2- 172.21.15.1:0/652466770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd410f440 0x7f7bd419fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:11.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.192+0000 7f7bd2a00700 1 -- 172.21.15.1:0/652466770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd41a0340 con 0x7f7bd4074c70 2024-04-26T01:49:11.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.193+0000 7f7bd2a00700 1 --2- 172.21.15.1:0/652466770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bd4074c70 0x7f7bd419b380 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400d950 tx=0x7f7bc400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:11.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.193+0000 7f7bcb400700 1 -- 172.21.15.1:0/652466770 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4016070 con 0x7f7bd4074c70 2024-04-26T01:49:11.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.193+0000 7f7bcb400700 1 -- 172.21.15.1:0/652466770 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc4009df0 con 0x7f7bd4074c70 2024-04-26T01:49:11.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.193+0000 7f7bd9a00700 1 -- 172.21.15.1:0/652466770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd41a0630 con 0x7f7bd4074c70 2024-04-26T01:49:11.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.193+0000 7f7bcb400700 1 -- 172.21.15.1:0/652466770 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc400f3c0 con 0x7f7bd4074c70 2024-04-26T01:49:11.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.193+0000 7f7bd9a00700 1 -- 172.21.15.1:0/652466770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd41a7eb0 con 0x7f7bd4074c70 2024-04-26T01:49:11.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.194+0000 7f7bd9a00700 1 -- 172.21.15.1:0/652466770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7bd4063150 con 0x7f7bd4074c70 2024-04-26T01:49:11.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.196+0000 7f7bcb400700 1 -- 172.21.15.1:0/652466770 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f7bc4019070 con 0x7f7bd4074c70 2024-04-26T01:49:11.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.197+0000 7f7bcb400700 1 --2- 172.21.15.1:0/652466770 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f7bb80a3d50 0x7f7bb80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:11.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.198+0000 7f7bcb400700 1 -- 172.21.15.1:0/652466770 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f7bc40d0610 con 0x7f7bd4074c70 2024-04-26T01:49:11.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.198+0000 7f7bd2000700 1 --2- 172.21.15.1:0/652466770 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f7bb80a3d50 0x7f7bb80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:11.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.198+0000 7f7bcb400700 1 --2- 172.21.15.1:0/652466770 >> [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587] conn(0x7f7bb80ad990 0x7f7bb80afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:11.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.198+0000 7f7bcb400700 1 -- 172.21.15.1:0/652466770 --> [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7bc4013ec0 con 0x7f7bb80ad990 2024-04-26T01:49:11.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.198+0000 7f7bcb400700 1 -- 172.21.15.1:0/652466770 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f7bc40d09c0 con 0x7f7bd4074c70 2024-04-26T01:49:11.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.198+0000 7f7bd3400700 1 --2- 172.21.15.1:0/652466770 >> [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587] conn(0x7f7bb80ad990 0x7f7bb80afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:11.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.198+0000 7f7bd2000700 1 --2- 172.21.15.1:0/652466770 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f7bb80a3d50 0x7f7bb80a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7bd41025d0 tx=0x7f7bc000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:11.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.199+0000 7f7bd3400700 1 --2- 172.21.15.1:0/652466770 >> [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587] conn(0x7f7bb80ad990 0x7f7bb80afe30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:11.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.200+0000 7f7bcb400700 1 -- 172.21.15.1:0/652466770 <== osd.6 v2:172.21.15.1:6818/444030587 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7bc4013ec0 con 0x7f7bb80ad990 2024-04-26T01:49:11.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.218+0000 7f7bd9a00700 1 -- 172.21.15.1:0/652466770 --> [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7bd4063550 con 0x7f7bb80ad990 2024-04-26T01:49:11.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.219+0000 7f7bcb400700 1 -- 172.21.15.1:0/652466770 <== osd.6 v2:172.21.15.1:6818/444030587 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7bd4063550 con 0x7f7bb80ad990 2024-04-26T01:49:11.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.219+0000 7f7bd9a00700 1 -- 172.21.15.1:0/652466770 >> [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587] conn(0x7f7bb80ad990 msgr2=0x7f7bb80afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:11.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.219+0000 7f7bd9a00700 1 --2- 172.21.15.1:0/652466770 >> [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587] conn(0x7f7bb80ad990 0x7f7bb80afe30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:11.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.219+0000 7f7bd9a00700 1 -- 172.21.15.1:0/652466770 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f7bb80a3d50 msgr2=0x7f7bb80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:11.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.219+0000 7f7bd9a00700 1 --2- 172.21.15.1:0/652466770 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f7bb80a3d50 0x7f7bb80a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7bd41025d0 tx=0x7f7bc000b260 comp rx=0 tx=0).stop 2024-04-26T01:49:11.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.219+0000 7f7bd9a00700 1 -- 172.21.15.1:0/652466770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bd4074c70 msgr2=0x7f7bd419b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:11.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.219+0000 7f7bd9a00700 1 --2- 172.21.15.1:0/652466770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bd4074c70 0x7f7bd419b380 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400d950 tx=0x7f7bc400dc60 comp rx=0 tx=0).stop 2024-04-26T01:49:11.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.219+0000 7f7bd3400700 1 -- 172.21.15.1:0/652466770 reap_dead start 2024-04-26T01:49:11.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.220+0000 7f7bd9a00700 1 -- 172.21.15.1:0/652466770 shutdown_connections 2024-04-26T01:49:11.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.220+0000 7f7bd9a00700 1 -- 172.21.15.1:0/652466770 >> 172.21.15.1:0/652466770 conn(0x7f7bd4079ba0 msgr2=0x7f7bd40ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:11.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.220+0000 7f7bd9a00700 1 -- 172.21.15.1:0/652466770 shutdown_connections 2024-04-26T01:49:11.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:10.220+0000 7f7bd9a00700 1 -- 172.21.15.1:0/652466770 wait complete. 2024-04-26T01:49:11.561 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:11.563 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:12.188 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:11 smithi001 ceph-mon[27822]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-26T01:49:12.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:11 smithi097 ceph-mon[32853]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-26T01:49:12.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:11 smithi173 ceph-mon[32553]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-26T01:49:13.101 INFO:teuthology.orchestra.run.smithi001.stdout:261993005111 2024-04-26T01:49:13.101 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-26T01:49:14.058 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:13 smithi001 ceph-mon[27822]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:14.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:13 smithi173 ceph-mon[32553]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:14.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:13 smithi097 ceph-mon[32853]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:14.687 INFO:teuthology.orchestra.run.smithi001.stdout:223338299450 2024-04-26T01:49:14.687 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-26T01:49:15.395 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:15.402 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:16.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:15 smithi001 ceph-mon[27822]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:16.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:15 smithi173 ceph-mon[32553]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:15 smithi097 ceph-mon[32853]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:18.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:17 smithi001 ceph-mon[27822]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.759+0000 7fb1f7e00700 1 -- 172.21.15.1:0/2787568159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1f810f030 msgr2=0x7fb1f810f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.759+0000 7fb1f7e00700 1 --2- 172.21.15.1:0/2787568159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1f810f030 0x7fb1f810f410 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8009990 tx=0x7fb1e8009ca0 comp rx=0 tx=0).stop 2024-04-26T01:49:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.759+0000 7fb1f7e00700 1 -- 172.21.15.1:0/2787568159 shutdown_connections 2024-04-26T01:49:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.759+0000 7fb1f7e00700 1 --2- 172.21.15.1:0/2787568159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1f8109940 0x7fb1f8115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.759+0000 7fb1f7e00700 1 --2- 172.21.15.1:0/2787568159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1f810f030 0x7fb1f810f410 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.759+0000 7fb1f7e00700 1 --2- 172.21.15.1:0/2787568159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1f8108f80 0x7fb1f8109400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.759+0000 7fb1f7e00700 1 -- 172.21.15.1:0/2787568159 >> 172.21.15.1:0/2787568159 conn(0x7fb1f80787d0 msgr2=0x7fb1f8107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.759+0000 7fb1f7e00700 1 -- 172.21.15.1:0/2787568159 shutdown_connections 2024-04-26T01:49:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.759+0000 7fb1f7e00700 1 -- 172.21.15.1:0/2787568159 wait complete. 2024-04-26T01:49:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.761+0000 7fb1f7e00700 1 Processor -- start 2024-04-26T01:49:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.761+0000 7fb1f7e00700 1 -- start start 2024-04-26T01:49:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.761+0000 7fb1f7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1f8108f80 0x7fb1f819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.761+0000 7fb1f7e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1f8109940 0x7fb1f819faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.761+0000 7fb1f7e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1f810f030 0x7fb1f81a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.761+0000 7fb1f7e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f8119140 con 0x7fb1f8108f80 2024-04-26T01:49:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.761+0000 7fb1f7e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f8118fc0 con 0x7fb1f810f030 2024-04-26T01:49:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.761+0000 7fb1f7e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f81192c0 con 0x7fb1f8109940 2024-04-26T01:49:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.762+0000 7fb1f6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1f8108f80 0x7fb1f819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.762+0000 7fb1f6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1f8108f80 0x7fb1f819f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59462/0 (socket says 172.21.15.1:59462) 2024-04-26T01:49:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.762+0000 7fb1f6a00700 1 -- 172.21.15.1:0/3505924589 learned_addr learned my addr 172.21.15.1:0/3505924589 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.762+0000 7fb1f7400700 1 --2- 172.21.15.1:0/3505924589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1f810f030 0x7fb1f81a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.762+0000 7fb1f6000700 1 --2- 172.21.15.1:0/3505924589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1f8109940 0x7fb1f819faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.762+0000 7fb1f6a00700 1 -- 172.21.15.1:0/3505924589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1f8109940 msgr2=0x7fb1f819faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.762+0000 7fb1f6a00700 1 --2- 172.21.15.1:0/3505924589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1f8109940 0x7fb1f819faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.762+0000 7fb1f6a00700 1 -- 172.21.15.1:0/3505924589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1f810f030 msgr2=0x7fb1f81a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.762+0000 7fb1f6a00700 1 --2- 172.21.15.1:0/3505924589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1f810f030 0x7fb1f81a3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.762+0000 7fb1f6a00700 1 -- 172.21.15.1:0/3505924589 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f81a4570 con 0x7fb1f8108f80 2024-04-26T01:49:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.762+0000 7fb1f6a00700 1 --2- 172.21.15.1:0/3505924589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1f8108f80 0x7fb1f819f5b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8005910 tx=0x7fb1e800b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.763+0000 7fb1ef400700 1 -- 172.21.15.1:0/3505924589 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8027070 con 0x7fb1f8108f80 2024-04-26T01:49:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.763+0000 7fb1f7e00700 1 -- 172.21.15.1:0/3505924589 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f81a4800 con 0x7fb1f8108f80 2024-04-26T01:49:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.763+0000 7fb1ef400700 1 -- 172.21.15.1:0/3505924589 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e8022470 con 0x7fb1f8108f80 2024-04-26T01:49:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.763+0000 7fb1ef400700 1 -- 172.21.15.1:0/3505924589 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8017650 con 0x7fb1f8108f80 2024-04-26T01:49:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.763+0000 7fb1f7e00700 1 -- 172.21.15.1:0/3505924589 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f81ac0e0 con 0x7fb1f8108f80 2024-04-26T01:49:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.764+0000 7fb1f7e00700 1 -- 172.21.15.1:0/3505924589 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb1f80631d0 con 0x7fb1f8108f80 2024-04-26T01:49:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.766+0000 7fb1ef400700 1 -- 172.21.15.1:0/3505924589 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fb1e802a070 con 0x7fb1f8108f80 2024-04-26T01:49:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.767+0000 7fb1ef400700 1 --2- 172.21.15.1:0/3505924589 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb1dc0a3cd0 0x7fb1dc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.768+0000 7fb1ef400700 1 -- 172.21.15.1:0/3505924589 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fb1e80d1770 con 0x7fb1f8108f80 2024-04-26T01:49:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.768+0000 7fb1f6000700 1 --2- 172.21.15.1:0/3505924589 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb1dc0a3cd0 0x7fb1dc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.768+0000 7fb1ef400700 1 --2- 172.21.15.1:0/3505924589 >> [v2:172.21.15.97:6800/3755022621,v1:172.21.15.97:6801/3755022621] conn(0x7fb1dc0ad910 0x7fb1dc0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.768+0000 7fb1ef400700 1 -- 172.21.15.1:0/3505924589 --> [v2:172.21.15.97:6800/3755022621,v1:172.21.15.97:6801/3755022621] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb1e8015ec0 con 0x7fb1dc0ad910 2024-04-26T01:49:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.768+0000 7fb1ef400700 1 -- 172.21.15.1:0/3505924589 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fb1e80d1b20 con 0x7fb1f8108f80 2024-04-26T01:49:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.768+0000 7fb1f7400700 1 --2- 172.21.15.1:0/3505924589 >> [v2:172.21.15.97:6800/3755022621,v1:172.21.15.97:6801/3755022621] conn(0x7fb1dc0ad910 0x7fb1dc0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.768+0000 7fb1f6000700 1 --2- 172.21.15.1:0/3505924589 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb1dc0a3cd0 0x7fb1dc0a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb1f810e6d0 tx=0x7fb1e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.769+0000 7fb1f7400700 1 --2- 172.21.15.1:0/3505924589 >> [v2:172.21.15.97:6800/3755022621,v1:172.21.15.97:6801/3755022621] conn(0x7fb1dc0ad910 0x7fb1dc0afdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.770+0000 7fb1ef400700 1 -- 172.21.15.1:0/3505924589 <== osd.1 v2:172.21.15.97:6800/3755022621 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb1e8015ec0 con 0x7fb1dc0ad910 2024-04-26T01:49:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.791+0000 7fb1f7e00700 1 -- 172.21.15.1:0/3505924589 --> [v2:172.21.15.97:6800/3755022621,v1:172.21.15.97:6801/3755022621] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb1f80635d0 con 0x7fb1dc0ad910 2024-04-26T01:49:18.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.791+0000 7fb1ef400700 1 -- 172.21.15.1:0/3505924589 <== osd.1 v2:172.21.15.97:6800/3755022621 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb1f80635d0 con 0x7fb1dc0ad910 2024-04-26T01:49:18.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.792+0000 7fb1f7e00700 1 -- 172.21.15.1:0/3505924589 >> [v2:172.21.15.97:6800/3755022621,v1:172.21.15.97:6801/3755022621] conn(0x7fb1dc0ad910 msgr2=0x7fb1dc0afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:18.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.792+0000 7fb1f7e00700 1 --2- 172.21.15.1:0/3505924589 >> [v2:172.21.15.97:6800/3755022621,v1:172.21.15.97:6801/3755022621] conn(0x7fb1dc0ad910 0x7fb1dc0afdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:18.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.792+0000 7fb1f7e00700 1 -- 172.21.15.1:0/3505924589 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb1dc0a3cd0 msgr2=0x7fb1dc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:18.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.792+0000 7fb1f7e00700 1 --2- 172.21.15.1:0/3505924589 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb1dc0a3cd0 0x7fb1dc0a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb1f810e6d0 tx=0x7fb1e4009260 comp rx=0 tx=0).stop 2024-04-26T01:49:18.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.792+0000 7fb1f7e00700 1 -- 172.21.15.1:0/3505924589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1f8108f80 msgr2=0x7fb1f819f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:18.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.792+0000 7fb1f7e00700 1 --2- 172.21.15.1:0/3505924589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1f8108f80 0x7fb1f819f5b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8005910 tx=0x7fb1e800b670 comp rx=0 tx=0).stop 2024-04-26T01:49:18.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.792+0000 7fb1f7400700 1 -- 172.21.15.1:0/3505924589 reap_dead start 2024-04-26T01:49:18.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.792+0000 7fb1f7e00700 1 -- 172.21.15.1:0/3505924589 shutdown_connections 2024-04-26T01:49:18.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.792+0000 7fb1f7e00700 1 -- 172.21.15.1:0/3505924589 >> 172.21.15.1:0/3505924589 conn(0x7fb1f80787d0 msgr2=0x7fb1f8079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:18.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.793+0000 7fb1f7e00700 1 -- 172.21.15.1:0/3505924589 shutdown_connections 2024-04-26T01:49:18.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:17.793+0000 7fb1f7e00700 1 -- 172.21.15.1:0/3505924589 wait complete. 2024-04-26T01:49:18.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:17 smithi173 ceph-mon[32553]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:18.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:17 smithi097 ceph-mon[32853]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:19.146 INFO:teuthology.orchestra.run.smithi001.stdout:201863462976 2024-04-26T01:49:19.147 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-26T01:49:19.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.196+0000 7fd014c00700 1 -- 172.21.15.1:0/2695559621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd010075590 msgr2=0x7fd0101136d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:19.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.196+0000 7fd014c00700 1 --2- 172.21.15.1:0/2695559621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd010075590 0x7fd0101136d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcff8009990 tx=0x7fcff8009ca0 comp rx=0 tx=0).stop 2024-04-26T01:49:19.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.196+0000 7fd014c00700 1 -- 172.21.15.1:0/2695559621 shutdown_connections 2024-04-26T01:49:19.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.196+0000 7fd014c00700 1 --2- 172.21.15.1:0/2695559621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd010113c10 0x7fd010116000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:19.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.196+0000 7fd014c00700 1 --2- 172.21.15.1:0/2695559621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd010074c70 0x7fd010075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:19.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.196+0000 7fd014c00700 1 --2- 172.21.15.1:0/2695559621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd010075590 0x7fd0101136d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:19.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.196+0000 7fd014c00700 1 -- 172.21.15.1:0/2695559621 >> 172.21.15.1:0/2695559621 conn(0x7fd010079bf0 msgr2=0x7fd01007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:19.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.197+0000 7fd014c00700 1 -- 172.21.15.1:0/2695559621 shutdown_connections 2024-04-26T01:49:19.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.197+0000 7fd014c00700 1 -- 172.21.15.1:0/2695559621 wait complete. 2024-04-26T01:49:19.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.198+0000 7fd014c00700 1 Processor -- start 2024-04-26T01:49:19.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.198+0000 7fd014c00700 1 -- start start 2024-04-26T01:49:19.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.198+0000 7fd014c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd010074c70 0x7fd0101a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:19.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.198+0000 7fd014c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd010075590 0x7fd0101a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:19.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.198+0000 7fd014c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd010113c10 0x7fd01019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:19.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.198+0000 7fd014c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd010119260 con 0x7fd010075590 2024-04-26T01:49:19.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.198+0000 7fd014c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd0101190e0 con 0x7fd010074c70 2024-04-26T01:49:19.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.198+0000 7fd014c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd0101193e0 con 0x7fd010113c10 2024-04-26T01:49:19.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.199+0000 7fd007e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd010075590 0x7fd0101a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:19.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.199+0000 7fd007e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd010075590 0x7fd0101a5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59436/0 (socket says 172.21.15.1:59436) 2024-04-26T01:49:19.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.199+0000 7fd007e00700 1 -- 172.21.15.1:0/3457380195 learned_addr learned my addr 172.21.15.1:0/3457380195 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:19.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.199+0000 7fd00e800700 1 --2- 172.21.15.1:0/3457380195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd010113c10 0x7fd01019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:19.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.199+0000 7fd00de00700 1 --2- 172.21.15.1:0/3457380195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd010074c70 0x7fd0101a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:19.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.199+0000 7fd00e800700 1 -- 172.21.15.1:0/3457380195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd010074c70 msgr2=0x7fd0101a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:19.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.199+0000 7fd00e800700 1 --2- 172.21.15.1:0/3457380195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd010074c70 0x7fd0101a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:19.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.199+0000 7fd00e800700 1 -- 172.21.15.1:0/3457380195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd010075590 msgr2=0x7fd0101a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:19.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.199+0000 7fd00e800700 1 --2- 172.21.15.1:0/3457380195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd010075590 0x7fd0101a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:19.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.200+0000 7fd00e800700 1 -- 172.21.15.1:0/3457380195 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01019fd00 con 0x7fd010113c10 2024-04-26T01:49:19.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.200+0000 7fd00e800700 1 --2- 172.21.15.1:0/3457380195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd010113c10 0x7fd01019f5a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd00800efc0 tx=0x7fd00800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:19.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.200+0000 7fd006a00700 1 -- 172.21.15.1:0/3457380195 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0080092e0 con 0x7fd010113c10 2024-04-26T01:49:19.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.200+0000 7fd006a00700 1 -- 172.21.15.1:0/3457380195 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd008004bb0 con 0x7fd010113c10 2024-04-26T01:49:19.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.200+0000 7fd014c00700 1 -- 172.21.15.1:0/3457380195 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd01019ff90 con 0x7fd010113c10 2024-04-26T01:49:19.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.200+0000 7fd006a00700 1 -- 172.21.15.1:0/3457380195 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008004020 con 0x7fd010113c10 2024-04-26T01:49:19.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.200+0000 7fd014c00700 1 -- 172.21.15.1:0/3457380195 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0101ac2b0 con 0x7fd010113c10 2024-04-26T01:49:19.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.201+0000 7fd014c00700 1 -- 172.21.15.1:0/3457380195 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd010110110 con 0x7fd010113c10 2024-04-26T01:49:19.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.203+0000 7fd006a00700 1 -- 172.21.15.1:0/3457380195 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fd00801b030 con 0x7fd010113c10 2024-04-26T01:49:19.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.205+0000 7fd006a00700 1 --2- 172.21.15.1:0/3457380195 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fcff40a3d50 0x7fcff40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:19.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.205+0000 7fd006a00700 1 -- 172.21.15.1:0/3457380195 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fd0080d0080 con 0x7fd010113c10 2024-04-26T01:49:19.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.205+0000 7fd00de00700 1 --2- 172.21.15.1:0/3457380195 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fcff40a3d50 0x7fcff40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:19.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.205+0000 7fd006a00700 1 --2- 172.21.15.1:0/3457380195 >> [v2:172.21.15.173:6800/734861771,v1:172.21.15.173:6801/734861771] conn(0x7fcff40ad990 0x7fcff40afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:19.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.205+0000 7fd006a00700 1 -- 172.21.15.1:0/3457380195 --> [v2:172.21.15.173:6800/734861771,v1:172.21.15.173:6801/734861771] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcff40b0500 con 0x7fcff40ad990 2024-04-26T01:49:19.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.205+0000 7fd006a00700 1 -- 172.21.15.1:0/3457380195 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fd0080d04d0 con 0x7fd010113c10 2024-04-26T01:49:19.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.206+0000 7fd007e00700 1 --2- 172.21.15.1:0/3457380195 >> [v2:172.21.15.173:6800/734861771,v1:172.21.15.173:6801/734861771] conn(0x7fcff40ad990 0x7fcff40afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:19.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.206+0000 7fd00de00700 1 --2- 172.21.15.1:0/3457380195 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fcff40a3d50 0x7fcff40a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd000009c40 tx=0x7fd000009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:19.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.206+0000 7fd007e00700 1 --2- 172.21.15.1:0/3457380195 >> [v2:172.21.15.173:6800/734861771,v1:172.21.15.173:6801/734861771] conn(0x7fcff40ad990 0x7fcff40afe30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:19.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.207+0000 7fd006a00700 1 -- 172.21.15.1:0/3457380195 <== osd.2 v2:172.21.15.173:6800/734861771 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcff40b0500 con 0x7fcff40ad990 2024-04-26T01:49:19.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.225+0000 7fd014c00700 1 -- 172.21.15.1:0/3457380195 --> [v2:172.21.15.173:6800/734861771,v1:172.21.15.173:6801/734861771] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd010076390 con 0x7fcff40ad990 2024-04-26T01:49:19.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.226+0000 7fd006a00700 1 -- 172.21.15.1:0/3457380195 <== osd.2 v2:172.21.15.173:6800/734861771 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd010076390 con 0x7fcff40ad990 2024-04-26T01:49:19.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.227+0000 7fd014c00700 1 -- 172.21.15.1:0/3457380195 >> [v2:172.21.15.173:6800/734861771,v1:172.21.15.173:6801/734861771] conn(0x7fcff40ad990 msgr2=0x7fcff40afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:19.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.227+0000 7fd014c00700 1 --2- 172.21.15.1:0/3457380195 >> [v2:172.21.15.173:6800/734861771,v1:172.21.15.173:6801/734861771] conn(0x7fcff40ad990 0x7fcff40afe30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:19.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.227+0000 7fd014c00700 1 -- 172.21.15.1:0/3457380195 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fcff40a3d50 msgr2=0x7fcff40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:19.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.227+0000 7fd014c00700 1 --2- 172.21.15.1:0/3457380195 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fcff40a3d50 0x7fcff40a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd000009c40 tx=0x7fd000009260 comp rx=0 tx=0).stop 2024-04-26T01:49:19.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.227+0000 7fd014c00700 1 -- 172.21.15.1:0/3457380195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd010113c10 msgr2=0x7fd01019f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:19.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.227+0000 7fd014c00700 1 --2- 172.21.15.1:0/3457380195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd010113c10 0x7fd01019f5a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd00800efc0 tx=0x7fd00800c3a0 comp rx=0 tx=0).stop 2024-04-26T01:49:19.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.227+0000 7fd00e800700 1 -- 172.21.15.1:0/3457380195 reap_dead start 2024-04-26T01:49:19.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.227+0000 7fd014c00700 1 -- 172.21.15.1:0/3457380195 shutdown_connections 2024-04-26T01:49:19.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.227+0000 7fd014c00700 1 -- 172.21.15.1:0/3457380195 >> 172.21.15.1:0/3457380195 conn(0x7fd010079bf0 msgr2=0x7fd01010f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:19.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.227+0000 7fd014c00700 1 -- 172.21.15.1:0/3457380195 shutdown_connections 2024-04-26T01:49:19.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:16.228+0000 7fd014c00700 1 -- 172.21.15.1:0/3457380195 wait complete. 2024-04-26T01:49:19.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.770+0000 7fe3aa600700 1 -- 172.21.15.1:0/3641818841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3a410f440 msgr2=0x7fe3a4111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.770+0000 7fe3aa600700 1 --2- 172.21.15.1:0/3641818841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3a410f440 0x7fe3a4111830 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00b0a0 tx=0x7fe39c00b3b0 comp rx=0 tx=0).stop 2024-04-26T01:49:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.770+0000 7fe3aa600700 1 -- 172.21.15.1:0/3641818841 shutdown_connections 2024-04-26T01:49:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.770+0000 7fe3aa600700 1 --2- 172.21.15.1:0/3641818841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3a410f440 0x7fe3a4111830 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.770+0000 7fe3aa600700 1 --2- 172.21.15.1:0/3641818841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe3a4074c70 0x7fe3a4075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.770+0000 7fe3aa600700 1 --2- 172.21.15.1:0/3641818841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3a4075590 0x7fe3a410ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.770+0000 7fe3aa600700 1 -- 172.21.15.1:0/3641818841 >> 172.21.15.1:0/3641818841 conn(0x7fe3a4079ba0 msgr2=0x7fe3a4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.770+0000 7fe3aa600700 1 -- 172.21.15.1:0/3641818841 shutdown_connections 2024-04-26T01:49:19.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.770+0000 7fe3aa600700 1 -- 172.21.15.1:0/3641818841 wait complete. 2024-04-26T01:49:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.771+0000 7fe3aa600700 1 Processor -- start 2024-04-26T01:49:19.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.771+0000 7fe3aa600700 1 -- start start 2024-04-26T01:49:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.772+0000 7fe3aa600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3a4074c70 0x7fe3a419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.772+0000 7fe3aa600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3a4075590 0x7fe3a419b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.772+0000 7fe3aa600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe3a410f440 0x7fe3a419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.772+0000 7fe3aa600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a4114e20 con 0x7fe3a410f440 2024-04-26T01:49:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.772+0000 7fe3aa600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a4114ca0 con 0x7fe3a4074c70 2024-04-26T01:49:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.772+0000 7fe3aa600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a4114fa0 con 0x7fe3a4075590 2024-04-26T01:49:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.772+0000 7fe3a3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe3a410f440 0x7fe3a419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.772+0000 7fe3a3400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3a4074c70 0x7fe3a419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.772+0000 7fe3a3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe3a410f440 0x7fe3a419fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59482/0 (socket says 172.21.15.1:59482) 2024-04-26T01:49:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.772+0000 7fe39be00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3a4075590 0x7fe3a419b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.772+0000 7fe3a3e00700 1 -- 172.21.15.1:0/2327119353 learned_addr learned my addr 172.21.15.1:0/2327119353 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.773+0000 7fe39be00700 1 -- 172.21.15.1:0/2327119353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3a4074c70 msgr2=0x7fe3a419b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.773+0000 7fe39be00700 1 --2- 172.21.15.1:0/2327119353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3a4074c70 0x7fe3a419b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:19.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.773+0000 7fe39be00700 1 -- 172.21.15.1:0/2327119353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe3a410f440 msgr2=0x7fe3a419fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:19.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.773+0000 7fe39be00700 1 --2- 172.21.15.1:0/2327119353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe3a410f440 0x7fe3a419fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:19.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.773+0000 7fe39be00700 1 -- 172.21.15.1:0/2327119353 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a41a02a0 con 0x7fe3a4075590 2024-04-26T01:49:19.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.773+0000 7fe3a3e00700 1 --2- 172.21.15.1:0/2327119353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe3a410f440 0x7fe3a419fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:49:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.773+0000 7fe39be00700 1 --2- 172.21.15.1:0/2327119353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3a4075590 0x7fe3a419b7d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00e9c0 tx=0x7fe38c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.773+0000 7fe3a0c00700 1 -- 172.21.15.1:0/2327119353 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c004220 con 0x7fe3a4075590 2024-04-26T01:49:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.773+0000 7fe3a0c00700 1 -- 172.21.15.1:0/2327119353 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe38c0043c0 con 0x7fe3a4075590 2024-04-26T01:49:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.773+0000 7fe3a0c00700 1 -- 172.21.15.1:0/2327119353 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c003c50 con 0x7fe3a4075590 2024-04-26T01:49:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.773+0000 7fe3aa600700 1 -- 172.21.15.1:0/2327119353 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a41a0590 con 0x7fe3a4075590 2024-04-26T01:49:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.774+0000 7fe3aa600700 1 -- 172.21.15.1:0/2327119353 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a41a7e10 con 0x7fe3a4075590 2024-04-26T01:49:19.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.775+0000 7fe3aa600700 1 -- 172.21.15.1:0/2327119353 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe3a40631d0 con 0x7fe3a4075590 2024-04-26T01:49:19.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.777+0000 7fe3a0c00700 1 -- 172.21.15.1:0/2327119353 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fe38c00c7c0 con 0x7fe3a4075590 2024-04-26T01:49:19.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.778+0000 7fe3a0c00700 1 --2- 172.21.15.1:0/2327119353 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe3880a3cd0 0x7fe3880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:19.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.778+0000 7fe3a0c00700 1 -- 172.21.15.1:0/2327119353 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fe38c0d09e0 con 0x7fe3a4075590 2024-04-26T01:49:19.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.779+0000 7fe3a3400700 1 --2- 172.21.15.1:0/2327119353 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe3880a3cd0 0x7fe3880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:19.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.779+0000 7fe3a0c00700 1 --2- 172.21.15.1:0/2327119353 >> [v2:172.21.15.97:6808/2494639397,v1:172.21.15.97:6809/2494639397] conn(0x7fe3880ad910 0x7fe3880afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:19.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.779+0000 7fe3a0c00700 1 -- 172.21.15.1:0/2327119353 --> [v2:172.21.15.97:6808/2494639397,v1:172.21.15.97:6809/2494639397] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe38c022e50 con 0x7fe3880ad910 2024-04-26T01:49:19.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.779+0000 7fe3a0c00700 1 -- 172.21.15.1:0/2327119353 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fe38c0d0d90 con 0x7fe3a4075590 2024-04-26T01:49:19.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.779+0000 7fe3a3400700 1 --2- 172.21.15.1:0/2327119353 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe3880a3cd0 0x7fe3880a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe394009c60 tx=0x7fe394009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:19.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.779+0000 7fe3a3e00700 1 --2- 172.21.15.1:0/2327119353 >> [v2:172.21.15.97:6808/2494639397,v1:172.21.15.97:6809/2494639397] conn(0x7fe3880ad910 0x7fe3880afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:19.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.780+0000 7fe3a3e00700 1 --2- 172.21.15.1:0/2327119353 >> [v2:172.21.15.97:6808/2494639397,v1:172.21.15.97:6809/2494639397] conn(0x7fe3880ad910 0x7fe3880afdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:19.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.781+0000 7fe3a0c00700 1 -- 172.21.15.1:0/2327119353 <== osd.4 v2:172.21.15.97:6808/2494639397 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe38c022e50 con 0x7fe3880ad910 2024-04-26T01:49:19.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.799+0000 7fe3aa600700 1 -- 172.21.15.1:0/2327119353 --> [v2:172.21.15.97:6808/2494639397,v1:172.21.15.97:6809/2494639397] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe3a40635d0 con 0x7fe3880ad910 2024-04-26T01:49:19.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.800+0000 7fe3a0c00700 1 -- 172.21.15.1:0/2327119353 <== osd.4 v2:172.21.15.97:6808/2494639397 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe3a40635d0 con 0x7fe3880ad910 2024-04-26T01:49:19.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.801+0000 7fe3aa600700 1 -- 172.21.15.1:0/2327119353 >> [v2:172.21.15.97:6808/2494639397,v1:172.21.15.97:6809/2494639397] conn(0x7fe3880ad910 msgr2=0x7fe3880afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:19.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.801+0000 7fe3aa600700 1 --2- 172.21.15.1:0/2327119353 >> [v2:172.21.15.97:6808/2494639397,v1:172.21.15.97:6809/2494639397] conn(0x7fe3880ad910 0x7fe3880afdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:19.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.801+0000 7fe3aa600700 1 -- 172.21.15.1:0/2327119353 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe3880a3cd0 msgr2=0x7fe3880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:19.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.801+0000 7fe3aa600700 1 --2- 172.21.15.1:0/2327119353 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe3880a3cd0 0x7fe3880a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe394009c60 tx=0x7fe394009260 comp rx=0 tx=0).stop 2024-04-26T01:49:19.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.801+0000 7fe3aa600700 1 -- 172.21.15.1:0/2327119353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3a4075590 msgr2=0x7fe3a419b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:19.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.801+0000 7fe3aa600700 1 --2- 172.21.15.1:0/2327119353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3a4075590 0x7fe3a419b7d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00e9c0 tx=0x7fe38c00ecd0 comp rx=0 tx=0).stop 2024-04-26T01:49:19.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.801+0000 7fe3a3e00700 1 -- 172.21.15.1:0/2327119353 reap_dead start 2024-04-26T01:49:19.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.801+0000 7fe3aa600700 1 -- 172.21.15.1:0/2327119353 shutdown_connections 2024-04-26T01:49:19.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.801+0000 7fe3aa600700 1 -- 172.21.15.1:0/2327119353 >> 172.21.15.1:0/2327119353 conn(0x7fe3a4079ba0 msgr2=0x7fe3a40ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:19.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.801+0000 7fe3aa600700 1 -- 172.21.15.1:0/2327119353 shutdown_connections 2024-04-26T01:49:19.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:19.801+0000 7fe3aa600700 1 -- 172.21.15.1:0/2327119353 wait complete. 2024-04-26T01:49:20.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:20 smithi001 ceph-mon[27822]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:20.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:20 smithi173 ceph-mon[32553]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:20.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:20 smithi097 ceph-mon[32853]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:21.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.818+0000 7f2d26e00700 1 -- 172.21.15.1:0/3168047967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d20104db0 msgr2=0x7f2d20105190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.819+0000 7f2d26e00700 1 --2- 172.21.15.1:0/3168047967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d20104db0 0x7f2d20105190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2d140098f0 tx=0x7f2d14009c00 comp rx=0 tx=0).stop 2024-04-26T01:49:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.819+0000 7f2d26e00700 1 -- 172.21.15.1:0/3168047967 shutdown_connections 2024-04-26T01:49:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.819+0000 7f2d26e00700 1 --2- 172.21.15.1:0/3168047967 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d20109e90 0x7f2d20111a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.819+0000 7f2d26e00700 1 --2- 172.21.15.1:0/3168047967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d20104db0 0x7f2d20105190 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.819+0000 7f2d26e00700 1 --2- 172.21.15.1:0/3168047967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d20105760 0x7f2d201097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.819+0000 7f2d26e00700 1 -- 172.21.15.1:0/3168047967 >> 172.21.15.1:0/3168047967 conn(0x7f2d20100620 msgr2=0x7f2d20102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.820+0000 7f2d26e00700 1 -- 172.21.15.1:0/3168047967 shutdown_connections 2024-04-26T01:49:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.820+0000 7f2d26e00700 1 -- 172.21.15.1:0/3168047967 wait complete. 2024-04-26T01:49:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.821+0000 7f2d26e00700 1 Processor -- start 2024-04-26T01:49:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.822+0000 7f2d26e00700 1 -- start start 2024-04-26T01:49:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.822+0000 7f2d26e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d20104db0 0x7f2d2019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.822+0000 7f2d26e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d20105760 0x7f2d2019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.822+0000 7f2d26e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d20109e90 0x7f2d2019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.822+0000 7f2d26e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2d20114dc0 con 0x7f2d20105760 2024-04-26T01:49:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.822+0000 7f2d26e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2d20114c40 con 0x7f2d20109e90 2024-04-26T01:49:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.822+0000 7f2d26e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2d20114f40 con 0x7f2d20104db0 2024-04-26T01:49:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.823+0000 7f2d25a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d20104db0 0x7f2d2019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.823+0000 7f2d25a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d20104db0 0x7f2d2019b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:53850/0 (socket says 172.21.15.1:53850) 2024-04-26T01:49:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.823+0000 7f2d25a00700 1 -- 172.21.15.1:0/3221816383 learned_addr learned my addr 172.21.15.1:0/3221816383 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.823+0000 7f2d25a00700 1 -- 172.21.15.1:0/3221816383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d20109e90 msgr2=0x7f2d2019fb70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T01:49:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.823+0000 7f2d25a00700 1 --2- 172.21.15.1:0/3221816383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d20109e90 0x7f2d2019fb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.823+0000 7f2d25a00700 1 -- 172.21.15.1:0/3221816383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d20105760 msgr2=0x7f2d2019b830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:49:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.824+0000 7f2d25000700 1 --2- 172.21.15.1:0/3221816383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d20105760 0x7f2d2019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.824+0000 7f2d25a00700 1 --2- 172.21.15.1:0/3221816383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d20105760 0x7f2d2019b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.824+0000 7f2d25a00700 1 -- 172.21.15.1:0/3221816383 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d201a02b0 con 0x7f2d20104db0 2024-04-26T01:49:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.824+0000 7f2d25000700 1 --2- 172.21.15.1:0/3221816383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d20105760 0x7f2d2019b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:49:21.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.824+0000 7f2d25a00700 1 --2- 172.21.15.1:0/3221816383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d20104db0 0x7f2d2019b2f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009230 tx=0x7f2d1400fb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.825+0000 7f2d1e000700 1 -- 172.21.15.1:0/3221816383 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14005170 con 0x7f2d20104db0 2024-04-26T01:49:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.825+0000 7f2d1e000700 1 -- 172.21.15.1:0/3221816383 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d1400b560 con 0x7f2d20104db0 2024-04-26T01:49:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.825+0000 7f2d26e00700 1 -- 172.21.15.1:0/3221816383 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d201a0540 con 0x7f2d20104db0 2024-04-26T01:49:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.825+0000 7f2d1e000700 1 -- 172.21.15.1:0/3221816383 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14003b60 con 0x7f2d20104db0 2024-04-26T01:49:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.825+0000 7f2d26e00700 1 -- 172.21.15.1:0/3221816383 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d20076e80 con 0x7f2d20104db0 2024-04-26T01:49:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.827+0000 7f2d26e00700 1 -- 172.21.15.1:0/3221816383 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d200631d0 con 0x7f2d20104db0 2024-04-26T01:49:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.830+0000 7f2d1e000700 1 -- 172.21.15.1:0/3221816383 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f2d14003d00 con 0x7f2d20104db0 2024-04-26T01:49:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.833+0000 7f2d1e000700 1 --2- 172.21.15.1:0/3221816383 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2d080a3d50 0x7f2d080a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.833+0000 7f2d1e000700 1 -- 172.21.15.1:0/3221816383 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f2d140d17f0 con 0x7f2d20104db0 2024-04-26T01:49:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.833+0000 7f2d25000700 1 --2- 172.21.15.1:0/3221816383 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2d080a3d50 0x7f2d080a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.833+0000 7f2d1e000700 1 --2- 172.21.15.1:0/3221816383 >> [v2:172.21.15.97:6824/1693425268,v1:172.21.15.97:6825/1693425268] conn(0x7f2d080ad990 0x7f2d080afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.833+0000 7f2d1e000700 1 -- 172.21.15.1:0/3221816383 --> [v2:172.21.15.97:6824/1693425268,v1:172.21.15.97:6825/1693425268] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d080b0500 con 0x7f2d080ad990 2024-04-26T01:49:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.833+0000 7f2d1e000700 1 -- 172.21.15.1:0/3221816383 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f2d140d1c40 con 0x7f2d20104db0 2024-04-26T01:49:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.834+0000 7f2d26400700 1 --2- 172.21.15.1:0/3221816383 >> [v2:172.21.15.97:6824/1693425268,v1:172.21.15.97:6825/1693425268] conn(0x7f2d080ad990 0x7f2d080afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.834+0000 7f2d25000700 1 --2- 172.21.15.1:0/3221816383 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2d080a3d50 0x7f2d080a6210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2d20100b40 tx=0x7f2d0c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:21.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.834+0000 7f2d26400700 1 --2- 172.21.15.1:0/3221816383 >> [v2:172.21.15.97:6824/1693425268,v1:172.21.15.97:6825/1693425268] conn(0x7f2d080ad990 0x7f2d080afe30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:21.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.835+0000 7f2d1e000700 1 -- 172.21.15.1:0/3221816383 <== osd.10 v2:172.21.15.97:6824/1693425268 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d080b0500 con 0x7f2d080ad990 2024-04-26T01:49:21.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.854+0000 7f2d26e00700 1 -- 172.21.15.1:0/3221816383 --> [v2:172.21.15.97:6824/1693425268,v1:172.21.15.97:6825/1693425268] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2d200635d0 con 0x7f2d080ad990 2024-04-26T01:49:21.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.855+0000 7f2d1e000700 1 -- 172.21.15.1:0/3221816383 <== osd.10 v2:172.21.15.97:6824/1693425268 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2d200635d0 con 0x7f2d080ad990 2024-04-26T01:49:21.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.855+0000 7f2d26e00700 1 -- 172.21.15.1:0/3221816383 >> [v2:172.21.15.97:6824/1693425268,v1:172.21.15.97:6825/1693425268] conn(0x7f2d080ad990 msgr2=0x7f2d080afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:21.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.855+0000 7f2d26e00700 1 --2- 172.21.15.1:0/3221816383 >> [v2:172.21.15.97:6824/1693425268,v1:172.21.15.97:6825/1693425268] conn(0x7f2d080ad990 0x7f2d080afe30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:21.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.855+0000 7f2d26e00700 1 -- 172.21.15.1:0/3221816383 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2d080a3d50 msgr2=0x7f2d080a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:21.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.855+0000 7f2d26e00700 1 --2- 172.21.15.1:0/3221816383 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2d080a3d50 0x7f2d080a6210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2d20100b40 tx=0x7f2d0c009260 comp rx=0 tx=0).stop 2024-04-26T01:49:21.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.855+0000 7f2d26e00700 1 -- 172.21.15.1:0/3221816383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d20104db0 msgr2=0x7f2d2019b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:21.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.855+0000 7f2d26e00700 1 --2- 172.21.15.1:0/3221816383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d20104db0 0x7f2d2019b2f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009230 tx=0x7f2d1400fb20 comp rx=0 tx=0).stop 2024-04-26T01:49:21.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.855+0000 7f2d26400700 1 -- 172.21.15.1:0/3221816383 reap_dead start 2024-04-26T01:49:21.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.856+0000 7f2d26e00700 1 -- 172.21.15.1:0/3221816383 shutdown_connections 2024-04-26T01:49:21.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.856+0000 7f2d26e00700 1 -- 172.21.15.1:0/3221816383 >> 172.21.15.1:0/3221816383 conn(0x7f2d20100620 msgr2=0x7f2d20101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:21.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.856+0000 7f2d26e00700 1 -- 172.21.15.1:0/3221816383 shutdown_connections 2024-04-26T01:49:21.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:20.856+0000 7f2d26e00700 1 -- 172.21.15.1:0/3221816383 wait complete. 2024-04-26T01:49:22.270 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:22 smithi001 ceph-mon[27822]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:22.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:22 smithi173 ceph-mon[32553]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:22 smithi097 ceph-mon[32853]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:22.572 INFO:teuthology.orchestra.run.smithi001.stdout:141733920845 2024-04-26T01:49:22.573 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-26T01:49:23.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:23 smithi173 ceph-mon[32553]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-26T01:49:23.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:23 smithi001 ceph-mon[27822]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-26T01:49:23.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:23 smithi097 ceph-mon[32853]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-26T01:49:24.122 INFO:teuthology.orchestra.run.smithi001.stdout:55834574944 2024-04-26T01:49:24.122 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-26T01:49:24.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:24 smithi173 ceph-mon[32553]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-26T01:49:24.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:24 smithi001 ceph-mon[27822]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-26T01:49:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:24 smithi097 ceph-mon[32853]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-26T01:49:24.780 INFO:teuthology.orchestra.run.smithi001.stdout:77309411417 2024-04-26T01:49:24.780 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-26T01:49:25.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.005+0000 7f716d800700 1 -- 172.21.15.1:0/2678286561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f716810f230 msgr2=0x7f7168111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:25.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.005+0000 7f716d800700 1 --2- 172.21.15.1:0/2678286561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f716810f230 0x7f7168111620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7160009f90 tx=0x7f716000b330 comp rx=0 tx=0).stop 2024-04-26T01:49:25.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.005+0000 7f716d800700 1 -- 172.21.15.1:0/2678286561 shutdown_connections 2024-04-26T01:49:25.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.005+0000 7f716d800700 1 --2- 172.21.15.1:0/2678286561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f716810f230 0x7f7168111620 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:25.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.005+0000 7f716d800700 1 --2- 172.21.15.1:0/2678286561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7168074d60 0x7f7168075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:25.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.005+0000 7f716d800700 1 --2- 172.21.15.1:0/2678286561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7168075680 0x7f716810ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:25.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.005+0000 7f716d800700 1 -- 172.21.15.1:0/2678286561 >> 172.21.15.1:0/2678286561 conn(0x7f71680fe4c0 msgr2=0x7f71681008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:25.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.005+0000 7f716d800700 1 -- 172.21.15.1:0/2678286561 shutdown_connections 2024-04-26T01:49:25.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.005+0000 7f716d800700 1 -- 172.21.15.1:0/2678286561 wait complete. 2024-04-26T01:49:25.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.006+0000 7f716d800700 1 Processor -- start 2024-04-26T01:49:25.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.006+0000 7f716d800700 1 -- start start 2024-04-26T01:49:25.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.007+0000 7f716d800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7168074d60 0x7f716819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:25.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.007+0000 7f716d800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7168075680 0x7f716819b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:25.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.007+0000 7f716d800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f716810f230 0x7f716819faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:25.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.007+0000 7f716d800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7168114d80 con 0x7f7168075680 2024-04-26T01:49:25.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.007+0000 7f716d800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7168114c00 con 0x7f7168074d60 2024-04-26T01:49:25.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.007+0000 7f716d800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7168114f00 con 0x7f716810f230 2024-04-26T01:49:25.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.007+0000 7f7167400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7168075680 0x7f716819b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:25.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.007+0000 7f7167400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7168075680 0x7f716819b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55726/0 (socket says 172.21.15.1:55726) 2024-04-26T01:49:25.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.007+0000 7f7167400700 1 -- 172.21.15.1:0/738351842 learned_addr learned my addr 172.21.15.1:0/738351842 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:25.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.007+0000 7f716ce00700 1 --2- 172.21.15.1:0/738351842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f716810f230 0x7f716819faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:25.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.008+0000 7f716ce00700 1 -- 172.21.15.1:0/738351842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7168074d60 msgr2=0x7f716819b270 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T01:49:25.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.008+0000 7f716ce00700 1 --2- 172.21.15.1:0/738351842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7168074d60 0x7f716819b270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:25.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.008+0000 7f716ce00700 1 -- 172.21.15.1:0/738351842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7168075680 msgr2=0x7f716819b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:25.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.008+0000 7f716ce00700 1 --2- 172.21.15.1:0/738351842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7168075680 0x7f716819b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:25.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.008+0000 7f716ce00700 1 -- 172.21.15.1:0/738351842 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71681a0230 con 0x7f716810f230 2024-04-26T01:49:25.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.008+0000 7f7167400700 1 --2- 172.21.15.1:0/738351842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7168075680 0x7f716819b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:49:25.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.008+0000 7f716ce00700 1 --2- 172.21.15.1:0/738351842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f716810f230 0x7f716819faf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f716000b6c0 tx=0x7f716000cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:25.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.009+0000 7f7164c00700 1 -- 172.21.15.1:0/738351842 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160021070 con 0x7f716810f230 2024-04-26T01:49:25.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.009+0000 7f7164c00700 1 -- 172.21.15.1:0/738351842 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7160007ac0 con 0x7f716810f230 2024-04-26T01:49:25.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.009+0000 7f716d800700 1 -- 172.21.15.1:0/738351842 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71681a04c0 con 0x7f716810f230 2024-04-26T01:49:25.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.009+0000 7f7164c00700 1 -- 172.21.15.1:0/738351842 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160005040 con 0x7f716810f230 2024-04-26T01:49:25.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.009+0000 7f716d800700 1 -- 172.21.15.1:0/738351842 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71681a7da0 con 0x7f716810f230 2024-04-26T01:49:25.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.011+0000 7f716d800700 1 -- 172.21.15.1:0/738351842 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f71680631d0 con 0x7f716810f230 2024-04-26T01:49:25.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.013+0000 7f7164c00700 1 -- 172.21.15.1:0/738351842 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f7160024070 con 0x7f716810f230 2024-04-26T01:49:25.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.015+0000 7f7164c00700 1 --2- 172.21.15.1:0/738351842 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f714c0a3cc0 0x7f714c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:25.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.015+0000 7f7164c00700 1 -- 172.21.15.1:0/738351842 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f71600d1250 con 0x7f716810f230 2024-04-26T01:49:25.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.016+0000 7f7167e00700 1 --2- 172.21.15.1:0/738351842 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f714c0a3cc0 0x7f714c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:25.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.016+0000 7f7164c00700 1 --2- 172.21.15.1:0/738351842 >> [v2:172.21.15.1:6802/3176633965,v1:172.21.15.1:6803/3176633965] conn(0x7f714c0ad900 0x7f714c0afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:25.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.016+0000 7f7164c00700 1 -- 172.21.15.1:0/738351842 --> [v2:172.21.15.1:6802/3176633965,v1:172.21.15.1:6803/3176633965] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f716001fe80 con 0x7f714c0ad900 2024-04-26T01:49:25.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.016+0000 7f7164c00700 1 -- 172.21.15.1:0/738351842 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f71600d8050 con 0x7f716810f230 2024-04-26T01:49:25.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.016+0000 7f7167400700 1 --2- 172.21.15.1:0/738351842 >> [v2:172.21.15.1:6802/3176633965,v1:172.21.15.1:6803/3176633965] conn(0x7f714c0ad900 0x7f714c0afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:25.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.017+0000 7f7167e00700 1 --2- 172.21.15.1:0/738351842 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f714c0a3cc0 0x7f714c0a6180 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7158009af0 tx=0x7f7158009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:25.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.017+0000 7f7167400700 1 --2- 172.21.15.1:0/738351842 >> [v2:172.21.15.1:6802/3176633965,v1:172.21.15.1:6803/3176633965] conn(0x7f714c0ad900 0x7f714c0afda0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:25.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.019+0000 7f7164c00700 1 -- 172.21.15.1:0/738351842 <== osd.0 v2:172.21.15.1:6802/3176633965 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f716001fe80 con 0x7f714c0ad900 2024-04-26T01:49:25.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.038+0000 7f716d800700 1 -- 172.21.15.1:0/738351842 --> [v2:172.21.15.1:6802/3176633965,v1:172.21.15.1:6803/3176633965] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7168063650 con 0x7f714c0ad900 2024-04-26T01:49:25.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.039+0000 7f7164c00700 1 -- 172.21.15.1:0/738351842 <== osd.0 v2:172.21.15.1:6802/3176633965 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7168063650 con 0x7f714c0ad900 2024-04-26T01:49:25.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.040+0000 7f716d800700 1 -- 172.21.15.1:0/738351842 >> [v2:172.21.15.1:6802/3176633965,v1:172.21.15.1:6803/3176633965] conn(0x7f714c0ad900 msgr2=0x7f714c0afda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:25.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.040+0000 7f716d800700 1 --2- 172.21.15.1:0/738351842 >> [v2:172.21.15.1:6802/3176633965,v1:172.21.15.1:6803/3176633965] conn(0x7f714c0ad900 0x7f714c0afda0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:25.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.040+0000 7f716d800700 1 -- 172.21.15.1:0/738351842 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f714c0a3cc0 msgr2=0x7f714c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:25.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.040+0000 7f716d800700 1 --2- 172.21.15.1:0/738351842 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f714c0a3cc0 0x7f714c0a6180 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7158009af0 tx=0x7f7158009260 comp rx=0 tx=0).stop 2024-04-26T01:49:25.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.040+0000 7f716d800700 1 -- 172.21.15.1:0/738351842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f716810f230 msgr2=0x7f716819faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:25.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.040+0000 7f716d800700 1 --2- 172.21.15.1:0/738351842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f716810f230 0x7f716819faf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f716000b6c0 tx=0x7f716000cc30 comp rx=0 tx=0).stop 2024-04-26T01:49:25.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.040+0000 7f716ce00700 1 -- 172.21.15.1:0/738351842 reap_dead start 2024-04-26T01:49:25.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.040+0000 7f716d800700 1 -- 172.21.15.1:0/738351842 shutdown_connections 2024-04-26T01:49:25.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.040+0000 7f716d800700 1 -- 172.21.15.1:0/738351842 >> 172.21.15.1:0/738351842 conn(0x7f71680fe4c0 msgr2=0x7f71680ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:25.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.040+0000 7f716d800700 1 -- 172.21.15.1:0/738351842 shutdown_connections 2024-04-26T01:49:25.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:25.040+0000 7f716d800700 1 -- 172.21.15.1:0/738351842 wait complete. 2024-04-26T01:49:26.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:26 smithi173 ceph-mon[32553]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-26T01:49:26.320 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:26 smithi001 ceph-mon[27822]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-26T01:49:26.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:26 smithi097 ceph-mon[32853]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-26T01:49:27.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.113+0000 7f0b42600700 1 -- 172.21.15.1:0/1206507458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b3c103450 msgr2=0x7f0b3c10f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:27.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.113+0000 7f0b42600700 1 --2- 172.21.15.1:0/1206507458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b3c103450 0x7f0b3c10f8e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400b0a0 tx=0x7f0b3400b3b0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.114+0000 7f0b42600700 1 -- 172.21.15.1:0/1206507458 shutdown_connections 2024-04-26T01:49:27.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.114+0000 7f0b42600700 1 --2- 172.21.15.1:0/1206507458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b3c103450 0x7f0b3c10f8e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.114+0000 7f0b42600700 1 --2- 172.21.15.1:0/1206507458 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b3c108a80 0x7f0b3c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.114+0000 7f0b42600700 1 --2- 172.21.15.1:0/1206507458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3c102a90 0x7f0b3c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.114+0000 7f0b42600700 1 -- 172.21.15.1:0/1206507458 >> 172.21.15.1:0/1206507458 conn(0x7f0b3c079bf0 msgr2=0x7f0b3c07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:27.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.114+0000 7f0b42600700 1 -- 172.21.15.1:0/1206507458 shutdown_connections 2024-04-26T01:49:27.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.114+0000 7f0b42600700 1 -- 172.21.15.1:0/1206507458 wait complete. 2024-04-26T01:49:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.115+0000 7f0b42600700 1 Processor -- start 2024-04-26T01:49:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.115+0000 7f0b42600700 1 -- start start 2024-04-26T01:49:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.115+0000 7f0b42600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b3c102a90 0x7f0b3c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.115+0000 7f0b42600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b3c103450 0x7f0b3c075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.115+0000 7f0b42600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3c108a80 0x7f0b3c0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.115+0000 7f0b42600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3c112c80 con 0x7f0b3c102a90 2024-04-26T01:49:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.115+0000 7f0b42600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3c112b00 con 0x7f0b3c108a80 2024-04-26T01:49:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.115+0000 7f0b42600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3c112e00 con 0x7f0b3c103450 2024-04-26T01:49:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.116+0000 7f0b3b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b3c102a90 0x7f0b3c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.116+0000 7f0b3be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3c108a80 0x7f0b3c0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.116+0000 7f0b3be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3c108a80 0x7f0b3c0757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:58126/0 (socket says 172.21.15.1:58126) 2024-04-26T01:49:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.116+0000 7f0b3be00700 1 -- 172.21.15.1:0/1385172552 learned_addr learned my addr 172.21.15.1:0/1385172552 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.116+0000 7f0b3aa00700 1 --2- 172.21.15.1:0/1385172552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b3c103450 0x7f0b3c075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:27.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.116+0000 7f0b3be00700 1 -- 172.21.15.1:0/1385172552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b3c103450 msgr2=0x7f0b3c075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:27.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.116+0000 7f0b3be00700 1 --2- 172.21.15.1:0/1385172552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b3c103450 0x7f0b3c075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.116+0000 7f0b3be00700 1 -- 172.21.15.1:0/1385172552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b3c102a90 msgr2=0x7f0b3c074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:27.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.116+0000 7f0b3be00700 1 --2- 172.21.15.1:0/1385172552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b3c102a90 0x7f0b3c074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.116+0000 7f0b3be00700 1 -- 172.21.15.1:0/1385172552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3c1a7650 con 0x7f0b3c108a80 2024-04-26T01:49:27.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.116+0000 7f0b3b400700 1 --2- 172.21.15.1:0/1385172552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b3c102a90 0x7f0b3c074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:49:27.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.117+0000 7f0b3be00700 1 --2- 172.21.15.1:0/1385172552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3c108a80 0x7f0b3c0757c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400b070 tx=0x7f0b34009c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:27.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.117+0000 7f0b33e00700 1 -- 172.21.15.1:0/1385172552 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34021070 con 0x7f0b3c108a80 2024-04-26T01:49:27.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.117+0000 7f0b42600700 1 -- 172.21.15.1:0/1385172552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b3c1a78e0 con 0x7f0b3c108a80 2024-04-26T01:49:27.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.117+0000 7f0b33e00700 1 -- 172.21.15.1:0/1385172552 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b34004e40 con 0x7f0b3c108a80 2024-04-26T01:49:27.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.117+0000 7f0b33e00700 1 -- 172.21.15.1:0/1385172552 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34004630 con 0x7f0b3c108a80 2024-04-26T01:49:27.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.117+0000 7f0b42600700 1 -- 172.21.15.1:0/1385172552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b3c1a7e70 con 0x7f0b3c108a80 2024-04-26T01:49:27.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.118+0000 7f0b42600700 1 -- 172.21.15.1:0/1385172552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b3c063110 con 0x7f0b3c108a80 2024-04-26T01:49:27.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.121+0000 7f0b33e00700 1 -- 172.21.15.1:0/1385172552 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f0b34007980 con 0x7f0b3c108a80 2024-04-26T01:49:27.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.122+0000 7f0b33e00700 1 --2- 172.21.15.1:0/1385172552 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0b200a3c40 0x7f0b200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:27.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.122+0000 7f0b33e00700 1 -- 172.21.15.1:0/1385172552 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f0b3400f020 con 0x7f0b3c108a80 2024-04-26T01:49:27.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.123+0000 7f0b3b400700 1 --2- 172.21.15.1:0/1385172552 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0b200a3c40 0x7f0b200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:27.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.123+0000 7f0b33e00700 1 --2- 172.21.15.1:0/1385172552 >> [v2:172.21.15.1:6810/3349961961,v1:172.21.15.1:6811/3349961961] conn(0x7f0b200ad880 0x7f0b200afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:27.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.123+0000 7f0b33e00700 1 -- 172.21.15.1:0/1385172552 --> [v2:172.21.15.1:6810/3349961961,v1:172.21.15.1:6811/3349961961] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b3401fe80 con 0x7f0b200ad880 2024-04-26T01:49:27.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.123+0000 7f0b33e00700 1 -- 172.21.15.1:0/1385172552 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f0b340d1d20 con 0x7f0b3c108a80 2024-04-26T01:49:27.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.123+0000 7f0b3aa00700 1 --2- 172.21.15.1:0/1385172552 >> [v2:172.21.15.1:6810/3349961961,v1:172.21.15.1:6811/3349961961] conn(0x7f0b200ad880 0x7f0b200afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:27.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.123+0000 7f0b3b400700 1 --2- 172.21.15.1:0/1385172552 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0b200a3c40 0x7f0b200a6100 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c009fd0 tx=0x7f0b2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:27.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.124+0000 7f0b3aa00700 1 --2- 172.21.15.1:0/1385172552 >> [v2:172.21.15.1:6810/3349961961,v1:172.21.15.1:6811/3349961961] conn(0x7f0b200ad880 0x7f0b200afd20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:27.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.125+0000 7f0b33e00700 1 -- 172.21.15.1:0/1385172552 <== osd.3 v2:172.21.15.1:6810/3349961961 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0b3401fe80 con 0x7f0b200ad880 2024-04-26T01:49:27.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.148+0000 7f0b42600700 1 -- 172.21.15.1:0/1385172552 --> [v2:172.21.15.1:6810/3349961961,v1:172.21.15.1:6811/3349961961] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0b3c063510 con 0x7f0b200ad880 2024-04-26T01:49:27.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.149+0000 7f0b33e00700 1 -- 172.21.15.1:0/1385172552 <== osd.3 v2:172.21.15.1:6810/3349961961 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0b3c063510 con 0x7f0b200ad880 2024-04-26T01:49:27.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.150+0000 7f0b42600700 1 -- 172.21.15.1:0/1385172552 >> [v2:172.21.15.1:6810/3349961961,v1:172.21.15.1:6811/3349961961] conn(0x7f0b200ad880 msgr2=0x7f0b200afd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:27.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.150+0000 7f0b42600700 1 --2- 172.21.15.1:0/1385172552 >> [v2:172.21.15.1:6810/3349961961,v1:172.21.15.1:6811/3349961961] conn(0x7f0b200ad880 0x7f0b200afd20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.150+0000 7f0b42600700 1 -- 172.21.15.1:0/1385172552 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0b200a3c40 msgr2=0x7f0b200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:27.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.150+0000 7f0b42600700 1 --2- 172.21.15.1:0/1385172552 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0b200a3c40 0x7f0b200a6100 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c009fd0 tx=0x7f0b2c009260 comp rx=0 tx=0).stop 2024-04-26T01:49:27.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.150+0000 7f0b42600700 1 -- 172.21.15.1:0/1385172552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3c108a80 msgr2=0x7f0b3c0757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:27.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.150+0000 7f0b42600700 1 --2- 172.21.15.1:0/1385172552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3c108a80 0x7f0b3c0757c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400b070 tx=0x7f0b34009c90 comp rx=0 tx=0).stop 2024-04-26T01:49:27.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.150+0000 7f0b3be00700 1 -- 172.21.15.1:0/1385172552 reap_dead start 2024-04-26T01:49:27.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.150+0000 7f0b42600700 1 -- 172.21.15.1:0/1385172552 shutdown_connections 2024-04-26T01:49:27.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.150+0000 7f0b42600700 1 -- 172.21.15.1:0/1385172552 >> 172.21.15.1:0/1385172552 conn(0x7f0b3c079bf0 msgr2=0x7f0b3c1016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:27.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.150+0000 7f0b42600700 1 -- 172.21.15.1:0/1385172552 shutdown_connections 2024-04-26T01:49:27.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:27.150+0000 7f0b42600700 1 -- 172.21.15.1:0/1385172552 wait complete. 2024-04-26T01:49:27.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.293+0000 7feaf3200700 1 -- 172.21.15.1:0/4000787930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec109c80 msgr2=0x7feaec10dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:27.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.293+0000 7feaf3200700 1 --2- 172.21.15.1:0/4000787930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec109c80 0x7feaec10dcd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fead80099c0 tx=0x7fead8009cd0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.293+0000 7feaf3200700 1 -- 172.21.15.1:0/4000787930 shutdown_connections 2024-04-26T01:49:27.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.293+0000 7feaf3200700 1 --2- 172.21.15.1:0/4000787930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaec10e3b0 0x7feaec115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.293+0000 7feaf3200700 1 --2- 172.21.15.1:0/4000787930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feaec1092d0 0x7feaec1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.293+0000 7feaf3200700 1 --2- 172.21.15.1:0/4000787930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec109c80 0x7feaec10dcd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.293+0000 7feaf3200700 1 -- 172.21.15.1:0/4000787930 >> 172.21.15.1:0/4000787930 conn(0x7feaec076210 msgr2=0x7feaec107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:27.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.293+0000 7feaf3200700 1 -- 172.21.15.1:0/4000787930 shutdown_connections 2024-04-26T01:49:27.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.293+0000 7feaf3200700 1 -- 172.21.15.1:0/4000787930 wait complete. 2024-04-26T01:49:27.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.294+0000 7feaf3200700 1 Processor -- start 2024-04-26T01:49:27.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.295+0000 7feaf3200700 1 -- start start 2024-04-26T01:49:27.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.295+0000 7feaf3200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec1092d0 0x7feaec19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:27.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.295+0000 7feaf3200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaec109c80 0x7feaec19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:27.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.295+0000 7feaf3200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feaec10e3b0 0x7feaec1a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:27.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.295+0000 7feaf3200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feaec119220 con 0x7feaec1092d0 2024-04-26T01:49:27.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.295+0000 7feaf3200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feaec1190a0 con 0x7feaec109c80 2024-04-26T01:49:27.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.295+0000 7feaf3200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feaec1193a0 con 0x7feaec10e3b0 2024-04-26T01:49:27.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.295+0000 7feaebe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec1092d0 0x7feaec19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:27.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.295+0000 7feaebe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec1092d0 0x7feaec19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55742/0 (socket says 172.21.15.1:55742) 2024-04-26T01:49:27.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.295+0000 7feaebe00700 1 -- 172.21.15.1:0/3036059246 learned_addr learned my addr 172.21.15.1:0/3036059246 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:27.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.295+0000 7feaf1200700 1 --2- 172.21.15.1:0/3036059246 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feaec10e3b0 0x7feaec1a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:27.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.295+0000 7feaeb400700 1 --2- 172.21.15.1:0/3036059246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaec109c80 0x7feaec19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:27.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.296+0000 7feaebe00700 1 -- 172.21.15.1:0/3036059246 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feaec10e3b0 msgr2=0x7feaec1a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:27.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.296+0000 7feaebe00700 1 --2- 172.21.15.1:0/3036059246 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feaec10e3b0 0x7feaec1a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.296+0000 7feaebe00700 1 -- 172.21.15.1:0/3036059246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaec109c80 msgr2=0x7feaec19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:27.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.296+0000 7feaebe00700 1 --2- 172.21.15.1:0/3036059246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaec109c80 0x7feaec19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.296+0000 7feaebe00700 1 -- 172.21.15.1:0/3036059246 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaec1a4760 con 0x7feaec1092d0 2024-04-26T01:49:27.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.296+0000 7feaeb400700 1 --2- 172.21.15.1:0/3036059246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feaec109c80 0x7feaec19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:49:27.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.296+0000 7feaebe00700 1 --2- 172.21.15.1:0/3036059246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec1092d0 0x7feaec19f750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feadc00e9a0 tx=0x7feadc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:27.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.296+0000 7feae8c00700 1 -- 172.21.15.1:0/3036059246 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc00cbd0 con 0x7feaec1092d0 2024-04-26T01:49:27.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.296+0000 7feae8c00700 1 -- 172.21.15.1:0/3036059246 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feadc00cd70 con 0x7feaec1092d0 2024-04-26T01:49:27.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.296+0000 7feae8c00700 1 -- 172.21.15.1:0/3036059246 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc018680 con 0x7feaec1092d0 2024-04-26T01:49:27.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.296+0000 7feaf3200700 1 -- 172.21.15.1:0/3036059246 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaec1a4a50 con 0x7feaec1092d0 2024-04-26T01:49:27.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.297+0000 7feaf3200700 1 -- 172.21.15.1:0/3036059246 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaec1ac2d0 con 0x7feaec1092d0 2024-04-26T01:49:27.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.297+0000 7feaf3200700 1 -- 172.21.15.1:0/3036059246 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feaec063150 con 0x7feaec1092d0 2024-04-26T01:49:27.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.300+0000 7feae8c00700 1 -- 172.21.15.1:0/3036059246 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7feadc010430 con 0x7feaec1092d0 2024-04-26T01:49:27.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.301+0000 7feae8c00700 1 --2- 172.21.15.1:0/3036059246 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fead00a3cc0 0x7fead00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:27.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.301+0000 7feae8c00700 1 -- 172.21.15.1:0/3036059246 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7feadc0d0ec0 con 0x7feaec1092d0 2024-04-26T01:49:27.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.301+0000 7feaeb400700 1 --2- 172.21.15.1:0/3036059246 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fead00a3cc0 0x7fead00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:27.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.302+0000 7feae8c00700 1 --2- 172.21.15.1:0/3036059246 >> [v2:172.21.15.173:6808/1026507957,v1:172.21.15.173:6809/1026507957] conn(0x7fead00ad900 0x7fead00afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:27.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.302+0000 7feae8c00700 1 -- 172.21.15.1:0/3036059246 --> [v2:172.21.15.173:6808/1026507957,v1:172.21.15.173:6809/1026507957] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feadc027e50 con 0x7fead00ad900 2024-04-26T01:49:27.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.302+0000 7feae8c00700 1 -- 172.21.15.1:0/3036059246 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7feadc0d1270 con 0x7feaec1092d0 2024-04-26T01:49:27.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.302+0000 7feaeb400700 1 --2- 172.21.15.1:0/3036059246 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fead00a3cc0 0x7fead00a6180 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fead8000c00 tx=0x7fead8005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:27.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.302+0000 7feaf1200700 1 --2- 172.21.15.1:0/3036059246 >> [v2:172.21.15.173:6808/1026507957,v1:172.21.15.173:6809/1026507957] conn(0x7fead00ad900 0x7fead00afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:27.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.303+0000 7feaf1200700 1 --2- 172.21.15.1:0/3036059246 >> [v2:172.21.15.173:6808/1026507957,v1:172.21.15.173:6809/1026507957] conn(0x7fead00ad900 0x7fead00afda0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:27.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.304+0000 7feae8c00700 1 -- 172.21.15.1:0/3036059246 <== osd.5 v2:172.21.15.173:6808/1026507957 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feadc027e50 con 0x7fead00ad900 2024-04-26T01:49:27.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.322+0000 7feaf3200700 1 -- 172.21.15.1:0/3036059246 --> [v2:172.21.15.173:6808/1026507957,v1:172.21.15.173:6809/1026507957] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feaec0635d0 con 0x7fead00ad900 2024-04-26T01:49:27.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.323+0000 7feae8c00700 1 -- 172.21.15.1:0/3036059246 <== osd.5 v2:172.21.15.173:6808/1026507957 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7feaec0635d0 con 0x7fead00ad900 2024-04-26T01:49:27.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.323+0000 7feaf3200700 1 -- 172.21.15.1:0/3036059246 >> [v2:172.21.15.173:6808/1026507957,v1:172.21.15.173:6809/1026507957] conn(0x7fead00ad900 msgr2=0x7fead00afda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:27.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.323+0000 7feaf3200700 1 --2- 172.21.15.1:0/3036059246 >> [v2:172.21.15.173:6808/1026507957,v1:172.21.15.173:6809/1026507957] conn(0x7fead00ad900 0x7fead00afda0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.324+0000 7feaf3200700 1 -- 172.21.15.1:0/3036059246 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fead00a3cc0 msgr2=0x7fead00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:27.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.324+0000 7feaf3200700 1 --2- 172.21.15.1:0/3036059246 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fead00a3cc0 0x7fead00a6180 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fead8000c00 tx=0x7fead8005ab0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.324+0000 7feaf3200700 1 -- 172.21.15.1:0/3036059246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec1092d0 msgr2=0x7feaec19f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:27.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.324+0000 7feaf3200700 1 --2- 172.21.15.1:0/3036059246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec1092d0 0x7feaec19f750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feadc00e9a0 tx=0x7feadc00ecb0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.324+0000 7feaf1200700 1 -- 172.21.15.1:0/3036059246 reap_dead start 2024-04-26T01:49:27.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.324+0000 7feaf3200700 1 -- 172.21.15.1:0/3036059246 shutdown_connections 2024-04-26T01:49:27.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.324+0000 7feaf3200700 1 -- 172.21.15.1:0/3036059246 >> 172.21.15.1:0/3036059246 conn(0x7feaec076210 msgr2=0x7feaec0768d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:27.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.324+0000 7feaf3200700 1 -- 172.21.15.1:0/3036059246 shutdown_connections 2024-04-26T01:49:27.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:26.324+0000 7feaf3200700 1 -- 172.21.15.1:0/3036059246 wait complete. 2024-04-26T01:49:28.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:28 smithi173 ceph-mon[32553]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-26T01:49:28.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:28 smithi001 ceph-mon[27822]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-26T01:49:28.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:28 smithi097 ceph-mon[32853]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-26T01:49:28.548 INFO:teuthology.orchestra.run.smithi001.stdout:34359738469 2024-04-26T01:49:28.548 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-26T01:49:29.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.002+0000 7fa06a200700 1 -- 172.21.15.1:0/2002480783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa064103490 msgr2=0x7fa06410f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:29.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.002+0000 7fa06a200700 1 --2- 172.21.15.1:0/2002480783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa064103490 0x7fa06410f970 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00b0a0 tx=0x7fa05c00b3b0 comp rx=0 tx=0).stop 2024-04-26T01:49:29.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.002+0000 7fa06a200700 1 -- 172.21.15.1:0/2002480783 shutdown_connections 2024-04-26T01:49:29.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.002+0000 7fa06a200700 1 --2- 172.21.15.1:0/2002480783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa064103490 0x7fa06410f970 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:29.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.002+0000 7fa06a200700 1 --2- 172.21.15.1:0/2002480783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa064108aa0 0x7fa064108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:29.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.002+0000 7fa06a200700 1 --2- 172.21.15.1:0/2002480783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa064102ad0 0x7fa064102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:29.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.002+0000 7fa06a200700 1 -- 172.21.15.1:0/2002480783 >> 172.21.15.1:0/2002480783 conn(0x7fa064079c30 msgr2=0x7fa06407a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:29.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.002+0000 7fa06a200700 1 -- 172.21.15.1:0/2002480783 shutdown_connections 2024-04-26T01:49:29.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.002+0000 7fa06a200700 1 -- 172.21.15.1:0/2002480783 wait complete. 2024-04-26T01:49:29.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.003+0000 7fa06a200700 1 Processor -- start 2024-04-26T01:49:29.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.003+0000 7fa06a200700 1 -- start start 2024-04-26T01:49:29.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.004+0000 7fa06a200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa064102ad0 0x7fa064074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:29.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.004+0000 7fa06a200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa064103490 0x7fa064075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:29.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.004+0000 7fa06a200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa064108aa0 0x7fa0640757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:29.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.004+0000 7fa06a200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa064112d30 con 0x7fa064108aa0 2024-04-26T01:49:29.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.004+0000 7fa06a200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa064112bb0 con 0x7fa064103490 2024-04-26T01:49:29.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.004+0000 7fa06a200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa064112eb0 con 0x7fa064102ad0 2024-04-26T01:49:29.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.004+0000 7fa063400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa064102ad0 0x7fa064074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:29.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.004+0000 7fa063400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa064102ad0 0x7fa064074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:33704/0 (socket says 172.21.15.1:33704) 2024-04-26T01:49:29.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.004+0000 7fa062a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa064103490 0x7fa064075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:29.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.004+0000 7fa063400700 1 -- 172.21.15.1:0/3292659571 learned_addr learned my addr 172.21.15.1:0/3292659571 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:29.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.004+0000 7fa063e00700 1 --2- 172.21.15.1:0/3292659571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa064108aa0 0x7fa0640757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:29.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.005+0000 7fa063400700 1 -- 172.21.15.1:0/3292659571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa064103490 msgr2=0x7fa064075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:29.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.005+0000 7fa063400700 1 --2- 172.21.15.1:0/3292659571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa064103490 0x7fa064075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:29.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.005+0000 7fa063400700 1 -- 172.21.15.1:0/3292659571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa064108aa0 msgr2=0x7fa0640757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:29.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.005+0000 7fa063400700 1 --2- 172.21.15.1:0/3292659571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa064108aa0 0x7fa0640757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:29.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.005+0000 7fa063400700 1 -- 172.21.15.1:0/3292659571 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0641a76a0 con 0x7fa064102ad0 2024-04-26T01:49:29.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.005+0000 7fa063400700 1 --2- 172.21.15.1:0/3292659571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa064102ad0 0x7fa064074d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa05400e9c0 tx=0x7fa05400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:29.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.005+0000 7fa05be00700 1 -- 172.21.15.1:0/3292659571 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05400cbd0 con 0x7fa064102ad0 2024-04-26T01:49:29.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.005+0000 7fa05be00700 1 -- 172.21.15.1:0/3292659571 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa05400cd70 con 0x7fa064102ad0 2024-04-26T01:49:29.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.005+0000 7fa06a200700 1 -- 172.21.15.1:0/3292659571 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0641a7990 con 0x7fa064102ad0 2024-04-26T01:49:29.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.005+0000 7fa05be00700 1 -- 172.21.15.1:0/3292659571 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054005610 con 0x7fa064102ad0 2024-04-26T01:49:29.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.005+0000 7fa06a200700 1 -- 172.21.15.1:0/3292659571 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0641a7ed0 con 0x7fa064102ad0 2024-04-26T01:49:29.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.006+0000 7fa06a200700 1 -- 172.21.15.1:0/3292659571 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa064063110 con 0x7fa064102ad0 2024-04-26T01:49:29.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.008+0000 7fa05be00700 1 -- 172.21.15.1:0/3292659571 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fa054004750 con 0x7fa064102ad0 2024-04-26T01:49:29.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.010+0000 7fa05be00700 1 --2- 172.21.15.1:0/3292659571 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa0480a3cc0 0x7fa0480a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:29.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.010+0000 7fa05be00700 1 -- 172.21.15.1:0/3292659571 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fa0540d0680 con 0x7fa064102ad0 2024-04-26T01:49:29.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.010+0000 7fa062a00700 1 --2- 172.21.15.1:0/3292659571 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa0480a3cc0 0x7fa0480a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:29.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.010+0000 7fa05be00700 1 --2- 172.21.15.1:0/3292659571 >> [v2:172.21.15.173:6824/1141524960,v1:172.21.15.173:6825/1141524960] conn(0x7fa0480ad900 0x7fa0480afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:29.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.010+0000 7fa05be00700 1 -- 172.21.15.1:0/3292659571 --> [v2:172.21.15.173:6824/1141524960,v1:172.21.15.173:6825/1141524960] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa0480b0470 con 0x7fa0480ad900 2024-04-26T01:49:29.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.010+0000 7fa05be00700 1 -- 172.21.15.1:0/3292659571 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fa0540d7050 con 0x7fa064102ad0 2024-04-26T01:49:29.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.011+0000 7fa062a00700 1 --2- 172.21.15.1:0/3292659571 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa0480a3cc0 0x7fa0480a6180 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa06407a2a0 tx=0x7fa050009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:29.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.011+0000 7fa063e00700 1 --2- 172.21.15.1:0/3292659571 >> [v2:172.21.15.173:6824/1141524960,v1:172.21.15.173:6825/1141524960] conn(0x7fa0480ad900 0x7fa0480afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:29.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.011+0000 7fa063e00700 1 --2- 172.21.15.1:0/3292659571 >> [v2:172.21.15.173:6824/1141524960,v1:172.21.15.173:6825/1141524960] conn(0x7fa0480ad900 0x7fa0480afda0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:29.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.012+0000 7fa05be00700 1 -- 172.21.15.1:0/3292659571 <== osd.11 v2:172.21.15.173:6824/1141524960 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa0480b0470 con 0x7fa0480ad900 2024-04-26T01:49:29.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.031+0000 7fa06a200700 1 -- 172.21.15.1:0/3292659571 --> [v2:172.21.15.173:6824/1141524960,v1:172.21.15.173:6825/1141524960] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa064063510 con 0x7fa0480ad900 2024-04-26T01:49:29.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.032+0000 7fa05be00700 1 -- 172.21.15.1:0/3292659571 <== osd.11 v2:172.21.15.173:6824/1141524960 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa064063510 con 0x7fa0480ad900 2024-04-26T01:49:29.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.032+0000 7fa06a200700 1 -- 172.21.15.1:0/3292659571 >> [v2:172.21.15.173:6824/1141524960,v1:172.21.15.173:6825/1141524960] conn(0x7fa0480ad900 msgr2=0x7fa0480afda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:29.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.032+0000 7fa06a200700 1 --2- 172.21.15.1:0/3292659571 >> [v2:172.21.15.173:6824/1141524960,v1:172.21.15.173:6825/1141524960] conn(0x7fa0480ad900 0x7fa0480afda0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:29.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.033+0000 7fa06a200700 1 -- 172.21.15.1:0/3292659571 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa0480a3cc0 msgr2=0x7fa0480a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:29.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.033+0000 7fa06a200700 1 --2- 172.21.15.1:0/3292659571 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa0480a3cc0 0x7fa0480a6180 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa06407a2a0 tx=0x7fa050009260 comp rx=0 tx=0).stop 2024-04-26T01:49:29.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.033+0000 7fa06a200700 1 -- 172.21.15.1:0/3292659571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa064102ad0 msgr2=0x7fa064074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:29.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.033+0000 7fa06a200700 1 --2- 172.21.15.1:0/3292659571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa064102ad0 0x7fa064074d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa05400e9c0 tx=0x7fa05400ecd0 comp rx=0 tx=0).stop 2024-04-26T01:49:29.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.033+0000 7fa063e00700 1 -- 172.21.15.1:0/3292659571 reap_dead start 2024-04-26T01:49:29.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.033+0000 7fa06a200700 1 -- 172.21.15.1:0/3292659571 shutdown_connections 2024-04-26T01:49:29.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.033+0000 7fa06a200700 1 -- 172.21.15.1:0/3292659571 >> 172.21.15.1:0/3292659571 conn(0x7fa064079c30 msgr2=0x7fa064100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:29.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.033+0000 7fa06a200700 1 -- 172.21.15.1:0/3292659571 shutdown_connections 2024-04-26T01:49:29.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.033+0000 7fa06a200700 1 -- 172.21.15.1:0/3292659571 wait complete. 2024-04-26T01:49:29.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:29 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:29.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:29 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:29.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:29 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:30.324 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:30 smithi001 ceph-mon[27822]: pgmap v386: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-26T01:49:30.324 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:30 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:49:30.324 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:30 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:49:30.324 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:30 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:49:30.324 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:30 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:49:30.327 INFO:teuthology.orchestra.run.smithi001.stdout:167503724619 2024-04-26T01:49:30.327 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-26T01:49:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.123+0000 7f705ba00700 1 -- 172.21.15.1:0/939121143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7054108a20 msgr2=0x7f7054108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.123+0000 7f705ba00700 1 --2- 172.21.15.1:0/939121143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7054108a20 0x7f7054108e00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f70480098f0 tx=0x7f7048009c00 comp rx=0 tx=0).stop 2024-04-26T01:49:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.123+0000 7f705ba00700 1 -- 172.21.15.1:0/939121143 shutdown_connections 2024-04-26T01:49:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.123+0000 7f705ba00700 1 --2- 172.21.15.1:0/939121143 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705406b980 0x7f705410f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.123+0000 7f705ba00700 1 --2- 172.21.15.1:0/939121143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7054108a20 0x7f7054108e00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.123+0000 7f705ba00700 1 --2- 172.21.15.1:0/939121143 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f705406afe0 0x7f705406b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.123+0000 7f705ba00700 1 -- 172.21.15.1:0/939121143 >> 172.21.15.1:0/939121143 conn(0x7f70540788b0 msgr2=0x7f7054078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.123+0000 7f705ba00700 1 -- 172.21.15.1:0/939121143 shutdown_connections 2024-04-26T01:49:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.123+0000 7f705ba00700 1 -- 172.21.15.1:0/939121143 wait complete. 2024-04-26T01:49:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.124+0000 7f705ba00700 1 Processor -- start 2024-04-26T01:49:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.124+0000 7f705ba00700 1 -- start start 2024-04-26T01:49:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.125+0000 7f705ba00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705406afe0 0x7f7054101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.125+0000 7f705ba00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f705406b980 0x7f7054101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.125+0000 7f705ba00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7054108a20 0x7f7054102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.125+0000 7f705ba00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7054112bf0 con 0x7f705406b980 2024-04-26T01:49:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.125+0000 7f705ba00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7054112a70 con 0x7f7054108a20 2024-04-26T01:49:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.125+0000 7f705ba00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7054112d70 con 0x7f705406afe0 2024-04-26T01:49:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.125+0000 7f7059000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705406afe0 0x7f7054101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.125+0000 7f7059000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705406afe0 0x7f7054101a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:33720/0 (socket says 172.21.15.1:33720) 2024-04-26T01:49:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.125+0000 7f7059000700 1 -- 172.21.15.1:0/1175630513 learned_addr learned my addr 172.21.15.1:0/1175630513 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.125+0000 7f7053e00700 1 --2- 172.21.15.1:0/1175630513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f705406b980 0x7f7054101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.125+0000 7f7059a00700 1 --2- 172.21.15.1:0/1175630513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7054108a20 0x7f7054102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.125+0000 7f7059000700 1 -- 172.21.15.1:0/1175630513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7054108a20 msgr2=0x7f7054102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.125+0000 7f7059000700 1 --2- 172.21.15.1:0/1175630513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7054108a20 0x7f7054102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.125+0000 7f7059000700 1 -- 172.21.15.1:0/1175630513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f705406b980 msgr2=0x7f7054101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.125+0000 7f7059000700 1 --2- 172.21.15.1:0/1175630513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f705406b980 0x7f7054101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.125+0000 7f7059000700 1 -- 172.21.15.1:0/1175630513 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70541a76d0 con 0x7f705406afe0 2024-04-26T01:49:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.126+0000 7f7059000700 1 --2- 172.21.15.1:0/1175630513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705406afe0 0x7f7054101a00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7048009230 tx=0x7f704800fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.126+0000 7f7051600700 1 -- 172.21.15.1:0/1175630513 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048005140 con 0x7f705406afe0 2024-04-26T01:49:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.126+0000 7f705ba00700 1 -- 172.21.15.1:0/1175630513 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70541a7960 con 0x7f705406afe0 2024-04-26T01:49:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.126+0000 7f7051600700 1 -- 172.21.15.1:0/1175630513 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f704800b560 con 0x7f705406afe0 2024-04-26T01:49:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.126+0000 7f7051600700 1 -- 172.21.15.1:0/1175630513 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048003970 con 0x7f705406afe0 2024-04-26T01:49:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.126+0000 7f705ba00700 1 -- 172.21.15.1:0/1175630513 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70541a7e10 con 0x7f705406afe0 2024-04-26T01:49:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.127+0000 7f705ba00700 1 -- 172.21.15.1:0/1175630513 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7054063110 con 0x7f705406afe0 2024-04-26T01:49:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.129+0000 7f7051600700 1 -- 172.21.15.1:0/1175630513 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f704800ba50 con 0x7f705406afe0 2024-04-26T01:49:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.130+0000 7f7051600700 1 --2- 172.21.15.1:0/1175630513 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f703c0a3cc0 0x7f703c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.130+0000 7f7051600700 1 -- 172.21.15.1:0/1175630513 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f70480d1810 con 0x7f705406afe0 2024-04-26T01:49:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.131+0000 7f7053e00700 1 --2- 172.21.15.1:0/1175630513 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f703c0a3cc0 0x7f703c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.131+0000 7f7051600700 1 --2- 172.21.15.1:0/1175630513 >> [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839] conn(0x7f703c0ad900 0x7f703c0afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.131+0000 7f7051600700 1 -- 172.21.15.1:0/1175630513 --> [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f704800eea0 con 0x7f703c0ad900 2024-04-26T01:49:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.131+0000 7f7051600700 1 -- 172.21.15.1:0/1175630513 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f70480d1bc0 con 0x7f705406afe0 2024-04-26T01:49:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.131+0000 7f7059a00700 1 --2- 172.21.15.1:0/1175630513 >> [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839] conn(0x7f703c0ad900 0x7f703c0afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.131+0000 7f7053e00700 1 --2- 172.21.15.1:0/1175630513 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f703c0a3cc0 0x7f703c0a6180 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7054078f60 tx=0x7f7040009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.132+0000 7f7059a00700 1 --2- 172.21.15.1:0/1175630513 >> [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839] conn(0x7f703c0ad900 0x7f703c0afda0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.133+0000 7f7051600700 1 -- 172.21.15.1:0/1175630513 <== osd.9 v2:172.21.15.1:6826/2973486839 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f704800eea0 con 0x7f703c0ad900 2024-04-26T01:49:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.151+0000 7f705ba00700 1 -- 172.21.15.1:0/1175630513 --> [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7054063590 con 0x7f703c0ad900 2024-04-26T01:49:30.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.152+0000 7f7051600700 1 -- 172.21.15.1:0/1175630513 <== osd.9 v2:172.21.15.1:6826/2973486839 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7054063590 con 0x7f703c0ad900 2024-04-26T01:49:30.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.152+0000 7f705ba00700 1 -- 172.21.15.1:0/1175630513 >> [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839] conn(0x7f703c0ad900 msgr2=0x7f703c0afda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:30.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.152+0000 7f705ba00700 1 --2- 172.21.15.1:0/1175630513 >> [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839] conn(0x7f703c0ad900 0x7f703c0afda0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:30.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.152+0000 7f705ba00700 1 -- 172.21.15.1:0/1175630513 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f703c0a3cc0 msgr2=0x7f703c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:30.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.152+0000 7f705ba00700 1 --2- 172.21.15.1:0/1175630513 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f703c0a3cc0 0x7f703c0a6180 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7054078f60 tx=0x7f7040009310 comp rx=0 tx=0).stop 2024-04-26T01:49:30.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.152+0000 7f705ba00700 1 -- 172.21.15.1:0/1175630513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705406afe0 msgr2=0x7f7054101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:30.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.152+0000 7f705ba00700 1 --2- 172.21.15.1:0/1175630513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705406afe0 0x7f7054101a00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7048009230 tx=0x7f704800fa70 comp rx=0 tx=0).stop 2024-04-26T01:49:30.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.152+0000 7f7059a00700 1 -- 172.21.15.1:0/1175630513 reap_dead start 2024-04-26T01:49:30.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.152+0000 7f705ba00700 1 -- 172.21.15.1:0/1175630513 shutdown_connections 2024-04-26T01:49:30.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.152+0000 7f705ba00700 1 -- 172.21.15.1:0/1175630513 >> 172.21.15.1:0/1175630513 conn(0x7f70540788b0 msgr2=0x7f70540ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:30.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.153+0000 7f705ba00700 1 -- 172.21.15.1:0/1175630513 shutdown_connections 2024-04-26T01:49:30.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:29.153+0000 7f705ba00700 1 -- 172.21.15.1:0/1175630513 wait complete. 2024-04-26T01:49:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:30 smithi097 ceph-mon[32853]: pgmap v386: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-26T01:49:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:30 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:49:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:30 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:49:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:30 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:49:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:30 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:49:30.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:30 smithi173 ceph-mon[32553]: pgmap v386: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-26T01:49:30.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:30 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:49:30.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:30 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:49:30.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:30 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:49:30.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:30 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:49:30.867 INFO:teuthology.orchestra.run.smithi001.stdout:317827579947 2024-04-26T01:49:30.867 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-26T01:49:30.986 INFO:teuthology.orchestra.run.smithi001.stdout:107374182484 2024-04-26T01:49:30.986 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-26T01:49:31.473 INFO:teuthology.orchestra.run.smithi001.stdout:296352743475 2024-04-26T01:49:31.474 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-26T01:49:31.795 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:31.911 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:31.913 INFO:teuthology.orchestra.run.smithi001.stdout:352187318312 2024-04-26T01:49:31.914 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-26T01:49:31.999 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:31 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:49:32.000 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:31 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:49:32.000 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:31 smithi001 ceph-mon[27822]: pgmap v387: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-26T01:49:32.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:31 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:49:32.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:31 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:49:32.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:31 smithi173 ceph-mon[32553]: pgmap v387: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-26T01:49:32.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:49:32.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:31 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:49:32.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:31 smithi097 ceph-mon[32853]: pgmap v387: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-26T01:49:34.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:33 smithi001 ceph-mon[27822]: pgmap v388: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-26T01:49:34.168 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:34.175 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:34.176 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:34.178 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:34.196 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:34.196 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:34.201 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:34.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:33 smithi097 ceph-mon[32853]: pgmap v388: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-26T01:49:34.297 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:34.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:33 smithi173 ceph-mon[32553]: pgmap v388: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-26T01:49:35.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.967+0000 7fc090600700 1 -- 172.21.15.1:0/2132303379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc088113920 msgr2=0x7fc088115d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:35.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.967+0000 7fc090600700 1 --2- 172.21.15.1:0/2132303379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc088113920 0x7fc088115d10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc08000b0a0 tx=0x7fc08000b3b0 comp rx=0 tx=0).stop 2024-04-26T01:49:35.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.967+0000 7fc090600700 1 -- 172.21.15.1:0/2132303379 shutdown_connections 2024-04-26T01:49:35.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.967+0000 7fc090600700 1 --2- 172.21.15.1:0/2132303379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc088113920 0x7fc088115d10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:35.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.967+0000 7fc090600700 1 --2- 172.21.15.1:0/2132303379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc088076f70 0x7fc088077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:35.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.967+0000 7fc090600700 1 --2- 172.21.15.1:0/2132303379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc088077890 0x7fc0881133e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:35.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.967+0000 7fc090600700 1 -- 172.21.15.1:0/2132303379 >> 172.21.15.1:0/2132303379 conn(0x7fc088100770 msgr2=0x7fc088102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:35.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.967+0000 7fc090600700 1 -- 172.21.15.1:0/2132303379 shutdown_connections 2024-04-26T01:49:35.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.967+0000 7fc090600700 1 -- 172.21.15.1:0/2132303379 wait complete. 2024-04-26T01:49:35.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.968+0000 7fc090600700 1 Processor -- start 2024-04-26T01:49:35.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.968+0000 7fc090600700 1 -- start start 2024-04-26T01:49:35.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.969+0000 7fc090600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc088076f70 0x7fc08819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:35.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.969+0000 7fc090600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc088077890 0x7fc08819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:35.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.969+0000 7fc090600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc088113920 0x7fc0881a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:35.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.969+0000 7fc090600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc088119270 con 0x7fc088077890 2024-04-26T01:49:35.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.969+0000 7fc090600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc0881190f0 con 0x7fc088113920 2024-04-26T01:49:35.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.969+0000 7fc090600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc0881193f0 con 0x7fc088076f70 2024-04-26T01:49:35.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.969+0000 7fc08d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc088077890 0x7fc08819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:35.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.969+0000 7fc08dc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc088076f70 0x7fc08819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:35.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.969+0000 7fc08d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc088077890 0x7fc08819fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54952/0 (socket says 172.21.15.1:54952) 2024-04-26T01:49:35.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.969+0000 7fc08d200700 1 -- 172.21.15.1:0/2423174465 learned_addr learned my addr 172.21.15.1:0/2423174465 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:35.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.969+0000 7fc08e600700 1 --2- 172.21.15.1:0/2423174465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc088113920 0x7fc0881a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:35.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.970+0000 7fc08dc00700 1 -- 172.21.15.1:0/2423174465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc088113920 msgr2=0x7fc0881a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:35.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.970+0000 7fc08dc00700 1 --2- 172.21.15.1:0/2423174465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc088113920 0x7fc0881a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:35.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.970+0000 7fc08dc00700 1 -- 172.21.15.1:0/2423174465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc088077890 msgr2=0x7fc08819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:35.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.970+0000 7fc08dc00700 1 --2- 172.21.15.1:0/2423174465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc088077890 0x7fc08819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:35.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.970+0000 7fc08dc00700 1 -- 172.21.15.1:0/2423174465 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0881a4740 con 0x7fc088076f70 2024-04-26T01:49:35.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.970+0000 7fc08d200700 1 --2- 172.21.15.1:0/2423174465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc088077890 0x7fc08819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:49:35.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.970+0000 7fc08dc00700 1 --2- 172.21.15.1:0/2423174465 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc088076f70 0x7fc08819f780 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00e9f0 tx=0x7fc07c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:35.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.970+0000 7fc086000700 1 -- 172.21.15.1:0/2423174465 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c004220 con 0x7fc088076f70 2024-04-26T01:49:35.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.970+0000 7fc090600700 1 -- 172.21.15.1:0/2423174465 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0881a4a30 con 0x7fc088076f70 2024-04-26T01:49:35.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.970+0000 7fc086000700 1 -- 172.21.15.1:0/2423174465 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc07c0043c0 con 0x7fc088076f70 2024-04-26T01:49:35.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.970+0000 7fc086000700 1 -- 172.21.15.1:0/2423174465 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c005020 con 0x7fc088076f70 2024-04-26T01:49:35.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.970+0000 7fc090600700 1 -- 172.21.15.1:0/2423174465 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0881ac2b0 con 0x7fc088076f70 2024-04-26T01:49:35.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.971+0000 7fc090600700 1 -- 172.21.15.1:0/2423174465 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc088069730 con 0x7fc088076f70 2024-04-26T01:49:35.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.973+0000 7fc086000700 1 -- 172.21.15.1:0/2423174465 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fc07c004750 con 0x7fc088076f70 2024-04-26T01:49:35.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.975+0000 7fc086000700 1 --2- 172.21.15.1:0/2423174465 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fc0700a3cc0 0x7fc0700a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:35.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.975+0000 7fc086000700 1 -- 172.21.15.1:0/2423174465 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fc07c0d0330 con 0x7fc088076f70 2024-04-26T01:49:35.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.975+0000 7fc08d200700 1 --2- 172.21.15.1:0/2423174465 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fc0700a3cc0 0x7fc0700a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:35.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.976+0000 7fc08d200700 1 --2- 172.21.15.1:0/2423174465 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fc0700a3cc0 0x7fc0700a6180 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc088103e00 tx=0x7fc074009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:35.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:34.979+0000 7fc086000700 1 -- 172.21.15.1:0/2423174465 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc07c09cc90 con 0x7fc088076f70 2024-04-26T01:49:35.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:35.224+0000 7fc090600700 1 -- 172.21.15.1:0/2423174465 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fc0881a0a90 con 0x7fc088076f70 2024-04-26T01:49:35.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:35.225+0000 7fc086000700 1 -- 172.21.15.1:0/2423174465 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc07c09c590 con 0x7fc088076f70 2024-04-26T01:49:35.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:35.228+0000 7fc090600700 1 -- 172.21.15.1:0/2423174465 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fc0700a3cc0 msgr2=0x7fc0700a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:35.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:35.228+0000 7fc090600700 1 --2- 172.21.15.1:0/2423174465 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fc0700a3cc0 0x7fc0700a6180 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc088103e00 tx=0x7fc074009260 comp rx=0 tx=0).stop 2024-04-26T01:49:35.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:35.228+0000 7fc090600700 1 -- 172.21.15.1:0/2423174465 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc088076f70 msgr2=0x7fc08819f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:35.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:35.228+0000 7fc090600700 1 --2- 172.21.15.1:0/2423174465 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc088076f70 0x7fc08819f780 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00e9f0 tx=0x7fc07c00ed00 comp rx=0 tx=0).stop 2024-04-26T01:49:35.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:35.229+0000 7fc090600700 1 -- 172.21.15.1:0/2423174465 shutdown_connections 2024-04-26T01:49:35.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:35.229+0000 7fc090600700 1 --2- 172.21.15.1:0/2423174465 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fc0700a3cc0 0x7fc0700a6180 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:35.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:35.229+0000 7fc090600700 1 --2- 172.21.15.1:0/2423174465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc088113920 0x7fc0881a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:35.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:35.229+0000 7fc090600700 1 --2- 172.21.15.1:0/2423174465 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc088076f70 0x7fc08819f780 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:35.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:35.229+0000 7fc090600700 1 --2- 172.21.15.1:0/2423174465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc088077890 0x7fc08819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:35.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:35.229+0000 7fc090600700 1 -- 172.21.15.1:0/2423174465 >> 172.21.15.1:0/2423174465 conn(0x7fc088100770 msgr2=0x7fc088102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:35.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:35.229+0000 7fc090600700 1 -- 172.21.15.1:0/2423174465 shutdown_connections 2024-04-26T01:49:35.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:35.229+0000 7fc090600700 1 -- 172.21.15.1:0/2423174465 wait complete. 2024-04-26T01:49:35.341 INFO:teuthology.orchestra.run.smithi001.stdout:261993005116 2024-04-26T01:49:36.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:35 smithi001 ceph-mon[27822]: pgmap v389: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T01:49:36.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:35 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2423174465' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T01:49:36.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:35 smithi097 ceph-mon[32853]: pgmap v389: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T01:49:36.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:35 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/2423174465' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T01:49:36.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:35 smithi173 ceph-mon[32553]: pgmap v389: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T01:49:36.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:35 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/2423174465' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T01:49:36.469 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:38.080 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:37 smithi001 ceph-mon[27822]: pgmap v390: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:38.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:37 smithi097 ceph-mon[32853]: pgmap v390: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:38.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:37 smithi173 ceph-mon[32553]: pgmap v390: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:40.082 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:39 smithi001 ceph-mon[27822]: pgmap v391: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:40.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:39 smithi097 ceph-mon[32853]: pgmap v391: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:40.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:39 smithi173 ceph-mon[32553]: pgmap v391: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:42.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.004+0000 7f98a5a00700 1 -- 172.21.15.1:0/1729175327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98a0074e70 msgr2=0x7f98a0075250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:42.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.004+0000 7f98a5a00700 1 --2- 172.21.15.1:0/1729175327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98a0074e70 0x7f98a0075250 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9890009990 tx=0x7f9890009ca0 comp rx=0 tx=0).stop 2024-04-26T01:49:42.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.004+0000 7f98a5a00700 1 -- 172.21.15.1:0/1729175327 shutdown_connections 2024-04-26T01:49:42.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.004+0000 7f98a5a00700 1 --2- 172.21.15.1:0/1729175327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98a010f1a0 0x7f98a0111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:42.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.004+0000 7f98a5a00700 1 --2- 172.21.15.1:0/1729175327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98a0074e70 0x7f98a0075250 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:42.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.004+0000 7f98a5a00700 1 --2- 172.21.15.1:0/1729175327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98a0075790 0x7f98a010ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:42.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.004+0000 7f98a5a00700 1 -- 172.21.15.1:0/1729175327 >> 172.21.15.1:0/1729175327 conn(0x7f98a00fe490 msgr2=0x7f98a01008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:42.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.005+0000 7f98a5a00700 1 -- 172.21.15.1:0/1729175327 shutdown_connections 2024-04-26T01:49:42.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.005+0000 7f98a5a00700 1 -- 172.21.15.1:0/1729175327 wait complete. 2024-04-26T01:49:42.004 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:41 smithi001 ceph-mon[27822]: pgmap v392: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:42.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.006+0000 7f98a5a00700 1 Processor -- start 2024-04-26T01:49:42.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.007+0000 7f98a5a00700 1 -- start start 2024-04-26T01:49:42.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.007+0000 7f98a5a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98a0074e70 0x7f98a019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:42.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.007+0000 7f98a5a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98a0075790 0x7f98a019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:42.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.007+0000 7f98a5a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98a010f1a0 0x7f98a019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:42.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.007+0000 7f98a5a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f98a0114d80 con 0x7f98a0074e70 2024-04-26T01:49:42.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.007+0000 7f98a5a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f98a0114c00 con 0x7f98a010f1a0 2024-04-26T01:49:42.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.007+0000 7f98a5a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f98a0114f00 con 0x7f98a0075790 2024-04-26T01:49:42.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.007+0000 7f989f400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98a0075790 0x7f98a019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:42.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.007+0000 7f989fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98a0074e70 0x7f98a019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:42.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.007+0000 7f989f400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98a0075790 0x7f98a019b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:44608/0 (socket says 172.21.15.1:44608) 2024-04-26T01:49:42.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.007+0000 7f989f400700 1 -- 172.21.15.1:0/3817964789 learned_addr learned my addr 172.21.15.1:0/3817964789 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:42.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.008+0000 7f98a5000700 1 --2- 172.21.15.1:0/3817964789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98a010f1a0 0x7f98a019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:42.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.008+0000 7f989f400700 1 -- 172.21.15.1:0/3817964789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98a010f1a0 msgr2=0x7f98a019fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:42.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.008+0000 7f989f400700 1 --2- 172.21.15.1:0/3817964789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98a010f1a0 0x7f98a019fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:42.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.008+0000 7f989f400700 1 -- 172.21.15.1:0/3817964789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98a0074e70 msgr2=0x7f98a019b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:42.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.008+0000 7f989f400700 1 --2- 172.21.15.1:0/3817964789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98a0074e70 0x7f98a019b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:42.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.008+0000 7f989f400700 1 -- 172.21.15.1:0/3817964789 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a01a02b0 con 0x7f98a0075790 2024-04-26T01:49:42.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.008+0000 7f989fe00700 1 --2- 172.21.15.1:0/3817964789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98a0074e70 0x7f98a019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:49:42.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.008+0000 7f989f400700 1 --2- 172.21.15.1:0/3817964789 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98a0075790 0x7f98a019b830 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f988c00e9a0 tx=0x7f988c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:42.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.009+0000 7f989cc00700 1 -- 172.21.15.1:0/3817964789 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c00ca10 con 0x7f98a0075790 2024-04-26T01:49:42.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.009+0000 7f989cc00700 1 -- 172.21.15.1:0/3817964789 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f988c00cbb0 con 0x7f98a0075790 2024-04-26T01:49:42.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.009+0000 7f989cc00700 1 -- 172.21.15.1:0/3817964789 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c018680 con 0x7f98a0075790 2024-04-26T01:49:42.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.009+0000 7f98a5a00700 1 -- 172.21.15.1:0/3817964789 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a01a05a0 con 0x7f98a0075790 2024-04-26T01:49:42.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.009+0000 7f98a5a00700 1 -- 172.21.15.1:0/3817964789 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a01a7e20 con 0x7f98a0075790 2024-04-26T01:49:42.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.010+0000 7f98a5a00700 1 -- 172.21.15.1:0/3817964789 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a0069730 con 0x7f98a0075790 2024-04-26T01:49:42.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.012+0000 7f989cc00700 1 -- 172.21.15.1:0/3817964789 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f988c010430 con 0x7f98a0075790 2024-04-26T01:49:42.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.014+0000 7f989cc00700 1 --2- 172.21.15.1:0/3817964789 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f98840a3cc0 0x7f98840a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:42.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.014+0000 7f989cc00700 1 -- 172.21.15.1:0/3817964789 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f988c0d0620 con 0x7f98a0075790 2024-04-26T01:49:42.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.014+0000 7f989fe00700 1 --2- 172.21.15.1:0/3817964789 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f98840a3cc0 0x7f98840a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:42.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.015+0000 7f989fe00700 1 --2- 172.21.15.1:0/3817964789 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f98840a3cc0 0x7f98840a6180 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9890006010 tx=0x7f989000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:42.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.016+0000 7f989cc00700 1 -- 172.21.15.1:0/3817964789 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f988c09cf80 con 0x7f98a0075790 2024-04-26T01:49:42.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:41 smithi097 ceph-mon[32853]: pgmap v392: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:42.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.268+0000 7f98a5a00700 1 -- 172.21.15.1:0/3817964789 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f98a002df80 con 0x7f98a0075790 2024-04-26T01:49:42.267 INFO:teuthology.orchestra.run.smithi001.stdout:77309411422 2024-04-26T01:49:42.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.269+0000 7f989cc00700 1 -- 172.21.15.1:0/3817964789 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f988c09c880 con 0x7f98a0075790 2024-04-26T01:49:42.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.271+0000 7f98a5a00700 1 -- 172.21.15.1:0/3817964789 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f98840a3cc0 msgr2=0x7f98840a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:42.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.271+0000 7f98a5a00700 1 --2- 172.21.15.1:0/3817964789 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f98840a3cc0 0x7f98840a6180 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9890006010 tx=0x7f989000b260 comp rx=0 tx=0).stop 2024-04-26T01:49:42.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.271+0000 7f98a5a00700 1 -- 172.21.15.1:0/3817964789 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98a0075790 msgr2=0x7f98a019b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:42.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.271+0000 7f98a5a00700 1 --2- 172.21.15.1:0/3817964789 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98a0075790 0x7f98a019b830 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f988c00e9a0 tx=0x7f988c00ecb0 comp rx=0 tx=0).stop 2024-04-26T01:49:42.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.272+0000 7f98a5a00700 1 -- 172.21.15.1:0/3817964789 shutdown_connections 2024-04-26T01:49:42.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.272+0000 7f98a5a00700 1 --2- 172.21.15.1:0/3817964789 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f98840a3cc0 0x7f98840a6180 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:42.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.272+0000 7f98a5a00700 1 --2- 172.21.15.1:0/3817964789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98a010f1a0 0x7f98a019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:42.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.272+0000 7f98a5a00700 1 --2- 172.21.15.1:0/3817964789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98a0074e70 0x7f98a019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:42.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.272+0000 7f98a5a00700 1 --2- 172.21.15.1:0/3817964789 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98a0075790 0x7f98a019b830 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:42.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.272+0000 7f98a5a00700 1 -- 172.21.15.1:0/3817964789 >> 172.21.15.1:0/3817964789 conn(0x7f98a00fe490 msgr2=0x7f98a00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:42.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.272+0000 7f98a5a00700 1 -- 172.21.15.1:0/3817964789 shutdown_connections 2024-04-26T01:49:42.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:42.272+0000 7f98a5a00700 1 -- 172.21.15.1:0/3817964789 wait complete. 2024-04-26T01:49:42.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:41 smithi173 ceph-mon[32553]: pgmap v392: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:42.993 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005111 got 261993005116 for osd.8 2024-04-26T01:49:42.993 DEBUG:teuthology.parallel:result is None 2024-04-26T01:49:43.220 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:42 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3817964789' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T01:49:43.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:42 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3817964789' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T01:49:43.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:42 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3817964789' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T01:49:44.188 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:43 smithi001 ceph-mon[27822]: pgmap v393: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:44.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:43 smithi097 ceph-mon[32853]: pgmap v393: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:44.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:43 smithi173 ceph-mon[32553]: pgmap v393: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:45.181 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:46.107 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:45 smithi001 ceph-mon[27822]: pgmap v394: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:46.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:45 smithi097 ceph-mon[32853]: pgmap v394: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:46.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:45 smithi173 ceph-mon[32553]: pgmap v394: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:47.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:46 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/2537530812' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T01:49:47.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:46 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/2537530812' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T01:49:47.329 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:46 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2537530812' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T01:49:47.387 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411417 got 77309411422 for osd.2 2024-04-26T01:49:47.387 DEBUG:teuthology.parallel:result is None 2024-04-26T01:49:47.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.107+0000 7f1ed2c00700 1 -- 172.21.15.1:0/2515986286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ecc10e3b0 msgr2=0x7f1ecc115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:47.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.107+0000 7f1ed2c00700 1 --2- 172.21.15.1:0/2515986286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ecc10e3b0 0x7f1ecc115f80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400b0d0 tx=0x7f1ec400b3e0 comp rx=0 tx=0).stop 2024-04-26T01:49:47.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.108+0000 7f1ed2c00700 1 -- 172.21.15.1:0/2515986286 shutdown_connections 2024-04-26T01:49:47.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.108+0000 7f1ed2c00700 1 --2- 172.21.15.1:0/2515986286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ecc10e3b0 0x7f1ecc115f80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:47.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.108+0000 7f1ed2c00700 1 --2- 172.21.15.1:0/2515986286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ecc1092d0 0x7f1ecc1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:47.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.108+0000 7f1ed2c00700 1 --2- 172.21.15.1:0/2515986286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ecc109c80 0x7f1ecc10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:47.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.108+0000 7f1ed2c00700 1 -- 172.21.15.1:0/2515986286 >> 172.21.15.1:0/2515986286 conn(0x7f1ecc076210 msgr2=0x7f1ecc107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:47.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.108+0000 7f1ed2c00700 1 -- 172.21.15.1:0/2515986286 shutdown_connections 2024-04-26T01:49:47.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.108+0000 7f1ed2c00700 1 -- 172.21.15.1:0/2515986286 wait complete. 2024-04-26T01:49:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.109+0000 7f1ed2c00700 1 Processor -- start 2024-04-26T01:49:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.109+0000 7f1ed2c00700 1 -- start start 2024-04-26T01:49:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.109+0000 7f1ed2c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ecc1092d0 0x7f1ecc19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.109+0000 7f1ed2c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ecc109c80 0x7f1ecc19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.109+0000 7f1ed2c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ecc10e3b0 0x7f1ecc1a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.109+0000 7f1ed2c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc1192a0 con 0x7f1ecc109c80 2024-04-26T01:49:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.109+0000 7f1ed2c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc119120 con 0x7f1ecc10e3b0 2024-04-26T01:49:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.109+0000 7f1ed2c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc119420 con 0x7f1ecc1092d0 2024-04-26T01:49:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.110+0000 7f1ed0c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ecc10e3b0 0x7f1ecc1a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.110+0000 7f1ecbe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ecc1092d0 0x7f1ecc19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.110+0000 7f1ed0c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ecc10e3b0 0x7f1ecc1a3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:52168/0 (socket says 172.21.15.1:52168) 2024-04-26T01:49:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.110+0000 7f1ed0c00700 1 -- 172.21.15.1:0/2537530812 learned_addr learned my addr 172.21.15.1:0/2537530812 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.110+0000 7f1ecb400700 1 --2- 172.21.15.1:0/2537530812 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ecc109c80 0x7f1ecc19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.110+0000 7f1ed0c00700 1 -- 172.21.15.1:0/2537530812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ecc1092d0 msgr2=0x7f1ecc19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.110+0000 7f1ed0c00700 1 --2- 172.21.15.1:0/2537530812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ecc1092d0 0x7f1ecc19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.110+0000 7f1ed0c00700 1 -- 172.21.15.1:0/2537530812 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ecc109c80 msgr2=0x7f1ecc19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.110+0000 7f1ed0c00700 1 --2- 172.21.15.1:0/2537530812 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ecc109c80 0x7f1ecc19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.110+0000 7f1ed0c00700 1 -- 172.21.15.1:0/2537530812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ecc1a46e0 con 0x7f1ecc10e3b0 2024-04-26T01:49:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.110+0000 7f1ecbe00700 1 --2- 172.21.15.1:0/2537530812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ecc1092d0 0x7f1ecc19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:49:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.110+0000 7f1ecb400700 1 --2- 172.21.15.1:0/2537530812 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ecc109c80 0x7f1ecc19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:49:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.111+0000 7f1ed0c00700 1 --2- 172.21.15.1:0/2537530812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ecc10e3b0 0x7f1ecc1a3fa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40079a0 tx=0x7f1ec40079d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.111+0000 7f1ec8c00700 1 -- 172.21.15.1:0/2537530812 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec401f070 con 0x7f1ecc10e3b0 2024-04-26T01:49:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.111+0000 7f1ed2c00700 1 -- 172.21.15.1:0/2537530812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ecc1a4970 con 0x7f1ecc10e3b0 2024-04-26T01:49:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.111+0000 7f1ec8c00700 1 -- 172.21.15.1:0/2537530812 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ec4003c00 con 0x7f1ecc10e3b0 2024-04-26T01:49:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.111+0000 7f1ec8c00700 1 -- 172.21.15.1:0/2537530812 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec401b430 con 0x7f1ecc10e3b0 2024-04-26T01:49:47.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.111+0000 7f1ed2c00700 1 -- 172.21.15.1:0/2537530812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ecc1ac250 con 0x7f1ecc10e3b0 2024-04-26T01:49:47.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.112+0000 7f1ed2c00700 1 -- 172.21.15.1:0/2537530812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ecc069730 con 0x7f1ecc10e3b0 2024-04-26T01:49:47.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.115+0000 7f1ec8c00700 1 -- 172.21.15.1:0/2537530812 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f1ec4022070 con 0x7f1ecc10e3b0 2024-04-26T01:49:47.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.116+0000 7f1ec8c00700 1 --2- 172.21.15.1:0/2537530812 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f1eb00a3c40 0x7f1eb00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:47.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.116+0000 7f1ec8c00700 1 -- 172.21.15.1:0/2537530812 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f1ec400f020 con 0x7f1ecc10e3b0 2024-04-26T01:49:47.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.116+0000 7f1ecbe00700 1 --2- 172.21.15.1:0/2537530812 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f1eb00a3c40 0x7f1eb00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:47.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.117+0000 7f1ecbe00700 1 --2- 172.21.15.1:0/2537530812 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f1eb00a3c40 0x7f1eb00a6100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc011490 tx=0x7f1ebc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:47.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.118+0000 7f1ec8c00700 1 -- 172.21.15.1:0/2537530812 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ec4029020 con 0x7f1ecc10e3b0 2024-04-26T01:49:47.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.361+0000 7f1ed2c00700 1 -- 172.21.15.1:0/2537530812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1ecc02df80 con 0x7f1ecc10e3b0 2024-04-26T01:49:47.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.361+0000 7f1ec8c00700 1 -- 172.21.15.1:0/2537530812 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1ec40a4030 con 0x7f1ecc10e3b0 2024-04-26T01:49:47.576 INFO:teuthology.orchestra.run.smithi001.stdout:34359738473 2024-04-26T01:49:47.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.365+0000 7f1ed2c00700 1 -- 172.21.15.1:0/2537530812 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f1eb00a3c40 msgr2=0x7f1eb00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:47.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.365+0000 7f1ed2c00700 1 --2- 172.21.15.1:0/2537530812 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f1eb00a3c40 0x7f1eb00a6100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc011490 tx=0x7f1ebc009260 comp rx=0 tx=0).stop 2024-04-26T01:49:47.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.365+0000 7f1ed2c00700 1 -- 172.21.15.1:0/2537530812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ecc10e3b0 msgr2=0x7f1ecc1a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:47.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.365+0000 7f1ed2c00700 1 --2- 172.21.15.1:0/2537530812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ecc10e3b0 0x7f1ecc1a3fa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40079a0 tx=0x7f1ec40079d0 comp rx=0 tx=0).stop 2024-04-26T01:49:47.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.365+0000 7f1ed2c00700 1 -- 172.21.15.1:0/2537530812 shutdown_connections 2024-04-26T01:49:47.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.365+0000 7f1ed2c00700 1 --2- 172.21.15.1:0/2537530812 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f1eb00a3c40 0x7f1eb00a6100 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:47.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.365+0000 7f1ed2c00700 1 --2- 172.21.15.1:0/2537530812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ecc10e3b0 0x7f1ecc1a3fa0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:47.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.365+0000 7f1ed2c00700 1 --2- 172.21.15.1:0/2537530812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ecc1092d0 0x7f1ecc19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:47.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.365+0000 7f1ed2c00700 1 --2- 172.21.15.1:0/2537530812 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ecc109c80 0x7f1ecc19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:47.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.365+0000 7f1ed2c00700 1 -- 172.21.15.1:0/2537530812 >> 172.21.15.1:0/2537530812 conn(0x7f1ecc076210 msgr2=0x7f1ecc076750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:47.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.365+0000 7f1ed2c00700 1 -- 172.21.15.1:0/2537530812 shutdown_connections 2024-04-26T01:49:47.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:46.365+0000 7f1ed2c00700 1 -- 172.21.15.1:0/2537530812 wait complete. 2024-04-26T01:49:48.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:47 smithi097 ceph-mon[32853]: pgmap v395: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:48.285 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:47 smithi001 ceph-mon[27822]: pgmap v395: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:48.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:47 smithi173 ceph-mon[32553]: pgmap v395: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:48.780 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738469 got 34359738473 for osd.0 2024-04-26T01:49:48.780 DEBUG:teuthology.parallel:result is None 2024-04-26T01:49:49.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:48 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/367435180' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T01:49:49.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:48 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/367435180' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T01:49:49.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:48 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/367435180' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T01:49:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.447+0000 7fd8bb000700 1 -- 172.21.15.1:0/2195574946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b4108a50 msgr2=0x7fd8b4108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.447+0000 7fd8bb000700 1 --2- 172.21.15.1:0/2195574946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b4108a50 0x7fd8b4108e30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009990 tx=0x7fd8a4009ca0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.447+0000 7fd8bb000700 1 -- 172.21.15.1:0/2195574946 shutdown_connections 2024-04-26T01:49:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.447+0000 7fd8bb000700 1 --2- 172.21.15.1:0/2195574946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8b4103450 0x7fd8b410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.447+0000 7fd8bb000700 1 --2- 172.21.15.1:0/2195574946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b4108a50 0x7fd8b4108e30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.448+0000 7fd8bb000700 1 --2- 172.21.15.1:0/2195574946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8b4102a90 0x7fd8b4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.448+0000 7fd8bb000700 1 -- 172.21.15.1:0/2195574946 >> 172.21.15.1:0/2195574946 conn(0x7fd8b4079ba0 msgr2=0x7fd8b4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.448+0000 7fd8bb000700 1 -- 172.21.15.1:0/2195574946 shutdown_connections 2024-04-26T01:49:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.448+0000 7fd8bb000700 1 -- 172.21.15.1:0/2195574946 wait complete. 2024-04-26T01:49:49.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.449+0000 7fd8bb000700 1 Processor -- start 2024-04-26T01:49:49.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.449+0000 7fd8bb000700 1 -- start start 2024-04-26T01:49:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.449+0000 7fd8bb000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b4102a90 0x7fd8b419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.449+0000 7fd8bb000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8b4103450 0x7fd8b419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.449+0000 7fd8bb000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8b4108a50 0x7fd8b419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.450+0000 7fd8bb000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b4112c60 con 0x7fd8b4102a90 2024-04-26T01:49:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.450+0000 7fd8bb000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b4112ae0 con 0x7fd8b4103450 2024-04-26T01:49:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.450+0000 7fd8bb000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b4112de0 con 0x7fd8b4108a50 2024-04-26T01:49:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.450+0000 7fd8b3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b4102a90 0x7fd8b419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.450+0000 7fd8b3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b4102a90 0x7fd8b419b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55092/0 (socket says 172.21.15.1:55092) 2024-04-26T01:49:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.450+0000 7fd8b3e00700 1 -- 172.21.15.1:0/4218650107 learned_addr learned my addr 172.21.15.1:0/4218650107 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.450+0000 7fd8b3400700 1 --2- 172.21.15.1:0/4218650107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8b4103450 0x7fd8b419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:49.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.450+0000 7fd8b9000700 1 --2- 172.21.15.1:0/4218650107 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8b4108a50 0x7fd8b419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:49.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.450+0000 7fd8b3e00700 1 -- 172.21.15.1:0/4218650107 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8b4108a50 msgr2=0x7fd8b419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:49.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.450+0000 7fd8b3e00700 1 --2- 172.21.15.1:0/4218650107 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8b4108a50 0x7fd8b419fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.450+0000 7fd8b3e00700 1 -- 172.21.15.1:0/4218650107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8b4103450 msgr2=0x7fd8b419b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:49.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.450+0000 7fd8b3e00700 1 --2- 172.21.15.1:0/4218650107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8b4103450 0x7fd8b419b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.450+0000 7fd8b3e00700 1 -- 172.21.15.1:0/4218650107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b41a03f0 con 0x7fd8b4102a90 2024-04-26T01:49:49.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.450+0000 7fd8b9000700 1 --2- 172.21.15.1:0/4218650107 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8b4108a50 0x7fd8b419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:49:49.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.450+0000 7fd8b3400700 1 --2- 172.21.15.1:0/4218650107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8b4103450 0x7fd8b419b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:49:49.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.451+0000 7fd8b3e00700 1 --2- 172.21.15.1:0/4218650107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b4102a90 0x7fd8b419b3e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4000c00 tx=0x7fd8a40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:49.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.451+0000 7fd8b0c00700 1 -- 172.21.15.1:0/4218650107 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4027070 con 0x7fd8b4102a90 2024-04-26T01:49:49.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.451+0000 7fd8b0c00700 1 -- 172.21.15.1:0/4218650107 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8a4022470 con 0x7fd8b4102a90 2024-04-26T01:49:49.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.451+0000 7fd8bb000700 1 -- 172.21.15.1:0/4218650107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b41a0680 con 0x7fd8b4102a90 2024-04-26T01:49:49.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.451+0000 7fd8bb000700 1 -- 172.21.15.1:0/4218650107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b41a7f60 con 0x7fd8b4102a90 2024-04-26T01:49:49.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.451+0000 7fd8b0c00700 1 -- 172.21.15.1:0/4218650107 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4017650 con 0x7fd8b4102a90 2024-04-26T01:49:49.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.452+0000 7fd8bb000700 1 -- 172.21.15.1:0/4218650107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b4069730 con 0x7fd8b4102a90 2024-04-26T01:49:49.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.456+0000 7fd8b0c00700 1 -- 172.21.15.1:0/4218650107 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fd8a402a070 con 0x7fd8b4102a90 2024-04-26T01:49:49.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.457+0000 7fd8b0c00700 1 --2- 172.21.15.1:0/4218650107 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fd8980a3cc0 0x7fd8980a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:49.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.457+0000 7fd8b0c00700 1 -- 172.21.15.1:0/4218650107 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fd8a40d1200 con 0x7fd8b4102a90 2024-04-26T01:49:49.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.458+0000 7fd8b3400700 1 --2- 172.21.15.1:0/4218650107 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fd8980a3cc0 0x7fd8980a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:49.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.458+0000 7fd8b3400700 1 --2- 172.21.15.1:0/4218650107 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fd8980a3cc0 0x7fd8980a6180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd8b410fcf0 tx=0x7fd8a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:49.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.461+0000 7fd8b0c00700 1 -- 172.21.15.1:0/4218650107 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd8a4013070 con 0x7fd8b4102a90 2024-04-26T01:49:49.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.618+0000 7fb6cc000700 1 -- 172.21.15.1:0/2240548089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6c4109fc0 msgr2=0x7fb6c4111b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:49.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.618+0000 7fb6cc000700 1 --2- 172.21.15.1:0/2240548089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6c4109fc0 0x7fb6c4111b90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc00b0d0 tx=0x7fb6bc00b3e0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.618+0000 7fb6cc000700 1 -- 172.21.15.1:0/2240548089 shutdown_connections 2024-04-26T01:49:49.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.618+0000 7fb6cc000700 1 --2- 172.21.15.1:0/2240548089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6c4109fc0 0x7fb6c4111b90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.618+0000 7fb6cc000700 1 --2- 172.21.15.1:0/2240548089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6c4104ee0 0x7fb6c41052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.618+0000 7fb6cc000700 1 --2- 172.21.15.1:0/2240548089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6c4105890 0x7fb6c41098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.618+0000 7fb6cc000700 1 -- 172.21.15.1:0/2240548089 >> 172.21.15.1:0/2240548089 conn(0x7fb6c4100770 msgr2=0x7fb6c4102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:49.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.618+0000 7fb6cc000700 1 -- 172.21.15.1:0/2240548089 shutdown_connections 2024-04-26T01:49:49.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.618+0000 7fb6cc000700 1 -- 172.21.15.1:0/2240548089 wait complete. 2024-04-26T01:49:49.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.620+0000 7fb6cc000700 1 Processor -- start 2024-04-26T01:49:49.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.620+0000 7fb6cc000700 1 -- start start 2024-04-26T01:49:49.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.620+0000 7fb6cc000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6c4104ee0 0x7fb6c419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:49.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.620+0000 7fb6cc000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6c4105890 0x7fb6c419b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:49.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.620+0000 7fb6cc000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6c4109fc0 0x7fb6c419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:49.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.620+0000 7fb6cc000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c4114ea0 con 0x7fb6c4105890 2024-04-26T01:49:49.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.620+0000 7fb6cc000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c4114d20 con 0x7fb6c4109fc0 2024-04-26T01:49:49.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.620+0000 7fb6cc000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c4115020 con 0x7fb6c4104ee0 2024-04-26T01:49:49.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.621+0000 7fb6c9600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6c4104ee0 0x7fb6c419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:49.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.621+0000 7fb6c9600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6c4104ee0 0x7fb6c419b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:38328/0 (socket says 172.21.15.1:38328) 2024-04-26T01:49:49.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.621+0000 7fb6c9600700 1 -- 172.21.15.1:0/367435180 learned_addr learned my addr 172.21.15.1:0/367435180 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:49.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.621+0000 7fb6c8c00700 1 --2- 172.21.15.1:0/367435180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6c4105890 0x7fb6c419b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:49.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.621+0000 7fb6ca000700 1 --2- 172.21.15.1:0/367435180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6c4109fc0 0x7fb6c419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.621+0000 7fb6c9600700 1 -- 172.21.15.1:0/367435180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6c4109fc0 msgr2=0x7fb6c419fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.621+0000 7fb6c9600700 1 --2- 172.21.15.1:0/367435180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6c4109fc0 0x7fb6c419fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.621+0000 7fb6c9600700 1 -- 172.21.15.1:0/367435180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6c4105890 msgr2=0x7fb6c419b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.621+0000 7fb6c9600700 1 --2- 172.21.15.1:0/367435180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6c4105890 0x7fb6c419b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.621+0000 7fb6c9600700 1 -- 172.21.15.1:0/367435180 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c41a0330 con 0x7fb6c4104ee0 2024-04-26T01:49:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.622+0000 7fb6c9600700 1 --2- 172.21.15.1:0/367435180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6c4104ee0 0x7fb6c419b370 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800e9a0 tx=0x7fb6b800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.622+0000 7fb6c2000700 1 -- 172.21.15.1:0/367435180 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b800ca10 con 0x7fb6c4104ee0 2024-04-26T01:49:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.622+0000 7fb6c2000700 1 -- 172.21.15.1:0/367435180 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6b800cbb0 con 0x7fb6c4104ee0 2024-04-26T01:49:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.622+0000 7fb6c2000700 1 -- 172.21.15.1:0/367435180 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8018680 con 0x7fb6c4104ee0 2024-04-26T01:49:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.622+0000 7fb6cc000700 1 -- 172.21.15.1:0/367435180 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c41a0620 con 0x7fb6c4104ee0 2024-04-26T01:49:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.622+0000 7fb6cc000700 1 -- 172.21.15.1:0/367435180 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c41a7ea0 con 0x7fb6c4104ee0 2024-04-26T01:49:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.623+0000 7fb6cc000700 1 -- 172.21.15.1:0/367435180 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c4069730 con 0x7fb6c4104ee0 2024-04-26T01:49:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.625+0000 7fb6c2000700 1 -- 172.21.15.1:0/367435180 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fb6b8010430 con 0x7fb6c4104ee0 2024-04-26T01:49:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.627+0000 7fb6c2000700 1 --2- 172.21.15.1:0/367435180 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb6ac0a3cc0 0x7fb6ac0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:49.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.627+0000 7fb6c2000700 1 -- 172.21.15.1:0/367435180 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fb6b80d0570 con 0x7fb6c4104ee0 2024-04-26T01:49:49.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.627+0000 7fb6c8c00700 1 --2- 172.21.15.1:0/367435180 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb6ac0a3cc0 0x7fb6ac0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:49.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.628+0000 7fb6c8c00700 1 --2- 172.21.15.1:0/367435180 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb6ac0a3cc0 0x7fb6ac0a6180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009680 tx=0x7fb6b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:49.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.631+0000 7fb6c2000700 1 -- 172.21.15.1:0/367435180 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6b809ced0 con 0x7fb6c4104ee0 2024-04-26T01:49:49.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.869+0000 7fb6cc000700 1 -- 172.21.15.1:0/367435180 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb6c402df80 con 0x7fb6c4104ee0 2024-04-26T01:49:49.470 INFO:teuthology.orchestra.run.smithi001.stdout:55834574949 2024-04-26T01:49:49.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.871+0000 7fb6c2000700 1 -- 172.21.15.1:0/367435180 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb6b809c7d0 con 0x7fb6c4104ee0 2024-04-26T01:49:49.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.874+0000 7fb6cc000700 1 -- 172.21.15.1:0/367435180 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb6ac0a3cc0 msgr2=0x7fb6ac0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:49.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.874+0000 7fb6cc000700 1 --2- 172.21.15.1:0/367435180 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb6ac0a3cc0 0x7fb6ac0a6180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009680 tx=0x7fb6b0009260 comp rx=0 tx=0).stop 2024-04-26T01:49:49.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.874+0000 7fb6cc000700 1 -- 172.21.15.1:0/367435180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6c4104ee0 msgr2=0x7fb6c419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:49.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.874+0000 7fb6cc000700 1 --2- 172.21.15.1:0/367435180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6c4104ee0 0x7fb6c419b370 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800e9a0 tx=0x7fb6b800ecb0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.875+0000 7fb6cc000700 1 -- 172.21.15.1:0/367435180 shutdown_connections 2024-04-26T01:49:49.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.875+0000 7fb6cc000700 1 --2- 172.21.15.1:0/367435180 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb6ac0a3cc0 0x7fb6ac0a6180 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.875+0000 7fb6cc000700 1 --2- 172.21.15.1:0/367435180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6c4109fc0 0x7fb6c419fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.875+0000 7fb6cc000700 1 --2- 172.21.15.1:0/367435180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6c4104ee0 0x7fb6c419b370 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.875+0000 7fb6cc000700 1 --2- 172.21.15.1:0/367435180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6c4105890 0x7fb6c419b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.875+0000 7fb6cc000700 1 -- 172.21.15.1:0/367435180 >> 172.21.15.1:0/367435180 conn(0x7fb6c4100770 msgr2=0x7fb6c41012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:49.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.875+0000 7fb6cc000700 1 -- 172.21.15.1:0/367435180 shutdown_connections 2024-04-26T01:49:49.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:48.875+0000 7fb6cc000700 1 -- 172.21.15.1:0/367435180 wait complete. 2024-04-26T01:49:49.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.701+0000 7fd8bb000700 1 -- 172.21.15.1:0/4218650107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd8b41a8190 con 0x7fd8b4102a90 2024-04-26T01:49:49.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.702+0000 7fd8b0c00700 1 -- 172.21.15.1:0/4218650107 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd8a409dbf0 con 0x7fd8b4102a90 2024-04-26T01:49:49.700 INFO:teuthology.orchestra.run.smithi001.stdout:167503724623 2024-04-26T01:49:49.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.706+0000 7fd8bb000700 1 -- 172.21.15.1:0/4218650107 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fd8980a3cc0 msgr2=0x7fd8980a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:49.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.706+0000 7fd8bb000700 1 --2- 172.21.15.1:0/4218650107 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fd8980a3cc0 0x7fd8980a6180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd8b410fcf0 tx=0x7fd8a0009260 comp rx=0 tx=0).stop 2024-04-26T01:49:49.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.706+0000 7fd8bb000700 1 -- 172.21.15.1:0/4218650107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b4102a90 msgr2=0x7fd8b419b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:49.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.706+0000 7fd8bb000700 1 --2- 172.21.15.1:0/4218650107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b4102a90 0x7fd8b419b3e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4000c00 tx=0x7fd8a40057c0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.706+0000 7fd8bb000700 1 -- 172.21.15.1:0/4218650107 shutdown_connections 2024-04-26T01:49:49.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.706+0000 7fd8bb000700 1 --2- 172.21.15.1:0/4218650107 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fd8980a3cc0 0x7fd8980a6180 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.706+0000 7fd8bb000700 1 --2- 172.21.15.1:0/4218650107 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8b4108a50 0x7fd8b419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.706+0000 7fd8bb000700 1 --2- 172.21.15.1:0/4218650107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b4102a90 0x7fd8b419b3e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.706+0000 7fd8bb000700 1 --2- 172.21.15.1:0/4218650107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8b4103450 0x7fd8b419b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:49.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.706+0000 7fd8bb000700 1 -- 172.21.15.1:0/4218650107 >> 172.21.15.1:0/4218650107 conn(0x7fd8b4079ba0 msgr2=0x7fd8b41015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:49.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.706+0000 7fd8bb000700 1 -- 172.21.15.1:0/4218650107 shutdown_connections 2024-04-26T01:49:49.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:49.706+0000 7fd8bb000700 1 -- 172.21.15.1:0/4218650107 wait complete. 2024-04-26T01:49:50.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:49 smithi097 ceph-mon[32853]: pgmap v396: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:50.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:49 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/4218650107' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T01:49:50.248 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:49 smithi001 ceph-mon[27822]: pgmap v396: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:50.249 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:49 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/4218650107' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T01:49:50.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:49 smithi173 ceph-mon[32553]: pgmap v396: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:50.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:49 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/4218650107' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T01:49:51.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.968+0000 7f1851600700 1 -- 172.21.15.1:0/1129086635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f184c103450 msgr2=0x7f184c10f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:51.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.968+0000 7f1851600700 1 --2- 172.21.15.1:0/1129086635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f184c103450 0x7f184c10f8b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1844009990 tx=0x7f1844009ca0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.968+0000 7f1851600700 1 -- 172.21.15.1:0/1129086635 shutdown_connections 2024-04-26T01:49:51.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.968+0000 7f1851600700 1 --2- 172.21.15.1:0/1129086635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f184c103450 0x7f184c10f8b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.968+0000 7f1851600700 1 --2- 172.21.15.1:0/1129086635 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f184c108a50 0x7f184c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.968+0000 7f1851600700 1 --2- 172.21.15.1:0/1129086635 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f184c102a90 0x7f184c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.968+0000 7f1851600700 1 -- 172.21.15.1:0/1129086635 >> 172.21.15.1:0/1129086635 conn(0x7f184c079ba0 msgr2=0x7f184c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:51.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.968+0000 7f1851600700 1 -- 172.21.15.1:0/1129086635 shutdown_connections 2024-04-26T01:49:51.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.968+0000 7f1851600700 1 -- 172.21.15.1:0/1129086635 wait complete. 2024-04-26T01:49:51.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.969+0000 7f1851600700 1 Processor -- start 2024-04-26T01:49:51.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.970+0000 7f1851600700 1 -- start start 2024-04-26T01:49:51.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.970+0000 7f1851600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f184c102a90 0x7f184c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:51.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.970+0000 7f1851600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f184c103450 0x7f184c075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:51.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.970+0000 7f1851600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f184c108a50 0x7f184c0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:51.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.970+0000 7f1851600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f184c112c50 con 0x7f184c102a90 2024-04-26T01:49:51.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.970+0000 7f1851600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f184c112ad0 con 0x7f184c108a50 2024-04-26T01:49:51.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.970+0000 7f1851600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f184c112dd0 con 0x7f184c103450 2024-04-26T01:49:51.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.970+0000 7f184aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f184c102a90 0x7f184c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:51.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.970+0000 7f184aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f184c102a90 0x7f184c074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55106/0 (socket says 172.21.15.1:55106) 2024-04-26T01:49:51.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.970+0000 7f184aa00700 1 -- 172.21.15.1:0/721648192 learned_addr learned my addr 172.21.15.1:0/721648192 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:51.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.971+0000 7f1842000700 1 --2- 172.21.15.1:0/721648192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f184c103450 0x7f184c075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:51.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.971+0000 7f184b400700 1 --2- 172.21.15.1:0/721648192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f184c108a50 0x7f184c0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:51.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.971+0000 7f184aa00700 1 -- 172.21.15.1:0/721648192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f184c103450 msgr2=0x7f184c075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:51.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.971+0000 7f184aa00700 1 --2- 172.21.15.1:0/721648192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f184c103450 0x7f184c075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.971+0000 7f184aa00700 1 -- 172.21.15.1:0/721648192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f184c108a50 msgr2=0x7f184c0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:51.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.971+0000 7f184aa00700 1 --2- 172.21.15.1:0/721648192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f184c108a50 0x7f184c0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.971+0000 7f184aa00700 1 -- 172.21.15.1:0/721648192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184c1a7660 con 0x7f184c102a90 2024-04-26T01:49:51.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.971+0000 7f184b400700 1 --2- 172.21.15.1:0/721648192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f184c108a50 0x7f184c0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:49:51.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.971+0000 7f184aa00700 1 --2- 172.21.15.1:0/721648192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f184c102a90 0x7f184c074d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f183c00e9a0 tx=0x7f183c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:51.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.971+0000 7f1843e00700 1 -- 172.21.15.1:0/721648192 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c00ca10 con 0x7f184c102a90 2024-04-26T01:49:51.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.972+0000 7f1843e00700 1 -- 172.21.15.1:0/721648192 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f183c00cbb0 con 0x7f184c102a90 2024-04-26T01:49:51.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.972+0000 7f1851600700 1 -- 172.21.15.1:0/721648192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184c1a7950 con 0x7f184c102a90 2024-04-26T01:49:51.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.972+0000 7f1843e00700 1 -- 172.21.15.1:0/721648192 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c018680 con 0x7f184c102a90 2024-04-26T01:49:51.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.972+0000 7f1851600700 1 -- 172.21.15.1:0/721648192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f184c1a7e90 con 0x7f184c102a90 2024-04-26T01:49:51.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.973+0000 7f1851600700 1 -- 172.21.15.1:0/721648192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f184c069730 con 0x7f184c102a90 2024-04-26T01:49:51.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.975+0000 7f1843e00700 1 -- 172.21.15.1:0/721648192 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f183c010430 con 0x7f184c102a90 2024-04-26T01:49:51.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.976+0000 7f1843e00700 1 --2- 172.21.15.1:0/721648192 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f18300a3c40 0x7f18300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:51.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.976+0000 7f1843e00700 1 -- 172.21.15.1:0/721648192 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f183c0d0520 con 0x7f184c102a90 2024-04-26T01:49:51.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.977+0000 7f1842000700 1 --2- 172.21.15.1:0/721648192 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f18300a3c40 0x7f18300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:51.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.977+0000 7f1842000700 1 --2- 172.21.15.1:0/721648192 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f18300a3c40 0x7f18300a6100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f184c07a180 tx=0x7f183400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:51.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:50.981+0000 7f1843e00700 1 -- 172.21.15.1:0/721648192 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f183c09cee0 con 0x7f184c102a90 2024-04-26T01:49:51.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.229+0000 7f1851600700 1 -- 172.21.15.1:0/721648192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f184c02e140 con 0x7f184c102a90 2024-04-26T01:49:51.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.230+0000 7f1843e00700 1 -- 172.21.15.1:0/721648192 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f183c09c7e0 con 0x7f184c102a90 2024-04-26T01:49:51.228 INFO:teuthology.orchestra.run.smithi001.stdout:201863462984 2024-04-26T01:49:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.232+0000 7f1851600700 1 -- 172.21.15.1:0/721648192 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f18300a3c40 msgr2=0x7f18300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.232+0000 7f1851600700 1 --2- 172.21.15.1:0/721648192 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f18300a3c40 0x7f18300a6100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f184c07a180 tx=0x7f183400b040 comp rx=0 tx=0).stop 2024-04-26T01:49:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.232+0000 7f1851600700 1 -- 172.21.15.1:0/721648192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f184c102a90 msgr2=0x7f184c074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.232+0000 7f1851600700 1 --2- 172.21.15.1:0/721648192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f184c102a90 0x7f184c074d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f183c00e9a0 tx=0x7f183c00ecb0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.233+0000 7f1851600700 1 -- 172.21.15.1:0/721648192 shutdown_connections 2024-04-26T01:49:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.233+0000 7f1851600700 1 --2- 172.21.15.1:0/721648192 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f18300a3c40 0x7f18300a6100 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.233+0000 7f1851600700 1 --2- 172.21.15.1:0/721648192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f184c108a50 0x7f184c0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.233+0000 7f1851600700 1 --2- 172.21.15.1:0/721648192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f184c102a90 0x7f184c074d40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.233+0000 7f1851600700 1 --2- 172.21.15.1:0/721648192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f184c103450 0x7f184c075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.233+0000 7f1851600700 1 -- 172.21.15.1:0/721648192 >> 172.21.15.1:0/721648192 conn(0x7f184c079ba0 msgr2=0x7f184c101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.233+0000 7f1851600700 1 -- 172.21.15.1:0/721648192 shutdown_connections 2024-04-26T01:49:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.233+0000 7f1851600700 1 -- 172.21.15.1:0/721648192 wait complete. 2024-04-26T01:49:51.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.398+0000 7fb834e00700 1 -- 172.21.15.1:0/3053218579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb830101400 msgr2=0x7fb830101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:51.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.398+0000 7fb834e00700 1 --2- 172.21.15.1:0/3053218579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb830101400 0x7fb830101860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009990 tx=0x7fb81c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.398+0000 7fb834e00700 1 -- 172.21.15.1:0/3053218579 shutdown_connections 2024-04-26T01:49:51.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.398+0000 7fb834e00700 1 --2- 172.21.15.1:0/3053218579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb830101da0 0x7fb830111b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.398+0000 7fb834e00700 1 --2- 172.21.15.1:0/3053218579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8301038b0 0x7fb830103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.398+0000 7fb834e00700 1 --2- 172.21.15.1:0/3053218579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb830101400 0x7fb830101860 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.398+0000 7fb834e00700 1 -- 172.21.15.1:0/3053218579 >> 172.21.15.1:0/3053218579 conn(0x7fb830076c60 msgr2=0x7fb830077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:51.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.398+0000 7fb834e00700 1 -- 172.21.15.1:0/3053218579 shutdown_connections 2024-04-26T01:49:51.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.399+0000 7fb834e00700 1 -- 172.21.15.1:0/3053218579 wait complete. 2024-04-26T01:49:51.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.400+0000 7fb834e00700 1 Processor -- start 2024-04-26T01:49:51.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.400+0000 7fb834e00700 1 -- start start 2024-04-26T01:49:51.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.400+0000 7fb834e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb830101400 0x7fb83019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:51.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.400+0000 7fb834e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb830101da0 0x7fb83019b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:51.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.400+0000 7fb834e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8301038b0 0x7fb83019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:51.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.400+0000 7fb834e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb830114f40 con 0x7fb830101400 2024-04-26T01:49:51.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.400+0000 7fb834e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb830114dc0 con 0x7fb830101da0 2024-04-26T01:49:51.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.400+0000 7fb834e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb8301150c0 con 0x7fb8301038b0 2024-04-26T01:49:51.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.401+0000 7fb82de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb830101400 0x7fb83019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:51.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.401+0000 7fb82d400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb830101da0 0x7fb83019b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:51.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.401+0000 7fb82e800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8301038b0 0x7fb83019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:51.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.401+0000 7fb82d400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb830101da0 0x7fb83019b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:52214/0 (socket says 172.21.15.1:52214) 2024-04-26T01:49:51.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.401+0000 7fb82e800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8301038b0 0x7fb83019fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:38390/0 (socket says 172.21.15.1:38390) 2024-04-26T01:49:51.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.401+0000 7fb82d400700 1 -- 172.21.15.1:0/2302776650 learned_addr learned my addr 172.21.15.1:0/2302776650 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:51.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.401+0000 7fb82d400700 1 -- 172.21.15.1:0/2302776650 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8301038b0 msgr2=0x7fb83019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:51.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.401+0000 7fb82d400700 1 --2- 172.21.15.1:0/2302776650 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8301038b0 0x7fb83019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.401+0000 7fb82d400700 1 -- 172.21.15.1:0/2302776650 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb830101400 msgr2=0x7fb83019b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:51.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.401+0000 7fb82d400700 1 --2- 172.21.15.1:0/2302776650 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb830101400 0x7fb83019b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.401+0000 7fb82d400700 1 -- 172.21.15.1:0/2302776650 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8301a03e0 con 0x7fb830101da0 2024-04-26T01:49:51.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.401+0000 7fb82d400700 1 --2- 172.21.15.1:0/2302776650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb830101da0 0x7fb83019b910 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0057c0 tx=0x7fb81c005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:51.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.402+0000 7fb826a00700 1 -- 172.21.15.1:0/2302776650 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c026070 con 0x7fb830101da0 2024-04-26T01:49:51.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.402+0000 7fb826a00700 1 -- 172.21.15.1:0/2302776650 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb81c021410 con 0x7fb830101da0 2024-04-26T01:49:51.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.402+0000 7fb826a00700 1 -- 172.21.15.1:0/2302776650 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c017650 con 0x7fb830101da0 2024-04-26T01:49:51.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.402+0000 7fb834e00700 1 -- 172.21.15.1:0/2302776650 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8301a0670 con 0x7fb830101da0 2024-04-26T01:49:51.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.402+0000 7fb834e00700 1 -- 172.21.15.1:0/2302776650 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8301a7f50 con 0x7fb830101da0 2024-04-26T01:49:51.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.403+0000 7fb834e00700 1 -- 172.21.15.1:0/2302776650 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb830069730 con 0x7fb830101da0 2024-04-26T01:49:51.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.405+0000 7fb826a00700 1 -- 172.21.15.1:0/2302776650 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fb81c009d10 con 0x7fb830101da0 2024-04-26T01:49:51.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.407+0000 7fb826a00700 1 --2- 172.21.15.1:0/2302776650 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb8140a3c40 0x7fb8140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:51.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.407+0000 7fb826a00700 1 -- 172.21.15.1:0/2302776650 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fb81c0d0ee0 con 0x7fb830101da0 2024-04-26T01:49:51.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.407+0000 7fb82de00700 1 --2- 172.21.15.1:0/2302776650 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb8140a3c40 0x7fb8140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:51.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.408+0000 7fb82de00700 1 --2- 172.21.15.1:0/2302776650 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb8140a3c40 0x7fb8140a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb8300782a0 tx=0x7fb820009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:51.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.409+0000 7fb826a00700 1 -- 172.21.15.1:0/2302776650 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb81c013070 con 0x7fb830101da0 2024-04-26T01:49:51.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.649+0000 7fb834e00700 1 -- 172.21.15.1:0/2302776650 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb83002df80 con 0x7fb830101da0 2024-04-26T01:49:51.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.649+0000 7fb826a00700 1 -- 172.21.15.1:0/2302776650 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb81c09d960 con 0x7fb830101da0 2024-04-26T01:49:51.648 INFO:teuthology.orchestra.run.smithi001.stdout:141733920851 2024-04-26T01:49:51.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.652+0000 7fb834e00700 1 -- 172.21.15.1:0/2302776650 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb8140a3c40 msgr2=0x7fb8140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:51.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.652+0000 7fb834e00700 1 --2- 172.21.15.1:0/2302776650 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb8140a3c40 0x7fb8140a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb8300782a0 tx=0x7fb820009260 comp rx=0 tx=0).stop 2024-04-26T01:49:51.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.652+0000 7fb834e00700 1 -- 172.21.15.1:0/2302776650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb830101da0 msgr2=0x7fb83019b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:51.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.652+0000 7fb834e00700 1 --2- 172.21.15.1:0/2302776650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb830101da0 0x7fb83019b910 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0057c0 tx=0x7fb81c005ef0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.653+0000 7fb834e00700 1 -- 172.21.15.1:0/2302776650 shutdown_connections 2024-04-26T01:49:51.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.653+0000 7fb834e00700 1 --2- 172.21.15.1:0/2302776650 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb8140a3c40 0x7fb8140a6100 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.653+0000 7fb834e00700 1 --2- 172.21.15.1:0/2302776650 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8301038b0 0x7fb83019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.653+0000 7fb834e00700 1 --2- 172.21.15.1:0/2302776650 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb830101400 0x7fb83019b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.653+0000 7fb834e00700 1 --2- 172.21.15.1:0/2302776650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb830101da0 0x7fb83019b910 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.653+0000 7fb834e00700 1 -- 172.21.15.1:0/2302776650 >> 172.21.15.1:0/2302776650 conn(0x7fb830076c60 msgr2=0x7fb8300ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:51.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.655+0000 7fb834e00700 1 -- 172.21.15.1:0/2302776650 shutdown_connections 2024-04-26T01:49:51.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:51.655+0000 7fb834e00700 1 -- 172.21.15.1:0/2302776650 wait complete. 2024-04-26T01:49:52.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:51 smithi097 ceph-mon[32853]: pgmap v397: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:52.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:51 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/721648192' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T01:49:52.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:51 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/2302776650' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T01:49:52.295 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:51 smithi001 ceph-mon[27822]: pgmap v397: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:52.295 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:51 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/721648192' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T01:49:52.295 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:51 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2302776650' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T01:49:52.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:51 smithi173 ceph-mon[32553]: pgmap v397: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:49:52.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:51 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/721648192' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T01:49:52.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:51 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/2302776650' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T01:49:53.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.728+0000 7f703e800700 1 -- 172.21.15.1:0/2263109088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7038102a90 msgr2=0x7f7038102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:53.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.728+0000 7f703e800700 1 --2- 172.21.15.1:0/2263109088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7038102a90 0x7f7038102f10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7024009990 tx=0x7f7024009ca0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.728+0000 7f703e800700 1 -- 172.21.15.1:0/2263109088 shutdown_connections 2024-04-26T01:49:53.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.728+0000 7f703e800700 1 --2- 172.21.15.1:0/2263109088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7038103450 0x7f703810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.728+0000 7f703e800700 1 --2- 172.21.15.1:0/2263109088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7038108a50 0x7f7038108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.728+0000 7f703e800700 1 --2- 172.21.15.1:0/2263109088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7038102a90 0x7f7038102f10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.728+0000 7f703e800700 1 -- 172.21.15.1:0/2263109088 >> 172.21.15.1:0/2263109088 conn(0x7f7038079ba0 msgr2=0x7f7038079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:53.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.728+0000 7f703e800700 1 -- 172.21.15.1:0/2263109088 shutdown_connections 2024-04-26T01:49:53.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.729+0000 7f703e800700 1 -- 172.21.15.1:0/2263109088 wait complete. 2024-04-26T01:49:53.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.730+0000 7f703e800700 1 Processor -- start 2024-04-26T01:49:53.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.730+0000 7f703e800700 1 -- start start 2024-04-26T01:49:53.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.731+0000 7f703e800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7038102a90 0x7f703819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:53.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.731+0000 7f703e800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7038103450 0x7f703819b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:53.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.731+0000 7f703e800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7038108a50 0x7f703819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:53.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.731+0000 7f703e800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7038112c30 con 0x7f7038102a90 2024-04-26T01:49:53.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.731+0000 7f703e800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7038112ab0 con 0x7f7038108a50 2024-04-26T01:49:53.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.731+0000 7f703e800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7038112db0 con 0x7f7038103450 2024-04-26T01:49:53.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.731+0000 7f7037400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7038102a90 0x7f703819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:53.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.731+0000 7f7037400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7038102a90 0x7f703819b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58276/0 (socket says 172.21.15.1:58276) 2024-04-26T01:49:53.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.731+0000 7f7037400700 1 -- 172.21.15.1:0/2768298113 learned_addr learned my addr 172.21.15.1:0/2768298113 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:53.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.731+0000 7f7036a00700 1 --2- 172.21.15.1:0/2768298113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7038103450 0x7f703819b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:53.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.731+0000 7f7037e00700 1 --2- 172.21.15.1:0/2768298113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7038108a50 0x7f703819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:53.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.732+0000 7f7037400700 1 -- 172.21.15.1:0/2768298113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7038103450 msgr2=0x7f703819b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:53.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.732+0000 7f7037400700 1 --2- 172.21.15.1:0/2768298113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7038103450 0x7f703819b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.732+0000 7f7037400700 1 -- 172.21.15.1:0/2768298113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7038108a50 msgr2=0x7f703819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:53.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.732+0000 7f7037400700 1 --2- 172.21.15.1:0/2768298113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7038108a50 0x7f703819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.732+0000 7f7037400700 1 -- 172.21.15.1:0/2768298113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70381a0370 con 0x7f7038102a90 2024-04-26T01:49:53.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.732+0000 7f7036a00700 1 --2- 172.21.15.1:0/2768298113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7038103450 0x7f703819b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:49:53.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.732+0000 7f7037400700 1 --2- 172.21.15.1:0/2768298113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7038102a90 0x7f703819b360 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f702800e9a0 tx=0x7f702800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:53.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.732+0000 7f702fe00700 1 -- 172.21.15.1:0/2768298113 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702800ca10 con 0x7f7038102a90 2024-04-26T01:49:53.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.732+0000 7f702fe00700 1 -- 172.21.15.1:0/2768298113 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f702800cbb0 con 0x7f7038102a90 2024-04-26T01:49:53.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.732+0000 7f703e800700 1 -- 172.21.15.1:0/2768298113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70381a0660 con 0x7f7038102a90 2024-04-26T01:49:53.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.733+0000 7f702fe00700 1 -- 172.21.15.1:0/2768298113 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028018680 con 0x7f7038102a90 2024-04-26T01:49:53.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.733+0000 7f703e800700 1 -- 172.21.15.1:0/2768298113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70381a7ee0 con 0x7f7038102a90 2024-04-26T01:49:53.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.734+0000 7f703e800700 1 -- 172.21.15.1:0/2768298113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7038069730 con 0x7f7038102a90 2024-04-26T01:49:53.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.736+0000 7f702fe00700 1 -- 172.21.15.1:0/2768298113 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f7028010430 con 0x7f7038102a90 2024-04-26T01:49:53.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.737+0000 7f702fe00700 1 --2- 172.21.15.1:0/2768298113 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f701c0a3cc0 0x7f701c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:53.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.738+0000 7f702fe00700 1 -- 172.21.15.1:0/2768298113 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f70280d0510 con 0x7f7038102a90 2024-04-26T01:49:53.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.738+0000 7f7036a00700 1 --2- 172.21.15.1:0/2768298113 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f701c0a3cc0 0x7f701c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:53.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.738+0000 7f7036a00700 1 --2- 172.21.15.1:0/2768298113 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f701c0a3cc0 0x7f701c0a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f703807a4a0 tx=0x7f702400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:53.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.742+0000 7f702fe00700 1 -- 172.21.15.1:0/2768298113 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f702809ce70 con 0x7f7038102a90 2024-04-26T01:49:53.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.990+0000 7f703e800700 1 -- 172.21.15.1:0/2768298113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f703802df80 con 0x7f7038102a90 2024-04-26T01:49:53.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.990+0000 7f702fe00700 1 -- 172.21.15.1:0/2768298113 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f702809c770 con 0x7f7038102a90 2024-04-26T01:49:53.989 INFO:teuthology.orchestra.run.smithi001.stdout:223338299459 2024-04-26T01:49:53.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.993+0000 7f703e800700 1 -- 172.21.15.1:0/2768298113 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f701c0a3cc0 msgr2=0x7f701c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:53.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.993+0000 7f703e800700 1 --2- 172.21.15.1:0/2768298113 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f701c0a3cc0 0x7f701c0a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f703807a4a0 tx=0x7f702400b260 comp rx=0 tx=0).stop 2024-04-26T01:49:53.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.993+0000 7f703e800700 1 -- 172.21.15.1:0/2768298113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7038102a90 msgr2=0x7f703819b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:53.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.993+0000 7f703e800700 1 --2- 172.21.15.1:0/2768298113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7038102a90 0x7f703819b360 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f702800e9a0 tx=0x7f702800ecb0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.993+0000 7f703e800700 1 -- 172.21.15.1:0/2768298113 shutdown_connections 2024-04-26T01:49:53.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.993+0000 7f703e800700 1 --2- 172.21.15.1:0/2768298113 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f701c0a3cc0 0x7f701c0a6180 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.993+0000 7f703e800700 1 --2- 172.21.15.1:0/2768298113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7038108a50 0x7f703819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.993+0000 7f703e800700 1 --2- 172.21.15.1:0/2768298113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7038102a90 0x7f703819b360 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.993+0000 7f703e800700 1 --2- 172.21.15.1:0/2768298113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7038103450 0x7f703819b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:53.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.993+0000 7f703e800700 1 -- 172.21.15.1:0/2768298113 >> 172.21.15.1:0/2768298113 conn(0x7f7038079ba0 msgr2=0x7f703810df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:53.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.994+0000 7f703e800700 1 -- 172.21.15.1:0/2768298113 shutdown_connections 2024-04-26T01:49:53.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.994+0000 7f703e800700 1 -- 172.21.15.1:0/2768298113 wait complete. 2024-04-26T01:49:54.023 INFO:teuthology.orchestra.run.smithi001.stdout:317827579954 2024-04-26T01:49:54.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.811+0000 7ffae5400700 1 -- 172.21.15.1:0/2148881284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae010f440 msgr2=0x7ffae0111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.811+0000 7ffae5400700 1 --2- 172.21.15.1:0/2148881284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae010f440 0x7ffae0111830 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffad800b0a0 tx=0x7ffad800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:49:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.811+0000 7ffae5400700 1 -- 172.21.15.1:0/2148881284 shutdown_connections 2024-04-26T01:49:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.811+0000 7ffae5400700 1 --2- 172.21.15.1:0/2148881284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae010f440 0x7ffae0111830 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.811+0000 7ffae5400700 1 --2- 172.21.15.1:0/2148881284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffae0074c70 0x7ffae0075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.811+0000 7ffae5400700 1 --2- 172.21.15.1:0/2148881284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffae0075590 0x7ffae010ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.811+0000 7ffae5400700 1 -- 172.21.15.1:0/2148881284 >> 172.21.15.1:0/2148881284 conn(0x7ffae0079ba0 msgr2=0x7ffae0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.811+0000 7ffae5400700 1 -- 172.21.15.1:0/2148881284 shutdown_connections 2024-04-26T01:49:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.811+0000 7ffae5400700 1 -- 172.21.15.1:0/2148881284 wait complete. 2024-04-26T01:49:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.813+0000 7ffae5400700 1 Processor -- start 2024-04-26T01:49:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.813+0000 7ffae5400700 1 -- start start 2024-04-26T01:49:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.814+0000 7ffae5400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffae0074c70 0x7ffae019b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.814+0000 7ffae5400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffae0075590 0x7ffae019b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.814+0000 7ffae5400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae010f440 0x7ffae019fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.814+0000 7ffae5400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffae0114e50 con 0x7ffae010f440 2024-04-26T01:49:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.814+0000 7ffae5400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffae0114cd0 con 0x7ffae0075590 2024-04-26T01:49:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.814+0000 7ffae5400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffae0114fd0 con 0x7ffae0074c70 2024-04-26T01:49:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.814+0000 7ffadf200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae010f440 0x7ffae019fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.814+0000 7ffadf200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae010f440 0x7ffae019fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58260/0 (socket says 172.21.15.1:58260) 2024-04-26T01:49:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.814+0000 7ffadf200700 1 -- 172.21.15.1:0/3482395715 learned_addr learned my addr 172.21.15.1:0/3482395715 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.814+0000 7ffad7e00700 1 --2- 172.21.15.1:0/3482395715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffae0075590 0x7ffae019b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.814+0000 7ffade800700 1 --2- 172.21.15.1:0/3482395715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffae0074c70 0x7ffae019b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.814+0000 7ffadf200700 1 -- 172.21.15.1:0/3482395715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffae0074c70 msgr2=0x7ffae019b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.814+0000 7ffadf200700 1 --2- 172.21.15.1:0/3482395715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffae0074c70 0x7ffae019b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.814+0000 7ffadf200700 1 -- 172.21.15.1:0/3482395715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffae0075590 msgr2=0x7ffae019b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.814+0000 7ffadf200700 1 --2- 172.21.15.1:0/3482395715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffae0075590 0x7ffae019b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.814+0000 7ffadf200700 1 -- 172.21.15.1:0/3482395715 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae01a0280 con 0x7ffae010f440 2024-04-26T01:49:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.815+0000 7ffadf200700 1 --2- 172.21.15.1:0/3482395715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae010f440 0x7ffae019fb40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffad800b070 tx=0x7ffad8004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.815+0000 7ffad7400700 1 -- 172.21.15.1:0/3482395715 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8021070 con 0x7ffae010f440 2024-04-26T01:49:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.815+0000 7ffad7400700 1 -- 172.21.15.1:0/3482395715 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffad8007e30 con 0x7ffae010f440 2024-04-26T01:49:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.815+0000 7ffae5400700 1 -- 172.21.15.1:0/3482395715 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae01a0510 con 0x7ffae010f440 2024-04-26T01:49:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.815+0000 7ffae5400700 1 -- 172.21.15.1:0/3482395715 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae01a7df0 con 0x7ffae010f440 2024-04-26T01:49:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.815+0000 7ffad7400700 1 -- 172.21.15.1:0/3482395715 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad80045e0 con 0x7ffae010f440 2024-04-26T01:49:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.816+0000 7ffae5400700 1 -- 172.21.15.1:0/3482395715 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffae0069730 con 0x7ffae010f440 2024-04-26T01:49:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.818+0000 7ffad7400700 1 -- 172.21.15.1:0/3482395715 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7ffad8003c90 con 0x7ffae010f440 2024-04-26T01:49:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.820+0000 7ffad7400700 1 --2- 172.21.15.1:0/3482395715 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ffac40a3c40 0x7ffac40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.820+0000 7ffad7400700 1 -- 172.21.15.1:0/3482395715 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7ffad80d1380 con 0x7ffae010f440 2024-04-26T01:49:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.820+0000 7ffade800700 1 --2- 172.21.15.1:0/3482395715 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ffac40a3c40 0x7ffac40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.821+0000 7ffade800700 1 --2- 172.21.15.1:0/3482395715 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ffac40a3c40 0x7ffac40a6100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009c60 tx=0x7ffad0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:52.824+0000 7ffad7400700 1 -- 172.21.15.1:0/3482395715 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffad809dd70 con 0x7ffae010f440 2024-04-26T01:49:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.054+0000 7ffae5400700 1 -- 172.21.15.1:0/3482395715 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7ffae002df80 con 0x7ffae010f440 2024-04-26T01:49:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.055+0000 7ffad7400700 1 -- 172.21.15.1:0/3482395715 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ffad809d670 con 0x7ffae010f440 2024-04-26T01:49:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.058+0000 7ffae5400700 1 -- 172.21.15.1:0/3482395715 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ffac40a3c40 msgr2=0x7ffac40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.058+0000 7ffae5400700 1 --2- 172.21.15.1:0/3482395715 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ffac40a3c40 0x7ffac40a6100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009c60 tx=0x7ffad0009260 comp rx=0 tx=0).stop 2024-04-26T01:49:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.058+0000 7ffae5400700 1 -- 172.21.15.1:0/3482395715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae010f440 msgr2=0x7ffae019fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.058+0000 7ffae5400700 1 --2- 172.21.15.1:0/3482395715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae010f440 0x7ffae019fb40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffad800b070 tx=0x7ffad8004c60 comp rx=0 tx=0).stop 2024-04-26T01:49:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.058+0000 7ffae5400700 1 -- 172.21.15.1:0/3482395715 shutdown_connections 2024-04-26T01:49:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.058+0000 7ffae5400700 1 --2- 172.21.15.1:0/3482395715 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ffac40a3c40 0x7ffac40a6100 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.058+0000 7ffae5400700 1 --2- 172.21.15.1:0/3482395715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae010f440 0x7ffae019fb40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.058+0000 7ffae5400700 1 --2- 172.21.15.1:0/3482395715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffae0074c70 0x7ffae019b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.058+0000 7ffae5400700 1 --2- 172.21.15.1:0/3482395715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffae0075590 0x7ffae019b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.058+0000 7ffae5400700 1 -- 172.21.15.1:0/3482395715 >> 172.21.15.1:0/3482395715 conn(0x7ffae0079ba0 msgr2=0x7ffae00ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.059+0000 7ffae5400700 1 -- 172.21.15.1:0/3482395715 shutdown_connections 2024-04-26T01:49:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:53.059+0000 7ffae5400700 1 -- 172.21.15.1:0/3482395715 wait complete. 2024-04-26T01:49:54.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:53 smithi097 ceph-mon[32853]: pgmap v398: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:54.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:53 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3482395715' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T01:49:54.267 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:53 smithi001 ceph-mon[27822]: pgmap v398: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:54.267 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:53 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3482395715' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T01:49:54.271 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724619 got 167503724623 for osd.5 2024-04-26T01:49:54.271 DEBUG:teuthology.parallel:result is None 2024-04-26T01:49:54.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:53 smithi173 ceph-mon[32553]: pgmap v398: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:49:54.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:53 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3482395715' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T01:49:55.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:54 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2768298113' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T01:49:55.191 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920845 got 141733920851 for osd.4 2024-04-26T01:49:55.191 DEBUG:teuthology.parallel:result is None 2024-04-26T01:49:55.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:54 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/2768298113' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T01:49:55.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:54 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/2768298113' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T01:49:55.666 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462976 got 201863462984 for osd.6 2024-04-26T01:49:55.667 DEBUG:teuthology.parallel:result is None 2024-04-26T01:49:55.767 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579947 got 317827579954 for osd.10 2024-04-26T01:49:55.768 DEBUG:teuthology.parallel:result is None 2024-04-26T01:49:56.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:55 smithi001 ceph-mon[27822]: pgmap v399: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T01:49:56.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:55 smithi173 ceph-mon[32553]: pgmap v399: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T01:49:56.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:55 smithi097 ceph-mon[32853]: pgmap v399: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T01:49:57.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.647+0000 7f644c000700 1 -- 172.21.15.1:0/815102283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6444077890 msgr2=0x7f64441133e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:57.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.647+0000 7f644c000700 1 --2- 172.21.15.1:0/815102283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6444077890 0x7f64441133e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6430009990 tx=0x7f6430009ca0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.647+0000 7f644c000700 1 -- 172.21.15.1:0/815102283 shutdown_connections 2024-04-26T01:49:57.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.647+0000 7f644c000700 1 --2- 172.21.15.1:0/815102283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6444113920 0x7f6444115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.647+0000 7f644c000700 1 --2- 172.21.15.1:0/815102283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6444076f70 0x7f6444077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.647+0000 7f644c000700 1 --2- 172.21.15.1:0/815102283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6444077890 0x7f64441133e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.647+0000 7f644c000700 1 -- 172.21.15.1:0/815102283 >> 172.21.15.1:0/815102283 conn(0x7f6444100770 msgr2=0x7f6444102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:57.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.647+0000 7f644c000700 1 -- 172.21.15.1:0/815102283 shutdown_connections 2024-04-26T01:49:57.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.647+0000 7f644c000700 1 -- 172.21.15.1:0/815102283 wait complete. 2024-04-26T01:49:57.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.648+0000 7f644c000700 1 Processor -- start 2024-04-26T01:49:57.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.648+0000 7f644c000700 1 -- start start 2024-04-26T01:49:57.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.649+0000 7f644c000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6444076f70 0x7f64441a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:57.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.649+0000 7f644c000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6444077890 0x7f64441a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:57.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.649+0000 7f644c000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6444113920 0x7f644419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:57.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.649+0000 7f644c000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f64441192a0 con 0x7f6444113920 2024-04-26T01:49:57.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.649+0000 7f644c000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6444119120 con 0x7f6444077890 2024-04-26T01:49:57.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.649+0000 7f644c000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6444119420 con 0x7f6444076f70 2024-04-26T01:49:57.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.649+0000 7f6448c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6444077890 0x7f64441a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:57.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.649+0000 7f6448c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6444077890 0x7f64441a5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:53506/0 (socket says 172.21.15.1:53506) 2024-04-26T01:49:57.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.649+0000 7f6448c00700 1 -- 172.21.15.1:0/1362557378 learned_addr learned my addr 172.21.15.1:0/1362557378 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:57.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.649+0000 7f6449600700 1 --2- 172.21.15.1:0/1362557378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6444076f70 0x7f64441a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:57.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.649+0000 7f644a000700 1 --2- 172.21.15.1:0/1362557378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6444113920 0x7f644419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:57.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.649+0000 7f6448c00700 1 -- 172.21.15.1:0/1362557378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6444076f70 msgr2=0x7f64441a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:57.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.649+0000 7f6448c00700 1 --2- 172.21.15.1:0/1362557378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6444076f70 0x7f64441a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.649+0000 7f6448c00700 1 -- 172.21.15.1:0/1362557378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6444113920 msgr2=0x7f644419f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:57.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.649+0000 7f6448c00700 1 --2- 172.21.15.1:0/1362557378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6444113920 0x7f644419f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.650+0000 7f6448c00700 1 -- 172.21.15.1:0/1362557378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644419fda0 con 0x7f6444077890 2024-04-26T01:49:57.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.650+0000 7f6448c00700 1 --2- 172.21.15.1:0/1362557378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6444077890 0x7f64441a5970 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6430000c00 tx=0x7f643000bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:57.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.650+0000 7f6442000700 1 -- 172.21.15.1:0/1362557378 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430027070 con 0x7f6444077890 2024-04-26T01:49:57.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.650+0000 7f6442000700 1 -- 172.21.15.1:0/1362557378 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6430022470 con 0x7f6444077890 2024-04-26T01:49:57.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.650+0000 7f644c000700 1 -- 172.21.15.1:0/1362557378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64441a0030 con 0x7f6444077890 2024-04-26T01:49:57.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.650+0000 7f6442000700 1 -- 172.21.15.1:0/1362557378 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430017650 con 0x7f6444077890 2024-04-26T01:49:57.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.650+0000 7f644c000700 1 -- 172.21.15.1:0/1362557378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64441ac300 con 0x7f6444077890 2024-04-26T01:49:57.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.651+0000 7f644c000700 1 -- 172.21.15.1:0/1362557378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6444069730 con 0x7f6444077890 2024-04-26T01:49:57.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.653+0000 7f6442000700 1 -- 172.21.15.1:0/1362557378 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f643002a070 con 0x7f6444077890 2024-04-26T01:49:57.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.655+0000 7f6442000700 1 --2- 172.21.15.1:0/1362557378 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f642c0a3c40 0x7f642c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:57.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.655+0000 7f6442000700 1 -- 172.21.15.1:0/1362557378 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f64300d11e0 con 0x7f6444077890 2024-04-26T01:49:57.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.655+0000 7f6449600700 1 --2- 172.21.15.1:0/1362557378 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f642c0a3c40 0x7f642c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:57.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.656+0000 7f6449600700 1 --2- 172.21.15.1:0/1362557378 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f642c0a3c40 0x7f642c0a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f643800f540 tx=0x7f643800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:57.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.659+0000 7f6442000700 1 -- 172.21.15.1:0/1362557378 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6430013070 con 0x7f6444077890 2024-04-26T01:49:57.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.239+0000 7f8302800700 1 -- 172.21.15.1:0/3149928779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82fc108a50 msgr2=0x7f82fc108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:57.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.239+0000 7f8302800700 1 --2- 172.21.15.1:0/3149928779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82fc108a50 0x7f82fc108e30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009990 tx=0x7f82ec009ca0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.239+0000 7f8302800700 1 -- 172.21.15.1:0/3149928779 shutdown_connections 2024-04-26T01:49:57.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.239+0000 7f8302800700 1 --2- 172.21.15.1:0/3149928779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82fc103450 0x7f82fc10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.239+0000 7f8302800700 1 --2- 172.21.15.1:0/3149928779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82fc108a50 0x7f82fc108e30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.239+0000 7f8302800700 1 --2- 172.21.15.1:0/3149928779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82fc102a90 0x7f82fc102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.239+0000 7f8302800700 1 -- 172.21.15.1:0/3149928779 >> 172.21.15.1:0/3149928779 conn(0x7f82fc079ba0 msgr2=0x7f82fc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.239+0000 7f8302800700 1 -- 172.21.15.1:0/3149928779 shutdown_connections 2024-04-26T01:49:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.239+0000 7f8302800700 1 -- 172.21.15.1:0/3149928779 wait complete. 2024-04-26T01:49:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.240+0000 7f8302800700 1 Processor -- start 2024-04-26T01:49:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.240+0000 7f8302800700 1 -- start start 2024-04-26T01:49:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.241+0000 7f8302800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82fc102a90 0x7f82fc199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.241+0000 7f8302800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82fc103450 0x7f82fc1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.241+0000 7f8302800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82fc108a50 0x7f82fc19da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.241+0000 7f8302800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc112c30 con 0x7f82fc102a90 2024-04-26T01:49:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.241+0000 7f8302800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc112ab0 con 0x7f82fc103450 2024-04-26T01:49:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.241+0000 7f8302800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc112db0 con 0x7f82fc108a50 2024-04-26T01:49:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.241+0000 7f82fb400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82fc102a90 0x7f82fc199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:57.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.241+0000 7f82fb400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82fc102a90 0x7f82fc199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58292/0 (socket says 172.21.15.1:58292) 2024-04-26T01:49:57.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.241+0000 7f82fb400700 1 -- 172.21.15.1:0/1317744888 learned_addr learned my addr 172.21.15.1:0/1317744888 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:49:57.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.241+0000 7f82faa00700 1 --2- 172.21.15.1:0/1317744888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82fc103450 0x7f82fc1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:57.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.241+0000 7f82fbe00700 1 --2- 172.21.15.1:0/1317744888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82fc108a50 0x7f82fc19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:57.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.242+0000 7f82fb400700 1 -- 172.21.15.1:0/1317744888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82fc108a50 msgr2=0x7f82fc19da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:57.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.242+0000 7f82fb400700 1 --2- 172.21.15.1:0/1317744888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82fc108a50 0x7f82fc19da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.242+0000 7f82fb400700 1 -- 172.21.15.1:0/1317744888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82fc103450 msgr2=0x7f82fc1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:57.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.242+0000 7f82fb400700 1 --2- 172.21.15.1:0/1317744888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82fc103450 0x7f82fc1996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.242+0000 7f82fb400700 1 -- 172.21.15.1:0/1317744888 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82fc19e170 con 0x7f82fc102a90 2024-04-26T01:49:57.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.242+0000 7f82faa00700 1 --2- 172.21.15.1:0/1317744888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82fc103450 0x7f82fc1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:49:57.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.242+0000 7f82fb400700 1 --2- 172.21.15.1:0/1317744888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82fc102a90 0x7f82fc199160 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f82ec005910 tx=0x7f82ec00b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:57.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.242+0000 7f82f3e00700 1 -- 172.21.15.1:0/1317744888 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec027070 con 0x7f82fc102a90 2024-04-26T01:49:57.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.242+0000 7f82f3e00700 1 -- 172.21.15.1:0/1317744888 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82ec022470 con 0x7f82fc102a90 2024-04-26T01:49:57.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.242+0000 7f8302800700 1 -- 172.21.15.1:0/1317744888 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82fc19e400 con 0x7f82fc102a90 2024-04-26T01:49:57.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.242+0000 7f82f3e00700 1 -- 172.21.15.1:0/1317744888 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec017650 con 0x7f82fc102a90 2024-04-26T01:49:57.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.243+0000 7f8302800700 1 -- 172.21.15.1:0/1317744888 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82fc1a5ce0 con 0x7f82fc102a90 2024-04-26T01:49:57.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.244+0000 7f8302800700 1 -- 172.21.15.1:0/1317744888 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82fc069730 con 0x7f82fc102a90 2024-04-26T01:49:57.843 INFO:teuthology.orchestra.run.smithi001.stdout:296352743481 2024-04-26T01:49:57.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.246+0000 7f82f3e00700 1 -- 172.21.15.1:0/1317744888 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f82ec02a070 con 0x7f82fc102a90 2024-04-26T01:49:57.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.248+0000 7f82f3e00700 1 --2- 172.21.15.1:0/1317744888 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f82e00a3cc0 0x7f82e00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:57.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.248+0000 7f82f3e00700 1 -- 172.21.15.1:0/1317744888 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f82ec0d0fc0 con 0x7f82fc102a90 2024-04-26T01:49:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.248+0000 7f82faa00700 1 --2- 172.21.15.1:0/1317744888 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f82e00a3cc0 0x7f82e00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.249+0000 7f82faa00700 1 --2- 172.21.15.1:0/1317744888 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f82e00a3cc0 0x7f82e00a6180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f82fc07a1f0 tx=0x7f82e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.252+0000 7f82f3e00700 1 -- 172.21.15.1:0/1317744888 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f82ec013070 con 0x7f82fc102a90 2024-04-26T01:49:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.493+0000 7f8302800700 1 -- 172.21.15.1:0/1317744888 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f82fc02df80 con 0x7f82fc102a90 2024-04-26T01:49:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.493+0000 7f82f3e00700 1 -- 172.21.15.1:0/1317744888 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f82ec09dab0 con 0x7f82fc102a90 2024-04-26T01:49:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.496+0000 7f8302800700 1 -- 172.21.15.1:0/1317744888 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f82e00a3cc0 msgr2=0x7f82e00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.496+0000 7f8302800700 1 --2- 172.21.15.1:0/1317744888 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f82e00a3cc0 0x7f82e00a6180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f82fc07a1f0 tx=0x7f82e8009260 comp rx=0 tx=0).stop 2024-04-26T01:49:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.496+0000 7f8302800700 1 -- 172.21.15.1:0/1317744888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82fc102a90 msgr2=0x7f82fc199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.496+0000 7f8302800700 1 --2- 172.21.15.1:0/1317744888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82fc102a90 0x7f82fc199160 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f82ec005910 tx=0x7f82ec00b670 comp rx=0 tx=0).stop 2024-04-26T01:49:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.497+0000 7f8302800700 1 -- 172.21.15.1:0/1317744888 shutdown_connections 2024-04-26T01:49:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.497+0000 7f8302800700 1 --2- 172.21.15.1:0/1317744888 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f82e00a3cc0 0x7f82e00a6180 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.497+0000 7f8302800700 1 --2- 172.21.15.1:0/1317744888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82fc108a50 0x7f82fc19da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.497+0000 7f8302800700 1 --2- 172.21.15.1:0/1317744888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82fc102a90 0x7f82fc199160 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.497+0000 7f8302800700 1 --2- 172.21.15.1:0/1317744888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82fc103450 0x7f82fc1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.497+0000 7f8302800700 1 -- 172.21.15.1:0/1317744888 >> 172.21.15.1:0/1317744888 conn(0x7f82fc079ba0 msgr2=0x7f82fc1018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:57.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.497+0000 7f8302800700 1 -- 172.21.15.1:0/1317744888 shutdown_connections 2024-04-26T01:49:57.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.497+0000 7f8302800700 1 -- 172.21.15.1:0/1317744888 wait complete. 2024-04-26T01:49:57.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.896+0000 7f644c000700 1 -- 172.21.15.1:0/1362557378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f644402df80 con 0x7f6444077890 2024-04-26T01:49:57.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.897+0000 7f6442000700 1 -- 172.21.15.1:0/1362557378 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f643009dbd0 con 0x7f6444077890 2024-04-26T01:49:57.895 INFO:teuthology.orchestra.run.smithi001.stdout:352187318318 2024-04-26T01:49:57.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.900+0000 7f644c000700 1 -- 172.21.15.1:0/1362557378 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f642c0a3c40 msgr2=0x7f642c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:57.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.900+0000 7f644c000700 1 --2- 172.21.15.1:0/1362557378 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f642c0a3c40 0x7f642c0a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f643800f540 tx=0x7f643800b040 comp rx=0 tx=0).stop 2024-04-26T01:49:57.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.900+0000 7f644c000700 1 -- 172.21.15.1:0/1362557378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6444077890 msgr2=0x7f64441a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:57.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.900+0000 7f644c000700 1 --2- 172.21.15.1:0/1362557378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6444077890 0x7f64441a5970 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6430000c00 tx=0x7f643000bfd0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.901+0000 7f644c000700 1 -- 172.21.15.1:0/1362557378 shutdown_connections 2024-04-26T01:49:57.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.901+0000 7f644c000700 1 --2- 172.21.15.1:0/1362557378 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f642c0a3c40 0x7f642c0a6100 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.901+0000 7f644c000700 1 --2- 172.21.15.1:0/1362557378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6444113920 0x7f644419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.901+0000 7f644c000700 1 --2- 172.21.15.1:0/1362557378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6444076f70 0x7f64441a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.901+0000 7f644c000700 1 --2- 172.21.15.1:0/1362557378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6444077890 0x7f64441a5970 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:57.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.901+0000 7f644c000700 1 -- 172.21.15.1:0/1362557378 >> 172.21.15.1:0/1362557378 conn(0x7f6444100770 msgr2=0x7f6444102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:57.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.901+0000 7f644c000700 1 -- 172.21.15.1:0/1362557378 shutdown_connections 2024-04-26T01:49:57.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:57.901+0000 7f644c000700 1 -- 172.21.15.1:0/1362557378 wait complete. 2024-04-26T01:49:58.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:57 smithi001 ceph-mon[27822]: pgmap v400: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-26T01:49:58.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:57 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1317744888' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T01:49:58.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:57 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1362557378' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T01:49:58.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:57 smithi173 ceph-mon[32553]: pgmap v400: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-26T01:49:58.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:57 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1317744888' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T01:49:58.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:57 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1362557378' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T01:49:58.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:57 smithi097 ceph-mon[32853]: pgmap v400: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-26T01:49:58.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:57 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1317744888' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T01:49:58.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:57 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1362557378' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T01:49:58.796 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743475 got 296352743481 for osd.9 2024-04-26T01:49:58.797 DEBUG:teuthology.parallel:result is None 2024-04-26T01:49:59.190 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574944 got 55834574949 for osd.1 2024-04-26T01:49:59.191 DEBUG:teuthology.parallel:result is None 2024-04-26T01:49:59.591 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299450 got 223338299459 for osd.7 2024-04-26T01:49:59.591 DEBUG:teuthology.parallel:result is None 2024-04-26T01:50:00.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:49:59 smithi001 ceph-mon[27822]: pgmap v401: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-04-26T01:50:00.189 INFO:tasks.cephadm.ceph_manager.ceph:need seq 352187318312 got 352187318318 for osd.11 2024-04-26T01:50:00.190 DEBUG:teuthology.parallel:result is None 2024-04-26T01:50:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.995+0000 7f8b9da00700 1 -- 172.21.15.1:0/1133411687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b98108a30 msgr2=0x7f8b98108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.995+0000 7f8b9da00700 1 --2- 172.21.15.1:0/1133411687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b98108a30 0x7f8b98108e10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009990 tx=0x7f8b88009ca0 comp rx=0 tx=0).stop 2024-04-26T01:50:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.995+0000 7f8b9da00700 1 -- 172.21.15.1:0/1133411687 shutdown_connections 2024-04-26T01:50:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.995+0000 7f8b9da00700 1 --2- 172.21.15.1:0/1133411687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b9806b0f0 0x7f8b9810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.995+0000 7f8b9da00700 1 --2- 172.21.15.1:0/1133411687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b98108a30 0x7f8b98108e10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.995+0000 7f8b9da00700 1 --2- 172.21.15.1:0/1133411687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b9806a750 0x7f8b9806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.995+0000 7f8b9da00700 1 -- 172.21.15.1:0/1133411687 >> 172.21.15.1:0/1133411687 conn(0x7f8b980788f0 msgr2=0x7f8b98078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.995+0000 7f8b9da00700 1 -- 172.21.15.1:0/1133411687 shutdown_connections 2024-04-26T01:50:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.996+0000 7f8b9da00700 1 -- 172.21.15.1:0/1133411687 wait complete. 2024-04-26T01:50:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.997+0000 7f8b9da00700 1 Processor -- start 2024-04-26T01:50:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.997+0000 7f8b9da00700 1 -- start start 2024-04-26T01:50:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.998+0000 7f8b9da00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b9806a750 0x7f8b98101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.998+0000 7f8b9da00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b9806b0f0 0x7f8b98101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.998+0000 7f8b9da00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b98108a30 0x7f8b98102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.998+0000 7f8b9da00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98112c00 con 0x7f8b9806a750 2024-04-26T01:50:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.998+0000 7f8b9da00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98112a80 con 0x7f8b9806b0f0 2024-04-26T01:50:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.998+0000 7f8b9da00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98112d80 con 0x7f8b98108a30 2024-04-26T01:50:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.998+0000 7f8b96a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b9806a750 0x7f8b98101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:00.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.998+0000 7f8b96a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b9806a750 0x7f8b98101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58332/0 (socket says 172.21.15.1:58332) 2024-04-26T01:50:00.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.998+0000 7f8b96a00700 1 -- 172.21.15.1:0/734756539 learned_addr learned my addr 172.21.15.1:0/734756539 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:00.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.998+0000 7f8b8e000700 1 --2- 172.21.15.1:0/734756539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b9806b0f0 0x7f8b98101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:00.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.998+0000 7f8b97400700 1 --2- 172.21.15.1:0/734756539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b98108a30 0x7f8b98102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:00.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.999+0000 7f8b96a00700 1 -- 172.21.15.1:0/734756539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b98108a30 msgr2=0x7f8b98102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:00.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.999+0000 7f8b96a00700 1 --2- 172.21.15.1:0/734756539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b98108a30 0x7f8b98102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:00.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.999+0000 7f8b96a00700 1 -- 172.21.15.1:0/734756539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b9806b0f0 msgr2=0x7f8b98101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:00.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.999+0000 7f8b96a00700 1 --2- 172.21.15.1:0/734756539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b9806b0f0 0x7f8b98101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:00.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.999+0000 7f8b96a00700 1 -- 172.21.15.1:0/734756539 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b981a7680 con 0x7f8b9806a750 2024-04-26T01:50:00.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:49:59.999+0000 7f8b96a00700 1 --2- 172.21.15.1:0/734756539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b9806a750 0x7f8b98101a10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8b88000c00 tx=0x7f8b880057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:00.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.000+0000 7f8b8fe00700 1 -- 172.21.15.1:0/734756539 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88027070 con 0x7f8b9806a750 2024-04-26T01:50:00.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.000+0000 7f8b9da00700 1 -- 172.21.15.1:0/734756539 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b981a7910 con 0x7f8b9806a750 2024-04-26T01:50:00.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.000+0000 7f8b8fe00700 1 -- 172.21.15.1:0/734756539 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b88022470 con 0x7f8b9806a750 2024-04-26T01:50:00.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.000+0000 7f8b8fe00700 1 -- 172.21.15.1:0/734756539 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88017650 con 0x7f8b9806a750 2024-04-26T01:50:00.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.000+0000 7f8b9da00700 1 -- 172.21.15.1:0/734756539 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b981a7dc0 con 0x7f8b9806a750 2024-04-26T01:50:00.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.001+0000 7f8b9da00700 1 -- 172.21.15.1:0/734756539 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b98069730 con 0x7f8b9806a750 2024-04-26T01:50:00.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.003+0000 7f8b8fe00700 1 -- 172.21.15.1:0/734756539 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f8b8802a070 con 0x7f8b9806a750 2024-04-26T01:50:00.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.004+0000 7f8b8fe00700 1 --2- 172.21.15.1:0/734756539 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8b7c0a3cc0 0x7f8b7c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:00.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.005+0000 7f8b8fe00700 1 -- 172.21.15.1:0/734756539 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f8b880d0fe0 con 0x7f8b9806a750 2024-04-26T01:50:00.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.005+0000 7f8b8e000700 1 --2- 172.21.15.1:0/734756539 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8b7c0a3cc0 0x7f8b7c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:00.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.005+0000 7f8b8e000700 1 --2- 172.21.15.1:0/734756539 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8b7c0a3cc0 0x7f8b7c0a6180 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8b98078f60 tx=0x7f8b8000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:00.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.009+0000 7f8b8fe00700 1 -- 172.21.15.1:0/734756539 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b88013070 con 0x7f8b9806a750 2024-04-26T01:50:00.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.251+0000 7f8b9da00700 1 -- 172.21.15.1:0/734756539 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f8b9802e140 con 0x7f8b9806a750 2024-04-26T01:50:00.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.252+0000 7f8b8fe00700 1 -- 172.21.15.1:0/734756539 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8b8809da30 con 0x7f8b9806a750 2024-04-26T01:50:00.282 INFO:teuthology.orchestra.run.smithi001.stdout:107374182491 2024-04-26T01:50:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.255+0000 7f8b9da00700 1 -- 172.21.15.1:0/734756539 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8b7c0a3cc0 msgr2=0x7f8b7c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.255+0000 7f8b9da00700 1 --2- 172.21.15.1:0/734756539 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8b7c0a3cc0 0x7f8b7c0a6180 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8b98078f60 tx=0x7f8b8000b040 comp rx=0 tx=0).stop 2024-04-26T01:50:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.255+0000 7f8b9da00700 1 -- 172.21.15.1:0/734756539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b9806a750 msgr2=0x7f8b98101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.255+0000 7f8b9da00700 1 --2- 172.21.15.1:0/734756539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b9806a750 0x7f8b98101a10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8b88000c00 tx=0x7f8b880057c0 comp rx=0 tx=0).stop 2024-04-26T01:50:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.255+0000 7f8b9da00700 1 -- 172.21.15.1:0/734756539 shutdown_connections 2024-04-26T01:50:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.255+0000 7f8b9da00700 1 --2- 172.21.15.1:0/734756539 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8b7c0a3cc0 0x7f8b7c0a6180 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.255+0000 7f8b9da00700 1 --2- 172.21.15.1:0/734756539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b98108a30 0x7f8b98102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.255+0000 7f8b9da00700 1 --2- 172.21.15.1:0/734756539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b9806a750 0x7f8b98101a10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.255+0000 7f8b9da00700 1 --2- 172.21.15.1:0/734756539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b9806b0f0 0x7f8b98101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.255+0000 7f8b9da00700 1 -- 172.21.15.1:0/734756539 >> 172.21.15.1:0/734756539 conn(0x7f8b980788f0 msgr2=0x7f8b980ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.255+0000 7f8b9da00700 1 -- 172.21.15.1:0/734756539 shutdown_connections 2024-04-26T01:50:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:00.255+0000 7f8b9da00700 1 -- 172.21.15.1:0/734756539 wait complete. 2024-04-26T01:50:00.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:49:59 smithi173 ceph-mon[32553]: pgmap v401: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-04-26T01:50:00.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:49:59 smithi097 ceph-mon[32853]: pgmap v401: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-04-26T01:50:01.073 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182484 got 107374182491 for osd.3 2024-04-26T01:50:01.073 DEBUG:teuthology.parallel:result is None 2024-04-26T01:50:01.073 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T01:50:01.074 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:50:01.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:01 smithi173 ceph-mon[32553]: overall HEALTH_OK 2024-04-26T01:50:01.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:01 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/734756539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T01:50:01.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:01 smithi001 ceph-mon[27822]: overall HEALTH_OK 2024-04-26T01:50:01.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:01 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/734756539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T01:50:01.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:01 smithi097 ceph-mon[32853]: overall HEALTH_OK 2024-04-26T01:50:01.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:01 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/734756539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T01:50:01.547 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:02.762 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:02 smithi001 ceph-mon[27822]: pgmap v402: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-04-26T01:50:02.763 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:02 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:50:02.763 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:02 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:50:02.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:02 smithi173 ceph-mon[32553]: pgmap v402: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-04-26T01:50:02.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:02 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:50:02.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:02 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:50:02.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:02 smithi097 ceph-mon[32853]: pgmap v402: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-04-26T01:50:02.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:02 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:50:02.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:02 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:50:03.709 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:03.710 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:50:03.710 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:50:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.780+0000 7f81ba000700 1 -- 172.21.15.1:0/4092825683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81b406a750 msgr2=0x7f81b406abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.780+0000 7f81ba000700 1 --2- 172.21.15.1:0/4092825683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81b406a750 0x7f81b406abb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009990 tx=0x7f81a0009ca0 comp rx=0 tx=0).stop 2024-04-26T01:50:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.780+0000 7f81ba000700 1 -- 172.21.15.1:0/4092825683 shutdown_connections 2024-04-26T01:50:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.780+0000 7f81ba000700 1 --2- 172.21.15.1:0/4092825683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81b406b0f0 0x7f81b410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.780+0000 7f81ba000700 1 --2- 172.21.15.1:0/4092825683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81b4108a30 0x7f81b4108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.780+0000 7f81ba000700 1 --2- 172.21.15.1:0/4092825683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81b406a750 0x7f81b406abb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.780+0000 7f81ba000700 1 -- 172.21.15.1:0/4092825683 >> 172.21.15.1:0/4092825683 conn(0x7f81b40788f0 msgr2=0x7f81b4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.780+0000 7f81ba000700 1 -- 172.21.15.1:0/4092825683 shutdown_connections 2024-04-26T01:50:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.780+0000 7f81ba000700 1 -- 172.21.15.1:0/4092825683 wait complete. 2024-04-26T01:50:03.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.781+0000 7f81ba000700 1 Processor -- start 2024-04-26T01:50:03.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.782+0000 7f81ba000700 1 -- start start 2024-04-26T01:50:03.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.782+0000 7f81ba000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81b406a750 0x7f81b4101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:03.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.782+0000 7f81ba000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81b406b0f0 0x7f81b4101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:03.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.782+0000 7f81ba000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81b4108a30 0x7f81b4102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:03.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.782+0000 7f81ba000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f81b4112c20 con 0x7f81b406b0f0 2024-04-26T01:50:03.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.782+0000 7f81ba000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f81b4112aa0 con 0x7f81b4108a30 2024-04-26T01:50:03.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.782+0000 7f81ba000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f81b4112da0 con 0x7f81b406a750 2024-04-26T01:50:03.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.782+0000 7f81b3400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81b406a750 0x7f81b4101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:03.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.782+0000 7f81b2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81b406b0f0 0x7f81b4101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:03.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.782+0000 7f81b3400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81b406a750 0x7f81b4101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:57136/0 (socket says 172.21.15.1:57136) 2024-04-26T01:50:03.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.782+0000 7f81b2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81b406b0f0 0x7f81b4101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35330/0 (socket says 172.21.15.1:35330) 2024-04-26T01:50:03.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.782+0000 7f81b3400700 1 -- 172.21.15.1:0/1196668291 learned_addr learned my addr 172.21.15.1:0/1196668291 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:03.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.782+0000 7f81b3e00700 1 --2- 172.21.15.1:0/1196668291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81b4108a30 0x7f81b4102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:03.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.783+0000 7f81b3400700 1 -- 172.21.15.1:0/1196668291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81b4108a30 msgr2=0x7f81b4102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:03.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.783+0000 7f81b3400700 1 --2- 172.21.15.1:0/1196668291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81b4108a30 0x7f81b4102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:03.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.783+0000 7f81b3400700 1 -- 172.21.15.1:0/1196668291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81b406b0f0 msgr2=0x7f81b4101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:03.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.783+0000 7f81b3400700 1 --2- 172.21.15.1:0/1196668291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81b406b0f0 0x7f81b4101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.783+0000 7f81b3400700 1 -- 172.21.15.1:0/1196668291 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b41abb20 con 0x7f81b406a750 2024-04-26T01:50:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.783+0000 7f81b2a00700 1 --2- 172.21.15.1:0/1196668291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81b406b0f0 0x7f81b4101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.783+0000 7f81b3400700 1 --2- 172.21.15.1:0/1196668291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81b406a750 0x7f81b4101a10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f81a400e9c0 tx=0x7f81a400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.784+0000 7f81abe00700 1 -- 172.21.15.1:0/1196668291 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a400cbd0 con 0x7f81b406a750 2024-04-26T01:50:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.784+0000 7f81abe00700 1 -- 172.21.15.1:0/1196668291 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81a400ee60 con 0x7f81b406a750 2024-04-26T01:50:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.784+0000 7f81ba000700 1 -- 172.21.15.1:0/1196668291 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b41abdb0 con 0x7f81b406a750 2024-04-26T01:50:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.784+0000 7f81abe00700 1 -- 172.21.15.1:0/1196668291 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a40055c0 con 0x7f81b406a750 2024-04-26T01:50:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.784+0000 7f81ba000700 1 -- 172.21.15.1:0/1196668291 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b41ac1b0 con 0x7f81b406a750 2024-04-26T01:50:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.785+0000 7f81ba000700 1 -- 172.21.15.1:0/1196668291 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b4104170 con 0x7f81b406a750 2024-04-26T01:50:03.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.788+0000 7f81abe00700 1 -- 172.21.15.1:0/1196668291 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f81a4004750 con 0x7f81b406a750 2024-04-26T01:50:03.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.789+0000 7f81abe00700 1 --2- 172.21.15.1:0/1196668291 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f81980a3c40 0x7f81980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:03.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.790+0000 7f81abe00700 1 -- 172.21.15.1:0/1196668291 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f81a40d06c0 con 0x7f81b406a750 2024-04-26T01:50:03.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.790+0000 7f81b2a00700 1 --2- 172.21.15.1:0/1196668291 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f81980a3c40 0x7f81980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:03.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.790+0000 7f81b2a00700 1 --2- 172.21.15.1:0/1196668291 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f81980a3c40 0x7f81980a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f81b4103260 tx=0x7f81a000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:03.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:03.794+0000 7f81abe00700 1 -- 172.21.15.1:0/1196668291 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81a409d020 con 0x7f81b406a750 2024-04-26T01:50:03.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:03.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:50:03.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:50:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:50:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:50:04.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:04.021+0000 7f81ba000700 1 -- 172.21.15.1:0/1196668291 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f81b4062e80 con 0x7f81980a3c40 2024-04-26T01:50:04.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:04.045+0000 7f81abe00700 1 -- 172.21.15.1:0/1196668291 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258838 (secure 0 0 0) 0x7f81b4062e80 con 0x7f81980a3c40 2024-04-26T01:50:04.050 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:50:04.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:04.053+0000 7f81ba000700 1 -- 172.21.15.1:0/1196668291 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f81980a3c40 msgr2=0x7f81980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:04.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:04.053+0000 7f81ba000700 1 --2- 172.21.15.1:0/1196668291 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f81980a3c40 0x7f81980a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f81b4103260 tx=0x7f81a000b260 comp rx=0 tx=0).stop 2024-04-26T01:50:04.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:04.053+0000 7f81ba000700 1 -- 172.21.15.1:0/1196668291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81b406a750 msgr2=0x7f81b4101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:04.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:04.053+0000 7f81ba000700 1 --2- 172.21.15.1:0/1196668291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81b406a750 0x7f81b4101a10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f81a400e9c0 tx=0x7f81a400ecd0 comp rx=0 tx=0).stop 2024-04-26T01:50:04.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:04.053+0000 7f81ba000700 1 -- 172.21.15.1:0/1196668291 shutdown_connections 2024-04-26T01:50:04.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:04.053+0000 7f81ba000700 1 --2- 172.21.15.1:0/1196668291 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f81980a3c40 0x7f81980a6100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:04.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:04.053+0000 7f81ba000700 1 --2- 172.21.15.1:0/1196668291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81b4108a30 0x7f81b4102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:04.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:04.053+0000 7f81ba000700 1 --2- 172.21.15.1:0/1196668291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81b406a750 0x7f81b4101a10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:04.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:04.053+0000 7f81ba000700 1 --2- 172.21.15.1:0/1196668291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81b406b0f0 0x7f81b4101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:04.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:04.053+0000 7f81ba000700 1 -- 172.21.15.1:0/1196668291 >> 172.21.15.1:0/1196668291 conn(0x7f81b40788f0 msgr2=0x7f81b40ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:04.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:04.053+0000 7f81ba000700 1 -- 172.21.15.1:0/1196668291 shutdown_connections 2024-04-26T01:50:04.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:04.053+0000 7f81ba000700 1 -- 172.21.15.1:0/1196668291 wait complete. 2024-04-26T01:50:04.054 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-04-26T01:50:04.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:04 smithi173 ceph-mon[32553]: pgmap v403: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-26T01:50:04.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:04 smithi001 ceph-mon[27822]: pgmap v403: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-26T01:50:04.976 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2024-04-26T01:50:02.860093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655340,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":946,"num_read_kb":594,"num_write":353,"num_write_kb":3055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":784,"ondisk_log_size":784,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358972,"kb_used_data":13980,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582852,"statfs":{"total":1151940427776,"available":1151572840448,"internally_reserved":0,"allocated":14315520,"data_stored":7843200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":43,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005056"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.815061+0000","last_change":"2024-04-26T01:47:27.140418+0000","last_active":"2024-04-26T01:49:20.815061+0000","last_peered":"2024-04-26T01:49:20.815061+0000","last_clean":"2024-04-26T01:49:20.815061+0000","last_became_active":"2024-04-26T01:47:27.140017+0000","last_became_peered":"2024-04-26T01:47:27.140017+0000","last_unstale":"2024-04-26T01:49:20.815061+0000","last_undegraded":"2024-04-26T01:49:20.815061+0000","last_fullsized":"2024-04-26T01:49:20.815061+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:46:08.621222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.831303+0000","last_change":"2024-04-26T01:47:27.138746+0000","last_active":"2024-04-26T01:49:20.831303+0000","last_peered":"2024-04-26T01:49:20.831303+0000","last_clean":"2024-04-26T01:49:20.831303+0000","last_became_active":"2024-04-26T01:47:27.136163+0000","last_became_peered":"2024-04-26T01:47:27.136163+0000","last_unstale":"2024-04-26T01:49:20.831303+0000","last_undegraded":"2024-04-26T01:49:20.831303+0000","last_fullsized":"2024-04-26T01:49:20.831303+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:01:36.195332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.833057+0000","last_change":"2024-04-26T01:47:27.127017+0000","last_active":"2024-04-26T01:49:20.833057+0000","last_peered":"2024-04-26T01:49:20.833057+0000","last_clean":"2024-04-26T01:49:20.833057+0000","last_became_active":"2024-04-26T01:47:27.126610+0000","last_became_peered":"2024-04-26T01:47:27.126610+0000","last_unstale":"2024-04-26T01:49:20.833057+0000","last_undegraded":"2024-04-26T01:49:20.833057+0000","last_fullsized":"2024-04-26T01:49:20.833057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:05:12.784486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.791490+0000","last_change":"2024-04-26T01:47:27.117640+0000","last_active":"2024-04-26T01:49:20.791490+0000","last_peered":"2024-04-26T01:49:20.791490+0000","last_clean":"2024-04-26T01:49:20.791490+0000","last_became_active":"2024-04-26T01:47:27.117266+0000","last_became_peered":"2024-04-26T01:47:27.117266+0000","last_unstale":"2024-04-26T01:49:20.791490+0000","last_undegraded":"2024-04-26T01:49:20.791490+0000","last_fullsized":"2024-04-26T01:49:20.791490+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:26:57.824560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.797486+0000","last_change":"2024-04-26T01:47:27.128185+0000","last_active":"2024-04-26T01:49:20.797486+0000","last_peered":"2024-04-26T01:49:20.797486+0000","last_clean":"2024-04-26T01:49:20.797486+0000","last_became_active":"2024-04-26T01:47:27.127794+0000","last_became_peered":"2024-04-26T01:47:27.127794+0000","last_unstale":"2024-04-26T01:49:20.797486+0000","last_undegraded":"2024-04-26T01:49:20.797486+0000","last_fullsized":"2024-04-26T01:49:20.797486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:09:52.774007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.803115+0000","last_change":"2024-04-26T01:48:22.748601+0000","last_active":"2024-04-26T01:49:20.803115+0000","last_peered":"2024-04-26T01:49:20.803115+0000","last_clean":"2024-04-26T01:49:20.803115+0000","last_became_active":"2024-04-26T01:48:22.748229+0000","last_became_peered":"2024-04-26T01:48:22.748229+0000","last_unstale":"2024-04-26T01:49:20.803115+0000","last_undegraded":"2024-04-26T01:49:20.803115+0000","last_fullsized":"2024-04-26T01:49:20.803115+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:46:30.433063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.814447+0000","last_change":"2024-04-26T01:47:27.123786+0000","last_active":"2024-04-26T01:49:20.814447+0000","last_peered":"2024-04-26T01:49:20.814447+0000","last_clean":"2024-04-26T01:49:20.814447+0000","last_became_active":"2024-04-26T01:47:27.123473+0000","last_became_peered":"2024-04-26T01:47:27.123473+0000","last_unstale":"2024-04-26T01:49:20.814447+0000","last_undegraded":"2024-04-26T01:49:20.814447+0000","last_fullsized":"2024-04-26T01:49:20.814447+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:48:55.031147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.825042+0000","last_change":"2024-04-26T01:47:27.118756+0000","last_active":"2024-04-26T01:49:20.825042+0000","last_peered":"2024-04-26T01:49:20.825042+0000","last_clean":"2024-04-26T01:49:20.825042+0000","last_became_active":"2024-04-26T01:47:27.118406+0000","last_became_peered":"2024-04-26T01:47:27.118406+0000","last_unstale":"2024-04-26T01:49:20.825042+0000","last_undegraded":"2024-04-26T01:49:20.825042+0000","last_fullsized":"2024-04-26T01:49:20.825042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:05.793682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.814403+0000","last_change":"2024-04-26T01:47:27.117622+0000","last_active":"2024-04-26T01:49:20.814403+0000","last_peered":"2024-04-26T01:49:20.814403+0000","last_clean":"2024-04-26T01:49:20.814403+0000","last_became_active":"2024-04-26T01:47:27.115206+0000","last_became_peered":"2024-04-26T01:47:27.115206+0000","last_unstale":"2024-04-26T01:49:20.814403+0000","last_undegraded":"2024-04-26T01:49:20.814403+0000","last_fullsized":"2024-04-26T01:49:20.814403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:13:17.866630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.805457+0000","last_change":"2024-04-26T01:47:27.119524+0000","last_active":"2024-04-26T01:49:20.805457+0000","last_peered":"2024-04-26T01:49:20.805457+0000","last_clean":"2024-04-26T01:49:20.805457+0000","last_became_active":"2024-04-26T01:47:27.119008+0000","last_became_peered":"2024-04-26T01:47:27.119008+0000","last_unstale":"2024-04-26T01:49:20.805457+0000","last_undegraded":"2024-04-26T01:49:20.805457+0000","last_fullsized":"2024-04-26T01:49:20.805457+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:37:31.894486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.814077+0000","last_change":"2024-04-26T01:47:27.122361+0000","last_active":"2024-04-26T01:49:20.814077+0000","last_peered":"2024-04-26T01:49:20.814077+0000","last_clean":"2024-04-26T01:49:20.814077+0000","last_became_active":"2024-04-26T01:47:27.121748+0000","last_became_peered":"2024-04-26T01:47:27.121748+0000","last_unstale":"2024-04-26T01:49:20.814077+0000","last_undegraded":"2024-04-26T01:49:20.814077+0000","last_fullsized":"2024-04-26T01:49:20.814077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:18.680909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.803636+0000","last_change":"2024-04-26T01:47:27.116833+0000","last_active":"2024-04-26T01:49:20.803636+0000","last_peered":"2024-04-26T01:49:20.803636+0000","last_clean":"2024-04-26T01:49:20.803636+0000","last_became_active":"2024-04-26T01:47:27.116508+0000","last_became_peered":"2024-04-26T01:47:27.116508+0000","last_unstale":"2024-04-26T01:49:20.803636+0000","last_undegraded":"2024-04-26T01:49:20.803636+0000","last_fullsized":"2024-04-26T01:49:20.803636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:27:45.864521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.825409+0000","last_change":"2024-04-26T01:47:27.125342+0000","last_active":"2024-04-26T01:49:20.825409+0000","last_peered":"2024-04-26T01:49:20.825409+0000","last_clean":"2024-04-26T01:49:20.825409+0000","last_became_active":"2024-04-26T01:47:27.124862+0000","last_became_peered":"2024-04-26T01:47:27.124862+0000","last_unstale":"2024-04-26T01:49:20.825409+0000","last_undegraded":"2024-04-26T01:49:20.825409+0000","last_fullsized":"2024-04-26T01:49:20.825409+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:48.654711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.831441+0000","last_change":"2024-04-26T01:47:27.127365+0000","last_active":"2024-04-26T01:49:20.831441+0000","last_peered":"2024-04-26T01:49:20.831441+0000","last_clean":"2024-04-26T01:49:20.831441+0000","last_became_active":"2024-04-26T01:47:27.126868+0000","last_became_peered":"2024-04-26T01:47:27.126868+0000","last_unstale":"2024-04-26T01:49:20.831441+0000","last_undegraded":"2024-04-26T01:49:20.831441+0000","last_fullsized":"2024-04-26T01:49:20.831441+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:07:46.400333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.831822+0000","last_change":"2024-04-26T01:47:27.135584+0000","last_active":"2024-04-26T01:49:20.831822+0000","last_peered":"2024-04-26T01:49:20.831822+0000","last_clean":"2024-04-26T01:49:20.831822+0000","last_became_active":"2024-04-26T01:47:27.134849+0000","last_became_peered":"2024-04-26T01:47:27.134849+0000","last_unstale":"2024-04-26T01:49:20.831822+0000","last_undegraded":"2024-04-26T01:49:20.831822+0000","last_fullsized":"2024-04-26T01:49:20.831822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:57:48.821595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.834029+0000","last_change":"2024-04-26T01:47:27.136182+0000","last_active":"2024-04-26T01:49:20.834029+0000","last_peered":"2024-04-26T01:49:20.834029+0000","last_clean":"2024-04-26T01:49:20.834029+0000","last_became_active":"2024-04-26T01:47:27.135787+0000","last_became_peered":"2024-04-26T01:47:27.135787+0000","last_unstale":"2024-04-26T01:49:20.834029+0000","last_undegraded":"2024-04-26T01:49:20.834029+0000","last_fullsized":"2024-04-26T01:49:20.834029+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:38:14.275898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.815346+0000","last_change":"2024-04-26T01:47:27.140700+0000","last_active":"2024-04-26T01:49:20.815346+0000","last_peered":"2024-04-26T01:49:20.815346+0000","last_clean":"2024-04-26T01:49:20.815346+0000","last_became_active":"2024-04-26T01:47:27.140025+0000","last_became_peered":"2024-04-26T01:47:27.140025+0000","last_unstale":"2024-04-26T01:49:20.815346+0000","last_undegraded":"2024-04-26T01:49:20.815346+0000","last_fullsized":"2024-04-26T01:49:20.815346+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:50.465236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.803099+0000","last_change":"2024-04-26T01:47:27.117256+0000","last_active":"2024-04-26T01:49:20.803099+0000","last_peered":"2024-04-26T01:49:20.803099+0000","last_clean":"2024-04-26T01:49:20.803099+0000","last_became_active":"2024-04-26T01:47:27.116825+0000","last_became_peered":"2024-04-26T01:47:27.116825+0000","last_unstale":"2024-04-26T01:49:20.803099+0000","last_undegraded":"2024-04-26T01:49:20.803099+0000","last_fullsized":"2024-04-26T01:49:20.803099+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:23.313712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.820168+0000","last_change":"2024-04-26T01:47:27.130233+0000","last_active":"2024-04-26T01:49:20.820168+0000","last_peered":"2024-04-26T01:49:20.820168+0000","last_clean":"2024-04-26T01:49:20.820168+0000","last_became_active":"2024-04-26T01:47:27.129852+0000","last_became_peered":"2024-04-26T01:47:27.129852+0000","last_unstale":"2024-04-26T01:49:20.820168+0000","last_undegraded":"2024-04-26T01:49:20.820168+0000","last_fullsized":"2024-04-26T01:49:20.820168+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:49:01.407264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.749156+0000","last_change":"2024-04-26T01:47:23.068477+0000","last_active":"2024-04-26T01:48:22.749156+0000","last_peered":"2024-04-26T01:48:22.749156+0000","last_clean":"2024-04-26T01:48:22.749156+0000","last_became_active":"2024-04-26T01:47:23.068061+0000","last_became_peered":"2024-04-26T01:47:23.068061+0000","last_unstale":"2024-04-26T01:48:22.749156+0000","last_undegraded":"2024-04-26T01:48:22.749156+0000","last_fullsized":"2024-04-26T01:48:22.749156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:02.153724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.811871+0000","last_change":"2024-04-26T01:47:27.130385+0000","last_active":"2024-04-26T01:49:20.811871+0000","last_peered":"2024-04-26T01:49:20.811871+0000","last_clean":"2024-04-26T01:49:20.811871+0000","last_became_active":"2024-04-26T01:47:27.129912+0000","last_became_peered":"2024-04-26T01:47:27.129912+0000","last_unstale":"2024-04-26T01:49:20.811871+0000","last_undegraded":"2024-04-26T01:49:20.811871+0000","last_fullsized":"2024-04-26T01:49:20.811871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:26:03.010544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.408077+0000","last_change":"2024-04-26T01:47:23.110806+0000","last_active":"2024-04-26T01:48:22.408077+0000","last_peered":"2024-04-26T01:48:22.408077+0000","last_clean":"2024-04-26T01:48:22.408077+0000","last_became_active":"2024-04-26T01:47:23.110441+0000","last_became_peered":"2024-04-26T01:47:23.110441+0000","last_unstale":"2024-04-26T01:48:22.408077+0000","last_undegraded":"2024-04-26T01:48:22.408077+0000","last_fullsized":"2024-04-26T01:48:22.408077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:39.239172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.832835+0000","last_change":"2024-04-26T01:47:27.138628+0000","last_active":"2024-04-26T01:49:20.832835+0000","last_peered":"2024-04-26T01:49:20.832835+0000","last_clean":"2024-04-26T01:49:20.832835+0000","last_became_active":"2024-04-26T01:47:27.130483+0000","last_became_peered":"2024-04-26T01:47:27.130483+0000","last_unstale":"2024-04-26T01:49:20.832835+0000","last_undegraded":"2024-04-26T01:49:20.832835+0000","last_fullsized":"2024-04-26T01:49:20.832835+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:40:41.242856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.397187+0000","last_change":"2024-04-26T01:47:23.060066+0000","last_active":"2024-04-26T01:48:22.397187+0000","last_peered":"2024-04-26T01:48:22.397187+0000","last_clean":"2024-04-26T01:48:22.397187+0000","last_became_active":"2024-04-26T01:47:23.059526+0000","last_became_peered":"2024-04-26T01:47:23.059526+0000","last_unstale":"2024-04-26T01:48:22.397187+0000","last_undegraded":"2024-04-26T01:48:22.397187+0000","last_fullsized":"2024-04-26T01:48:22.397187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:28:22.623695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.827183+0000","last_change":"2024-04-26T01:47:27.128343+0000","last_active":"2024-04-26T01:49:20.827183+0000","last_peered":"2024-04-26T01:49:20.827183+0000","last_clean":"2024-04-26T01:49:20.827183+0000","last_became_active":"2024-04-26T01:47:27.127920+0000","last_became_peered":"2024-04-26T01:47:27.127920+0000","last_unstale":"2024-04-26T01:49:20.827183+0000","last_undegraded":"2024-04-26T01:49:20.827183+0000","last_fullsized":"2024-04-26T01:49:20.827183+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:06.725033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396988+0000","last_change":"2024-04-26T01:47:23.057783+0000","last_active":"2024-04-26T01:48:22.396988+0000","last_peered":"2024-04-26T01:48:22.396988+0000","last_clean":"2024-04-26T01:48:22.396988+0000","last_became_active":"2024-04-26T01:47:23.057391+0000","last_became_peered":"2024-04-26T01:47:23.057391+0000","last_unstale":"2024-04-26T01:48:22.396988+0000","last_undegraded":"2024-04-26T01:48:22.396988+0000","last_fullsized":"2024-04-26T01:48:22.396988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:10:10.050787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.807130+0000","last_change":"2024-04-26T01:47:27.116458+0000","last_active":"2024-04-26T01:49:20.807130+0000","last_peered":"2024-04-26T01:49:20.807130+0000","last_clean":"2024-04-26T01:49:20.807130+0000","last_became_active":"2024-04-26T01:47:27.116084+0000","last_became_peered":"2024-04-26T01:47:27.116084+0000","last_unstale":"2024-04-26T01:49:20.807130+0000","last_undegraded":"2024-04-26T01:49:20.807130+0000","last_fullsized":"2024-04-26T01:49:20.807130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:06:34.517478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":163,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:57.448416+0000","last_change":"2024-04-26T01:47:23.060657+0000","last_active":"2024-04-26T01:49:57.448416+0000","last_peered":"2024-04-26T01:49:57.448416+0000","last_clean":"2024-04-26T01:49:57.448416+0000","last_became_active":"2024-04-26T01:47:23.060294+0000","last_became_peered":"2024-04-26T01:47:23.060294+0000","last_unstale":"2024-04-26T01:49:57.448416+0000","last_undegraded":"2024-04-26T01:49:57.448416+0000","last_fullsized":"2024-04-26T01:49:57.448416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:33.979717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.794722+0000","last_change":"2024-04-26T01:47:27.120587+0000","last_active":"2024-04-26T01:49:20.794722+0000","last_peered":"2024-04-26T01:49:20.794722+0000","last_clean":"2024-04-26T01:49:20.794722+0000","last_became_active":"2024-04-26T01:47:27.120236+0000","last_became_peered":"2024-04-26T01:47:27.120236+0000","last_unstale":"2024-04-26T01:49:20.794722+0000","last_undegraded":"2024-04-26T01:49:20.794722+0000","last_fullsized":"2024-04-26T01:49:20.794722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:30:38.488741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405133+0000","last_change":"2024-04-26T01:47:23.069395+0000","last_active":"2024-04-26T01:48:22.405133+0000","last_peered":"2024-04-26T01:48:22.405133+0000","last_clean":"2024-04-26T01:48:22.405133+0000","last_became_active":"2024-04-26T01:47:23.069102+0000","last_became_peered":"2024-04-26T01:47:23.069102+0000","last_unstale":"2024-04-26T01:48:22.405133+0000","last_undegraded":"2024-04-26T01:48:22.405133+0000","last_fullsized":"2024-04-26T01:48:22.405133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:45.162604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.805279+0000","last_change":"2024-04-26T01:47:27.123859+0000","last_active":"2024-04-26T01:49:20.805279+0000","last_peered":"2024-04-26T01:49:20.805279+0000","last_clean":"2024-04-26T01:49:20.805279+0000","last_became_active":"2024-04-26T01:47:27.123438+0000","last_became_peered":"2024-04-26T01:47:27.123438+0000","last_unstale":"2024-04-26T01:49:20.805279+0000","last_undegraded":"2024-04-26T01:49:20.805279+0000","last_fullsized":"2024-04-26T01:49:20.805279+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:33:53.542809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.401196+0000","last_change":"2024-04-26T01:47:23.071544+0000","last_active":"2024-04-26T01:48:22.401196+0000","last_peered":"2024-04-26T01:48:22.401196+0000","last_clean":"2024-04-26T01:48:22.401196+0000","last_became_active":"2024-04-26T01:47:23.071191+0000","last_became_peered":"2024-04-26T01:47:23.071191+0000","last_unstale":"2024-04-26T01:48:22.401196+0000","last_undegraded":"2024-04-26T01:48:22.401196+0000","last_fullsized":"2024-04-26T01:48:22.401196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:58.889561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.821717+0000","last_change":"2024-04-26T01:47:27.131887+0000","last_active":"2024-04-26T01:49:20.821717+0000","last_peered":"2024-04-26T01:49:20.821717+0000","last_clean":"2024-04-26T01:49:20.821717+0000","last_became_active":"2024-04-26T01:47:27.127973+0000","last_became_peered":"2024-04-26T01:47:27.127973+0000","last_unstale":"2024-04-26T01:49:20.821717+0000","last_undegraded":"2024-04-26T01:49:20.821717+0000","last_fullsized":"2024-04-26T01:49:20.821717+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:54:35.305779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745664+0000","last_change":"2024-04-26T01:47:23.061353+0000","last_active":"2024-04-26T01:48:22.745664+0000","last_peered":"2024-04-26T01:48:22.745664+0000","last_clean":"2024-04-26T01:48:22.745664+0000","last_became_active":"2024-04-26T01:47:23.061037+0000","last_became_peered":"2024-04-26T01:47:23.061037+0000","last_unstale":"2024-04-26T01:48:22.745664+0000","last_undegraded":"2024-04-26T01:48:22.745664+0000","last_fullsized":"2024-04-26T01:48:22.745664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:13:59.617969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.820690+0000","last_change":"2024-04-26T01:47:27.122303+0000","last_active":"2024-04-26T01:49:20.820690+0000","last_peered":"2024-04-26T01:49:20.820690+0000","last_clean":"2024-04-26T01:49:20.820690+0000","last_became_active":"2024-04-26T01:47:27.121919+0000","last_became_peered":"2024-04-26T01:47:27.121919+0000","last_unstale":"2024-04-26T01:49:20.820690+0000","last_undegraded":"2024-04-26T01:49:20.820690+0000","last_fullsized":"2024-04-26T01:49:20.820690+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:21.676663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":161,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:59.450941+0000","last_change":"2024-04-26T01:47:23.057961+0000","last_active":"2024-04-26T01:49:59.450941+0000","last_peered":"2024-04-26T01:49:59.450941+0000","last_clean":"2024-04-26T01:49:59.450941+0000","last_became_active":"2024-04-26T01:47:23.057529+0000","last_became_peered":"2024-04-26T01:47:23.057529+0000","last_unstale":"2024-04-26T01:49:59.450941+0000","last_undegraded":"2024-04-26T01:49:59.450941+0000","last_fullsized":"2024-04-26T01:49:59.450941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:14.420265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":131,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.813684+0000","last_change":"2024-04-26T01:47:27.120083+0000","last_active":"2024-04-26T01:49:20.813684+0000","last_peered":"2024-04-26T01:49:20.813684+0000","last_clean":"2024-04-26T01:49:20.813684+0000","last_became_active":"2024-04-26T01:47:27.118410+0000","last_became_peered":"2024-04-26T01:47:27.118410+0000","last_unstale":"2024-04-26T01:49:20.813684+0000","last_undegraded":"2024-04-26T01:49:20.813684+0000","last_fullsized":"2024-04-26T01:49:20.813684+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:28.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400284+0000","last_change":"2024-04-26T01:47:23.059221+0000","last_active":"2024-04-26T01:48:22.400284+0000","last_peered":"2024-04-26T01:48:22.400284+0000","last_clean":"2024-04-26T01:48:22.400284+0000","last_became_active":"2024-04-26T01:47:23.058881+0000","last_became_peered":"2024-04-26T01:47:23.058881+0000","last_unstale":"2024-04-26T01:48:22.400284+0000","last_undegraded":"2024-04-26T01:48:22.400284+0000","last_fullsized":"2024-04-26T01:48:22.400284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:30.433696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.829291+0000","last_change":"2024-04-26T01:47:27.136763+0000","last_active":"2024-04-26T01:49:20.829291+0000","last_peered":"2024-04-26T01:49:20.829291+0000","last_clean":"2024-04-26T01:49:20.829291+0000","last_became_active":"2024-04-26T01:47:27.133642+0000","last_became_peered":"2024-04-26T01:47:27.133642+0000","last_unstale":"2024-04-26T01:49:20.829291+0000","last_undegraded":"2024-04-26T01:49:20.829291+0000","last_fullsized":"2024-04-26T01:49:20.829291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:05.901139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746141+0000","last_change":"2024-04-26T01:47:23.108883+0000","last_active":"2024-04-26T01:48:22.746141+0000","last_peered":"2024-04-26T01:48:22.746141+0000","last_clean":"2024-04-26T01:48:22.746141+0000","last_became_active":"2024-04-26T01:47:23.108499+0000","last_became_peered":"2024-04-26T01:47:23.108499+0000","last_unstale":"2024-04-26T01:48:22.746141+0000","last_undegraded":"2024-04-26T01:48:22.746141+0000","last_fullsized":"2024-04-26T01:48:22.746141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:14:17.555416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.827266+0000","last_change":"2024-04-26T01:48:22.410651+0000","last_active":"2024-04-26T01:49:20.827266+0000","last_peered":"2024-04-26T01:49:20.827266+0000","last_clean":"2024-04-26T01:49:20.827266+0000","last_became_active":"2024-04-26T01:48:22.410359+0000","last_became_peered":"2024-04-26T01:48:22.410359+0000","last_unstale":"2024-04-26T01:49:20.827266+0000","last_undegraded":"2024-04-26T01:49:20.827266+0000","last_fullsized":"2024-04-26T01:49:20.827266+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:53:20.686915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'32","reported_seq":331,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:00.460302+0000","last_change":"2024-04-26T01:47:23.054893+0000","last_active":"2024-04-26T01:50:00.460302+0000","last_peered":"2024-04-26T01:50:00.460302+0000","last_clean":"2024-04-26T01:50:00.460302+0000","last_became_active":"2024-04-26T01:47:23.054558+0000","last_became_peered":"2024-04-26T01:47:23.054558+0000","last_unstale":"2024-04-26T01:50:00.460302+0000","last_undegraded":"2024-04-26T01:50:00.460302+0000","last_fullsized":"2024-04-26T01:50:00.460302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:59.356841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60596,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":272,"num_read_kb":195,"num_write":33,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.815218+0000","last_change":"2024-04-26T01:47:27.116934+0000","last_active":"2024-04-26T01:49:20.815218+0000","last_peered":"2024-04-26T01:49:20.815218+0000","last_clean":"2024-04-26T01:49:20.815218+0000","last_became_active":"2024-04-26T01:47:27.116527+0000","last_became_peered":"2024-04-26T01:47:27.116527+0000","last_unstale":"2024-04-26T01:49:20.815218+0000","last_undegraded":"2024-04-26T01:49:20.815218+0000","last_fullsized":"2024-04-26T01:49:20.815218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:09:45.820533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399124+0000","last_change":"2024-04-26T01:47:23.059032+0000","last_active":"2024-04-26T01:48:22.399124+0000","last_peered":"2024-04-26T01:48:22.399124+0000","last_clean":"2024-04-26T01:48:22.399124+0000","last_became_active":"2024-04-26T01:47:23.057992+0000","last_became_peered":"2024-04-26T01:47:23.057992+0000","last_unstale":"2024-04-26T01:48:22.399124+0000","last_undegraded":"2024-04-26T01:48:22.399124+0000","last_fullsized":"2024-04-26T01:48:22.399124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:46:53.164925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.810758+0000","last_change":"2024-04-26T01:47:27.134574+0000","last_active":"2024-04-26T01:49:20.810758+0000","last_peered":"2024-04-26T01:49:20.810758+0000","last_clean":"2024-04-26T01:49:20.810758+0000","last_became_active":"2024-04-26T01:47:27.134184+0000","last_became_peered":"2024-04-26T01:47:27.134184+0000","last_unstale":"2024-04-26T01:49:20.810758+0000","last_undegraded":"2024-04-26T01:49:20.810758+0000","last_fullsized":"2024-04-26T01:49:20.810758+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:42:26.428014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745707+0000","last_change":"2024-04-26T01:47:23.062649+0000","last_active":"2024-04-26T01:48:22.745707+0000","last_peered":"2024-04-26T01:48:22.745707+0000","last_clean":"2024-04-26T01:48:22.745707+0000","last_became_active":"2024-04-26T01:47:23.062282+0000","last_became_peered":"2024-04-26T01:47:23.062282+0000","last_unstale":"2024-04-26T01:48:22.745707+0000","last_undegraded":"2024-04-26T01:48:22.745707+0000","last_fullsized":"2024-04-26T01:48:22.745707+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:38.699105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.826254+0000","last_change":"2024-04-26T01:47:27.124720+0000","last_active":"2024-04-26T01:49:20.826254+0000","last_peered":"2024-04-26T01:49:20.826254+0000","last_clean":"2024-04-26T01:49:20.826254+0000","last_became_active":"2024-04-26T01:47:27.124378+0000","last_became_peered":"2024-04-26T01:47:27.124378+0000","last_unstale":"2024-04-26T01:49:20.826254+0000","last_undegraded":"2024-04-26T01:49:20.826254+0000","last_fullsized":"2024-04-26T01:49:20.826254+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:45.462741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":165,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:57.513298+0000","last_change":"2024-04-26T01:47:23.056305+0000","last_active":"2024-04-26T01:49:57.513298+0000","last_peered":"2024-04-26T01:49:57.513298+0000","last_clean":"2024-04-26T01:49:57.513298+0000","last_became_active":"2024-04-26T01:47:23.055905+0000","last_became_peered":"2024-04-26T01:47:23.055905+0000","last_unstale":"2024-04-26T01:49:57.513298+0000","last_undegraded":"2024-04-26T01:49:57.513298+0000","last_fullsized":"2024-04-26T01:49:57.513298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:00:14.609673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":134,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.814879+0000","last_change":"2024-04-26T01:47:27.119880+0000","last_active":"2024-04-26T01:49:20.814879+0000","last_peered":"2024-04-26T01:49:20.814879+0000","last_clean":"2024-04-26T01:49:20.814879+0000","last_became_active":"2024-04-26T01:47:27.119483+0000","last_became_peered":"2024-04-26T01:47:27.119483+0000","last_unstale":"2024-04-26T01:49:20.814879+0000","last_undegraded":"2024-04-26T01:49:20.814879+0000","last_fullsized":"2024-04-26T01:49:20.814879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:15.523150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":67,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.759238+0000","last_change":"2024-04-26T01:48:22.758774+0000","last_active":"2024-04-26T01:48:22.759238+0000","last_peered":"2024-04-26T01:48:22.759238+0000","last_clean":"2024-04-26T01:48:22.759238+0000","last_became_active":"2024-04-26T01:48:22.747354+0000","last_became_peered":"2024-04-26T01:48:22.747354+0000","last_unstale":"2024-04-26T01:48:22.759238+0000","last_undegraded":"2024-04-26T01:48:22.759238+0000","last_fullsized":"2024-04-26T01:48:22.759238+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:23:15.886774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.806186+0000","last_change":"2024-04-26T01:47:27.113508+0000","last_active":"2024-04-26T01:49:20.806186+0000","last_peered":"2024-04-26T01:49:20.806186+0000","last_clean":"2024-04-26T01:49:20.806186+0000","last_became_active":"2024-04-26T01:47:27.113066+0000","last_became_peered":"2024-04-26T01:47:27.113066+0000","last_unstale":"2024-04-26T01:49:20.806186+0000","last_undegraded":"2024-04-26T01:49:20.806186+0000","last_fullsized":"2024-04-26T01:49:20.806186+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:37.525486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.397112+0000","last_change":"2024-04-26T01:47:23.059644+0000","last_active":"2024-04-26T01:48:22.397112+0000","last_peered":"2024-04-26T01:48:22.397112+0000","last_clean":"2024-04-26T01:48:22.397112+0000","last_became_active":"2024-04-26T01:47:23.058966+0000","last_became_peered":"2024-04-26T01:47:23.058966+0000","last_unstale":"2024-04-26T01:48:22.397112+0000","last_undegraded":"2024-04-26T01:48:22.397112+0000","last_fullsized":"2024-04-26T01:48:22.397112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:57:36.475935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.796390+0000","last_change":"2024-04-26T01:48:22.749354+0000","last_active":"2024-04-26T01:49:20.796390+0000","last_peered":"2024-04-26T01:49:20.796390+0000","last_clean":"2024-04-26T01:49:20.796390+0000","last_became_active":"2024-04-26T01:48:22.748978+0000","last_became_peered":"2024-04-26T01:48:22.748978+0000","last_unstale":"2024-04-26T01:49:20.796390+0000","last_undegraded":"2024-04-26T01:49:20.796390+0000","last_fullsized":"2024-04-26T01:49:20.796390+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:51:16.087091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746722+0000","last_change":"2024-04-26T01:47:23.110360+0000","last_active":"2024-04-26T01:48:22.746722+0000","last_peered":"2024-04-26T01:48:22.746722+0000","last_clean":"2024-04-26T01:48:22.746722+0000","last_became_active":"2024-04-26T01:47:23.109744+0000","last_became_peered":"2024-04-26T01:47:23.109744+0000","last_unstale":"2024-04-26T01:48:22.746722+0000","last_undegraded":"2024-04-26T01:48:22.746722+0000","last_fullsized":"2024-04-26T01:48:22.746722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:40:13.658396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.825600+0000","last_change":"2024-04-26T01:47:27.125399+0000","last_active":"2024-04-26T01:49:20.825600+0000","last_peered":"2024-04-26T01:49:20.825600+0000","last_clean":"2024-04-26T01:49:20.825600+0000","last_became_active":"2024-04-26T01:47:27.124871+0000","last_became_peered":"2024-04-26T01:47:27.124871+0000","last_unstale":"2024-04-26T01:49:20.825600+0000","last_undegraded":"2024-04-26T01:49:20.825600+0000","last_fullsized":"2024-04-26T01:49:20.825600+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:28:48.285206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400231+0000","last_change":"2024-04-26T01:47:23.067862+0000","last_active":"2024-04-26T01:48:22.400231+0000","last_peered":"2024-04-26T01:48:22.400231+0000","last_clean":"2024-04-26T01:48:22.400231+0000","last_became_active":"2024-04-26T01:47:23.067526+0000","last_became_peered":"2024-04-26T01:47:23.067526+0000","last_unstale":"2024-04-26T01:48:22.400231+0000","last_undegraded":"2024-04-26T01:48:22.400231+0000","last_fullsized":"2024-04-26T01:48:22.400231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:22.668405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.804364+0000","last_change":"2024-04-26T01:48:22.748548+0000","last_active":"2024-04-26T01:49:20.804364+0000","last_peered":"2024-04-26T01:49:20.804364+0000","last_clean":"2024-04-26T01:49:20.804364+0000","last_became_active":"2024-04-26T01:48:22.748182+0000","last_became_peered":"2024-04-26T01:48:22.748182+0000","last_unstale":"2024-04-26T01:49:20.804364+0000","last_undegraded":"2024-04-26T01:49:20.804364+0000","last_fullsized":"2024-04-26T01:49:20.804364+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:20:27.596827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.798153+0000","last_change":"2024-04-26T01:47:27.118307+0000","last_active":"2024-04-26T01:49:20.798153+0000","last_peered":"2024-04-26T01:49:20.798153+0000","last_clean":"2024-04-26T01:49:20.798153+0000","last_became_active":"2024-04-26T01:47:27.117986+0000","last_became_peered":"2024-04-26T01:47:27.117986+0000","last_unstale":"2024-04-26T01:49:20.798153+0000","last_undegraded":"2024-04-26T01:49:20.798153+0000","last_fullsized":"2024-04-26T01:49:20.798153+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:40.667143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":163,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:59.517387+0000","last_change":"2024-04-26T01:47:23.109973+0000","last_active":"2024-04-26T01:49:59.517387+0000","last_peered":"2024-04-26T01:49:59.517387+0000","last_clean":"2024-04-26T01:49:59.517387+0000","last_became_active":"2024-04-26T01:47:23.109446+0000","last_became_peered":"2024-04-26T01:47:23.109446+0000","last_unstale":"2024-04-26T01:49:59.517387+0000","last_undegraded":"2024-04-26T01:49:59.517387+0000","last_fullsized":"2024-04-26T01:49:59.517387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:49:00.684164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":131,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.821119+0000","last_change":"2024-04-26T01:47:27.131502+0000","last_active":"2024-04-26T01:49:20.821119+0000","last_peered":"2024-04-26T01:49:20.821119+0000","last_clean":"2024-04-26T01:49:20.821119+0000","last_became_active":"2024-04-26T01:47:27.127697+0000","last_became_peered":"2024-04-26T01:47:27.127697+0000","last_unstale":"2024-04-26T01:49:20.821119+0000","last_undegraded":"2024-04-26T01:49:20.821119+0000","last_fullsized":"2024-04-26T01:49:20.821119+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:39:15.458189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746759+0000","last_change":"2024-04-26T01:47:23.066596+0000","last_active":"2024-04-26T01:48:22.746759+0000","last_peered":"2024-04-26T01:48:22.746759+0000","last_clean":"2024-04-26T01:48:22.746759+0000","last_became_active":"2024-04-26T01:47:23.066216+0000","last_became_peered":"2024-04-26T01:47:23.066216+0000","last_unstale":"2024-04-26T01:48:22.746759+0000","last_undegraded":"2024-04-26T01:48:22.746759+0000","last_fullsized":"2024-04-26T01:48:22.746759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:08:31.544831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400162+0000","last_change":"2024-04-26T01:45:54.834501+0000","last_active":"2024-04-26T01:48:22.400162+0000","last_peered":"2024-04-26T01:48:22.400162+0000","last_clean":"2024-04-26T01:48:22.400162+0000","last_became_active":"2024-04-26T01:45:54.833868+0000","last_became_peered":"2024-04-26T01:45:54.833868+0000","last_unstale":"2024-04-26T01:48:22.400162+0000","last_undegraded":"2024-04-26T01:48:22.400162+0000","last_fullsized":"2024-04-26T01:48:22.400162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:42:05.228956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:42:05.228956+0000","last_clean_scrub_stamp":"2024-04-26T01:42:05.228956+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:06:58.383315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.797852+0000","last_change":"2024-04-26T01:47:27.126475+0000","last_active":"2024-04-26T01:49:20.797852+0000","last_peered":"2024-04-26T01:49:20.797852+0000","last_clean":"2024-04-26T01:49:20.797852+0000","last_became_active":"2024-04-26T01:47:27.123760+0000","last_became_peered":"2024-04-26T01:47:27.123760+0000","last_unstale":"2024-04-26T01:49:20.797852+0000","last_undegraded":"2024-04-26T01:49:20.797852+0000","last_fullsized":"2024-04-26T01:49:20.797852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:38:27.473909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400412+0000","last_change":"2024-04-26T01:47:23.059810+0000","last_active":"2024-04-26T01:48:22.400412+0000","last_peered":"2024-04-26T01:48:22.400412+0000","last_clean":"2024-04-26T01:48:22.400412+0000","last_became_active":"2024-04-26T01:47:23.059504+0000","last_became_peered":"2024-04-26T01:47:23.059504+0000","last_unstale":"2024-04-26T01:48:22.400412+0000","last_undegraded":"2024-04-26T01:48:22.400412+0000","last_fullsized":"2024-04-26T01:48:22.400412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:50:51.220951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.397081+0000","last_change":"2024-04-26T01:47:23.109460+0000","last_active":"2024-04-26T01:48:22.397081+0000","last_peered":"2024-04-26T01:48:22.397081+0000","last_clean":"2024-04-26T01:48:22.397081+0000","last_became_active":"2024-04-26T01:47:23.109090+0000","last_became_peered":"2024-04-26T01:47:23.109090+0000","last_unstale":"2024-04-26T01:48:22.397081+0000","last_undegraded":"2024-04-26T01:48:22.397081+0000","last_fullsized":"2024-04-26T01:48:22.397081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:56.821312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.802492+0000","last_change":"2024-04-26T01:47:27.129537+0000","last_active":"2024-04-26T01:49:20.802492+0000","last_peered":"2024-04-26T01:49:20.802492+0000","last_clean":"2024-04-26T01:49:20.802492+0000","last_became_active":"2024-04-26T01:47:27.128124+0000","last_became_peered":"2024-04-26T01:47:27.128124+0000","last_unstale":"2024-04-26T01:49:20.802492+0000","last_undegraded":"2024-04-26T01:49:20.802492+0000","last_fullsized":"2024-04-26T01:49:20.802492+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:14:26.262197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:21.141469+0000","last_change":"2024-04-26T01:47:23.064511+0000","last_active":"2024-04-26T01:49:21.141469+0000","last_peered":"2024-04-26T01:49:21.141469+0000","last_clean":"2024-04-26T01:49:21.141469+0000","last_became_active":"2024-04-26T01:47:23.064080+0000","last_became_peered":"2024-04-26T01:47:23.064080+0000","last_unstale":"2024-04-26T01:49:21.141469+0000","last_undegraded":"2024-04-26T01:49:21.141469+0000","last_fullsized":"2024-04-26T01:49:21.141469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:37.562644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.814995+0000","last_change":"2024-04-26T01:47:27.124057+0000","last_active":"2024-04-26T01:49:20.814995+0000","last_peered":"2024-04-26T01:49:20.814995+0000","last_clean":"2024-04-26T01:49:20.814995+0000","last_became_active":"2024-04-26T01:47:27.123636+0000","last_became_peered":"2024-04-26T01:47:27.123636+0000","last_unstale":"2024-04-26T01:49:20.814995+0000","last_undegraded":"2024-04-26T01:49:20.814995+0000","last_fullsized":"2024-04-26T01:49:20.814995+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:56:55.691697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746211+0000","last_change":"2024-04-26T01:47:23.136571+0000","last_active":"2024-04-26T01:48:22.746211+0000","last_peered":"2024-04-26T01:48:22.746211+0000","last_clean":"2024-04-26T01:48:22.746211+0000","last_became_active":"2024-04-26T01:47:23.136205+0000","last_became_peered":"2024-04-26T01:47:23.136205+0000","last_unstale":"2024-04-26T01:48:22.746211+0000","last_undegraded":"2024-04-26T01:48:22.746211+0000","last_fullsized":"2024-04-26T01:48:22.746211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:24.418797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.828229+0000","last_change":"2024-04-26T01:48:22.747570+0000","last_active":"2024-04-26T01:49:20.828229+0000","last_peered":"2024-04-26T01:49:20.828229+0000","last_clean":"2024-04-26T01:49:20.828229+0000","last_became_active":"2024-04-26T01:48:22.747147+0000","last_became_peered":"2024-04-26T01:48:22.747147+0000","last_unstale":"2024-04-26T01:49:20.828229+0000","last_undegraded":"2024-04-26T01:49:20.828229+0000","last_fullsized":"2024-04-26T01:49:20.828229+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:37.034388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.414985+0000","last_change":"2024-04-26T01:48:22.414610+0000","last_active":"2024-04-26T01:48:22.414985+0000","last_peered":"2024-04-26T01:48:22.414985+0000","last_clean":"2024-04-26T01:48:22.414985+0000","last_became_active":"2024-04-26T01:48:22.410275+0000","last_became_peered":"2024-04-26T01:48:22.410275+0000","last_unstale":"2024-04-26T01:48:22.414985+0000","last_undegraded":"2024-04-26T01:48:22.414985+0000","last_fullsized":"2024-04-26T01:48:22.414985+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:20:48.131988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.783601+0000","last_change":"2024-04-26T01:47:27.115498+0000","last_active":"2024-04-26T01:49:20.783601+0000","last_peered":"2024-04-26T01:49:20.783601+0000","last_clean":"2024-04-26T01:49:20.783601+0000","last_became_active":"2024-04-26T01:47:27.115134+0000","last_became_peered":"2024-04-26T01:47:27.115134+0000","last_unstale":"2024-04-26T01:49:20.783601+0000","last_undegraded":"2024-04-26T01:49:20.783601+0000","last_fullsized":"2024-04-26T01:49:20.783601+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:20.795224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405406+0000","last_change":"2024-04-26T01:47:23.065366+0000","last_active":"2024-04-26T01:48:22.405406+0000","last_peered":"2024-04-26T01:48:22.405406+0000","last_clean":"2024-04-26T01:48:22.405406+0000","last_became_active":"2024-04-26T01:47:23.063216+0000","last_became_peered":"2024-04-26T01:47:23.063216+0000","last_unstale":"2024-04-26T01:48:22.405406+0000","last_undegraded":"2024-04-26T01:48:22.405406+0000","last_fullsized":"2024-04-26T01:48:22.405406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:06:05.261907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.798214+0000","last_change":"2024-04-26T01:47:27.127938+0000","last_active":"2024-04-26T01:49:20.798214+0000","last_peered":"2024-04-26T01:49:20.798214+0000","last_clean":"2024-04-26T01:49:20.798214+0000","last_became_active":"2024-04-26T01:47:27.127553+0000","last_became_peered":"2024-04-26T01:47:27.127553+0000","last_unstale":"2024-04-26T01:49:20.798214+0000","last_undegraded":"2024-04-26T01:49:20.798214+0000","last_fullsized":"2024-04-26T01:49:20.798214+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:17:51.115349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398786+0000","last_change":"2024-04-26T01:47:23.060475+0000","last_active":"2024-04-26T01:48:22.398786+0000","last_peered":"2024-04-26T01:48:22.398786+0000","last_clean":"2024-04-26T01:48:22.398786+0000","last_became_active":"2024-04-26T01:47:23.060098+0000","last_became_peered":"2024-04-26T01:47:23.060098+0000","last_unstale":"2024-04-26T01:48:22.398786+0000","last_undegraded":"2024-04-26T01:48:22.398786+0000","last_fullsized":"2024-04-26T01:48:22.398786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:49.009455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.833701+0000","last_change":"2024-04-26T01:47:27.137491+0000","last_active":"2024-04-26T01:49:20.833701+0000","last_peered":"2024-04-26T01:49:20.833701+0000","last_clean":"2024-04-26T01:49:20.833701+0000","last_became_active":"2024-04-26T01:47:27.137059+0000","last_became_peered":"2024-04-26T01:47:27.137059+0000","last_unstale":"2024-04-26T01:49:20.833701+0000","last_undegraded":"2024-04-26T01:49:20.833701+0000","last_fullsized":"2024-04-26T01:49:20.833701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:20:25.004074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405524+0000","last_change":"2024-04-26T01:48:22.405071+0000","last_active":"2024-04-26T01:48:22.405524+0000","last_peered":"2024-04-26T01:48:22.405524+0000","last_clean":"2024-04-26T01:48:22.405524+0000","last_became_active":"2024-04-26T01:48:22.401282+0000","last_became_peered":"2024-04-26T01:48:22.401282+0000","last_unstale":"2024-04-26T01:48:22.405524+0000","last_undegraded":"2024-04-26T01:48:22.405524+0000","last_fullsized":"2024-04-26T01:48:22.405524+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:24:53.628014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.821279+0000","last_change":"2024-04-26T01:47:27.128253+0000","last_active":"2024-04-26T01:49:20.821279+0000","last_peered":"2024-04-26T01:49:20.821279+0000","last_clean":"2024-04-26T01:49:20.821279+0000","last_became_active":"2024-04-26T01:47:27.127869+0000","last_became_peered":"2024-04-26T01:47:27.127869+0000","last_unstale":"2024-04-26T01:49:20.821279+0000","last_undegraded":"2024-04-26T01:49:20.821279+0000","last_fullsized":"2024-04-26T01:49:20.821279+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:11:23.476938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399460+0000","last_change":"2024-04-26T01:47:23.136862+0000","last_active":"2024-04-26T01:48:22.399460+0000","last_peered":"2024-04-26T01:48:22.399460+0000","last_clean":"2024-04-26T01:48:22.399460+0000","last_became_active":"2024-04-26T01:47:23.136380+0000","last_became_peered":"2024-04-26T01:47:23.136380+0000","last_unstale":"2024-04-26T01:48:22.399460+0000","last_undegraded":"2024-04-26T01:48:22.399460+0000","last_fullsized":"2024-04-26T01:48:22.399460+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:39:18.103559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.815096+0000","last_change":"2024-04-26T01:47:27.113363+0000","last_active":"2024-04-26T01:49:20.815096+0000","last_peered":"2024-04-26T01:49:20.815096+0000","last_clean":"2024-04-26T01:49:20.815096+0000","last_became_active":"2024-04-26T01:47:27.113040+0000","last_became_peered":"2024-04-26T01:47:27.113040+0000","last_unstale":"2024-04-26T01:49:20.815096+0000","last_undegraded":"2024-04-26T01:49:20.815096+0000","last_fullsized":"2024-04-26T01:49:20.815096+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:42:16.428697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746035+0000","last_change":"2024-04-26T01:47:23.067257+0000","last_active":"2024-04-26T01:48:22.746035+0000","last_peered":"2024-04-26T01:48:22.746035+0000","last_clean":"2024-04-26T01:48:22.746035+0000","last_became_active":"2024-04-26T01:47:23.066858+0000","last_became_peered":"2024-04-26T01:47:23.066858+0000","last_unstale":"2024-04-26T01:48:22.746035+0000","last_undegraded":"2024-04-26T01:48:22.746035+0000","last_fullsized":"2024-04-26T01:48:22.746035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:22:07.848074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.820466+0000","last_change":"2024-04-26T01:47:27.125885+0000","last_active":"2024-04-26T01:49:20.820466+0000","last_peered":"2024-04-26T01:49:20.820466+0000","last_clean":"2024-04-26T01:49:20.820466+0000","last_became_active":"2024-04-26T01:47:27.125524+0000","last_became_peered":"2024-04-26T01:47:27.125524+0000","last_unstale":"2024-04-26T01:49:20.820466+0000","last_undegraded":"2024-04-26T01:49:20.820466+0000","last_fullsized":"2024-04-26T01:49:20.820466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:39.282023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.407777+0000","last_change":"2024-04-26T01:47:23.071051+0000","last_active":"2024-04-26T01:48:22.407777+0000","last_peered":"2024-04-26T01:48:22.407777+0000","last_clean":"2024-04-26T01:48:22.407777+0000","last_became_active":"2024-04-26T01:47:23.069524+0000","last_became_peered":"2024-04-26T01:47:23.069524+0000","last_unstale":"2024-04-26T01:48:22.407777+0000","last_undegraded":"2024-04-26T01:48:22.407777+0000","last_fullsized":"2024-04-26T01:48:22.407777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:24:57.643100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.815496+0000","last_change":"2024-04-26T01:47:27.114325+0000","last_active":"2024-04-26T01:49:20.815496+0000","last_peered":"2024-04-26T01:49:20.815496+0000","last_clean":"2024-04-26T01:49:20.815496+0000","last_became_active":"2024-04-26T01:47:27.113952+0000","last_became_peered":"2024-04-26T01:47:27.113952+0000","last_unstale":"2024-04-26T01:49:20.815496+0000","last_undegraded":"2024-04-26T01:49:20.815496+0000","last_fullsized":"2024-04-26T01:49:20.815496+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:39:19.167175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.797195+0000","last_change":"2024-04-26T01:47:27.118709+0000","last_active":"2024-04-26T01:49:20.797195+0000","last_peered":"2024-04-26T01:49:20.797195+0000","last_clean":"2024-04-26T01:49:20.797195+0000","last_became_active":"2024-04-26T01:47:27.118341+0000","last_became_peered":"2024-04-26T01:47:27.118341+0000","last_unstale":"2024-04-26T01:49:20.797195+0000","last_undegraded":"2024-04-26T01:49:20.797195+0000","last_fullsized":"2024-04-26T01:49:20.797195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:13.736180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.795672+0000","last_change":"2024-04-26T01:47:27.119239+0000","last_active":"2024-04-26T01:49:20.795672+0000","last_peered":"2024-04-26T01:49:20.795672+0000","last_clean":"2024-04-26T01:49:20.795672+0000","last_became_active":"2024-04-26T01:47:27.118877+0000","last_became_peered":"2024-04-26T01:47:27.118877+0000","last_unstale":"2024-04-26T01:49:20.795672+0000","last_undegraded":"2024-04-26T01:49:20.795672+0000","last_fullsized":"2024-04-26T01:49:20.795672+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:04:10.033517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.819917+0000","last_change":"2024-04-26T01:47:27.142848+0000","last_active":"2024-04-26T01:49:20.819917+0000","last_peered":"2024-04-26T01:49:20.819917+0000","last_clean":"2024-04-26T01:49:20.819917+0000","last_became_active":"2024-04-26T01:47:27.142580+0000","last_became_peered":"2024-04-26T01:47:27.142580+0000","last_unstale":"2024-04-26T01:49:20.819917+0000","last_undegraded":"2024-04-26T01:49:20.819917+0000","last_fullsized":"2024-04-26T01:49:20.819917+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:15:46.923922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.831630+0000","last_change":"2024-04-26T01:47:27.134817+0000","last_active":"2024-04-26T01:49:20.831630+0000","last_peered":"2024-04-26T01:49:20.831630+0000","last_clean":"2024-04-26T01:49:20.831630+0000","last_became_active":"2024-04-26T01:47:27.134024+0000","last_became_peered":"2024-04-26T01:47:27.134024+0000","last_unstale":"2024-04-26T01:49:20.831630+0000","last_undegraded":"2024-04-26T01:49:20.831630+0000","last_fullsized":"2024-04-26T01:49:20.831630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:56.746427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.828596+0000","last_change":"2024-04-26T01:47:27.143137+0000","last_active":"2024-04-26T01:49:20.828596+0000","last_peered":"2024-04-26T01:49:20.828596+0000","last_clean":"2024-04-26T01:49:20.828596+0000","last_became_active":"2024-04-26T01:47:27.142625+0000","last_became_peered":"2024-04-26T01:47:27.142625+0000","last_unstale":"2024-04-26T01:49:20.828596+0000","last_undegraded":"2024-04-26T01:49:20.828596+0000","last_fullsized":"2024-04-26T01:49:20.828596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:42:50.468382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.798582+0000","last_change":"2024-04-26T01:48:22.749582+0000","last_active":"2024-04-26T01:49:20.798582+0000","last_peered":"2024-04-26T01:49:20.798582+0000","last_clean":"2024-04-26T01:49:20.798582+0000","last_became_active":"2024-04-26T01:48:22.749191+0000","last_became_peered":"2024-04-26T01:48:22.749191+0000","last_unstale":"2024-04-26T01:49:20.798582+0000","last_undegraded":"2024-04-26T01:49:20.798582+0000","last_fullsized":"2024-04-26T01:49:20.798582+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:14.614842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.813096+0000","last_change":"2024-04-26T01:47:27.123964+0000","last_active":"2024-04-26T01:49:20.813096+0000","last_peered":"2024-04-26T01:49:20.813096+0000","last_clean":"2024-04-26T01:49:20.813096+0000","last_became_active":"2024-04-26T01:47:27.123552+0000","last_became_peered":"2024-04-26T01:47:27.123552+0000","last_unstale":"2024-04-26T01:49:20.813096+0000","last_undegraded":"2024-04-26T01:49:20.813096+0000","last_fullsized":"2024-04-26T01:49:20.813096+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:33:12.566105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.811676+0000","last_change":"2024-04-26T01:48:22.412621+0000","last_active":"2024-04-26T01:49:20.811676+0000","last_peered":"2024-04-26T01:49:20.811676+0000","last_clean":"2024-04-26T01:49:20.811676+0000","last_became_active":"2024-04-26T01:48:22.411430+0000","last_became_peered":"2024-04-26T01:48:22.411430+0000","last_unstale":"2024-04-26T01:49:20.811676+0000","last_undegraded":"2024-04-26T01:49:20.811676+0000","last_fullsized":"2024-04-26T01:49:20.811676+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:34.152338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.807847+0000","last_change":"2024-04-26T01:47:27.131437+0000","last_active":"2024-04-26T01:49:20.807847+0000","last_peered":"2024-04-26T01:49:20.807847+0000","last_clean":"2024-04-26T01:49:20.807847+0000","last_became_active":"2024-04-26T01:47:27.131138+0000","last_became_peered":"2024-04-26T01:47:27.131138+0000","last_unstale":"2024-04-26T01:49:20.807847+0000","last_undegraded":"2024-04-26T01:49:20.807847+0000","last_fullsized":"2024-04-26T01:49:20.807847+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:55:39.132972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.820594+0000","last_change":"2024-04-26T01:47:27.120928+0000","last_active":"2024-04-26T01:49:20.820594+0000","last_peered":"2024-04-26T01:49:20.820594+0000","last_clean":"2024-04-26T01:49:20.820594+0000","last_became_active":"2024-04-26T01:47:27.120012+0000","last_became_peered":"2024-04-26T01:47:27.120012+0000","last_unstale":"2024-04-26T01:49:20.820594+0000","last_undegraded":"2024-04-26T01:49:20.820594+0000","last_fullsized":"2024-04-26T01:49:20.820594+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:19:44.209876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.824181+0000","last_change":"2024-04-26T01:47:27.138766+0000","last_active":"2024-04-26T01:49:20.824181+0000","last_peered":"2024-04-26T01:49:20.824181+0000","last_clean":"2024-04-26T01:49:20.824181+0000","last_became_active":"2024-04-26T01:47:27.138413+0000","last_became_peered":"2024-04-26T01:47:27.138413+0000","last_unstale":"2024-04-26T01:49:20.824181+0000","last_undegraded":"2024-04-26T01:49:20.824181+0000","last_fullsized":"2024-04-26T01:49:20.824181+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:51:13.117307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.833198+0000","last_change":"2024-04-26T01:47:27.141338+0000","last_active":"2024-04-26T01:49:20.833198+0000","last_peered":"2024-04-26T01:49:20.833198+0000","last_clean":"2024-04-26T01:49:20.833198+0000","last_became_active":"2024-04-26T01:47:27.140825+0000","last_became_peered":"2024-04-26T01:47:27.140825+0000","last_unstale":"2024-04-26T01:49:20.833198+0000","last_undegraded":"2024-04-26T01:49:20.833198+0000","last_fullsized":"2024-04-26T01:49:20.833198+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:12:08.680876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.795200+0000","last_change":"2024-04-26T01:47:27.114919+0000","last_active":"2024-04-26T01:49:20.795200+0000","last_peered":"2024-04-26T01:49:20.795200+0000","last_clean":"2024-04-26T01:49:20.795200+0000","last_became_active":"2024-04-26T01:47:27.114528+0000","last_became_peered":"2024-04-26T01:47:27.114528+0000","last_unstale":"2024-04-26T01:49:20.795200+0000","last_undegraded":"2024-04-26T01:49:20.795200+0000","last_fullsized":"2024-04-26T01:49:20.795200+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:15:25.620566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64738,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":810,"num_read_kb":475,"num_write":170,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318319,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":490347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579956,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28684,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716468,"statfs":{"total":95995035648,"available":95965663232,"internally_reserved":0,"allocated":1708032,"data_stored":1142956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743482,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":490583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005122,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":490503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299461,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1082574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462987,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":489972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1077248,"data_stored":550361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738477,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":491243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":551599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182492,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":490283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920853,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":491053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724626,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32772,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712380,"statfs":{"total":95995035648,"available":95961477120,"internally_reserved":0,"allocated":1634304,"data_stored":1081726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:50:04.980 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:50:04.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:04 smithi097 ceph-mon[32853]: pgmap v403: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-26T01:50:05.481 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:05.799 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:05 smithi001 ceph-mon[27822]: from='client.24865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:50:05.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:05 smithi173 ceph-mon[32553]: from='client.24865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:50:05.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:05 smithi097 ceph-mon[32853]: from='client.24865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:50:06.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:06 smithi173 ceph-mon[32553]: pgmap v404: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T01:50:06.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:06 smithi001 ceph-mon[27822]: pgmap v404: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T01:50:06.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:06 smithi097 ceph-mon[32853]: pgmap v404: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-26T01:50:07.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.779+0000 7f3418a00700 1 -- 172.21.15.1:0/1084887264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34141044c0 msgr2=0x7f341410c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:07.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.779+0000 7f3418a00700 1 --2- 172.21.15.1:0/1084887264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34141044c0 0x7f341410c9b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3400009990 tx=0x7f3400009ca0 comp rx=0 tx=0).stop 2024-04-26T01:50:07.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.779+0000 7f3418a00700 1 -- 172.21.15.1:0/1084887264 shutdown_connections 2024-04-26T01:50:07.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.779+0000 7f3418a00700 1 --2- 172.21.15.1:0/1084887264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f341410cfd0 0x7f341410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:07.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.779+0000 7f3418a00700 1 --2- 172.21.15.1:0/1084887264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3414103ba0 0x7f3414103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:07.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.779+0000 7f3418a00700 1 --2- 172.21.15.1:0/1084887264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34141044c0 0x7f341410c9b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:07.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.779+0000 7f3418a00700 1 -- 172.21.15.1:0/1084887264 >> 172.21.15.1:0/1084887264 conn(0x7f34140fd3f0 msgr2=0x7f34140ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:07.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.779+0000 7f3418a00700 1 -- 172.21.15.1:0/1084887264 shutdown_connections 2024-04-26T01:50:07.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.779+0000 7f3418a00700 1 -- 172.21.15.1:0/1084887264 wait complete. 2024-04-26T01:50:07.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.781+0000 7f3418a00700 1 Processor -- start 2024-04-26T01:50:07.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.781+0000 7f3418a00700 1 -- start start 2024-04-26T01:50:07.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.781+0000 7f3418a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3414103ba0 0x7f341419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:07.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.781+0000 7f3418a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34141044c0 0x7f341419b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:07.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.781+0000 7f3418a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f341410cfd0 0x7f341419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:07.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.782+0000 7f3418a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3414112bb0 con 0x7f341410cfd0 2024-04-26T01:50:07.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.782+0000 7f3418a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3414112a30 con 0x7f34141044c0 2024-04-26T01:50:07.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.782+0000 7f3418a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3414112d30 con 0x7f3414103ba0 2024-04-26T01:50:07.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.782+0000 7f3413400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3414103ba0 0x7f341419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:07.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.782+0000 7f3412a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34141044c0 0x7f341419b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:07.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.782+0000 7f3413400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3414103ba0 0x7f341419b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:57146/0 (socket says 172.21.15.1:57146) 2024-04-26T01:50:07.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.782+0000 7f3412a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34141044c0 0x7f341419b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:34570/0 (socket says 172.21.15.1:34570) 2024-04-26T01:50:07.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.782+0000 7f3413400700 1 -- 172.21.15.1:0/4041524415 learned_addr learned my addr 172.21.15.1:0/4041524415 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:07.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.782+0000 7f3413e00700 1 --2- 172.21.15.1:0/4041524415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f341410cfd0 0x7f341419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:07.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.782+0000 7f3413400700 1 -- 172.21.15.1:0/4041524415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34141044c0 msgr2=0x7f341419b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:07.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.782+0000 7f3413400700 1 --2- 172.21.15.1:0/4041524415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34141044c0 0x7f341419b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:07.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.782+0000 7f3413400700 1 -- 172.21.15.1:0/4041524415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f341410cfd0 msgr2=0x7f341419fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.782+0000 7f3413400700 1 --2- 172.21.15.1:0/4041524415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f341410cfd0 0x7f341419fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.782+0000 7f3413400700 1 -- 172.21.15.1:0/4041524415 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34141a0310 con 0x7f3414103ba0 2024-04-26T01:50:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.782+0000 7f3413e00700 1 --2- 172.21.15.1:0/4041524415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f341410cfd0 0x7f341419fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.782+0000 7f3412a00700 1 --2- 172.21.15.1:0/4041524415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34141044c0 0x7f341419b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.783+0000 7f3413400700 1 --2- 172.21.15.1:0/4041524415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3414103ba0 0x7f341419b300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f340400d980 tx=0x7f340400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.783+0000 7f340be00700 1 -- 172.21.15.1:0/4041524415 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3404016070 con 0x7f3414103ba0 2024-04-26T01:50:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.783+0000 7f340be00700 1 -- 172.21.15.1:0/4041524415 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3404004830 con 0x7f3414103ba0 2024-04-26T01:50:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.783+0000 7f3418a00700 1 -- 172.21.15.1:0/4041524415 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34141a0600 con 0x7f3414103ba0 2024-04-26T01:50:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.783+0000 7f340be00700 1 -- 172.21.15.1:0/4041524415 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3404005540 con 0x7f3414103ba0 2024-04-26T01:50:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.783+0000 7f3418a00700 1 -- 172.21.15.1:0/4041524415 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34141a7e80 con 0x7f3414103ba0 2024-04-26T01:50:07.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.785+0000 7f3418a00700 1 -- 172.21.15.1:0/4041524415 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3414069730 con 0x7f3414103ba0 2024-04-26T01:50:07.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.787+0000 7f340be00700 1 -- 172.21.15.1:0/4041524415 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f3404004220 con 0x7f3414103ba0 2024-04-26T01:50:07.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.788+0000 7f340be00700 1 --2- 172.21.15.1:0/4041524415 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f33f80a3c40 0x7f33f80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:07.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.788+0000 7f340be00700 1 -- 172.21.15.1:0/4041524415 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f34040d0220 con 0x7f3414103ba0 2024-04-26T01:50:07.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.788+0000 7f3412a00700 1 --2- 172.21.15.1:0/4041524415 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f33f80a3c40 0x7f33f80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:07.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.789+0000 7f3412a00700 1 --2- 172.21.15.1:0/4041524415 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f33f80a3c40 0x7f33f80a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f340000b5c0 tx=0x7f3400005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:07.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:07.793+0000 7f340be00700 1 -- 172.21.15.1:0/4041524415 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f340401f020 con 0x7f3414103ba0 2024-04-26T01:50:08.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:08.017+0000 7f3418a00700 1 -- 172.21.15.1:0/4041524415 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f341419c660 con 0x7f33f80a3c40 2024-04-26T01:50:08.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:08.040+0000 7f340be00700 1 -- 172.21.15.1:0/4041524415 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258838 (secure 0 0 0) 0x7f341419c660 con 0x7f33f80a3c40 2024-04-26T01:50:08.042 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:50:08.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:08.045+0000 7f3418a00700 1 -- 172.21.15.1:0/4041524415 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f33f80a3c40 msgr2=0x7f33f80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:08.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:08.045+0000 7f3418a00700 1 --2- 172.21.15.1:0/4041524415 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f33f80a3c40 0x7f33f80a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f340000b5c0 tx=0x7f3400005960 comp rx=0 tx=0).stop 2024-04-26T01:50:08.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:08.045+0000 7f3418a00700 1 -- 172.21.15.1:0/4041524415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3414103ba0 msgr2=0x7f341419b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:08.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:08.045+0000 7f3418a00700 1 --2- 172.21.15.1:0/4041524415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3414103ba0 0x7f341419b300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f340400d980 tx=0x7f340400dc90 comp rx=0 tx=0).stop 2024-04-26T01:50:08.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:08.046+0000 7f3418a00700 1 -- 172.21.15.1:0/4041524415 shutdown_connections 2024-04-26T01:50:08.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:08.046+0000 7f3418a00700 1 --2- 172.21.15.1:0/4041524415 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f33f80a3c40 0x7f33f80a6100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:08.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:08.046+0000 7f3418a00700 1 --2- 172.21.15.1:0/4041524415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f341410cfd0 0x7f341419fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:08.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:08.046+0000 7f3418a00700 1 --2- 172.21.15.1:0/4041524415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3414103ba0 0x7f341419b300 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:08.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:08.046+0000 7f3418a00700 1 --2- 172.21.15.1:0/4041524415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34141044c0 0x7f341419b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:08.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:08.046+0000 7f3418a00700 1 -- 172.21.15.1:0/4041524415 >> 172.21.15.1:0/4041524415 conn(0x7f34140fd3f0 msgr2=0x7f34140fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:08.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:08.046+0000 7f3418a00700 1 -- 172.21.15.1:0/4041524415 shutdown_connections 2024-04-26T01:50:08.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:08.046+0000 7f3418a00700 1 -- 172.21.15.1:0/4041524415 wait complete. 2024-04-26T01:50:08.046 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-04-26T01:50:08.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:08 smithi173 ceph-mon[32553]: pgmap v405: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:08.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:08 smithi001 ceph-mon[27822]: pgmap v405: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:08.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:08 smithi097 ceph-mon[32853]: pgmap v405: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:08.998 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-04-26T01:50:06.862509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655340,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":976,"num_read_kb":609,"num_write":353,"num_write_kb":3055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":784,"ondisk_log_size":784,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358972,"kb_used_data":13980,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582852,"statfs":{"total":1151940427776,"available":1151572840448,"internally_reserved":0,"allocated":14315520,"data_stored":7843200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":44,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005877"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.815061+0000","last_change":"2024-04-26T01:47:27.140418+0000","last_active":"2024-04-26T01:49:20.815061+0000","last_peered":"2024-04-26T01:49:20.815061+0000","last_clean":"2024-04-26T01:49:20.815061+0000","last_became_active":"2024-04-26T01:47:27.140017+0000","last_became_peered":"2024-04-26T01:47:27.140017+0000","last_unstale":"2024-04-26T01:49:20.815061+0000","last_undegraded":"2024-04-26T01:49:20.815061+0000","last_fullsized":"2024-04-26T01:49:20.815061+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:46:08.621222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.831303+0000","last_change":"2024-04-26T01:47:27.138746+0000","last_active":"2024-04-26T01:49:20.831303+0000","last_peered":"2024-04-26T01:49:20.831303+0000","last_clean":"2024-04-26T01:49:20.831303+0000","last_became_active":"2024-04-26T01:47:27.136163+0000","last_became_peered":"2024-04-26T01:47:27.136163+0000","last_unstale":"2024-04-26T01:49:20.831303+0000","last_undegraded":"2024-04-26T01:49:20.831303+0000","last_fullsized":"2024-04-26T01:49:20.831303+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:01:36.195332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.833057+0000","last_change":"2024-04-26T01:47:27.127017+0000","last_active":"2024-04-26T01:49:20.833057+0000","last_peered":"2024-04-26T01:49:20.833057+0000","last_clean":"2024-04-26T01:49:20.833057+0000","last_became_active":"2024-04-26T01:47:27.126610+0000","last_became_peered":"2024-04-26T01:47:27.126610+0000","last_unstale":"2024-04-26T01:49:20.833057+0000","last_undegraded":"2024-04-26T01:49:20.833057+0000","last_fullsized":"2024-04-26T01:49:20.833057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:05:12.784486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.791490+0000","last_change":"2024-04-26T01:47:27.117640+0000","last_active":"2024-04-26T01:49:20.791490+0000","last_peered":"2024-04-26T01:49:20.791490+0000","last_clean":"2024-04-26T01:49:20.791490+0000","last_became_active":"2024-04-26T01:47:27.117266+0000","last_became_peered":"2024-04-26T01:47:27.117266+0000","last_unstale":"2024-04-26T01:49:20.791490+0000","last_undegraded":"2024-04-26T01:49:20.791490+0000","last_fullsized":"2024-04-26T01:49:20.791490+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:26:57.824560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.797486+0000","last_change":"2024-04-26T01:47:27.128185+0000","last_active":"2024-04-26T01:49:20.797486+0000","last_peered":"2024-04-26T01:49:20.797486+0000","last_clean":"2024-04-26T01:49:20.797486+0000","last_became_active":"2024-04-26T01:47:27.127794+0000","last_became_peered":"2024-04-26T01:47:27.127794+0000","last_unstale":"2024-04-26T01:49:20.797486+0000","last_undegraded":"2024-04-26T01:49:20.797486+0000","last_fullsized":"2024-04-26T01:49:20.797486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:09:52.774007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.803115+0000","last_change":"2024-04-26T01:48:22.748601+0000","last_active":"2024-04-26T01:49:20.803115+0000","last_peered":"2024-04-26T01:49:20.803115+0000","last_clean":"2024-04-26T01:49:20.803115+0000","last_became_active":"2024-04-26T01:48:22.748229+0000","last_became_peered":"2024-04-26T01:48:22.748229+0000","last_unstale":"2024-04-26T01:49:20.803115+0000","last_undegraded":"2024-04-26T01:49:20.803115+0000","last_fullsized":"2024-04-26T01:49:20.803115+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:46:30.433063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.814447+0000","last_change":"2024-04-26T01:47:27.123786+0000","last_active":"2024-04-26T01:49:20.814447+0000","last_peered":"2024-04-26T01:49:20.814447+0000","last_clean":"2024-04-26T01:49:20.814447+0000","last_became_active":"2024-04-26T01:47:27.123473+0000","last_became_peered":"2024-04-26T01:47:27.123473+0000","last_unstale":"2024-04-26T01:49:20.814447+0000","last_undegraded":"2024-04-26T01:49:20.814447+0000","last_fullsized":"2024-04-26T01:49:20.814447+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:48:55.031147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.825042+0000","last_change":"2024-04-26T01:47:27.118756+0000","last_active":"2024-04-26T01:49:20.825042+0000","last_peered":"2024-04-26T01:49:20.825042+0000","last_clean":"2024-04-26T01:49:20.825042+0000","last_became_active":"2024-04-26T01:47:27.118406+0000","last_became_peered":"2024-04-26T01:47:27.118406+0000","last_unstale":"2024-04-26T01:49:20.825042+0000","last_undegraded":"2024-04-26T01:49:20.825042+0000","last_fullsized":"2024-04-26T01:49:20.825042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:05.793682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.814403+0000","last_change":"2024-04-26T01:47:27.117622+0000","last_active":"2024-04-26T01:49:20.814403+0000","last_peered":"2024-04-26T01:49:20.814403+0000","last_clean":"2024-04-26T01:49:20.814403+0000","last_became_active":"2024-04-26T01:47:27.115206+0000","last_became_peered":"2024-04-26T01:47:27.115206+0000","last_unstale":"2024-04-26T01:49:20.814403+0000","last_undegraded":"2024-04-26T01:49:20.814403+0000","last_fullsized":"2024-04-26T01:49:20.814403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:13:17.866630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.805457+0000","last_change":"2024-04-26T01:47:27.119524+0000","last_active":"2024-04-26T01:49:20.805457+0000","last_peered":"2024-04-26T01:49:20.805457+0000","last_clean":"2024-04-26T01:49:20.805457+0000","last_became_active":"2024-04-26T01:47:27.119008+0000","last_became_peered":"2024-04-26T01:47:27.119008+0000","last_unstale":"2024-04-26T01:49:20.805457+0000","last_undegraded":"2024-04-26T01:49:20.805457+0000","last_fullsized":"2024-04-26T01:49:20.805457+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:37:31.894486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.814077+0000","last_change":"2024-04-26T01:47:27.122361+0000","last_active":"2024-04-26T01:49:20.814077+0000","last_peered":"2024-04-26T01:49:20.814077+0000","last_clean":"2024-04-26T01:49:20.814077+0000","last_became_active":"2024-04-26T01:47:27.121748+0000","last_became_peered":"2024-04-26T01:47:27.121748+0000","last_unstale":"2024-04-26T01:49:20.814077+0000","last_undegraded":"2024-04-26T01:49:20.814077+0000","last_fullsized":"2024-04-26T01:49:20.814077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:18.680909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.803636+0000","last_change":"2024-04-26T01:47:27.116833+0000","last_active":"2024-04-26T01:49:20.803636+0000","last_peered":"2024-04-26T01:49:20.803636+0000","last_clean":"2024-04-26T01:49:20.803636+0000","last_became_active":"2024-04-26T01:47:27.116508+0000","last_became_peered":"2024-04-26T01:47:27.116508+0000","last_unstale":"2024-04-26T01:49:20.803636+0000","last_undegraded":"2024-04-26T01:49:20.803636+0000","last_fullsized":"2024-04-26T01:49:20.803636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:27:45.864521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.825409+0000","last_change":"2024-04-26T01:47:27.125342+0000","last_active":"2024-04-26T01:49:20.825409+0000","last_peered":"2024-04-26T01:49:20.825409+0000","last_clean":"2024-04-26T01:49:20.825409+0000","last_became_active":"2024-04-26T01:47:27.124862+0000","last_became_peered":"2024-04-26T01:47:27.124862+0000","last_unstale":"2024-04-26T01:49:20.825409+0000","last_undegraded":"2024-04-26T01:49:20.825409+0000","last_fullsized":"2024-04-26T01:49:20.825409+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:48.654711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.831441+0000","last_change":"2024-04-26T01:47:27.127365+0000","last_active":"2024-04-26T01:49:20.831441+0000","last_peered":"2024-04-26T01:49:20.831441+0000","last_clean":"2024-04-26T01:49:20.831441+0000","last_became_active":"2024-04-26T01:47:27.126868+0000","last_became_peered":"2024-04-26T01:47:27.126868+0000","last_unstale":"2024-04-26T01:49:20.831441+0000","last_undegraded":"2024-04-26T01:49:20.831441+0000","last_fullsized":"2024-04-26T01:49:20.831441+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:07:46.400333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.831822+0000","last_change":"2024-04-26T01:47:27.135584+0000","last_active":"2024-04-26T01:49:20.831822+0000","last_peered":"2024-04-26T01:49:20.831822+0000","last_clean":"2024-04-26T01:49:20.831822+0000","last_became_active":"2024-04-26T01:47:27.134849+0000","last_became_peered":"2024-04-26T01:47:27.134849+0000","last_unstale":"2024-04-26T01:49:20.831822+0000","last_undegraded":"2024-04-26T01:49:20.831822+0000","last_fullsized":"2024-04-26T01:49:20.831822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:57:48.821595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.834029+0000","last_change":"2024-04-26T01:47:27.136182+0000","last_active":"2024-04-26T01:49:20.834029+0000","last_peered":"2024-04-26T01:49:20.834029+0000","last_clean":"2024-04-26T01:49:20.834029+0000","last_became_active":"2024-04-26T01:47:27.135787+0000","last_became_peered":"2024-04-26T01:47:27.135787+0000","last_unstale":"2024-04-26T01:49:20.834029+0000","last_undegraded":"2024-04-26T01:49:20.834029+0000","last_fullsized":"2024-04-26T01:49:20.834029+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:38:14.275898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.815346+0000","last_change":"2024-04-26T01:47:27.140700+0000","last_active":"2024-04-26T01:49:20.815346+0000","last_peered":"2024-04-26T01:49:20.815346+0000","last_clean":"2024-04-26T01:49:20.815346+0000","last_became_active":"2024-04-26T01:47:27.140025+0000","last_became_peered":"2024-04-26T01:47:27.140025+0000","last_unstale":"2024-04-26T01:49:20.815346+0000","last_undegraded":"2024-04-26T01:49:20.815346+0000","last_fullsized":"2024-04-26T01:49:20.815346+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:50.465236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.803099+0000","last_change":"2024-04-26T01:47:27.117256+0000","last_active":"2024-04-26T01:49:20.803099+0000","last_peered":"2024-04-26T01:49:20.803099+0000","last_clean":"2024-04-26T01:49:20.803099+0000","last_became_active":"2024-04-26T01:47:27.116825+0000","last_became_peered":"2024-04-26T01:47:27.116825+0000","last_unstale":"2024-04-26T01:49:20.803099+0000","last_undegraded":"2024-04-26T01:49:20.803099+0000","last_fullsized":"2024-04-26T01:49:20.803099+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:23.313712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.820168+0000","last_change":"2024-04-26T01:47:27.130233+0000","last_active":"2024-04-26T01:49:20.820168+0000","last_peered":"2024-04-26T01:49:20.820168+0000","last_clean":"2024-04-26T01:49:20.820168+0000","last_became_active":"2024-04-26T01:47:27.129852+0000","last_became_peered":"2024-04-26T01:47:27.129852+0000","last_unstale":"2024-04-26T01:49:20.820168+0000","last_undegraded":"2024-04-26T01:49:20.820168+0000","last_fullsized":"2024-04-26T01:49:20.820168+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:49:01.407264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.749156+0000","last_change":"2024-04-26T01:47:23.068477+0000","last_active":"2024-04-26T01:48:22.749156+0000","last_peered":"2024-04-26T01:48:22.749156+0000","last_clean":"2024-04-26T01:48:22.749156+0000","last_became_active":"2024-04-26T01:47:23.068061+0000","last_became_peered":"2024-04-26T01:47:23.068061+0000","last_unstale":"2024-04-26T01:48:22.749156+0000","last_undegraded":"2024-04-26T01:48:22.749156+0000","last_fullsized":"2024-04-26T01:48:22.749156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:02.153724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.811871+0000","last_change":"2024-04-26T01:47:27.130385+0000","last_active":"2024-04-26T01:49:20.811871+0000","last_peered":"2024-04-26T01:49:20.811871+0000","last_clean":"2024-04-26T01:49:20.811871+0000","last_became_active":"2024-04-26T01:47:27.129912+0000","last_became_peered":"2024-04-26T01:47:27.129912+0000","last_unstale":"2024-04-26T01:49:20.811871+0000","last_undegraded":"2024-04-26T01:49:20.811871+0000","last_fullsized":"2024-04-26T01:49:20.811871+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:26:03.010544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.408077+0000","last_change":"2024-04-26T01:47:23.110806+0000","last_active":"2024-04-26T01:48:22.408077+0000","last_peered":"2024-04-26T01:48:22.408077+0000","last_clean":"2024-04-26T01:48:22.408077+0000","last_became_active":"2024-04-26T01:47:23.110441+0000","last_became_peered":"2024-04-26T01:47:23.110441+0000","last_unstale":"2024-04-26T01:48:22.408077+0000","last_undegraded":"2024-04-26T01:48:22.408077+0000","last_fullsized":"2024-04-26T01:48:22.408077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:39.239172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.832835+0000","last_change":"2024-04-26T01:47:27.138628+0000","last_active":"2024-04-26T01:49:20.832835+0000","last_peered":"2024-04-26T01:49:20.832835+0000","last_clean":"2024-04-26T01:49:20.832835+0000","last_became_active":"2024-04-26T01:47:27.130483+0000","last_became_peered":"2024-04-26T01:47:27.130483+0000","last_unstale":"2024-04-26T01:49:20.832835+0000","last_undegraded":"2024-04-26T01:49:20.832835+0000","last_fullsized":"2024-04-26T01:49:20.832835+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:40:41.242856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.397187+0000","last_change":"2024-04-26T01:47:23.060066+0000","last_active":"2024-04-26T01:48:22.397187+0000","last_peered":"2024-04-26T01:48:22.397187+0000","last_clean":"2024-04-26T01:48:22.397187+0000","last_became_active":"2024-04-26T01:47:23.059526+0000","last_became_peered":"2024-04-26T01:47:23.059526+0000","last_unstale":"2024-04-26T01:48:22.397187+0000","last_undegraded":"2024-04-26T01:48:22.397187+0000","last_fullsized":"2024-04-26T01:48:22.397187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:28:22.623695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.827183+0000","last_change":"2024-04-26T01:47:27.128343+0000","last_active":"2024-04-26T01:49:20.827183+0000","last_peered":"2024-04-26T01:49:20.827183+0000","last_clean":"2024-04-26T01:49:20.827183+0000","last_became_active":"2024-04-26T01:47:27.127920+0000","last_became_peered":"2024-04-26T01:47:27.127920+0000","last_unstale":"2024-04-26T01:49:20.827183+0000","last_undegraded":"2024-04-26T01:49:20.827183+0000","last_fullsized":"2024-04-26T01:49:20.827183+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:06.725033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.396988+0000","last_change":"2024-04-26T01:47:23.057783+0000","last_active":"2024-04-26T01:48:22.396988+0000","last_peered":"2024-04-26T01:48:22.396988+0000","last_clean":"2024-04-26T01:48:22.396988+0000","last_became_active":"2024-04-26T01:47:23.057391+0000","last_became_peered":"2024-04-26T01:47:23.057391+0000","last_unstale":"2024-04-26T01:48:22.396988+0000","last_undegraded":"2024-04-26T01:48:22.396988+0000","last_fullsized":"2024-04-26T01:48:22.396988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:10:10.050787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.807130+0000","last_change":"2024-04-26T01:47:27.116458+0000","last_active":"2024-04-26T01:49:20.807130+0000","last_peered":"2024-04-26T01:49:20.807130+0000","last_clean":"2024-04-26T01:49:20.807130+0000","last_became_active":"2024-04-26T01:47:27.116084+0000","last_became_peered":"2024-04-26T01:47:27.116084+0000","last_unstale":"2024-04-26T01:49:20.807130+0000","last_undegraded":"2024-04-26T01:49:20.807130+0000","last_fullsized":"2024-04-26T01:49:20.807130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:06:34.517478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":168,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:02.456026+0000","last_change":"2024-04-26T01:47:23.060657+0000","last_active":"2024-04-26T01:50:02.456026+0000","last_peered":"2024-04-26T01:50:02.456026+0000","last_clean":"2024-04-26T01:50:02.456026+0000","last_became_active":"2024-04-26T01:47:23.060294+0000","last_became_peered":"2024-04-26T01:47:23.060294+0000","last_unstale":"2024-04-26T01:50:02.456026+0000","last_undegraded":"2024-04-26T01:50:02.456026+0000","last_fullsized":"2024-04-26T01:50:02.456026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:33.979717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.794722+0000","last_change":"2024-04-26T01:47:27.120587+0000","last_active":"2024-04-26T01:49:20.794722+0000","last_peered":"2024-04-26T01:49:20.794722+0000","last_clean":"2024-04-26T01:49:20.794722+0000","last_became_active":"2024-04-26T01:47:27.120236+0000","last_became_peered":"2024-04-26T01:47:27.120236+0000","last_unstale":"2024-04-26T01:49:20.794722+0000","last_undegraded":"2024-04-26T01:49:20.794722+0000","last_fullsized":"2024-04-26T01:49:20.794722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:30:38.488741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405133+0000","last_change":"2024-04-26T01:47:23.069395+0000","last_active":"2024-04-26T01:48:22.405133+0000","last_peered":"2024-04-26T01:48:22.405133+0000","last_clean":"2024-04-26T01:48:22.405133+0000","last_became_active":"2024-04-26T01:47:23.069102+0000","last_became_peered":"2024-04-26T01:47:23.069102+0000","last_unstale":"2024-04-26T01:48:22.405133+0000","last_undegraded":"2024-04-26T01:48:22.405133+0000","last_fullsized":"2024-04-26T01:48:22.405133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:45.162604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.805279+0000","last_change":"2024-04-26T01:47:27.123859+0000","last_active":"2024-04-26T01:49:20.805279+0000","last_peered":"2024-04-26T01:49:20.805279+0000","last_clean":"2024-04-26T01:49:20.805279+0000","last_became_active":"2024-04-26T01:47:27.123438+0000","last_became_peered":"2024-04-26T01:47:27.123438+0000","last_unstale":"2024-04-26T01:49:20.805279+0000","last_undegraded":"2024-04-26T01:49:20.805279+0000","last_fullsized":"2024-04-26T01:49:20.805279+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:33:53.542809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.401196+0000","last_change":"2024-04-26T01:47:23.071544+0000","last_active":"2024-04-26T01:48:22.401196+0000","last_peered":"2024-04-26T01:48:22.401196+0000","last_clean":"2024-04-26T01:48:22.401196+0000","last_became_active":"2024-04-26T01:47:23.071191+0000","last_became_peered":"2024-04-26T01:47:23.071191+0000","last_unstale":"2024-04-26T01:48:22.401196+0000","last_undegraded":"2024-04-26T01:48:22.401196+0000","last_fullsized":"2024-04-26T01:48:22.401196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:58.889561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.821717+0000","last_change":"2024-04-26T01:47:27.131887+0000","last_active":"2024-04-26T01:49:20.821717+0000","last_peered":"2024-04-26T01:49:20.821717+0000","last_clean":"2024-04-26T01:49:20.821717+0000","last_became_active":"2024-04-26T01:47:27.127973+0000","last_became_peered":"2024-04-26T01:47:27.127973+0000","last_unstale":"2024-04-26T01:49:20.821717+0000","last_undegraded":"2024-04-26T01:49:20.821717+0000","last_fullsized":"2024-04-26T01:49:20.821717+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:54:35.305779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745664+0000","last_change":"2024-04-26T01:47:23.061353+0000","last_active":"2024-04-26T01:48:22.745664+0000","last_peered":"2024-04-26T01:48:22.745664+0000","last_clean":"2024-04-26T01:48:22.745664+0000","last_became_active":"2024-04-26T01:47:23.061037+0000","last_became_peered":"2024-04-26T01:47:23.061037+0000","last_unstale":"2024-04-26T01:48:22.745664+0000","last_undegraded":"2024-04-26T01:48:22.745664+0000","last_fullsized":"2024-04-26T01:48:22.745664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:13:59.617969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.820690+0000","last_change":"2024-04-26T01:47:27.122303+0000","last_active":"2024-04-26T01:49:20.820690+0000","last_peered":"2024-04-26T01:49:20.820690+0000","last_clean":"2024-04-26T01:49:20.820690+0000","last_became_active":"2024-04-26T01:47:27.121919+0000","last_became_peered":"2024-04-26T01:47:27.121919+0000","last_unstale":"2024-04-26T01:49:20.820690+0000","last_undegraded":"2024-04-26T01:49:20.820690+0000","last_fullsized":"2024-04-26T01:49:20.820690+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:21.676663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":166,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:04.458610+0000","last_change":"2024-04-26T01:47:23.057961+0000","last_active":"2024-04-26T01:50:04.458610+0000","last_peered":"2024-04-26T01:50:04.458610+0000","last_clean":"2024-04-26T01:50:04.458610+0000","last_became_active":"2024-04-26T01:47:23.057529+0000","last_became_peered":"2024-04-26T01:47:23.057529+0000","last_unstale":"2024-04-26T01:50:04.458610+0000","last_undegraded":"2024-04-26T01:50:04.458610+0000","last_fullsized":"2024-04-26T01:50:04.458610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:14.420265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":136,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.813684+0000","last_change":"2024-04-26T01:47:27.120083+0000","last_active":"2024-04-26T01:49:20.813684+0000","last_peered":"2024-04-26T01:49:20.813684+0000","last_clean":"2024-04-26T01:49:20.813684+0000","last_became_active":"2024-04-26T01:47:27.118410+0000","last_became_peered":"2024-04-26T01:47:27.118410+0000","last_unstale":"2024-04-26T01:49:20.813684+0000","last_undegraded":"2024-04-26T01:49:20.813684+0000","last_fullsized":"2024-04-26T01:49:20.813684+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:28.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400284+0000","last_change":"2024-04-26T01:47:23.059221+0000","last_active":"2024-04-26T01:48:22.400284+0000","last_peered":"2024-04-26T01:48:22.400284+0000","last_clean":"2024-04-26T01:48:22.400284+0000","last_became_active":"2024-04-26T01:47:23.058881+0000","last_became_peered":"2024-04-26T01:47:23.058881+0000","last_unstale":"2024-04-26T01:48:22.400284+0000","last_undegraded":"2024-04-26T01:48:22.400284+0000","last_fullsized":"2024-04-26T01:48:22.400284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:30.433696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.829291+0000","last_change":"2024-04-26T01:47:27.136763+0000","last_active":"2024-04-26T01:49:20.829291+0000","last_peered":"2024-04-26T01:49:20.829291+0000","last_clean":"2024-04-26T01:49:20.829291+0000","last_became_active":"2024-04-26T01:47:27.133642+0000","last_became_peered":"2024-04-26T01:47:27.133642+0000","last_unstale":"2024-04-26T01:49:20.829291+0000","last_undegraded":"2024-04-26T01:49:20.829291+0000","last_fullsized":"2024-04-26T01:49:20.829291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:05.901139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746141+0000","last_change":"2024-04-26T01:47:23.108883+0000","last_active":"2024-04-26T01:48:22.746141+0000","last_peered":"2024-04-26T01:48:22.746141+0000","last_clean":"2024-04-26T01:48:22.746141+0000","last_became_active":"2024-04-26T01:47:23.108499+0000","last_became_peered":"2024-04-26T01:47:23.108499+0000","last_unstale":"2024-04-26T01:48:22.746141+0000","last_undegraded":"2024-04-26T01:48:22.746141+0000","last_fullsized":"2024-04-26T01:48:22.746141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:14:17.555416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.827266+0000","last_change":"2024-04-26T01:48:22.410651+0000","last_active":"2024-04-26T01:49:20.827266+0000","last_peered":"2024-04-26T01:49:20.827266+0000","last_clean":"2024-04-26T01:49:20.827266+0000","last_became_active":"2024-04-26T01:48:22.410359+0000","last_became_peered":"2024-04-26T01:48:22.410359+0000","last_unstale":"2024-04-26T01:49:20.827266+0000","last_undegraded":"2024-04-26T01:49:20.827266+0000","last_fullsized":"2024-04-26T01:49:20.827266+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:53:20.686915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'32","reported_seq":341,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:05.468445+0000","last_change":"2024-04-26T01:47:23.054893+0000","last_active":"2024-04-26T01:50:05.468445+0000","last_peered":"2024-04-26T01:50:05.468445+0000","last_clean":"2024-04-26T01:50:05.468445+0000","last_became_active":"2024-04-26T01:47:23.054558+0000","last_became_peered":"2024-04-26T01:47:23.054558+0000","last_unstale":"2024-04-26T01:50:05.468445+0000","last_undegraded":"2024-04-26T01:50:05.468445+0000","last_fullsized":"2024-04-26T01:50:05.468445+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:59.356841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60596,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":282,"num_read_kb":200,"num_write":33,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.815218+0000","last_change":"2024-04-26T01:47:27.116934+0000","last_active":"2024-04-26T01:49:20.815218+0000","last_peered":"2024-04-26T01:49:20.815218+0000","last_clean":"2024-04-26T01:49:20.815218+0000","last_became_active":"2024-04-26T01:47:27.116527+0000","last_became_peered":"2024-04-26T01:47:27.116527+0000","last_unstale":"2024-04-26T01:49:20.815218+0000","last_undegraded":"2024-04-26T01:49:20.815218+0000","last_fullsized":"2024-04-26T01:49:20.815218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:09:45.820533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399124+0000","last_change":"2024-04-26T01:47:23.059032+0000","last_active":"2024-04-26T01:48:22.399124+0000","last_peered":"2024-04-26T01:48:22.399124+0000","last_clean":"2024-04-26T01:48:22.399124+0000","last_became_active":"2024-04-26T01:47:23.057992+0000","last_became_peered":"2024-04-26T01:47:23.057992+0000","last_unstale":"2024-04-26T01:48:22.399124+0000","last_undegraded":"2024-04-26T01:48:22.399124+0000","last_fullsized":"2024-04-26T01:48:22.399124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:46:53.164925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.810758+0000","last_change":"2024-04-26T01:47:27.134574+0000","last_active":"2024-04-26T01:49:20.810758+0000","last_peered":"2024-04-26T01:49:20.810758+0000","last_clean":"2024-04-26T01:49:20.810758+0000","last_became_active":"2024-04-26T01:47:27.134184+0000","last_became_peered":"2024-04-26T01:47:27.134184+0000","last_unstale":"2024-04-26T01:49:20.810758+0000","last_undegraded":"2024-04-26T01:49:20.810758+0000","last_fullsized":"2024-04-26T01:49:20.810758+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:42:26.428014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.745707+0000","last_change":"2024-04-26T01:47:23.062649+0000","last_active":"2024-04-26T01:48:22.745707+0000","last_peered":"2024-04-26T01:48:22.745707+0000","last_clean":"2024-04-26T01:48:22.745707+0000","last_became_active":"2024-04-26T01:47:23.062282+0000","last_became_peered":"2024-04-26T01:47:23.062282+0000","last_unstale":"2024-04-26T01:48:22.745707+0000","last_undegraded":"2024-04-26T01:48:22.745707+0000","last_fullsized":"2024-04-26T01:48:22.745707+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:38.699105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.826254+0000","last_change":"2024-04-26T01:47:27.124720+0000","last_active":"2024-04-26T01:49:20.826254+0000","last_peered":"2024-04-26T01:49:20.826254+0000","last_clean":"2024-04-26T01:49:20.826254+0000","last_became_active":"2024-04-26T01:47:27.124378+0000","last_became_peered":"2024-04-26T01:47:27.124378+0000","last_unstale":"2024-04-26T01:49:20.826254+0000","last_undegraded":"2024-04-26T01:49:20.826254+0000","last_fullsized":"2024-04-26T01:49:20.826254+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:45.462741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":170,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:02.523624+0000","last_change":"2024-04-26T01:47:23.056305+0000","last_active":"2024-04-26T01:50:02.523624+0000","last_peered":"2024-04-26T01:50:02.523624+0000","last_clean":"2024-04-26T01:50:02.523624+0000","last_became_active":"2024-04-26T01:47:23.055905+0000","last_became_peered":"2024-04-26T01:47:23.055905+0000","last_unstale":"2024-04-26T01:50:02.523624+0000","last_undegraded":"2024-04-26T01:50:02.523624+0000","last_fullsized":"2024-04-26T01:50:02.523624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:00:14.609673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":139,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.814879+0000","last_change":"2024-04-26T01:47:27.119880+0000","last_active":"2024-04-26T01:49:20.814879+0000","last_peered":"2024-04-26T01:49:20.814879+0000","last_clean":"2024-04-26T01:49:20.814879+0000","last_became_active":"2024-04-26T01:47:27.119483+0000","last_became_peered":"2024-04-26T01:47:27.119483+0000","last_unstale":"2024-04-26T01:49:20.814879+0000","last_undegraded":"2024-04-26T01:49:20.814879+0000","last_fullsized":"2024-04-26T01:49:20.814879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:15.523150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":67,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.759238+0000","last_change":"2024-04-26T01:48:22.758774+0000","last_active":"2024-04-26T01:48:22.759238+0000","last_peered":"2024-04-26T01:48:22.759238+0000","last_clean":"2024-04-26T01:48:22.759238+0000","last_became_active":"2024-04-26T01:48:22.747354+0000","last_became_peered":"2024-04-26T01:48:22.747354+0000","last_unstale":"2024-04-26T01:48:22.759238+0000","last_undegraded":"2024-04-26T01:48:22.759238+0000","last_fullsized":"2024-04-26T01:48:22.759238+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:23:15.886774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.806186+0000","last_change":"2024-04-26T01:47:27.113508+0000","last_active":"2024-04-26T01:49:20.806186+0000","last_peered":"2024-04-26T01:49:20.806186+0000","last_clean":"2024-04-26T01:49:20.806186+0000","last_became_active":"2024-04-26T01:47:27.113066+0000","last_became_peered":"2024-04-26T01:47:27.113066+0000","last_unstale":"2024-04-26T01:49:20.806186+0000","last_undegraded":"2024-04-26T01:49:20.806186+0000","last_fullsized":"2024-04-26T01:49:20.806186+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:37.525486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.397112+0000","last_change":"2024-04-26T01:47:23.059644+0000","last_active":"2024-04-26T01:48:22.397112+0000","last_peered":"2024-04-26T01:48:22.397112+0000","last_clean":"2024-04-26T01:48:22.397112+0000","last_became_active":"2024-04-26T01:47:23.058966+0000","last_became_peered":"2024-04-26T01:47:23.058966+0000","last_unstale":"2024-04-26T01:48:22.397112+0000","last_undegraded":"2024-04-26T01:48:22.397112+0000","last_fullsized":"2024-04-26T01:48:22.397112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:57:36.475935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.796390+0000","last_change":"2024-04-26T01:48:22.749354+0000","last_active":"2024-04-26T01:49:20.796390+0000","last_peered":"2024-04-26T01:49:20.796390+0000","last_clean":"2024-04-26T01:49:20.796390+0000","last_became_active":"2024-04-26T01:48:22.748978+0000","last_became_peered":"2024-04-26T01:48:22.748978+0000","last_unstale":"2024-04-26T01:49:20.796390+0000","last_undegraded":"2024-04-26T01:49:20.796390+0000","last_fullsized":"2024-04-26T01:49:20.796390+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:51:16.087091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746722+0000","last_change":"2024-04-26T01:47:23.110360+0000","last_active":"2024-04-26T01:48:22.746722+0000","last_peered":"2024-04-26T01:48:22.746722+0000","last_clean":"2024-04-26T01:48:22.746722+0000","last_became_active":"2024-04-26T01:47:23.109744+0000","last_became_peered":"2024-04-26T01:47:23.109744+0000","last_unstale":"2024-04-26T01:48:22.746722+0000","last_undegraded":"2024-04-26T01:48:22.746722+0000","last_fullsized":"2024-04-26T01:48:22.746722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:40:13.658396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.825600+0000","last_change":"2024-04-26T01:47:27.125399+0000","last_active":"2024-04-26T01:49:20.825600+0000","last_peered":"2024-04-26T01:49:20.825600+0000","last_clean":"2024-04-26T01:49:20.825600+0000","last_became_active":"2024-04-26T01:47:27.124871+0000","last_became_peered":"2024-04-26T01:47:27.124871+0000","last_unstale":"2024-04-26T01:49:20.825600+0000","last_undegraded":"2024-04-26T01:49:20.825600+0000","last_fullsized":"2024-04-26T01:49:20.825600+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:28:48.285206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400231+0000","last_change":"2024-04-26T01:47:23.067862+0000","last_active":"2024-04-26T01:48:22.400231+0000","last_peered":"2024-04-26T01:48:22.400231+0000","last_clean":"2024-04-26T01:48:22.400231+0000","last_became_active":"2024-04-26T01:47:23.067526+0000","last_became_peered":"2024-04-26T01:47:23.067526+0000","last_unstale":"2024-04-26T01:48:22.400231+0000","last_undegraded":"2024-04-26T01:48:22.400231+0000","last_fullsized":"2024-04-26T01:48:22.400231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:22.668405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.804364+0000","last_change":"2024-04-26T01:48:22.748548+0000","last_active":"2024-04-26T01:49:20.804364+0000","last_peered":"2024-04-26T01:49:20.804364+0000","last_clean":"2024-04-26T01:49:20.804364+0000","last_became_active":"2024-04-26T01:48:22.748182+0000","last_became_peered":"2024-04-26T01:48:22.748182+0000","last_unstale":"2024-04-26T01:49:20.804364+0000","last_undegraded":"2024-04-26T01:49:20.804364+0000","last_fullsized":"2024-04-26T01:49:20.804364+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:20:27.596827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.798153+0000","last_change":"2024-04-26T01:47:27.118307+0000","last_active":"2024-04-26T01:49:20.798153+0000","last_peered":"2024-04-26T01:49:20.798153+0000","last_clean":"2024-04-26T01:49:20.798153+0000","last_became_active":"2024-04-26T01:47:27.117986+0000","last_became_peered":"2024-04-26T01:47:27.117986+0000","last_unstale":"2024-04-26T01:49:20.798153+0000","last_undegraded":"2024-04-26T01:49:20.798153+0000","last_fullsized":"2024-04-26T01:49:20.798153+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:40.667143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":168,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:50:04.528027+0000","last_change":"2024-04-26T01:47:23.109973+0000","last_active":"2024-04-26T01:50:04.528027+0000","last_peered":"2024-04-26T01:50:04.528027+0000","last_clean":"2024-04-26T01:50:04.528027+0000","last_became_active":"2024-04-26T01:47:23.109446+0000","last_became_peered":"2024-04-26T01:47:23.109446+0000","last_unstale":"2024-04-26T01:50:04.528027+0000","last_undegraded":"2024-04-26T01:50:04.528027+0000","last_fullsized":"2024-04-26T01:50:04.528027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:49:00.684164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":136,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.821119+0000","last_change":"2024-04-26T01:47:27.131502+0000","last_active":"2024-04-26T01:49:20.821119+0000","last_peered":"2024-04-26T01:49:20.821119+0000","last_clean":"2024-04-26T01:49:20.821119+0000","last_became_active":"2024-04-26T01:47:27.127697+0000","last_became_peered":"2024-04-26T01:47:27.127697+0000","last_unstale":"2024-04-26T01:49:20.821119+0000","last_undegraded":"2024-04-26T01:49:20.821119+0000","last_fullsized":"2024-04-26T01:49:20.821119+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:39:15.458189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746759+0000","last_change":"2024-04-26T01:47:23.066596+0000","last_active":"2024-04-26T01:48:22.746759+0000","last_peered":"2024-04-26T01:48:22.746759+0000","last_clean":"2024-04-26T01:48:22.746759+0000","last_became_active":"2024-04-26T01:47:23.066216+0000","last_became_peered":"2024-04-26T01:47:23.066216+0000","last_unstale":"2024-04-26T01:48:22.746759+0000","last_undegraded":"2024-04-26T01:48:22.746759+0000","last_fullsized":"2024-04-26T01:48:22.746759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:08:31.544831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400162+0000","last_change":"2024-04-26T01:45:54.834501+0000","last_active":"2024-04-26T01:48:22.400162+0000","last_peered":"2024-04-26T01:48:22.400162+0000","last_clean":"2024-04-26T01:48:22.400162+0000","last_became_active":"2024-04-26T01:45:54.833868+0000","last_became_peered":"2024-04-26T01:45:54.833868+0000","last_unstale":"2024-04-26T01:48:22.400162+0000","last_undegraded":"2024-04-26T01:48:22.400162+0000","last_fullsized":"2024-04-26T01:48:22.400162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:42:05.228956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:42:05.228956+0000","last_clean_scrub_stamp":"2024-04-26T01:42:05.228956+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:06:58.383315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.797852+0000","last_change":"2024-04-26T01:47:27.126475+0000","last_active":"2024-04-26T01:49:20.797852+0000","last_peered":"2024-04-26T01:49:20.797852+0000","last_clean":"2024-04-26T01:49:20.797852+0000","last_became_active":"2024-04-26T01:47:27.123760+0000","last_became_peered":"2024-04-26T01:47:27.123760+0000","last_unstale":"2024-04-26T01:49:20.797852+0000","last_undegraded":"2024-04-26T01:49:20.797852+0000","last_fullsized":"2024-04-26T01:49:20.797852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:38:27.473909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.400412+0000","last_change":"2024-04-26T01:47:23.059810+0000","last_active":"2024-04-26T01:48:22.400412+0000","last_peered":"2024-04-26T01:48:22.400412+0000","last_clean":"2024-04-26T01:48:22.400412+0000","last_became_active":"2024-04-26T01:47:23.059504+0000","last_became_peered":"2024-04-26T01:47:23.059504+0000","last_unstale":"2024-04-26T01:48:22.400412+0000","last_undegraded":"2024-04-26T01:48:22.400412+0000","last_fullsized":"2024-04-26T01:48:22.400412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:50:51.220951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.397081+0000","last_change":"2024-04-26T01:47:23.109460+0000","last_active":"2024-04-26T01:48:22.397081+0000","last_peered":"2024-04-26T01:48:22.397081+0000","last_clean":"2024-04-26T01:48:22.397081+0000","last_became_active":"2024-04-26T01:47:23.109090+0000","last_became_peered":"2024-04-26T01:47:23.109090+0000","last_unstale":"2024-04-26T01:48:22.397081+0000","last_undegraded":"2024-04-26T01:48:22.397081+0000","last_fullsized":"2024-04-26T01:48:22.397081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:56.821312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.802492+0000","last_change":"2024-04-26T01:47:27.129537+0000","last_active":"2024-04-26T01:49:20.802492+0000","last_peered":"2024-04-26T01:49:20.802492+0000","last_clean":"2024-04-26T01:49:20.802492+0000","last_became_active":"2024-04-26T01:47:27.128124+0000","last_became_peered":"2024-04-26T01:47:27.128124+0000","last_unstale":"2024-04-26T01:49:20.802492+0000","last_undegraded":"2024-04-26T01:49:20.802492+0000","last_fullsized":"2024-04-26T01:49:20.802492+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:14:26.262197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:21.141469+0000","last_change":"2024-04-26T01:47:23.064511+0000","last_active":"2024-04-26T01:49:21.141469+0000","last_peered":"2024-04-26T01:49:21.141469+0000","last_clean":"2024-04-26T01:49:21.141469+0000","last_became_active":"2024-04-26T01:47:23.064080+0000","last_became_peered":"2024-04-26T01:47:23.064080+0000","last_unstale":"2024-04-26T01:49:21.141469+0000","last_undegraded":"2024-04-26T01:49:21.141469+0000","last_fullsized":"2024-04-26T01:49:21.141469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:37.562644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.814995+0000","last_change":"2024-04-26T01:47:27.124057+0000","last_active":"2024-04-26T01:49:20.814995+0000","last_peered":"2024-04-26T01:49:20.814995+0000","last_clean":"2024-04-26T01:49:20.814995+0000","last_became_active":"2024-04-26T01:47:27.123636+0000","last_became_peered":"2024-04-26T01:47:27.123636+0000","last_unstale":"2024-04-26T01:49:20.814995+0000","last_undegraded":"2024-04-26T01:49:20.814995+0000","last_fullsized":"2024-04-26T01:49:20.814995+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:56:55.691697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746211+0000","last_change":"2024-04-26T01:47:23.136571+0000","last_active":"2024-04-26T01:48:22.746211+0000","last_peered":"2024-04-26T01:48:22.746211+0000","last_clean":"2024-04-26T01:48:22.746211+0000","last_became_active":"2024-04-26T01:47:23.136205+0000","last_became_peered":"2024-04-26T01:47:23.136205+0000","last_unstale":"2024-04-26T01:48:22.746211+0000","last_undegraded":"2024-04-26T01:48:22.746211+0000","last_fullsized":"2024-04-26T01:48:22.746211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:24.418797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.828229+0000","last_change":"2024-04-26T01:48:22.747570+0000","last_active":"2024-04-26T01:49:20.828229+0000","last_peered":"2024-04-26T01:49:20.828229+0000","last_clean":"2024-04-26T01:49:20.828229+0000","last_became_active":"2024-04-26T01:48:22.747147+0000","last_became_peered":"2024-04-26T01:48:22.747147+0000","last_unstale":"2024-04-26T01:49:20.828229+0000","last_undegraded":"2024-04-26T01:49:20.828229+0000","last_fullsized":"2024-04-26T01:49:20.828229+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:37.034388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.414985+0000","last_change":"2024-04-26T01:48:22.414610+0000","last_active":"2024-04-26T01:48:22.414985+0000","last_peered":"2024-04-26T01:48:22.414985+0000","last_clean":"2024-04-26T01:48:22.414985+0000","last_became_active":"2024-04-26T01:48:22.410275+0000","last_became_peered":"2024-04-26T01:48:22.410275+0000","last_unstale":"2024-04-26T01:48:22.414985+0000","last_undegraded":"2024-04-26T01:48:22.414985+0000","last_fullsized":"2024-04-26T01:48:22.414985+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:20:48.131988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.783601+0000","last_change":"2024-04-26T01:47:27.115498+0000","last_active":"2024-04-26T01:49:20.783601+0000","last_peered":"2024-04-26T01:49:20.783601+0000","last_clean":"2024-04-26T01:49:20.783601+0000","last_became_active":"2024-04-26T01:47:27.115134+0000","last_became_peered":"2024-04-26T01:47:27.115134+0000","last_unstale":"2024-04-26T01:49:20.783601+0000","last_undegraded":"2024-04-26T01:49:20.783601+0000","last_fullsized":"2024-04-26T01:49:20.783601+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:20.795224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405406+0000","last_change":"2024-04-26T01:47:23.065366+0000","last_active":"2024-04-26T01:48:22.405406+0000","last_peered":"2024-04-26T01:48:22.405406+0000","last_clean":"2024-04-26T01:48:22.405406+0000","last_became_active":"2024-04-26T01:47:23.063216+0000","last_became_peered":"2024-04-26T01:47:23.063216+0000","last_unstale":"2024-04-26T01:48:22.405406+0000","last_undegraded":"2024-04-26T01:48:22.405406+0000","last_fullsized":"2024-04-26T01:48:22.405406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:06:05.261907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.798214+0000","last_change":"2024-04-26T01:47:27.127938+0000","last_active":"2024-04-26T01:49:20.798214+0000","last_peered":"2024-04-26T01:49:20.798214+0000","last_clean":"2024-04-26T01:49:20.798214+0000","last_became_active":"2024-04-26T01:47:27.127553+0000","last_became_peered":"2024-04-26T01:47:27.127553+0000","last_unstale":"2024-04-26T01:49:20.798214+0000","last_undegraded":"2024-04-26T01:49:20.798214+0000","last_fullsized":"2024-04-26T01:49:20.798214+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:17:51.115349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.398786+0000","last_change":"2024-04-26T01:47:23.060475+0000","last_active":"2024-04-26T01:48:22.398786+0000","last_peered":"2024-04-26T01:48:22.398786+0000","last_clean":"2024-04-26T01:48:22.398786+0000","last_became_active":"2024-04-26T01:47:23.060098+0000","last_became_peered":"2024-04-26T01:47:23.060098+0000","last_unstale":"2024-04-26T01:48:22.398786+0000","last_undegraded":"2024-04-26T01:48:22.398786+0000","last_fullsized":"2024-04-26T01:48:22.398786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:49.009455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.833701+0000","last_change":"2024-04-26T01:47:27.137491+0000","last_active":"2024-04-26T01:49:20.833701+0000","last_peered":"2024-04-26T01:49:20.833701+0000","last_clean":"2024-04-26T01:49:20.833701+0000","last_became_active":"2024-04-26T01:47:27.137059+0000","last_became_peered":"2024-04-26T01:47:27.137059+0000","last_unstale":"2024-04-26T01:49:20.833701+0000","last_undegraded":"2024-04-26T01:49:20.833701+0000","last_fullsized":"2024-04-26T01:49:20.833701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:20:25.004074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.405524+0000","last_change":"2024-04-26T01:48:22.405071+0000","last_active":"2024-04-26T01:48:22.405524+0000","last_peered":"2024-04-26T01:48:22.405524+0000","last_clean":"2024-04-26T01:48:22.405524+0000","last_became_active":"2024-04-26T01:48:22.401282+0000","last_became_peered":"2024-04-26T01:48:22.401282+0000","last_unstale":"2024-04-26T01:48:22.405524+0000","last_undegraded":"2024-04-26T01:48:22.405524+0000","last_fullsized":"2024-04-26T01:48:22.405524+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:24:53.628014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.821279+0000","last_change":"2024-04-26T01:47:27.128253+0000","last_active":"2024-04-26T01:49:20.821279+0000","last_peered":"2024-04-26T01:49:20.821279+0000","last_clean":"2024-04-26T01:49:20.821279+0000","last_became_active":"2024-04-26T01:47:27.127869+0000","last_became_peered":"2024-04-26T01:47:27.127869+0000","last_unstale":"2024-04-26T01:49:20.821279+0000","last_undegraded":"2024-04-26T01:49:20.821279+0000","last_fullsized":"2024-04-26T01:49:20.821279+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:11:23.476938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.399460+0000","last_change":"2024-04-26T01:47:23.136862+0000","last_active":"2024-04-26T01:48:22.399460+0000","last_peered":"2024-04-26T01:48:22.399460+0000","last_clean":"2024-04-26T01:48:22.399460+0000","last_became_active":"2024-04-26T01:47:23.136380+0000","last_became_peered":"2024-04-26T01:47:23.136380+0000","last_unstale":"2024-04-26T01:48:22.399460+0000","last_undegraded":"2024-04-26T01:48:22.399460+0000","last_fullsized":"2024-04-26T01:48:22.399460+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:39:18.103559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.815096+0000","last_change":"2024-04-26T01:47:27.113363+0000","last_active":"2024-04-26T01:49:20.815096+0000","last_peered":"2024-04-26T01:49:20.815096+0000","last_clean":"2024-04-26T01:49:20.815096+0000","last_became_active":"2024-04-26T01:47:27.113040+0000","last_became_peered":"2024-04-26T01:47:27.113040+0000","last_unstale":"2024-04-26T01:49:20.815096+0000","last_undegraded":"2024-04-26T01:49:20.815096+0000","last_fullsized":"2024-04-26T01:49:20.815096+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:42:16.428697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.746035+0000","last_change":"2024-04-26T01:47:23.067257+0000","last_active":"2024-04-26T01:48:22.746035+0000","last_peered":"2024-04-26T01:48:22.746035+0000","last_clean":"2024-04-26T01:48:22.746035+0000","last_became_active":"2024-04-26T01:47:23.066858+0000","last_became_peered":"2024-04-26T01:47:23.066858+0000","last_unstale":"2024-04-26T01:48:22.746035+0000","last_undegraded":"2024-04-26T01:48:22.746035+0000","last_fullsized":"2024-04-26T01:48:22.746035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:22:07.848074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.820466+0000","last_change":"2024-04-26T01:47:27.125885+0000","last_active":"2024-04-26T01:49:20.820466+0000","last_peered":"2024-04-26T01:49:20.820466+0000","last_clean":"2024-04-26T01:49:20.820466+0000","last_became_active":"2024-04-26T01:47:27.125524+0000","last_became_peered":"2024-04-26T01:47:27.125524+0000","last_unstale":"2024-04-26T01:49:20.820466+0000","last_undegraded":"2024-04-26T01:49:20.820466+0000","last_fullsized":"2024-04-26T01:49:20.820466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:39.282023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:48:22.407777+0000","last_change":"2024-04-26T01:47:23.071051+0000","last_active":"2024-04-26T01:48:22.407777+0000","last_peered":"2024-04-26T01:48:22.407777+0000","last_clean":"2024-04-26T01:48:22.407777+0000","last_became_active":"2024-04-26T01:47:23.069524+0000","last_became_peered":"2024-04-26T01:47:23.069524+0000","last_unstale":"2024-04-26T01:48:22.407777+0000","last_undegraded":"2024-04-26T01:48:22.407777+0000","last_fullsized":"2024-04-26T01:48:22.407777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:24:57.643100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.815496+0000","last_change":"2024-04-26T01:47:27.114325+0000","last_active":"2024-04-26T01:49:20.815496+0000","last_peered":"2024-04-26T01:49:20.815496+0000","last_clean":"2024-04-26T01:49:20.815496+0000","last_became_active":"2024-04-26T01:47:27.113952+0000","last_became_peered":"2024-04-26T01:47:27.113952+0000","last_unstale":"2024-04-26T01:49:20.815496+0000","last_undegraded":"2024-04-26T01:49:20.815496+0000","last_fullsized":"2024-04-26T01:49:20.815496+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:39:19.167175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.797195+0000","last_change":"2024-04-26T01:47:27.118709+0000","last_active":"2024-04-26T01:49:20.797195+0000","last_peered":"2024-04-26T01:49:20.797195+0000","last_clean":"2024-04-26T01:49:20.797195+0000","last_became_active":"2024-04-26T01:47:27.118341+0000","last_became_peered":"2024-04-26T01:47:27.118341+0000","last_unstale":"2024-04-26T01:49:20.797195+0000","last_undegraded":"2024-04-26T01:49:20.797195+0000","last_fullsized":"2024-04-26T01:49:20.797195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:13.736180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.795672+0000","last_change":"2024-04-26T01:47:27.119239+0000","last_active":"2024-04-26T01:49:20.795672+0000","last_peered":"2024-04-26T01:49:20.795672+0000","last_clean":"2024-04-26T01:49:20.795672+0000","last_became_active":"2024-04-26T01:47:27.118877+0000","last_became_peered":"2024-04-26T01:47:27.118877+0000","last_unstale":"2024-04-26T01:49:20.795672+0000","last_undegraded":"2024-04-26T01:49:20.795672+0000","last_fullsized":"2024-04-26T01:49:20.795672+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:04:10.033517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.819917+0000","last_change":"2024-04-26T01:47:27.142848+0000","last_active":"2024-04-26T01:49:20.819917+0000","last_peered":"2024-04-26T01:49:20.819917+0000","last_clean":"2024-04-26T01:49:20.819917+0000","last_became_active":"2024-04-26T01:47:27.142580+0000","last_became_peered":"2024-04-26T01:47:27.142580+0000","last_unstale":"2024-04-26T01:49:20.819917+0000","last_undegraded":"2024-04-26T01:49:20.819917+0000","last_fullsized":"2024-04-26T01:49:20.819917+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:15:46.923922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.831630+0000","last_change":"2024-04-26T01:47:27.134817+0000","last_active":"2024-04-26T01:49:20.831630+0000","last_peered":"2024-04-26T01:49:20.831630+0000","last_clean":"2024-04-26T01:49:20.831630+0000","last_became_active":"2024-04-26T01:47:27.134024+0000","last_became_peered":"2024-04-26T01:47:27.134024+0000","last_unstale":"2024-04-26T01:49:20.831630+0000","last_undegraded":"2024-04-26T01:49:20.831630+0000","last_fullsized":"2024-04-26T01:49:20.831630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:56.746427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.828596+0000","last_change":"2024-04-26T01:47:27.143137+0000","last_active":"2024-04-26T01:49:20.828596+0000","last_peered":"2024-04-26T01:49:20.828596+0000","last_clean":"2024-04-26T01:49:20.828596+0000","last_became_active":"2024-04-26T01:47:27.142625+0000","last_became_peered":"2024-04-26T01:47:27.142625+0000","last_unstale":"2024-04-26T01:49:20.828596+0000","last_undegraded":"2024-04-26T01:49:20.828596+0000","last_fullsized":"2024-04-26T01:49:20.828596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:42:50.468382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.798582+0000","last_change":"2024-04-26T01:48:22.749582+0000","last_active":"2024-04-26T01:49:20.798582+0000","last_peered":"2024-04-26T01:49:20.798582+0000","last_clean":"2024-04-26T01:49:20.798582+0000","last_became_active":"2024-04-26T01:48:22.749191+0000","last_became_peered":"2024-04-26T01:48:22.749191+0000","last_unstale":"2024-04-26T01:49:20.798582+0000","last_undegraded":"2024-04-26T01:49:20.798582+0000","last_fullsized":"2024-04-26T01:49:20.798582+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:14.614842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.813096+0000","last_change":"2024-04-26T01:47:27.123964+0000","last_active":"2024-04-26T01:49:20.813096+0000","last_peered":"2024-04-26T01:49:20.813096+0000","last_clean":"2024-04-26T01:49:20.813096+0000","last_became_active":"2024-04-26T01:47:27.123552+0000","last_became_peered":"2024-04-26T01:47:27.123552+0000","last_unstale":"2024-04-26T01:49:20.813096+0000","last_undegraded":"2024-04-26T01:49:20.813096+0000","last_fullsized":"2024-04-26T01:49:20.813096+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:33:12.566105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.811676+0000","last_change":"2024-04-26T01:48:22.412621+0000","last_active":"2024-04-26T01:49:20.811676+0000","last_peered":"2024-04-26T01:49:20.811676+0000","last_clean":"2024-04-26T01:49:20.811676+0000","last_became_active":"2024-04-26T01:48:22.411430+0000","last_became_peered":"2024-04-26T01:48:22.411430+0000","last_unstale":"2024-04-26T01:49:20.811676+0000","last_undegraded":"2024-04-26T01:49:20.811676+0000","last_fullsized":"2024-04-26T01:49:20.811676+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:34.152338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.807847+0000","last_change":"2024-04-26T01:47:27.131437+0000","last_active":"2024-04-26T01:49:20.807847+0000","last_peered":"2024-04-26T01:49:20.807847+0000","last_clean":"2024-04-26T01:49:20.807847+0000","last_became_active":"2024-04-26T01:47:27.131138+0000","last_became_peered":"2024-04-26T01:47:27.131138+0000","last_unstale":"2024-04-26T01:49:20.807847+0000","last_undegraded":"2024-04-26T01:49:20.807847+0000","last_fullsized":"2024-04-26T01:49:20.807847+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:55:39.132972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.820594+0000","last_change":"2024-04-26T01:47:27.120928+0000","last_active":"2024-04-26T01:49:20.820594+0000","last_peered":"2024-04-26T01:49:20.820594+0000","last_clean":"2024-04-26T01:49:20.820594+0000","last_became_active":"2024-04-26T01:47:27.120012+0000","last_became_peered":"2024-04-26T01:47:27.120012+0000","last_unstale":"2024-04-26T01:49:20.820594+0000","last_undegraded":"2024-04-26T01:49:20.820594+0000","last_fullsized":"2024-04-26T01:49:20.820594+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:19:44.209876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.824181+0000","last_change":"2024-04-26T01:47:27.138766+0000","last_active":"2024-04-26T01:49:20.824181+0000","last_peered":"2024-04-26T01:49:20.824181+0000","last_clean":"2024-04-26T01:49:20.824181+0000","last_became_active":"2024-04-26T01:47:27.138413+0000","last_became_peered":"2024-04-26T01:47:27.138413+0000","last_unstale":"2024-04-26T01:49:20.824181+0000","last_undegraded":"2024-04-26T01:49:20.824181+0000","last_fullsized":"2024-04-26T01:49:20.824181+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:51:13.117307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.833198+0000","last_change":"2024-04-26T01:47:27.141338+0000","last_active":"2024-04-26T01:49:20.833198+0000","last_peered":"2024-04-26T01:49:20.833198+0000","last_clean":"2024-04-26T01:49:20.833198+0000","last_became_active":"2024-04-26T01:47:27.140825+0000","last_became_peered":"2024-04-26T01:47:27.140825+0000","last_unstale":"2024-04-26T01:49:20.833198+0000","last_undegraded":"2024-04-26T01:49:20.833198+0000","last_fullsized":"2024-04-26T01:49:20.833198+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:12:08.680876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:49:20.795200+0000","last_change":"2024-04-26T01:47:27.114919+0000","last_active":"2024-04-26T01:49:20.795200+0000","last_peered":"2024-04-26T01:49:20.795200+0000","last_clean":"2024-04-26T01:49:20.795200+0000","last_became_active":"2024-04-26T01:47:27.114528+0000","last_became_peered":"2024-04-26T01:47:27.114528+0000","last_unstale":"2024-04-26T01:49:20.795200+0000","last_undegraded":"2024-04-26T01:49:20.795200+0000","last_fullsized":"2024-04-26T01:49:20.795200+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:15:25.620566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64738,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":840,"num_read_kb":490,"num_write":170,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318320,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":490347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579957,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28684,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716468,"statfs":{"total":95995035648,"available":95965663232,"internally_reserved":0,"allocated":1708032,"data_stored":1142956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743483,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":490583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005122,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":490503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299462,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1082574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462988,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":489972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1077248,"data_stored":550361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738478,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":491243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":551599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182492,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":490283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":491053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724627,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32772,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712380,"statfs":{"total":95995035648,"available":95961477120,"internally_reserved":0,"allocated":1634304,"data_stored":1081726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:50:09.002 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T01:50:09.002 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T01:50:09.002 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-26T01:50:09.003 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-26T01:50:09.506 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:09.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:09 smithi173 ceph-mon[32553]: from='client.24871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:50:09.859 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:09 smithi001 ceph-mon[27822]: from='client.24871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:50:09.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:09 smithi097 ceph-mon[32853]: from='client.24871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:50:10.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:10 smithi173 ceph-mon[32553]: pgmap v406: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:10.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:10 smithi001 ceph-mon[27822]: pgmap v406: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:10.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:10 smithi097 ceph-mon[32853]: pgmap v406: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:11.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.938+0000 7f1960000700 1 -- 172.21.15.1:0/2533660503 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19581063f0 msgr2=0x7f195810dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:11.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.938+0000 7f1960000700 1 --2- 172.21.15.1:0/2533660503 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19581063f0 0x7f195810dfc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f195000c1b0 tx=0x7f195000c4c0 comp rx=0 tx=0).stop 2024-04-26T01:50:11.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.938+0000 7f1960000700 1 -- 172.21.15.1:0/2533660503 shutdown_connections 2024-04-26T01:50:11.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.938+0000 7f1960000700 1 --2- 172.21.15.1:0/2533660503 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19581063f0 0x7f195810dfc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:11.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.938+0000 7f1960000700 1 --2- 172.21.15.1:0/2533660503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1958101360 0x7f1958101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:11.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.938+0000 7f1960000700 1 --2- 172.21.15.1:0/2533660503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1958101d10 0x7f1958105d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:11.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.938+0000 7f1960000700 1 -- 172.21.15.1:0/2533660503 >> 172.21.15.1:0/2533660503 conn(0x7f1958077fb0 msgr2=0x7f19580ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:11.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.938+0000 7f1960000700 1 -- 172.21.15.1:0/2533660503 shutdown_connections 2024-04-26T01:50:11.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.938+0000 7f1960000700 1 -- 172.21.15.1:0/2533660503 wait complete. 2024-04-26T01:50:11.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.939+0000 7f1960000700 1 Processor -- start 2024-04-26T01:50:11.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.939+0000 7f1960000700 1 -- start start 2024-04-26T01:50:11.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.939+0000 7f1960000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1958101360 0x7f195819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:11.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.939+0000 7f1960000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1958101d10 0x7f195819b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:11.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.939+0000 7f1960000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19581063f0 0x7f195819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:11.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.939+0000 7f1960000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1958110a50 con 0x7f19581063f0 2024-04-26T01:50:11.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.939+0000 7f1960000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f19581108d0 con 0x7f1958101d10 2024-04-26T01:50:11.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.939+0000 7f1960000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1958110bd0 con 0x7f1958101360 2024-04-26T01:50:11.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.940+0000 7f195e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19581063f0 0x7f195819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:11.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.940+0000 7f195e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19581063f0 0x7f195819fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35364/0 (socket says 172.21.15.1:35364) 2024-04-26T01:50:11.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.940+0000 7f195e000700 1 -- 172.21.15.1:0/934861883 learned_addr learned my addr 172.21.15.1:0/934861883 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:11.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.940+0000 7f195d600700 1 --2- 172.21.15.1:0/934861883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1958101360 0x7f195819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:11.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.940+0000 7f195cc00700 1 --2- 172.21.15.1:0/934861883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1958101d10 0x7f195819b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:11.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.940+0000 7f195e000700 1 -- 172.21.15.1:0/934861883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1958101360 msgr2=0x7f195819b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:11.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.940+0000 7f195e000700 1 --2- 172.21.15.1:0/934861883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1958101360 0x7f195819b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:11.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.940+0000 7f195e000700 1 -- 172.21.15.1:0/934861883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1958101d10 msgr2=0x7f195819b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:11.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.940+0000 7f195e000700 1 --2- 172.21.15.1:0/934861883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1958101d10 0x7f195819b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:11.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.940+0000 7f195e000700 1 -- 172.21.15.1:0/934861883 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19581a03a0 con 0x7f19581063f0 2024-04-26T01:50:11.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.941+0000 7f195e000700 1 --2- 172.21.15.1:0/934861883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19581063f0 0x7f195819fc60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f195000c180 tx=0x7f1950004c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:11.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.941+0000 7f1956000700 1 -- 172.21.15.1:0/934861883 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950022020 con 0x7f19581063f0 2024-04-26T01:50:11.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.941+0000 7f1960000700 1 -- 172.21.15.1:0/934861883 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19581a0630 con 0x7f19581063f0 2024-04-26T01:50:11.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.941+0000 7f1956000700 1 -- 172.21.15.1:0/934861883 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1950008bd0 con 0x7f19581063f0 2024-04-26T01:50:11.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.941+0000 7f1960000700 1 -- 172.21.15.1:0/934861883 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19581a7f10 con 0x7f19581063f0 2024-04-26T01:50:11.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.941+0000 7f1956000700 1 -- 172.21.15.1:0/934861883 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195001ee30 con 0x7f19581063f0 2024-04-26T01:50:11.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.942+0000 7f1960000700 1 -- 172.21.15.1:0/934861883 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19581027f0 con 0x7f19581063f0 2024-04-26T01:50:11.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.944+0000 7f1956000700 1 -- 172.21.15.1:0/934861883 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f1950025020 con 0x7f19581063f0 2024-04-26T01:50:11.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.946+0000 7f1956000700 1 --2- 172.21.15.1:0/934861883 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f19400a3c40 0x7f19400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:11.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.946+0000 7f1956000700 1 -- 172.21.15.1:0/934861883 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f19500d1610 con 0x7f19581063f0 2024-04-26T01:50:11.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.946+0000 7f195d600700 1 --2- 172.21.15.1:0/934861883 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f19400a3c40 0x7f19400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:11.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.947+0000 7f195d600700 1 --2- 172.21.15.1:0/934861883 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f19400a3c40 0x7f19400a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f194c009680 tx=0x7f194c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:11.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:11.951+0000 7f1956000700 1 -- 172.21.15.1:0/934861883 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f195009e000 con 0x7f19581063f0 2024-04-26T01:50:12.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:12.231+0000 7f1960000700 1 -- 172.21.15.1:0/934861883 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f195802df80 con 0x7f19581063f0 2024-04-26T01:50:12.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:12.231+0000 7f1956000700 1 -- 172.21.15.1:0/934861883 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f195009d900 con 0x7f19581063f0 2024-04-26T01:50:12.229 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:50:12.230 INFO:teuthology.orchestra.run.smithi001.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T01:50:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:12.234+0000 7f1960000700 1 -- 172.21.15.1:0/934861883 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f19400a3c40 msgr2=0x7f19400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:12.234+0000 7f1960000700 1 --2- 172.21.15.1:0/934861883 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f19400a3c40 0x7f19400a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f194c009680 tx=0x7f194c009260 comp rx=0 tx=0).stop 2024-04-26T01:50:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:12.234+0000 7f1960000700 1 -- 172.21.15.1:0/934861883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19581063f0 msgr2=0x7f195819fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:12.234+0000 7f1960000700 1 --2- 172.21.15.1:0/934861883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19581063f0 0x7f195819fc60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f195000c180 tx=0x7f1950004c40 comp rx=0 tx=0).stop 2024-04-26T01:50:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:12.234+0000 7f1960000700 1 -- 172.21.15.1:0/934861883 shutdown_connections 2024-04-26T01:50:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:12.234+0000 7f1960000700 1 --2- 172.21.15.1:0/934861883 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f19400a3c40 0x7f19400a6100 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:12.235+0000 7f1960000700 1 --2- 172.21.15.1:0/934861883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19581063f0 0x7f195819fc60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:12.235+0000 7f1960000700 1 --2- 172.21.15.1:0/934861883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1958101360 0x7f195819b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:12.235+0000 7f1960000700 1 --2- 172.21.15.1:0/934861883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1958101d10 0x7f195819b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:12.235+0000 7f1960000700 1 -- 172.21.15.1:0/934861883 >> 172.21.15.1:0/934861883 conn(0x7f1958077fb0 msgr2=0x7f1958104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:12.235+0000 7f1960000700 1 -- 172.21.15.1:0/934861883 shutdown_connections 2024-04-26T01:50:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:12.235+0000 7f1960000700 1 -- 172.21.15.1:0/934861883 wait complete. 2024-04-26T01:50:12.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:12 smithi173 ceph-mon[32553]: pgmap v407: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:12.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:12 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/934861883' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T01:50:12.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:12 smithi001 ceph-mon[27822]: pgmap v407: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:12.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:12 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/934861883' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T01:50:12.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:12 smithi097 ceph-mon[32853]: pgmap v407: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:12.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:12 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/934861883' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T01:50:13.125 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-26T01:50:13.125 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:50:13.604 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:14.798 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:14 smithi001 ceph-mon[27822]: pgmap v408: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:14.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:14 smithi173 ceph-mon[32553]: pgmap v408: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:14.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:14 smithi097 ceph-mon[32853]: pgmap v408: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:15.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.760+0000 7f567c000700 1 -- 172.21.15.1:0/3476986384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5674074c70 msgr2=0x7f5674075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:15.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.760+0000 7f567c000700 1 --2- 172.21.15.1:0/3476986384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5674074c70 0x7f5674075050 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5668009990 tx=0x7f5668009ca0 comp rx=0 tx=0).stop 2024-04-26T01:50:15.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.760+0000 7f567c000700 1 -- 172.21.15.1:0/3476986384 shutdown_connections 2024-04-26T01:50:15.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.760+0000 7f567c000700 1 --2- 172.21.15.1:0/3476986384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f567410f440 0x7f5674111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:15.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.760+0000 7f567c000700 1 --2- 172.21.15.1:0/3476986384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5674074c70 0x7f5674075050 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:15.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.760+0000 7f567c000700 1 --2- 172.21.15.1:0/3476986384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5674075590 0x7f567410ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:15.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.760+0000 7f567c000700 1 -- 172.21.15.1:0/3476986384 >> 172.21.15.1:0/3476986384 conn(0x7f5674079ba0 msgr2=0x7f5674079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:15.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.760+0000 7f567c000700 1 -- 172.21.15.1:0/3476986384 shutdown_connections 2024-04-26T01:50:15.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.760+0000 7f567c000700 1 -- 172.21.15.1:0/3476986384 wait complete. 2024-04-26T01:50:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.762+0000 7f567c000700 1 Processor -- start 2024-04-26T01:50:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.762+0000 7f567c000700 1 -- start start 2024-04-26T01:50:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.762+0000 7f567c000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5674075590 0x7f567419b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.762+0000 7f567c000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f567410f440 0x7f567419ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.763+0000 7f567c000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f567419fe10 0x7f56741a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.763+0000 7f567c000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5674114e60 con 0x7f567419fe10 2024-04-26T01:50:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.763+0000 7f567c000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5674114ce0 con 0x7f5674075590 2024-04-26T01:50:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.763+0000 7f567c000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5674114fe0 con 0x7f567410f440 2024-04-26T01:50:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.763+0000 7f567a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f567419fe10 0x7f56741a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.763+0000 7f567a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f567419fe10 0x7f56741a0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38962/0 (socket says 172.21.15.1:38962) 2024-04-26T01:50:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.763+0000 7f567a000700 1 -- 172.21.15.1:0/3995132138 learned_addr learned my addr 172.21.15.1:0/3995132138 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:15.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.763+0000 7f5678c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f567410f440 0x7f567419ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:15.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.763+0000 7f5679600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5674075590 0x7f567419b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:15.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.763+0000 7f567a000700 1 -- 172.21.15.1:0/3995132138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f567410f440 msgr2=0x7f567419ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:15.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.763+0000 7f567a000700 1 --2- 172.21.15.1:0/3995132138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f567410f440 0x7f567419ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:15.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.763+0000 7f567a000700 1 -- 172.21.15.1:0/3995132138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5674075590 msgr2=0x7f567419b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:15.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.763+0000 7f567a000700 1 --2- 172.21.15.1:0/3995132138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5674075590 0x7f567419b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:15.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.763+0000 7f567a000700 1 -- 172.21.15.1:0/3995132138 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56741a0930 con 0x7f567419fe10 2024-04-26T01:50:15.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.764+0000 7f567a000700 1 --2- 172.21.15.1:0/3995132138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f567419fe10 0x7f56741a0290 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f566c00ef90 tx=0x7f566c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:15.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.764+0000 7f5672000700 1 -- 172.21.15.1:0/3995132138 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c0092e0 con 0x7f567419fe10 2024-04-26T01:50:15.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.764+0000 7f5672000700 1 -- 172.21.15.1:0/3995132138 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566c004bb0 con 0x7f567419fe10 2024-04-26T01:50:15.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.764+0000 7f5672000700 1 -- 172.21.15.1:0/3995132138 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c004280 con 0x7f567419fe10 2024-04-26T01:50:15.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.764+0000 7f567c000700 1 -- 172.21.15.1:0/3995132138 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56741a0bc0 con 0x7f567419fe10 2024-04-26T01:50:15.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.764+0000 7f567c000700 1 -- 172.21.15.1:0/3995132138 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56741a83f0 con 0x7f567419fe10 2024-04-26T01:50:15.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.765+0000 7f567c000700 1 -- 172.21.15.1:0/3995132138 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5674069730 con 0x7f567419fe10 2024-04-26T01:50:15.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.767+0000 7f5672000700 1 -- 172.21.15.1:0/3995132138 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f566c01b030 con 0x7f567419fe10 2024-04-26T01:50:15.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.769+0000 7f5672000700 1 --2- 172.21.15.1:0/3995132138 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f565c0a3cc0 0x7f565c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:15.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.769+0000 7f5672000700 1 -- 172.21.15.1:0/3995132138 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f566c0cfea0 con 0x7f567419fe10 2024-04-26T01:50:15.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.769+0000 7f5679600700 1 --2- 172.21.15.1:0/3995132138 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f565c0a3cc0 0x7f565c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:15.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.770+0000 7f5679600700 1 --2- 172.21.15.1:0/3995132138 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f565c0a3cc0 0x7f565c0a6180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5668000c00 tx=0x7f5668005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:15.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:15.774+0000 7f5672000700 1 -- 172.21.15.1:0/3995132138 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f566c022020 con 0x7f567419fe10 2024-04-26T01:50:16.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:16.058+0000 7f567c000700 1 -- 172.21.15.1:0/3995132138 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f567402df80 con 0x7f567419fe10 2024-04-26T01:50:16.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:16.059+0000 7f5672000700 1 -- 172.21.15.1:0/3995132138 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11841 (secure 0 0 0) 0x7f566c010040 con 0x7f567419fe10 2024-04-26T01:50:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:50:16.058 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:47:54.077766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2952462325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2952462325},{"type":"v1","addr":"172.21.15.173:6835","nonce":2952462325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24514,24518,24499]},"id":1}]} 2024-04-26T01:50:16.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:16.062+0000 7f567c000700 1 -- 172.21.15.1:0/3995132138 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f565c0a3cc0 msgr2=0x7f565c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:16.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:16.062+0000 7f567c000700 1 --2- 172.21.15.1:0/3995132138 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f565c0a3cc0 0x7f565c0a6180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5668000c00 tx=0x7f5668005fd0 comp rx=0 tx=0).stop 2024-04-26T01:50:16.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:16.062+0000 7f567c000700 1 -- 172.21.15.1:0/3995132138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f567419fe10 msgr2=0x7f56741a0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:16.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:16.062+0000 7f567c000700 1 --2- 172.21.15.1:0/3995132138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f567419fe10 0x7f56741a0290 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f566c00ef90 tx=0x7f566c00c3a0 comp rx=0 tx=0).stop 2024-04-26T01:50:16.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:16.062+0000 7f567c000700 1 -- 172.21.15.1:0/3995132138 shutdown_connections 2024-04-26T01:50:16.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:16.062+0000 7f567c000700 1 --2- 172.21.15.1:0/3995132138 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f565c0a3cc0 0x7f565c0a6180 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:16.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:16.062+0000 7f567c000700 1 --2- 172.21.15.1:0/3995132138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f567419fe10 0x7f56741a0290 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:16.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:16.062+0000 7f567c000700 1 --2- 172.21.15.1:0/3995132138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5674075590 0x7f567419b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:16.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:16.062+0000 7f567c000700 1 --2- 172.21.15.1:0/3995132138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f567410f440 0x7f567419ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:16.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:16.062+0000 7f567c000700 1 -- 172.21.15.1:0/3995132138 >> 172.21.15.1:0/3995132138 conn(0x7f5674079ba0 msgr2=0x7f56740ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:16.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:16.063+0000 7f567c000700 1 -- 172.21.15.1:0/3995132138 shutdown_connections 2024-04-26T01:50:16.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:16.063+0000 7f567c000700 1 -- 172.21.15.1:0/3995132138 wait complete. 2024-04-26T01:50:16.063 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-04-26T01:50:16.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:16 smithi173 ceph-mon[32553]: pgmap v409: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:16.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:16 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3995132138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:50:16.863 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:16 smithi001 ceph-mon[27822]: pgmap v409: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:16.863 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:16 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3995132138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:50:16.924 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:50:16.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:16 smithi097 ceph-mon[32853]: pgmap v409: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:16.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:16 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3995132138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:50:17.401 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:18.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:18 smithi001 ceph-mon[27822]: pgmap v410: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:18.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:18 smithi173 ceph-mon[32553]: pgmap v410: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:18.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:18 smithi097 ceph-mon[32853]: pgmap v410: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.564+0000 7f8f58c00700 1 -- 172.21.15.1:0/328316988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f5410e3b0 msgr2=0x7f8f54115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.564+0000 7f8f58c00700 1 --2- 172.21.15.1:0/328316988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f5410e3b0 0x7f8f54115f80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00b0d0 tx=0x7f8f4c00b3e0 comp rx=0 tx=0).stop 2024-04-26T01:50:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.565+0000 7f8f58c00700 1 -- 172.21.15.1:0/328316988 shutdown_connections 2024-04-26T01:50:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.565+0000 7f8f58c00700 1 --2- 172.21.15.1:0/328316988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f5410e3b0 0x7f8f54115f80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.565+0000 7f8f58c00700 1 --2- 172.21.15.1:0/328316988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f541092d0 0x7f8f541096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.565+0000 7f8f58c00700 1 --2- 172.21.15.1:0/328316988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f54109c80 0x7f8f5410dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.565+0000 7f8f58c00700 1 -- 172.21.15.1:0/328316988 >> 172.21.15.1:0/328316988 conn(0x7f8f54076210 msgr2=0x7f8f54107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.565+0000 7f8f58c00700 1 -- 172.21.15.1:0/328316988 shutdown_connections 2024-04-26T01:50:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.565+0000 7f8f58c00700 1 -- 172.21.15.1:0/328316988 wait complete. 2024-04-26T01:50:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.566+0000 7f8f58c00700 1 Processor -- start 2024-04-26T01:50:19.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.566+0000 7f8f58c00700 1 -- start start 2024-04-26T01:50:19.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.566+0000 7f8f58c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f541092d0 0x7f8f5419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:19.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.566+0000 7f8f58c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f54109c80 0x7f8f5419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:19.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.567+0000 7f8f58c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f5410e3b0 0x7f8f541a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:19.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.567+0000 7f8f58c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8f541192a0 con 0x7f8f54109c80 2024-04-26T01:50:19.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.567+0000 7f8f58c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8f54119120 con 0x7f8f541092d0 2024-04-26T01:50:19.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.567+0000 7f8f58c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8f54119420 con 0x7f8f5410e3b0 2024-04-26T01:50:19.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.567+0000 7f8f51400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f54109c80 0x7f8f5419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:19.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.567+0000 7f8f51e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f541092d0 0x7f8f5419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:19.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.567+0000 7f8f51400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f54109c80 0x7f8f5419fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38976/0 (socket says 172.21.15.1:38976) 2024-04-26T01:50:19.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.567+0000 7f8f51400700 1 -- 172.21.15.1:0/2026772816 learned_addr learned my addr 172.21.15.1:0/2026772816 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:19.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.567+0000 7f8f52800700 1 --2- 172.21.15.1:0/2026772816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f5410e3b0 0x7f8f541a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:19.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.567+0000 7f8f51e00700 1 -- 172.21.15.1:0/2026772816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f5410e3b0 msgr2=0x7f8f541a3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:19.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.567+0000 7f8f51e00700 1 --2- 172.21.15.1:0/2026772816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f5410e3b0 0x7f8f541a3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:19.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.567+0000 7f8f51e00700 1 -- 172.21.15.1:0/2026772816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f54109c80 msgr2=0x7f8f5419fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:19.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.567+0000 7f8f51e00700 1 --2- 172.21.15.1:0/2026772816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f54109c80 0x7f8f5419fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:19.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.567+0000 7f8f51e00700 1 -- 172.21.15.1:0/2026772816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f541a46d0 con 0x7f8f541092d0 2024-04-26T01:50:19.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.567+0000 7f8f51400700 1 --2- 172.21.15.1:0/2026772816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f54109c80 0x7f8f5419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:19.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.568+0000 7f8f51e00700 1 --2- 172.21.15.1:0/2026772816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f541092d0 0x7f8f5419f710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400d750 tx=0x7f8f4400da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:19.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.568+0000 7f8f4aa00700 1 -- 172.21.15.1:0/2026772816 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44014030 con 0x7f8f541092d0 2024-04-26T01:50:19.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.568+0000 7f8f4aa00700 1 -- 172.21.15.1:0/2026772816 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f440044d0 con 0x7f8f541092d0 2024-04-26T01:50:19.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.568+0000 7f8f58c00700 1 -- 172.21.15.1:0/2026772816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f541a49c0 con 0x7f8f541092d0 2024-04-26T01:50:19.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.568+0000 7f8f4aa00700 1 -- 172.21.15.1:0/2026772816 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44005420 con 0x7f8f541092d0 2024-04-26T01:50:19.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.568+0000 7f8f58c00700 1 -- 172.21.15.1:0/2026772816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f541ac240 con 0x7f8f541092d0 2024-04-26T01:50:19.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.569+0000 7f8f58c00700 1 -- 172.21.15.1:0/2026772816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f54069730 con 0x7f8f541092d0 2024-04-26T01:50:19.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.571+0000 7f8f4aa00700 1 -- 172.21.15.1:0/2026772816 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f8f44004a50 con 0x7f8f541092d0 2024-04-26T01:50:19.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.572+0000 7f8f4aa00700 1 --2- 172.21.15.1:0/2026772816 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8f380a3cc0 0x7f8f380a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:19.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.573+0000 7f8f4aa00700 1 -- 172.21.15.1:0/2026772816 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f8f440cffd0 con 0x7f8f541092d0 2024-04-26T01:50:19.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.573+0000 7f8f51400700 1 --2- 172.21.15.1:0/2026772816 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8f380a3cc0 0x7f8f380a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:19.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.573+0000 7f8f51400700 1 --2- 172.21.15.1:0/2026772816 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8f380a3cc0 0x7f8f380a6180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8f5410f7a0 tx=0x7f8f40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:19.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.577+0000 7f8f4aa00700 1 -- 172.21.15.1:0/2026772816 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f4409c930 con 0x7f8f541092d0 2024-04-26T01:50:19.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.857+0000 7f8f58c00700 1 -- 172.21.15.1:0/2026772816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f5402df80 con 0x7f8f541092d0 2024-04-26T01:50:19.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.858+0000 7f8f4aa00700 1 -- 172.21.15.1:0/2026772816 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11841 (secure 0 0 0) 0x7f8f4409c230 con 0x7f8f541092d0 2024-04-26T01:50:19.856 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:50:19.857 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:47:54.077766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2952462325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2952462325},{"type":"v1","addr":"172.21.15.173:6835","nonce":2952462325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24514,24518,24499]},"id":1}]} 2024-04-26T01:50:19.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.863+0000 7f8f58c00700 1 -- 172.21.15.1:0/2026772816 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8f380a3cc0 msgr2=0x7f8f380a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:19.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.863+0000 7f8f58c00700 1 --2- 172.21.15.1:0/2026772816 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8f380a3cc0 0x7f8f380a6180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8f5410f7a0 tx=0x7f8f40009260 comp rx=0 tx=0).stop 2024-04-26T01:50:19.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.863+0000 7f8f58c00700 1 -- 172.21.15.1:0/2026772816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f541092d0 msgr2=0x7f8f5419f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:19.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.863+0000 7f8f58c00700 1 --2- 172.21.15.1:0/2026772816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f541092d0 0x7f8f5419f710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400d750 tx=0x7f8f4400da60 comp rx=0 tx=0).stop 2024-04-26T01:50:19.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.863+0000 7f8f58c00700 1 -- 172.21.15.1:0/2026772816 shutdown_connections 2024-04-26T01:50:19.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.863+0000 7f8f58c00700 1 --2- 172.21.15.1:0/2026772816 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f8f380a3cc0 0x7f8f380a6180 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:19.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.863+0000 7f8f58c00700 1 --2- 172.21.15.1:0/2026772816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f5410e3b0 0x7f8f541a3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:19.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.863+0000 7f8f58c00700 1 --2- 172.21.15.1:0/2026772816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f541092d0 0x7f8f5419f710 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:19.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.863+0000 7f8f58c00700 1 --2- 172.21.15.1:0/2026772816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f54109c80 0x7f8f5419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:19.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.863+0000 7f8f58c00700 1 -- 172.21.15.1:0/2026772816 >> 172.21.15.1:0/2026772816 conn(0x7f8f54076210 msgr2=0x7f8f540767d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:19.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.864+0000 7f8f58c00700 1 -- 172.21.15.1:0/2026772816 shutdown_connections 2024-04-26T01:50:19.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:19.864+0000 7f8f58c00700 1 -- 172.21.15.1:0/2026772816 wait complete. 2024-04-26T01:50:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-04-26T01:50:20.774 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:50:20.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:20 smithi173 ceph-mon[32553]: pgmap v411: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:20.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:20 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/2026772816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:50:20.857 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:20 smithi001 ceph-mon[27822]: pgmap v411: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:20.857 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:20 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2026772816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:50:20.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:20 smithi097 ceph-mon[32853]: pgmap v411: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:20.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:20 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/2026772816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:50:21.240 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:22.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:22 smithi001 ceph-mon[27822]: pgmap v412: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:22.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:22 smithi097 ceph-mon[32853]: pgmap v412: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:23.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:22 smithi173 ceph-mon[32553]: pgmap v412: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:23.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.483+0000 7f3395e00700 1 -- 172.21.15.1:0/2368162593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3390075700 msgr2=0x7f3390113440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.483+0000 7f3395e00700 1 --2- 172.21.15.1:0/2368162593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3390075700 0x7f3390113440 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3378009990 tx=0x7f3378009ca0 comp rx=0 tx=0).stop 2024-04-26T01:50:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.483+0000 7f3395e00700 1 -- 172.21.15.1:0/2368162593 shutdown_connections 2024-04-26T01:50:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.483+0000 7f3395e00700 1 --2- 172.21.15.1:0/2368162593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3390113980 0x7f3390115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.483+0000 7f3395e00700 1 --2- 172.21.15.1:0/2368162593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3390074de0 0x7f33900751c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.483+0000 7f3395e00700 1 --2- 172.21.15.1:0/2368162593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3390075700 0x7f3390113440 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.483+0000 7f3395e00700 1 -- 172.21.15.1:0/2368162593 >> 172.21.15.1:0/2368162593 conn(0x7f33900fe4c0 msgr2=0x7f33901008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.483+0000 7f3395e00700 1 -- 172.21.15.1:0/2368162593 shutdown_connections 2024-04-26T01:50:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.483+0000 7f3395e00700 1 -- 172.21.15.1:0/2368162593 wait complete. 2024-04-26T01:50:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.484+0000 7f3395e00700 1 Processor -- start 2024-04-26T01:50:23.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.484+0000 7f3395e00700 1 -- start start 2024-04-26T01:50:23.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.484+0000 7f3395e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3390074de0 0x7f339019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:23.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.484+0000 7f3395e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3390075700 0x7f339019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:23.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.484+0000 7f3395e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3390113980 0x7f33901a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:23.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.484+0000 7f3395e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f33901191c0 con 0x7f3390074de0 2024-04-26T01:50:23.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.484+0000 7f3395e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3390119040 con 0x7f3390113980 2024-04-26T01:50:23.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.484+0000 7f3395e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3390119340 con 0x7f3390075700 2024-04-26T01:50:23.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.485+0000 7f3394a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3390074de0 0x7f339019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.485+0000 7f3394a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3390074de0 0x7f339019f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43604/0 (socket says 172.21.15.1:43604) 2024-04-26T01:50:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.485+0000 7f3395400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3390113980 0x7f33901a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.485+0000 7f3394a00700 1 -- 172.21.15.1:0/4201998325 learned_addr learned my addr 172.21.15.1:0/4201998325 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.485+0000 7f3387e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3390075700 0x7f339019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.485+0000 7f3395400700 1 -- 172.21.15.1:0/4201998325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3390075700 msgr2=0x7f339019fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.485+0000 7f3395400700 1 --2- 172.21.15.1:0/4201998325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3390075700 0x7f339019fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.485+0000 7f3395400700 1 -- 172.21.15.1:0/4201998325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3390074de0 msgr2=0x7f339019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.485+0000 7f3395400700 1 --2- 172.21.15.1:0/4201998325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3390074de0 0x7f339019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.485+0000 7f3395400700 1 -- 172.21.15.1:0/4201998325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33901a4670 con 0x7f3390113980 2024-04-26T01:50:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.486+0000 7f3395400700 1 --2- 172.21.15.1:0/4201998325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3390113980 0x7f33901a3f30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f338800e940 tx=0x7f338800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.486+0000 7f338e000700 1 -- 172.21.15.1:0/4201998325 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388004220 con 0x7f3390113980 2024-04-26T01:50:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.486+0000 7f3395e00700 1 -- 172.21.15.1:0/4201998325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33901a4960 con 0x7f3390113980 2024-04-26T01:50:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.486+0000 7f338e000700 1 -- 172.21.15.1:0/4201998325 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33880043c0 con 0x7f3390113980 2024-04-26T01:50:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.486+0000 7f338e000700 1 -- 172.21.15.1:0/4201998325 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388010700 con 0x7f3390113980 2024-04-26T01:50:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.486+0000 7f3395e00700 1 -- 172.21.15.1:0/4201998325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33901ac1e0 con 0x7f3390113980 2024-04-26T01:50:23.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.487+0000 7f3395e00700 1 -- 172.21.15.1:0/4201998325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3390069730 con 0x7f3390113980 2024-04-26T01:50:23.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.489+0000 7f338e000700 1 -- 172.21.15.1:0/4201998325 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f3388004750 con 0x7f3390113980 2024-04-26T01:50:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.491+0000 7f338e000700 1 --2- 172.21.15.1:0/4201998325 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f33740a3c40 0x7f33740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.491+0000 7f338e000700 1 -- 172.21.15.1:0/4201998325 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f33880d00c0 con 0x7f3390113980 2024-04-26T01:50:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.491+0000 7f3394a00700 1 --2- 172.21.15.1:0/4201998325 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f33740a3c40 0x7f33740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.492+0000 7f3394a00700 1 --2- 172.21.15.1:0/4201998325 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f33740a3c40 0x7f33740a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3380009440 tx=0x7f338000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:23.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.495+0000 7f338e000700 1 -- 172.21.15.1:0/4201998325 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f338809ca20 con 0x7f3390113980 2024-04-26T01:50:23.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.730+0000 7f3395e00700 1 -- 172.21.15.1:0/4201998325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33901a09c0 con 0x7f3390113980 2024-04-26T01:50:23.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.732+0000 7f338e000700 1 -- 172.21.15.1:0/4201998325 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22649 (secure 0 0 0) 0x7f3388004d70 con 0x7f3390113980 2024-04-26T01:50:23.731 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:50:23.731 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":92,"fsid":"77cfcc80-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:38:03.531182+0000","modified":"2024-04-26T01:48:22.381867+0000","last_up_change":"2024-04-26T01:46:16.151275+0000","last_in_change":"2024-04-26T01:45:56.514228+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:42:04.805488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:47:21.957421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:47:25.860260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb10ae12-09ee-4d6b-bc78-8c6016613b81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6803","nonce":3176633965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6805","nonce":3176633965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6809","nonce":3176633965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6807","nonce":3176633965}]},"public_addr":"172.21.15.1:6803/3176633965","cluster_addr":"172.21.15.1:6805/3176633965","heartbeat_back_addr":"172.21.15.1:6809/3176633965","heartbeat_front_addr":"172.21.15.1:6807/3176633965","state":["exists","up"]},{"osd":1,"uuid":"acd12888-6ff6-40bc-99e7-e5c395e85b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6801","nonce":3755022621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6803","nonce":3755022621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6807","nonce":3755022621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6805","nonce":3755022621}]},"public_addr":"172.21.15.97:6801/3755022621","cluster_addr":"172.21.15.97:6803/3755022621","heartbeat_back_addr":"172.21.15.97:6807/3755022621","heartbeat_front_addr":"172.21.15.97:6805/3755022621","state":["exists","up"]},{"osd":2,"uuid":"79971508-e256-4145-8111-a75666e886fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6801","nonce":734861771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6803","nonce":734861771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6807","nonce":734861771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6805","nonce":734861771}]},"public_addr":"172.21.15.173:6801/734861771","cluster_addr":"172.21.15.173:6803/734861771","heartbeat_back_addr":"172.21.15.173:6807/734861771","heartbeat_front_addr":"172.21.15.173:6805/734861771","state":["exists","up"]},{"osd":3,"uuid":"a8ccc4a9-0f31-446a-8bcb-4691532397c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6811","nonce":3349961961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6813","nonce":3349961961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6817","nonce":3349961961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6815","nonce":3349961961}]},"public_addr":"172.21.15.1:6811/3349961961","cluster_addr":"172.21.15.1:6813/3349961961","heartbeat_back_addr":"172.21.15.1:6817/3349961961","heartbeat_front_addr":"172.21.15.1:6815/3349961961","state":["exists","up"]},{"osd":4,"uuid":"5b724648-4df7-40fc-89d5-55f170cdd806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6809","nonce":2494639397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6811","nonce":2494639397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6815","nonce":2494639397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6813","nonce":2494639397}]},"public_addr":"172.21.15.97:6809/2494639397","cluster_addr":"172.21.15.97:6811/2494639397","heartbeat_back_addr":"172.21.15.97:6815/2494639397","heartbeat_front_addr":"172.21.15.97:6813/2494639397","state":["exists","up"]},{"osd":5,"uuid":"a5a24a9c-3007-4b87-8562-40c98c41ef8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6809","nonce":1026507957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6811","nonce":1026507957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6815","nonce":1026507957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6813","nonce":1026507957}]},"public_addr":"172.21.15.173:6809/1026507957","cluster_addr":"172.21.15.173:6811/1026507957","heartbeat_back_addr":"172.21.15.173:6815/1026507957","heartbeat_front_addr":"172.21.15.173:6813/1026507957","state":["exists","up"]},{"osd":6,"uuid":"97cd2e88-a56c-4b1a-8104-65b23a77986a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6819","nonce":444030587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6821","nonce":444030587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6825","nonce":444030587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6823","nonce":444030587}]},"public_addr":"172.21.15.1:6819/444030587","cluster_addr":"172.21.15.1:6821/444030587","heartbeat_back_addr":"172.21.15.1:6825/444030587","heartbeat_front_addr":"172.21.15.1:6823/444030587","state":["exists","up"]},{"osd":7,"uuid":"afc0c2dd-e112-4358-b3aa-944366d5d383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6817","nonce":2600022531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6819","nonce":2600022531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6823","nonce":2600022531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6821","nonce":2600022531}]},"public_addr":"172.21.15.97:6817/2600022531","cluster_addr":"172.21.15.97:6819/2600022531","heartbeat_back_addr":"172.21.15.97:6823/2600022531","heartbeat_front_addr":"172.21.15.97:6821/2600022531","state":["exists","up"]},{"osd":8,"uuid":"41fa612b-793e-44da-af8b-44d17a9de303","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6817","nonce":1334205527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6819","nonce":1334205527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6823","nonce":1334205527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6821","nonce":1334205527}]},"public_addr":"172.21.15.173:6817/1334205527","cluster_addr":"172.21.15.173:6819/1334205527","heartbeat_back_addr":"172.21.15.173:6823/1334205527","heartbeat_front_addr":"172.21.15.173:6821/1334205527","state":["exists","up"]},{"osd":9,"uuid":"bbbb163b-f8c4-4be5-8f80-82445e8d407f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6827","nonce":2973486839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6829","nonce":2973486839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6833","nonce":2973486839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6831","nonce":2973486839}]},"public_addr":"172.21.15.1:6827/2973486839","cluster_addr":"172.21.15.1:6829/2973486839","heartbeat_back_addr":"172.21.15.1:6833/2973486839","heartbeat_front_addr":"172.21.15.1:6831/2973486839","state":["exists","up"]},{"osd":10,"uuid":"1b447b34-0600-4727-826d-b3d68adfb18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6825","nonce":1693425268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6827","nonce":1693425268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6831","nonce":1693425268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6829","nonce":1693425268}]},"public_addr":"172.21.15.97:6825/1693425268","cluster_addr":"172.21.15.97:6827/1693425268","heartbeat_back_addr":"172.21.15.97:6831/1693425268","heartbeat_front_addr":"172.21.15.97:6829/1693425268","state":["exists","up"]},{"osd":11,"uuid":"f439aae1-f50a-4176-889d-36fa5131541a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6825","nonce":1141524960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6827","nonce":1141524960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6831","nonce":1141524960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6829","nonce":1141524960}]},"public_addr":"172.21.15.173:6825/1141524960","cluster_addr":"172.21.15.173:6827/1141524960","heartbeat_back_addr":"172.21.15.173:6831/1141524960","heartbeat_front_addr":"172.21.15.173:6829/1141524960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:12.955643+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:38.846652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:02.556062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:34.900933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:01.170878+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:24.842573+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:58.304440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:24.868689+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:48.772003+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:22.611681+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:49.470779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:14.163984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6800/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/1000222947":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/457958597":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6801/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:6800/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/2715798962":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/1923058728":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/2489038777":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/2869502448":"2024-04-27T01:38:45.974031+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:50:23.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.735+0000 7f3395e00700 1 -- 172.21.15.1:0/4201998325 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f33740a3c40 msgr2=0x7f33740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:23.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.735+0000 7f3395e00700 1 --2- 172.21.15.1:0/4201998325 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f33740a3c40 0x7f33740a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3380009440 tx=0x7f338000b040 comp rx=0 tx=0).stop 2024-04-26T01:50:23.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.735+0000 7f3395e00700 1 -- 172.21.15.1:0/4201998325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3390113980 msgr2=0x7f33901a3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:23.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.735+0000 7f3395e00700 1 --2- 172.21.15.1:0/4201998325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3390113980 0x7f33901a3f30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f338800e940 tx=0x7f338800ec50 comp rx=0 tx=0).stop 2024-04-26T01:50:23.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.735+0000 7f3395e00700 1 -- 172.21.15.1:0/4201998325 shutdown_connections 2024-04-26T01:50:23.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.735+0000 7f3395e00700 1 --2- 172.21.15.1:0/4201998325 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f33740a3c40 0x7f33740a6100 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:23.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.735+0000 7f3395e00700 1 --2- 172.21.15.1:0/4201998325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3390113980 0x7f33901a3f30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:23.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.735+0000 7f3395e00700 1 --2- 172.21.15.1:0/4201998325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3390074de0 0x7f339019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:23.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.735+0000 7f3395e00700 1 --2- 172.21.15.1:0/4201998325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3390075700 0x7f339019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:23.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.735+0000 7f3395e00700 1 -- 172.21.15.1:0/4201998325 >> 172.21.15.1:0/4201998325 conn(0x7f33900fe4c0 msgr2=0x7f33900ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:23.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.736+0000 7f3395e00700 1 -- 172.21.15.1:0/4201998325 shutdown_connections 2024-04-26T01:50:23.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:23.736+0000 7f3395e00700 1 -- 172.21.15.1:0/4201998325 wait complete. 2024-04-26T01:50:24.672 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:50:24.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:24 smithi001 ceph-mon[27822]: pgmap v413: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:24.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:24 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/4201998325' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:50:24.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:24 smithi097 ceph-mon[32853]: pgmap v413: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:24.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:24 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/4201998325' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:50:25.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:24 smithi173 ceph-mon[32553]: pgmap v413: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:25.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:24 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/4201998325' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:50:25.167 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:26.759 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:26 smithi001 ceph-mon[27822]: pgmap v414: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:26.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:26 smithi097 ceph-mon[32853]: pgmap v414: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:27.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:26 smithi173 ceph-mon[32553]: pgmap v414: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:27.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.314+0000 7f0d10600700 1 -- 172.21.15.1:0/802439175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d08109c80 msgr2=0x7f0d0810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:27.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.314+0000 7f0d10600700 1 --2- 172.21.15.1:0/802439175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d08109c80 0x7f0d0810dcd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40099c0 tx=0x7f0cf4009cd0 comp rx=0 tx=0).stop 2024-04-26T01:50:27.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.315+0000 7f0d10600700 1 -- 172.21.15.1:0/802439175 shutdown_connections 2024-04-26T01:50:27.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.315+0000 7f0d10600700 1 --2- 172.21.15.1:0/802439175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d0810e3b0 0x7f0d08115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:27.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.315+0000 7f0d10600700 1 --2- 172.21.15.1:0/802439175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d081092d0 0x7f0d081096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:27.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.315+0000 7f0d10600700 1 --2- 172.21.15.1:0/802439175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d08109c80 0x7f0d0810dcd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:27.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.315+0000 7f0d10600700 1 -- 172.21.15.1:0/802439175 >> 172.21.15.1:0/802439175 conn(0x7f0d08076210 msgr2=0x7f0d08107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:27.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.315+0000 7f0d10600700 1 -- 172.21.15.1:0/802439175 shutdown_connections 2024-04-26T01:50:27.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.315+0000 7f0d10600700 1 -- 172.21.15.1:0/802439175 wait complete. 2024-04-26T01:50:27.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.316+0000 7f0d10600700 1 Processor -- start 2024-04-26T01:50:27.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.316+0000 7f0d10600700 1 -- start start 2024-04-26T01:50:27.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.316+0000 7f0d10600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d081092d0 0x7f0d080741b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:27.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.316+0000 7f0d10600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d08109c80 0x7f0d080746f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:27.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.317+0000 7f0d10600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d0810e3b0 0x7f0d0806d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:27.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.317+0000 7f0d10600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0d08119240 con 0x7f0d08109c80 2024-04-26T01:50:27.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.317+0000 7f0d10600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0d081190c0 con 0x7f0d081092d0 2024-04-26T01:50:27.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.317+0000 7f0d10600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0d081193c0 con 0x7f0d0810e3b0 2024-04-26T01:50:27.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.317+0000 7f0d0d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d08109c80 0x7f0d080746f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:27.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.317+0000 7f0d0dc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d081092d0 0x7f0d080741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:27.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.317+0000 7f0d0dc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d081092d0 0x7f0d080741b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:56914/0 (socket says 172.21.15.1:56914) 2024-04-26T01:50:27.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.317+0000 7f0d0d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d08109c80 0x7f0d080746f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43628/0 (socket says 172.21.15.1:43628) 2024-04-26T01:50:27.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.317+0000 7f0d0dc00700 1 -- 172.21.15.1:0/1695045656 learned_addr learned my addr 172.21.15.1:0/1695045656 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:27.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.317+0000 7f0d0e600700 1 --2- 172.21.15.1:0/1695045656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d0810e3b0 0x7f0d0806d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:27.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.317+0000 7f0d0dc00700 1 -- 172.21.15.1:0/1695045656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d0810e3b0 msgr2=0x7f0d0806d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:27.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.317+0000 7f0d0dc00700 1 --2- 172.21.15.1:0/1695045656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d0810e3b0 0x7f0d0806d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:27.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.317+0000 7f0d0dc00700 1 -- 172.21.15.1:0/1695045656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d08109c80 msgr2=0x7f0d080746f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:27.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.317+0000 7f0d0dc00700 1 --2- 172.21.15.1:0/1695045656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d08109c80 0x7f0d080746f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:27.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.317+0000 7f0d0dc00700 1 -- 172.21.15.1:0/1695045656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0806dd50 con 0x7f0d081092d0 2024-04-26T01:50:27.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.317+0000 7f0d0d200700 1 --2- 172.21.15.1:0/1695045656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d08109c80 0x7f0d080746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:27.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.318+0000 7f0d0dc00700 1 --2- 172.21.15.1:0/1695045656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d081092d0 0x7f0d080741b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00eb30 tx=0x7f0cfc00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:27.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.318+0000 7f0d06000700 1 -- 172.21.15.1:0/1695045656 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc00cbd0 con 0x7f0d081092d0 2024-04-26T01:50:27.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.318+0000 7f0d06000700 1 -- 172.21.15.1:0/1695045656 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cfc0044d0 con 0x7f0d081092d0 2024-04-26T01:50:27.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.318+0000 7f0d10600700 1 -- 172.21.15.1:0/1695045656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0806e040 con 0x7f0d081092d0 2024-04-26T01:50:27.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.318+0000 7f0d06000700 1 -- 172.21.15.1:0/1695045656 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc003e30 con 0x7f0d081092d0 2024-04-26T01:50:27.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.318+0000 7f0d10600700 1 -- 172.21.15.1:0/1695045656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d08071fc0 con 0x7f0d081092d0 2024-04-26T01:50:27.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.319+0000 7f0d10600700 1 -- 172.21.15.1:0/1695045656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d0810a7b0 con 0x7f0d081092d0 2024-04-26T01:50:27.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.321+0000 7f0d06000700 1 -- 172.21.15.1:0/1695045656 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f0cfc004000 con 0x7f0d081092d0 2024-04-26T01:50:27.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.323+0000 7f0d06000700 1 --2- 172.21.15.1:0/1695045656 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0cf00a3c40 0x7f0cf00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:27.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.323+0000 7f0d06000700 1 -- 172.21.15.1:0/1695045656 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f0cfc0d07f0 con 0x7f0d081092d0 2024-04-26T01:50:27.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.323+0000 7f0d0d200700 1 --2- 172.21.15.1:0/1695045656 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0cf00a3c40 0x7f0cf00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:27.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.324+0000 7f0d0d200700 1 --2- 172.21.15.1:0/1695045656 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0cf00a3c40 0x7f0cf00a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4000c00 tx=0x7f0cf4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:27.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.325+0000 7f0d06000700 1 -- 172.21.15.1:0/1695045656 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0cfc0042f0 con 0x7f0d081092d0 2024-04-26T01:50:27.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.607+0000 7f0d10600700 1 -- 172.21.15.1:0/1695045656 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d08072a20 con 0x7f0d081092d0 2024-04-26T01:50:27.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.608+0000 7f0d06000700 1 -- 172.21.15.1:0/1695045656 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11841 (secure 0 0 0) 0x7f0cfc09d150 con 0x7f0d081092d0 2024-04-26T01:50:27.607 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:50:27.607 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:47:54.077766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2952462325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2952462325},{"type":"v1","addr":"172.21.15.173:6835","nonce":2952462325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24514,24518,24499]},"id":1}]} 2024-04-26T01:50:27.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.612+0000 7f0d10600700 1 -- 172.21.15.1:0/1695045656 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0cf00a3c40 msgr2=0x7f0cf00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:27.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.612+0000 7f0d10600700 1 --2- 172.21.15.1:0/1695045656 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0cf00a3c40 0x7f0cf00a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4000c00 tx=0x7f0cf4005fb0 comp rx=0 tx=0).stop 2024-04-26T01:50:27.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.612+0000 7f0d10600700 1 -- 172.21.15.1:0/1695045656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d081092d0 msgr2=0x7f0d080741b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:27.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.612+0000 7f0d10600700 1 --2- 172.21.15.1:0/1695045656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d081092d0 0x7f0d080741b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00eb30 tx=0x7f0cfc00ee40 comp rx=0 tx=0).stop 2024-04-26T01:50:27.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.613+0000 7f0d10600700 1 -- 172.21.15.1:0/1695045656 shutdown_connections 2024-04-26T01:50:27.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.613+0000 7f0d10600700 1 --2- 172.21.15.1:0/1695045656 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f0cf00a3c40 0x7f0cf00a6100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:27.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.613+0000 7f0d10600700 1 --2- 172.21.15.1:0/1695045656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d0810e3b0 0x7f0d0806d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:27.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.613+0000 7f0d10600700 1 --2- 172.21.15.1:0/1695045656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d081092d0 0x7f0d080741b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:27.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.613+0000 7f0d10600700 1 --2- 172.21.15.1:0/1695045656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d08109c80 0x7f0d080746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:27.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.613+0000 7f0d10600700 1 -- 172.21.15.1:0/1695045656 >> 172.21.15.1:0/1695045656 conn(0x7f0d08076210 msgr2=0x7f0d08076830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:27.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.613+0000 7f0d10600700 1 -- 172.21.15.1:0/1695045656 shutdown_connections 2024-04-26T01:50:27.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:27.613+0000 7f0d10600700 1 -- 172.21.15.1:0/1695045656 wait complete. 2024-04-26T01:50:27.613 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-04-26T01:50:28.562 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T01:47:29.999521+0000', 'modified': '2024-04-26T01:47:54.077766+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24518, 'mds_1': 24514, 'mds_2': 24499}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24464': {'gid': 24464, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6835/763675408', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 763675408}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 763675408}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6837/3607142398', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 3607142398}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 3607142398}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.173:6835/2952462325', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 2952462325}, {'type': 'v1', 'addr': '172.21.15.173:6835', 'nonce': 2952462325}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.173:6837/2502298793', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6836', 'nonce': 2502298793}, {'type': 'v1', 'addr': '172.21.15.173:6837', 'nonce': 2502298793}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24514': {'gid': 24514, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.173:6839/373692430', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6838', 'nonce': 373692430}, {'type': 'v1', 'addr': '172.21.15.173:6839', 'nonce': 373692430}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.97:6839/1315554162', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 1315554162}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 1315554162}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24518, 'qdb_cluster': [24514, 24518, 24499]} 2024-04-26T01:50:28.562 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-26T01:50:28.562 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph tell mds.c status 2024-04-26T01:50:28.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:28 smithi001 ceph-mon[27822]: pgmap v415: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:28.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:28 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1695045656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:50:28.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:28 smithi097 ceph-mon[32853]: pgmap v415: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:28.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:28 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1695045656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:50:29.033 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:29.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:28 smithi173 ceph-mon[32553]: pgmap v415: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:29.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:28 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1695045656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:50:30.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:30 smithi001 ceph-mon[27822]: pgmap v416: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:30.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:30 smithi097 ceph-mon[32853]: pgmap v416: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:31.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:30 smithi173 ceph-mon[32553]: pgmap v416: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:31.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.247+0000 7fd7db800700 1 -- 172.21.15.1:0/557208111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7d4077890 msgr2=0x7fd7d41133b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:31.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.247+0000 7fd7db800700 1 --2- 172.21.15.1:0/557208111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7d4077890 0x7fd7d41133b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009990 tx=0x7fd7c0009ca0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.248+0000 7fd7db800700 1 -- 172.21.15.1:0/557208111 shutdown_connections 2024-04-26T01:50:31.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.248+0000 7fd7db800700 1 --2- 172.21.15.1:0/557208111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7d41138f0 0x7fd7d4115ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.248+0000 7fd7db800700 1 --2- 172.21.15.1:0/557208111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7d4076f70 0x7fd7d4077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.248+0000 7fd7db800700 1 --2- 172.21.15.1:0/557208111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7d4077890 0x7fd7d41133b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.248+0000 7fd7db800700 1 -- 172.21.15.1:0/557208111 >> 172.21.15.1:0/557208111 conn(0x7fd7d4100780 msgr2=0x7fd7d4102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:31.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.248+0000 7fd7db800700 1 -- 172.21.15.1:0/557208111 shutdown_connections 2024-04-26T01:50:31.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.248+0000 7fd7db800700 1 -- 172.21.15.1:0/557208111 wait complete. 2024-04-26T01:50:31.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.249+0000 7fd7db800700 1 Processor -- start 2024-04-26T01:50:31.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.249+0000 7fd7db800700 1 -- start start 2024-04-26T01:50:31.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.249+0000 7fd7db800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7d4076f70 0x7fd7d41a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:31.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.249+0000 7fd7db800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7d4077890 0x7fd7d41a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:31.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.249+0000 7fd7db800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7d41138f0 0x7fd7d419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:31.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.249+0000 7fd7db800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d41192c0 con 0x7fd7d41138f0 2024-04-26T01:50:31.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.249+0000 7fd7db800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d4119140 con 0x7fd7d4076f70 2024-04-26T01:50:31.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.249+0000 7fd7db800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d4119440 con 0x7fd7d4077890 2024-04-26T01:50:31.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.250+0000 7fd7d8e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7d4076f70 0x7fd7d41a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:31.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.250+0000 7fd7d3e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7d4077890 0x7fd7d41a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:31.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.250+0000 7fd7d3e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7d4077890 0x7fd7d41a5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:49082/0 (socket says 172.21.15.1:49082) 2024-04-26T01:50:31.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.250+0000 7fd7d3e00700 1 -- 172.21.15.1:0/996033481 learned_addr learned my addr 172.21.15.1:0/996033481 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:31.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.250+0000 7fd7d9800700 1 --2- 172.21.15.1:0/996033481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7d41138f0 0x7fd7d419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:31.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.250+0000 7fd7d3e00700 1 -- 172.21.15.1:0/996033481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7d4076f70 msgr2=0x7fd7d41a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:31.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.250+0000 7fd7d3e00700 1 --2- 172.21.15.1:0/996033481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7d4076f70 0x7fd7d41a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.250+0000 7fd7d3e00700 1 -- 172.21.15.1:0/996033481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7d41138f0 msgr2=0x7fd7d419f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.250+0000 7fd7d3e00700 1 --2- 172.21.15.1:0/996033481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7d41138f0 0x7fd7d419f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.250+0000 7fd7d3e00700 1 -- 172.21.15.1:0/996033481 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d419fda0 con 0x7fd7d4077890 2024-04-26T01:50:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.250+0000 7fd7d9800700 1 --2- 172.21.15.1:0/996033481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7d41138f0 0x7fd7d419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.250+0000 7fd7d8e00700 1 --2- 172.21.15.1:0/996033481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7d4076f70 0x7fd7d41a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.251+0000 7fd7d3e00700 1 --2- 172.21.15.1:0/996033481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7d4077890 0x7fd7d41a5970 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0000c00 tx=0x7fd7c00057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.251+0000 7fd7d1600700 1 -- 172.21.15.1:0/996033481 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0027070 con 0x7fd7d4077890 2024-04-26T01:50:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.251+0000 7fd7d1600700 1 -- 172.21.15.1:0/996033481 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7c0022470 con 0x7fd7d4077890 2024-04-26T01:50:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.251+0000 7fd7db800700 1 -- 172.21.15.1:0/996033481 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d41a0030 con 0x7fd7d4077890 2024-04-26T01:50:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.251+0000 7fd7d1600700 1 -- 172.21.15.1:0/996033481 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0017650 con 0x7fd7d4077890 2024-04-26T01:50:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.251+0000 7fd7db800700 1 -- 172.21.15.1:0/996033481 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d41ac300 con 0x7fd7d4077890 2024-04-26T01:50:31.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.255+0000 7fd7d1600700 1 -- 172.21.15.1:0/996033481 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fd7c002a070 con 0x7fd7d4077890 2024-04-26T01:50:31.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.256+0000 7fd7e616eb80 1 Processor -- start 2024-04-26T01:50:31.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.256+0000 7fd7e616eb80 1 -- start start 2024-04-26T01:50:31.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.256+0000 7fd7d1600700 1 --2- 172.21.15.1:0/996033481 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fd7bc0a3cc0 0x7fd7bc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.256+0000 7fd7d1600700 1 -- 172.21.15.1:0/996033481 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fd7c00d1070 con 0x7fd7d4077890 2024-04-26T01:50:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.256+0000 7fd7d9800700 1 --2- 172.21.15.1:0/996033481 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fd7bc0a3cc0 0x7fd7bc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.256+0000 7fd7e616eb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea136780 0x559dea136b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.256+0000 7fd7e616eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea1370a0 0x559dea1309b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.257+0000 7fd7e616eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea130ef0 0x559dea131330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.257+0000 7fd7e616eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559dea082e40 con 0x559dea1370a0 2024-04-26T01:50:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.257+0000 7fd7e616eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559dea0296e0 con 0x559dea130ef0 2024-04-26T01:50:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.257+0000 7fd7e616eb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559dea07ee00 con 0x559dea136780 2024-04-26T01:50:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.257+0000 7fd7d9800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea130ef0 0x559dea131330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.257+0000 7fd7d3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea1370a0 0x559dea1309b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:31.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.257+0000 7fd7d9800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea130ef0 0x559dea131330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:56940/0 (socket says 172.21.15.1:56940) 2024-04-26T01:50:31.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.257+0000 7fd7d9800700 1 -- 172.21.15.1:0/2594555997 learned_addr learned my addr 172.21.15.1:0/2594555997 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:31.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.257+0000 7fd7d8e00700 1 --2- 172.21.15.1:0/2594555997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea136780 0x559dea136b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:31.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.257+0000 7fd7d8e00700 1 -- 172.21.15.1:0/2594555997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea130ef0 msgr2=0x559dea131330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:31.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.257+0000 7fd7d8e00700 1 --2- 172.21.15.1:0/2594555997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea130ef0 0x559dea131330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.257+0000 7fd7d8e00700 1 -- 172.21.15.1:0/2594555997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea1370a0 msgr2=0x559dea1309b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:31.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.257+0000 7fd7d8e00700 1 --2- 172.21.15.1:0/2594555997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea1370a0 0x559dea1309b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.257+0000 7fd7d8e00700 1 -- 172.21.15.1:0/2594555997 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dea01f630 con 0x559dea136780 2024-04-26T01:50:31.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.257+0000 7fd7d3e00700 1 --2- 172.21.15.1:0/2594555997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea1370a0 0x559dea1309b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:31.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.257+0000 7fd7d9800700 1 --2- 172.21.15.1:0/2594555997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea130ef0 0x559dea131330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:31.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.258+0000 7fd7d9800700 1 --2- 172.21.15.1:0/996033481 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fd7bc0a3cc0 0x7fd7bc0a6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0046f0 tx=0x7fd7cc003d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:31.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.258+0000 7fd7d8e00700 1 --2- 172.21.15.1:0/2594555997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea136780 0x559dea136b60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80114f0 tx=0x7fd7c8011800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e69bae73d3494691 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:31.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.258+0000 7fd7c6000700 1 -- 172.21.15.1:0/2594555997 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8004220 con 0x559dea136780 2024-04-26T01:50:31.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.258+0000 7fd7c6000700 1 -- 172.21.15.1:0/2594555997 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7c80043c0 con 0x559dea136780 2024-04-26T01:50:31.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.258+0000 7fd7c6000700 1 -- 172.21.15.1:0/2594555997 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8005620 con 0x559dea136780 2024-04-26T01:50:31.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.259+0000 7fd7e616eb80 1 -- 172.21.15.1:0/2594555997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea136780 msgr2=0x559dea136b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:31.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.259+0000 7fd7e616eb80 1 --2- 172.21.15.1:0/2594555997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea136780 0x559dea136b60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80114f0 tx=0x7fd7c8011800 comp rx=0 tx=0).stop 2024-04-26T01:50:31.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.259+0000 7fd7e616eb80 1 -- 172.21.15.1:0/2594555997 shutdown_connections 2024-04-26T01:50:31.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.259+0000 7fd7e616eb80 1 --2- 172.21.15.1:0/2594555997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea130ef0 0x559dea131330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.259+0000 7fd7e616eb80 1 --2- 172.21.15.1:0/2594555997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea136780 0x559dea136b60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.259+0000 7fd7e616eb80 1 --2- 172.21.15.1:0/2594555997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea1370a0 0x559dea1309b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.259+0000 7fd7e616eb80 1 -- 172.21.15.1:0/2594555997 >> 172.21.15.1:0/2594555997 conn(0x559dea05ef00 msgr2=0x559dea12b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:31.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.259+0000 7fd7e616eb80 1 -- 172.21.15.1:0/2594555997 shutdown_connections 2024-04-26T01:50:31.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.259+0000 7fd7e616eb80 1 -- 172.21.15.1:0/2594555997 wait complete. 2024-04-26T01:50:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.260+0000 7fd7e616eb80 1 -- start start 2024-04-26T01:50:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.260+0000 7fd7e616eb80 1 Processor -- start 2024-04-26T01:50:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.260+0000 7fd7e616eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea130ef0 0x559dea13cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.260+0000 7fd7e616eb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea136780 0x559dea139f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.260+0000 7fd7e616eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea13a480 0x559dea13a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.260+0000 7fd7e616eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559dea13d920 con 0x559dea130ef0 2024-04-26T01:50:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.260+0000 7fd7e616eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559dea03c810 con 0x559dea13a480 2024-04-26T01:50:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.260+0000 7fd7e616eb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559dea13daa0 con 0x559dea136780 2024-04-26T01:50:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.261+0000 7fd7d8e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea130ef0 0x559dea13cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.261+0000 7fd7e616eb80 10 client.0 _ll_register_callbacks cb 0x559dea01b6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:50:31.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.261+0000 7fd7d8e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea130ef0 0x559dea13cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43674/0 (socket says 172.21.15.1:43674) 2024-04-26T01:50:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.261+0000 7fd7d8e00700 1 -- 172.21.15.1:0/3339401551 learned_addr learned my addr 172.21.15.1:0/3339401551 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.261+0000 7fd7d3e00700 1 --2- 172.21.15.1:0/3339401551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea136780 0x559dea139f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.261+0000 7fd7d9800700 1 --2- 172.21.15.1:0/3339401551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea13a480 0x559dea13a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.261+0000 7fd7d8e00700 1 -- 172.21.15.1:0/3339401551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea136780 msgr2=0x559dea139f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.261+0000 7fd7d8e00700 1 --2- 172.21.15.1:0/3339401551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea136780 0x559dea139f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.261+0000 7fd7d8e00700 1 -- 172.21.15.1:0/3339401551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea13a480 msgr2=0x559dea13a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.261+0000 7fd7d8e00700 1 --2- 172.21.15.1:0/3339401551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea13a480 0x559dea13a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.261+0000 7fd7d8e00700 1 -- 172.21.15.1:0/3339401551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559de9fd0880 con 0x559dea130ef0 2024-04-26T01:50:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.261+0000 7fd7d3e00700 1 --2- 172.21.15.1:0/3339401551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea136780 0x559dea139f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.261+0000 7fd7d9800700 1 --2- 172.21.15.1:0/3339401551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea13a480 0x559dea13a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.262+0000 7fd7d8e00700 1 --2- 172.21.15.1:0/3339401551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea130ef0 0x559dea13cc40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8000c00 tx=0x7fd7c8009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c187c022689246d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.262+0000 7fd7baa00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:50:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.262+0000 7fd7baa00700 1 -- 172.21.15.1:0/3339401551 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8028020 con 0x559dea130ef0 2024-04-26T01:50:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.262+0000 7fd7baa00700 1 -- 172.21.15.1:0/3339401551 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fd7c8004950 con 0x559dea130ef0 2024-04-26T01:50:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.262+0000 7fd7e616eb80 1 -- 172.21.15.1:0/3339401551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7c80043c0 con 0x559dea130ef0 2024-04-26T01:50:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.262+0000 7fd7baa00700 1 -- 172.21.15.1:0/3339401551 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd7c801d070 con 0x559dea130ef0 2024-04-26T01:50:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.262+0000 7fd7e616eb80 10 client.14781 fetch_fsmap learned FSMap version 25 2024-04-26T01:50:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.263+0000 7fd7e616eb80 1 -- 172.21.15.1:0/3339401551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559dea019aa0 con 0x559dea130ef0 2024-04-26T01:50:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.263+0000 7fd7baa00700 1 -- 172.21.15.1:0/3339401551 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7c80043c0 con 0x559dea130ef0 2024-04-26T01:50:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.263+0000 7fd7baa00700 1 -- 172.21.15.1:0/3339401551 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fd7c800e470 con 0x559dea130ef0 2024-04-26T01:50:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.263+0000 7fd7e616eb80 10 client.14781 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:50:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.263+0000 7fd7e616eb80 10 client.14781 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T01:50:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.263+0000 7fd7e616eb80 20 client.14781 populate_metadata read hostname 'smithi001' 2024-04-26T01:50:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.263+0000 7fd7e616eb80 1 --2- 172.21.15.1:0/3339401551 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x559dea14a770 0x559dea14bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:31.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.263+0000 7fd7e616eb80 4 client.14781 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:50:31.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.263+0000 7fd7e616eb80 1 -- 172.21.15.1:0/3339401551 --> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dea090630 con 0x559dea14a770 2024-04-26T01:50:31.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.264+0000 7fd7d3e00700 1 --2- 172.21.15.1:0/3339401551 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x559dea14a770 0x559dea14bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:31.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.264+0000 7fd7d3e00700 1 --2- 172.21.15.1:0/3339401551 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x559dea14a770 0x559dea14bbe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=955ea1b61a1d455e server_cookie=608a037187a9be94 in_seq=0 out_seq=0 2024-04-26T01:50:31.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.264+0000 7fd7baa00700 10 client.14781 ms_handle_connect on v2:172.21.15.173:6836/2502298793 2024-04-26T01:50:31.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.265+0000 7fd7baa00700 1 -- 172.21.15.1:0/3339401551 <== mds.2 v2:172.21.15.173:6836/2502298793 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559dea090630 con 0x559dea14a770 2024-04-26T01:50:31.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.265+0000 7fd7baa00700 10 client.14781 handle_command_reply: tid=0 2024-04-26T01:50:31.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.265+0000 7fd7e616eb80 1 client.14781 shutdown 2024-04-26T01:50:31.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.265+0000 7fd7e616eb80 1 -- 172.21.15.1:0/3339401551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea130ef0 msgr2=0x559dea13cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:31.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.265+0000 7fd7e616eb80 1 --2- 172.21.15.1:0/3339401551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea130ef0 0x559dea13cc40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8000c00 tx=0x7fd7c8009560 comp rx=0 tx=0).stop 2024-04-26T01:50:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.265+0000 7fd7e616eb80 1 -- 172.21.15.1:0/3339401551 shutdown_connections 2024-04-26T01:50:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.265+0000 7fd7e616eb80 1 --2- 172.21.15.1:0/3339401551 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x559dea14a770 0x559dea14bbe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.265+0000 7fd7e616eb80 1 --2- 172.21.15.1:0/3339401551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea13a480 0x559dea13a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.265+0000 7fd7e616eb80 1 --2- 172.21.15.1:0/3339401551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea130ef0 0x559dea13cc40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.265+0000 7fd7e616eb80 1 --2- 172.21.15.1:0/3339401551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea136780 0x559dea139f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.266+0000 7fd7e616eb80 1 -- 172.21.15.1:0/3339401551 >> 172.21.15.1:0/3339401551 conn(0x559dea05ef00 msgr2=0x559dea12ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.266+0000 7fd7baa00700 0 client.14781 ms_handle_reset on v2:172.21.15.173:6836/2502298793 2024-04-26T01:50:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.266+0000 7fd7e616eb80 1 -- 172.21.15.1:0/3339401551 shutdown_connections 2024-04-26T01:50:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.266+0000 7fd7e616eb80 1 -- 172.21.15.1:0/3339401551 wait complete. 2024-04-26T01:50:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.266+0000 7fd7e616eb80 20 client.14781 trim_cache size 0 max 16384 2024-04-26T01:50:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.289+0000 7fd7e616eb80 1 Processor -- start 2024-04-26T01:50:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.289+0000 7fd7e616eb80 1 -- start start 2024-04-26T01:50:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.289+0000 7fd7e616eb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea130ef0 0x559dea14bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.289+0000 7fd7e616eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea136780 0x559dea146e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.289+0000 7fd7e616eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea13a480 0x559dea1312d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.289+0000 7fd7e616eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559dea149f20 con 0x559dea13a480 2024-04-26T01:50:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.289+0000 7fd7e616eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559dea14a5f0 con 0x559dea136780 2024-04-26T01:50:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.289+0000 7fd7e616eb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559dea14acc0 con 0x559dea130ef0 2024-04-26T01:50:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.290+0000 7fd7d9800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea13a480 0x559dea1312d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.290+0000 7fd7d8e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea130ef0 0x559dea14bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.290+0000 7fd7d9800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea13a480 0x559dea1312d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43676/0 (socket says 172.21.15.1:43676) 2024-04-26T01:50:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.290+0000 7fd7d3e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea136780 0x559dea146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.290+0000 7fd7d9800700 1 -- 172.21.15.1:0/3324023056 learned_addr learned my addr 172.21.15.1:0/3324023056 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:31.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.290+0000 7fd7d8e00700 1 -- 172.21.15.1:0/3324023056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea136780 msgr2=0x559dea146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.290+0000 7fd7d8e00700 1 --2- 172.21.15.1:0/3324023056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea136780 0x559dea146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.290+0000 7fd7d8e00700 1 -- 172.21.15.1:0/3324023056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea13a480 msgr2=0x559dea1312d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.290+0000 7fd7d8e00700 1 --2- 172.21.15.1:0/3324023056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea13a480 0x559dea1312d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.290+0000 7fd7d8e00700 1 -- 172.21.15.1:0/3324023056 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dea13c880 con 0x559dea130ef0 2024-04-26T01:50:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.291+0000 7fd7d8e00700 1 --2- 172.21.15.1:0/3324023056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea130ef0 0x559dea14bbe0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009f80 tx=0x7fd7c802cf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baf963a1aa157fcf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.291+0000 7fd7c5600700 1 -- 172.21.15.1:0/3324023056 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8039020 con 0x559dea130ef0 2024-04-26T01:50:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.291+0000 7fd7c5600700 1 -- 172.21.15.1:0/3324023056 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7c801e040 con 0x559dea130ef0 2024-04-26T01:50:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.291+0000 7fd7c5600700 1 -- 172.21.15.1:0/3324023056 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8028040 con 0x559dea130ef0 2024-04-26T01:50:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.292+0000 7fd7e616eb80 1 -- 172.21.15.1:0/3324023056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea130ef0 msgr2=0x559dea14bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.292+0000 7fd7e616eb80 1 --2- 172.21.15.1:0/3324023056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea130ef0 0x559dea14bbe0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009f80 tx=0x7fd7c802cf80 comp rx=0 tx=0).stop 2024-04-26T01:50:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.292+0000 7fd7e616eb80 1 -- 172.21.15.1:0/3324023056 shutdown_connections 2024-04-26T01:50:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.292+0000 7fd7e616eb80 1 --2- 172.21.15.1:0/3324023056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea13a480 0x559dea1312d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.292+0000 7fd7e616eb80 1 --2- 172.21.15.1:0/3324023056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea130ef0 0x559dea14bbe0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.292+0000 7fd7e616eb80 1 --2- 172.21.15.1:0/3324023056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea136780 0x559dea146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.292+0000 7fd7e616eb80 1 -- 172.21.15.1:0/3324023056 >> 172.21.15.1:0/3324023056 conn(0x559dea05ef00 msgr2=0x559dea1496f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:31.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.292+0000 7fd7e616eb80 1 -- 172.21.15.1:0/3324023056 shutdown_connections 2024-04-26T01:50:31.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.292+0000 7fd7e616eb80 1 -- 172.21.15.1:0/3324023056 wait complete. 2024-04-26T01:50:31.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.292+0000 7fd7e616eb80 1 -- start start 2024-04-26T01:50:31.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.293+0000 7fd7e616eb80 1 Processor -- start 2024-04-26T01:50:31.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.293+0000 7fd7e616eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea130ef0 0x559dea12a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:31.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.293+0000 7fd7e616eb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea136780 0x559dea12ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:31.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.293+0000 7fd7e616eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea13a480 0x559dea12b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:31.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.293+0000 7fd7e616eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559dea14ba60 con 0x559dea130ef0 2024-04-26T01:50:31.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.293+0000 7fd7e616eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559dea14b390 con 0x559dea13a480 2024-04-26T01:50:31.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.293+0000 7fd7e616eb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559dea14c130 con 0x559dea136780 2024-04-26T01:50:31.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.293+0000 7fd7e616eb80 10 client.0 _ll_register_callbacks cb 0x559dea01b6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:50:31.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.293+0000 7fd7d3e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea136780 0x559dea12ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:31.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.294+0000 7fd7d3e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea136780 0x559dea12ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:49112/0 (socket says 172.21.15.1:49112) 2024-04-26T01:50:31.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.294+0000 7fd7d3e00700 1 -- 172.21.15.1:0/2459588432 learned_addr learned my addr 172.21.15.1:0/2459588432 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:31.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.294+0000 7fd7d8e00700 1 --2- 172.21.15.1:0/2459588432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea130ef0 0x559dea12a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:31.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.294+0000 7fd7d9800700 1 --2- 172.21.15.1:0/2459588432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea13a480 0x559dea12b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:31.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.294+0000 7fd7d3e00700 1 -- 172.21.15.1:0/2459588432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea13a480 msgr2=0x559dea12b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:31.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.294+0000 7fd7d3e00700 1 --2- 172.21.15.1:0/2459588432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea13a480 0x559dea12b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.294+0000 7fd7d3e00700 1 -- 172.21.15.1:0/2459588432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea130ef0 msgr2=0x559dea12a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:31.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.294+0000 7fd7d3e00700 1 --2- 172.21.15.1:0/2459588432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea130ef0 0x559dea12a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.294+0000 7fd7d3e00700 1 -- 172.21.15.1:0/2459588432 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dea153120 con 0x559dea136780 2024-04-26T01:50:31.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.294+0000 7fd7d8e00700 1 --2- 172.21.15.1:0/2459588432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea130ef0 0x559dea12a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:50:31.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.294+0000 7fd7d3e00700 1 --2- 172.21.15.1:0/2459588432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea136780 0x559dea12ae40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0099ce0 tx=0x7fd7c0005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1013f7c4ddcd8d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:31.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.294+0000 7fd7baa00700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:50:31.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.294+0000 7fd7baa00700 1 -- 172.21.15.1:0/2459588432 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c00b6060 con 0x559dea136780 2024-04-26T01:50:31.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.294+0000 7fd7baa00700 1 -- 172.21.15.1:0/2459588432 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fd7c009b660 con 0x559dea136780 2024-04-26T01:50:31.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.294+0000 7fd7e616eb80 1 -- 172.21.15.1:0/2459588432 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7c801e040 con 0x559dea136780 2024-04-26T01:50:31.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.295+0000 7fd7baa00700 1 -- 172.21.15.1:0/2459588432 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd7c00bc170 con 0x559dea136780 2024-04-26T01:50:31.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.295+0000 7fd7e616eb80 10 client.24919 fetch_fsmap learned FSMap version 25 2024-04-26T01:50:31.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.295+0000 7fd7e616eb80 1 -- 172.21.15.1:0/2459588432 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559dea1b1ab0 con 0x559dea136780 2024-04-26T01:50:31.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.295+0000 7fd7baa00700 1 -- 172.21.15.1:0/2459588432 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7c801e040 con 0x559dea136780 2024-04-26T01:50:31.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.295+0000 7fd7baa00700 1 -- 172.21.15.1:0/2459588432 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fd7c007f020 con 0x559dea136780 2024-04-26T01:50:31.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.295+0000 7fd7e616eb80 10 client.24919 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:50:31.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.295+0000 7fd7e616eb80 10 client.24919 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T01:50:31.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.295+0000 7fd7e616eb80 20 client.24919 populate_metadata read hostname 'smithi001' 2024-04-26T01:50:31.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.295+0000 7fd7e616eb80 1 --2- 172.21.15.1:0/2459588432 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x559dea1b6680 0x559dea1b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:31.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.295+0000 7fd7e616eb80 4 client.24919 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T01:50:31.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.295+0000 7fd7e616eb80 1 -- 172.21.15.1:0/2459588432 --> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] -- command(tid 0: {"prefix": "status"}) -- 0x559dea08a6d0 con 0x559dea1b6680 2024-04-26T01:50:31.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.296+0000 7fd7d8e00700 1 --2- 172.21.15.1:0/2459588432 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x559dea1b6680 0x559dea1b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:31.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.296+0000 7fd7d8e00700 1 --2- 172.21.15.1:0/2459588432 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x559dea1b6680 0x559dea1b5390 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=154a577b3241d3b1 server_cookie=502ce797277e8f7c in_seq=0 out_seq=0 2024-04-26T01:50:31.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.296+0000 7fd7baa00700 10 client.24919 ms_handle_connect on v2:172.21.15.173:6836/2502298793 2024-04-26T01:50:31.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.297+0000 7fd7baa00700 1 -- 172.21.15.1:0/2459588432 <== mds.2 v2:172.21.15.173:6836/2502298793 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x559dea08a6d0 con 0x559dea1b6680 2024-04-26T01:50:31.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.297+0000 7fd7baa00700 10 client.24919 handle_command_reply: tid=0 2024-04-26T01:50:31.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.297+0000 7fd7e616eb80 1 client.24919 shutdown 2024-04-26T01:50:31.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.297+0000 7fd7e616eb80 1 -- 172.21.15.1:0/2459588432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea136780 msgr2=0x559dea12ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:31.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.297+0000 7fd7e616eb80 1 --2- 172.21.15.1:0/2459588432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea136780 0x559dea12ae40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0099ce0 tx=0x7fd7c0005040 comp rx=0 tx=0).stop 2024-04-26T01:50:31.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.297+0000 7fd7e616eb80 1 -- 172.21.15.1:0/2459588432 shutdown_connections 2024-04-26T01:50:31.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.297+0000 7fd7e616eb80 1 --2- 172.21.15.1:0/2459588432 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x559dea1b6680 0x559dea1b5390 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.297+0000 7fd7e616eb80 1 --2- 172.21.15.1:0/2459588432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dea13a480 0x559dea12b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.297+0000 7fd7e616eb80 1 --2- 172.21.15.1:0/2459588432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559dea130ef0 0x559dea12a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.297+0000 7fd7e616eb80 1 --2- 172.21.15.1:0/2459588432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559dea136780 0x559dea12ae40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.297+0000 7fd7baa00700 0 client.24919 ms_handle_reset on v2:172.21.15.173:6836/2502298793 2024-04-26T01:50:31.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.297+0000 7fd7e616eb80 1 -- 172.21.15.1:0/2459588432 >> 172.21.15.1:0/2459588432 conn(0x559dea05ef00 msgr2=0x559dea1312d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:31.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.297+0000 7fd7e616eb80 1 -- 172.21.15.1:0/2459588432 shutdown_connections 2024-04-26T01:50:31.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.297+0000 7fd7e616eb80 1 -- 172.21.15.1:0/2459588432 wait complete. 2024-04-26T01:50:31.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.297+0000 7fd7e616eb80 20 client.24919 trim_cache size 0 max 16384 2024-04-26T01:50:31.298 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T01:50:31.298 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "77cfcc80-036d-11ef-bc93-c7b262605968", 2024-04-26T01:50:31.298 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 2, 2024-04-26T01:50:31.298 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24499, 2024-04-26T01:50:31.298 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-04-26T01:50:31.298 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-04-26T01:50:31.299 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-04-26T01:50:31.299 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 164.083401194, 2024-04-26T01:50:31.299 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 25, 2024-04-26T01:50:31.299 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 92, 2024-04-26T01:50:31.299 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T01:50:31.299 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 219.99047911 2024-04-26T01:50:31.299 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T01:50:31.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.298+0000 7fd7db800700 1 -- 172.21.15.1:0/996033481 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fd7bc0a3cc0 msgr2=0x7fd7bc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:31.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.298+0000 7fd7db800700 1 --2- 172.21.15.1:0/996033481 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fd7bc0a3cc0 0x7fd7bc0a6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0046f0 tx=0x7fd7cc003d30 comp rx=0 tx=0).stop 2024-04-26T01:50:31.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.298+0000 7fd7db800700 1 -- 172.21.15.1:0/996033481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7d4077890 msgr2=0x7fd7d41a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:31.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.298+0000 7fd7db800700 1 --2- 172.21.15.1:0/996033481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7d4077890 0x7fd7d41a5970 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0000c00 tx=0x7fd7c00057c0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.298+0000 7fd7db800700 1 -- 172.21.15.1:0/996033481 shutdown_connections 2024-04-26T01:50:31.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.298+0000 7fd7db800700 1 --2- 172.21.15.1:0/996033481 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fd7bc0a3cc0 0x7fd7bc0a6180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.298+0000 7fd7db800700 1 --2- 172.21.15.1:0/996033481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7d41138f0 0x7fd7d419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.298+0000 7fd7db800700 1 --2- 172.21.15.1:0/996033481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7d4076f70 0x7fd7d41a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.298+0000 7fd7db800700 1 --2- 172.21.15.1:0/996033481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7d4077890 0x7fd7d41a5970 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:31.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.298+0000 7fd7db800700 1 -- 172.21.15.1:0/996033481 >> 172.21.15.1:0/996033481 conn(0x7fd7d4100780 msgr2=0x7fd7d4102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:31.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.298+0000 7fd7db800700 1 -- 172.21.15.1:0/996033481 shutdown_connections 2024-04-26T01:50:31.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:31.298+0000 7fd7db800700 1 -- 172.21.15.1:0/996033481 wait complete. 2024-04-26T01:50:32.213 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-26T01:50:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:32.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:32 smithi001 ceph-mon[27822]: pgmap v417: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:32.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:32 smithi097 ceph-mon[32853]: pgmap v417: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:33.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:32 smithi173 ceph-mon[32553]: pgmap v417: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:34.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:34 smithi001 ceph-mon[27822]: pgmap v418: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:34.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.956+0000 7fa149200700 1 -- 172.21.15.1:0/2927758454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa144109c80 msgr2=0x7fa14410dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:34.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.956+0000 7fa149200700 1 --2- 172.21.15.1:0/2927758454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa144109c80 0x7fa14410dcd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa1300098f0 tx=0x7fa130009c00 comp rx=0 tx=0).stop 2024-04-26T01:50:34.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.956+0000 7fa149200700 1 -- 172.21.15.1:0/2927758454 shutdown_connections 2024-04-26T01:50:34.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.956+0000 7fa149200700 1 --2- 172.21.15.1:0/2927758454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa14410e3b0 0x7fa144115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:34.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.956+0000 7fa149200700 1 --2- 172.21.15.1:0/2927758454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1441092d0 0x7fa1441096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:34.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.956+0000 7fa149200700 1 --2- 172.21.15.1:0/2927758454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa144109c80 0x7fa14410dcd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:34.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.956+0000 7fa149200700 1 -- 172.21.15.1:0/2927758454 >> 172.21.15.1:0/2927758454 conn(0x7fa144076210 msgr2=0x7fa144107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:34.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.956+0000 7fa149200700 1 -- 172.21.15.1:0/2927758454 shutdown_connections 2024-04-26T01:50:34.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.957+0000 7fa149200700 1 -- 172.21.15.1:0/2927758454 wait complete. 2024-04-26T01:50:34.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.958+0000 7fa149200700 1 Processor -- start 2024-04-26T01:50:34.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.958+0000 7fa149200700 1 -- start start 2024-04-26T01:50:34.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.959+0000 7fa149200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1441092d0 0x7fa14419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:34.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.959+0000 7fa149200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa144109c80 0x7fa14419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:34.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.959+0000 7fa149200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa14410e3b0 0x7fa1441a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:34.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.959+0000 7fa149200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa144119240 con 0x7fa1441092d0 2024-04-26T01:50:34.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.959+0000 7fa149200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa1441190c0 con 0x7fa144109c80 2024-04-26T01:50:34.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.959+0000 7fa149200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa1441193c0 con 0x7fa14410e3b0 2024-04-26T01:50:34.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.959+0000 7fa141e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1441092d0 0x7fa14419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:34.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.959+0000 7fa141e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1441092d0 0x7fa14419f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54998/0 (socket says 172.21.15.1:54998) 2024-04-26T01:50:34.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.959+0000 7fa141e00700 1 -- 172.21.15.1:0/784750361 learned_addr learned my addr 172.21.15.1:0/784750361 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:34.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.959+0000 7fa141400700 1 --2- 172.21.15.1:0/784750361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa144109c80 0x7fa14419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:34.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.959+0000 7fa142800700 1 --2- 172.21.15.1:0/784750361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa14410e3b0 0x7fa1441a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:34.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.960+0000 7fa141400700 1 -- 172.21.15.1:0/784750361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa14410e3b0 msgr2=0x7fa1441a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:34.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.960+0000 7fa141400700 1 --2- 172.21.15.1:0/784750361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa14410e3b0 0x7fa1441a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:34.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.960+0000 7fa141400700 1 -- 172.21.15.1:0/784750361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1441092d0 msgr2=0x7fa14419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:34.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.960+0000 7fa141400700 1 --2- 172.21.15.1:0/784750361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1441092d0 0x7fa14419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:34.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.960+0000 7fa141400700 1 -- 172.21.15.1:0/784750361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1441a4760 con 0x7fa144109c80 2024-04-26T01:50:34.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.960+0000 7fa141e00700 1 --2- 172.21.15.1:0/784750361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1441092d0 0x7fa14419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:34.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.960+0000 7fa141400700 1 --2- 172.21.15.1:0/784750361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa144109c80 0x7fa14419fc90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa130000c00 tx=0x7fa13000fa20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:34.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.960+0000 7fa13aa00700 1 -- 172.21.15.1:0/784750361 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130005040 con 0x7fa144109c80 2024-04-26T01:50:34.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.960+0000 7fa149200700 1 -- 172.21.15.1:0/784750361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1441a49f0 con 0x7fa144109c80 2024-04-26T01:50:34.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.960+0000 7fa13aa00700 1 -- 172.21.15.1:0/784750361 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa130005690 con 0x7fa144109c80 2024-04-26T01:50:34.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.960+0000 7fa13aa00700 1 -- 172.21.15.1:0/784750361 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1300038e0 con 0x7fa144109c80 2024-04-26T01:50:34.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.960+0000 7fa149200700 1 -- 172.21.15.1:0/784750361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1441ac2d0 con 0x7fa144109c80 2024-04-26T01:50:34.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.964+0000 7fa13aa00700 1 -- 172.21.15.1:0/784750361 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fa1300051e0 con 0x7fa144109c80 2024-04-26T01:50:34.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.965+0000 7fa153b9db80 1 Processor -- start 2024-04-26T01:50:34.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.965+0000 7fa13aa00700 1 --2- 172.21.15.1:0/784750361 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa1280a3c40 0x7fa1280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:34.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.965+0000 7fa13aa00700 1 -- 172.21.15.1:0/784750361 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fa1300d0bc0 con 0x7fa144109c80 2024-04-26T01:50:34.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.965+0000 7fa153b9db80 1 -- start start 2024-04-26T01:50:34.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.966+0000 7fa141e00700 1 --2- 172.21.15.1:0/784750361 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa1280a3c40 0x7fa1280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:34.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.966+0000 7fa153b9db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5beb2f0 0x5605a5beb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:34.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.966+0000 7fa153b9db80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5bebc10 0x5605a5bf6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:34.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.966+0000 7fa153b9db80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bf6ab0 0x5605a5bf8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:34.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.966+0000 7fa153b9db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5605a5b3ee40 con 0x5605a5beb2f0 2024-04-26T01:50:34.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.966+0000 7fa153b9db80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5605a5ae56e0 con 0x5605a5bf6ab0 2024-04-26T01:50:34.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.966+0000 7fa153b9db80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5605a5b3ae00 con 0x5605a5bebc10 2024-04-26T01:50:34.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.966+0000 7fa141400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5bebc10 0x5605a5bf6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:34.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.966+0000 7fa141e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5beb2f0 0x5605a5beb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:34.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.966+0000 7fa141400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5bebc10 0x5605a5bf6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:36658/0 (socket says 172.21.15.1:36658) 2024-04-26T01:50:34.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.966+0000 7fa141400700 1 -- 172.21.15.1:0/2342595586 learned_addr learned my addr 172.21.15.1:0/2342595586 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:34.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.967+0000 7fa142800700 1 --2- 172.21.15.1:0/2342595586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bf6ab0 0x5605a5bf8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:34.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.967+0000 7fa141e00700 1 --2- 172.21.15.1:0/784750361 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa1280a3c40 0x7fa1280a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa134009750 tx=0x7fa134011a50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:34.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.967+0000 7fa141400700 1 -- 172.21.15.1:0/2342595586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bf6ab0 msgr2=0x5605a5bf8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:34.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.967+0000 7fa141400700 1 --2- 172.21.15.1:0/2342595586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bf6ab0 0x5605a5bf8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:34.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.967+0000 7fa141400700 1 -- 172.21.15.1:0/2342595586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5beb2f0 msgr2=0x5605a5beb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:34.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.967+0000 7fa141400700 1 --2- 172.21.15.1:0/2342595586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5beb2f0 0x5605a5beb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:34.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.967+0000 7fa141400700 1 -- 172.21.15.1:0/2342595586 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605a5adb630 con 0x5605a5bebc10 2024-04-26T01:50:34.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.967+0000 7fa141400700 1 --2- 172.21.15.1:0/2342595586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5bebc10 0x5605a5bf6570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa130099310 tx=0x7fa130005550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=276a2995b7f83171 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:34.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.967+0000 7fa12f400700 1 -- 172.21.15.1:0/2342595586 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1300a1260 con 0x5605a5bebc10 2024-04-26T01:50:34.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.967+0000 7fa12f400700 1 -- 172.21.15.1:0/2342595586 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1300a5a90 con 0x5605a5bebc10 2024-04-26T01:50:34.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.967+0000 7fa12f400700 1 -- 172.21.15.1:0/2342595586 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13009d370 con 0x5605a5bebc10 2024-04-26T01:50:34.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.968+0000 7fa153b9db80 1 -- 172.21.15.1:0/2342595586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5bebc10 msgr2=0x5605a5bf6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:34.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.968+0000 7fa153b9db80 1 --2- 172.21.15.1:0/2342595586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5bebc10 0x5605a5bf6570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa130099310 tx=0x7fa130005550 comp rx=0 tx=0).stop 2024-04-26T01:50:34.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.968+0000 7fa153b9db80 1 -- 172.21.15.1:0/2342595586 shutdown_connections 2024-04-26T01:50:34.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.968+0000 7fa153b9db80 1 --2- 172.21.15.1:0/2342595586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bf6ab0 0x5605a5bf8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:34.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.968+0000 7fa153b9db80 1 --2- 172.21.15.1:0/2342595586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5beb2f0 0x5605a5beb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:34.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.968+0000 7fa153b9db80 1 --2- 172.21.15.1:0/2342595586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5bebc10 0x5605a5bf6570 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:34.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.968+0000 7fa153b9db80 1 -- 172.21.15.1:0/2342595586 >> 172.21.15.1:0/2342595586 conn(0x5605a5b1a550 msgr2=0x5605a5be7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:34.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.968+0000 7fa153b9db80 1 -- 172.21.15.1:0/2342595586 shutdown_connections 2024-04-26T01:50:34.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.968+0000 7fa153b9db80 1 -- 172.21.15.1:0/2342595586 wait complete. 2024-04-26T01:50:34.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.969+0000 7fa153b9db80 1 -- start start 2024-04-26T01:50:34.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.969+0000 7fa153b9db80 1 Processor -- start 2024-04-26T01:50:34.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.970+0000 7fa153b9db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5beb2f0 0x5605a5bf6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:34.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.970+0000 7fa153b9db80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5bebc10 0x5605a5bf3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:34.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.970+0000 7fa153b9db80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bf6ab0 0x5605a5bf3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:34.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.970+0000 7fa153b9db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5605a5bf93e0 con 0x5605a5beb2f0 2024-04-26T01:50:34.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.970+0000 7fa153b9db80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5605a5af8810 con 0x5605a5bf6ab0 2024-04-26T01:50:34.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.970+0000 7fa153b9db80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5605a5bf9560 con 0x5605a5bebc10 2024-04-26T01:50:34.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.970+0000 7fa153b9db80 10 client.0 _ll_register_callbacks cb 0x5605a5ad76d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:50:34.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.970+0000 7fa141e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5beb2f0 0x5605a5bf6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:34.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.970+0000 7fa141e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5beb2f0 0x5605a5bf6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55010/0 (socket says 172.21.15.1:55010) 2024-04-26T01:50:34.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.970+0000 7fa141e00700 1 -- 172.21.15.1:0/521742787 learned_addr learned my addr 172.21.15.1:0/521742787 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:34.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.970+0000 7fa142800700 1 --2- 172.21.15.1:0/521742787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bf6ab0 0x5605a5bf3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:34.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.970+0000 7fa141400700 1 --2- 172.21.15.1:0/521742787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5bebc10 0x5605a5bf3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:34.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.971+0000 7fa141e00700 1 -- 172.21.15.1:0/521742787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5bebc10 msgr2=0x5605a5bf3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:34.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.971+0000 7fa141e00700 1 --2- 172.21.15.1:0/521742787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5bebc10 0x5605a5bf3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:34.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.971+0000 7fa141e00700 1 -- 172.21.15.1:0/521742787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bf6ab0 msgr2=0x5605a5bf3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:34.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.971+0000 7fa141e00700 1 --2- 172.21.15.1:0/521742787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bf6ab0 0x5605a5bf3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:34.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.971+0000 7fa141e00700 1 -- 172.21.15.1:0/521742787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605a5a8c880 con 0x5605a5beb2f0 2024-04-26T01:50:34.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.971+0000 7fa141400700 1 --2- 172.21.15.1:0/521742787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5bebc10 0x5605a5bf3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:34.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.971+0000 7fa141e00700 1 --2- 172.21.15.1:0/521742787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5beb2f0 0x5605a5bf6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa13401eaf0 tx=0x7fa1340047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c82203754259fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:34.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.971+0000 7fa123e00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:50:34.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.971+0000 7fa123e00700 1 -- 172.21.15.1:0/521742787 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa134017040 con 0x5605a5beb2f0 2024-04-26T01:50:34.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.971+0000 7fa123e00700 1 -- 172.21.15.1:0/521742787 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fa13400fb00 con 0x5605a5beb2f0 2024-04-26T01:50:34.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.971+0000 7fa153b9db80 1 -- 172.21.15.1:0/521742787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1300a5a90 con 0x5605a5beb2f0 2024-04-26T01:50:34.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.972+0000 7fa123e00700 1 -- 172.21.15.1:0/521742787 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa13400fd00 con 0x5605a5beb2f0 2024-04-26T01:50:34.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.972+0000 7fa153b9db80 10 client.14799 fetch_fsmap learned FSMap version 25 2024-04-26T01:50:34.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.972+0000 7fa153b9db80 1 -- 172.21.15.1:0/521742787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5605a5ad5aa0 con 0x5605a5beb2f0 2024-04-26T01:50:34.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.972+0000 7fa123e00700 1 -- 172.21.15.1:0/521742787 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1300a5a90 con 0x5605a5beb2f0 2024-04-26T01:50:34.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.972+0000 7fa123e00700 1 -- 172.21.15.1:0/521742787 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fa134025030 con 0x5605a5beb2f0 2024-04-26T01:50:34.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.972+0000 7fa153b9db80 10 client.14799 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:50:34.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.972+0000 7fa153b9db80 10 client.14799 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T01:50:34.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.972+0000 7fa153b9db80 20 client.14799 populate_metadata read hostname 'smithi001' 2024-04-26T01:50:34.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.972+0000 7fa153b9db80 1 --2- 172.21.15.1:0/521742787 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x5605a5c061a0 0x5605a5c07610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:34.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.972+0000 7fa153b9db80 4 client.14799 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:50:34.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.972+0000 7fa153b9db80 1 -- 172.21.15.1:0/521742787 --> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605a5b4c630 con 0x5605a5c061a0 2024-04-26T01:50:34.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.973+0000 7fa141400700 1 --2- 172.21.15.1:0/521742787 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x5605a5c061a0 0x5605a5c07610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:34.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.973+0000 7fa141400700 1 --2- 172.21.15.1:0/521742787 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x5605a5c061a0 0x5605a5c07610 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea82fbfe50d3941b server_cookie=c229c8bf01d11bdd in_seq=0 out_seq=0 2024-04-26T01:50:34.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.973+0000 7fa123e00700 10 client.14799 ms_handle_connect on v2:172.21.15.173:6838/373692430 2024-04-26T01:50:34.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.974+0000 7fa123e00700 1 -- 172.21.15.1:0/521742787 <== mds.1 v2:172.21.15.173:6838/373692430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5605a5b4c630 con 0x5605a5c061a0 2024-04-26T01:50:34.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.974+0000 7fa123e00700 10 client.14799 handle_command_reply: tid=0 2024-04-26T01:50:34.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.974+0000 7fa153b9db80 1 client.14799 shutdown 2024-04-26T01:50:34.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.974+0000 7fa153b9db80 1 -- 172.21.15.1:0/521742787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5beb2f0 msgr2=0x5605a5bf6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:34.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.974+0000 7fa153b9db80 1 --2- 172.21.15.1:0/521742787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5beb2f0 0x5605a5bf6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa13401eaf0 tx=0x7fa1340047c0 comp rx=0 tx=0).stop 2024-04-26T01:50:34.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.975+0000 7fa153b9db80 1 -- 172.21.15.1:0/521742787 shutdown_connections 2024-04-26T01:50:34.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.975+0000 7fa153b9db80 1 --2- 172.21.15.1:0/521742787 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x5605a5c061a0 0x5605a5c07610 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:34.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.975+0000 7fa153b9db80 1 --2- 172.21.15.1:0/521742787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bf6ab0 0x5605a5bf3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:34.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.975+0000 7fa153b9db80 1 --2- 172.21.15.1:0/521742787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5beb2f0 0x5605a5bf6210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:34.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.975+0000 7fa153b9db80 1 --2- 172.21.15.1:0/521742787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5bebc10 0x5605a5bf3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:34.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.975+0000 7fa123e00700 0 client.14799 ms_handle_reset on v2:172.21.15.173:6838/373692430 2024-04-26T01:50:34.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.975+0000 7fa153b9db80 1 -- 172.21.15.1:0/521742787 >> 172.21.15.1:0/521742787 conn(0x5605a5b1a550 msgr2=0x5605a5be6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:34.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.975+0000 7fa153b9db80 1 -- 172.21.15.1:0/521742787 shutdown_connections 2024-04-26T01:50:34.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.975+0000 7fa153b9db80 1 -- 172.21.15.1:0/521742787 wait complete. 2024-04-26T01:50:34.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.975+0000 7fa153b9db80 20 client.14799 trim_cache size 0 max 16384 2024-04-26T01:50:34.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:34 smithi097 ceph-mon[32853]: pgmap v418: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:34.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.998+0000 7fa153b9db80 1 Processor -- start 2024-04-26T01:50:34.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.998+0000 7fa153b9db80 1 -- start start 2024-04-26T01:50:34.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.998+0000 7fa153b9db80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5beb2f0 0x5605a5bf3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:34.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.998+0000 7fa153b9db80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bebc10 0x5605a5be70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:34.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.998+0000 7fa153b9db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5bf6ab0 0x5605a5be7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:34.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.998+0000 7fa153b9db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5605a5c05950 con 0x5605a5bf6ab0 2024-04-26T01:50:34.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.998+0000 7fa153b9db80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5605a5c06020 con 0x5605a5bebc10 2024-04-26T01:50:34.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.998+0000 7fa153b9db80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5605a5c066f0 con 0x5605a5beb2f0 2024-04-26T01:50:34.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.999+0000 7fa141e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5beb2f0 0x5605a5bf3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:34.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.999+0000 7fa141400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bebc10 0x5605a5be70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:34.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.999+0000 7fa142800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5bf6ab0 0x5605a5be7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:34.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.999+0000 7fa141e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5beb2f0 0x5605a5bf3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:36670/0 (socket says 172.21.15.1:36670) 2024-04-26T01:50:34.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.999+0000 7fa141e00700 1 -- 172.21.15.1:0/4023571955 learned_addr learned my addr 172.21.15.1:0/4023571955 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:34.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.999+0000 7fa141e00700 1 -- 172.21.15.1:0/4023571955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bebc10 msgr2=0x5605a5be70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:34.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.999+0000 7fa141e00700 1 --2- 172.21.15.1:0/4023571955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bebc10 0x5605a5be70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:34.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.999+0000 7fa141e00700 1 -- 172.21.15.1:0/4023571955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5bf6ab0 msgr2=0x5605a5be7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:34.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.999+0000 7fa141e00700 1 --2- 172.21.15.1:0/4023571955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5bf6ab0 0x5605a5be7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:34.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.999+0000 7fa141e00700 1 -- 172.21.15.1:0/4023571955 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605a5bee2d0 con 0x5605a5beb2f0 2024-04-26T01:50:34.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.999+0000 7fa141400700 1 --2- 172.21.15.1:0/4023571955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bebc10 0x5605a5be70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:34.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:34.999+0000 7fa141e00700 1 --2- 172.21.15.1:0/4023571955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5beb2f0 0x5605a5bf3a00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa13401dd50 tx=0x7fa13400fc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb4d1c0a19a74e72 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:34.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.000+0000 7fa12ea00700 1 -- 172.21.15.1:0/4023571955 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa134035070 con 0x5605a5beb2f0 2024-04-26T01:50:35.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.000+0000 7fa12ea00700 1 -- 172.21.15.1:0/4023571955 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa134019050 con 0x5605a5beb2f0 2024-04-26T01:50:35.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.000+0000 7fa12ea00700 1 -- 172.21.15.1:0/4023571955 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa134015030 con 0x5605a5beb2f0 2024-04-26T01:50:35.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.000+0000 7fa153b9db80 1 -- 172.21.15.1:0/4023571955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5beb2f0 msgr2=0x5605a5bf3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:35.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.000+0000 7fa153b9db80 1 --2- 172.21.15.1:0/4023571955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5beb2f0 0x5605a5bf3a00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa13401dd50 tx=0x7fa13400fc90 comp rx=0 tx=0).stop 2024-04-26T01:50:35.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.000+0000 7fa153b9db80 1 -- 172.21.15.1:0/4023571955 shutdown_connections 2024-04-26T01:50:35.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.000+0000 7fa153b9db80 1 --2- 172.21.15.1:0/4023571955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5bf6ab0 0x5605a5be7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:35.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.000+0000 7fa153b9db80 1 --2- 172.21.15.1:0/4023571955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5beb2f0 0x5605a5bf3a00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:35.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.000+0000 7fa153b9db80 1 --2- 172.21.15.1:0/4023571955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bebc10 0x5605a5be70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:35.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.000+0000 7fa153b9db80 1 -- 172.21.15.1:0/4023571955 >> 172.21.15.1:0/4023571955 conn(0x5605a5b1a550 msgr2=0x5605a5bf32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:35.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.001+0000 7fa153b9db80 1 -- 172.21.15.1:0/4023571955 shutdown_connections 2024-04-26T01:50:35.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.001+0000 7fa153b9db80 1 -- 172.21.15.1:0/4023571955 wait complete. 2024-04-26T01:50:35.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.001+0000 7fa153b9db80 1 -- start start 2024-04-26T01:50:35.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.001+0000 7fa153b9db80 1 Processor -- start 2024-04-26T01:50:35.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.002+0000 7fa153b9db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5beb2f0 0x5605a5c10fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:35.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.002+0000 7fa153b9db80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5bebc10 0x5605a5bf7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:35.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.002+0000 7fa153b9db80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bf6ab0 0x5605a5bf8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:35.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.002+0000 7fa153b9db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5605a5c07490 con 0x5605a5beb2f0 2024-04-26T01:50:35.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.002+0000 7fa153b9db80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5605a5c06dc0 con 0x5605a5bf6ab0 2024-04-26T01:50:35.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.002+0000 7fa153b9db80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5605a5c07b60 con 0x5605a5bebc10 2024-04-26T01:50:35.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.002+0000 7fa153b9db80 10 client.0 _ll_register_callbacks cb 0x5605a5ad76d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:50:35.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.002+0000 7fa141400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5bebc10 0x5605a5bf7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:35.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.002+0000 7fa141e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5beb2f0 0x5605a5c10fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:35.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.002+0000 7fa141400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5bebc10 0x5605a5bf7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:36672/0 (socket says 172.21.15.1:36672) 2024-04-26T01:50:35.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.002+0000 7fa141400700 1 -- 172.21.15.1:0/2455513296 learned_addr learned my addr 172.21.15.1:0/2455513296 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:35.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.002+0000 7fa142800700 1 --2- 172.21.15.1:0/2455513296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bf6ab0 0x5605a5bf8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:35.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.003+0000 7fa141400700 1 -- 172.21.15.1:0/2455513296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bf6ab0 msgr2=0x5605a5bf8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:35.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.003+0000 7fa141400700 1 --2- 172.21.15.1:0/2455513296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bf6ab0 0x5605a5bf8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:35.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.003+0000 7fa141400700 1 -- 172.21.15.1:0/2455513296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5beb2f0 msgr2=0x5605a5c10fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:35.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.003+0000 7fa141400700 1 --2- 172.21.15.1:0/2455513296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5beb2f0 0x5605a5c10fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:35.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.003+0000 7fa141400700 1 -- 172.21.15.1:0/2455513296 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605a5c0ca50 con 0x5605a5bebc10 2024-04-26T01:50:35.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.003+0000 7fa142800700 1 --2- 172.21.15.1:0/2455513296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bf6ab0 0x5605a5bf8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:50:35.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.003+0000 7fa141400700 1 --2- 172.21.15.1:0/2455513296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5bebc10 0x5605a5bf7a50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa13009cbb0 tx=0x7fa1300054d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=312440a1c18e4a89 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:35.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.003+0000 7fa123e00700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:50:35.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.003+0000 7fa123e00700 1 -- 172.21.15.1:0/2455513296 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1300b62b0 con 0x5605a5bebc10 2024-04-26T01:50:35.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.003+0000 7fa153b9db80 1 -- 172.21.15.1:0/2455513296 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa134019050 con 0x5605a5bebc10 2024-04-26T01:50:35.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.003+0000 7fa123e00700 1 -- 172.21.15.1:0/2455513296 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fa130009400 con 0x5605a5bebc10 2024-04-26T01:50:35.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.003+0000 7fa123e00700 1 -- 172.21.15.1:0/2455513296 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa13009d110 con 0x5605a5bebc10 2024-04-26T01:50:35.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.003+0000 7fa153b9db80 10 client.24937 fetch_fsmap learned FSMap version 25 2024-04-26T01:50:35.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.003+0000 7fa153b9db80 1 -- 172.21.15.1:0/2455513296 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5605a5c6d420 con 0x5605a5bebc10 2024-04-26T01:50:35.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.004+0000 7fa123e00700 1 -- 172.21.15.1:0/2455513296 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa134019050 con 0x5605a5bebc10 2024-04-26T01:50:35.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.004+0000 7fa123e00700 1 -- 172.21.15.1:0/2455513296 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fa1300a5600 con 0x5605a5bebc10 2024-04-26T01:50:35.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.004+0000 7fa153b9db80 10 client.24937 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:50:35.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.004+0000 7fa153b9db80 10 client.24937 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T01:50:35.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.004+0000 7fa153b9db80 20 client.24937 populate_metadata read hostname 'smithi001' 2024-04-26T01:50:35.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.004+0000 7fa153b9db80 1 --2- 172.21.15.1:0/2455513296 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x5605a5c71bc0 0x5605a5c708d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:35.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.004+0000 7fa153b9db80 4 client.24937 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T01:50:35.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.004+0000 7fa153b9db80 1 -- 172.21.15.1:0/2455513296 --> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] -- command(tid 0: {"prefix": "status"}) -- 0x5605a5b466d0 con 0x5605a5c71bc0 2024-04-26T01:50:35.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.004+0000 7fa141e00700 1 --2- 172.21.15.1:0/2455513296 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x5605a5c71bc0 0x5605a5c708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:35.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.005+0000 7fa141e00700 1 --2- 172.21.15.1:0/2455513296 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x5605a5c71bc0 0x5605a5c708d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d89dfc4175ea971 server_cookie=ff20d29e8b4816cc in_seq=0 out_seq=0 2024-04-26T01:50:35.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.005+0000 7fa123e00700 10 client.24937 ms_handle_connect on v2:172.21.15.173:6838/373692430 2024-04-26T01:50:35.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.005+0000 7fa123e00700 1 -- 172.21.15.1:0/2455513296 <== mds.1 v2:172.21.15.173:6838/373692430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5605a5b466d0 con 0x5605a5c71bc0 2024-04-26T01:50:35.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.005+0000 7fa123e00700 10 client.24937 handle_command_reply: tid=0 2024-04-26T01:50:35.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.005+0000 7fa153b9db80 1 client.24937 shutdown 2024-04-26T01:50:35.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.005+0000 7fa153b9db80 1 -- 172.21.15.1:0/2455513296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5bebc10 msgr2=0x5605a5bf7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:35.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.005+0000 7fa153b9db80 1 --2- 172.21.15.1:0/2455513296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5bebc10 0x5605a5bf7a50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa13009cbb0 tx=0x7fa1300054d0 comp rx=0 tx=0).stop 2024-04-26T01:50:35.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.005+0000 7fa153b9db80 1 -- 172.21.15.1:0/2455513296 shutdown_connections 2024-04-26T01:50:35.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.005+0000 7fa153b9db80 1 --2- 172.21.15.1:0/2455513296 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x5605a5c71bc0 0x5605a5c708d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:35.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.005+0000 7fa153b9db80 1 --2- 172.21.15.1:0/2455513296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605a5bf6ab0 0x5605a5bf8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:35.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.005+0000 7fa153b9db80 1 --2- 172.21.15.1:0/2455513296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605a5beb2f0 0x5605a5c10fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:35.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.005+0000 7fa153b9db80 1 --2- 172.21.15.1:0/2455513296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5605a5bebc10 0x5605a5bf7a50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:35.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.005+0000 7fa123e00700 0 client.24937 ms_handle_reset on v2:172.21.15.173:6838/373692430 2024-04-26T01:50:35.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.005+0000 7fa153b9db80 1 -- 172.21.15.1:0/2455513296 >> 172.21.15.1:0/2455513296 conn(0x5605a5b1a550 msgr2=0x5605a5bee4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:35.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.006+0000 7fa153b9db80 1 -- 172.21.15.1:0/2455513296 shutdown_connections 2024-04-26T01:50:35.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.006+0000 7fa153b9db80 1 -- 172.21.15.1:0/2455513296 wait complete. 2024-04-26T01:50:35.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.006+0000 7fa153b9db80 20 client.24937 trim_cache size 0 max 16384 2024-04-26T01:50:35.007 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T01:50:35.008 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "77cfcc80-036d-11ef-bc93-c7b262605968", 2024-04-26T01:50:35.008 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 1, 2024-04-26T01:50:35.008 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24514, 2024-04-26T01:50:35.008 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-04-26T01:50:35.008 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-04-26T01:50:35.008 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-04-26T01:50:35.008 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 168.80441618099999, 2024-04-26T01:50:35.008 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 25, 2024-04-26T01:50:35.008 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 92, 2024-04-26T01:50:35.008 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T01:50:35.008 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 218.14610077899999 2024-04-26T01:50:35.008 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T01:50:35.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.006+0000 7fa149200700 1 -- 172.21.15.1:0/784750361 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa1280a3c40 msgr2=0x7fa1280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:35.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.006+0000 7fa149200700 1 --2- 172.21.15.1:0/784750361 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa1280a3c40 0x7fa1280a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa134009750 tx=0x7fa134011a50 comp rx=0 tx=0).stop 2024-04-26T01:50:35.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.006+0000 7fa149200700 1 -- 172.21.15.1:0/784750361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa144109c80 msgr2=0x7fa14419fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:35.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.006+0000 7fa149200700 1 --2- 172.21.15.1:0/784750361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa144109c80 0x7fa14419fc90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa130000c00 tx=0x7fa13000fa20 comp rx=0 tx=0).stop 2024-04-26T01:50:35.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.007+0000 7fa149200700 1 -- 172.21.15.1:0/784750361 shutdown_connections 2024-04-26T01:50:35.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.007+0000 7fa149200700 1 --2- 172.21.15.1:0/784750361 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fa1280a3c40 0x7fa1280a6100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:35.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.007+0000 7fa149200700 1 --2- 172.21.15.1:0/784750361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa14410e3b0 0x7fa1441a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:35.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.007+0000 7fa149200700 1 --2- 172.21.15.1:0/784750361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1441092d0 0x7fa14419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:35.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.007+0000 7fa149200700 1 --2- 172.21.15.1:0/784750361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa144109c80 0x7fa14419fc90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:35.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.007+0000 7fa149200700 1 -- 172.21.15.1:0/784750361 >> 172.21.15.1:0/784750361 conn(0x7fa144076210 msgr2=0x7fa1441086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:35.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.007+0000 7fa149200700 1 -- 172.21.15.1:0/784750361 shutdown_connections 2024-04-26T01:50:35.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:35.007+0000 7fa149200700 1 -- 172.21.15.1:0/784750361 wait complete. 2024-04-26T01:50:35.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:34 smithi173 ceph-mon[32553]: pgmap v418: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:35.981 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-26T01:50:36.470 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:36.858 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:36 smithi001 ceph-mon[27822]: pgmap v419: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:36.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:36 smithi097 ceph-mon[32853]: pgmap v419: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:37.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:36 smithi173 ceph-mon[32553]: pgmap v419: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:38.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.651+0000 7fb887800700 1 -- 172.21.15.1:0/1457246882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb880105860 msgr2=0x7fb8801098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:38.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.651+0000 7fb887800700 1 --2- 172.21.15.1:0/1457246882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb880105860 0x7fb8801098b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0099c0 tx=0x7fb86c009cd0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.652+0000 7fb887800700 1 -- 172.21.15.1:0/1457246882 shutdown_connections 2024-04-26T01:50:38.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.652+0000 7fb887800700 1 --2- 172.21.15.1:0/1457246882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb880109f90 0x7fb880111b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.652+0000 7fb887800700 1 --2- 172.21.15.1:0/1457246882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb880104eb0 0x7fb880105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.652+0000 7fb887800700 1 --2- 172.21.15.1:0/1457246882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb880105860 0x7fb8801098b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.652+0000 7fb887800700 1 -- 172.21.15.1:0/1457246882 >> 172.21.15.1:0/1457246882 conn(0x7fb880100740 msgr2=0x7fb880102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:38.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.652+0000 7fb887800700 1 -- 172.21.15.1:0/1457246882 shutdown_connections 2024-04-26T01:50:38.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.652+0000 7fb887800700 1 -- 172.21.15.1:0/1457246882 wait complete. 2024-04-26T01:50:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.653+0000 7fb887800700 1 Processor -- start 2024-04-26T01:50:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.653+0000 7fb887800700 1 -- start start 2024-04-26T01:50:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.654+0000 7fb887800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb880104eb0 0x7fb88019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.654+0000 7fb887800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb880105860 0x7fb88019b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.654+0000 7fb887800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb880109f90 0x7fb88019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.654+0000 7fb887800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb880114e50 con 0x7fb880105860 2024-04-26T01:50:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.654+0000 7fb887800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb880114cd0 con 0x7fb880104eb0 2024-04-26T01:50:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.654+0000 7fb887800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb880114fd0 con 0x7fb880109f90 2024-04-26T01:50:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.654+0000 7fb87fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb880105860 0x7fb88019b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.654+0000 7fb884e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb880104eb0 0x7fb88019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.654+0000 7fb87fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb880105860 0x7fb88019b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55038/0 (socket says 172.21.15.1:55038) 2024-04-26T01:50:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.654+0000 7fb884e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb880104eb0 0x7fb88019b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:43974/0 (socket says 172.21.15.1:43974) 2024-04-26T01:50:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.654+0000 7fb87fe00700 1 -- 172.21.15.1:0/2652709990 learned_addr learned my addr 172.21.15.1:0/2652709990 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:38.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.654+0000 7fb885800700 1 --2- 172.21.15.1:0/2652709990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb880109f90 0x7fb88019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:38.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.654+0000 7fb884e00700 1 -- 172.21.15.1:0/2652709990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb880109f90 msgr2=0x7fb88019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:38.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.654+0000 7fb884e00700 1 --2- 172.21.15.1:0/2652709990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb880109f90 0x7fb88019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.655+0000 7fb884e00700 1 -- 172.21.15.1:0/2652709990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb880105860 msgr2=0x7fb88019b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:38.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.655+0000 7fb884e00700 1 --2- 172.21.15.1:0/2652709990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb880105860 0x7fb88019b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.655+0000 7fb884e00700 1 -- 172.21.15.1:0/2652709990 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8801a0390 con 0x7fb880104eb0 2024-04-26T01:50:38.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.655+0000 7fb87fe00700 1 --2- 172.21.15.1:0/2652709990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb880105860 0x7fb88019b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:38.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.655+0000 7fb884e00700 1 --2- 172.21.15.1:0/2652709990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb880104eb0 0x7fb88019b380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb87400e9f0 tx=0x7fb87400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:38.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.655+0000 7fb87d600700 1 -- 172.21.15.1:0/2652709990 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87400cb70 con 0x7fb880104eb0 2024-04-26T01:50:38.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.655+0000 7fb887800700 1 -- 172.21.15.1:0/2652709990 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8801a0680 con 0x7fb880104eb0 2024-04-26T01:50:38.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.655+0000 7fb87d600700 1 -- 172.21.15.1:0/2652709990 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb87400ee90 con 0x7fb880104eb0 2024-04-26T01:50:38.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.655+0000 7fb87d600700 1 -- 172.21.15.1:0/2652709990 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb874003d40 con 0x7fb880104eb0 2024-04-26T01:50:38.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.655+0000 7fb887800700 1 -- 172.21.15.1:0/2652709990 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8801a7f00 con 0x7fb880104eb0 2024-04-26T01:50:38.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.658+0000 7fb87d600700 1 -- 172.21.15.1:0/2652709990 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fb874003ee0 con 0x7fb880104eb0 2024-04-26T01:50:38.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.660+0000 7fb87d600700 1 --2- 172.21.15.1:0/2652709990 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb8680a3c40 0x7fb8680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:38.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.660+0000 7fb87d600700 1 -- 172.21.15.1:0/2652709990 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fb8740d0530 con 0x7fb880104eb0 2024-04-26T01:50:38.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.660+0000 7fb87fe00700 1 --2- 172.21.15.1:0/2652709990 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb8680a3c40 0x7fb8680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:38.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.660+0000 7fb89218bb80 1 Processor -- start 2024-04-26T01:50:38.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.661+0000 7fb89218bb80 1 -- start start 2024-04-26T01:50:38.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.661+0000 7fb87fe00700 1 --2- 172.21.15.1:0/2652709990 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb8680a3c40 0x7fb8680a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb86c003760 tx=0x7fb86c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:38.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.661+0000 7fb89218bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f370e780 0x55a8f370eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:38.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.661+0000 7fb89218bb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8f370f0a0 0x55a8f37089b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:38.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.661+0000 7fb89218bb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f3708ef0 0x55a8f3709330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:38.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.661+0000 7fb89218bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a8f365ae40 con 0x55a8f370e780 2024-04-26T01:50:38.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.661+0000 7fb89218bb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a8f36016e0 con 0x55a8f3708ef0 2024-04-26T01:50:38.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.661+0000 7fb89218bb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a8f3656e00 con 0x55a8f370f0a0 2024-04-26T01:50:38.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.661+0000 7fb884e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f370e780 0x55a8f370eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:38.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.661+0000 7fb884e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f370e780 0x55a8f370eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55052/0 (socket says 172.21.15.1:55052) 2024-04-26T01:50:38.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.661+0000 7fb884e00700 1 -- 172.21.15.1:0/743470916 learned_addr learned my addr 172.21.15.1:0/743470916 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:38.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.661+0000 7fb87fe00700 1 --2- 172.21.15.1:0/743470916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8f370f0a0 0x55a8f37089b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:38.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.661+0000 7fb885800700 1 --2- 172.21.15.1:0/743470916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f3708ef0 0x55a8f3709330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:38.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.662+0000 7fb884e00700 1 -- 172.21.15.1:0/743470916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8f370f0a0 msgr2=0x55a8f37089b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:38.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.662+0000 7fb884e00700 1 --2- 172.21.15.1:0/743470916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8f370f0a0 0x55a8f37089b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.662+0000 7fb884e00700 1 -- 172.21.15.1:0/743470916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f3708ef0 msgr2=0x55a8f3709330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:38.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.662+0000 7fb884e00700 1 --2- 172.21.15.1:0/743470916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f3708ef0 0x55a8f3709330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.662+0000 7fb884e00700 1 -- 172.21.15.1:0/743470916 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8f35f7630 con 0x55a8f370e780 2024-04-26T01:50:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.662+0000 7fb884e00700 1 --2- 172.21.15.1:0/743470916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f370e780 0x55a8f370eb60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb874005cb0 tx=0x7fb87409c1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39adebccabf1d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.662+0000 7fb872000700 1 -- 172.21.15.1:0/743470916 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8740a0b40 con 0x55a8f370e780 2024-04-26T01:50:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.662+0000 7fb872000700 1 -- 172.21.15.1:0/743470916 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8740a5400 con 0x55a8f370e780 2024-04-26T01:50:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.662+0000 7fb872000700 1 -- 172.21.15.1:0/743470916 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8740d00f0 con 0x55a8f370e780 2024-04-26T01:50:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.663+0000 7fb89218bb80 1 -- 172.21.15.1:0/743470916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f370e780 msgr2=0x55a8f370eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.663+0000 7fb89218bb80 1 --2- 172.21.15.1:0/743470916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f370e780 0x55a8f370eb60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb874005cb0 tx=0x7fb87409c1c0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.663+0000 7fb89218bb80 1 -- 172.21.15.1:0/743470916 shutdown_connections 2024-04-26T01:50:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.663+0000 7fb89218bb80 1 --2- 172.21.15.1:0/743470916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f3708ef0 0x55a8f3709330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.663+0000 7fb89218bb80 1 --2- 172.21.15.1:0/743470916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f370e780 0x55a8f370eb60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.663+0000 7fb89218bb80 1 --2- 172.21.15.1:0/743470916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8f370f0a0 0x55a8f37089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.663+0000 7fb89218bb80 1 -- 172.21.15.1:0/743470916 >> 172.21.15.1:0/743470916 conn(0x55a8f3636f00 msgr2=0x55a8f37038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.663+0000 7fb89218bb80 1 -- 172.21.15.1:0/743470916 shutdown_connections 2024-04-26T01:50:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.663+0000 7fb89218bb80 1 -- 172.21.15.1:0/743470916 wait complete. 2024-04-26T01:50:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.664+0000 7fb89218bb80 1 -- start start 2024-04-26T01:50:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.664+0000 7fb89218bb80 1 Processor -- start 2024-04-26T01:50:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.665+0000 7fb89218bb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8f3708ef0 0x55a8f3714d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.665+0000 7fb89218bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f370e780 0x55a8f37120a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.665+0000 7fb89218bb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f37125e0 0x55a8f3712a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.665+0000 7fb89218bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a8f3715920 con 0x55a8f370e780 2024-04-26T01:50:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.665+0000 7fb89218bb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a8f3614810 con 0x55a8f37125e0 2024-04-26T01:50:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.665+0000 7fb89218bb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a8f3715aa0 con 0x55a8f3708ef0 2024-04-26T01:50:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.665+0000 7fb89218bb80 10 client.0 _ll_register_callbacks cb 0x55a8f35f36d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:50:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.665+0000 7fb87fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f370e780 0x55a8f37120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.665+0000 7fb87fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f370e780 0x55a8f37120a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55068/0 (socket says 172.21.15.1:55068) 2024-04-26T01:50:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.665+0000 7fb87fe00700 1 -- 172.21.15.1:0/1059105759 learned_addr learned my addr 172.21.15.1:0/1059105759 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.665+0000 7fb884e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8f3708ef0 0x55a8f3714d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.665+0000 7fb885800700 1 --2- 172.21.15.1:0/1059105759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f37125e0 0x55a8f3712a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.665+0000 7fb87fe00700 1 -- 172.21.15.1:0/1059105759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8f3708ef0 msgr2=0x55a8f3714d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.665+0000 7fb87fe00700 1 --2- 172.21.15.1:0/1059105759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8f3708ef0 0x55a8f3714d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.665+0000 7fb87fe00700 1 -- 172.21.15.1:0/1059105759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f37125e0 msgr2=0x55a8f3712a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.665+0000 7fb87fe00700 1 --2- 172.21.15.1:0/1059105759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f37125e0 0x55a8f3712a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.665+0000 7fb87fe00700 1 -- 172.21.15.1:0/1059105759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8f35a8880 con 0x55a8f370e780 2024-04-26T01:50:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.665+0000 7fb885800700 1 --2- 172.21.15.1:0/1059105759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f37125e0 0x55a8f3712a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.666+0000 7fb87fe00700 1 --2- 172.21.15.1:0/1059105759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f370e780 0x55a8f37120a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00b580 tx=0x7fb86c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d14b0a4fe5fde2d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.666+0000 7fb866a00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:50:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.666+0000 7fb866a00700 1 -- 172.21.15.1:0/1059105759 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c003e60 con 0x55a8f370e780 2024-04-26T01:50:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.666+0000 7fb866a00700 1 -- 172.21.15.1:0/1059105759 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fb86c023460 con 0x55a8f370e780 2024-04-26T01:50:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.666+0000 7fb89218bb80 1 -- 172.21.15.1:0/1059105759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8740a5400 con 0x55a8f370e780 2024-04-26T01:50:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.666+0000 7fb866a00700 1 -- 172.21.15.1:0/1059105759 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb86c023660 con 0x55a8f370e780 2024-04-26T01:50:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.667+0000 7fb89218bb80 10 client.14817 fetch_fsmap learned FSMap version 25 2024-04-26T01:50:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.667+0000 7fb89218bb80 1 -- 172.21.15.1:0/1059105759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a8f35f1aa0 con 0x55a8f370e780 2024-04-26T01:50:38.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.667+0000 7fb866a00700 1 -- 172.21.15.1:0/1059105759 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8740a5400 con 0x55a8f370e780 2024-04-26T01:50:38.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.667+0000 7fb866a00700 1 -- 172.21.15.1:0/1059105759 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fb86c038810 con 0x55a8f370e780 2024-04-26T01:50:38.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.667+0000 7fb89218bb80 10 client.14817 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:50:38.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.667+0000 7fb89218bb80 10 client.14817 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T01:50:38.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.667+0000 7fb89218bb80 20 client.14817 populate_metadata read hostname 'smithi001' 2024-04-26T01:50:38.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.667+0000 7fb89218bb80 1 --2- 172.21.15.1:0/1059105759 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a8f3722b50 0x55a8f3723fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:38.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.667+0000 7fb89218bb80 4 client.14817 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:50:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.667+0000 7fb89218bb80 1 -- 172.21.15.1:0/1059105759 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8f3668630 con 0x55a8f3722b50 2024-04-26T01:50:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.668+0000 7fb884e00700 1 --2- 172.21.15.1:0/1059105759 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a8f3722b50 0x55a8f3723fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.668+0000 7fb884e00700 1 --2- 172.21.15.1:0/1059105759 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a8f3722b50 0x55a8f3723fc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50828c76939e96ef server_cookie=9d0425da0293eb9e in_seq=0 out_seq=0 2024-04-26T01:50:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.668+0000 7fb866a00700 10 client.14817 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:50:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.669+0000 7fb866a00700 1 -- 172.21.15.1:0/1059105759 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a8f3668630 con 0x55a8f3722b50 2024-04-26T01:50:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.669+0000 7fb866a00700 10 client.14817 handle_command_reply: tid=0 2024-04-26T01:50:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.669+0000 7fb89218bb80 1 client.14817 shutdown 2024-04-26T01:50:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.670+0000 7fb89218bb80 1 -- 172.21.15.1:0/1059105759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f370e780 msgr2=0x55a8f37120a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.670+0000 7fb89218bb80 1 --2- 172.21.15.1:0/1059105759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f370e780 0x55a8f37120a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00b580 tx=0x7fb86c009cd0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.670+0000 7fb89218bb80 1 -- 172.21.15.1:0/1059105759 shutdown_connections 2024-04-26T01:50:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.670+0000 7fb89218bb80 1 --2- 172.21.15.1:0/1059105759 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a8f3722b50 0x55a8f3723fc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.670+0000 7fb89218bb80 1 --2- 172.21.15.1:0/1059105759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f37125e0 0x55a8f3712a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.670+0000 7fb89218bb80 1 --2- 172.21.15.1:0/1059105759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8f3708ef0 0x55a8f3714d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.670+0000 7fb89218bb80 1 --2- 172.21.15.1:0/1059105759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f370e780 0x55a8f37120a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.670+0000 7fb866a00700 0 client.14817 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:50:38.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.670+0000 7fb89218bb80 1 -- 172.21.15.1:0/1059105759 >> 172.21.15.1:0/1059105759 conn(0x55a8f3636f00 msgr2=0x55a8f3702e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:38.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.670+0000 7fb89218bb80 1 -- 172.21.15.1:0/1059105759 shutdown_connections 2024-04-26T01:50:38.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.670+0000 7fb89218bb80 1 -- 172.21.15.1:0/1059105759 wait complete. 2024-04-26T01:50:38.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.670+0000 7fb89218bb80 20 client.14817 trim_cache size 0 max 16384 2024-04-26T01:50:38.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.692+0000 7fb89218bb80 1 Processor -- start 2024-04-26T01:50:38.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.693+0000 7fb89218bb80 1 -- start start 2024-04-26T01:50:38.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.693+0000 7fb89218bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f3708ef0 0x55a8f37089b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:38.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.693+0000 7fb89218bb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8f370e780 0x55a8f37035d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:38.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.693+0000 7fb89218bb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f37125e0 0x55a8f372d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:38.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.693+0000 7fb89218bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a8f3722300 con 0x55a8f3708ef0 2024-04-26T01:50:38.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.693+0000 7fb89218bb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a8f37229d0 con 0x55a8f37125e0 2024-04-26T01:50:38.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.693+0000 7fb89218bb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a8f37230a0 con 0x55a8f370e780 2024-04-26T01:50:38.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.693+0000 7fb884e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f3708ef0 0x55a8f37089b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:38.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.693+0000 7fb884e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f3708ef0 0x55a8f37089b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55078/0 (socket says 172.21.15.1:55078) 2024-04-26T01:50:38.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.693+0000 7fb884e00700 1 -- 172.21.15.1:0/3247435286 learned_addr learned my addr 172.21.15.1:0/3247435286 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:38.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.693+0000 7fb87fe00700 1 --2- 172.21.15.1:0/3247435286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8f370e780 0x55a8f37035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:38.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.693+0000 7fb885800700 1 --2- 172.21.15.1:0/3247435286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f37125e0 0x55a8f372d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:38.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.694+0000 7fb884e00700 1 -- 172.21.15.1:0/3247435286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8f370e780 msgr2=0x55a8f37035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:38.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.694+0000 7fb884e00700 1 --2- 172.21.15.1:0/3247435286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8f370e780 0x55a8f37035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.694+0000 7fb884e00700 1 -- 172.21.15.1:0/3247435286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f37125e0 msgr2=0x55a8f372d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:38.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.694+0000 7fb884e00700 1 --2- 172.21.15.1:0/3247435286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f37125e0 0x55a8f372d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.694+0000 7fb884e00700 1 -- 172.21.15.1:0/3247435286 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8f370e3a0 con 0x55a8f3708ef0 2024-04-26T01:50:38.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.694+0000 7fb885800700 1 --2- 172.21.15.1:0/3247435286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f37125e0 0x55a8f372d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:50:38.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.694+0000 7fb884e00700 1 --2- 172.21.15.1:0/3247435286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f3708ef0 0x55a8f37089b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb874087350 tx=0x7fb87409c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1039986a2a0e05 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:38.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.694+0000 7fb871600700 1 -- 172.21.15.1:0/3247435286 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8740a5400 con 0x55a8f3708ef0 2024-04-26T01:50:38.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.694+0000 7fb871600700 1 -- 172.21.15.1:0/3247435286 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb87409aa90 con 0x55a8f3708ef0 2024-04-26T01:50:38.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.694+0000 7fb871600700 1 -- 172.21.15.1:0/3247435286 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87409c9f0 con 0x55a8f3708ef0 2024-04-26T01:50:38.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.695+0000 7fb89218bb80 1 -- 172.21.15.1:0/3247435286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f3708ef0 msgr2=0x55a8f37089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:38.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.695+0000 7fb89218bb80 1 --2- 172.21.15.1:0/3247435286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f3708ef0 0x55a8f37089b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb874087350 tx=0x7fb87409c910 comp rx=0 tx=0).stop 2024-04-26T01:50:38.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.695+0000 7fb89218bb80 1 -- 172.21.15.1:0/3247435286 shutdown_connections 2024-04-26T01:50:38.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.695+0000 7fb89218bb80 1 --2- 172.21.15.1:0/3247435286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f37125e0 0x55a8f372d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.695+0000 7fb89218bb80 1 --2- 172.21.15.1:0/3247435286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f3708ef0 0x55a8f37089b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.695+0000 7fb89218bb80 1 --2- 172.21.15.1:0/3247435286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8f370e780 0x55a8f37035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.695+0000 7fb89218bb80 1 -- 172.21.15.1:0/3247435286 >> 172.21.15.1:0/3247435286 conn(0x55a8f3636f00 msgr2=0x55a8f370c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:38.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.695+0000 7fb89218bb80 1 -- 172.21.15.1:0/3247435286 shutdown_connections 2024-04-26T01:50:38.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.695+0000 7fb89218bb80 1 -- 172.21.15.1:0/3247435286 wait complete. 2024-04-26T01:50:38.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.696+0000 7fb89218bb80 1 -- start start 2024-04-26T01:50:38.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.696+0000 7fb89218bb80 1 Processor -- start 2024-04-26T01:50:38.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.696+0000 7fb89218bb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f3708ef0 0x55a8f3702dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:38.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.696+0000 7fb89218bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f370e780 0x55a8f3703310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:38.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.696+0000 7fb89218bb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8f37125e0 0x55a8f372b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:38.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.696+0000 7fb89218bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a8f3723e40 con 0x55a8f370e780 2024-04-26T01:50:38.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.696+0000 7fb89218bb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a8f3723770 con 0x55a8f3708ef0 2024-04-26T01:50:38.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.696+0000 7fb89218bb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a8f3724510 con 0x55a8f37125e0 2024-04-26T01:50:38.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.696+0000 7fb89218bb80 10 client.0 _ll_register_callbacks cb 0x55a8f35f36d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:50:38.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.697+0000 7fb884e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f3708ef0 0x55a8f3702dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:38.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.697+0000 7fb884e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f3708ef0 0x55a8f3702dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:44000/0 (socket says 172.21.15.1:44000) 2024-04-26T01:50:38.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.697+0000 7fb884e00700 1 -- 172.21.15.1:0/1155939616 learned_addr learned my addr 172.21.15.1:0/1155939616 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:38.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.697+0000 7fb885800700 1 --2- 172.21.15.1:0/1155939616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8f37125e0 0x55a8f372b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:38.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.697+0000 7fb87fe00700 1 --2- 172.21.15.1:0/1155939616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f370e780 0x55a8f3703310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:38.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.697+0000 7fb884e00700 1 -- 172.21.15.1:0/1155939616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8f37125e0 msgr2=0x55a8f372b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:38.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.697+0000 7fb884e00700 1 --2- 172.21.15.1:0/1155939616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8f37125e0 0x55a8f372b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.697+0000 7fb884e00700 1 -- 172.21.15.1:0/1155939616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f370e780 msgr2=0x55a8f3703310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:50:38.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.697+0000 7fb884e00700 1 --2- 172.21.15.1:0/1155939616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f370e780 0x55a8f3703310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.697+0000 7fb884e00700 1 -- 172.21.15.1:0/1155939616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8f372bea0 con 0x55a8f3708ef0 2024-04-26T01:50:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.697+0000 7fb884e00700 1 --2- 172.21.15.1:0/1155939616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f3708ef0 0x55a8f3702dd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb874009230 tx=0x7fb87409cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=597456d8e4138815 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.697+0000 7fb866a00700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:50:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.698+0000 7fb866a00700 1 -- 172.21.15.1:0/1155939616 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8740a5400 con 0x55a8f3708ef0 2024-04-26T01:50:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.698+0000 7fb866a00700 1 -- 172.21.15.1:0/1155939616 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fb8740670b0 con 0x55a8f3708ef0 2024-04-26T01:50:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.698+0000 7fb89218bb80 1 -- 172.21.15.1:0/1155939616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb87409aa90 con 0x55a8f3708ef0 2024-04-26T01:50:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.698+0000 7fb866a00700 1 -- 172.21.15.1:0/1155939616 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb8740672b0 con 0x55a8f3708ef0 2024-04-26T01:50:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.698+0000 7fb89218bb80 10 client.24926 fetch_fsmap learned FSMap version 25 2024-04-26T01:50:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.698+0000 7fb89218bb80 1 -- 172.21.15.1:0/1155939616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a8f378a340 con 0x55a8f3708ef0 2024-04-26T01:50:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.698+0000 7fb866a00700 1 -- 172.21.15.1:0/1155939616 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb87409aa90 con 0x55a8f3708ef0 2024-04-26T01:50:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.698+0000 7fb866a00700 1 -- 172.21.15.1:0/1155939616 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fb8740d0070 con 0x55a8f3708ef0 2024-04-26T01:50:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.698+0000 7fb89218bb80 10 client.24926 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:50:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.698+0000 7fb89218bb80 10 client.24926 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T01:50:38.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.698+0000 7fb89218bb80 20 client.24926 populate_metadata read hostname 'smithi001' 2024-04-26T01:50:38.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.698+0000 7fb89218bb80 1 --2- 172.21.15.1:0/1155939616 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a8f378e640 0x55a8f378d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:38.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.698+0000 7fb89218bb80 4 client.24926 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T01:50:38.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.698+0000 7fb89218bb80 1 -- 172.21.15.1:0/1155939616 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "status"}) -- 0x55a8f36626d0 con 0x55a8f378e640 2024-04-26T01:50:38.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.699+0000 7fb87fe00700 1 --2- 172.21.15.1:0/1155939616 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a8f378e640 0x55a8f378d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:38.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.699+0000 7fb87fe00700 1 --2- 172.21.15.1:0/1155939616 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a8f378e640 0x55a8f378d350 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b2448380975cee server_cookie=79efd71afa68c420 in_seq=0 out_seq=0 2024-04-26T01:50:38.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.699+0000 7fb866a00700 10 client.24926 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:50:38.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.699+0000 7fb866a00700 1 -- 172.21.15.1:0/1155939616 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55a8f36626d0 con 0x55a8f378e640 2024-04-26T01:50:38.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.699+0000 7fb866a00700 10 client.24926 handle_command_reply: tid=0 2024-04-26T01:50:38.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.699+0000 7fb89218bb80 1 client.24926 shutdown 2024-04-26T01:50:38.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.700+0000 7fb89218bb80 1 -- 172.21.15.1:0/1155939616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f3708ef0 msgr2=0x55a8f3702dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:38.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.700+0000 7fb89218bb80 1 --2- 172.21.15.1:0/1155939616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f3708ef0 0x55a8f3702dd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb874009230 tx=0x7fb87409cf90 comp rx=0 tx=0).stop 2024-04-26T01:50:38.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.700+0000 7fb89218bb80 1 -- 172.21.15.1:0/1155939616 shutdown_connections 2024-04-26T01:50:38.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.700+0000 7fb89218bb80 1 --2- 172.21.15.1:0/1155939616 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a8f378e640 0x55a8f378d350 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.700+0000 7fb89218bb80 1 --2- 172.21.15.1:0/1155939616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8f37125e0 0x55a8f372b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.700+0000 7fb89218bb80 1 --2- 172.21.15.1:0/1155939616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a8f3708ef0 0x55a8f3702dd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.700+0000 7fb89218bb80 1 --2- 172.21.15.1:0/1155939616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8f370e780 0x55a8f3703310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.700+0000 7fb866a00700 0 client.24926 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:50:38.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.700+0000 7fb89218bb80 1 -- 172.21.15.1:0/1155939616 >> 172.21.15.1:0/1155939616 conn(0x55a8f3636f00 msgr2=0x55a8f3710930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:38.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.700+0000 7fb89218bb80 1 -- 172.21.15.1:0/1155939616 shutdown_connections 2024-04-26T01:50:38.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.700+0000 7fb89218bb80 1 -- 172.21.15.1:0/1155939616 wait complete. 2024-04-26T01:50:38.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.700+0000 7fb89218bb80 20 client.24926 trim_cache size 0 max 16384 2024-04-26T01:50:38.702 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T01:50:38.702 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "77cfcc80-036d-11ef-bc93-c7b262605968", 2024-04-26T01:50:38.702 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-04-26T01:50:38.702 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24518, 2024-04-26T01:50:38.703 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-04-26T01:50:38.703 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-04-26T01:50:38.703 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-04-26T01:50:38.703 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 188.581804293, 2024-04-26T01:50:38.703 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 25, 2024-04-26T01:50:38.703 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 92, 2024-04-26T01:50:38.703 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T01:50:38.703 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 209.069439905 2024-04-26T01:50:38.703 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T01:50:38.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.701+0000 7fb887800700 1 -- 172.21.15.1:0/2652709990 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb8680a3c40 msgr2=0x7fb8680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:38.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.701+0000 7fb887800700 1 --2- 172.21.15.1:0/2652709990 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb8680a3c40 0x7fb8680a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb86c003760 tx=0x7fb86c005fb0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.701+0000 7fb887800700 1 -- 172.21.15.1:0/2652709990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb880104eb0 msgr2=0x7fb88019b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:38.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.701+0000 7fb887800700 1 --2- 172.21.15.1:0/2652709990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb880104eb0 0x7fb88019b380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb87400e9f0 tx=0x7fb87400ed00 comp rx=0 tx=0).stop 2024-04-26T01:50:38.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.701+0000 7fb887800700 1 -- 172.21.15.1:0/2652709990 shutdown_connections 2024-04-26T01:50:38.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.701+0000 7fb887800700 1 --2- 172.21.15.1:0/2652709990 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb8680a3c40 0x7fb8680a6100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.701+0000 7fb887800700 1 --2- 172.21.15.1:0/2652709990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb880109f90 0x7fb88019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.701+0000 7fb887800700 1 --2- 172.21.15.1:0/2652709990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb880104eb0 0x7fb88019b380 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.701+0000 7fb887800700 1 --2- 172.21.15.1:0/2652709990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb880105860 0x7fb88019b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:38.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.701+0000 7fb887800700 1 -- 172.21.15.1:0/2652709990 >> 172.21.15.1:0/2652709990 conn(0x7fb880100740 msgr2=0x7fb880102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:38.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.701+0000 7fb887800700 1 -- 172.21.15.1:0/2652709990 shutdown_connections 2024-04-26T01:50:38.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:38.701+0000 7fb887800700 1 -- 172.21.15.1:0/2652709990 wait complete. 2024-04-26T01:50:38.868 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:38 smithi173 ceph-mon[32553]: pgmap v420: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:38.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:38 smithi001 ceph-mon[27822]: pgmap v420: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:38.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:38 smithi097 ceph-mon[32853]: pgmap v420: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:39.564 INFO:tasks.cephadm:Setup complete, yielding 2024-04-26T01:50:39.565 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T01:50:39.574 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-04-26T01:50:39.575 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-26T01:50:40.051 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:40.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:40 smithi001 ceph-mon[27822]: pgmap v421: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:40 smithi097 ceph-mon[32853]: pgmap v421: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:41.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:40 smithi173 ceph-mon[32553]: pgmap v421: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:42.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.184+0000 7efc34e00700 1 -- 172.21.15.1:0/469482546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc301090f0 msgr2=0x7efc30109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:42.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.184+0000 7efc34e00700 1 --2- 172.21.15.1:0/469482546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc301090f0 0x7efc30109510 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efc200099c0 tx=0x7efc20009cd0 comp rx=0 tx=0).stop 2024-04-26T01:50:42.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.184+0000 7efc34e00700 1 -- 172.21.15.1:0/469482546 shutdown_connections 2024-04-26T01:50:42.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.184+0000 7efc34e00700 1 --2- 172.21.15.1:0/469482546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc3010ac90 0x7efc30111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:42.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.184+0000 7efc34e00700 1 --2- 172.21.15.1:0/469482546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc301090f0 0x7efc30109510 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:42.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.184+0000 7efc34e00700 1 --2- 172.21.15.1:0/469482546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc3010a2f0 0x7efc3010a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:42.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.184+0000 7efc34e00700 1 -- 172.21.15.1:0/469482546 >> 172.21.15.1:0/469482546 conn(0x7efc300780c0 msgr2=0x7efc3007a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:42.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.185+0000 7efc34e00700 1 -- 172.21.15.1:0/469482546 shutdown_connections 2024-04-26T01:50:42.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.185+0000 7efc34e00700 1 -- 172.21.15.1:0/469482546 wait complete. 2024-04-26T01:50:42.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.186+0000 7efc34e00700 1 Processor -- start 2024-04-26T01:50:42.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.186+0000 7efc34e00700 1 -- start start 2024-04-26T01:50:42.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.186+0000 7efc34e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc301090f0 0x7efc3019f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:42.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.186+0000 7efc34e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc3010a2f0 0x7efc3019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:42.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.186+0000 7efc34e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc3010ac90 0x7efc301a6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:42.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.186+0000 7efc34e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efc30114be0 con 0x7efc3010a2f0 2024-04-26T01:50:42.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.186+0000 7efc34e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efc30114a60 con 0x7efc301090f0 2024-04-26T01:50:42.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.186+0000 7efc34e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efc30114d60 con 0x7efc3010ac90 2024-04-26T01:50:42.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.187+0000 7efc2d400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc3010a2f0 0x7efc3019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:42.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.187+0000 7efc2de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc301090f0 0x7efc3019f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:42.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.187+0000 7efc2de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc301090f0 0x7efc3019f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:44006/0 (socket says 172.21.15.1:44006) 2024-04-26T01:50:42.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.187+0000 7efc2d400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc3010a2f0 0x7efc3019f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55116/0 (socket says 172.21.15.1:55116) 2024-04-26T01:50:42.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.187+0000 7efc2de00700 1 -- 172.21.15.1:0/3301423683 learned_addr learned my addr 172.21.15.1:0/3301423683 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:42.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.187+0000 7efc2e800700 1 --2- 172.21.15.1:0/3301423683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc3010ac90 0x7efc301a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:42.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.187+0000 7efc2de00700 1 -- 172.21.15.1:0/3301423683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc3010ac90 msgr2=0x7efc301a6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:42.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.187+0000 7efc2de00700 1 --2- 172.21.15.1:0/3301423683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc3010ac90 0x7efc301a6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:42.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.187+0000 7efc2de00700 1 -- 172.21.15.1:0/3301423683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc3010a2f0 msgr2=0x7efc3019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:42.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.187+0000 7efc2de00700 1 --2- 172.21.15.1:0/3301423683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc3010a2f0 0x7efc3019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:42.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.187+0000 7efc2de00700 1 -- 172.21.15.1:0/3301423683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc301a6ec0 con 0x7efc301090f0 2024-04-26T01:50:42.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.187+0000 7efc2d400700 1 --2- 172.21.15.1:0/3301423683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc3010a2f0 0x7efc3019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:42.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.188+0000 7efc2de00700 1 --2- 172.21.15.1:0/3301423683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc301090f0 0x7efc3019f2b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efc2000b580 tx=0x7efc20003e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:42.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.188+0000 7efc26a00700 1 -- 172.21.15.1:0/3301423683 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20009e60 con 0x7efc301090f0 2024-04-26T01:50:42.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.188+0000 7efc34e00700 1 -- 172.21.15.1:0/3301423683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc301a7150 con 0x7efc301090f0 2024-04-26T01:50:42.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.188+0000 7efc26a00700 1 -- 172.21.15.1:0/3301423683 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc2000f460 con 0x7efc301090f0 2024-04-26T01:50:42.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.188+0000 7efc34e00700 1 -- 172.21.15.1:0/3301423683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc301a7600 con 0x7efc301090f0 2024-04-26T01:50:42.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.188+0000 7efc26a00700 1 -- 172.21.15.1:0/3301423683 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20017650 con 0x7efc301090f0 2024-04-26T01:50:42.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.189+0000 7efc34e00700 1 -- 172.21.15.1:0/3301423683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc30069730 con 0x7efc301090f0 2024-04-26T01:50:42.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.191+0000 7efc26a00700 1 -- 172.21.15.1:0/3301423683 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7efc2000f8f0 con 0x7efc301090f0 2024-04-26T01:50:42.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.193+0000 7efc26a00700 1 --2- 172.21.15.1:0/3301423683 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7efc140a3cc0 0x7efc140a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:42.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.193+0000 7efc26a00700 1 -- 172.21.15.1:0/3301423683 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7efc200d11b0 con 0x7efc301090f0 2024-04-26T01:50:42.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.193+0000 7efc2d400700 1 --2- 172.21.15.1:0/3301423683 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7efc140a3cc0 0x7efc140a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:42.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.194+0000 7efc2d400700 1 --2- 172.21.15.1:0/3301423683 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7efc140a3cc0 0x7efc140a6180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efc1c010420 tx=0x7efc1c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:42.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.197+0000 7efc26a00700 1 -- 172.21.15.1:0/3301423683 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc20013070 con 0x7efc301090f0 2024-04-26T01:50:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.454+0000 7efc34e00700 1 -- 172.21.15.1:0/3301423683 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7efc30062e80 con 0x7efc140a3cc0 2024-04-26T01:50:42.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.456+0000 7efc26a00700 1 -- 172.21.15.1:0/3301423683 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7efc30062e80 con 0x7efc140a3cc0 2024-04-26T01:50:42.454 INFO:teuthology.orchestra.run.smithi001.stdout:Backend: cephadm 2024-04-26T01:50:42.454 INFO:teuthology.orchestra.run.smithi001.stdout:Available: Yes 2024-04-26T01:50:42.454 INFO:teuthology.orchestra.run.smithi001.stdout:Paused: No 2024-04-26T01:50:42.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.459+0000 7efc34e00700 1 -- 172.21.15.1:0/3301423683 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7efc140a3cc0 msgr2=0x7efc140a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:42.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.459+0000 7efc34e00700 1 --2- 172.21.15.1:0/3301423683 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7efc140a3cc0 0x7efc140a6180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efc1c010420 tx=0x7efc1c0073d0 comp rx=0 tx=0).stop 2024-04-26T01:50:42.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.459+0000 7efc34e00700 1 -- 172.21.15.1:0/3301423683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc301090f0 msgr2=0x7efc3019f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:42.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.459+0000 7efc34e00700 1 --2- 172.21.15.1:0/3301423683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc301090f0 0x7efc3019f2b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efc2000b580 tx=0x7efc20003e60 comp rx=0 tx=0).stop 2024-04-26T01:50:42.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.459+0000 7efc34e00700 1 -- 172.21.15.1:0/3301423683 shutdown_connections 2024-04-26T01:50:42.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.459+0000 7efc34e00700 1 --2- 172.21.15.1:0/3301423683 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7efc140a3cc0 0x7efc140a6180 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:42.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.459+0000 7efc34e00700 1 --2- 172.21.15.1:0/3301423683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc3010ac90 0x7efc301a6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:42.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.459+0000 7efc34e00700 1 --2- 172.21.15.1:0/3301423683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc301090f0 0x7efc3019f2b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:42.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.459+0000 7efc34e00700 1 --2- 172.21.15.1:0/3301423683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc3010a2f0 0x7efc3019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:42.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.459+0000 7efc34e00700 1 -- 172.21.15.1:0/3301423683 >> 172.21.15.1:0/3301423683 conn(0x7efc300780c0 msgr2=0x7efc3010d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:42.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.459+0000 7efc34e00700 1 -- 172.21.15.1:0/3301423683 shutdown_connections 2024-04-26T01:50:42.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:42.459+0000 7efc34e00700 1 -- 172.21.15.1:0/3301423683 wait complete. 2024-04-26T01:50:42.842 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:42 smithi001 ceph-mon[27822]: pgmap v422: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:50:42.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:42 smithi097 ceph-mon[32853]: pgmap v422: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:50:43.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:42 smithi173 ceph-mon[32553]: pgmap v422: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:50:43.299 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-26T01:50:43.782 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:43.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:43 smithi001 ceph-mon[27822]: from='client.24932 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:43.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:43 smithi097 ceph-mon[32853]: from='client.24932 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:44.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:43 smithi173 ceph-mon[32553]: from='client.24932 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:44.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:44 smithi097 ceph-mon[32853]: pgmap v423: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:44.997 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:44 smithi001 ceph-mon[27822]: pgmap v423: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:45.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:44 smithi173 ceph-mon[32553]: pgmap v423: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:45.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.916+0000 7fcafce00700 1 -- 172.21.15.1:0/455557010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcaf8101320 msgr2=0x7fcaf8101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:45.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.916+0000 7fcafce00700 1 --2- 172.21.15.1:0/455557010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcaf8101320 0x7fcaf8101740 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcae800b0a0 tx=0x7fcae800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:50:45.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.916+0000 7fcafce00700 1 -- 172.21.15.1:0/455557010 shutdown_connections 2024-04-26T01:50:45.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.916+0000 7fcafce00700 1 --2- 172.21.15.1:0/455557010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcaf8102640 0x7fcaf8102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:45.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.916+0000 7fcafce00700 1 --2- 172.21.15.1:0/455557010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcaf8101320 0x7fcaf8101740 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:45.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.916+0000 7fcafce00700 1 --2- 172.21.15.1:0/455557010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf8101c80 0x7fcaf8102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:45.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.916+0000 7fcafce00700 1 -- 172.21.15.1:0/455557010 >> 172.21.15.1:0/455557010 conn(0x7fcaf80fcf40 msgr2=0x7fcaf80ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:45.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.916+0000 7fcafce00700 1 -- 172.21.15.1:0/455557010 shutdown_connections 2024-04-26T01:50:45.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.916+0000 7fcafce00700 1 -- 172.21.15.1:0/455557010 wait complete. 2024-04-26T01:50:45.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.918+0000 7fcafce00700 1 Processor -- start 2024-04-26T01:50:45.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.918+0000 7fcafce00700 1 -- start start 2024-04-26T01:50:45.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.919+0000 7fcafce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcaf8101320 0x7fcaf819ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:45.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.919+0000 7fcafce00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcaf8101c80 0x7fcaf819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:45.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.919+0000 7fcafce00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf8102640 0x7fcaf81a2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:45.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.919+0000 7fcafce00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf8110740 con 0x7fcaf8101320 2024-04-26T01:50:45.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.919+0000 7fcafce00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81105c0 con 0x7fcaf8102640 2024-04-26T01:50:45.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.919+0000 7fcafce00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81108c0 con 0x7fcaf8101c80 2024-04-26T01:50:45.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.919+0000 7fcaf5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcaf8101320 0x7fcaf819ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:45.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.919+0000 7fcaf5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcaf8101320 0x7fcaf819ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54412/0 (socket says 172.21.15.1:54412) 2024-04-26T01:50:45.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.919+0000 7fcaf5e00700 1 -- 172.21.15.1:0/3786449088 learned_addr learned my addr 172.21.15.1:0/3786449088 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:45.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.919+0000 7fcaed400700 1 --2- 172.21.15.1:0/3786449088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcaf8101c80 0x7fcaf819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:45.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.919+0000 7fcaf6800700 1 --2- 172.21.15.1:0/3786449088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf8102640 0x7fcaf81a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:45.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.920+0000 7fcaf5e00700 1 -- 172.21.15.1:0/3786449088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcaf8101c80 msgr2=0x7fcaf819b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:45.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.920+0000 7fcaf5e00700 1 --2- 172.21.15.1:0/3786449088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcaf8101c80 0x7fcaf819b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:45.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.920+0000 7fcaf5e00700 1 -- 172.21.15.1:0/3786449088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf8102640 msgr2=0x7fcaf81a2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:45.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.920+0000 7fcaf5e00700 1 --2- 172.21.15.1:0/3786449088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf8102640 0x7fcaf81a2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:45.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.920+0000 7fcaf5e00700 1 -- 172.21.15.1:0/3786449088 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf81a2ae0 con 0x7fcaf8101320 2024-04-26T01:50:45.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.920+0000 7fcaed400700 1 --2- 172.21.15.1:0/3786449088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcaf8101c80 0x7fcaf819b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:45.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.920+0000 7fcaf6800700 1 --2- 172.21.15.1:0/3786449088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf8102640 0x7fcaf81a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:45.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.920+0000 7fcaf5e00700 1 --2- 172.21.15.1:0/3786449088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcaf8101320 0x7fcaf819ae80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcae800d010 tx=0x7fcae8004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:45.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.920+0000 7fcaef400700 1 -- 172.21.15.1:0/3786449088 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8021070 con 0x7fcaf8101320 2024-04-26T01:50:45.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.920+0000 7fcafce00700 1 -- 172.21.15.1:0/3786449088 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf81a2d70 con 0x7fcaf8101320 2024-04-26T01:50:45.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.921+0000 7fcafce00700 1 -- 172.21.15.1:0/3786449088 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf81a31c0 con 0x7fcaf8101320 2024-04-26T01:50:45.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.921+0000 7fcaef400700 1 -- 172.21.15.1:0/3786449088 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcae8007e30 con 0x7fcaf8101320 2024-04-26T01:50:45.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.921+0000 7fcaef400700 1 -- 172.21.15.1:0/3786449088 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8004770 con 0x7fcaf8101320 2024-04-26T01:50:45.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.922+0000 7fcafce00700 1 -- 172.21.15.1:0/3786449088 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf8069730 con 0x7fcaf8101320 2024-04-26T01:50:45.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.924+0000 7fcaef400700 1 -- 172.21.15.1:0/3786449088 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fcae8007980 con 0x7fcaf8101320 2024-04-26T01:50:45.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.925+0000 7fcaef400700 1 --2- 172.21.15.1:0/3786449088 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fcadc0a3cc0 0x7fcadc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:45.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.926+0000 7fcaef400700 1 -- 172.21.15.1:0/3786449088 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fcae800f020 con 0x7fcaf8101320 2024-04-26T01:50:45.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.926+0000 7fcaed400700 1 --2- 172.21.15.1:0/3786449088 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fcadc0a3cc0 0x7fcadc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:45.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.927+0000 7fcaed400700 1 --2- 172.21.15.1:0/3786449088 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fcadc0a3cc0 0x7fcadc0a6180 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcae0011490 tx=0x7fcae0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:45.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:45.930+0000 7fcaef400700 1 -- 172.21.15.1:0/3786449088 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcae802a020 con 0x7fcaf8101320 2024-04-26T01:50:46.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:46.178+0000 7fcafce00700 1 -- 172.21.15.1:0/3786449088 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fcaf8062e80 con 0x7fcadc0a3cc0 2024-04-26T01:50:46.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:46.215+0000 7fcaef400700 1 -- 172.21.15.1:0/3786449088 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fcaf8062e80 con 0x7fcadc0a3cc0 2024-04-26T01:50:46.214 INFO:teuthology.orchestra.run.smithi001.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T01:50:46.214 INFO:teuthology.orchestra.run.smithi001.stdout:mds.a smithi001 running (3m) 44s ago 3m 12.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 61178bdc637a 2024-04-26T01:50:46.214 INFO:teuthology.orchestra.run.smithi001.stdout:mds.b smithi097 running (3m) 76s ago 3m 27.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 1ded51ec221f 2024-04-26T01:50:46.214 INFO:teuthology.orchestra.run.smithi001.stdout:mds.c smithi173 running (3m) 76s ago 3m 22.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 e9cd24ca2a36 2024-04-26T01:50:46.214 INFO:teuthology.orchestra.run.smithi001.stdout:mds.d smithi001 running (4m) 44s ago 4m 13.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 49c38e895f49 2024-04-26T01:50:46.214 INFO:teuthology.orchestra.run.smithi001.stdout:mds.e smithi097 running (4m) 76s ago 4m 19.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 1ae8bb4ee485 2024-04-26T01:50:46.214 INFO:teuthology.orchestra.run.smithi001.stdout:mds.f smithi173 running (3m) 76s ago 3m 20.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 a584bc72024d 2024-04-26T01:50:46.214 INFO:teuthology.orchestra.run.smithi001.stdout:mds.g smithi001 running (3m) 44s ago 3m 13.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 01bfdcf9c32e 2024-04-26T01:50:46.214 INFO:teuthology.orchestra.run.smithi001.stdout:mds.h smithi097 running (3m) 76s ago 3m 19.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 529af91b6a60 2024-04-26T01:50:46.214 INFO:teuthology.orchestra.run.smithi001.stdout:mds.i smithi173 running (3m) 76s ago 3m 19.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3fcc96c76357 2024-04-26T01:50:46.214 INFO:teuthology.orchestra.run.smithi001.stdout:mds.j smithi001 running (4m) 44s ago 4m 13.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 caccc4506c76 2024-04-26T01:50:46.214 INFO:teuthology.orchestra.run.smithi001.stdout:mds.k smithi097 running (4m) 76s ago 4m 15.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 4fb93c802074 2024-04-26T01:50:46.215 INFO:teuthology.orchestra.run.smithi001.stdout:mds.l smithi173 running (4m) 76s ago 4m 16.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 4d65dc01c89e 2024-04-26T01:50:46.215 INFO:teuthology.orchestra.run.smithi001.stdout:mgr.x smithi001 *:9283,8765 running (12m) 44s ago 12m 516M - 19.0.0-3156-gd69f13d8 da4b004c09e8 8387a05c9c89 2024-04-26T01:50:46.215 INFO:teuthology.orchestra.run.smithi001.stdout:mgr.y smithi097 *:8443,8765 running (10m) 76s ago 10m 455M - 19.0.0-3156-gd69f13d8 da4b004c09e8 b29fd37f9ca7 2024-04-26T01:50:46.215 INFO:teuthology.orchestra.run.smithi001.stdout:mgr.z smithi173 *:8443,8765 running (10m) 76s ago 10m 454M - 19.0.0-3156-gd69f13d8 da4b004c09e8 28ffc01326f1 2024-04-26T01:50:46.215 INFO:teuthology.orchestra.run.smithi001.stdout:mon.a smithi001 running (12m) 44s ago 12m 49.5M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 42fcca49f2d3 2024-04-26T01:50:46.215 INFO:teuthology.orchestra.run.smithi001.stdout:mon.b smithi097 running (10m) 76s ago 10m 44.2M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 e232f8872610 2024-04-26T01:50:46.215 INFO:teuthology.orchestra.run.smithi001.stdout:mon.c smithi173 running (10m) 76s ago 10m 41.9M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 6dd41fdc413d 2024-04-26T01:50:46.215 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0 smithi001 running (9m) 44s ago 9m 49.2M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 bdf5bb806559 2024-04-26T01:50:46.215 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 smithi097 running (9m) 76s ago 9m 59.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 f62b60ea36ac 2024-04-26T01:50:46.215 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2 smithi173 running (8m) 76s ago 8m 53.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 aef5d1859314 2024-04-26T01:50:46.215 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 smithi001 running (8m) 44s ago 8m 47.1M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 fca45df65be1 2024-04-26T01:50:46.215 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 smithi097 running (7m) 76s ago 7m 46.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 2a40c1f1c767 2024-04-26T01:50:46.216 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5 smithi173 running (7m) 76s ago 7m 56.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 b45ebcdd3f7f 2024-04-26T01:50:46.216 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6 smithi001 running (6m) 44s ago 6m 45.4M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 b179dfef39c7 2024-04-26T01:50:46.216 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7 smithi097 running (6m) 76s ago 6m 48.8M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 22aeccdef0a9 2024-04-26T01:50:46.216 INFO:teuthology.orchestra.run.smithi001.stdout:osd.8 smithi173 running (6m) 76s ago 6m 58.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 a2694a52152e 2024-04-26T01:50:46.216 INFO:teuthology.orchestra.run.smithi001.stdout:osd.9 smithi001 running (5m) 44s ago 5m 45.5M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 bb4a71d9cbfd 2024-04-26T01:50:46.216 INFO:teuthology.orchestra.run.smithi001.stdout:osd.10 smithi097 running (5m) 76s ago 5m 47.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 50f8e9572f89 2024-04-26T01:50:46.216 INFO:teuthology.orchestra.run.smithi001.stdout:osd.11 smithi173 running (4m) 76s ago 4m 48.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 da2d77cd0880 2024-04-26T01:50:46.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:46.218+0000 7fcafce00700 1 -- 172.21.15.1:0/3786449088 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fcadc0a3cc0 msgr2=0x7fcadc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:46.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:46.218+0000 7fcafce00700 1 --2- 172.21.15.1:0/3786449088 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fcadc0a3cc0 0x7fcadc0a6180 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcae0011490 tx=0x7fcae0009260 comp rx=0 tx=0).stop 2024-04-26T01:50:46.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:46.218+0000 7fcafce00700 1 -- 172.21.15.1:0/3786449088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcaf8101320 msgr2=0x7fcaf819ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:46.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:46.218+0000 7fcafce00700 1 --2- 172.21.15.1:0/3786449088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcaf8101320 0x7fcaf819ae80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcae800d010 tx=0x7fcae8004d70 comp rx=0 tx=0).stop 2024-04-26T01:50:46.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:46.218+0000 7fcafce00700 1 -- 172.21.15.1:0/3786449088 shutdown_connections 2024-04-26T01:50:46.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:46.218+0000 7fcafce00700 1 --2- 172.21.15.1:0/3786449088 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fcadc0a3cc0 0x7fcadc0a6180 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:46.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:46.218+0000 7fcafce00700 1 --2- 172.21.15.1:0/3786449088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf8102640 0x7fcaf81a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:46.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:46.218+0000 7fcafce00700 1 --2- 172.21.15.1:0/3786449088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcaf8101320 0x7fcaf819ae80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:46.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:46.218+0000 7fcafce00700 1 --2- 172.21.15.1:0/3786449088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcaf8101c80 0x7fcaf819b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:46.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:46.218+0000 7fcafce00700 1 -- 172.21.15.1:0/3786449088 >> 172.21.15.1:0/3786449088 conn(0x7fcaf80fcf40 msgr2=0x7fcaf80fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:46.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:46.218+0000 7fcafce00700 1 -- 172.21.15.1:0/3786449088 shutdown_connections 2024-04-26T01:50:46.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:46.218+0000 7fcafce00700 1 -- 172.21.15.1:0/3786449088 wait complete. 2024-04-26T01:50:46.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:46 smithi001 ceph-mon[27822]: pgmap v424: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:46.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:46 smithi097 ceph-mon[32853]: pgmap v424: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:47.063 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-26T01:50:47.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:46 smithi173 ceph-mon[32553]: pgmap v424: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:47.541 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:47.906 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:47 smithi001 ceph-mon[27822]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:47.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:47 smithi097 ceph-mon[32853]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:48.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:47 smithi173 ceph-mon[32553]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:48.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:48 smithi001 ceph-mon[27822]: pgmap v425: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:48.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:48 smithi097 ceph-mon[32853]: pgmap v425: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:49.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:48 smithi173 ceph-mon[32553]: pgmap v425: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:49.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.769+0000 7f9735e00700 1 -- 172.21.15.1:0/2976862183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9730078640 msgr2=0x7f9730078ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:49.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.769+0000 7f9735e00700 1 --2- 172.21.15.1:0/2976862183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9730078640 0x7f9730078ac0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f971c009990 tx=0x7f971c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:50:49.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.769+0000 7f9735e00700 1 -- 172.21.15.1:0/2976862183 shutdown_connections 2024-04-26T01:50:49.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.769+0000 7f9735e00700 1 --2- 172.21.15.1:0/2976862183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9730079000 0x7f9730079880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:49.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.769+0000 7f9735e00700 1 --2- 172.21.15.1:0/2976862183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97300773f0 0x7f9730077810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:49.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.769+0000 7f9735e00700 1 --2- 172.21.15.1:0/2976862183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9730078640 0x7f9730078ac0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:49.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.769+0000 7f9735e00700 1 -- 172.21.15.1:0/2976862183 >> 172.21.15.1:0/2976862183 conn(0x7f9730100290 msgr2=0x7f97301026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:49.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.769+0000 7f9735e00700 1 -- 172.21.15.1:0/2976862183 shutdown_connections 2024-04-26T01:50:49.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.770+0000 7f9735e00700 1 -- 172.21.15.1:0/2976862183 wait complete. 2024-04-26T01:50:49.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.771+0000 7f9735e00700 1 Processor -- start 2024-04-26T01:50:49.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.771+0000 7f9735e00700 1 -- start start 2024-04-26T01:50:49.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.771+0000 7f9735e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97300773f0 0x7f973019f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:49.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.771+0000 7f9735e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9730078640 0x7f973019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:49.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.771+0000 7f9735e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9730079000 0x7f97301a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.771+0000 7f9735e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9730114bd0 con 0x7f9730078640 2024-04-26T01:50:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.771+0000 7f9735e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9730114a50 con 0x7f97300773f0 2024-04-26T01:50:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.771+0000 7f9735e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9730114d50 con 0x7f9730079000 2024-04-26T01:50:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.772+0000 7f972e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9730078640 0x7f973019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.772+0000 7f972e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9730078640 0x7f973019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54416/0 (socket says 172.21.15.1:54416) 2024-04-26T01:50:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.772+0000 7f972e000700 1 -- 172.21.15.1:0/2872152993 learned_addr learned my addr 172.21.15.1:0/2872152993 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.772+0000 7f972ea00700 1 --2- 172.21.15.1:0/2872152993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97300773f0 0x7f973019f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.772+0000 7f972f400700 1 --2- 172.21.15.1:0/2872152993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9730079000 0x7f97301a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.772+0000 7f972e000700 1 -- 172.21.15.1:0/2872152993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9730079000 msgr2=0x7f97301a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.772+0000 7f972e000700 1 --2- 172.21.15.1:0/2872152993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9730079000 0x7f97301a67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.772+0000 7f972e000700 1 -- 172.21.15.1:0/2872152993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97300773f0 msgr2=0x7f973019f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.772+0000 7f972e000700 1 --2- 172.21.15.1:0/2872152993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97300773f0 0x7f973019f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:49.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.772+0000 7f972e000700 1 -- 172.21.15.1:0/2872152993 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97301a6e70 con 0x7f9730078640 2024-04-26T01:50:49.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.772+0000 7f972ea00700 1 --2- 172.21.15.1:0/2872152993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97300773f0 0x7f973019f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:49.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.773+0000 7f972e000700 1 --2- 172.21.15.1:0/2872152993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9730078640 0x7f973019f750 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f971c000c00 tx=0x7f971c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:49.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.773+0000 7f9734a00700 1 -- 172.21.15.1:0/2872152993 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c027070 con 0x7f9730078640 2024-04-26T01:50:49.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.773+0000 7f9734a00700 1 -- 172.21.15.1:0/2872152993 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f971c022470 con 0x7f9730078640 2024-04-26T01:50:49.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.773+0000 7f9735e00700 1 -- 172.21.15.1:0/2872152993 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97301a7100 con 0x7f9730078640 2024-04-26T01:50:49.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.773+0000 7f9734a00700 1 -- 172.21.15.1:0/2872152993 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c017650 con 0x7f9730078640 2024-04-26T01:50:49.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.773+0000 7f9735e00700 1 -- 172.21.15.1:0/2872152993 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97301a75b0 con 0x7f9730078640 2024-04-26T01:50:49.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.774+0000 7f9735e00700 1 -- 172.21.15.1:0/2872152993 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9730069730 con 0x7f9730078640 2024-04-26T01:50:49.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.776+0000 7f9734a00700 1 -- 172.21.15.1:0/2872152993 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f971c009d10 con 0x7f9730078640 2024-04-26T01:50:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.778+0000 7f9734a00700 1 --2- 172.21.15.1:0/2872152993 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f97140a3cc0 0x7f97140a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.778+0000 7f9734a00700 1 -- 172.21.15.1:0/2872152993 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f971c0d0e20 con 0x7f9730078640 2024-04-26T01:50:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.778+0000 7f972ea00700 1 --2- 172.21.15.1:0/2872152993 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f97140a3cc0 0x7f97140a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:49.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.779+0000 7f972ea00700 1 --2- 172.21.15.1:0/2872152993 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f97140a3cc0 0x7f97140a6180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9720011490 tx=0x7f9720009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:49.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:49.783+0000 7f9734a00700 1 -- 172.21.15.1:0/2872152993 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f971c013070 con 0x7f9730078640 2024-04-26T01:50:50.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:50.022+0000 7f9735e00700 1 -- 172.21.15.1:0/2872152993 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f9730062f40 con 0x7f97140a3cc0 2024-04-26T01:50:50.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:50.039+0000 7f9734a00700 1 -- 172.21.15.1:0/2872152993 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f9730062f40 con 0x7f97140a3cc0 2024-04-26T01:50:50.037 INFO:teuthology.orchestra.run.smithi001.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-26T01:50:50.038 INFO:teuthology.orchestra.run.smithi001.stdout:mds.all 12/12 80s ago 4m smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l;count:12 2024-04-26T01:50:50.038 INFO:teuthology.orchestra.run.smithi001.stdout:mgr 3/3 80s ago 10m smithi001=x;smithi097=y;smithi173=z;count:3 2024-04-26T01:50:50.038 INFO:teuthology.orchestra.run.smithi001.stdout:mon 3/3 80s ago 10m smithi001:172.21.15.1=a;smithi097:172.21.15.97=b;smithi173:172.21.15.173=c;count:3 2024-04-26T01:50:50.038 INFO:teuthology.orchestra.run.smithi001.stdout:osd 12 80s ago - 2024-04-26T01:50:50.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:50.042+0000 7f9735e00700 1 -- 172.21.15.1:0/2872152993 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f97140a3cc0 msgr2=0x7f97140a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:50.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:50.042+0000 7f9735e00700 1 --2- 172.21.15.1:0/2872152993 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f97140a3cc0 0x7f97140a6180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9720011490 tx=0x7f9720009260 comp rx=0 tx=0).stop 2024-04-26T01:50:50.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:50.042+0000 7f9735e00700 1 -- 172.21.15.1:0/2872152993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9730078640 msgr2=0x7f973019f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:50.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:50.042+0000 7f9735e00700 1 --2- 172.21.15.1:0/2872152993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9730078640 0x7f973019f750 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f971c000c00 tx=0x7f971c0047b0 comp rx=0 tx=0).stop 2024-04-26T01:50:50.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:50.042+0000 7f9735e00700 1 -- 172.21.15.1:0/2872152993 shutdown_connections 2024-04-26T01:50:50.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:50.042+0000 7f9735e00700 1 --2- 172.21.15.1:0/2872152993 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f97140a3cc0 0x7f97140a6180 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:50.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:50.042+0000 7f9735e00700 1 --2- 172.21.15.1:0/2872152993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9730079000 0x7f97301a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:50.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:50.042+0000 7f9735e00700 1 --2- 172.21.15.1:0/2872152993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97300773f0 0x7f973019f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:50.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:50.043+0000 7f9735e00700 1 --2- 172.21.15.1:0/2872152993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9730078640 0x7f973019f750 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:50.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:50.043+0000 7f9735e00700 1 -- 172.21.15.1:0/2872152993 >> 172.21.15.1:0/2872152993 conn(0x7f9730100290 msgr2=0x7f9730101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:50.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:50.043+0000 7f9735e00700 1 -- 172.21.15.1:0/2872152993 shutdown_connections 2024-04-26T01:50:50.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:50.043+0000 7f9735e00700 1 -- 172.21.15.1:0/2872152993 wait complete. 2024-04-26T01:50:50.826 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:50 smithi001 ceph-mon[27822]: pgmap v426: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:50.902 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-26T01:50:50.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:50 smithi097 ceph-mon[32853]: pgmap v426: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:51.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:50 smithi173 ceph-mon[32553]: pgmap v426: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:51.389 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:51.794 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:51 smithi001 ceph-mon[27822]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:51.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:51 smithi097 ceph-mon[32853]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:52.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:51 smithi173 ceph-mon[32553]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:52.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:52 smithi097 ceph-mon[32853]: pgmap v427: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:50:53.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:52 smithi173 ceph-mon[32553]: pgmap v427: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:50:53.100 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:52 smithi001 ceph-mon[27822]: pgmap v427: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:50:53.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.687+0000 7f3893200700 1 -- 172.21.15.1:0/2267890050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c076a70 msgr2=0x7f388c074ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:53.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.687+0000 7f3893200700 1 --2- 172.21.15.1:0/2267890050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c076a70 0x7f388c074ed0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f387c009990 tx=0x7f387c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.687+0000 7f3893200700 1 -- 172.21.15.1:0/2267890050 shutdown_connections 2024-04-26T01:50:53.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.687+0000 7f3893200700 1 --2- 172.21.15.1:0/2267890050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f388c10a980 0x7f388c10ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.687+0000 7f3893200700 1 --2- 172.21.15.1:0/2267890050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c076a70 0x7f388c074ed0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.687+0000 7f3893200700 1 --2- 172.21.15.1:0/2267890050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c0754a0 0x7f388c075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.687+0000 7f3893200700 1 -- 172.21.15.1:0/2267890050 >> 172.21.15.1:0/2267890050 conn(0x7f388c0fe010 msgr2=0x7f388c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:53.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.688+0000 7f3893200700 1 -- 172.21.15.1:0/2267890050 shutdown_connections 2024-04-26T01:50:53.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.688+0000 7f3893200700 1 -- 172.21.15.1:0/2267890050 wait complete. 2024-04-26T01:50:53.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.689+0000 7f3893200700 1 Processor -- start 2024-04-26T01:50:53.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.689+0000 7f3893200700 1 -- start start 2024-04-26T01:50:53.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.689+0000 7f3893200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f388c0754a0 0x7f388c19ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:53.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.689+0000 7f3893200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c076a70 0x7f388c19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:53.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.689+0000 7f3893200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c10a980 0x7f388c1a2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:53.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.689+0000 7f3893200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f388c110790 con 0x7f388c0754a0 2024-04-26T01:50:53.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.689+0000 7f3893200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f388c110610 con 0x7f388c10a980 2024-04-26T01:50:53.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.689+0000 7f3893200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f388c110910 con 0x7f388c076a70 2024-04-26T01:50:53.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.690+0000 7f388be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f388c0754a0 0x7f388c19ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:53.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.690+0000 7f388b400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c076a70 0x7f388c19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:53.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.690+0000 7f388be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f388c0754a0 0x7f388c19ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59312/0 (socket says 172.21.15.1:59312) 2024-04-26T01:50:53.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.690+0000 7f388be00700 1 -- 172.21.15.1:0/2350946162 learned_addr learned my addr 172.21.15.1:0/2350946162 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:53.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.690+0000 7f3891200700 1 --2- 172.21.15.1:0/2350946162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c10a980 0x7f388c1a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:53.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.690+0000 7f388be00700 1 -- 172.21.15.1:0/2350946162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c076a70 msgr2=0x7f388c19b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:53.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.690+0000 7f388be00700 1 --2- 172.21.15.1:0/2350946162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c076a70 0x7f388c19b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.690+0000 7f388be00700 1 -- 172.21.15.1:0/2350946162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c10a980 msgr2=0x7f388c1a2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:53.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.690+0000 7f388be00700 1 --2- 172.21.15.1:0/2350946162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c10a980 0x7f388c1a2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.690+0000 7f388be00700 1 -- 172.21.15.1:0/2350946162 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388c1a2ae0 con 0x7f388c0754a0 2024-04-26T01:50:53.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.690+0000 7f388b400700 1 --2- 172.21.15.1:0/2350946162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c076a70 0x7f388c19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:53.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.690+0000 7f3891200700 1 --2- 172.21.15.1:0/2350946162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c10a980 0x7f388c1a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:53.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.691+0000 7f388be00700 1 --2- 172.21.15.1:0/2350946162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f388c0754a0 0x7f388c19ae80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f387c0047e0 tx=0x7f387c009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:53.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.691+0000 7f3888c00700 1 -- 172.21.15.1:0/2350946162 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c026070 con 0x7f388c0754a0 2024-04-26T01:50:53.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.691+0000 7f3888c00700 1 -- 172.21.15.1:0/2350946162 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f387c021410 con 0x7f388c0754a0 2024-04-26T01:50:53.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.691+0000 7f3888c00700 1 -- 172.21.15.1:0/2350946162 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c017650 con 0x7f388c0754a0 2024-04-26T01:50:53.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.691+0000 7f3893200700 1 -- 172.21.15.1:0/2350946162 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388c1a2d70 con 0x7f388c0754a0 2024-04-26T01:50:53.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.691+0000 7f3893200700 1 -- 172.21.15.1:0/2350946162 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f388c1a3170 con 0x7f388c0754a0 2024-04-26T01:50:53.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.692+0000 7f3893200700 1 -- 172.21.15.1:0/2350946162 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f388c069730 con 0x7f388c0754a0 2024-04-26T01:50:53.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.696+0000 7f3888c00700 1 -- 172.21.15.1:0/2350946162 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f387c029070 con 0x7f388c0754a0 2024-04-26T01:50:53.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.698+0000 7f3888c00700 1 --2- 172.21.15.1:0/2350946162 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f38700a3cc0 0x7f38700a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:53.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.698+0000 7f3888c00700 1 -- 172.21.15.1:0/2350946162 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f387c0d1230 con 0x7f388c0754a0 2024-04-26T01:50:53.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.698+0000 7f388b400700 1 --2- 172.21.15.1:0/2350946162 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f38700a3cc0 0x7f38700a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:53.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.699+0000 7f388b400700 1 --2- 172.21.15.1:0/2350946162 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f38700a3cc0 0x7f38700a6180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f388c0767a0 tx=0x7f3878009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:53.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.702+0000 7f3888c00700 1 -- 172.21.15.1:0/2350946162 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f387c05dff0 con 0x7f388c0754a0 2024-04-26T01:50:53.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.936+0000 7f3893200700 1 -- 172.21.15.1:0/2350946162 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f388c062e80 con 0x7f38700a3cc0 2024-04-26T01:50:53.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.939+0000 7f3888c00700 1 -- 172.21.15.1:0/2350946162 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f388c062e80 con 0x7f38700a3cc0 2024-04-26T01:50:53.937 INFO:teuthology.orchestra.run.smithi001.stdout:HOST ADDR LABELS STATUS 2024-04-26T01:50:53.937 INFO:teuthology.orchestra.run.smithi001.stdout:smithi001 172.21.15.1 2024-04-26T01:50:53.938 INFO:teuthology.orchestra.run.smithi001.stdout:smithi097 172.21.15.97 2024-04-26T01:50:53.938 INFO:teuthology.orchestra.run.smithi001.stdout:smithi173 172.21.15.173 2024-04-26T01:50:53.938 INFO:teuthology.orchestra.run.smithi001.stdout:3 hosts in cluster 2024-04-26T01:50:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.942+0000 7f3893200700 1 -- 172.21.15.1:0/2350946162 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f38700a3cc0 msgr2=0x7f38700a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.942+0000 7f3893200700 1 --2- 172.21.15.1:0/2350946162 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f38700a3cc0 0x7f38700a6180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f388c0767a0 tx=0x7f3878009260 comp rx=0 tx=0).stop 2024-04-26T01:50:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.942+0000 7f3893200700 1 -- 172.21.15.1:0/2350946162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f388c0754a0 msgr2=0x7f388c19ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.942+0000 7f3893200700 1 --2- 172.21.15.1:0/2350946162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f388c0754a0 0x7f388c19ae80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f387c0047e0 tx=0x7f387c009510 comp rx=0 tx=0).stop 2024-04-26T01:50:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.942+0000 7f3893200700 1 -- 172.21.15.1:0/2350946162 shutdown_connections 2024-04-26T01:50:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.942+0000 7f3893200700 1 --2- 172.21.15.1:0/2350946162 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f38700a3cc0 0x7f38700a6180 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.942+0000 7f3893200700 1 --2- 172.21.15.1:0/2350946162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f388c10a980 0x7f388c1a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.942+0000 7f3893200700 1 --2- 172.21.15.1:0/2350946162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f388c0754a0 0x7f388c19ae80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.942+0000 7f3893200700 1 --2- 172.21.15.1:0/2350946162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c076a70 0x7f388c19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.942+0000 7f3893200700 1 -- 172.21.15.1:0/2350946162 >> 172.21.15.1:0/2350946162 conn(0x7f388c0fe010 msgr2=0x7f388c106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:53.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.943+0000 7f3893200700 1 -- 172.21.15.1:0/2350946162 shutdown_connections 2024-04-26T01:50:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:53.943+0000 7f3893200700 1 -- 172.21.15.1:0/2350946162 wait complete. 2024-04-26T01:50:54.699 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-26T01:50:54.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:54 smithi097 ceph-mon[32853]: pgmap v428: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:55.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:54 smithi173 ceph-mon[32553]: pgmap v428: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:55.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:54 smithi001 ceph-mon[27822]: pgmap v428: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:55.181 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:55.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:55 smithi097 ceph-mon[32853]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:56.062 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:55 smithi001 ceph-mon[27822]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:56.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:55 smithi173 ceph-mon[32553]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:56.920 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:56 smithi001 ceph-mon[27822]: pgmap v429: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:56.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:56 smithi097 ceph-mon[32853]: pgmap v429: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:57.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:56 smithi173 ceph-mon[32553]: pgmap v429: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:50:57.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.210+0000 7fb3d4a00700 1 -- 172.21.15.1:0/2554297739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d0104600 msgr2=0x7fb3d010aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:57.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.210+0000 7fb3d4a00700 1 --2- 172.21.15.1:0/2554297739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d0104600 0x7fb3d010aeb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800b0a0 tx=0x7fb3c800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.210+0000 7fb3d4a00700 1 -- 172.21.15.1:0/2554297739 shutdown_connections 2024-04-26T01:50:57.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.210+0000 7fb3d4a00700 1 --2- 172.21.15.1:0/2554297739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d0104600 0x7fb3d010aeb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.210+0000 7fb3d4a00700 1 --2- 172.21.15.1:0/2554297739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d0100c80 0x7fb3d01010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.210+0000 7fb3d4a00700 1 --2- 172.21.15.1:0/2554297739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3d0103c60 0x7fb3d01040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.210+0000 7fb3d4a00700 1 -- 172.21.15.1:0/2554297739 >> 172.21.15.1:0/2554297739 conn(0x7fb3d00fc5e0 msgr2=0x7fb3d00fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:57.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.210+0000 7fb3d4a00700 1 -- 172.21.15.1:0/2554297739 shutdown_connections 2024-04-26T01:50:57.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.211+0000 7fb3d4a00700 1 -- 172.21.15.1:0/2554297739 wait complete. 2024-04-26T01:50:57.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.212+0000 7fb3d4a00700 1 Processor -- start 2024-04-26T01:50:57.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.212+0000 7fb3d4a00700 1 -- start start 2024-04-26T01:50:57.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.212+0000 7fb3d4a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d0100c80 0x7fb3d019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:57.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.212+0000 7fb3d4a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3d0103c60 0x7fb3d019b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:57.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.212+0000 7fb3d4a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d0104600 0x7fb3d01a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:57.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.212+0000 7fb3d4a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d010e480 con 0x7fb3d0104600 2024-04-26T01:50:57.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.212+0000 7fb3d4a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d010e300 con 0x7fb3d0100c80 2024-04-26T01:50:57.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.212+0000 7fb3d4a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d010e600 con 0x7fb3d0103c60 2024-04-26T01:50:57.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.212+0000 7fb3cea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3d0103c60 0x7fb3d019b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:57.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.213+0000 7fb3cfe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d0104600 0x7fb3d01a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:57.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.213+0000 7fb3cea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3d0103c60 0x7fb3d019b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:51530/0 (socket says 172.21.15.1:51530) 2024-04-26T01:50:57.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.213+0000 7fb3cea00700 1 -- 172.21.15.1:0/2815982162 learned_addr learned my addr 172.21.15.1:0/2815982162 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:50:57.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.213+0000 7fb3cf400700 1 --2- 172.21.15.1:0/2815982162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d0100c80 0x7fb3d019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:57.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.213+0000 7fb3cea00700 1 -- 172.21.15.1:0/2815982162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d0100c80 msgr2=0x7fb3d019ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:57.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.213+0000 7fb3cea00700 1 --2- 172.21.15.1:0/2815982162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d0100c80 0x7fb3d019ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.213+0000 7fb3cea00700 1 -- 172.21.15.1:0/2815982162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d0104600 msgr2=0x7fb3d01a21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:57.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.213+0000 7fb3cea00700 1 --2- 172.21.15.1:0/2815982162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d0104600 0x7fb3d01a21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.213+0000 7fb3cea00700 1 -- 172.21.15.1:0/2815982162 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d01a28b0 con 0x7fb3d0103c60 2024-04-26T01:50:57.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.213+0000 7fb3cfe00700 1 --2- 172.21.15.1:0/2815982162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d0104600 0x7fb3d01a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:50:57.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.213+0000 7fb3cea00700 1 --2- 172.21.15.1:0/2815982162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3d0103c60 0x7fb3d019b1a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00e9a0 tx=0x7fb3bc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:57.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.214+0000 7fb3c7e00700 1 -- 172.21.15.1:0/2815982162 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc00ca10 con 0x7fb3d0103c60 2024-04-26T01:50:57.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.214+0000 7fb3c7e00700 1 -- 172.21.15.1:0/2815982162 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3bc00cbb0 con 0x7fb3d0103c60 2024-04-26T01:50:57.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.214+0000 7fb3d4a00700 1 -- 172.21.15.1:0/2815982162 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d01a2ba0 con 0x7fb3d0103c60 2024-04-26T01:50:57.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.214+0000 7fb3c7e00700 1 -- 172.21.15.1:0/2815982162 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc018680 con 0x7fb3d0103c60 2024-04-26T01:50:57.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.214+0000 7fb3d4a00700 1 -- 172.21.15.1:0/2815982162 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d01a3130 con 0x7fb3d0103c60 2024-04-26T01:50:57.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.215+0000 7fb3d4a00700 1 -- 172.21.15.1:0/2815982162 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3d0069730 con 0x7fb3d0103c60 2024-04-26T01:50:57.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.217+0000 7fb3c7e00700 1 -- 172.21.15.1:0/2815982162 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fb3bc010430 con 0x7fb3d0103c60 2024-04-26T01:50:57.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.218+0000 7fb3c7e00700 1 --2- 172.21.15.1:0/2815982162 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb3b40a3cc0 0x7fb3b40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:57.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.219+0000 7fb3c7e00700 1 -- 172.21.15.1:0/2815982162 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fb3bc0d05b0 con 0x7fb3d0103c60 2024-04-26T01:50:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.219+0000 7fb3cf400700 1 --2- 172.21.15.1:0/2815982162 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb3b40a3cc0 0x7fb3b40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.220+0000 7fb3cf400700 1 --2- 172.21.15.1:0/2815982162 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb3b40a3cc0 0x7fb3b40a6180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb3d019c140 tx=0x7fb3c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.223+0000 7fb3c7e00700 1 -- 172.21.15.1:0/2815982162 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb3bc09cf10 con 0x7fb3d0103c60 2024-04-26T01:50:57.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.467+0000 7fb3d4a00700 1 -- 172.21.15.1:0/2815982162 --> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fb3d0103170 con 0x7fb3b40a3cc0 2024-04-26T01:50:57.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.471+0000 7fb3c7e00700 1 -- 172.21.15.1:0/2815982162 <== mgr.14152 v2:172.21.15.1:6800/2140737121 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fb3d0103170 con 0x7fb3b40a3cc0 2024-04-26T01:50:57.469 INFO:teuthology.orchestra.run.smithi001.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-26T01:50:57.469 INFO:teuthology.orchestra.run.smithi001.stdout:smithi001 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513000D480DGN 447G No 5m ago Has a FileSystem, LVM detected 2024-04-26T01:50:57.469 INFO:teuthology.orchestra.run.smithi001.stdout:smithi097 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013T400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-04-26T01:50:57.470 INFO:teuthology.orchestra.run.smithi001.stdout:smithi173 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204019S400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-26T01:50:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.474+0000 7fb3d4a00700 1 -- 172.21.15.1:0/2815982162 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb3b40a3cc0 msgr2=0x7fb3b40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.474+0000 7fb3d4a00700 1 --2- 172.21.15.1:0/2815982162 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb3b40a3cc0 0x7fb3b40a6180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb3d019c140 tx=0x7fb3c0009260 comp rx=0 tx=0).stop 2024-04-26T01:50:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.474+0000 7fb3d4a00700 1 -- 172.21.15.1:0/2815982162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3d0103c60 msgr2=0x7fb3d019b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.474+0000 7fb3d4a00700 1 --2- 172.21.15.1:0/2815982162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3d0103c60 0x7fb3d019b1a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00e9a0 tx=0x7fb3bc00ecb0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.474+0000 7fb3d4a00700 1 -- 172.21.15.1:0/2815982162 shutdown_connections 2024-04-26T01:50:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.474+0000 7fb3d4a00700 1 --2- 172.21.15.1:0/2815982162 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fb3b40a3cc0 0x7fb3b40a6180 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.474+0000 7fb3d4a00700 1 --2- 172.21.15.1:0/2815982162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d0104600 0x7fb3d01a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.474+0000 7fb3d4a00700 1 --2- 172.21.15.1:0/2815982162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d0100c80 0x7fb3d019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.474+0000 7fb3d4a00700 1 --2- 172.21.15.1:0/2815982162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3d0103c60 0x7fb3d019b1a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.474+0000 7fb3d4a00700 1 -- 172.21.15.1:0/2815982162 >> 172.21.15.1:0/2815982162 conn(0x7fb3d00fc5e0 msgr2=0x7fb3d0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.475+0000 7fb3d4a00700 1 -- 172.21.15.1:0/2815982162 shutdown_connections 2024-04-26T01:50:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:50:57.475+0000 7fb3d4a00700 1 -- 172.21.15.1:0/2815982162 wait complete. 2024-04-26T01:50:58.242 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T01:50:58.251 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-04-26T01:50:58.251 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-26T01:50:58.724 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:50:58.972 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:58 smithi001 ceph-mon[27822]: pgmap v430: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:58.972 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:50:58 smithi001 ceph-mon[27822]: from='client.24985 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:58.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:58 smithi097 ceph-mon[32853]: pgmap v430: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:58.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:50:58 smithi097 ceph-mon[32853]: from='client.24985 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:59.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:58 smithi173 ceph-mon[32553]: pgmap v430: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:50:59.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:50:58 smithi173 ceph-mon[32553]: from='client.24985 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:00.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.947+0000 7f3e7a600700 1 -- 172.21.15.1:0/314138894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e74102aa0 msgr2=0x7f3e74102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:00.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.947+0000 7f3e7a600700 1 --2- 172.21.15.1:0/314138894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e74102aa0 0x7f3e74102ec0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3e640099c0 tx=0x7f3e64009cd0 comp rx=0 tx=0).stop 2024-04-26T01:51:00.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.947+0000 7f3e7a600700 1 -- 172.21.15.1:0/314138894 shutdown_connections 2024-04-26T01:51:00.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.947+0000 7f3e7a600700 1 --2- 172.21.15.1:0/314138894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e74104640 0x7f3e7410aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:00.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.947+0000 7f3e7a600700 1 --2- 172.21.15.1:0/314138894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e74102aa0 0x7f3e74102ec0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:00.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.947+0000 7f3e7a600700 1 --2- 172.21.15.1:0/314138894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e74103ca0 0x7f3e74104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:00.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.947+0000 7f3e7a600700 1 -- 172.21.15.1:0/314138894 >> 172.21.15.1:0/314138894 conn(0x7f3e740fe040 msgr2=0x7f3e74100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:00.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.947+0000 7f3e7a600700 1 -- 172.21.15.1:0/314138894 shutdown_connections 2024-04-26T01:51:00.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:00 smithi001 ceph-mon[27822]: pgmap v431: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:00.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.948+0000 7f3e7a600700 1 -- 172.21.15.1:0/314138894 wait complete. 2024-04-26T01:51:00.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.949+0000 7f3e7a600700 1 Processor -- start 2024-04-26T01:51:00.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.949+0000 7f3e7a600700 1 -- start start 2024-04-26T01:51:00.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.949+0000 7f3e7a600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e74102aa0 0x7f3e74196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:00.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.949+0000 7f3e7a600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e74103ca0 0x7f3e74196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:00.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.949+0000 7f3e7a600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e74104640 0x7f3e741af470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:00.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.949+0000 7f3e7a600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7410e560 con 0x7f3e74102aa0 2024-04-26T01:51:00.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.949+0000 7f3e7a600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7410e3e0 con 0x7f3e74104640 2024-04-26T01:51:00.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.949+0000 7f3e7a600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7410e6e0 con 0x7f3e74103ca0 2024-04-26T01:51:00.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.950+0000 7f3e73400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e74102aa0 0x7f3e74196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:00.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.950+0000 7f3e72a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e74103ca0 0x7f3e74196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:00.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.950+0000 7f3e73400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e74102aa0 0x7f3e74196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59354/0 (socket says 172.21.15.1:59354) 2024-04-26T01:51:00.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.950+0000 7f3e73400700 1 -- 172.21.15.1:0/3037429848 learned_addr learned my addr 172.21.15.1:0/3037429848 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:00.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.950+0000 7f3e73e00700 1 --2- 172.21.15.1:0/3037429848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e74104640 0x7f3e741af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:00.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.950+0000 7f3e73e00700 1 -- 172.21.15.1:0/3037429848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e74103ca0 msgr2=0x7f3e74196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:00.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.950+0000 7f3e73e00700 1 --2- 172.21.15.1:0/3037429848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e74103ca0 0x7f3e74196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:00.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.950+0000 7f3e73e00700 1 -- 172.21.15.1:0/3037429848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e74102aa0 msgr2=0x7f3e74196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:00.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.950+0000 7f3e73e00700 1 --2- 172.21.15.1:0/3037429848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e74102aa0 0x7f3e74196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:00.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.950+0000 7f3e73e00700 1 -- 172.21.15.1:0/3037429848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7419c1c0 con 0x7f3e74104640 2024-04-26T01:51:00.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.951+0000 7f3e73e00700 1 --2- 172.21.15.1:0/3037429848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e74104640 0x7f3e741af470 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00efc0 tx=0x7f3e6c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:00.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.951+0000 7f3e6be00700 1 -- 172.21.15.1:0/3037429848 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c0092e0 con 0x7f3e74104640 2024-04-26T01:51:00.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.951+0000 7f3e7a600700 1 -- 172.21.15.1:0/3037429848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e741af9b0 con 0x7f3e74104640 2024-04-26T01:51:00.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.951+0000 7f3e6be00700 1 -- 172.21.15.1:0/3037429848 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e6c004bb0 con 0x7f3e74104640 2024-04-26T01:51:00.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.951+0000 7f3e6be00700 1 -- 172.21.15.1:0/3037429848 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c004020 con 0x7f3e74104640 2024-04-26T01:51:00.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.951+0000 7f3e7a600700 1 -- 172.21.15.1:0/3037429848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e741afe90 con 0x7f3e74104640 2024-04-26T01:51:00.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.952+0000 7f3e7a600700 1 -- 172.21.15.1:0/3037429848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e74069730 con 0x7f3e74104640 2024-04-26T01:51:00.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.954+0000 7f3e6be00700 1 -- 172.21.15.1:0/3037429848 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f3e6c01b030 con 0x7f3e74104640 2024-04-26T01:51:00.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.955+0000 7f3e6be00700 1 --2- 172.21.15.1:0/3037429848 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f3e580a3cc0 0x7f3e580a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:00.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.956+0000 7f3e6be00700 1 -- 172.21.15.1:0/3037429848 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f3e6c0cfe40 con 0x7f3e74104640 2024-04-26T01:51:00.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.956+0000 7f3e73400700 1 --2- 172.21.15.1:0/3037429848 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f3e580a3cc0 0x7f3e580a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:00.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.956+0000 7f3e73400700 1 --2- 172.21.15.1:0/3037429848 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f3e580a3cc0 0x7f3e580a6180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3e64003820 tx=0x7f3e64005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:00.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:00.960+0000 7f3e6be00700 1 -- 172.21.15.1:0/3037429848 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3e6c022020 con 0x7f3e74104640 2024-04-26T01:51:00.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:00 smithi097 ceph-mon[32853]: pgmap v431: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:01.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:00 smithi173 ceph-mon[32553]: pgmap v431: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:01.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:01.241+0000 7f3e7a600700 1 -- 172.21.15.1:0/3037429848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f3e74063650 con 0x7f3e74104640 2024-04-26T01:51:01.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:01.242+0000 7f3e6be00700 1 -- 172.21.15.1:0/3037429848 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3030 (secure 0 0 0) 0x7f3e6c010040 con 0x7f3e74104640 2024-04-26T01:51:01.241 INFO:teuthology.orchestra.run.smithi001.stdout:e25 2024-04-26T01:51:01.241 INFO:teuthology.orchestra.run.smithi001.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T01:51:01.241 INFO:teuthology.orchestra.run.smithi001.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T01:51:01.241 INFO:teuthology.orchestra.run.smithi001.stdout:legacy client fscid: 1 2024-04-26T01:51:01.241 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:51:01.241 INFO:teuthology.orchestra.run.smithi001.stdout:Filesystem 'cephfs' (1) 2024-04-26T01:51:01.241 INFO:teuthology.orchestra.run.smithi001.stdout:fs_name cephfs 2024-04-26T01:51:01.241 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 25 2024-04-26T01:51:01.241 INFO:teuthology.orchestra.run.smithi001.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-26T01:51:01.242 INFO:teuthology.orchestra.run.smithi001.stdout:created 2024-04-26T01:47:29.999521+0000 2024-04-26T01:51:01.242 INFO:teuthology.orchestra.run.smithi001.stdout:modified 2024-04-26T01:47:54.077766+0000 2024-04-26T01:51:01.242 INFO:teuthology.orchestra.run.smithi001.stdout:tableserver 0 2024-04-26T01:51:01.242 INFO:teuthology.orchestra.run.smithi001.stdout:root 0 2024-04-26T01:51:01.242 INFO:teuthology.orchestra.run.smithi001.stdout:session_timeout 300 2024-04-26T01:51:01.242 INFO:teuthology.orchestra.run.smithi001.stdout:session_autoclose 300 2024-04-26T01:51:01.242 INFO:teuthology.orchestra.run.smithi001.stdout:max_file_size 1099511627776 2024-04-26T01:51:01.242 INFO:teuthology.orchestra.run.smithi001.stdout:max_xattr_size 65536 2024-04-26T01:51:01.242 INFO:teuthology.orchestra.run.smithi001.stdout:required_client_features {} 2024-04-26T01:51:01.242 INFO:teuthology.orchestra.run.smithi001.stdout:last_failure 0 2024-04-26T01:51:01.242 INFO:teuthology.orchestra.run.smithi001.stdout:last_failure_osd_epoch 0 2024-04-26T01:51:01.243 INFO:teuthology.orchestra.run.smithi001.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T01:51:01.243 INFO:teuthology.orchestra.run.smithi001.stdout:max_mds 3 2024-04-26T01:51:01.243 INFO:teuthology.orchestra.run.smithi001.stdout:in 0,1,2 2024-04-26T01:51:01.243 INFO:teuthology.orchestra.run.smithi001.stdout:up {0=24518,1=24514,2=24499} 2024-04-26T01:51:01.243 INFO:teuthology.orchestra.run.smithi001.stdout:failed 2024-04-26T01:51:01.243 INFO:teuthology.orchestra.run.smithi001.stdout:damaged 2024-04-26T01:51:01.243 INFO:teuthology.orchestra.run.smithi001.stdout:stopped 2024-04-26T01:51:01.243 INFO:teuthology.orchestra.run.smithi001.stdout:data_pools [3] 2024-04-26T01:51:01.243 INFO:teuthology.orchestra.run.smithi001.stdout:metadata_pool 2 2024-04-26T01:51:01.243 INFO:teuthology.orchestra.run.smithi001.stdout:inline_data disabled 2024-04-26T01:51:01.243 INFO:teuthology.orchestra.run.smithi001.stdout:balancer 2024-04-26T01:51:01.243 INFO:teuthology.orchestra.run.smithi001.stdout:bal_rank_mask -1 2024-04-26T01:51:01.243 INFO:teuthology.orchestra.run.smithi001.stdout:standby_count_wanted 0 2024-04-26T01:51:01.244 INFO:teuthology.orchestra.run.smithi001.stdout:qdb_cluster leader: 24518 members: 24514,24518,24499 2024-04-26T01:51:01.244 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.b{0:24518} state up:active seq 7 addr [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:51:01.244 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.f{0:24487} state up:standby-replay seq 1 addr [v2:172.21.15.173:6834/2952462325,v1:172.21.15.173:6835/2952462325] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:51:01.244 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.i{1:24514} state up:active seq 14 addr [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:51:01.244 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.h{1:24479} state up:standby-replay seq 1 addr [v2:172.21.15.97:6836/3607142398,v1:172.21.15.97:6837/3607142398] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:51:01.244 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.c{2:24499} state up:active seq 15 addr [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:51:01.244 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.e{2:24464} state up:standby-replay seq 1 addr [v2:172.21.15.97:6834/763675408,v1:172.21.15.97:6835/763675408] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:51:01.244 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:51:01.244 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:51:01.244 INFO:teuthology.orchestra.run.smithi001.stdout:Standby daemons: 2024-04-26T01:51:01.244 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:51:01.244 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.j{-1:14364} state up:standby seq 1 addr [v2:172.21.15.1:6834/1477554934,v1:172.21.15.1:6835/1477554934] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:51:01.244 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.d{-1:14370} state up:standby seq 1 addr [v2:172.21.15.1:6836/76325266,v1:172.21.15.1:6837/76325266] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:51:01.244 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.g{-1:14400} state up:standby seq 1 addr [v2:172.21.15.1:6838/42404130,v1:172.21.15.1:6839/42404130] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:51:01.244 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.a{-1:14406} state up:standby seq 1 addr [v2:172.21.15.1:6840/3885053557,v1:172.21.15.1:6841/3885053557] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:51:01.245 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.k{-1:24434} state up:standby seq 1 addr [v2:172.21.15.97:6832/4061367802,v1:172.21.15.97:6833/4061367802] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:51:01.245 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.l{-1:24451} state up:standby seq 1 addr [v2:172.21.15.173:6832/188689332,v1:172.21.15.173:6833/188689332] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:51:01.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:01.245+0000 7f3e7a600700 1 -- 172.21.15.1:0/3037429848 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f3e580a3cc0 msgr2=0x7f3e580a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:01.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:01.245+0000 7f3e7a600700 1 --2- 172.21.15.1:0/3037429848 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f3e580a3cc0 0x7f3e580a6180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3e64003820 tx=0x7f3e64005fb0 comp rx=0 tx=0).stop 2024-04-26T01:51:01.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:01.245+0000 7f3e7a600700 1 -- 172.21.15.1:0/3037429848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e74104640 msgr2=0x7f3e741af470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:01.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:01.245+0000 7f3e7a600700 1 --2- 172.21.15.1:0/3037429848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e74104640 0x7f3e741af470 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00efc0 tx=0x7f3e6c00c3a0 comp rx=0 tx=0).stop 2024-04-26T01:51:01.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:01.245+0000 7f3e7a600700 1 -- 172.21.15.1:0/3037429848 shutdown_connections 2024-04-26T01:51:01.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:01.245+0000 7f3e7a600700 1 --2- 172.21.15.1:0/3037429848 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f3e580a3cc0 0x7f3e580a6180 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:01.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:01.245+0000 7f3e7a600700 1 --2- 172.21.15.1:0/3037429848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e74104640 0x7f3e741af470 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:01.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:01.245+0000 7f3e7a600700 1 --2- 172.21.15.1:0/3037429848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e74102aa0 0x7f3e74196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:01.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:01.245+0000 7f3e7a600700 1 --2- 172.21.15.1:0/3037429848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e74103ca0 0x7f3e74196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:01.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:01.245+0000 7f3e7a600700 1 -- 172.21.15.1:0/3037429848 >> 172.21.15.1:0/3037429848 conn(0x7f3e740fe040 msgr2=0x7f3e74104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:01.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:01.246+0000 7f3e7a600700 1 -- 172.21.15.1:0/3037429848 shutdown_connections 2024-04-26T01:51:01.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:01.246+0000 7f3e7a600700 1 -- 172.21.15.1:0/3037429848 wait complete. 2024-04-26T01:51:01.246 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-04-26T01:51:01.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:01 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3037429848' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T01:51:01.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:01 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3037429848' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T01:51:02.017 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-26T01:51:02.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:01 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3037429848' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T01:51:02.496 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:02.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:02 smithi097 ceph-mon[32853]: pgmap v432: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:51:02.998 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:02 smithi001 ceph-mon[27822]: pgmap v432: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:51:03.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:02 smithi173 ceph-mon[32553]: pgmap v432: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:51:03.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:03 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:51:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:03 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:51:04.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:03 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:51:04.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.703+0000 7f8261600700 1 -- 172.21.15.1:0/983363630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f825c076a70 msgr2=0x7f825c074ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:04.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.703+0000 7f8261600700 1 --2- 172.21.15.1:0/983363630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f825c076a70 0x7f825c074ed0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f824c009990 tx=0x7f824c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:51:04.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.703+0000 7f8261600700 1 -- 172.21.15.1:0/983363630 shutdown_connections 2024-04-26T01:51:04.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.703+0000 7f8261600700 1 --2- 172.21.15.1:0/983363630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f825c10a970 0x7f825c10ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:04.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.703+0000 7f8261600700 1 --2- 172.21.15.1:0/983363630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f825c076a70 0x7f825c074ed0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:04.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.703+0000 7f8261600700 1 --2- 172.21.15.1:0/983363630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f825c0754a0 0x7f825c075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:04.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.703+0000 7f8261600700 1 -- 172.21.15.1:0/983363630 >> 172.21.15.1:0/983363630 conn(0x7f825c0fe040 msgr2=0x7f825c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:04.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.703+0000 7f8261600700 1 -- 172.21.15.1:0/983363630 shutdown_connections 2024-04-26T01:51:04.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.703+0000 7f8261600700 1 -- 172.21.15.1:0/983363630 wait complete. 2024-04-26T01:51:04.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.704+0000 7f8261600700 1 Processor -- start 2024-04-26T01:51:04.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.704+0000 7f8261600700 1 -- start start 2024-04-26T01:51:04.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.705+0000 7f8261600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f825c0754a0 0x7f825c19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:04.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.705+0000 7f8261600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f825c076a70 0x7f825c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:04.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.705+0000 7f8261600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f825c10a970 0x7f825c1a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:04.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.705+0000 7f8261600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f825c110780 con 0x7f825c10a970 2024-04-26T01:51:04.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.705+0000 7f8261600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f825c110600 con 0x7f825c0754a0 2024-04-26T01:51:04.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.705+0000 7f8261600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f825c110900 con 0x7f825c076a70 2024-04-26T01:51:04.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.705+0000 7f825aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f825c0754a0 0x7f825c19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:04.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.705+0000 7f825aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f825c0754a0 0x7f825c19ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:35942/0 (socket says 172.21.15.1:35942) 2024-04-26T01:51:04.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.705+0000 7f825aa00700 1 -- 172.21.15.1:0/363716941 learned_addr learned my addr 172.21.15.1:0/363716941 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:04.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.705+0000 7f825b400700 1 --2- 172.21.15.1:0/363716941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f825c10a970 0x7f825c1a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:04.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.705+0000 7f825a000700 1 --2- 172.21.15.1:0/363716941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f825c076a70 0x7f825c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:04.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.706+0000 7f825aa00700 1 -- 172.21.15.1:0/363716941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f825c076a70 msgr2=0x7f825c19b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:04.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.706+0000 7f825aa00700 1 --2- 172.21.15.1:0/363716941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f825c076a70 0x7f825c19b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:04.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.706+0000 7f825aa00700 1 -- 172.21.15.1:0/363716941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f825c10a970 msgr2=0x7f825c1a2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:04.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.706+0000 7f825aa00700 1 --2- 172.21.15.1:0/363716941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f825c10a970 0x7f825c1a2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:04.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.706+0000 7f825aa00700 1 -- 172.21.15.1:0/363716941 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825c1a2aa0 con 0x7f825c0754a0 2024-04-26T01:51:04.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.706+0000 7f825a000700 1 --2- 172.21.15.1:0/363716941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f825c076a70 0x7f825c19b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:04.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.706+0000 7f825aa00700 1 --2- 172.21.15.1:0/363716941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f825c0754a0 0x7f825c19ae40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f824c0047b0 tx=0x7f824c009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:04.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.706+0000 7f8253400700 1 -- 172.21.15.1:0/363716941 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c026070 con 0x7f825c0754a0 2024-04-26T01:51:04.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.707+0000 7f8253400700 1 -- 172.21.15.1:0/363716941 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f824c021410 con 0x7f825c0754a0 2024-04-26T01:51:04.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.707+0000 7f8261600700 1 -- 172.21.15.1:0/363716941 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825c1a2d30 con 0x7f825c0754a0 2024-04-26T01:51:04.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.707+0000 7f8253400700 1 -- 172.21.15.1:0/363716941 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c017650 con 0x7f825c0754a0 2024-04-26T01:51:04.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.707+0000 7f8261600700 1 -- 172.21.15.1:0/363716941 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f825c1a3180 con 0x7f825c0754a0 2024-04-26T01:51:04.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.708+0000 7f8261600700 1 -- 172.21.15.1:0/363716941 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f825c069730 con 0x7f825c0754a0 2024-04-26T01:51:04.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.710+0000 7f8253400700 1 -- 172.21.15.1:0/363716941 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f824c029070 con 0x7f825c0754a0 2024-04-26T01:51:04.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.711+0000 7f8253400700 1 --2- 172.21.15.1:0/363716941 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f82400a3c40 0x7f82400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:04.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.711+0000 7f8253400700 1 -- 172.21.15.1:0/363716941 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f824c0d12c0 con 0x7f825c0754a0 2024-04-26T01:51:04.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.711+0000 7f825a000700 1 --2- 172.21.15.1:0/363716941 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f82400a3c40 0x7f82400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:04.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.712+0000 7f825a000700 1 --2- 172.21.15.1:0/363716941 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f82400a3c40 0x7f82400a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f825c0767a0 tx=0x7f8248009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:04.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.715+0000 7f8253400700 1 -- 172.21.15.1:0/363716941 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f824c05dff0 con 0x7f825c0754a0 2024-04-26T01:51:04.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:04 smithi001 ceph-mon[27822]: pgmap v433: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:04.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:04 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:51:04.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:04 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:51:04.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:04 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:51:04.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:04 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:51:04.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:04 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:04.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:04 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:51:04.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:04 smithi001 ceph-mon[27822]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:51:04.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.956+0000 7f8261600700 1 -- 172.21.15.1:0/363716941 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f825c02df80 con 0x7f825c0754a0 2024-04-26T01:51:04.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.957+0000 7f8253400700 1 -- 172.21.15.1:0/363716941 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) ==== 56+0+5294 (secure 0 0 0) 0x7f824c013070 con 0x7f825c0754a0 2024-04-26T01:51:04.956 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 92 2024-04-26T01:51:04.957 INFO:teuthology.orchestra.run.smithi001.stdout:fsid 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T01:51:04.957 INFO:teuthology.orchestra.run.smithi001.stdout:created 2024-04-26T01:38:03.531182+0000 2024-04-26T01:51:04.957 INFO:teuthology.orchestra.run.smithi001.stdout:modified 2024-04-26T01:48:22.381867+0000 2024-04-26T01:51:04.957 INFO:teuthology.orchestra.run.smithi001.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-26T01:51:04.957 INFO:teuthology.orchestra.run.smithi001.stdout:crush_version 26 2024-04-26T01:51:04.957 INFO:teuthology.orchestra.run.smithi001.stdout:full_ratio 0.95 2024-04-26T01:51:04.957 INFO:teuthology.orchestra.run.smithi001.stdout:backfillfull_ratio 0.9 2024-04-26T01:51:04.957 INFO:teuthology.orchestra.run.smithi001.stdout:nearfull_ratio 0.85 2024-04-26T01:51:04.957 INFO:teuthology.orchestra.run.smithi001.stdout:require_min_compat_client luminous 2024-04-26T01:51:04.957 INFO:teuthology.orchestra.run.smithi001.stdout:min_compat_client luminous 2024-04-26T01:51:04.957 INFO:teuthology.orchestra.run.smithi001.stdout:require_osd_release squid 2024-04-26T01:51:04.958 INFO:teuthology.orchestra.run.smithi001.stdout:stretch_mode_enabled false 2024-04-26T01:51:04.958 INFO:teuthology.orchestra.run.smithi001.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-26T01:51:04.958 INFO:teuthology.orchestra.run.smithi001.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-26T01:51:04.958 INFO:teuthology.orchestra.run.smithi001.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 90 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-26T01:51:04.958 INFO:teuthology.orchestra.run.smithi001.stdout:max_osd 12 2024-04-26T01:51:04.958 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0 up in weight 1 up_from 8 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6802/3176633965,v1:172.21.15.1:6803/3176633965] [v2:172.21.15.1:6804/3176633965,v1:172.21.15.1:6805/3176633965] exists,up bb10ae12-09ee-4d6b-bc78-8c6016613b81 2024-04-26T01:51:04.958 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 up in weight 1 up_from 13 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6800/3755022621,v1:172.21.15.97:6801/3755022621] [v2:172.21.15.97:6802/3755022621,v1:172.21.15.97:6803/3755022621] exists,up acd12888-6ff6-40bc-99e7-e5c395e85b1c 2024-04-26T01:51:04.958 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6800/734861771,v1:172.21.15.173:6801/734861771] [v2:172.21.15.173:6802/734861771,v1:172.21.15.173:6803/734861771] exists,up 79971508-e256-4145-8111-a75666e886fc 2024-04-26T01:51:04.958 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 up in weight 1 up_from 25 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6810/3349961961,v1:172.21.15.1:6811/3349961961] [v2:172.21.15.1:6812/3349961961,v1:172.21.15.1:6813/3349961961] exists,up a8ccc4a9-0f31-446a-8bcb-4691532397c2 2024-04-26T01:51:04.958 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 up in weight 1 up_from 33 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6808/2494639397,v1:172.21.15.97:6809/2494639397] [v2:172.21.15.97:6810/2494639397,v1:172.21.15.97:6811/2494639397] exists,up 5b724648-4df7-40fc-89d5-55f170cdd806 2024-04-26T01:51:04.958 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5 up in weight 1 up_from 39 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6808/1026507957,v1:172.21.15.173:6809/1026507957] [v2:172.21.15.173:6810/1026507957,v1:172.21.15.173:6811/1026507957] exists,up a5a24a9c-3007-4b87-8562-40c98c41ef8b 2024-04-26T01:51:04.959 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6 up in weight 1 up_from 47 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6818/444030587,v1:172.21.15.1:6819/444030587] [v2:172.21.15.1:6820/444030587,v1:172.21.15.1:6821/444030587] exists,up 97cd2e88-a56c-4b1a-8104-65b23a77986a 2024-04-26T01:51:04.959 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6816/2600022531,v1:172.21.15.97:6817/2600022531] [v2:172.21.15.97:6818/2600022531,v1:172.21.15.97:6819/2600022531] exists,up afc0c2dd-e112-4358-b3aa-944366d5d383 2024-04-26T01:51:04.959 INFO:teuthology.orchestra.run.smithi001.stdout:osd.8 up in weight 1 up_from 61 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6816/1334205527,v1:172.21.15.173:6817/1334205527] [v2:172.21.15.173:6818/1334205527,v1:172.21.15.173:6819/1334205527] exists,up 41fa612b-793e-44da-af8b-44d17a9de303 2024-04-26T01:51:04.959 INFO:teuthology.orchestra.run.smithi001.stdout:osd.9 up in weight 1 up_from 69 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6826/2973486839,v1:172.21.15.1:6827/2973486839] [v2:172.21.15.1:6828/2973486839,v1:172.21.15.1:6829/2973486839] exists,up bbbb163b-f8c4-4be5-8f80-82445e8d407f 2024-04-26T01:51:04.959 INFO:teuthology.orchestra.run.smithi001.stdout:osd.10 up in weight 1 up_from 74 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6824/1693425268,v1:172.21.15.97:6825/1693425268] [v2:172.21.15.97:6826/1693425268,v1:172.21.15.97:6827/1693425268] exists,up 1b447b34-0600-4727-826d-b3d68adfb18d 2024-04-26T01:51:04.959 INFO:teuthology.orchestra.run.smithi001.stdout:osd.11 up in weight 1 up_from 82 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6824/1141524960,v1:172.21.15.173:6825/1141524960] [v2:172.21.15.173:6826/1141524960,v1:172.21.15.173:6827/1141524960] exists,up f439aae1-f50a-4176-889d-36fa5131541a 2024-04-26T01:51:04.959 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 2.8 [2,11] 2024-04-26T01:51:04.959 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 2.b [8,11] 2024-04-26T01:51:04.959 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 2.10 [2,4] 2024-04-26T01:51:04.959 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.3 [4,9] 2024-04-26T01:51:04.959 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.6 [3,8] 2024-04-26T01:51:04.959 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.e [5,9] 2024-04-26T01:51:04.959 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.14 [4,6] 2024-04-26T01:51:04.959 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.25 [5,9] 2024-04-26T01:51:04.960 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.27 [4,8] 2024-04-26T01:51:04.960 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.3a [4,7] 2024-04-26T01:51:04.960 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6801/1870580982 expires 2024-04-27T01:39:20.645943+0000 2024-04-26T01:51:04.960 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6800/1870580982 expires 2024-04-27T01:39:20.645943+0000 2024-04-26T01:51:04.960 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/1000222947 expires 2024-04-27T01:39:20.645943+0000 2024-04-26T01:51:04.960 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/457958597 expires 2024-04-27T01:39:20.645943+0000 2024-04-26T01:51:04.960 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6801/810889357 expires 2024-04-27T01:38:45.974031+0000 2024-04-26T01:51:04.960 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6800/810889357 expires 2024-04-27T01:38:45.974031+0000 2024-04-26T01:51:04.960 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/2715798962 expires 2024-04-27T01:38:45.974031+0000 2024-04-26T01:51:04.960 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/1923058728 expires 2024-04-27T01:39:20.645943+0000 2024-04-26T01:51:04.960 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/2489038777 expires 2024-04-27T01:38:45.974031+0000 2024-04-26T01:51:04.960 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/2869502448 expires 2024-04-27T01:38:45.974031+0000 2024-04-26T01:51:04.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.962+0000 7f8261600700 1 -- 172.21.15.1:0/363716941 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f82400a3c40 msgr2=0x7f82400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:04.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.962+0000 7f8261600700 1 --2- 172.21.15.1:0/363716941 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f82400a3c40 0x7f82400a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f825c0767a0 tx=0x7f8248009260 comp rx=0 tx=0).stop 2024-04-26T01:51:04.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.962+0000 7f8261600700 1 -- 172.21.15.1:0/363716941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f825c0754a0 msgr2=0x7f825c19ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:04.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.962+0000 7f8261600700 1 --2- 172.21.15.1:0/363716941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f825c0754a0 0x7f825c19ae40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f824c0047b0 tx=0x7f824c009510 comp rx=0 tx=0).stop 2024-04-26T01:51:04.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.962+0000 7f8261600700 1 -- 172.21.15.1:0/363716941 shutdown_connections 2024-04-26T01:51:04.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.962+0000 7f8261600700 1 --2- 172.21.15.1:0/363716941 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f82400a3c40 0x7f82400a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:04.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.962+0000 7f8261600700 1 --2- 172.21.15.1:0/363716941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f825c10a970 0x7f825c1a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:04.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.962+0000 7f8261600700 1 --2- 172.21.15.1:0/363716941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f825c0754a0 0x7f825c19ae40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:04.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.962+0000 7f8261600700 1 --2- 172.21.15.1:0/363716941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f825c076a70 0x7f825c19b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:04.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.962+0000 7f8261600700 1 -- 172.21.15.1:0/363716941 >> 172.21.15.1:0/363716941 conn(0x7f825c0fe040 msgr2=0x7f825c106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:04.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.962+0000 7f8261600700 1 -- 172.21.15.1:0/363716941 shutdown_connections 2024-04-26T01:51:04.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:04.963+0000 7f8261600700 1 -- 172.21.15.1:0/363716941 wait complete. 2024-04-26T01:51:04.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:04 smithi097 ceph-mon[32853]: pgmap v433: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:04.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:04 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:51:04.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:04 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:51:04.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:04 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:51:04.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:04 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:51:04.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:04 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:04.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:04 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:51:04.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:04 smithi097 ceph-mon[32853]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:51:05.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:04 smithi173 ceph-mon[32553]: pgmap v433: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:05.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:04 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:51:05.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:04 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:51:05.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:04 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:51:05.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:04 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:51:05.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:04 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:05.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:04 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:51:05.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:04 smithi173 ceph-mon[32553]: from='mgr.14152 172.21.15.1:0/3199499842' entity='mgr.x' 2024-04-26T01:51:05.746 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-26T01:51:05.758 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:51:05.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:05 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/363716941' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T01:51:06.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:05 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/363716941' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T01:51:06.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:05 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/363716941' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T01:51:06.222 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:06.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:06 smithi097 ceph-mon[32853]: pgmap v434: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:07.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:06 smithi173 ceph-mon[32553]: pgmap v434: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:07.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:06 smithi001 ceph-mon[27822]: pgmap v434: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:07.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:07 smithi001 ceph-mon[27822]: pgmap v435: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:08.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:07 smithi173 ceph-mon[32553]: pgmap v435: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:08.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:07 smithi097 ceph-mon[32853]: pgmap v435: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:08.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.371+0000 7fc72f800700 1 -- 172.21.15.1:0/3313597641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc728102300 msgr2=0x7fc72810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:08.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.371+0000 7fc72f800700 1 --2- 172.21.15.1:0/3313597641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc728102300 0x7fc72810e7f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc72000b0a0 tx=0x7fc72000b3b0 comp rx=0 tx=0).stop 2024-04-26T01:51:08.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.371+0000 7fc72f800700 1 -- 172.21.15.1:0/3313597641 shutdown_connections 2024-04-26T01:51:08.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.371+0000 7fc72f800700 1 --2- 172.21.15.1:0/3313597641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc728102300 0x7fc72810e7f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:08.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.371+0000 7fc72f800700 1 --2- 172.21.15.1:0/3313597641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc728107990 0x7fc728107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:08.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.371+0000 7fc72f800700 1 --2- 172.21.15.1:0/3313597641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc728101940 0x7fc728101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:08.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.371+0000 7fc72f800700 1 -- 172.21.15.1:0/3313597641 >> 172.21.15.1:0/3313597641 conn(0x7fc7280788f0 msgr2=0x7fc728078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:08.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.371+0000 7fc72f800700 1 -- 172.21.15.1:0/3313597641 shutdown_connections 2024-04-26T01:51:08.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.371+0000 7fc72f800700 1 -- 172.21.15.1:0/3313597641 wait complete. 2024-04-26T01:51:08.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.373+0000 7fc72f800700 1 Processor -- start 2024-04-26T01:51:08.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.373+0000 7fc72f800700 1 -- start start 2024-04-26T01:51:08.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.373+0000 7fc72f800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc728101940 0x7fc72819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:08.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.373+0000 7fc72f800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc728102300 0x7fc72819b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:08.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.374+0000 7fc72f800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc728107990 0x7fc72819fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:08.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.374+0000 7fc72f800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc728111b30 con 0x7fc728101940 2024-04-26T01:51:08.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.374+0000 7fc72f800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc7281119b0 con 0x7fc728107990 2024-04-26T01:51:08.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.374+0000 7fc72f800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc728111cb0 con 0x7fc728102300 2024-04-26T01:51:08.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.374+0000 7fc72ce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc728101940 0x7fc72819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:08.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.374+0000 7fc72ce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc728101940 0x7fc72819b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59758/0 (socket says 172.21.15.1:59758) 2024-04-26T01:51:08.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.374+0000 7fc72ce00700 1 -- 172.21.15.1:0/1377173544 learned_addr learned my addr 172.21.15.1:0/1377173544 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:08.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.374+0000 7fc727e00700 1 --2- 172.21.15.1:0/1377173544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc728102300 0x7fc72819b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:08.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.374+0000 7fc72d800700 1 --2- 172.21.15.1:0/1377173544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc728107990 0x7fc72819fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:08.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.374+0000 7fc72d800700 1 -- 172.21.15.1:0/1377173544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc728102300 msgr2=0x7fc72819b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:08.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.374+0000 7fc72d800700 1 --2- 172.21.15.1:0/1377173544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc728102300 0x7fc72819b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:08.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.374+0000 7fc72d800700 1 -- 172.21.15.1:0/1377173544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc728101940 msgr2=0x7fc72819b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:08.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.374+0000 7fc72d800700 1 --2- 172.21.15.1:0/1377173544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc728101940 0x7fc72819b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:08.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.374+0000 7fc72d800700 1 -- 172.21.15.1:0/1377173544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7281a02f0 con 0x7fc728107990 2024-04-26T01:51:08.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.374+0000 7fc72ce00700 1 --2- 172.21.15.1:0/1377173544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc728101940 0x7fc72819b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:08.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.375+0000 7fc72d800700 1 --2- 172.21.15.1:0/1377173544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc728107990 0x7fc72819fbb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc72000b070 tx=0x7fc720004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:08.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.375+0000 7fc725600700 1 -- 172.21.15.1:0/1377173544 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc720021070 con 0x7fc728107990 2024-04-26T01:51:08.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.375+0000 7fc725600700 1 -- 172.21.15.1:0/1377173544 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc720007e30 con 0x7fc728107990 2024-04-26T01:51:08.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.375+0000 7fc725600700 1 -- 172.21.15.1:0/1377173544 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7200045f0 con 0x7fc728107990 2024-04-26T01:51:08.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.375+0000 7fc72f800700 1 -- 172.21.15.1:0/1377173544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7281a0580 con 0x7fc728107990 2024-04-26T01:51:08.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.375+0000 7fc72f800700 1 -- 172.21.15.1:0/1377173544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7281a7e60 con 0x7fc728107990 2024-04-26T01:51:08.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.376+0000 7fc72f800700 1 -- 172.21.15.1:0/1377173544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc728103060 con 0x7fc728107990 2024-04-26T01:51:08.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.379+0000 7fc725600700 1 -- 172.21.15.1:0/1377173544 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fc720007980 con 0x7fc728107990 2024-04-26T01:51:08.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.380+0000 7fc725600700 1 --2- 172.21.15.1:0/1377173544 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fc7100a3c40 0x7fc7100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:08.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.380+0000 7fc725600700 1 -- 172.21.15.1:0/1377173544 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fc7200d11c0 con 0x7fc728107990 2024-04-26T01:51:08.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.380+0000 7fc72ce00700 1 --2- 172.21.15.1:0/1377173544 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fc7100a3c40 0x7fc7100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:08.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.381+0000 7fc72ce00700 1 --2- 172.21.15.1:0/1377173544 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fc7100a3c40 0x7fc7100a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009680 tx=0x7fc71c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:08.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.382+0000 7fc725600700 1 -- 172.21.15.1:0/1377173544 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc72009dbb0 con 0x7fc728107990 2024-04-26T01:51:08.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.659+0000 7fc72f800700 1 -- 172.21.15.1:0/1377173544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7281a8030 con 0x7fc728107990 2024-04-26T01:51:08.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.661+0000 7fc725600700 1 -- 172.21.15.1:0/1377173544 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11841 (secure 0 0 0) 0x7fc72009d4b0 con 0x7fc728107990 2024-04-26T01:51:08.662 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:51:08.663 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:47:54.077766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2952462325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2952462325},{"type":"v1","addr":"172.21.15.173:6835","nonce":2952462325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24514,24518,24499]},"id":1}]} 2024-04-26T01:51:08.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.664+0000 7fc72f800700 1 -- 172.21.15.1:0/1377173544 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fc7100a3c40 msgr2=0x7fc7100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:08.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.664+0000 7fc72f800700 1 --2- 172.21.15.1:0/1377173544 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fc7100a3c40 0x7fc7100a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009680 tx=0x7fc71c009260 comp rx=0 tx=0).stop 2024-04-26T01:51:08.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.664+0000 7fc72f800700 1 -- 172.21.15.1:0/1377173544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc728107990 msgr2=0x7fc72819fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:08.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.664+0000 7fc72f800700 1 --2- 172.21.15.1:0/1377173544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc728107990 0x7fc72819fbb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc72000b070 tx=0x7fc720004c80 comp rx=0 tx=0).stop 2024-04-26T01:51:08.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.664+0000 7fc72f800700 1 -- 172.21.15.1:0/1377173544 shutdown_connections 2024-04-26T01:51:08.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.664+0000 7fc72f800700 1 --2- 172.21.15.1:0/1377173544 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fc7100a3c40 0x7fc7100a6100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:08.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.664+0000 7fc72f800700 1 --2- 172.21.15.1:0/1377173544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc728107990 0x7fc72819fbb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:08.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.664+0000 7fc72f800700 1 --2- 172.21.15.1:0/1377173544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc728101940 0x7fc72819b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:08.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.664+0000 7fc72f800700 1 --2- 172.21.15.1:0/1377173544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc728102300 0x7fc72819b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:08.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.664+0000 7fc72f800700 1 -- 172.21.15.1:0/1377173544 >> 172.21.15.1:0/1377173544 conn(0x7fc7280788f0 msgr2=0x7fc72806a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:08.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.664+0000 7fc72f800700 1 -- 172.21.15.1:0/1377173544 shutdown_connections 2024-04-26T01:51:08.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:08.664+0000 7fc72f800700 1 -- 172.21.15.1:0/1377173544 wait complete. 2024-04-26T01:51:08.664 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-04-26T01:51:09.056 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:08 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1377173544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:51:09.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:08 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1377173544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:51:09.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:08 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1377173544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:51:09.545 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:51:10.028 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:10.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:09 smithi173 ceph-mon[32553]: pgmap v436: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:10.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:09 smithi001 ceph-mon[27822]: pgmap v436: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:10.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:09 smithi097 ceph-mon[32853]: pgmap v436: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:12.006 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:11 smithi001 ceph-mon[27822]: pgmap v437: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:51:12.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:11 smithi097 ceph-mon[32853]: pgmap v437: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:51:12.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.269+0000 7f8172400700 1 -- 172.21.15.1:0/565111285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816c075590 msgr2=0x7f816c10ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:12.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.269+0000 7f8172400700 1 --2- 172.21.15.1:0/565111285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816c075590 0x7f816c10ef00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8154009990 tx=0x7f8154009ca0 comp rx=0 tx=0).stop 2024-04-26T01:51:12.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.269+0000 7f8172400700 1 -- 172.21.15.1:0/565111285 shutdown_connections 2024-04-26T01:51:12.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.269+0000 7f8172400700 1 --2- 172.21.15.1:0/565111285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f816c10f440 0x7f816c111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:12.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.269+0000 7f8172400700 1 --2- 172.21.15.1:0/565111285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f816c074c70 0x7f816c075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:12.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.269+0000 7f8172400700 1 --2- 172.21.15.1:0/565111285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816c075590 0x7f816c10ef00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:12.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.269+0000 7f8172400700 1 -- 172.21.15.1:0/565111285 >> 172.21.15.1:0/565111285 conn(0x7f816c079ba0 msgr2=0x7f816c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:12.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.269+0000 7f8172400700 1 -- 172.21.15.1:0/565111285 shutdown_connections 2024-04-26T01:51:12.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.269+0000 7f8172400700 1 -- 172.21.15.1:0/565111285 wait complete. 2024-04-26T01:51:12.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.270+0000 7f8172400700 1 Processor -- start 2024-04-26T01:51:12.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.270+0000 7f8172400700 1 -- start start 2024-04-26T01:51:12.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.271+0000 7f8172400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f816c074c70 0x7f816c19b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:12.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.271+0000 7f8172400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816c075590 0x7f816c19b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:12.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.271+0000 7f8172400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f816c10f440 0x7f816c19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:12.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.271+0000 7f8172400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f816c114e00 con 0x7f816c075590 2024-04-26T01:51:12.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.271+0000 7f8172400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f816c114c80 con 0x7f816c10f440 2024-04-26T01:51:12.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.271+0000 7f8172400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f816c114f80 con 0x7f816c074c70 2024-04-26T01:51:12.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.271+0000 7f8162a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816c075590 0x7f816c19b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:12.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.271+0000 7f816b400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f816c074c70 0x7f816c19b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:12.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.271+0000 7f8162a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816c075590 0x7f816c19b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59774/0 (socket says 172.21.15.1:59774) 2024-04-26T01:51:12.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.271+0000 7f8162a00700 1 -- 172.21.15.1:0/2195632448 learned_addr learned my addr 172.21.15.1:0/2195632448 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:12.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.271+0000 7f816be00700 1 --2- 172.21.15.1:0/2195632448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f816c10f440 0x7f816c19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:12.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.272+0000 7f8162a00700 1 -- 172.21.15.1:0/2195632448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f816c074c70 msgr2=0x7f816c19b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:12.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.272+0000 7f8162a00700 1 --2- 172.21.15.1:0/2195632448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f816c074c70 0x7f816c19b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:12.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.272+0000 7f8162a00700 1 -- 172.21.15.1:0/2195632448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f816c10f440 msgr2=0x7f816c19fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:12.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.272+0000 7f8162a00700 1 --2- 172.21.15.1:0/2195632448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f816c10f440 0x7f816c19fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:12.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.272+0000 7f8162a00700 1 -- 172.21.15.1:0/2195632448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816c1a0290 con 0x7f816c075590 2024-04-26T01:51:12.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.272+0000 7f816be00700 1 --2- 172.21.15.1:0/2195632448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f816c10f440 0x7f816c19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:12.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.272+0000 7f816b400700 1 --2- 172.21.15.1:0/2195632448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f816c074c70 0x7f816c19b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:12.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.272+0000 7f8162a00700 1 --2- 172.21.15.1:0/2195632448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816c075590 0x7f816c19b810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8154000c00 tx=0x7f8154009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:12.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.273+0000 7f8168c00700 1 -- 172.21.15.1:0/2195632448 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154027070 con 0x7f816c075590 2024-04-26T01:51:12.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.273+0000 7f8168c00700 1 -- 172.21.15.1:0/2195632448 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8154022470 con 0x7f816c075590 2024-04-26T01:51:12.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.273+0000 7f8172400700 1 -- 172.21.15.1:0/2195632448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816c1a0520 con 0x7f816c075590 2024-04-26T01:51:12.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.273+0000 7f8168c00700 1 -- 172.21.15.1:0/2195632448 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154017650 con 0x7f816c075590 2024-04-26T01:51:12.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.273+0000 7f8172400700 1 -- 172.21.15.1:0/2195632448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f816c1a7e00 con 0x7f816c075590 2024-04-26T01:51:12.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.274+0000 7f8172400700 1 -- 172.21.15.1:0/2195632448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f816c069730 con 0x7f816c075590 2024-04-26T01:51:12.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.276+0000 7f8168c00700 1 -- 172.21.15.1:0/2195632448 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f815402a070 con 0x7f816c075590 2024-04-26T01:51:12.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.277+0000 7f8168c00700 1 --2- 172.21.15.1:0/2195632448 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f81500a3cc0 0x7f81500a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:12.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.278+0000 7f8168c00700 1 -- 172.21.15.1:0/2195632448 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f81540d14b0 con 0x7f816c075590 2024-04-26T01:51:12.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.278+0000 7f816b400700 1 --2- 172.21.15.1:0/2195632448 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f81500a3cc0 0x7f81500a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:12.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.279+0000 7f816b400700 1 --2- 172.21.15.1:0/2195632448 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f81500a3cc0 0x7f81500a6180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f815c005950 tx=0x7f815c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:12.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.280+0000 7f8168c00700 1 -- 172.21.15.1:0/2195632448 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8154013070 con 0x7f816c075590 2024-04-26T01:51:12.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:11 smithi173 ceph-mon[32553]: pgmap v437: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:51:12.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.557+0000 7f8172400700 1 -- 172.21.15.1:0/2195632448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f816c19c5e0 con 0x7f816c075590 2024-04-26T01:51:12.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.558+0000 7f8168c00700 1 -- 172.21.15.1:0/2195632448 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11841 (secure 0 0 0) 0x7f815409df20 con 0x7f816c075590 2024-04-26T01:51:12.557 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:51:12.557 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:47:54.077766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2952462325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2952462325},{"type":"v1","addr":"172.21.15.173:6835","nonce":2952462325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24514,24518,24499]},"id":1}]} 2024-04-26T01:51:12.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.561+0000 7f8172400700 1 -- 172.21.15.1:0/2195632448 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f81500a3cc0 msgr2=0x7f81500a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:12.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.561+0000 7f8172400700 1 --2- 172.21.15.1:0/2195632448 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f81500a3cc0 0x7f81500a6180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f815c005950 tx=0x7f815c00b410 comp rx=0 tx=0).stop 2024-04-26T01:51:12.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.561+0000 7f8172400700 1 -- 172.21.15.1:0/2195632448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816c075590 msgr2=0x7f816c19b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:12.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.561+0000 7f8172400700 1 --2- 172.21.15.1:0/2195632448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816c075590 0x7f816c19b810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8154000c00 tx=0x7f8154009580 comp rx=0 tx=0).stop 2024-04-26T01:51:12.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.561+0000 7f8172400700 1 -- 172.21.15.1:0/2195632448 shutdown_connections 2024-04-26T01:51:12.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.561+0000 7f8172400700 1 --2- 172.21.15.1:0/2195632448 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f81500a3cc0 0x7f81500a6180 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:12.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.561+0000 7f8172400700 1 --2- 172.21.15.1:0/2195632448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f816c10f440 0x7f816c19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:12.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.561+0000 7f8172400700 1 --2- 172.21.15.1:0/2195632448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f816c074c70 0x7f816c19b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:12.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.561+0000 7f8172400700 1 --2- 172.21.15.1:0/2195632448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816c075590 0x7f816c19b810 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:12.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.561+0000 7f8172400700 1 -- 172.21.15.1:0/2195632448 >> 172.21.15.1:0/2195632448 conn(0x7f816c079ba0 msgr2=0x7f816c0ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:12.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.562+0000 7f8172400700 1 -- 172.21.15.1:0/2195632448 shutdown_connections 2024-04-26T01:51:12.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:12.562+0000 7f8172400700 1 -- 172.21.15.1:0/2195632448 wait complete. 2024-04-26T01:51:12.561 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-04-26T01:51:13.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:12 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/2195632448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:51:13.313 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:12 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2195632448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:51:13.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:12 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/2195632448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:51:13.389 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:51:13.869 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:14.213 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:13 smithi001 ceph-mon[27822]: pgmap v438: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:14.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:13 smithi097 ceph-mon[32853]: pgmap v438: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:14.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:13 smithi173 ceph-mon[32553]: pgmap v438: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:16.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.162+0000 7f313c800700 1 -- 172.21.15.1:0/4258138819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31341092d0 msgr2=0x7f31341096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:16.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.162+0000 7f313c800700 1 --2- 172.21.15.1:0/4258138819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31341092d0 0x7f31341096b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f31280099c0 tx=0x7f3128009cd0 comp rx=0 tx=0).stop 2024-04-26T01:51:16.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.162+0000 7f313c800700 1 -- 172.21.15.1:0/4258138819 shutdown_connections 2024-04-26T01:51:16.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.162+0000 7f313c800700 1 --2- 172.21.15.1:0/4258138819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f313410e3b0 0x7f3134115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:16.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.162+0000 7f313c800700 1 --2- 172.21.15.1:0/4258138819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31341092d0 0x7f31341096b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:16.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.162+0000 7f313c800700 1 --2- 172.21.15.1:0/4258138819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3134109c80 0x7f313410dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:16.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.162+0000 7f313c800700 1 -- 172.21.15.1:0/4258138819 >> 172.21.15.1:0/4258138819 conn(0x7f3134076210 msgr2=0x7f3134107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:16.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.162+0000 7f313c800700 1 -- 172.21.15.1:0/4258138819 shutdown_connections 2024-04-26T01:51:16.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.162+0000 7f313c800700 1 -- 172.21.15.1:0/4258138819 wait complete. 2024-04-26T01:51:16.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.164+0000 7f313c800700 1 Processor -- start 2024-04-26T01:51:16.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.164+0000 7f313c800700 1 -- start start 2024-04-26T01:51:16.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.164+0000 7f313c800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31341092d0 0x7f313419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:16.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.164+0000 7f313c800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3134109c80 0x7f313419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:16.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.164+0000 7f313c800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f313410e3b0 0x7f31341a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:16.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.164+0000 7f313c800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3134119240 con 0x7f31341092d0 2024-04-26T01:51:16.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.164+0000 7f313c800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f31341190c0 con 0x7f313410e3b0 2024-04-26T01:51:16.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.164+0000 7f313c800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f31341193c0 con 0x7f3134109c80 2024-04-26T01:51:16.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.164+0000 7f3139e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31341092d0 0x7f313419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:16.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.165+0000 7f3139e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31341092d0 0x7f313419f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35358/0 (socket says 172.21.15.1:35358) 2024-04-26T01:51:16.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.165+0000 7f3139e00700 1 -- 172.21.15.1:0/508590053 learned_addr learned my addr 172.21.15.1:0/508590053 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:16.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.165+0000 7f313a800700 1 --2- 172.21.15.1:0/508590053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f313410e3b0 0x7f31341a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:16.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.165+0000 7f3139400700 1 --2- 172.21.15.1:0/508590053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3134109c80 0x7f313419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:16.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.165+0000 7f3139e00700 1 -- 172.21.15.1:0/508590053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3134109c80 msgr2=0x7f313419fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:16.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.165+0000 7f3139e00700 1 --2- 172.21.15.1:0/508590053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3134109c80 0x7f313419fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:16.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.165+0000 7f3139e00700 1 -- 172.21.15.1:0/508590053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f313410e3b0 msgr2=0x7f31341a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:16.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.165+0000 7f3139e00700 1 --2- 172.21.15.1:0/508590053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f313410e3b0 0x7f31341a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:16.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.165+0000 7f3139e00700 1 -- 172.21.15.1:0/508590053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31341a4740 con 0x7f31341092d0 2024-04-26T01:51:16.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.165+0000 7f313a800700 1 --2- 172.21.15.1:0/508590053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f313410e3b0 0x7f31341a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:16.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.165+0000 7f3139e00700 1 --2- 172.21.15.1:0/508590053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31341092d0 0x7f313419f730 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3128006010 tx=0x7f3128004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:16.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.166+0000 7f312ea00700 1 -- 172.21.15.1:0/508590053 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128026070 con 0x7f31341092d0 2024-04-26T01:51:16.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.166+0000 7f312ea00700 1 -- 172.21.15.1:0/508590053 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3128004630 con 0x7f31341092d0 2024-04-26T01:51:16.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.166+0000 7f312ea00700 1 -- 172.21.15.1:0/508590053 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312800fb20 con 0x7f31341092d0 2024-04-26T01:51:16.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.166+0000 7f313c800700 1 -- 172.21.15.1:0/508590053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31341a49d0 con 0x7f31341092d0 2024-04-26T01:51:16.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.166+0000 7f313c800700 1 -- 172.21.15.1:0/508590053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31341ac2b0 con 0x7f31341092d0 2024-04-26T01:51:16.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.167+0000 7f313c800700 1 -- 172.21.15.1:0/508590053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3134069730 con 0x7f31341092d0 2024-04-26T01:51:16.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.171+0000 7f312ea00700 1 -- 172.21.15.1:0/508590053 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f3128029070 con 0x7f31341092d0 2024-04-26T01:51:16.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.172+0000 7f312ea00700 1 --2- 172.21.15.1:0/508590053 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f311c0a3c40 0x7f311c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:16.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.172+0000 7f312ea00700 1 -- 172.21.15.1:0/508590053 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f31280d0ff0 con 0x7f31341092d0 2024-04-26T01:51:16.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.172+0000 7f3139400700 1 --2- 172.21.15.1:0/508590053 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f311c0a3c40 0x7f311c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:16.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.174+0000 7f3139400700 1 --2- 172.21.15.1:0/508590053 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f311c0a3c40 0x7f311c0a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3124009680 tx=0x7f3124009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:16.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.175+0000 7f312ea00700 1 -- 172.21.15.1:0/508590053 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3128030080 con 0x7f31341092d0 2024-04-26T01:51:16.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:15 smithi097 ceph-mon[32853]: pgmap v439: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:16.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:15 smithi173 ceph-mon[32553]: pgmap v439: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:16.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:15 smithi001 ceph-mon[27822]: pgmap v439: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:16.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.408+0000 7f313c800700 1 -- 172.21.15.1:0/508590053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f313402df80 con 0x7f31341092d0 2024-04-26T01:51:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.411+0000 7f312ea00700 1 -- 172.21.15.1:0/508590053 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22649 (secure 0 0 0) 0x7f3128013070 con 0x7f31341092d0 2024-04-26T01:51:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:51:16.410 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":92,"fsid":"77cfcc80-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:38:03.531182+0000","modified":"2024-04-26T01:48:22.381867+0000","last_up_change":"2024-04-26T01:46:16.151275+0000","last_in_change":"2024-04-26T01:45:56.514228+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:42:04.805488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:47:21.957421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:47:25.860260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb10ae12-09ee-4d6b-bc78-8c6016613b81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6803","nonce":3176633965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6805","nonce":3176633965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6809","nonce":3176633965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6807","nonce":3176633965}]},"public_addr":"172.21.15.1:6803/3176633965","cluster_addr":"172.21.15.1:6805/3176633965","heartbeat_back_addr":"172.21.15.1:6809/3176633965","heartbeat_front_addr":"172.21.15.1:6807/3176633965","state":["exists","up"]},{"osd":1,"uuid":"acd12888-6ff6-40bc-99e7-e5c395e85b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6801","nonce":3755022621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6803","nonce":3755022621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6807","nonce":3755022621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6805","nonce":3755022621}]},"public_addr":"172.21.15.97:6801/3755022621","cluster_addr":"172.21.15.97:6803/3755022621","heartbeat_back_addr":"172.21.15.97:6807/3755022621","heartbeat_front_addr":"172.21.15.97:6805/3755022621","state":["exists","up"]},{"osd":2,"uuid":"79971508-e256-4145-8111-a75666e886fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6801","nonce":734861771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6803","nonce":734861771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6807","nonce":734861771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6805","nonce":734861771}]},"public_addr":"172.21.15.173:6801/734861771","cluster_addr":"172.21.15.173:6803/734861771","heartbeat_back_addr":"172.21.15.173:6807/734861771","heartbeat_front_addr":"172.21.15.173:6805/734861771","state":["exists","up"]},{"osd":3,"uuid":"a8ccc4a9-0f31-446a-8bcb-4691532397c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6811","nonce":3349961961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6813","nonce":3349961961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6817","nonce":3349961961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6815","nonce":3349961961}]},"public_addr":"172.21.15.1:6811/3349961961","cluster_addr":"172.21.15.1:6813/3349961961","heartbeat_back_addr":"172.21.15.1:6817/3349961961","heartbeat_front_addr":"172.21.15.1:6815/3349961961","state":["exists","up"]},{"osd":4,"uuid":"5b724648-4df7-40fc-89d5-55f170cdd806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6809","nonce":2494639397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6811","nonce":2494639397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6815","nonce":2494639397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6813","nonce":2494639397}]},"public_addr":"172.21.15.97:6809/2494639397","cluster_addr":"172.21.15.97:6811/2494639397","heartbeat_back_addr":"172.21.15.97:6815/2494639397","heartbeat_front_addr":"172.21.15.97:6813/2494639397","state":["exists","up"]},{"osd":5,"uuid":"a5a24a9c-3007-4b87-8562-40c98c41ef8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6809","nonce":1026507957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6811","nonce":1026507957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6815","nonce":1026507957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6813","nonce":1026507957}]},"public_addr":"172.21.15.173:6809/1026507957","cluster_addr":"172.21.15.173:6811/1026507957","heartbeat_back_addr":"172.21.15.173:6815/1026507957","heartbeat_front_addr":"172.21.15.173:6813/1026507957","state":["exists","up"]},{"osd":6,"uuid":"97cd2e88-a56c-4b1a-8104-65b23a77986a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6819","nonce":444030587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6821","nonce":444030587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6825","nonce":444030587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6823","nonce":444030587}]},"public_addr":"172.21.15.1:6819/444030587","cluster_addr":"172.21.15.1:6821/444030587","heartbeat_back_addr":"172.21.15.1:6825/444030587","heartbeat_front_addr":"172.21.15.1:6823/444030587","state":["exists","up"]},{"osd":7,"uuid":"afc0c2dd-e112-4358-b3aa-944366d5d383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6817","nonce":2600022531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6819","nonce":2600022531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6823","nonce":2600022531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6821","nonce":2600022531}]},"public_addr":"172.21.15.97:6817/2600022531","cluster_addr":"172.21.15.97:6819/2600022531","heartbeat_back_addr":"172.21.15.97:6823/2600022531","heartbeat_front_addr":"172.21.15.97:6821/2600022531","state":["exists","up"]},{"osd":8,"uuid":"41fa612b-793e-44da-af8b-44d17a9de303","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6817","nonce":1334205527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6819","nonce":1334205527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6823","nonce":1334205527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6821","nonce":1334205527}]},"public_addr":"172.21.15.173:6817/1334205527","cluster_addr":"172.21.15.173:6819/1334205527","heartbeat_back_addr":"172.21.15.173:6823/1334205527","heartbeat_front_addr":"172.21.15.173:6821/1334205527","state":["exists","up"]},{"osd":9,"uuid":"bbbb163b-f8c4-4be5-8f80-82445e8d407f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6827","nonce":2973486839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6829","nonce":2973486839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6833","nonce":2973486839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6831","nonce":2973486839}]},"public_addr":"172.21.15.1:6827/2973486839","cluster_addr":"172.21.15.1:6829/2973486839","heartbeat_back_addr":"172.21.15.1:6833/2973486839","heartbeat_front_addr":"172.21.15.1:6831/2973486839","state":["exists","up"]},{"osd":10,"uuid":"1b447b34-0600-4727-826d-b3d68adfb18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6825","nonce":1693425268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6827","nonce":1693425268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6831","nonce":1693425268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6829","nonce":1693425268}]},"public_addr":"172.21.15.97:6825/1693425268","cluster_addr":"172.21.15.97:6827/1693425268","heartbeat_back_addr":"172.21.15.97:6831/1693425268","heartbeat_front_addr":"172.21.15.97:6829/1693425268","state":["exists","up"]},{"osd":11,"uuid":"f439aae1-f50a-4176-889d-36fa5131541a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6825","nonce":1141524960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6827","nonce":1141524960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6831","nonce":1141524960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6829","nonce":1141524960}]},"public_addr":"172.21.15.173:6825/1141524960","cluster_addr":"172.21.15.173:6827/1141524960","heartbeat_back_addr":"172.21.15.173:6831/1141524960","heartbeat_front_addr":"172.21.15.173:6829/1141524960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:12.955643+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:38.846652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:02.556062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:34.900933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:01.170878+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:24.842573+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:58.304440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:24.868689+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:48.772003+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:22.611681+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:49.470779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:14.163984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:6801/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6800/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/1000222947":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/457958597":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6801/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:6800/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/2715798962":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/1923058728":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/2489038777":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/2869502448":"2024-04-27T01:38:45.974031+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:51:16.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.413+0000 7f313c800700 1 -- 172.21.15.1:0/508590053 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f311c0a3c40 msgr2=0x7f311c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:16.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.413+0000 7f313c800700 1 --2- 172.21.15.1:0/508590053 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f311c0a3c40 0x7f311c0a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3124009680 tx=0x7f3124009260 comp rx=0 tx=0).stop 2024-04-26T01:51:16.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.413+0000 7f313c800700 1 -- 172.21.15.1:0/508590053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31341092d0 msgr2=0x7f313419f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:16.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.413+0000 7f313c800700 1 --2- 172.21.15.1:0/508590053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31341092d0 0x7f313419f730 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3128006010 tx=0x7f3128004270 comp rx=0 tx=0).stop 2024-04-26T01:51:16.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.414+0000 7f313c800700 1 -- 172.21.15.1:0/508590053 shutdown_connections 2024-04-26T01:51:16.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.414+0000 7f313c800700 1 --2- 172.21.15.1:0/508590053 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f311c0a3c40 0x7f311c0a6100 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:16.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.414+0000 7f313c800700 1 --2- 172.21.15.1:0/508590053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f313410e3b0 0x7f31341a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:16.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.414+0000 7f313c800700 1 --2- 172.21.15.1:0/508590053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31341092d0 0x7f313419f730 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:16.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.414+0000 7f313c800700 1 --2- 172.21.15.1:0/508590053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3134109c80 0x7f313419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:16.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.414+0000 7f313c800700 1 -- 172.21.15.1:0/508590053 >> 172.21.15.1:0/508590053 conn(0x7f3134076210 msgr2=0x7f3134077e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:16.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.414+0000 7f313c800700 1 -- 172.21.15.1:0/508590053 shutdown_connections 2024-04-26T01:51:16.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:16.414+0000 7f313c800700 1 -- 172.21.15.1:0/508590053 wait complete. 2024-04-26T01:51:17.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:16 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/508590053' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:51:17.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:16 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/508590053' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:51:17.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:16 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/508590053' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:51:17.390 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T01:47:29.999521+0000', 'modified': '2024-04-26T01:47:54.077766+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24518, 'mds_1': 24514, 'mds_2': 24499}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24464': {'gid': 24464, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6835/763675408', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 763675408}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 763675408}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6837/3607142398', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 3607142398}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 3607142398}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.173:6835/2952462325', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 2952462325}, {'type': 'v1', 'addr': '172.21.15.173:6835', 'nonce': 2952462325}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.173:6837/2502298793', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6836', 'nonce': 2502298793}, {'type': 'v1', 'addr': '172.21.15.173:6837', 'nonce': 2502298793}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24514': {'gid': 24514, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.173:6839/373692430', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6838', 'nonce': 373692430}, {'type': 'v1', 'addr': '172.21.15.173:6839', 'nonce': 373692430}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.97:6839/1315554162', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 1315554162}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 1315554162}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24518, 'qdb_cluster': [24514, 24518, 24499]} 2024-04-26T01:51:17.390 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-26T01:51:17.390 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph tell mds.c status 2024-04-26T01:51:17.850 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:17 smithi097 ceph-mon[32853]: pgmap v440: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:18.253 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:17 smithi001 ceph-mon[27822]: pgmap v440: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:18.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:17 smithi173 ceph-mon[32553]: pgmap v440: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:20.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.069+0000 7fbc82600700 1 -- 172.21.15.1:0/4081709404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc7c104ee0 msgr2=0x7fbc7c1052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:20.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.069+0000 7fbc82600700 1 --2- 172.21.15.1:0/4081709404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc7c104ee0 0x7fbc7c1052c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0099c0 tx=0x7fbc6c009cd0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.069+0000 7fbc82600700 1 -- 172.21.15.1:0/4081709404 shutdown_connections 2024-04-26T01:51:20.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.069+0000 7fbc82600700 1 --2- 172.21.15.1:0/4081709404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc7c109fc0 0x7fbc7c111b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.069+0000 7fbc82600700 1 --2- 172.21.15.1:0/4081709404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc7c104ee0 0x7fbc7c1052c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.069+0000 7fbc82600700 1 --2- 172.21.15.1:0/4081709404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc7c105890 0x7fbc7c1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.070+0000 7fbc82600700 1 -- 172.21.15.1:0/4081709404 >> 172.21.15.1:0/4081709404 conn(0x7fbc7c100770 msgr2=0x7fbc7c102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:20.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.070+0000 7fbc82600700 1 -- 172.21.15.1:0/4081709404 shutdown_connections 2024-04-26T01:51:20.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.070+0000 7fbc82600700 1 -- 172.21.15.1:0/4081709404 wait complete. 2024-04-26T01:51:20.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.071+0000 7fbc82600700 1 Processor -- start 2024-04-26T01:51:20.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.071+0000 7fbc82600700 1 -- start start 2024-04-26T01:51:20.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.071+0000 7fbc82600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc7c104ee0 0x7fbc7c19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.071+0000 7fbc82600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc7c105890 0x7fbc7c19b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.071+0000 7fbc82600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc7c109fc0 0x7fbc7c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.071+0000 7fbc82600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c114ed0 con 0x7fbc7c104ee0 2024-04-26T01:51:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.071+0000 7fbc82600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c114d50 con 0x7fbc7c105890 2024-04-26T01:51:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.071+0000 7fbc82600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c115050 con 0x7fbc7c109fc0 2024-04-26T01:51:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.072+0000 7fbc7b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc7c104ee0 0x7fbc7c19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.072+0000 7fbc7b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc7c104ee0 0x7fbc7c19b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35380/0 (socket says 172.21.15.1:35380) 2024-04-26T01:51:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.072+0000 7fbc7b400700 1 -- 172.21.15.1:0/9145064 learned_addr learned my addr 172.21.15.1:0/9145064 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.072+0000 7fbc7be00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc7c109fc0 0x7fbc7c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.072+0000 7fbc72a00700 1 --2- 172.21.15.1:0/9145064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc7c105890 0x7fbc7c19b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:20.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.072+0000 7fbc7be00700 1 -- 172.21.15.1:0/9145064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc7c105890 msgr2=0x7fbc7c19b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:20.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.072+0000 7fbc7be00700 1 --2- 172.21.15.1:0/9145064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc7c105890 0x7fbc7c19b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.072+0000 7fbc7be00700 1 -- 172.21.15.1:0/9145064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc7c104ee0 msgr2=0x7fbc7c19b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:20.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.072+0000 7fbc7be00700 1 --2- 172.21.15.1:0/9145064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc7c104ee0 0x7fbc7c19b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.072+0000 7fbc7be00700 1 -- 172.21.15.1:0/9145064 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7c1a03c0 con 0x7fbc7c109fc0 2024-04-26T01:51:20.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.072+0000 7fbc7b400700 1 --2- 172.21.15.1:0/9145064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc7c104ee0 0x7fbc7c19b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:20.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.072+0000 7fbc7be00700 1 --2- 172.21.15.1:0/9145064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc7c109fc0 0x7fbc7c19fc80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400efc0 tx=0x7fbc7400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:20.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.073+0000 7fbc78c00700 1 -- 172.21.15.1:0/9145064 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc740092e0 con 0x7fbc7c109fc0 2024-04-26T01:51:20.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.073+0000 7fbc78c00700 1 -- 172.21.15.1:0/9145064 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc74004bb0 con 0x7fbc7c109fc0 2024-04-26T01:51:20.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.073+0000 7fbc82600700 1 -- 172.21.15.1:0/9145064 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc7c1a0650 con 0x7fbc7c109fc0 2024-04-26T01:51:20.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.073+0000 7fbc78c00700 1 -- 172.21.15.1:0/9145064 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74004020 con 0x7fbc7c109fc0 2024-04-26T01:51:20.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.073+0000 7fbc82600700 1 -- 172.21.15.1:0/9145064 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc7c1a7f30 con 0x7fbc7c109fc0 2024-04-26T01:51:20.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.076+0000 7fbc78c00700 1 -- 172.21.15.1:0/9145064 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fbc7401b030 con 0x7fbc7c109fc0 2024-04-26T01:51:20.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.078+0000 7fbc78c00700 1 --2- 172.21.15.1:0/9145064 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbc600a3c40 0x7fbc600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:20.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.078+0000 7fbc8d050b80 1 Processor -- start 2024-04-26T01:51:20.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.078+0000 7fbc78c00700 1 -- 172.21.15.1:0/9145064 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fbc740cfe70 con 0x7fbc7c109fc0 2024-04-26T01:51:20.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.078+0000 7fbc7b400700 1 --2- 172.21.15.1:0/9145064 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbc600a3c40 0x7fbc600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:20.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.078+0000 7fbc8d050b80 1 -- start start 2024-04-26T01:51:20.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.078+0000 7fbc8d050b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1ef780 0x556ffb1efb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:20.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.079+0000 7fbc8d050b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1f00a0 0x556ffb1e99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:20.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.079+0000 7fbc8d050b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1e9ef0 0x556ffb1ea330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:20.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.079+0000 7fbc8d050b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556ffb13be40 con 0x556ffb1ef780 2024-04-26T01:51:20.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.079+0000 7fbc8d050b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556ffb0e26e0 con 0x556ffb1f00a0 2024-04-26T01:51:20.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.079+0000 7fbc8d050b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556ffb137e00 con 0x556ffb1e9ef0 2024-04-26T01:51:20.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.079+0000 7fbc7b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1ef780 0x556ffb1efb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:20.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.079+0000 7fbc7b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1ef780 0x556ffb1efb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35394/0 (socket says 172.21.15.1:35394) 2024-04-26T01:51:20.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.079+0000 7fbc7b400700 1 -- 172.21.15.1:0/1863016578 learned_addr learned my addr 172.21.15.1:0/1863016578 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:20.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.079+0000 7fbc7be00700 1 --2- 172.21.15.1:0/1863016578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1e9ef0 0x556ffb1ea330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:20.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.079+0000 7fbc7b400700 1 --2- 172.21.15.1:0/9145064 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbc600a3c40 0x7fbc600a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c009990 tx=0x7fbc6c005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:20.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.079+0000 7fbc72a00700 1 --2- 172.21.15.1:0/1863016578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1f00a0 0x556ffb1e99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:20.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.079+0000 7fbc7b400700 1 -- 172.21.15.1:0/1863016578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1e9ef0 msgr2=0x556ffb1ea330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:20.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.079+0000 7fbc7b400700 1 --2- 172.21.15.1:0/1863016578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1e9ef0 0x556ffb1ea330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.079+0000 7fbc7b400700 1 -- 172.21.15.1:0/1863016578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1f00a0 msgr2=0x556ffb1e99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:20.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.079+0000 7fbc7b400700 1 --2- 172.21.15.1:0/1863016578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1f00a0 0x556ffb1e99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.079+0000 7fbc7b400700 1 -- 172.21.15.1:0/1863016578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ffb0d8630 con 0x556ffb1ef780 2024-04-26T01:51:20.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.079+0000 7fbc72a00700 1 --2- 172.21.15.1:0/1863016578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1f00a0 0x556ffb1e99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:51:20.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.080+0000 7fbc7b400700 1 --2- 172.21.15.1:0/1863016578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1ef780 0x556ffb1efb60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c000c00 tx=0x7fbc6c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f636c317335e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:20.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.080+0000 7fbc70c00700 1 -- 172.21.15.1:0/1863016578 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c0039c0 con 0x556ffb1ef780 2024-04-26T01:51:20.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.080+0000 7fbc70c00700 1 -- 172.21.15.1:0/1863016578 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc6c009590 con 0x556ffb1ef780 2024-04-26T01:51:20.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.080+0000 7fbc70c00700 1 -- 172.21.15.1:0/1863016578 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c024e50 con 0x556ffb1ef780 2024-04-26T01:51:20.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.081+0000 7fbc8d050b80 1 -- 172.21.15.1:0/1863016578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1ef780 msgr2=0x556ffb1efb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:20.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.081+0000 7fbc8d050b80 1 --2- 172.21.15.1:0/1863016578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1ef780 0x556ffb1efb60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c000c00 tx=0x7fbc6c020000 comp rx=0 tx=0).stop 2024-04-26T01:51:20.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.081+0000 7fbc8d050b80 1 -- 172.21.15.1:0/1863016578 shutdown_connections 2024-04-26T01:51:20.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.081+0000 7fbc8d050b80 1 --2- 172.21.15.1:0/1863016578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1e9ef0 0x556ffb1ea330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.081+0000 7fbc8d050b80 1 --2- 172.21.15.1:0/1863016578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1ef780 0x556ffb1efb60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.081+0000 7fbc8d050b80 1 --2- 172.21.15.1:0/1863016578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1f00a0 0x556ffb1e99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.081+0000 7fbc8d050b80 1 -- 172.21.15.1:0/1863016578 >> 172.21.15.1:0/1863016578 conn(0x556ffb117f00 msgr2=0x556ffb1e48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:20.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.081+0000 7fbc8d050b80 1 -- 172.21.15.1:0/1863016578 shutdown_connections 2024-04-26T01:51:20.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.081+0000 7fbc8d050b80 1 -- 172.21.15.1:0/1863016578 wait complete. 2024-04-26T01:51:20.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.081+0000 7fbc8d050b80 1 -- start start 2024-04-26T01:51:20.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.082+0000 7fbc8d050b80 1 Processor -- start 2024-04-26T01:51:20.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.082+0000 7fbc8d050b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1e9ef0 0x556ffb1f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:20.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.082+0000 7fbc8d050b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1ef780 0x556ffb1f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:20.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.082+0000 7fbc8d050b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1f34c0 0x556ffb1f3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:20.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.082+0000 7fbc8d050b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556ffb1f68c0 con 0x556ffb1ef780 2024-04-26T01:51:20.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.082+0000 7fbc8d050b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556ffb0f5810 con 0x556ffb1e9ef0 2024-04-26T01:51:20.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.082+0000 7fbc8d050b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556ffb1f6a40 con 0x556ffb1f34c0 2024-04-26T01:51:20.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.082+0000 7fbc8d050b80 10 client.0 _ll_register_callbacks cb 0x556ffb0d46d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:51:20.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.083+0000 7fbc72a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1ef780 0x556ffb1f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:20.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.083+0000 7fbc72a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1ef780 0x556ffb1f2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35402/0 (socket says 172.21.15.1:35402) 2024-04-26T01:51:20.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.083+0000 7fbc72a00700 1 -- 172.21.15.1:0/350399781 learned_addr learned my addr 172.21.15.1:0/350399781 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:20.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.083+0000 7fbc7b400700 1 --2- 172.21.15.1:0/350399781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1e9ef0 0x556ffb1f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:20.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.083+0000 7fbc7be00700 1 --2- 172.21.15.1:0/350399781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1f34c0 0x556ffb1f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:20.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.083+0000 7fbc7b400700 1 -- 172.21.15.1:0/350399781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1f34c0 msgr2=0x556ffb1f3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:20.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.083+0000 7fbc7b400700 1 --2- 172.21.15.1:0/350399781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1f34c0 0x556ffb1f3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.083+0000 7fbc7b400700 1 -- 172.21.15.1:0/350399781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1ef780 msgr2=0x556ffb1f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:20.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.083+0000 7fbc7b400700 1 --2- 172.21.15.1:0/350399781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1ef780 0x556ffb1f2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.083+0000 7fbc7b400700 1 -- 172.21.15.1:0/350399781 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ffb089880 con 0x556ffb1e9ef0 2024-04-26T01:51:20.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.083+0000 7fbc7be00700 1 --2- 172.21.15.1:0/350399781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1f34c0 0x556ffb1f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:20.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.084+0000 7fbc7b400700 1 --2- 172.21.15.1:0/350399781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1e9ef0 0x556ffb1f5c80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c006010 tx=0x7fbc6c0246a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f74266ee73f412f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:20.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.084+0000 7fbc69600700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:51:20.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.084+0000 7fbc69600700 1 -- 172.21.15.1:0/350399781 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c024850 con 0x556ffb1e9ef0 2024-04-26T01:51:20.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.084+0000 7fbc69600700 1 -- 172.21.15.1:0/350399781 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fbc6c0039d0 con 0x556ffb1e9ef0 2024-04-26T01:51:20.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.084+0000 7fbc8d050b80 1 -- 172.21.15.1:0/350399781 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc6c009590 con 0x556ffb1e9ef0 2024-04-26T01:51:20.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.084+0000 7fbc69600700 1 -- 172.21.15.1:0/350399781 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc6c003bd0 con 0x556ffb1e9ef0 2024-04-26T01:51:20.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.084+0000 7fbc8d050b80 10 client.24998 fetch_fsmap learned FSMap version 25 2024-04-26T01:51:20.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.084+0000 7fbc8d050b80 1 -- 172.21.15.1:0/350399781 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556ffb0d2aa0 con 0x556ffb1e9ef0 2024-04-26T01:51:20.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.085+0000 7fbc69600700 1 -- 172.21.15.1:0/350399781 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc6c009590 con 0x556ffb1e9ef0 2024-04-26T01:51:20.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.085+0000 7fbc69600700 1 -- 172.21.15.1:0/350399781 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fbc6c02d020 con 0x556ffb1e9ef0 2024-04-26T01:51:20.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.085+0000 7fbc8d050b80 10 client.24998 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:51:20.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.085+0000 7fbc8d050b80 10 client.24998 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T01:51:20.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.085+0000 7fbc8d050b80 20 client.24998 populate_metadata read hostname 'smithi001' 2024-04-26T01:51:20.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.085+0000 7fbc8d050b80 1 --2- 172.21.15.1:0/350399781 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x556ffb2026c0 0x556ffb2047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:20.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.085+0000 7fbc8d050b80 4 client.24998 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:51:20.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.085+0000 7fbc8d050b80 1 -- 172.21.15.1:0/350399781 --> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ffb149630 con 0x556ffb2026c0 2024-04-26T01:51:20.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.086+0000 7fbc72a00700 1 --2- 172.21.15.1:0/350399781 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x556ffb2026c0 0x556ffb2047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:20.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.086+0000 7fbc72a00700 1 --2- 172.21.15.1:0/350399781 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x556ffb2026c0 0x556ffb2047d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6890bcea4ab81bd server_cookie=8c9ed6d6eddc7aef in_seq=0 out_seq=0 2024-04-26T01:51:20.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.086+0000 7fbc69600700 10 client.24998 ms_handle_connect on v2:172.21.15.173:6836/2502298793 2024-04-26T01:51:20.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.087+0000 7fbc69600700 1 -- 172.21.15.1:0/350399781 <== mds.2 v2:172.21.15.173:6836/2502298793 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ffb149630 con 0x556ffb2026c0 2024-04-26T01:51:20.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.087+0000 7fbc69600700 10 client.24998 handle_command_reply: tid=0 2024-04-26T01:51:20.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.087+0000 7fbc8d050b80 1 client.24998 shutdown 2024-04-26T01:51:20.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.088+0000 7fbc8d050b80 1 -- 172.21.15.1:0/350399781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1e9ef0 msgr2=0x556ffb1f5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:20.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.088+0000 7fbc8d050b80 1 --2- 172.21.15.1:0/350399781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1e9ef0 0x556ffb1f5c80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c006010 tx=0x7fbc6c0246a0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.088+0000 7fbc8d050b80 1 -- 172.21.15.1:0/350399781 shutdown_connections 2024-04-26T01:51:20.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.088+0000 7fbc8d050b80 1 --2- 172.21.15.1:0/350399781 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x556ffb2026c0 0x556ffb2047d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.088+0000 7fbc8d050b80 1 --2- 172.21.15.1:0/350399781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1f34c0 0x556ffb1f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.088+0000 7fbc8d050b80 1 --2- 172.21.15.1:0/350399781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1e9ef0 0x556ffb1f5c80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.088+0000 7fbc8d050b80 1 --2- 172.21.15.1:0/350399781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1ef780 0x556ffb1f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.088+0000 7fbc8d050b80 1 -- 172.21.15.1:0/350399781 >> 172.21.15.1:0/350399781 conn(0x556ffb117f00 msgr2=0x556ffb1e3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:20.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.088+0000 7fbc69600700 0 client.24998 ms_handle_reset on v2:172.21.15.173:6836/2502298793 2024-04-26T01:51:20.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.088+0000 7fbc8d050b80 1 -- 172.21.15.1:0/350399781 shutdown_connections 2024-04-26T01:51:20.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.088+0000 7fbc8d050b80 1 -- 172.21.15.1:0/350399781 wait complete. 2024-04-26T01:51:20.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.088+0000 7fbc8d050b80 20 client.24998 trim_cache size 0 max 16384 2024-04-26T01:51:20.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.111+0000 7fbc8d050b80 1 Processor -- start 2024-04-26T01:51:20.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.111+0000 7fbc8d050b80 1 -- start start 2024-04-26T01:51:20.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.111+0000 7fbc8d050b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1e9ef0 0x556ffb1e99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:20.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.111+0000 7fbc8d050b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1ef780 0x556ffb1ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:20.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.111+0000 7fbc8d050b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1f34c0 0x556ffb1e45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:20.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.111+0000 7fbc8d050b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556ffb202b10 con 0x556ffb1f34c0 2024-04-26T01:51:20.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.111+0000 7fbc8d050b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556ffb2031e0 con 0x556ffb1ef780 2024-04-26T01:51:20.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.111+0000 7fbc8d050b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556ffb2038b0 con 0x556ffb1e9ef0 2024-04-26T01:51:20.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.112+0000 7fbc7be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1f34c0 0x556ffb1e45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:20.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.112+0000 7fbc7b400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1e9ef0 0x556ffb1e99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:20.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.112+0000 7fbc72a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1ef780 0x556ffb1ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:20.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.112+0000 7fbc7be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1f34c0 0x556ffb1e45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35416/0 (socket says 172.21.15.1:35416) 2024-04-26T01:51:20.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.112+0000 7fbc7be00700 1 -- 172.21.15.1:0/1303962767 learned_addr learned my addr 172.21.15.1:0/1303962767 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:20.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.112+0000 7fbc7b400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1e9ef0 0x556ffb1e99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:33600/0 (socket says 172.21.15.1:33600) 2024-04-26T01:51:20.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.112+0000 7fbc72a00700 1 -- 172.21.15.1:0/1303962767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1e9ef0 msgr2=0x556ffb1e99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:20.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.112+0000 7fbc72a00700 1 --2- 172.21.15.1:0/1303962767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1e9ef0 0x556ffb1e99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.112+0000 7fbc72a00700 1 -- 172.21.15.1:0/1303962767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1f34c0 msgr2=0x556ffb1e45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:20.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.112+0000 7fbc72a00700 1 --2- 172.21.15.1:0/1303962767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1f34c0 0x556ffb1e45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.112+0000 7fbc72a00700 1 -- 172.21.15.1:0/1303962767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ffb1f6260 con 0x556ffb1ef780 2024-04-26T01:51:20.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.112+0000 7fbc7be00700 1 --2- 172.21.15.1:0/1303962767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1f34c0 0x556ffb1e45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:20.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.112+0000 7fbc72a00700 1 --2- 172.21.15.1:0/1303962767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1ef780 0x556ffb1ffe10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x556ffb205190 tx=0x7fbc64009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2039a139ce29ab79 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:20.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.113+0000 7fbc6be00700 1 -- 172.21.15.1:0/1303962767 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc64024070 con 0x556ffb1ef780 2024-04-26T01:51:20.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.113+0000 7fbc6be00700 1 -- 172.21.15.1:0/1303962767 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc64010e10 con 0x556ffb1ef780 2024-04-26T01:51:20.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.113+0000 7fbc6be00700 1 -- 172.21.15.1:0/1303962767 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6401f070 con 0x556ffb1ef780 2024-04-26T01:51:20.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.113+0000 7fbc8d050b80 1 -- 172.21.15.1:0/1303962767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1ef780 msgr2=0x556ffb1ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:20.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.113+0000 7fbc8d050b80 1 --2- 172.21.15.1:0/1303962767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1ef780 0x556ffb1ffe10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x556ffb205190 tx=0x7fbc64009260 comp rx=0 tx=0).stop 2024-04-26T01:51:20.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.113+0000 7fbc8d050b80 1 -- 172.21.15.1:0/1303962767 shutdown_connections 2024-04-26T01:51:20.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.113+0000 7fbc8d050b80 1 --2- 172.21.15.1:0/1303962767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1f34c0 0x556ffb1e45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.113+0000 7fbc8d050b80 1 --2- 172.21.15.1:0/1303962767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1e9ef0 0x556ffb1e99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.113+0000 7fbc8d050b80 1 --2- 172.21.15.1:0/1303962767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1ef780 0x556ffb1ffe10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.113+0000 7fbc8d050b80 1 -- 172.21.15.1:0/1303962767 >> 172.21.15.1:0/1303962767 conn(0x556ffb117f00 msgr2=0x556ffb1f2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:20.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.114+0000 7fbc8d050b80 1 -- 172.21.15.1:0/1303962767 shutdown_connections 2024-04-26T01:51:20.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.114+0000 7fbc8d050b80 1 -- 172.21.15.1:0/1303962767 wait complete. 2024-04-26T01:51:20.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.114+0000 7fbc8d050b80 1 -- start start 2024-04-26T01:51:20.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.114+0000 7fbc8d050b80 1 Processor -- start 2024-04-26T01:51:20.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.115+0000 7fbc8d050b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1e9ef0 0x556ffb20dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:20.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.115+0000 7fbc8d050b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1ef780 0x556ffb20e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:20.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.115+0000 7fbc8d050b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1f34c0 0x556ffb208c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:20.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.115+0000 7fbc8d050b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556ffb204650 con 0x556ffb1f34c0 2024-04-26T01:51:20.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.115+0000 7fbc8d050b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556ffb203f80 con 0x556ffb1ef780 2024-04-26T01:51:20.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.115+0000 7fbc8d050b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556ffb204d20 con 0x556ffb1e9ef0 2024-04-26T01:51:20.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.115+0000 7fbc8d050b80 10 client.0 _ll_register_callbacks cb 0x556ffb0d46d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:51:20.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.115+0000 7fbc7be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1f34c0 0x556ffb208c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:20.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.115+0000 7fbc7be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1f34c0 0x556ffb208c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35426/0 (socket says 172.21.15.1:35426) 2024-04-26T01:51:20.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.115+0000 7fbc7be00700 1 -- 172.21.15.1:0/1908928595 learned_addr learned my addr 172.21.15.1:0/1908928595 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:20.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.115+0000 7fbc7b400700 1 --2- 172.21.15.1:0/1908928595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1e9ef0 0x556ffb20dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:20.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.115+0000 7fbc72a00700 1 --2- 172.21.15.1:0/1908928595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1ef780 0x556ffb20e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:20.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.115+0000 7fbc72a00700 1 -- 172.21.15.1:0/1908928595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1e9ef0 msgr2=0x556ffb20dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:20.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.116+0000 7fbc72a00700 1 --2- 172.21.15.1:0/1908928595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1e9ef0 0x556ffb20dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.116+0000 7fbc72a00700 1 -- 172.21.15.1:0/1908928595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1f34c0 msgr2=0x556ffb208c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:20.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.116+0000 7fbc72a00700 1 --2- 172.21.15.1:0/1908928595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1f34c0 0x556ffb208c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.116+0000 7fbc72a00700 1 -- 172.21.15.1:0/1908928595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ffb209270 con 0x556ffb1ef780 2024-04-26T01:51:20.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.116+0000 7fbc7be00700 1 --2- 172.21.15.1:0/1908928595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1f34c0 0x556ffb208c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:20.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.116+0000 7fbc72a00700 1 --2- 172.21.15.1:0/1908928595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1ef780 0x556ffb20e190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbc64005c20 tx=0x7fbc64010ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34bf1bbf8a71bd42 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:20.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.116+0000 7fbc69600700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:51:20.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.116+0000 7fbc69600700 1 -- 172.21.15.1:0/1908928595 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc64024040 con 0x556ffb1ef780 2024-04-26T01:51:20.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.116+0000 7fbc8d050b80 1 -- 172.21.15.1:0/1908928595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc64010e10 con 0x556ffb1ef780 2024-04-26T01:51:20.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.116+0000 7fbc69600700 1 -- 172.21.15.1:0/1908928595 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fbc640040d0 con 0x556ffb1ef780 2024-04-26T01:51:20.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.117+0000 7fbc69600700 1 -- 172.21.15.1:0/1908928595 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc64004310 con 0x556ffb1ef780 2024-04-26T01:51:20.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.117+0000 7fbc8d050b80 10 client.25004 fetch_fsmap learned FSMap version 25 2024-04-26T01:51:20.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.117+0000 7fbc8d050b80 1 -- 172.21.15.1:0/1908928595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556ffb26adf0 con 0x556ffb1ef780 2024-04-26T01:51:20.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.117+0000 7fbc69600700 1 -- 172.21.15.1:0/1908928595 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc64010e10 con 0x556ffb1ef780 2024-04-26T01:51:20.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.117+0000 7fbc69600700 1 -- 172.21.15.1:0/1908928595 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fbc6401f180 con 0x556ffb1ef780 2024-04-26T01:51:20.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.117+0000 7fbc8d050b80 10 client.25004 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:51:20.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.117+0000 7fbc8d050b80 10 client.25004 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T01:51:20.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.117+0000 7fbc8d050b80 20 client.25004 populate_metadata read hostname 'smithi001' 2024-04-26T01:51:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.117+0000 7fbc8d050b80 1 --2- 172.21.15.1:0/1908928595 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x556ffb26f590 0x556ffb26e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.117+0000 7fbc8d050b80 4 client.25004 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T01:51:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.117+0000 7fbc8d050b80 1 -- 172.21.15.1:0/1908928595 --> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] -- command(tid 0: {"prefix": "status"}) -- 0x556ffb1436d0 con 0x556ffb26f590 2024-04-26T01:51:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.118+0000 7fbc7b400700 1 --2- 172.21.15.1:0/1908928595 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x556ffb26f590 0x556ffb26e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.118+0000 7fbc7b400700 1 --2- 172.21.15.1:0/1908928595 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x556ffb26f590 0x556ffb26e2a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f0f86876c546e0b6 server_cookie=e8fcccfab5fb3108 in_seq=0 out_seq=0 2024-04-26T01:51:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.118+0000 7fbc69600700 10 client.25004 ms_handle_connect on v2:172.21.15.173:6836/2502298793 2024-04-26T01:51:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.119+0000 7fbc69600700 1 -- 172.21.15.1:0/1908928595 <== mds.2 v2:172.21.15.173:6836/2502298793 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556ffb1436d0 con 0x556ffb26f590 2024-04-26T01:51:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.119+0000 7fbc69600700 10 client.25004 handle_command_reply: tid=0 2024-04-26T01:51:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.119+0000 7fbc8d050b80 1 client.25004 shutdown 2024-04-26T01:51:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.119+0000 7fbc8d050b80 1 -- 172.21.15.1:0/1908928595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1ef780 msgr2=0x556ffb20e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.119+0000 7fbc8d050b80 1 --2- 172.21.15.1:0/1908928595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1ef780 0x556ffb20e190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbc64005c20 tx=0x7fbc64010ca0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.119+0000 7fbc8d050b80 1 -- 172.21.15.1:0/1908928595 shutdown_connections 2024-04-26T01:51:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.119+0000 7fbc8d050b80 1 --2- 172.21.15.1:0/1908928595 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x556ffb26f590 0x556ffb26e2a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.119+0000 7fbc8d050b80 1 --2- 172.21.15.1:0/1908928595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556ffb1f34c0 0x556ffb208c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.119+0000 7fbc8d050b80 1 --2- 172.21.15.1:0/1908928595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ffb1e9ef0 0x556ffb20dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.119+0000 7fbc8d050b80 1 --2- 172.21.15.1:0/1908928595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556ffb1ef780 0x556ffb20e190 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.119+0000 7fbc69600700 0 client.25004 ms_handle_reset on v2:172.21.15.173:6836/2502298793 2024-04-26T01:51:20.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.119+0000 7fbc8d050b80 1 -- 172.21.15.1:0/1908928595 >> 172.21.15.1:0/1908928595 conn(0x556ffb117f00 msgr2=0x556ffb1fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:20.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.119+0000 7fbc8d050b80 1 -- 172.21.15.1:0/1908928595 shutdown_connections 2024-04-26T01:51:20.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.119+0000 7fbc8d050b80 1 -- 172.21.15.1:0/1908928595 wait complete. 2024-04-26T01:51:20.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.119+0000 7fbc8d050b80 20 client.25004 trim_cache size 0 max 16384 2024-04-26T01:51:20.121 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T01:51:20.121 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "77cfcc80-036d-11ef-bc93-c7b262605968", 2024-04-26T01:51:20.121 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 2, 2024-04-26T01:51:20.121 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24499, 2024-04-26T01:51:20.121 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-04-26T01:51:20.122 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-04-26T01:51:20.122 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-04-26T01:51:20.122 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 212.90544245300001, 2024-04-26T01:51:20.122 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 25, 2024-04-26T01:51:20.122 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 92, 2024-04-26T01:51:20.122 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T01:51:20.122 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 268.81252063199997 2024-04-26T01:51:20.122 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T01:51:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.120+0000 7fbc82600700 1 -- 172.21.15.1:0/9145064 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbc600a3c40 msgr2=0x7fbc600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.120+0000 7fbc82600700 1 --2- 172.21.15.1:0/9145064 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbc600a3c40 0x7fbc600a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c009990 tx=0x7fbc6c005ab0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.120+0000 7fbc82600700 1 -- 172.21.15.1:0/9145064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc7c109fc0 msgr2=0x7fbc7c19fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.120+0000 7fbc82600700 1 --2- 172.21.15.1:0/9145064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc7c109fc0 0x7fbc7c19fc80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400efc0 tx=0x7fbc7400c3a0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.120+0000 7fbc82600700 1 -- 172.21.15.1:0/9145064 shutdown_connections 2024-04-26T01:51:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.120+0000 7fbc82600700 1 --2- 172.21.15.1:0/9145064 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fbc600a3c40 0x7fbc600a6100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.120+0000 7fbc82600700 1 --2- 172.21.15.1:0/9145064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc7c109fc0 0x7fbc7c19fc80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.120+0000 7fbc82600700 1 --2- 172.21.15.1:0/9145064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc7c104ee0 0x7fbc7c19b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.120+0000 7fbc82600700 1 --2- 172.21.15.1:0/9145064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc7c105890 0x7fbc7c19b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.120+0000 7fbc82600700 1 -- 172.21.15.1:0/9145064 >> 172.21.15.1:0/9145064 conn(0x7fbc7c100770 msgr2=0x7fbc7c102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.120+0000 7fbc82600700 1 -- 172.21.15.1:0/9145064 shutdown_connections 2024-04-26T01:51:20.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:20.120+0000 7fbc82600700 1 -- 172.21.15.1:0/9145064 wait complete. 2024-04-26T01:51:20.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:19 smithi097 ceph-mon[32853]: pgmap v441: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:20.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:19 smithi173 ceph-mon[32553]: pgmap v441: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:20.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:19 smithi001 ceph-mon[27822]: pgmap v441: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:21.119 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-26T01:51:21.570 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:22.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:21 smithi173 ceph-mon[32553]: pgmap v442: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:51:22.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:21 smithi001 ceph-mon[27822]: pgmap v442: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:51:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:21 smithi097 ceph-mon[32853]: pgmap v442: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:51:23.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.741+0000 7f2ed8200700 1 -- 172.21.15.1:0/3621941034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ed0105870 msgr2=0x7f2ed01098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:23.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.741+0000 7f2ed8200700 1 --2- 172.21.15.1:0/3621941034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ed0105870 0x7f2ed01098c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0099c0 tx=0x7f2ebc009cd0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.741+0000 7f2ed8200700 1 -- 172.21.15.1:0/3621941034 shutdown_connections 2024-04-26T01:51:23.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.741+0000 7f2ed8200700 1 --2- 172.21.15.1:0/3621941034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ed0109fa0 0x7f2ed0111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.741+0000 7f2ed8200700 1 --2- 172.21.15.1:0/3621941034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed0104ec0 0x7f2ed01052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.741+0000 7f2ed8200700 1 --2- 172.21.15.1:0/3621941034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ed0105870 0x7f2ed01098c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.741+0000 7f2ed8200700 1 -- 172.21.15.1:0/3621941034 >> 172.21.15.1:0/3621941034 conn(0x7f2ed0100750 msgr2=0x7f2ed0102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:23.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.741+0000 7f2ed8200700 1 -- 172.21.15.1:0/3621941034 shutdown_connections 2024-04-26T01:51:23.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.741+0000 7f2ed8200700 1 -- 172.21.15.1:0/3621941034 wait complete. 2024-04-26T01:51:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.742+0000 7f2ed8200700 1 Processor -- start 2024-04-26T01:51:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.742+0000 7f2ed8200700 1 -- start start 2024-04-26T01:51:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.742+0000 7f2ed8200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ed0104ec0 0x7f2ed019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.742+0000 7f2ed8200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed0105870 0x7f2ed019b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.742+0000 7f2ed8200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ed0109fa0 0x7f2ed019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.742+0000 7f2ed8200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed0114e60 con 0x7f2ed0109fa0 2024-04-26T01:51:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.742+0000 7f2ed8200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed0114ce0 con 0x7f2ed0105870 2024-04-26T01:51:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.742+0000 7f2ed8200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed0114fe0 con 0x7f2ed0104ec0 2024-04-26T01:51:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.743+0000 7f2ed4e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed0105870 0x7f2ed019b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.743+0000 7f2ed4e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed0105870 0x7f2ed019b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:52506/0 (socket says 172.21.15.1:52506) 2024-04-26T01:51:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.743+0000 7f2ed6200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ed0109fa0 0x7f2ed019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.743+0000 7f2ed4e00700 1 -- 172.21.15.1:0/1883980206 learned_addr learned my addr 172.21.15.1:0/1883980206 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.743+0000 7f2ed5800700 1 --2- 172.21.15.1:0/1883980206 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ed0104ec0 0x7f2ed019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.743+0000 7f2ed4e00700 1 -- 172.21.15.1:0/1883980206 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ed0104ec0 msgr2=0x7f2ed019b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.743+0000 7f2ed4e00700 1 --2- 172.21.15.1:0/1883980206 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ed0104ec0 0x7f2ed019b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.743+0000 7f2ed4e00700 1 -- 172.21.15.1:0/1883980206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ed0109fa0 msgr2=0x7f2ed019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.743+0000 7f2ed4e00700 1 --2- 172.21.15.1:0/1883980206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ed0109fa0 0x7f2ed019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.743+0000 7f2ed4e00700 1 -- 172.21.15.1:0/1883980206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed01a0390 con 0x7f2ed0105870 2024-04-26T01:51:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.743+0000 7f2ed4e00700 1 --2- 172.21.15.1:0/1883980206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed0105870 0x7f2ed019b910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00b580 tx=0x7f2ebc0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.744+0000 7f2ece000700 1 -- 172.21.15.1:0/1883980206 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc009e60 con 0x7f2ed0105870 2024-04-26T01:51:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.744+0000 7f2ece000700 1 -- 172.21.15.1:0/1883980206 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ebc0047c0 con 0x7f2ed0105870 2024-04-26T01:51:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.744+0000 7f2ece000700 1 -- 172.21.15.1:0/1883980206 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc005270 con 0x7f2ed0105870 2024-04-26T01:51:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.744+0000 7f2ed8200700 1 -- 172.21.15.1:0/1883980206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed01a0620 con 0x7f2ed0105870 2024-04-26T01:51:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.744+0000 7f2ed8200700 1 -- 172.21.15.1:0/1883980206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed01a0a50 con 0x7f2ed0105870 2024-04-26T01:51:23.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.747+0000 7f2ece000700 1 -- 172.21.15.1:0/1883980206 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f2ebc004970 con 0x7f2ed0105870 2024-04-26T01:51:23.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.748+0000 7f2ee2a9cb80 1 Processor -- start 2024-04-26T01:51:23.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.748+0000 7f2ee2a9cb80 1 -- start start 2024-04-26T01:51:23.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.749+0000 7f2ee2a9cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6ca780 0x5646fa6cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.749+0000 7f2ece000700 1 --2- 172.21.15.1:0/1883980206 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2eb80a3c40 0x7f2eb80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.749+0000 7f2ee2a9cb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6cb0a0 0x5646fa6c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.749+0000 7f2ee2a9cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6c4ef0 0x5646fa6c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.749+0000 7f2ee2a9cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5646fa616e40 con 0x5646fa6c4ef0 2024-04-26T01:51:23.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.749+0000 7f2ee2a9cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5646fa5bd6e0 con 0x5646fa6ca780 2024-04-26T01:51:23.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.749+0000 7f2ee2a9cb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5646fa612e00 con 0x5646fa6cb0a0 2024-04-26T01:51:23.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.749+0000 7f2ece000700 1 -- 172.21.15.1:0/1883980206 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f2ebc0d0d40 con 0x7f2ed0105870 2024-04-26T01:51:23.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.749+0000 7f2ed5800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6ca780 0x5646fa6cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.749+0000 7f2ed4e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6cb0a0 0x5646fa6c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.749+0000 7f2ed6200700 1 --2- 172.21.15.1:0/1883980206 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2eb80a3c40 0x7f2eb80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.749+0000 7f2ed5800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6ca780 0x5646fa6cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:52520/0 (socket says 172.21.15.1:52520) 2024-04-26T01:51:23.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.749+0000 7f2ed5800700 1 -- 172.21.15.1:0/509086093 learned_addr learned my addr 172.21.15.1:0/509086093 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:23.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.749+0000 7f2ed6200700 1 --2- 172.21.15.1:0/509086093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6c4ef0 0x5646fa6c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.750+0000 7f2ed5800700 1 -- 172.21.15.1:0/509086093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6cb0a0 msgr2=0x5646fa6c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:23.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.750+0000 7f2ed5800700 1 --2- 172.21.15.1:0/509086093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6cb0a0 0x5646fa6c49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.750+0000 7f2ed5800700 1 -- 172.21.15.1:0/509086093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6c4ef0 msgr2=0x5646fa6c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:23.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.750+0000 7f2ed5800700 1 --2- 172.21.15.1:0/509086093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6c4ef0 0x5646fa6c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.750+0000 7f2ed5800700 1 -- 172.21.15.1:0/509086093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646fa5b3630 con 0x5646fa6ca780 2024-04-26T01:51:23.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.750+0000 7f2ed4e00700 1 --2- 172.21.15.1:0/509086093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6cb0a0 0x5646fa6c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:23.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.750+0000 7f2ed5800700 1 --2- 172.21.15.1:0/509086093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6ca780 0x5646fa6cab60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009d80 tx=0x7f2ec4011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7989b1f09806d10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:23.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.750+0000 7f2ed6200700 1 --2- 172.21.15.1:0/1883980206 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2eb80a3c40 0x7f2eb80a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8007be0 tx=0x7f2ec800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:23.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.750+0000 7f2ec2a00700 1 -- 172.21.15.1:0/509086093 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4004230 con 0x5646fa6ca780 2024-04-26T01:51:23.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.750+0000 7f2ec2a00700 1 -- 172.21.15.1:0/509086093 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ec40043d0 con 0x5646fa6ca780 2024-04-26T01:51:23.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.750+0000 7f2ec2a00700 1 -- 172.21.15.1:0/509086093 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4019830 con 0x5646fa6ca780 2024-04-26T01:51:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.751+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/509086093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6ca780 msgr2=0x5646fa6cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.751+0000 7f2ee2a9cb80 1 --2- 172.21.15.1:0/509086093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6ca780 0x5646fa6cab60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009d80 tx=0x7f2ec4011720 comp rx=0 tx=0).stop 2024-04-26T01:51:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.751+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/509086093 shutdown_connections 2024-04-26T01:51:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.751+0000 7f2ee2a9cb80 1 --2- 172.21.15.1:0/509086093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6c4ef0 0x5646fa6c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.751+0000 7f2ee2a9cb80 1 --2- 172.21.15.1:0/509086093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6ca780 0x5646fa6cab60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.751+0000 7f2ee2a9cb80 1 --2- 172.21.15.1:0/509086093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6cb0a0 0x5646fa6c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.751+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/509086093 >> 172.21.15.1:0/509086093 conn(0x5646fa5f2f00 msgr2=0x5646fa6bf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.751+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/509086093 shutdown_connections 2024-04-26T01:51:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.751+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/509086093 wait complete. 2024-04-26T01:51:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.752+0000 7f2ee2a9cb80 1 -- start start 2024-04-26T01:51:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.752+0000 7f2ee2a9cb80 1 Processor -- start 2024-04-26T01:51:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.753+0000 7f2ee2a9cb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6c4ef0 0x5646fa6d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.753+0000 7f2ee2a9cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6ca780 0x5646fa6cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.753+0000 7f2ee2a9cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6ce510 0x5646fa6ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.753+0000 7f2ee2a9cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5646fa6d1920 con 0x5646fa6ce510 2024-04-26T01:51:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.753+0000 7f2ee2a9cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5646fa5d0810 con 0x5646fa6ca780 2024-04-26T01:51:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.753+0000 7f2ee2a9cb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5646fa6d1aa0 con 0x5646fa6c4ef0 2024-04-26T01:51:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.753+0000 7f2ee2a9cb80 10 client.0 _ll_register_callbacks cb 0x5646fa5af6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:51:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.753+0000 7f2ed6200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6ce510 0x5646fa6ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.753+0000 7f2ed4e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6ca780 0x5646fa6cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.753+0000 7f2ed6200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6ce510 0x5646fa6ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56786/0 (socket says 172.21.15.1:56786) 2024-04-26T01:51:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.753+0000 7f2ed6200700 1 -- 172.21.15.1:0/1605890723 learned_addr learned my addr 172.21.15.1:0/1605890723 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.753+0000 7f2ed5800700 1 --2- 172.21.15.1:0/1605890723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6c4ef0 0x5646fa6d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.753+0000 7f2ed6200700 1 -- 172.21.15.1:0/1605890723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6c4ef0 msgr2=0x5646fa6d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.753+0000 7f2ed6200700 1 --2- 172.21.15.1:0/1605890723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6c4ef0 0x5646fa6d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.753+0000 7f2ed6200700 1 -- 172.21.15.1:0/1605890723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6ca780 msgr2=0x5646fa6cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.753+0000 7f2ed6200700 1 --2- 172.21.15.1:0/1605890723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6ca780 0x5646fa6cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.753+0000 7f2ed6200700 1 -- 172.21.15.1:0/1605890723 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646fa564880 con 0x5646fa6ce510 2024-04-26T01:51:23.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.753+0000 7f2ed4e00700 1 --2- 172.21.15.1:0/1605890723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6ca780 0x5646fa6cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.754+0000 7f2ed6200700 1 --2- 172.21.15.1:0/1605890723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6ce510 0x5646fa6ce990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8003d30 tx=0x7f2ec8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69693388cdc97353 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.754+0000 7f2eb7400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:51:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.754+0000 7f2eb7400700 1 -- 172.21.15.1:0/1605890723 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8003d60 con 0x5646fa6ce510 2024-04-26T01:51:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.754+0000 7f2eb7400700 1 -- 172.21.15.1:0/1605890723 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f2ec8012040 con 0x5646fa6ce510 2024-04-26T01:51:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.754+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/1605890723 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ec40043d0 con 0x5646fa6ce510 2024-04-26T01:51:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.754+0000 7f2eb7400700 1 -- 172.21.15.1:0/1605890723 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ec8003f00 con 0x5646fa6ce510 2024-04-26T01:51:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.754+0000 7f2ee2a9cb80 10 client.14913 fetch_fsmap learned FSMap version 25 2024-04-26T01:51:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.754+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/1605890723 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5646fa5adaa0 con 0x5646fa6ce510 2024-04-26T01:51:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.755+0000 7f2eb7400700 1 -- 172.21.15.1:0/1605890723 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ec40043d0 con 0x5646fa6ce510 2024-04-26T01:51:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.755+0000 7f2eb7400700 1 -- 172.21.15.1:0/1605890723 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f2ec8018040 con 0x5646fa6ce510 2024-04-26T01:51:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.755+0000 7f2ee2a9cb80 10 client.14913 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:51:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.755+0000 7f2ee2a9cb80 10 client.14913 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T01:51:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.755+0000 7f2ee2a9cb80 20 client.14913 populate_metadata read hostname 'smithi001' 2024-04-26T01:51:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.755+0000 7f2ee2a9cb80 1 --2- 172.21.15.1:0/1605890723 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x5646fa6deaa0 0x5646fa6dff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.755+0000 7f2ee2a9cb80 4 client.14913 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:51:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.755+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/1605890723 --> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646fa624630 con 0x5646fa6deaa0 2024-04-26T01:51:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.755+0000 7f2ed5800700 1 --2- 172.21.15.1:0/1605890723 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x5646fa6deaa0 0x5646fa6dff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.756+0000 7f2ed5800700 1 --2- 172.21.15.1:0/1605890723 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x5646fa6deaa0 0x5646fa6dff10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ed95ce3655e0807 server_cookie=94b74ada484bd6c0 in_seq=0 out_seq=0 2024-04-26T01:51:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.756+0000 7f2eb7400700 10 client.14913 ms_handle_connect on v2:172.21.15.173:6838/373692430 2024-04-26T01:51:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.757+0000 7f2eb7400700 1 -- 172.21.15.1:0/1605890723 <== mds.1 v2:172.21.15.173:6838/373692430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5646fa624630 con 0x5646fa6deaa0 2024-04-26T01:51:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.757+0000 7f2eb7400700 10 client.14913 handle_command_reply: tid=0 2024-04-26T01:51:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.757+0000 7f2ee2a9cb80 1 client.14913 shutdown 2024-04-26T01:51:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.757+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/1605890723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6ce510 msgr2=0x5646fa6ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.757+0000 7f2ee2a9cb80 1 --2- 172.21.15.1:0/1605890723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6ce510 0x5646fa6ce990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8003d30 tx=0x7f2ec8004b40 comp rx=0 tx=0).stop 2024-04-26T01:51:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.757+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/1605890723 shutdown_connections 2024-04-26T01:51:23.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.757+0000 7f2ee2a9cb80 1 --2- 172.21.15.1:0/1605890723 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x5646fa6deaa0 0x5646fa6dff10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.757+0000 7f2ee2a9cb80 1 --2- 172.21.15.1:0/1605890723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6ce510 0x5646fa6ce990 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.757+0000 7f2ee2a9cb80 1 --2- 172.21.15.1:0/1605890723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6c4ef0 0x5646fa6d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.757+0000 7f2ee2a9cb80 1 --2- 172.21.15.1:0/1605890723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6ca780 0x5646fa6cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.757+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/1605890723 >> 172.21.15.1:0/1605890723 conn(0x5646fa5f2f00 msgr2=0x5646fa6bee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:23.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.757+0000 7f2eb7400700 0 client.14913 ms_handle_reset on v2:172.21.15.173:6838/373692430 2024-04-26T01:51:23.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.757+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/1605890723 shutdown_connections 2024-04-26T01:51:23.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.757+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/1605890723 wait complete. 2024-04-26T01:51:23.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.758+0000 7f2ee2a9cb80 20 client.14913 trim_cache size 0 max 16384 2024-04-26T01:51:23.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.779+0000 7f2ee2a9cb80 1 Processor -- start 2024-04-26T01:51:23.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.779+0000 7f2ee2a9cb80 1 -- start start 2024-04-26T01:51:23.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.779+0000 7f2ee2a9cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6c4ef0 0x5646fa6c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.779+0000 7f2ee2a9cb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6ca780 0x5646fa6daea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.779+0000 7f2ee2a9cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6ce510 0x5646fa6bf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.779+0000 7f2ee2a9cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5646fa6de250 con 0x5646fa6ce510 2024-04-26T01:51:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.779+0000 7f2ee2a9cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5646fa6de920 con 0x5646fa6c4ef0 2024-04-26T01:51:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.779+0000 7f2ee2a9cb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5646fa6deff0 con 0x5646fa6ca780 2024-04-26T01:51:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.780+0000 7f2ed6200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6ce510 0x5646fa6bf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.780+0000 7f2ed6200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6ce510 0x5646fa6bf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56792/0 (socket says 172.21.15.1:56792) 2024-04-26T01:51:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.780+0000 7f2ed6200700 1 -- 172.21.15.1:0/4041430353 learned_addr learned my addr 172.21.15.1:0/4041430353 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.780+0000 7f2ed5800700 1 --2- 172.21.15.1:0/4041430353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6c4ef0 0x5646fa6c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.780+0000 7f2ed4e00700 1 --2- 172.21.15.1:0/4041430353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6ca780 0x5646fa6daea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.780+0000 7f2ed6200700 1 -- 172.21.15.1:0/4041430353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6ca780 msgr2=0x5646fa6daea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.780+0000 7f2ed6200700 1 --2- 172.21.15.1:0/4041430353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6ca780 0x5646fa6daea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.780+0000 7f2ed6200700 1 -- 172.21.15.1:0/4041430353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6c4ef0 msgr2=0x5646fa6c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.780+0000 7f2ed6200700 1 --2- 172.21.15.1:0/4041430353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6c4ef0 0x5646fa6c49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.780+0000 7f2ed6200700 1 -- 172.21.15.1:0/4041430353 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646fa6cf570 con 0x5646fa6ce510 2024-04-26T01:51:23.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.780+0000 7f2ed5800700 1 --2- 172.21.15.1:0/4041430353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6c4ef0 0x5646fa6c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:23.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.780+0000 7f2ed6200700 1 --2- 172.21.15.1:0/4041430353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6ce510 0x5646fa6bf5d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8006070 tx=0x7f2ec8009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1753f9a2b89b61 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:23.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.781+0000 7f2ec2000700 1 -- 172.21.15.1:0/4041430353 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8007910 con 0x5646fa6ce510 2024-04-26T01:51:23.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.781+0000 7f2ec2000700 1 -- 172.21.15.1:0/4041430353 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ec8035020 con 0x5646fa6ce510 2024-04-26T01:51:23.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.781+0000 7f2ec2000700 1 -- 172.21.15.1:0/4041430353 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8025e10 con 0x5646fa6ce510 2024-04-26T01:51:23.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.781+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/4041430353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6ce510 msgr2=0x5646fa6bf5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:23.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.781+0000 7f2ee2a9cb80 1 --2- 172.21.15.1:0/4041430353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6ce510 0x5646fa6bf5d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8006070 tx=0x7f2ec8009900 comp rx=0 tx=0).stop 2024-04-26T01:51:23.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.781+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/4041430353 shutdown_connections 2024-04-26T01:51:23.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.781+0000 7f2ee2a9cb80 1 --2- 172.21.15.1:0/4041430353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6ce510 0x5646fa6bf5d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.781+0000 7f2ee2a9cb80 1 --2- 172.21.15.1:0/4041430353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6c4ef0 0x5646fa6c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.781+0000 7f2ee2a9cb80 1 --2- 172.21.15.1:0/4041430353 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6ca780 0x5646fa6daea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.781+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/4041430353 >> 172.21.15.1:0/4041430353 conn(0x5646fa5f2f00 msgr2=0x5646fa6cc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:23.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.782+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/4041430353 shutdown_connections 2024-04-26T01:51:23.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.782+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/4041430353 wait complete. 2024-04-26T01:51:23.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.782+0000 7f2ee2a9cb80 1 -- start start 2024-04-26T01:51:23.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.782+0000 7f2ee2a9cb80 1 Processor -- start 2024-04-26T01:51:23.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.783+0000 7f2ee2a9cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6c4ef0 0x5646fa6bf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.783+0000 7f2ee2a9cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6ca780 0x5646fa6bf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.783+0000 7f2ee2a9cb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6ce510 0x5646fa6e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.783+0000 7f2ee2a9cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5646fa6dfd90 con 0x5646fa6c4ef0 2024-04-26T01:51:23.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.783+0000 7f2ee2a9cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5646fa6df6c0 con 0x5646fa6ca780 2024-04-26T01:51:23.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.783+0000 7f2ee2a9cb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5646fa6e0460 con 0x5646fa6ce510 2024-04-26T01:51:23.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.783+0000 7f2ee2a9cb80 10 client.0 _ll_register_callbacks cb 0x5646fa5af6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:51:23.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.783+0000 7f2ed5800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6c4ef0 0x5646fa6bf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.783+0000 7f2ed5800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6c4ef0 0x5646fa6bf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56808/0 (socket says 172.21.15.1:56808) 2024-04-26T01:51:23.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.783+0000 7f2ed5800700 1 -- 172.21.15.1:0/4118157922 learned_addr learned my addr 172.21.15.1:0/4118157922 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.783+0000 7f2ed4e00700 1 --2- 172.21.15.1:0/4118157922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6ca780 0x5646fa6bf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.783+0000 7f2ed6200700 1 --2- 172.21.15.1:0/4118157922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6ce510 0x5646fa6e7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.783+0000 7f2ed5800700 1 -- 172.21.15.1:0/4118157922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6ce510 msgr2=0x5646fa6e7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.783+0000 7f2ed5800700 1 --2- 172.21.15.1:0/4118157922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6ce510 0x5646fa6e7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.783+0000 7f2ed5800700 1 -- 172.21.15.1:0/4118157922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6ca780 msgr2=0x5646fa6bf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.783+0000 7f2ed5800700 1 --2- 172.21.15.1:0/4118157922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6ca780 0x5646fa6bf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.783+0000 7f2ed5800700 1 -- 172.21.15.1:0/4118157922 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646fa6e7f50 con 0x5646fa6c4ef0 2024-04-26T01:51:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.783+0000 7f2ed6200700 1 --2- 172.21.15.1:0/4118157922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6ce510 0x5646fa6e7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:51:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.784+0000 7f2ed5800700 1 --2- 172.21.15.1:0/4118157922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6c4ef0 0x5646fa6bf090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009fd0 tx=0x7f2ec40058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35883590acc96920 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.784+0000 7f2eb7400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:51:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.784+0000 7f2eb7400700 1 -- 172.21.15.1:0/4118157922 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec402b520 con 0x5646fa6c4ef0 2024-04-26T01:51:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.784+0000 7f2eb7400700 1 -- 172.21.15.1:0/4118157922 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f2ec402b6c0 con 0x5646fa6c4ef0 2024-04-26T01:51:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.784+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/4118157922 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ec8035020 con 0x5646fa6c4ef0 2024-04-26T01:51:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.784+0000 7f2eb7400700 1 -- 172.21.15.1:0/4118157922 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ec402b8e0 con 0x5646fa6c4ef0 2024-04-26T01:51:23.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.784+0000 7f2ee2a9cb80 10 client.14919 fetch_fsmap learned FSMap version 25 2024-04-26T01:51:23.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.784+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/4118157922 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5646fa7463d0 con 0x5646fa6c4ef0 2024-04-26T01:51:23.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.785+0000 7f2eb7400700 1 -- 172.21.15.1:0/4118157922 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ec8035020 con 0x5646fa6c4ef0 2024-04-26T01:51:23.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.785+0000 7f2eb7400700 1 -- 172.21.15.1:0/4118157922 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f2ec4003cd0 con 0x5646fa6c4ef0 2024-04-26T01:51:23.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.785+0000 7f2ee2a9cb80 10 client.14919 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:51:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.785+0000 7f2ee2a9cb80 10 client.14919 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T01:51:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.785+0000 7f2ee2a9cb80 20 client.14919 populate_metadata read hostname 'smithi001' 2024-04-26T01:51:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.785+0000 7f2ee2a9cb80 1 --2- 172.21.15.1:0/4118157922 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x5646fa74ae20 0x5646fa749b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.785+0000 7f2ee2a9cb80 4 client.14919 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T01:51:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.785+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/4118157922 --> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] -- command(tid 0: {"prefix": "status"}) -- 0x5646fa61e6d0 con 0x5646fa74ae20 2024-04-26T01:51:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.785+0000 7f2ed4e00700 1 --2- 172.21.15.1:0/4118157922 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x5646fa74ae20 0x5646fa749b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.786+0000 7f2ed4e00700 1 --2- 172.21.15.1:0/4118157922 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x5646fa74ae20 0x5646fa749b30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=388739b2899dc261 server_cookie=6abc7d31ce8e3a89 in_seq=0 out_seq=0 2024-04-26T01:51:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.786+0000 7f2eb7400700 10 client.14919 ms_handle_connect on v2:172.21.15.173:6838/373692430 2024-04-26T01:51:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.786+0000 7f2eb7400700 1 -- 172.21.15.1:0/4118157922 <== mds.1 v2:172.21.15.173:6838/373692430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5646fa61e6d0 con 0x5646fa74ae20 2024-04-26T01:51:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.786+0000 7f2eb7400700 10 client.14919 handle_command_reply: tid=0 2024-04-26T01:51:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.786+0000 7f2ee2a9cb80 1 client.14919 shutdown 2024-04-26T01:51:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.786+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/4118157922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6c4ef0 msgr2=0x5646fa6bf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.786+0000 7f2ee2a9cb80 1 --2- 172.21.15.1:0/4118157922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6c4ef0 0x5646fa6bf090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009fd0 tx=0x7f2ec40058e0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.786+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/4118157922 shutdown_connections 2024-04-26T01:51:23.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.786+0000 7f2ee2a9cb80 1 --2- 172.21.15.1:0/4118157922 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x5646fa74ae20 0x5646fa749b30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.786+0000 7f2ee2a9cb80 1 --2- 172.21.15.1:0/4118157922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646fa6ce510 0x5646fa6e7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.786+0000 7f2ee2a9cb80 1 --2- 172.21.15.1:0/4118157922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fa6c4ef0 0x5646fa6bf090 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.786+0000 7f2ee2a9cb80 1 --2- 172.21.15.1:0/4118157922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646fa6ca780 0x5646fa6bf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.786+0000 7f2eb7400700 0 client.14919 ms_handle_reset on v2:172.21.15.173:6838/373692430 2024-04-26T01:51:23.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.786+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/4118157922 >> 172.21.15.1:0/4118157922 conn(0x5646fa5f2f00 msgr2=0x5646fa6cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:23.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.787+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/4118157922 shutdown_connections 2024-04-26T01:51:23.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.787+0000 7f2ee2a9cb80 1 -- 172.21.15.1:0/4118157922 wait complete. 2024-04-26T01:51:23.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.787+0000 7f2ee2a9cb80 20 client.14919 trim_cache size 0 max 16384 2024-04-26T01:51:23.788 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T01:51:23.788 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "77cfcc80-036d-11ef-bc93-c7b262605968", 2024-04-26T01:51:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 1, 2024-04-26T01:51:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24514, 2024-04-26T01:51:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-04-26T01:51:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-04-26T01:51:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-04-26T01:51:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 217.58541552, 2024-04-26T01:51:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 25, 2024-04-26T01:51:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 92, 2024-04-26T01:51:23.789 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T01:51:23.790 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 266.92709938399997 2024-04-26T01:51:23.790 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T01:51:23.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.787+0000 7f2ed8200700 1 -- 172.21.15.1:0/1883980206 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2eb80a3c40 msgr2=0x7f2eb80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:23.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.787+0000 7f2ed8200700 1 --2- 172.21.15.1:0/1883980206 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2eb80a3c40 0x7f2eb80a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8007be0 tx=0x7f2ec800b040 comp rx=0 tx=0).stop 2024-04-26T01:51:23.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.787+0000 7f2ed8200700 1 -- 172.21.15.1:0/1883980206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed0105870 msgr2=0x7f2ed019b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:23.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.787+0000 7f2ed8200700 1 --2- 172.21.15.1:0/1883980206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed0105870 0x7f2ed019b910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00b580 tx=0x7f2ebc0045e0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.787+0000 7f2ed8200700 1 -- 172.21.15.1:0/1883980206 shutdown_connections 2024-04-26T01:51:23.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.787+0000 7f2ed8200700 1 --2- 172.21.15.1:0/1883980206 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f2eb80a3c40 0x7f2eb80a6100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.787+0000 7f2ed8200700 1 --2- 172.21.15.1:0/1883980206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ed0109fa0 0x7f2ed019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.788+0000 7f2ed8200700 1 --2- 172.21.15.1:0/1883980206 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ed0104ec0 0x7f2ed019b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.788+0000 7f2ed8200700 1 --2- 172.21.15.1:0/1883980206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ed0105870 0x7f2ed019b910 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:23.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.788+0000 7f2ed8200700 1 -- 172.21.15.1:0/1883980206 >> 172.21.15.1:0/1883980206 conn(0x7f2ed0100750 msgr2=0x7f2ed0101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:23.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.788+0000 7f2ed8200700 1 -- 172.21.15.1:0/1883980206 shutdown_connections 2024-04-26T01:51:23.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:23.788+0000 7f2ed8200700 1 -- 172.21.15.1:0/1883980206 wait complete. 2024-04-26T01:51:24.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:23 smithi173 ceph-mon[32553]: pgmap v443: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:24.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:23 smithi001 ceph-mon[27822]: pgmap v443: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:23 smithi097 ceph-mon[32853]: pgmap v443: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:24.716 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-26T01:51:25.195 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:26.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:25 smithi173 ceph-mon[32553]: pgmap v444: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:26.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:25 smithi001 ceph-mon[27822]: pgmap v444: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:26.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:25 smithi097 ceph-mon[32853]: pgmap v444: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T01:51:27.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.369+0000 7f30de200700 1 -- 172.21.15.1:0/3735184942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30d806a750 msgr2=0x7f30d806abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:27.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.369+0000 7f30de200700 1 --2- 172.21.15.1:0/3735184942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30d806a750 0x7f30d806abb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009990 tx=0x7f30c0009ca0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.370+0000 7f30de200700 1 -- 172.21.15.1:0/3735184942 shutdown_connections 2024-04-26T01:51:27.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.370+0000 7f30de200700 1 --2- 172.21.15.1:0/3735184942 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30d806b0f0 0x7f30d810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.370+0000 7f30de200700 1 --2- 172.21.15.1:0/3735184942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d8108a30 0x7f30d8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.370+0000 7f30de200700 1 --2- 172.21.15.1:0/3735184942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30d806a750 0x7f30d806abb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.370+0000 7f30de200700 1 -- 172.21.15.1:0/3735184942 >> 172.21.15.1:0/3735184942 conn(0x7f30d80788f0 msgr2=0x7f30d8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:27.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.370+0000 7f30de200700 1 -- 172.21.15.1:0/3735184942 shutdown_connections 2024-04-26T01:51:27.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.370+0000 7f30de200700 1 -- 172.21.15.1:0/3735184942 wait complete. 2024-04-26T01:51:27.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.371+0000 7f30de200700 1 Processor -- start 2024-04-26T01:51:27.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.371+0000 7f30de200700 1 -- start start 2024-04-26T01:51:27.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.371+0000 7f30de200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30d806a750 0x7f30d819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:27.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.371+0000 7f30de200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d806b0f0 0x7f30d819b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:27.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.371+0000 7f30de200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30d8108a30 0x7f30d819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:27.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.372+0000 7f30de200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f30d8112c60 con 0x7f30d8108a30 2024-04-26T01:51:27.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.372+0000 7f30de200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f30d8112ae0 con 0x7f30d806b0f0 2024-04-26T01:51:27.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.372+0000 7f30de200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f30d8112de0 con 0x7f30d806a750 2024-04-26T01:51:27.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.372+0000 7f30d7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30d8108a30 0x7f30d819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:27.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.372+0000 7f30d7400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30d806a750 0x7f30d819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:27.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.372+0000 7f30d7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30d8108a30 0x7f30d819fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56824/0 (socket says 172.21.15.1:56824) 2024-04-26T01:51:27.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.372+0000 7f30cea00700 1 --2- 172.21.15.1:0/577048706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d806b0f0 0x7f30d819b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:27.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.372+0000 7f30d7e00700 1 -- 172.21.15.1:0/577048706 learned_addr learned my addr 172.21.15.1:0/577048706 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:27.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.372+0000 7f30d7400700 1 -- 172.21.15.1:0/577048706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d806b0f0 msgr2=0x7f30d819b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:27.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.372+0000 7f30d7400700 1 --2- 172.21.15.1:0/577048706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d806b0f0 0x7f30d819b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.372+0000 7f30d7400700 1 -- 172.21.15.1:0/577048706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30d8108a30 msgr2=0x7f30d819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:27.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.372+0000 7f30d7400700 1 --2- 172.21.15.1:0/577048706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30d8108a30 0x7f30d819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.372+0000 7f30d7400700 1 -- 172.21.15.1:0/577048706 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d81a0400 con 0x7f30d806a750 2024-04-26T01:51:27.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.372+0000 7f30d7e00700 1 --2- 172.21.15.1:0/577048706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30d8108a30 0x7f30d819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:27.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.373+0000 7f30d7400700 1 --2- 172.21.15.1:0/577048706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30d806a750 0x7f30d819b3f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f30c800e9f0 tx=0x7f30c800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:27.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.373+0000 7f30d4c00700 1 -- 172.21.15.1:0/577048706 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c800c9c0 con 0x7f30d806a750 2024-04-26T01:51:27.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.373+0000 7f30de200700 1 -- 172.21.15.1:0/577048706 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d81a06f0 con 0x7f30d806a750 2024-04-26T01:51:27.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.373+0000 7f30d4c00700 1 -- 172.21.15.1:0/577048706 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30c800cb60 con 0x7f30d806a750 2024-04-26T01:51:27.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.373+0000 7f30d4c00700 1 -- 172.21.15.1:0/577048706 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8018680 con 0x7f30d806a750 2024-04-26T01:51:27.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.373+0000 7f30de200700 1 -- 172.21.15.1:0/577048706 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d81a7f70 con 0x7f30d806a750 2024-04-26T01:51:27.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.377+0000 7f30d4c00700 1 -- 172.21.15.1:0/577048706 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f30c8010430 con 0x7f30d806a750 2024-04-26T01:51:27.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.378+0000 7f30d4c00700 1 --2- 172.21.15.1:0/577048706 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f30bc0a3c40 0x7f30bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:27.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.378+0000 7f30e8b6fb80 1 Processor -- start 2024-04-26T01:51:27.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.378+0000 7f30d4c00700 1 -- 172.21.15.1:0/577048706 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f30c80d0500 con 0x7f30d806a750 2024-04-26T01:51:27.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.378+0000 7f30e8b6fb80 1 -- start start 2024-04-26T01:51:27.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.378+0000 7f30cea00700 1 --2- 172.21.15.1:0/577048706 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f30bc0a3c40 0x7f30bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:27.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.379+0000 7f30e8b6fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc811780 0x5622dc811b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:27.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.379+0000 7f30e8b6fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc8120a0 0x5622dc80b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:27.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.379+0000 7f30e8b6fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc80bef0 0x5622dc80c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:27.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.379+0000 7f30e8b6fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5622dc75de40 con 0x5622dc8120a0 2024-04-26T01:51:27.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.379+0000 7f30e8b6fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5622dc7046e0 con 0x5622dc811780 2024-04-26T01:51:27.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.379+0000 7f30e8b6fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5622dc759e00 con 0x5622dc80bef0 2024-04-26T01:51:27.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.379+0000 7f30d7e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc80bef0 0x5622dc80c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:27.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.379+0000 7f30cea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc8120a0 0x5622dc80b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:27.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.379+0000 7f30d7e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc80bef0 0x5622dc80c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:52772/0 (socket says 172.21.15.1:52772) 2024-04-26T01:51:27.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.379+0000 7f30d7e00700 1 -- 172.21.15.1:0/1903436356 learned_addr learned my addr 172.21.15.1:0/1903436356 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:27.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.379+0000 7f30d7400700 1 --2- 172.21.15.1:0/1903436356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc811780 0x5622dc811b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:27.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.379+0000 7f30cea00700 1 --2- 172.21.15.1:0/577048706 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f30bc0a3c40 0x7f30bc0a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f30d8078f40 tx=0x7f30c0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:27.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.380+0000 7f30d7400700 1 -- 172.21.15.1:0/1903436356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc80bef0 msgr2=0x5622dc80c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:27.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.380+0000 7f30d7400700 1 --2- 172.21.15.1:0/1903436356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc80bef0 0x5622dc80c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.380+0000 7f30d7400700 1 -- 172.21.15.1:0/1903436356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc8120a0 msgr2=0x5622dc80b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:27.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.380+0000 7f30d7400700 1 --2- 172.21.15.1:0/1903436356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc8120a0 0x5622dc80b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.380+0000 7f30d7400700 1 -- 172.21.15.1:0/1903436356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622dc6fa630 con 0x5622dc811780 2024-04-26T01:51:27.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.380+0000 7f30d7400700 1 --2- 172.21.15.1:0/1903436356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc811780 0x5622dc811b60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f30c809c460 tx=0x7f30c809cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e32fc5e51e3bca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:27.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.380+0000 7f30ccc00700 1 -- 172.21.15.1:0/1903436356 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c80a53d0 con 0x5622dc811780 2024-04-26T01:51:27.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.380+0000 7f30ccc00700 1 -- 172.21.15.1:0/1903436356 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30c800cd10 con 0x5622dc811780 2024-04-26T01:51:27.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.380+0000 7f30ccc00700 1 -- 172.21.15.1:0/1903436356 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c80a4f40 con 0x5622dc811780 2024-04-26T01:51:27.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.381+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/1903436356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc811780 msgr2=0x5622dc811b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:27.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.381+0000 7f30e8b6fb80 1 --2- 172.21.15.1:0/1903436356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc811780 0x5622dc811b60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f30c809c460 tx=0x7f30c809cf30 comp rx=0 tx=0).stop 2024-04-26T01:51:27.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.381+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/1903436356 shutdown_connections 2024-04-26T01:51:27.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.381+0000 7f30e8b6fb80 1 --2- 172.21.15.1:0/1903436356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc80bef0 0x5622dc80c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.381+0000 7f30e8b6fb80 1 --2- 172.21.15.1:0/1903436356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc811780 0x5622dc811b60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.381+0000 7f30e8b6fb80 1 --2- 172.21.15.1:0/1903436356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc8120a0 0x5622dc80b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.381+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/1903436356 >> 172.21.15.1:0/1903436356 conn(0x5622dc739f00 msgr2=0x5622dc8068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:27.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.381+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/1903436356 shutdown_connections 2024-04-26T01:51:27.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.381+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/1903436356 wait complete. 2024-04-26T01:51:27.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.382+0000 7f30e8b6fb80 1 -- start start 2024-04-26T01:51:27.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.382+0000 7f30e8b6fb80 1 Processor -- start 2024-04-26T01:51:27.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.383+0000 7f30e8b6fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc80bef0 0x5622dc817cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:27.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.383+0000 7f30e8b6fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc811780 0x5622dc814fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:27.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.383+0000 7f30e8b6fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc815510 0x5622dc815990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:27.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.383+0000 7f30e8b6fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5622dc8188c0 con 0x5622dc80bef0 2024-04-26T01:51:27.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.383+0000 7f30e8b6fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5622dc717810 con 0x5622dc815510 2024-04-26T01:51:27.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.383+0000 7f30e8b6fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5622dc818a40 con 0x5622dc811780 2024-04-26T01:51:27.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.383+0000 7f30e8b6fb80 10 client.0 _ll_register_callbacks cb 0x5622dc6f66d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:51:27.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.383+0000 7f30d7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc80bef0 0x5622dc817cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:27.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.383+0000 7f30cea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc811780 0x5622dc814fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:27.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.383+0000 7f30cea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc811780 0x5622dc814fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:52776/0 (socket says 172.21.15.1:52776) 2024-04-26T01:51:27.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.383+0000 7f30d7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc80bef0 0x5622dc817cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56848/0 (socket says 172.21.15.1:56848) 2024-04-26T01:51:27.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.383+0000 7f30cea00700 1 -- 172.21.15.1:0/3214926361 learned_addr learned my addr 172.21.15.1:0/3214926361 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:27.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.383+0000 7f30d7e00700 1 --2- 172.21.15.1:0/3214926361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc815510 0x5622dc815990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:27.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.384+0000 7f30cea00700 1 -- 172.21.15.1:0/3214926361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc815510 msgr2=0x5622dc815990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:27.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.384+0000 7f30cea00700 1 --2- 172.21.15.1:0/3214926361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc815510 0x5622dc815990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.384+0000 7f30cea00700 1 -- 172.21.15.1:0/3214926361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc80bef0 msgr2=0x5622dc817cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:27.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.384+0000 7f30cea00700 1 --2- 172.21.15.1:0/3214926361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc80bef0 0x5622dc817cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.384+0000 7f30cea00700 1 -- 172.21.15.1:0/3214926361 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622dc6ab880 con 0x5622dc811780 2024-04-26T01:51:27.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.384+0000 7f30d7400700 1 --2- 172.21.15.1:0/3214926361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc80bef0 0x5622dc817cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:27.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.384+0000 7f30cea00700 1 --2- 172.21.15.1:0/3214926361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc811780 0x5622dc814fd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f30c0015000 tx=0x7f30c0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6356262f4c72c4b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:27.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.384+0000 7f30c5600700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:51:27.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.384+0000 7f30c5600700 1 -- 172.21.15.1:0/3214926361 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0003c90 con 0x5622dc811780 2024-04-26T01:51:27.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.384+0000 7f30c5600700 1 -- 172.21.15.1:0/3214926361 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f30c000f460 con 0x5622dc811780 2024-04-26T01:51:27.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.384+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/3214926361 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30c800cd10 con 0x5622dc811780 2024-04-26T01:51:27.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.385+0000 7f30c5600700 1 -- 172.21.15.1:0/3214926361 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f30c0003e30 con 0x5622dc811780 2024-04-26T01:51:27.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.385+0000 7f30e8b6fb80 10 client.25060 fetch_fsmap learned FSMap version 25 2024-04-26T01:51:27.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.385+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/3214926361 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5622dc6f4aa0 con 0x5622dc811780 2024-04-26T01:51:27.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.385+0000 7f30c5600700 1 -- 172.21.15.1:0/3214926361 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30c800cd10 con 0x5622dc811780 2024-04-26T01:51:27.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.385+0000 7f30c5600700 1 -- 172.21.15.1:0/3214926361 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f30c002f020 con 0x5622dc811780 2024-04-26T01:51:27.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.386+0000 7f30e8b6fb80 10 client.25060 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:51:27.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.386+0000 7f30e8b6fb80 10 client.25060 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T01:51:27.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.386+0000 7f30e8b6fb80 20 client.25060 populate_metadata read hostname 'smithi001' 2024-04-26T01:51:27.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.386+0000 7f30e8b6fb80 1 --2- 172.21.15.1:0/3214926361 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5622dc8259f0 0x5622dc826e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:27.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.386+0000 7f30e8b6fb80 4 client.25060 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:51:27.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.386+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/3214926361 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622dc76b630 con 0x5622dc8259f0 2024-04-26T01:51:27.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.386+0000 7f30d7400700 1 --2- 172.21.15.1:0/3214926361 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5622dc8259f0 0x5622dc826e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:27.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.387+0000 7f30d7400700 1 --2- 172.21.15.1:0/3214926361 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5622dc8259f0 0x5622dc826e60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c85305ee640f75e server_cookie=b30e84e1b034928a in_seq=0 out_seq=0 2024-04-26T01:51:27.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.387+0000 7f30c5600700 10 client.25060 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:51:27.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.387+0000 7f30c5600700 1 -- 172.21.15.1:0/3214926361 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5622dc76b630 con 0x5622dc8259f0 2024-04-26T01:51:27.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.387+0000 7f30c5600700 10 client.25060 handle_command_reply: tid=0 2024-04-26T01:51:27.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.388+0000 7f30e8b6fb80 1 client.25060 shutdown 2024-04-26T01:51:27.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.388+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/3214926361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc811780 msgr2=0x5622dc814fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:27.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.388+0000 7f30e8b6fb80 1 --2- 172.21.15.1:0/3214926361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc811780 0x5622dc814fd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f30c0015000 tx=0x7f30c0009310 comp rx=0 tx=0).stop 2024-04-26T01:51:27.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.388+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/3214926361 shutdown_connections 2024-04-26T01:51:27.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.388+0000 7f30e8b6fb80 1 --2- 172.21.15.1:0/3214926361 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5622dc8259f0 0x5622dc826e60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.388+0000 7f30e8b6fb80 1 --2- 172.21.15.1:0/3214926361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc815510 0x5622dc815990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.388+0000 7f30e8b6fb80 1 --2- 172.21.15.1:0/3214926361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc80bef0 0x5622dc817cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.388+0000 7f30e8b6fb80 1 --2- 172.21.15.1:0/3214926361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc811780 0x5622dc814fd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.388+0000 7f30c5600700 0 client.25060 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:51:27.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.388+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/3214926361 >> 172.21.15.1:0/3214926361 conn(0x5622dc739f00 msgr2=0x5622dc805e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:27.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.388+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/3214926361 shutdown_connections 2024-04-26T01:51:27.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.388+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/3214926361 wait complete. 2024-04-26T01:51:27.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.388+0000 7f30e8b6fb80 20 client.25060 trim_cache size 0 max 16384 2024-04-26T01:51:27.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.411+0000 7f30e8b6fb80 1 Processor -- start 2024-04-26T01:51:27.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.411+0000 7f30e8b6fb80 1 -- start start 2024-04-26T01:51:27.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.411+0000 7f30e8b6fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc80bef0 0x5622dc80b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:27.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.411+0000 7f30e8b6fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc811780 0x5622dc8065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:27.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.411+0000 7f30e8b6fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc815510 0x5622dc8302c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:27.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.411+0000 7f30e8b6fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5622dc8251a0 con 0x5622dc811780 2024-04-26T01:51:27.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.411+0000 7f30e8b6fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5622dc825870 con 0x5622dc815510 2024-04-26T01:51:27.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.411+0000 7f30e8b6fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5622dc825f40 con 0x5622dc80bef0 2024-04-26T01:51:27.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.412+0000 7f30d7400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc80bef0 0x5622dc80b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:27.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.412+0000 7f30d7400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc80bef0 0x5622dc80b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:52788/0 (socket says 172.21.15.1:52788) 2024-04-26T01:51:27.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.412+0000 7f30d7400700 1 -- 172.21.15.1:0/223676721 learned_addr learned my addr 172.21.15.1:0/223676721 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:27.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.412+0000 7f30cea00700 1 --2- 172.21.15.1:0/223676721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc811780 0x5622dc8065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:27.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.412+0000 7f30d7e00700 1 --2- 172.21.15.1:0/223676721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc815510 0x5622dc8302c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:27.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.412+0000 7f30d7400700 1 -- 172.21.15.1:0/223676721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc815510 msgr2=0x5622dc8302c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:27.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.412+0000 7f30d7400700 1 --2- 172.21.15.1:0/223676721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc815510 0x5622dc8302c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.412+0000 7f30d7400700 1 -- 172.21.15.1:0/223676721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc811780 msgr2=0x5622dc8065d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:27.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.412+0000 7f30d7400700 1 --2- 172.21.15.1:0/223676721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc811780 0x5622dc8065d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.412+0000 7f30d7400700 1 -- 172.21.15.1:0/223676721 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622dc8113a0 con 0x5622dc80bef0 2024-04-26T01:51:27.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.412+0000 7f30d7400700 1 --2- 172.21.15.1:0/223676721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc80bef0 0x5622dc80b9b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f30c8098bc0 tx=0x7f30c8009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dbac72a0bea6d10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:27.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.413+0000 7f30c7e00700 1 -- 172.21.15.1:0/223676721 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c80870b0 con 0x5622dc80bef0 2024-04-26T01:51:27.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.413+0000 7f30c7e00700 1 -- 172.21.15.1:0/223676721 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30c80a4f40 con 0x5622dc80bef0 2024-04-26T01:51:27.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.413+0000 7f30c7e00700 1 -- 172.21.15.1:0/223676721 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8010980 con 0x5622dc80bef0 2024-04-26T01:51:27.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.413+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/223676721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc80bef0 msgr2=0x5622dc80b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:27.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.413+0000 7f30e8b6fb80 1 --2- 172.21.15.1:0/223676721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc80bef0 0x5622dc80b9b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f30c8098bc0 tx=0x7f30c8009790 comp rx=0 tx=0).stop 2024-04-26T01:51:27.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.413+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/223676721 shutdown_connections 2024-04-26T01:51:27.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.413+0000 7f30e8b6fb80 1 --2- 172.21.15.1:0/223676721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc815510 0x5622dc8302c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.413+0000 7f30e8b6fb80 1 --2- 172.21.15.1:0/223676721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc80bef0 0x5622dc80b9b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.413+0000 7f30e8b6fb80 1 --2- 172.21.15.1:0/223676721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc811780 0x5622dc8065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.413+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/223676721 >> 172.21.15.1:0/223676721 conn(0x5622dc739f00 msgr2=0x5622dc80f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:27.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.414+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/223676721 shutdown_connections 2024-04-26T01:51:27.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.414+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/223676721 wait complete. 2024-04-26T01:51:27.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.414+0000 7f30e8b6fb80 1 -- start start 2024-04-26T01:51:27.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.414+0000 7f30e8b6fb80 1 Processor -- start 2024-04-26T01:51:27.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.415+0000 7f30e8b6fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc80bef0 0x5622dc8302e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:27.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.415+0000 7f30e8b6fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc811780 0x5622dc830820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:27.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.415+0000 7f30e8b6fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc815510 0x5622dc82b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:27.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.415+0000 7f30e8b6fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5622dc826ce0 con 0x5622dc811780 2024-04-26T01:51:27.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.415+0000 7f30e8b6fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5622dc826610 con 0x5622dc815510 2024-04-26T01:51:27.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.415+0000 7f30e8b6fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5622dc8273b0 con 0x5622dc80bef0 2024-04-26T01:51:27.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.415+0000 7f30e8b6fb80 10 client.0 _ll_register_callbacks cb 0x5622dc6f66d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:51:27.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.415+0000 7f30cea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc811780 0x5622dc830820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:27.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.415+0000 7f30cea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc811780 0x5622dc830820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56854/0 (socket says 172.21.15.1:56854) 2024-04-26T01:51:27.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.415+0000 7f30cea00700 1 -- 172.21.15.1:0/559068615 learned_addr learned my addr 172.21.15.1:0/559068615 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:27.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.415+0000 7f30d7400700 1 --2- 172.21.15.1:0/559068615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc80bef0 0x5622dc8302e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:27.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.415+0000 7f30d7e00700 1 --2- 172.21.15.1:0/559068615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc815510 0x5622dc82b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:27.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.415+0000 7f30cea00700 1 -- 172.21.15.1:0/559068615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc80bef0 msgr2=0x5622dc8302e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:27.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.415+0000 7f30cea00700 1 --2- 172.21.15.1:0/559068615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc80bef0 0x5622dc8302e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.416+0000 7f30cea00700 1 -- 172.21.15.1:0/559068615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc815510 msgr2=0x5622dc82b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:27.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.416+0000 7f30cea00700 1 --2- 172.21.15.1:0/559068615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc815510 0x5622dc82b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.416+0000 7f30cea00700 1 -- 172.21.15.1:0/559068615 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622dc82b9b0 con 0x5622dc811780 2024-04-26T01:51:27.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.416+0000 7f30d7e00700 1 --2- 172.21.15.1:0/559068615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc815510 0x5622dc82b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:51:27.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.416+0000 7f30cea00700 1 --2- 172.21.15.1:0/559068615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc811780 0x5622dc830820 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x5622dc8165a0 tx=0x7f30c0025b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6063152c90a82e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:27.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.416+0000 7f30c5600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:51:27.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.416+0000 7f30c5600700 1 -- 172.21.15.1:0/559068615 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0028040 con 0x5622dc811780 2024-04-26T01:51:27.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.416+0000 7f30c5600700 1 -- 172.21.15.1:0/559068615 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f30c000f660 con 0x5622dc811780 2024-04-26T01:51:27.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.416+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/559068615 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30c80a4f40 con 0x5622dc811780 2024-04-26T01:51:27.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.417+0000 7f30c5600700 1 -- 172.21.15.1:0/559068615 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f30c00058e0 con 0x5622dc811780 2024-04-26T01:51:27.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.417+0000 7f30e8b6fb80 10 client.14937 fetch_fsmap learned FSMap version 25 2024-04-26T01:51:27.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.417+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/559068615 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5622dc895d50 con 0x5622dc811780 2024-04-26T01:51:27.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.417+0000 7f30c5600700 1 -- 172.21.15.1:0/559068615 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30c80a4f40 con 0x5622dc811780 2024-04-26T01:51:27.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.417+0000 7f30c5600700 1 -- 172.21.15.1:0/559068615 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f30c002f040 con 0x5622dc811780 2024-04-26T01:51:27.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.417+0000 7f30e8b6fb80 10 client.14937 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T01:51:27.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.417+0000 7f30e8b6fb80 10 client.14937 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T01:51:27.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.417+0000 7f30e8b6fb80 20 client.14937 populate_metadata read hostname 'smithi001' 2024-04-26T01:51:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.417+0000 7f30e8b6fb80 1 --2- 172.21.15.1:0/559068615 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5622dc89a8d0 0x5622dc8995e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.417+0000 7f30e8b6fb80 4 client.14937 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T01:51:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.417+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/559068615 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "status"}) -- 0x5622dc7656d0 con 0x5622dc89a8d0 2024-04-26T01:51:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.418+0000 7f30d7400700 1 --2- 172.21.15.1:0/559068615 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5622dc89a8d0 0x5622dc8995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.418+0000 7f30d7400700 1 --2- 172.21.15.1:0/559068615 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5622dc89a8d0 0x5622dc8995e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c53cd1aaa2e23597 server_cookie=9ff32cbeac331bd3 in_seq=0 out_seq=0 2024-04-26T01:51:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.418+0000 7f30c5600700 10 client.14937 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:51:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.418+0000 7f30c5600700 1 -- 172.21.15.1:0/559068615 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5622dc7656d0 con 0x5622dc89a8d0 2024-04-26T01:51:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.418+0000 7f30c5600700 10 client.14937 handle_command_reply: tid=0 2024-04-26T01:51:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.418+0000 7f30e8b6fb80 1 client.14937 shutdown 2024-04-26T01:51:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.419+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/559068615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc811780 msgr2=0x5622dc830820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.419+0000 7f30e8b6fb80 1 --2- 172.21.15.1:0/559068615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc811780 0x5622dc830820 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x5622dc8165a0 tx=0x7f30c0025b80 comp rx=0 tx=0).stop 2024-04-26T01:51:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.419+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/559068615 shutdown_connections 2024-04-26T01:51:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.419+0000 7f30e8b6fb80 1 --2- 172.21.15.1:0/559068615 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5622dc89a8d0 0x5622dc8995e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.419+0000 7f30e8b6fb80 1 --2- 172.21.15.1:0/559068615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622dc815510 0x5622dc82b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.419+0000 7f30e8b6fb80 1 --2- 172.21.15.1:0/559068615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5622dc80bef0 0x5622dc8302e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.419+0000 7f30e8b6fb80 1 --2- 172.21.15.1:0/559068615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622dc811780 0x5622dc830820 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.419+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/559068615 >> 172.21.15.1:0/559068615 conn(0x5622dc739f00 msgr2=0x5622dc812490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:27.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.419+0000 7f30c5600700 0 client.14937 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:51:27.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.419+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/559068615 shutdown_connections 2024-04-26T01:51:27.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.419+0000 7f30e8b6fb80 1 -- 172.21.15.1:0/559068615 wait complete. 2024-04-26T01:51:27.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.419+0000 7f30e8b6fb80 20 client.14937 trim_cache size 0 max 16384 2024-04-26T01:51:27.420 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T01:51:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "77cfcc80-036d-11ef-bc93-c7b262605968", 2024-04-26T01:51:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-04-26T01:51:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24518, 2024-04-26T01:51:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-04-26T01:51:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-04-26T01:51:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-04-26T01:51:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 237.30075090899999, 2024-04-26T01:51:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 25, 2024-04-26T01:51:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 92, 2024-04-26T01:51:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T01:51:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 257.78838702100001 2024-04-26T01:51:27.421 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T01:51:27.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.420+0000 7f30de200700 1 -- 172.21.15.1:0/577048706 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f30bc0a3c40 msgr2=0x7f30bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:27.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.420+0000 7f30de200700 1 --2- 172.21.15.1:0/577048706 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f30bc0a3c40 0x7f30bc0a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f30d8078f40 tx=0x7f30c0005fd0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.420+0000 7f30de200700 1 -- 172.21.15.1:0/577048706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30d806a750 msgr2=0x7f30d819b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:27.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.420+0000 7f30de200700 1 --2- 172.21.15.1:0/577048706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30d806a750 0x7f30d819b3f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f30c800e9f0 tx=0x7f30c800ed00 comp rx=0 tx=0).stop 2024-04-26T01:51:27.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.420+0000 7f30de200700 1 -- 172.21.15.1:0/577048706 shutdown_connections 2024-04-26T01:51:27.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.420+0000 7f30de200700 1 --2- 172.21.15.1:0/577048706 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f30bc0a3c40 0x7f30bc0a6100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.420+0000 7f30de200700 1 --2- 172.21.15.1:0/577048706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30d8108a30 0x7f30d819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.420+0000 7f30de200700 1 --2- 172.21.15.1:0/577048706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30d806a750 0x7f30d819b3f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.420+0000 7f30de200700 1 --2- 172.21.15.1:0/577048706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d806b0f0 0x7f30d819b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:27.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.420+0000 7f30de200700 1 -- 172.21.15.1:0/577048706 >> 172.21.15.1:0/577048706 conn(0x7f30d80788f0 msgr2=0x7f30d810dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:27.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.420+0000 7f30de200700 1 -- 172.21.15.1:0/577048706 shutdown_connections 2024-04-26T01:51:27.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:27.420+0000 7f30de200700 1 -- 172.21.15.1:0/577048706 wait complete. 2024-04-26T01:51:28.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:28 smithi173 ceph-mon[32553]: pgmap v445: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:28.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:28 smithi001 ceph-mon[27822]: pgmap v445: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:28.472 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T01:51:28.481 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T01:51:28.481 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T01:51:28.482 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T01:51:28.482 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-26T01:51:28.483 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T01:51:28.483 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-04-26T01:51:28.483 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-26T01:51:28.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:28 smithi097 ceph-mon[32853]: pgmap v445: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T01:51:28.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.708+0000 7f6eb0c00640 1 -- 172.21.15.1:0/98919695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eac108470 msgr2=0x7f6eac108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:28.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.708+0000 7f6eb0c00640 1 --2- 172.21.15.1:0/98919695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eac108470 0x7f6eac108890 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009860 tx=0x7f6e9c02a340 comp rx=0 tx=0).stop 2024-04-26T01:51:28.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.708+0000 7f6eb0c00640 1 -- 172.21.15.1:0/98919695 shutdown_connections 2024-04-26T01:51:28.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.708+0000 7f6eb0c00640 1 --2- 172.21.15.1:0/98919695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6eac109fd0 0x7f6eac110860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:28.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.708+0000 7f6eb0c00640 1 --2- 172.21.15.1:0/98919695 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6eac109610 0x7f6eac109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:28.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.708+0000 7f6eb0c00640 1 --2- 172.21.15.1:0/98919695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eac108470 0x7f6eac108890 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:28.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.708+0000 7f6eb0c00640 1 -- 172.21.15.1:0/98919695 >> 172.21.15.1:0/98919695 conn(0x7f6eac0773e0 msgr2=0x7f6eac079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:28.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.708+0000 7f6eb0c00640 1 -- 172.21.15.1:0/98919695 shutdown_connections 2024-04-26T01:51:28.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.708+0000 7f6eb0c00640 1 -- 172.21.15.1:0/98919695 wait complete. 2024-04-26T01:51:28.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.709+0000 7f6eb0c00640 1 Processor -- start 2024-04-26T01:51:28.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.709+0000 7f6eb0c00640 1 -- start start 2024-04-26T01:51:28.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.709+0000 7f6eb0c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6eac108470 0x7f6eac19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:28.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.709+0000 7f6eb0c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6eac109610 0x7f6eac19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:28.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.709+0000 7f6eb0c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eac109fd0 0x7f6eac1a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:28.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.709+0000 7f6eb0c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac113f60 con 0x7f6eac109610 2024-04-26T01:51:28.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.709+0000 7f6eb0c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac113de0 con 0x7f6eac109fd0 2024-04-26T01:51:28.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.709+0000 7f6eb0c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac1140e0 con 0x7f6eac108470 2024-04-26T01:51:28.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.710+0000 7f6eaaa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6eac109610 0x7f6eac19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.710+0000 7f6eab400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6eac108470 0x7f6eac19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.710+0000 7f6eaaa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6eac109610 0x7f6eac19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56860/0 (socket says 172.21.15.1:56860) 2024-04-26T01:51:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.710+0000 7f6eaaa00640 1 -- 172.21.15.1:0/3222726085 learned_addr learned my addr 172.21.15.1:0/3222726085 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.710+0000 7f6eabe00640 1 --2- 172.21.15.1:0/3222726085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eac109fd0 0x7f6eac1a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.710+0000 7f6eaaa00640 1 -- 172.21.15.1:0/3222726085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6eac108470 msgr2=0x7f6eac19e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.710+0000 7f6eaaa00640 1 --2- 172.21.15.1:0/3222726085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6eac108470 0x7f6eac19e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.710+0000 7f6eaaa00640 1 -- 172.21.15.1:0/3222726085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eac109fd0 msgr2=0x7f6eac1a5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.710+0000 7f6eaaa00640 1 --2- 172.21.15.1:0/3222726085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eac109fd0 0x7f6eac1a5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.710+0000 7f6eaaa00640 1 -- 172.21.15.1:0/3222726085 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eac1a6220 con 0x7f6eac109610 2024-04-26T01:51:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.710+0000 7f6eaaa00640 1 --2- 172.21.15.1:0/3222726085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6eac109610 0x7f6eac19eb40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800ea60 tx=0x7f6e9800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.711+0000 7f6ea3e00640 1 -- 172.21.15.1:0/3222726085 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9800cde0 con 0x7f6eac109610 2024-04-26T01:51:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.711+0000 7f6ea3e00640 1 -- 172.21.15.1:0/3222726085 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e980035f0 con 0x7f6eac109610 2024-04-26T01:51:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.711+0000 7f6eb0c00640 1 -- 172.21.15.1:0/3222726085 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eac1a6510 con 0x7f6eac109610 2024-04-26T01:51:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.711+0000 7f6ea3e00640 1 -- 172.21.15.1:0/3222726085 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98010690 con 0x7f6eac109610 2024-04-26T01:51:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.711+0000 7f6eb0c00640 1 -- 172.21.15.1:0/3222726085 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eac1a6ab0 con 0x7f6eac109610 2024-04-26T01:51:28.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.712+0000 7f6eb0c00640 1 -- 172.21.15.1:0/3222726085 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eac10f7c0 con 0x7f6eac109610 2024-04-26T01:51:28.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.715+0000 7f6ea3e00640 1 -- 172.21.15.1:0/3222726085 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f6e980037a0 con 0x7f6eac109610 2024-04-26T01:51:28.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.716+0000 7f6ea3e00640 1 --2- 172.21.15.1:0/3222726085 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f6e900a3c10 0x7f6e900a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:28.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.716+0000 7f6ea3e00640 1 -- 172.21.15.1:0/3222726085 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f6e980d0690 con 0x7f6eac109610 2024-04-26T01:51:28.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.717+0000 7f6eab400640 1 --2- 172.21.15.1:0/3222726085 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f6e900a3c10 0x7f6e900a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:28.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.717+0000 7f6eab400640 1 --2- 172.21.15.1:0/3222726085 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f6e900a3c10 0x7f6e900a60d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009830 tx=0x7f6e9c030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:28.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.721+0000 7f6ea3e00640 1 -- 172.21.15.1:0/3222726085 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e9809cff0 con 0x7f6eac109610 2024-04-26T01:51:28.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:28.980+0000 7f6eb0c00640 1 -- 172.21.15.1:0/3222726085 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f6eac062b50 con 0x7f6eac109610 2024-04-26T01:51:29.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.020+0000 7f6ea3e00640 1 -- 172.21.15.1:0/3222726085 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f6e9809c8f0 con 0x7f6eac109610 2024-04-26T01:51:29.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.024+0000 7f6eb0c00640 1 -- 172.21.15.1:0/3222726085 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f6e900a3c10 msgr2=0x7f6e900a60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:29.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.024+0000 7f6eb0c00640 1 --2- 172.21.15.1:0/3222726085 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f6e900a3c10 0x7f6e900a60d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009830 tx=0x7f6e9c030450 comp rx=0 tx=0).stop 2024-04-26T01:51:29.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.024+0000 7f6eb0c00640 1 -- 172.21.15.1:0/3222726085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6eac109610 msgr2=0x7f6eac19eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:29.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.024+0000 7f6eb0c00640 1 --2- 172.21.15.1:0/3222726085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6eac109610 0x7f6eac19eb40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800ea60 tx=0x7f6e9800ef20 comp rx=0 tx=0).stop 2024-04-26T01:51:29.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.027+0000 7f6eb0c00640 1 -- 172.21.15.1:0/3222726085 shutdown_connections 2024-04-26T01:51:29.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.027+0000 7f6eb0c00640 1 --2- 172.21.15.1:0/3222726085 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f6e900a3c10 0x7f6e900a60d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.027+0000 7f6eb0c00640 1 --2- 172.21.15.1:0/3222726085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eac109fd0 0x7f6eac1a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.027+0000 7f6eb0c00640 1 --2- 172.21.15.1:0/3222726085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6eac109610 0x7f6eac19eb40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.027+0000 7f6eb0c00640 1 --2- 172.21.15.1:0/3222726085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6eac108470 0x7f6eac19e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.027+0000 7f6eb0c00640 1 -- 172.21.15.1:0/3222726085 >> 172.21.15.1:0/3222726085 conn(0x7f6eac0773e0 msgr2=0x7f6eac10c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:29.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.029+0000 7f6eb0c00640 1 -- 172.21.15.1:0/3222726085 shutdown_connections 2024-04-26T01:51:29.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.029+0000 7f6eb0c00640 1 -- 172.21.15.1:0/3222726085 wait complete. 2024-04-26T01:51:29.050 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-26T01:51:29.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.239+0000 7ff17c800640 1 -- 172.21.15.1:0/3184876418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1740784f0 msgr2=0x7ff174078d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.239+0000 7ff17c800640 1 --2- 172.21.15.1:0/3184876418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1740784f0 0x7ff174078d90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff170009900 tx=0x7ff17002a410 comp rx=0 tx=0).stop 2024-04-26T01:51:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.239+0000 7ff17c800640 1 -- 172.21.15.1:0/3184876418 shutdown_connections 2024-04-26T01:51:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.239+0000 7ff17c800640 1 --2- 172.21.15.1:0/3184876418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1740784f0 0x7ff174078d90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.239+0000 7ff17c800640 1 --2- 172.21.15.1:0/3184876418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff174077b50 0x7ff174077fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.239+0000 7ff17c800640 1 --2- 172.21.15.1:0/3184876418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff174076910 0x7ff174076d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.239+0000 7ff17c800640 1 -- 172.21.15.1:0/3184876418 >> 172.21.15.1:0/3184876418 conn(0x7ff1740ff770 msgr2=0x7ff174101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.239+0000 7ff17c800640 1 -- 172.21.15.1:0/3184876418 shutdown_connections 2024-04-26T01:51:29.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.239+0000 7ff17c800640 1 -- 172.21.15.1:0/3184876418 wait complete. 2024-04-26T01:51:29.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.240+0000 7ff17c800640 1 Processor -- start 2024-04-26T01:51:29.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.240+0000 7ff17c800640 1 -- start start 2024-04-26T01:51:29.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.240+0000 7ff17c800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff174076910 0x7ff17419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:29.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.240+0000 7ff17c800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff174077b50 0x7ff17419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:29.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.240+0000 7ff17c800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1740784f0 0x7ff1741a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.240+0000 7ff17c800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff174114160 con 0x7ff174077b50 2024-04-26T01:51:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.240+0000 7ff17c800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff174113fe0 con 0x7ff174076910 2024-04-26T01:51:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.240+0000 7ff17c800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff1741142e0 con 0x7ff1740784f0 2024-04-26T01:51:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.241+0000 7ff179e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff174076910 0x7ff17419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.241+0000 7ff179e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff174076910 0x7ff17419e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:52666/0 (socket says 172.21.15.1:52666) 2024-04-26T01:51:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.241+0000 7ff179e00640 1 -- 172.21.15.1:0/2044043740 learned_addr learned my addr 172.21.15.1:0/2044043740 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.241+0000 7ff179400640 1 --2- 172.21.15.1:0/2044043740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff174077b50 0x7ff17419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.241+0000 7ff17a800640 1 --2- 172.21.15.1:0/2044043740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1740784f0 0x7ff1741a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.241+0000 7ff179e00640 1 -- 172.21.15.1:0/2044043740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1740784f0 msgr2=0x7ff1741a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:29.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.241+0000 7ff179e00640 1 --2- 172.21.15.1:0/2044043740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1740784f0 0x7ff1741a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.241+0000 7ff179e00640 1 -- 172.21.15.1:0/2044043740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff174077b50 msgr2=0x7ff17419ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.241+0000 7ff179e00640 1 --2- 172.21.15.1:0/2044043740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff174077b50 0x7ff17419ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.241+0000 7ff179e00640 1 -- 172.21.15.1:0/2044043740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1741a6350 con 0x7ff174076910 2024-04-26T01:51:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.241+0000 7ff179400640 1 --2- 172.21.15.1:0/2044043740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff174077b50 0x7ff17419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:51:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.241+0000 7ff179e00640 1 --2- 172.21.15.1:0/2044043740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff174076910 0x7ff17419e730 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00ebc0 tx=0x7ff16c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.242+0000 7ff16aa00640 1 -- 172.21.15.1:0/2044043740 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c010630 con 0x7ff174076910 2024-04-26T01:51:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.242+0000 7ff16aa00640 1 -- 172.21.15.1:0/2044043740 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff16c010c80 con 0x7ff174076910 2024-04-26T01:51:29.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.242+0000 7ff17c800640 1 -- 172.21.15.1:0/2044043740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1741a6640 con 0x7ff174076910 2024-04-26T01:51:29.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.242+0000 7ff16aa00640 1 -- 172.21.15.1:0/2044043740 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c018b60 con 0x7ff174076910 2024-04-26T01:51:29.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.242+0000 7ff17c800640 1 -- 172.21.15.1:0/2044043740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1741a6b90 con 0x7ff174076910 2024-04-26T01:51:29.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.243+0000 7ff17c800640 1 -- 172.21.15.1:0/2044043740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff174110c90 con 0x7ff174076910 2024-04-26T01:51:29.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.245+0000 7ff16aa00640 1 -- 172.21.15.1:0/2044043740 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 19) ==== 138662+0+0 (secure 0 0 0) 0x7ff16c018d00 con 0x7ff174076910 2024-04-26T01:51:29.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.246+0000 7ff16aa00640 1 --2- 172.21.15.1:0/2044043740 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ff1600a3bf0 0x7ff1600a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:29.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.246+0000 7ff179400640 1 -- 172.21.15.1:0/2044043740 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ff1600a3bf0 msgr2=0x7ff1600a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2140737121 2024-04-26T01:51:29.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.246+0000 7ff16aa00640 1 -- 172.21.15.1:0/2044043740 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7ff16c0d0b60 con 0x7ff174076910 2024-04-26T01:51:29.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.246+0000 7ff179400640 1 --2- 172.21.15.1:0/2044043740 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ff1600a3bf0 0x7ff1600a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:51:29.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.250+0000 7ff16aa00640 1 -- 172.21.15.1:0/2044043740 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff16c09d4c0 con 0x7ff174076910 2024-04-26T01:51:29.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:29 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3222726085' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T01:51:29.317 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:29 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: ignoring --setuser ceph since I am not root 2024-04-26T01:51:29.317 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:29 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: ignoring --setgroup ceph since I am not root 2024-04-26T01:51:29.317 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:29 smithi173 ceph-mgr[34430]: -- 172.21.15.173:0/324725940 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5601cd6b65a0 con 0x5601cd68a800 2024-04-26T01:51:29.317 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:29 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:29.201+0000 7fbbb7ebc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:51:29.338 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:29 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3222726085' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T01:51:29.339 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:29 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: ignoring --setuser ceph since I am not root 2024-04-26T01:51:29.339 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:29 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: ignoring --setgroup ceph since I am not root 2024-04-26T01:51:29.339 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:29 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:29.339+0000 7f8dbc288200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:51:29.381 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:29 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3222726085' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T01:51:29.381 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:29 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: ignoring --setuser ceph since I am not root 2024-04-26T01:51:29.381 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:29 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: ignoring --setgroup ceph since I am not root 2024-04-26T01:51:29.381 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:29 smithi097 ceph-mgr[34240]: -- 172.21.15.97:0/4069722185 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ec4e7025a0 con 0x55ec4e6d7000 2024-04-26T01:51:29.381 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:29 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:29.198+0000 7f2b6e27d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:51:29.381 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:29 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:29.381+0000 7f2b6e27d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:51:29.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.447+0000 7ff179400640 1 -- 172.21.15.1:0/2044043740 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ff1600a3bf0 msgr2=0x7ff1600a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2140737121 2024-04-26T01:51:29.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.447+0000 7ff179400640 1 --2- 172.21.15.1:0/2044043740 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ff1600a3bf0 0x7ff1600a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:51:29.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.454+0000 7ff17c800640 1 -- 172.21.15.1:0/2044043740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7ff17402d5c0 con 0x7ff174076910 2024-04-26T01:51:29.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.460+0000 7ff16aa00640 1 -- 172.21.15.1:0/2044043740 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v26)=0 v26) ==== 138+0+0 (secure 0 0 0) 0x7ff16c09cdc0 con 0x7ff174076910 2024-04-26T01:51:29.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.463+0000 7ff17c800640 1 -- 172.21.15.1:0/2044043740 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ff1600a3bf0 msgr2=0x7ff1600a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:51:29.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.463+0000 7ff17c800640 1 --2- 172.21.15.1:0/2044043740 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ff1600a3bf0 0x7ff1600a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.463+0000 7ff17c800640 1 -- 172.21.15.1:0/2044043740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff174076910 msgr2=0x7ff17419e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:29.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.463+0000 7ff17c800640 1 --2- 172.21.15.1:0/2044043740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff174076910 0x7ff17419e730 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00ebc0 tx=0x7ff16c00c6a0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.463+0000 7ff17c800640 1 -- 172.21.15.1:0/2044043740 shutdown_connections 2024-04-26T01:51:29.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.463+0000 7ff17c800640 1 --2- 172.21.15.1:0/2044043740 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7ff1600a3bf0 0x7ff1600a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.463+0000 7ff17c800640 1 --2- 172.21.15.1:0/2044043740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1740784f0 0x7ff1741a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.463+0000 7ff17c800640 1 --2- 172.21.15.1:0/2044043740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff174077b50 0x7ff17419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.463+0000 7ff17c800640 1 --2- 172.21.15.1:0/2044043740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff174076910 0x7ff17419e730 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.463+0000 7ff17c800640 1 -- 172.21.15.1:0/2044043740 >> 172.21.15.1:0/2044043740 conn(0x7ff1740ff770 msgr2=0x7ff174100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:29.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.463+0000 7ff17c800640 1 -- 172.21.15.1:0/2044043740 shutdown_connections 2024-04-26T01:51:29.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.463+0000 7ff17c800640 1 -- 172.21.15.1:0/2044043740 wait complete. 2024-04-26T01:51:29.479 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-26T01:51:29.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.671+0000 7fe913800640 1 -- 172.21.15.1:0/2260658656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe90c105d90 msgr2=0x7fe90c10c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.671+0000 7fe913800640 1 --2- 172.21.15.1:0/2260658656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe90c105d90 0x7fe90c10c620 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe904009ca0 tx=0x7fe90402a8c0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.671+0000 7fe913800640 1 -- 172.21.15.1:0/2260658656 shutdown_connections 2024-04-26T01:51:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.671+0000 7fe913800640 1 --2- 172.21.15.1:0/2260658656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe90c105d90 0x7fe90c10c620 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.671+0000 7fe913800640 1 --2- 172.21.15.1:0/2260658656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe90c1053d0 0x7fe90c105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.671+0000 7fe913800640 1 --2- 172.21.15.1:0/2260658656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90c1041e0 0x7fe90c104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.671+0000 7fe913800640 1 -- 172.21.15.1:0/2260658656 >> 172.21.15.1:0/2260658656 conn(0x7fe90c0ff770 msgr2=0x7fe90c101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.671+0000 7fe913800640 1 -- 172.21.15.1:0/2260658656 shutdown_connections 2024-04-26T01:51:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.671+0000 7fe913800640 1 -- 172.21.15.1:0/2260658656 wait complete. 2024-04-26T01:51:29.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.672+0000 7fe913800640 1 Processor -- start 2024-04-26T01:51:29.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.672+0000 7fe913800640 1 -- start start 2024-04-26T01:51:29.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.673+0000 7fe913800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90c1041e0 0x7fe90c19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:29.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.673+0000 7fe913800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe90c1053d0 0x7fe90c19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.673+0000 7fe913800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe90c105d90 0x7fe90c1a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.673+0000 7fe913800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c10fd40 con 0x7fe90c105d90 2024-04-26T01:51:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.673+0000 7fe913800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c10fbc0 con 0x7fe90c1041e0 2024-04-26T01:51:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.673+0000 7fe913800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c10fec0 con 0x7fe90c1053d0 2024-04-26T01:51:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.673+0000 7fe910e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90c1041e0 0x7fe90c19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.673+0000 7fe910e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90c1041e0 0x7fe90c19a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:52684/0 (socket says 172.21.15.1:52684) 2024-04-26T01:51:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.673+0000 7fe910e00640 1 -- 172.21.15.1:0/4022354604 learned_addr learned my addr 172.21.15.1:0/4022354604 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.673+0000 7fe903e00640 1 --2- 172.21.15.1:0/4022354604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe90c1053d0 0x7fe90c19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.673+0000 7fe911800640 1 --2- 172.21.15.1:0/4022354604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe90c105d90 0x7fe90c1a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:29.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.673+0000 7fe910e00640 1 -- 172.21.15.1:0/4022354604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe90c1053d0 msgr2=0x7fe90c19a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:29.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.673+0000 7fe910e00640 1 --2- 172.21.15.1:0/4022354604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe90c1053d0 0x7fe90c19a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.673+0000 7fe910e00640 1 -- 172.21.15.1:0/4022354604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe90c105d90 msgr2=0x7fe90c1a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:29.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.673+0000 7fe910e00640 1 --2- 172.21.15.1:0/4022354604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe90c105d90 0x7fe90c1a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.674+0000 7fe910e00640 1 -- 172.21.15.1:0/4022354604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90c1a1fb0 con 0x7fe90c1041e0 2024-04-26T01:51:29.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.674+0000 7fe910e00640 1 --2- 172.21.15.1:0/4022354604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90c1041e0 0x7fe90c19a390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000e9c0 tx=0x7fe8f000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:29.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.674+0000 7fe90a000640 1 -- 172.21.15.1:0/4022354604 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f000cd80 con 0x7fe90c1041e0 2024-04-26T01:51:29.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.674+0000 7fe90a000640 1 -- 172.21.15.1:0/4022354604 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8f00035f0 con 0x7fe90c1041e0 2024-04-26T01:51:29.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.674+0000 7fe90a000640 1 -- 172.21.15.1:0/4022354604 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0010690 con 0x7fe90c1041e0 2024-04-26T01:51:29.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.674+0000 7fe913800640 1 -- 172.21.15.1:0/4022354604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90c1a22a0 con 0x7fe90c1041e0 2024-04-26T01:51:29.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.674+0000 7fe913800640 1 -- 172.21.15.1:0/4022354604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe90c1a27f0 con 0x7fe90c1041e0 2024-04-26T01:51:29.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.676+0000 7fe913800640 1 -- 172.21.15.1:0/4022354604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe90c10b500 con 0x7fe90c1041e0 2024-04-26T01:51:29.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.677+0000 7fe90a000640 1 -- 172.21.15.1:0/4022354604 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 19) ==== 138662+0+0 (secure 0 0 0) 0x7fe8f00037a0 con 0x7fe90c1041e0 2024-04-26T01:51:29.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.679+0000 7fe90a000640 1 --2- 172.21.15.1:0/4022354604 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe8e80a3bf0 0x7fe8e80a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:29.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.679+0000 7fe903e00640 1 -- 172.21.15.1:0/4022354604 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe8e80a3bf0 msgr2=0x7fe8e80a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2140737121 2024-04-26T01:51:29.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.679+0000 7fe90a000640 1 -- 172.21.15.1:0/4022354604 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7fe8f00d0840 con 0x7fe90c1041e0 2024-04-26T01:51:29.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.679+0000 7fe903e00640 1 --2- 172.21.15.1:0/4022354604 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe8e80a3bf0 0x7fe8e80a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:51:29.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.682+0000 7fe90a000640 1 -- 172.21.15.1:0/4022354604 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe8f009d1a0 con 0x7fe90c1041e0 2024-04-26T01:51:29.733 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:29 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:29.485+0000 7f2b6e27d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:51:29.787 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:29 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:29.385+0000 7fbbb7ebc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:51:29.788 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:29 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:29.486+0000 7fbbb7ebc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:51:29.788 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:29 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:29.789+0000 7fbbb7ebc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:51:29.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.880+0000 7fe903e00640 1 -- 172.21.15.1:0/4022354604 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe8e80a3bf0 msgr2=0x7fe8e80a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2140737121 2024-04-26T01:51:29.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.880+0000 7fe903e00640 1 --2- 172.21.15.1:0/4022354604 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe8e80a3bf0 0x7fe8e80a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:51:29.879 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:29 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:29.595+0000 7f8dbc288200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:51:29.879 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:29 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:29.742+0000 7f8dbc288200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:51:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.886+0000 7fe913800640 1 -- 172.21.15.1:0/4022354604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fe90c02d5c0 con 0x7fe90c1041e0 2024-04-26T01:51:29.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.892+0000 7fe90a000640 1 -- 172.21.15.1:0/4022354604 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v27)=0 v27) ==== 133+0+0 (secure 0 0 0) 0x7fe90c02d5c0 con 0x7fe90c1041e0 2024-04-26T01:51:29.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.895+0000 7fe913800640 1 -- 172.21.15.1:0/4022354604 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe8e80a3bf0 msgr2=0x7fe8e80a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:51:29.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.895+0000 7fe913800640 1 --2- 172.21.15.1:0/4022354604 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe8e80a3bf0 0x7fe8e80a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.895+0000 7fe913800640 1 -- 172.21.15.1:0/4022354604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90c1041e0 msgr2=0x7fe90c19a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:29.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.895+0000 7fe913800640 1 --2- 172.21.15.1:0/4022354604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90c1041e0 0x7fe90c19a390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000e9c0 tx=0x7fe8f000ee80 comp rx=0 tx=0).stop 2024-04-26T01:51:29.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.895+0000 7fe913800640 1 -- 172.21.15.1:0/4022354604 shutdown_connections 2024-04-26T01:51:29.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.895+0000 7fe913800640 1 --2- 172.21.15.1:0/4022354604 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7fe8e80a3bf0 0x7fe8e80a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.895+0000 7fe913800640 1 --2- 172.21.15.1:0/4022354604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe90c105d90 0x7fe90c1a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.895+0000 7fe913800640 1 --2- 172.21.15.1:0/4022354604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe90c1053d0 0x7fe90c19a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.895+0000 7fe913800640 1 --2- 172.21.15.1:0/4022354604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe90c1041e0 0x7fe90c19a390 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:29.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.895+0000 7fe913800640 1 -- 172.21.15.1:0/4022354604 >> 172.21.15.1:0/4022354604 conn(0x7fe90c0ff770 msgr2=0x7fe90c1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:29.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.895+0000 7fe913800640 1 -- 172.21.15.1:0/4022354604 shutdown_connections 2024-04-26T01:51:29.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:29.895+0000 7fe913800640 1 -- 172.21.15.1:0/4022354604 wait complete. 2024-04-26T01:51:29.910 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T01:51:29.934 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T01:51:29.934 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-26T01:51:29.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:51:29.934 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-04-26T01:51:29.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:51:29.986 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link delete ceph-brx 2024-04-26T01:51:30.066 INFO:teuthology.orchestra.run.smithi001.stderr:Cannot find device "ceph-brx" 2024-04-26T01:51:30.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:30 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3222726085' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T01:51:30.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:30 smithi173 ceph-mon[32553]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-26T01:51:30.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:30 smithi173 ceph-mon[32553]: from='client.? ' entity='client.admin' 2024-04-26T01:51:30.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:30 smithi173 ceph-mon[32553]: from='client.? ' entity='client.admin' 2024-04-26T01:51:30.067 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:29 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:29.871+0000 7fbbb7ebc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:51:30.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T01:51:30.069 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-26T01:51:30.070 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T01:51:30.070 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T01:51:30.070 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T01:51:30.162 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:30 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3222726085' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T01:51:30.162 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:30 smithi001 ceph-mon[27822]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-26T01:51:30.162 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:30 smithi001 ceph-mon[27822]: from='client.? ' entity='client.admin' 2024-04-26T01:51:30.162 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:30 smithi001 ceph-mon[27822]: from='client.? ' entity='client.admin' 2024-04-26T01:51:30.233 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:29 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:29.790+0000 7f2b6e27d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:51:30.233 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:29 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:29.874+0000 7f2b6e27d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:51:30.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:30 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3222726085' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T01:51:30.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:30 smithi097 ceph-mon[32853]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-26T01:51:30.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:30 smithi097 ceph-mon[32853]: from='client.? ' entity='client.admin' 2024-04-26T01:51:30.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:30 smithi097 ceph-mon[32853]: from='client.? ' entity='client.admin' 2024-04-26T01:51:30.492 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:30.629 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:30 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:30.163+0000 7f8dbc288200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:51:30.629 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:30 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:30.282+0000 7f8dbc288200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:51:31.316 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:31 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:31.049+0000 7fbbb7ebc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:51:31.317 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:31 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:31.219+0000 7fbbb7ebc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:51:31.385 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:31 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:31.048+0000 7f2b6e27d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:51:31.385 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:31 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:31.219+0000 7f2b6e27d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:51:31.385 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:31 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:31.387+0000 7f2b6e27d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:51:31.677 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:31 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:31.392+0000 7fbbb7ebc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:51:31.677 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:31 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:31.678+0000 7fbbb7ebc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:51:31.968 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:31 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:31.676+0000 7f2b6e27d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:51:31.968 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:31 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:31.768+0000 7f2b6e27d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:51:31.969 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:31 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:31.969+0000 7f2b6e27d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:51:31.972 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:31 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:31.770+0000 7fbbb7ebc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:51:31.972 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:31 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:31.973+0000 7fbbb7ebc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:51:32.175 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:31 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:31.915+0000 7f8dbc288200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:51:32.233 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:32 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:32.161+0000 7f2b6e27d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:51:32.317 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:32 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:32.165+0000 7fbbb7ebc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:51:32.490 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:32 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:32.175+0000 7f8dbc288200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:51:32.490 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:32 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:32.407+0000 7f8dbc288200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:51:32.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.648+0000 7f8253e00700 1 -- 172.21.15.1:0/2617360541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8254103b30 msgr2=0x7f8254103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:32.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.648+0000 7f8253e00700 1 --2- 172.21.15.1:0/2617360541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8254103b30 0x7f8254103f10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8244009990 tx=0x7f8244009ca0 comp rx=0 tx=0).stop 2024-04-26T01:51:32.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.648+0000 7f8253e00700 1 -- 172.21.15.1:0/2617360541 shutdown_connections 2024-04-26T01:51:32.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.648+0000 7f8253e00700 1 --2- 172.21.15.1:0/2617360541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f825410cf60 0x7f825410f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:32.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.648+0000 7f8253e00700 1 --2- 172.21.15.1:0/2617360541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8254103b30 0x7f8254103f10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:32.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.648+0000 7f8253e00700 1 --2- 172.21.15.1:0/2617360541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8254104450 0x7f825410c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:32.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.648+0000 7f8253e00700 1 -- 172.21.15.1:0/2617360541 >> 172.21.15.1:0/2617360541 conn(0x7f82540fd360 msgr2=0x7f82540ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:32.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.648+0000 7f8253e00700 1 -- 172.21.15.1:0/2617360541 shutdown_connections 2024-04-26T01:51:32.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.648+0000 7f8253e00700 1 -- 172.21.15.1:0/2617360541 wait complete. 2024-04-26T01:51:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.649+0000 7f8253e00700 1 Processor -- start 2024-04-26T01:51:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.650+0000 7f8253e00700 1 -- start start 2024-04-26T01:51:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.650+0000 7f8253e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8254103b30 0x7f825419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.650+0000 7f8253e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8254104450 0x7f825419b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.650+0000 7f8253e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f825410cf60 0x7f825419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.650+0000 7f8253e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8254112ba0 con 0x7f825410cf60 2024-04-26T01:51:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.650+0000 7f8253e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8254112a20 con 0x7f8254103b30 2024-04-26T01:51:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.650+0000 7f8253e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8254112d20 con 0x7f8254104450 2024-04-26T01:51:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.650+0000 7f8252000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8254104450 0x7f825419b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.650+0000 7f8252a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8254103b30 0x7f825419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.650+0000 7f8252000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8254104450 0x7f825419b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:52278/0 (socket says 172.21.15.1:52278) 2024-04-26T01:51:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.650+0000 7f8252000700 1 -- 172.21.15.1:0/3422303442 learned_addr learned my addr 172.21.15.1:0/3422303442 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:32.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.651+0000 7f8253400700 1 --2- 172.21.15.1:0/3422303442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f825410cf60 0x7f825419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:32.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.651+0000 7f8252a00700 1 -- 172.21.15.1:0/3422303442 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8254104450 msgr2=0x7f825419b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:32.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.651+0000 7f8252a00700 1 --2- 172.21.15.1:0/3422303442 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8254104450 0x7f825419b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:32.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.651+0000 7f8252a00700 1 -- 172.21.15.1:0/3422303442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f825410cf60 msgr2=0x7f825419fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:32.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.651+0000 7f8252a00700 1 --2- 172.21.15.1:0/3422303442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f825410cf60 0x7f825419fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:32.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.651+0000 7f8252a00700 1 -- 172.21.15.1:0/3422303442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82541a0310 con 0x7f8254103b30 2024-04-26T01:51:32.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.651+0000 7f8252a00700 1 --2- 172.21.15.1:0/3422303442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8254103b30 0x7f825419b300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8244000c00 tx=0x7f82440047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:32.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.651+0000 7f8258a00700 1 -- 172.21.15.1:0/3422303442 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8244026070 con 0x7f8254103b30 2024-04-26T01:51:32.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.652+0000 7f8258a00700 1 -- 172.21.15.1:0/3422303442 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8244021410 con 0x7f8254103b30 2024-04-26T01:51:32.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.652+0000 7f8253e00700 1 -- 172.21.15.1:0/3422303442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82541a05a0 con 0x7f8254103b30 2024-04-26T01:51:32.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.652+0000 7f8258a00700 1 -- 172.21.15.1:0/3422303442 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8244017650 con 0x7f8254103b30 2024-04-26T01:51:32.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.652+0000 7f8253e00700 1 -- 172.21.15.1:0/3422303442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82541a7e80 con 0x7f8254103b30 2024-04-26T01:51:32.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.653+0000 7f8253e00700 1 -- 172.21.15.1:0/3422303442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8254069730 con 0x7f8254103b30 2024-04-26T01:51:32.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.655+0000 7f8258a00700 1 -- 172.21.15.1:0/3422303442 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 19) ==== 138662+0+0 (secure 0 0 0) 0x7f8244029070 con 0x7f8254103b30 2024-04-26T01:51:32.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.656+0000 7f8258a00700 1 --2- 172.21.15.1:0/3422303442 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f82380a3d10 0x7f82380a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:32.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.656+0000 7f8252000700 1 -- 172.21.15.1:0/3422303442 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f82380a3d10 msgr2=0x7f82380a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2140737121 2024-04-26T01:51:32.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.656+0000 7f8258a00700 1 -- 172.21.15.1:0/3422303442 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f82440d1080 con 0x7f8254103b30 2024-04-26T01:51:32.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.656+0000 7f8252000700 1 --2- 172.21.15.1:0/3422303442 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f82380a3d10 0x7f82380a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:51:32.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.660+0000 7f8258a00700 1 -- 172.21.15.1:0/3422303442 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f824405dff0 con 0x7f8254103b30 2024-04-26T01:51:32.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.857+0000 7f8252000700 1 -- 172.21.15.1:0/3422303442 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f82380a3d10 msgr2=0x7f82380a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2140737121 2024-04-26T01:51:32.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.857+0000 7f8252000700 1 --2- 172.21.15.1:0/3422303442 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f82380a3d10 0x7f82380a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:51:32.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.950+0000 7f8253e00700 1 -- 172.21.15.1:0/3422303442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f825402df80 con 0x7f8254103b30 2024-04-26T01:51:32.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.951+0000 7f8258a00700 1 -- 172.21.15.1:0/3422303442 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f8244013070 con 0x7f8254103b30 2024-04-26T01:51:32.949 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T01:51:32.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.954+0000 7f8253e00700 1 -- 172.21.15.1:0/3422303442 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f82380a3d10 msgr2=0x7f82380a61d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:51:32.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.954+0000 7f8253e00700 1 --2- 172.21.15.1:0/3422303442 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f82380a3d10 0x7f82380a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:32.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.954+0000 7f8253e00700 1 -- 172.21.15.1:0/3422303442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8254103b30 msgr2=0x7f825419b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:32.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.954+0000 7f8253e00700 1 --2- 172.21.15.1:0/3422303442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8254103b30 0x7f825419b300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8244000c00 tx=0x7f82440047b0 comp rx=0 tx=0).stop 2024-04-26T01:51:32.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.954+0000 7f8253e00700 1 -- 172.21.15.1:0/3422303442 shutdown_connections 2024-04-26T01:51:32.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.954+0000 7f8253e00700 1 --2- 172.21.15.1:0/3422303442 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f82380a3d10 0x7f82380a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:32.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.954+0000 7f8253e00700 1 --2- 172.21.15.1:0/3422303442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f825410cf60 0x7f825419fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:32.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.954+0000 7f8253e00700 1 --2- 172.21.15.1:0/3422303442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8254103b30 0x7f825419b300 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:32.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.954+0000 7f8253e00700 1 --2- 172.21.15.1:0/3422303442 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8254104450 0x7f825419b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:32.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.954+0000 7f8253e00700 1 -- 172.21.15.1:0/3422303442 >> 172.21.15.1:0/3422303442 conn(0x7f82540fd360 msgr2=0x7f82540fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:32.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.954+0000 7f8253e00700 1 -- 172.21.15.1:0/3422303442 shutdown_connections 2024-04-26T01:51:32.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:32.954+0000 7f8253e00700 1 -- 172.21.15.1:0/3422303442 wait complete. 2024-04-26T01:51:33.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:33 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3422303442' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T01:51:33.067 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:32 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:32.761+0000 7fbbb7ebc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:51:33.123 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:33 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3422303442' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T01:51:33.124 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:32 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:32.831+0000 7f8dbc288200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:51:33.124 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:32 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:32.962+0000 7f8dbc288200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:51:33.168 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:32 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:32.756+0000 7f2b6e27d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:51:33.169 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:33 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3422303442' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T01:51:33.381 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:33 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:33.232+0000 7f8dbc288200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:51:33.467 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:33 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:33.170+0000 7f2b6e27d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:51:33.467 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:33 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:33.267+0000 7f2b6e27d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:51:33.475 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:33 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:33.179+0000 7fbbb7ebc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:51:33.475 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:33 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:33.275+0000 7fbbb7ebc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:51:33.475 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:33 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:33.476+0000 7fbbb7ebc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:51:33.733 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:33 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:33.469+0000 7f2b6e27d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:51:33.803 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:33 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:33.501+0000 7f8dbc288200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:51:33.889 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T01:51:33.889 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T01:51:33.889 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-04-26T01:51:33.889 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T01:51:33.889 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T01:51:33.889 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T01:51:33.889 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T01:51:33.889 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T01:51:33.889 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T01:51:33.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:51:33.890 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-04-26T01:51:33.939 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T01:51:33.939 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T01:51:33.939 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T01:51:33.939 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-04-26T01:51:33.939 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-04-26T01:51:33.939 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-04-26T01:51:33.939 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T01:51:33.939 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-04-26T01:51:33.939 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-04-26T01:51:33.939 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T01:51:33.939 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-04-26T01:51:33.939 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-04-26T01:51:33.940 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T01:51:33.940 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-04-26T01:51:33.940 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-04-26T01:51:33.940 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T01:51:33.940 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 41888sec preferred_lft 41888sec 2024-04-26T01:51:33.940 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-04-26T01:51:33.940 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-04-26T01:51:33.940 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T01:51:33.940 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-04-26T01:51:33.940 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-04-26T01:51:33.941 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T01:51:33.941 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T01:51:33.941 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-04-26T01:51:33.941 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add name ceph-brx type bridge 2024-04-26T01:51:33.942 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr flush dev ceph-brx 2024-04-26T01:51:33.942 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set ceph-brx up 2024-04-26T01:51:33.942 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T01:51:33.942 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-04-26T01:51:34.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T01:51:34.315 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:33 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:33.987+0000 7f2b6e27d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:51:34.315 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:34 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:34.078+0000 7f2b6e27d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:51:34.316 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:33 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:33.996+0000 7fbbb7ebc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:51:34.316 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:34 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:34.089+0000 7fbbb7ebc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:51:34.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:51:34.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:51:34.418 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T01:51:34.486 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-04-26T01:51:34.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:51:34.490 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-04-26T01:51:34.545 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-04-26T01:51:34.545 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T01:51:34.545 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T01:51:34.545 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T01:51:34.546 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T01:51:34.547 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T01:51:34.547 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-04-26T01:51:34.547 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T01:51:34.547 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T01:51:34.547 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T01:51:34.547 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-04-26T01:51:34.629 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:34 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:34.330+0000 7f8dbc288200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:51:34.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T01:51:34.657 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:34 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:34.317+0000 7f2b6e27d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:51:34.657 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:34 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:34.451+0000 7f2b6e27d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:51:34.657 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:34 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:34.544+0000 7f2b6e27d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:51:34.658 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:34 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:34.659+0000 7f2b6e27d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:51:34.669 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:34 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:34.330+0000 7fbbb7ebc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:51:34.670 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:34 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:34.462+0000 7fbbb7ebc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:51:34.670 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:34 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:34.556+0000 7fbbb7ebc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:51:34.973 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:34 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:34.671+0000 7fbbb7ebc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:51:35.233 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:34 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:34.958+0000 7f2b6e27d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:51:35.238 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:34 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:34.955+0000 7f8dbc288200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:51:35.238 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:35 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:35.094+0000 7f8dbc288200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:51:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:51:35.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:51:35.250 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-04-26T01:51:35.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:51:35.294 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list-id 2024-04-26T01:51:35.316 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:34 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:34.974+0000 7fbbb7ebc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:51:35.386 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T01:51:35.386 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-04-26T01:51:35.386 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T01:51:35.386 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T01:51:35.386 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-04-26T01:51:35.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T01:51:35.509 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:35 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:35.387+0000 7f8dbc288200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:51:35.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:51:35.564 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T01:51:35.565 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T01:51:35.565 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-04-26T01:51:35.565 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T01:51:35.565 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T01:51:35.565 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T01:51:35.565 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T01:51:35.565 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T01:51:35.565 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-04-26T01:51:35.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T01:51:35.841 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:35 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:35.559+0000 7f2b6e27d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:51:35.841 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 01:51:35 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y[34236]: 2024-04-26T01:51:35.773+0000 7f2b6e27d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:51:35.842 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:35 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:35.575+0000 7fbbb7ebc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:51:35.842 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 01:51:35 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z[34426]: 2024-04-26T01:51:35.789+0000 7fbbb7ebc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:51:35.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:51:35.861 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T01:51:35.861 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-04-26T01:51:35.861 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-04-26T01:51:35.861 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T01:51:35.861 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-04-26T01:51:35.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T01:51:36.015 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:35 smithi001 ceph-mon[27822]: Standby manager daemon y restarted 2024-04-26T01:51:36.015 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:35 smithi001 ceph-mon[27822]: Standby manager daemon y started 2024-04-26T01:51:36.015 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:35 smithi001 ceph-mon[27822]: from='mgr.? 172.21.15.97:0/1758313094' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T01:51:36.015 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:35 smithi001 ceph-mon[27822]: from='mgr.? 172.21.15.97:0/1758313094' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:51:36.015 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:35 smithi001 ceph-mon[27822]: from='mgr.? 172.21.15.97:0/1758313094' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T01:51:36.015 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:35 smithi001 ceph-mon[27822]: from='mgr.? 172.21.15.97:0/1758313094' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:51:36.015 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:35 smithi001 ceph-mon[27822]: Standby manager daemon z restarted 2024-04-26T01:51:36.016 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:35 smithi001 ceph-mon[27822]: Standby manager daemon z started 2024-04-26T01:51:36.016 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:35 smithi001 ceph-mon[27822]: from='mgr.? 172.21.15.173:0/2115600171' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T01:51:36.016 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:35 smithi001 ceph-mon[27822]: from='mgr.? 172.21.15.173:0/2115600171' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:51:36.016 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:35 smithi001 ceph-mon[27822]: from='mgr.? 172.21.15.173:0/2115600171' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T01:51:36.016 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:35 smithi001 ceph-mon[27822]: from='mgr.? 172.21.15.173:0/2115600171' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:51:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:51:36.048 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T01:51:36.048 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T01:51:36.092 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T01:51:36.093 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T01:51:36.093 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T01:51:36.183 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-26T01:51:36.183 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-26T01:51:36.183 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/ef5e77dd-fbc9-41b0-9f37-faa021010076 2024-04-26T01:51:36.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:51:36.183 DEBUG:teuthology.orchestra.run.smithi001:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/ef5e77dd-fbc9-41b0-9f37-faa021010076 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-26T01:51:36.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:35 smithi097 ceph-mon[32853]: Standby manager daemon y restarted 2024-04-26T01:51:36.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:35 smithi097 ceph-mon[32853]: Standby manager daemon y started 2024-04-26T01:51:36.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:35 smithi097 ceph-mon[32853]: from='mgr.? 172.21.15.97:0/1758313094' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T01:51:36.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:35 smithi097 ceph-mon[32853]: from='mgr.? 172.21.15.97:0/1758313094' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:51:36.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:35 smithi097 ceph-mon[32853]: from='mgr.? 172.21.15.97:0/1758313094' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T01:51:36.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:35 smithi097 ceph-mon[32853]: from='mgr.? 172.21.15.97:0/1758313094' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:51:36.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:35 smithi097 ceph-mon[32853]: Standby manager daemon z restarted 2024-04-26T01:51:36.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:35 smithi097 ceph-mon[32853]: Standby manager daemon z started 2024-04-26T01:51:36.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:35 smithi097 ceph-mon[32853]: from='mgr.? 172.21.15.173:0/2115600171' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T01:51:36.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:35 smithi097 ceph-mon[32853]: from='mgr.? 172.21.15.173:0/2115600171' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:51:36.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:35 smithi097 ceph-mon[32853]: from='mgr.? 172.21.15.173:0/2115600171' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T01:51:36.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:35 smithi097 ceph-mon[32853]: from='mgr.? 172.21.15.173:0/2115600171' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:51:36.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:35 smithi173 ceph-mon[32553]: Standby manager daemon y restarted 2024-04-26T01:51:36.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:35 smithi173 ceph-mon[32553]: Standby manager daemon y started 2024-04-26T01:51:36.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:35 smithi173 ceph-mon[32553]: from='mgr.? 172.21.15.97:0/1758313094' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T01:51:36.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:35 smithi173 ceph-mon[32553]: from='mgr.? 172.21.15.97:0/1758313094' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:51:36.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:35 smithi173 ceph-mon[32553]: from='mgr.? 172.21.15.97:0/1758313094' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T01:51:36.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:35 smithi173 ceph-mon[32553]: from='mgr.? 172.21.15.97:0/1758313094' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:51:36.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:35 smithi173 ceph-mon[32553]: Standby manager daemon z restarted 2024-04-26T01:51:36.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:35 smithi173 ceph-mon[32553]: Standby manager daemon z started 2024-04-26T01:51:36.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:35 smithi173 ceph-mon[32553]: from='mgr.? 172.21.15.173:0/2115600171' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T01:51:36.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:35 smithi173 ceph-mon[32553]: from='mgr.? 172.21.15.173:0/2115600171' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:51:36.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:35 smithi173 ceph-mon[32553]: from='mgr.? 172.21.15.173:0/2115600171' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T01:51:36.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:35 smithi173 ceph-mon[32553]: from='mgr.? 172.21.15.173:0/2115600171' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:51:36.318 INFO:teuthology.orchestra.run.smithi001.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-26T01:51:36.318 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-26T01:51:36.379 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:36 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:36.110+0000 7f8dbc288200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:51:36.380 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:36 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:36.250+0000 7f8dbc288200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:51:36.783 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:36 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:36.594+0000 7f8dbc288200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:51:37.041 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:36 smithi001 ceph-mon[27822]: mgrmap e20: x(active, since 12m), standbys: y, z 2024-04-26T01:51:37.041 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:36 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:36.784+0000 7f8dbc288200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:51:37.041 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:36 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:36.921+0000 7f8dbc288200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:51:37.051 INFO:teuthology.orchestra.run.smithi001.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-26T01:51:37.051 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-26T01:51:37.051 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: resolved to: "172.21.15.1:3300,172.21.15.97:3300,172.21.15.173:3300" 2024-04-26T01:51:37.051 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: trying mount with new device syntax: 0@77cfcc80-036d-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/ef5e77dd-fbc9-41b0-9f37-faa021010076 2024-04-26T01:51:37.051 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.1:3300/172.21.15.97:3300/172.21.15.173:3300" will pass to kernel 2024-04-26T01:51:37.054 INFO:teuthology.orchestra.run.smithi001.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T01:51:37.054 INFO:teuthology.orchestra.run.smithi001.stdout: You just mounted a file system that supports labels which does not 2024-04-26T01:51:37.054 INFO:teuthology.orchestra.run.smithi001.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T01:51:37.054 INFO:teuthology.orchestra.run.smithi001.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T01:51:37.055 INFO:teuthology.orchestra.run.smithi001.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T01:51:37.059 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T01:51:37.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:51:37.059 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T01:51:37.135 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T01:51:37.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:36 smithi097 ceph-mon[32853]: mgrmap e20: x(active, since 12m), standbys: y, z 2024-04-26T01:51:37.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T01:51:37.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:51:37.250 INFO:teuthology.orchestra.run.smithi001.stdout:client14979 2024-04-26T01:51:37.250 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-04-26T01:51:37.251 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T01:51:37.298 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T01:51:37.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:36 smithi173 ceph-mon[32553]: mgrmap e20: x(active, since 12m), standbys: y, z 2024-04-26T01:51:37.379 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:37 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:37.089+0000 7f8dbc288200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:51:37.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T01:51:37.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:51:37.454 INFO:teuthology.orchestra.run.smithi001.stdout:client14979 2024-04-26T01:51:37.454 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T01:51:37.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T01:51:37.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:51:37.607 INFO:teuthology.orchestra.run.smithi001.stdout:client14979 2024-04-26T01:51:37.608 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/77cfcc80-036d-11ef-bc93-c7b262605968.client14979/status') 2024-04-26T01:51:37.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T01:51:37.731 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:37 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:37.477+0000 7f8dbc288200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:51:37.738 INFO:teuthology.orchestra.run.smithi001.stdout:instance: client.14979 (3)192.168.0.1:0/3589575769 2024-04-26T01:51:37.739 INFO:teuthology.orchestra.run.smithi001.stdout:blocklisted: false 2024-04-26T01:51:37.739 INFO:teuthology.orchestra.run.smithi001.stderr:0+1 records in 2024-04-26T01:51:37.739 INFO:teuthology.orchestra.run.smithi001.stderr:0+1 records out 2024-04-26T01:51:37.739 INFO:teuthology.orchestra.run.smithi001.stderr:70 bytes copied, 8.0449e-05 s, 870 kB/s 2024-04-26T01:51:37.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:51:37.752 INFO:teuthology.run_tasks:Running task exec... 2024-04-26T01:51:37.761 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T01:51:37.762 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-04-26T01:51:37.762 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-26T01:51:37.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.955+0000 7f20fd000640 1 -- 172.21.15.1:0/2436658479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20f8103ff0 msgr2=0x7f20f8104410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:37.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.955+0000 7f20fd000640 1 --2- 172.21.15.1:0/2436658479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20f8103ff0 0x7f20f8104410 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009ca0 tx=0x7f20e802a8c0 comp rx=0 tx=0).stop 2024-04-26T01:51:37.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.955+0000 7f20fd000640 1 -- 172.21.15.1:0/2436658479 shutdown_connections 2024-04-26T01:51:37.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.955+0000 7f20fd000640 1 --2- 172.21.15.1:0/2436658479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f8105ba0 0x7f20f810c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:37.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.955+0000 7f20fd000640 1 --2- 172.21.15.1:0/2436658479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f81051e0 0x7f20f8105660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:37.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.955+0000 7f20fd000640 1 --2- 172.21.15.1:0/2436658479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20f8103ff0 0x7f20f8104410 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:37.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.955+0000 7f20fd000640 1 -- 172.21.15.1:0/2436658479 >> 172.21.15.1:0/2436658479 conn(0x7f20f80ff560 msgr2=0x7f20f81019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:37.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.955+0000 7f20fd000640 1 -- 172.21.15.1:0/2436658479 shutdown_connections 2024-04-26T01:51:37.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.955+0000 7f20fd000640 1 -- 172.21.15.1:0/2436658479 wait complete. 2024-04-26T01:51:37.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.957+0000 7f20fd000640 1 Processor -- start 2024-04-26T01:51:37.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.957+0000 7f20fd000640 1 -- start start 2024-04-26T01:51:37.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.957+0000 7f20fd000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f8103ff0 0x7f20f819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:37.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.957+0000 7f20fd000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20f81051e0 0x7f20f819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:37.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.957+0000 7f20fd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f8105ba0 0x7f20f81a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:37.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.957+0000 7f20fd000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f20f810fb30 con 0x7f20f81051e0 2024-04-26T01:51:37.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.957+0000 7f20fd000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f20f810f9b0 con 0x7f20f8105ba0 2024-04-26T01:51:37.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.957+0000 7f20fd000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f20f810fcb0 con 0x7f20f8103ff0 2024-04-26T01:51:37.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.957+0000 7f20eea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20f81051e0 0x7f20f819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:37.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.957+0000 7f20f7400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f8103ff0 0x7f20f819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:37.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.957+0000 7f20eea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20f81051e0 0x7f20f819a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56966/0 (socket says 172.21.15.1:56966) 2024-04-26T01:51:37.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.957+0000 7f20f7400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f8103ff0 0x7f20f819a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:52284/0 (socket says 172.21.15.1:52284) 2024-04-26T01:51:37.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.957+0000 7f20eea00640 1 -- 172.21.15.1:0/3800991885 learned_addr learned my addr 172.21.15.1:0/3800991885 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:37.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.958+0000 7f20f7e00640 1 --2- 172.21.15.1:0/3800991885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f8105ba0 0x7f20f81a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:37.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.958+0000 7f20eea00640 1 -- 172.21.15.1:0/3800991885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f8103ff0 msgr2=0x7f20f819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:37.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.958+0000 7f20eea00640 1 --2- 172.21.15.1:0/3800991885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f8103ff0 0x7f20f819a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:37.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.958+0000 7f20eea00640 1 -- 172.21.15.1:0/3800991885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f8105ba0 msgr2=0x7f20f81a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:37.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.958+0000 7f20eea00640 1 --2- 172.21.15.1:0/3800991885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f8105ba0 0x7f20f81a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:37.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.958+0000 7f20eea00640 1 -- 172.21.15.1:0/3800991885 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f81a1e60 con 0x7f20f81051e0 2024-04-26T01:51:37.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.958+0000 7f20f7e00640 1 --2- 172.21.15.1:0/3800991885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f8105ba0 0x7f20f81a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:37.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.958+0000 7f20eea00640 1 --2- 172.21.15.1:0/3800991885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20f81051e0 0x7f20f819a780 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f20e000e9f0 tx=0x7f20e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:37.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.959+0000 7f20f4c00640 1 -- 172.21.15.1:0/3800991885 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e000cd80 con 0x7f20f81051e0 2024-04-26T01:51:37.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.959+0000 7f20f4c00640 1 -- 172.21.15.1:0/3800991885 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e00054d0 con 0x7f20f81051e0 2024-04-26T01:51:37.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.959+0000 7f20fd000640 1 -- 172.21.15.1:0/3800991885 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f81a2150 con 0x7f20f81051e0 2024-04-26T01:51:37.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.959+0000 7f20f4c00640 1 -- 172.21.15.1:0/3800991885 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0010690 con 0x7f20f81051e0 2024-04-26T01:51:37.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.959+0000 7f20fd000640 1 -- 172.21.15.1:0/3800991885 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f81a26a0 con 0x7f20f81051e0 2024-04-26T01:51:37.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.960+0000 7f20fd000640 1 -- 172.21.15.1:0/3800991885 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f810b3b0 con 0x7f20f81051e0 2024-04-26T01:51:37.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.963+0000 7f20f4c00640 1 -- 172.21.15.1:0/3800991885 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 138662+0+0 (secure 0 0 0) 0x7f20e0004e30 con 0x7f20f81051e0 2024-04-26T01:51:37.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.964+0000 7f20f4c00640 1 --2- 172.21.15.1:0/3800991885 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f20dc0a3c60 0x7f20dc0a6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:37.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.964+0000 7f20f7400640 1 -- 172.21.15.1:0/3800991885 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f20dc0a3c60 msgr2=0x7f20dc0a6120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2140737121 2024-04-26T01:51:37.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.964+0000 7f20f4c00640 1 -- 172.21.15.1:0/3800991885 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f20e00d0a10 con 0x7f20f81051e0 2024-04-26T01:51:37.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.964+0000 7f20f7400640 1 --2- 172.21.15.1:0/3800991885 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f20dc0a3c60 0x7f20dc0a6120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:51:37.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:37.972+0000 7f20f4c00640 1 -- 172.21.15.1:0/3800991885 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20e0053ff0 con 0x7f20f81051e0 2024-04-26T01:51:38.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.165+0000 7f20f7400640 1 -- 172.21.15.1:0/3800991885 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f20dc0a3c60 msgr2=0x7f20dc0a6120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2140737121 2024-04-26T01:51:38.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.165+0000 7f20f7400640 1 --2- 172.21.15.1:0/3800991885 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f20dc0a3c60 0x7f20dc0a6120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:51:38.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.221+0000 7f20fd000640 1 -- 172.21.15.1:0/3800991885 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f20f802d5c0 con 0x7f20f81051e0 2024-04-26T01:51:38.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.274+0000 7f20f4c00640 1 -- 172.21.15.1:0/3800991885 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f20e009d3d0 con 0x7f20f81051e0 2024-04-26T01:51:38.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.277+0000 7f20fd000640 1 -- 172.21.15.1:0/3800991885 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f20dc0a3c60 msgr2=0x7f20dc0a6120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:51:38.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.277+0000 7f20fd000640 1 --2- 172.21.15.1:0/3800991885 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f20dc0a3c60 0x7f20dc0a6120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:38.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.277+0000 7f20fd000640 1 -- 172.21.15.1:0/3800991885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20f81051e0 msgr2=0x7f20f819a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:38.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.277+0000 7f20fd000640 1 --2- 172.21.15.1:0/3800991885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20f81051e0 0x7f20f819a780 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f20e000e9f0 tx=0x7f20e000edf0 comp rx=0 tx=0).stop 2024-04-26T01:51:38.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.277+0000 7f20fd000640 1 -- 172.21.15.1:0/3800991885 shutdown_connections 2024-04-26T01:51:38.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.277+0000 7f20fd000640 1 --2- 172.21.15.1:0/3800991885 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f20dc0a3c60 0x7f20dc0a6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:38.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.277+0000 7f20fd000640 1 --2- 172.21.15.1:0/3800991885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f8105ba0 0x7f20f81a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:38.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.277+0000 7f20fd000640 1 --2- 172.21.15.1:0/3800991885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20f81051e0 0x7f20f819a780 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:38.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.277+0000 7f20fd000640 1 --2- 172.21.15.1:0/3800991885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f8103ff0 0x7f20f819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:38.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.277+0000 7f20fd000640 1 -- 172.21.15.1:0/3800991885 >> 172.21.15.1:0/3800991885 conn(0x7f20f80ff560 msgr2=0x7f20f8108580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:38.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.278+0000 7f20fd000640 1 -- 172.21.15.1:0/3800991885 shutdown_connections 2024-04-26T01:51:38.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.278+0000 7f20fd000640 1 -- 172.21.15.1:0/3800991885 wait complete. 2024-04-26T01:51:38.300 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-26T01:51:38.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.503+0000 7f6fab800640 1 -- 172.21.15.1:0/551343089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fa4075f90 msgr2=0x7f6fa40743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:38.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.503+0000 7f6fab800640 1 --2- 172.21.15.1:0/551343089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fa4075f90 0x7f6fa40743e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6f98009ca0 tx=0x7f6f9802a8c0 comp rx=0 tx=0).stop 2024-04-26T01:51:38.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.503+0000 7f6fab800640 1 -- 172.21.15.1:0/551343089 shutdown_connections 2024-04-26T01:51:38.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.503+0000 7f6fab800640 1 --2- 172.21.15.1:0/551343089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fa4109e90 0x7f6fa410c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:38.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.503+0000 7f6fab800640 1 --2- 172.21.15.1:0/551343089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fa40749b0 0x7f6fa4074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:38.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.503+0000 7f6fab800640 1 --2- 172.21.15.1:0/551343089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fa4075f90 0x7f6fa40743e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:38.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.503+0000 7f6fab800640 1 -- 172.21.15.1:0/551343089 >> 172.21.15.1:0/551343089 conn(0x7f6fa40fd540 msgr2=0x7f6fa40ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:38.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.503+0000 7f6fab800640 1 -- 172.21.15.1:0/551343089 shutdown_connections 2024-04-26T01:51:38.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.503+0000 7f6fab800640 1 -- 172.21.15.1:0/551343089 wait complete. 2024-04-26T01:51:38.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.504+0000 7f6fab800640 1 Processor -- start 2024-04-26T01:51:38.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.505+0000 7f6fab800640 1 -- start start 2024-04-26T01:51:38.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.505+0000 7f6fab800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fa40749b0 0x7f6fa41052a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:38.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.505+0000 7f6fab800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fa4075f90 0x7f6fa41057e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:38.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.505+0000 7f6fab800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fa4109e90 0x7f6fa4101e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:38.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.505+0000 7f6fab800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa410fcd0 con 0x7f6fa40749b0 2024-04-26T01:51:38.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.505+0000 7f6fab800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa410fb50 con 0x7f6fa4075f90 2024-04-26T01:51:38.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.505+0000 7f6fab800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa410fe50 con 0x7f6fa4109e90 2024-04-26T01:51:38.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.505+0000 7f6fa8e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fa40749b0 0x7f6fa41052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:38.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.505+0000 7f6fa8e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fa40749b0 0x7f6fa41052a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56986/0 (socket says 172.21.15.1:56986) 2024-04-26T01:51:38.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.505+0000 7f6fa8e00640 1 -- 172.21.15.1:0/4064411824 learned_addr learned my addr 172.21.15.1:0/4064411824 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:38.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.505+0000 7f6fa9800640 1 --2- 172.21.15.1:0/4064411824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fa4109e90 0x7f6fa4101e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:38.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.505+0000 7f6fa3e00640 1 --2- 172.21.15.1:0/4064411824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fa4075f90 0x7f6fa41057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:38.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.506+0000 7f6fa8e00640 1 -- 172.21.15.1:0/4064411824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fa4109e90 msgr2=0x7f6fa4101e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:38.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.506+0000 7f6fa8e00640 1 --2- 172.21.15.1:0/4064411824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fa4109e90 0x7f6fa4101e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:38.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.506+0000 7f6fa8e00640 1 -- 172.21.15.1:0/4064411824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fa4075f90 msgr2=0x7f6fa41057e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:38.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.506+0000 7f6fa8e00640 1 --2- 172.21.15.1:0/4064411824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fa4075f90 0x7f6fa41057e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:38.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.506+0000 7f6fa8e00640 1 -- 172.21.15.1:0/4064411824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa41026d0 con 0x7f6fa40749b0 2024-04-26T01:51:38.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.506+0000 7f6fa9800640 1 --2- 172.21.15.1:0/4064411824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fa4109e90 0x7f6fa4101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:38.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.506+0000 7f6fa3e00640 1 --2- 172.21.15.1:0/4064411824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fa4075f90 0x7f6fa41057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:38.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.506+0000 7f6fa8e00640 1 --2- 172.21.15.1:0/4064411824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fa40749b0 0x7f6fa41052a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6f98006010 tx=0x7f6f980317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:38.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.506+0000 7f6fa1600640 1 -- 172.21.15.1:0/4064411824 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98003c30 con 0x7f6fa40749b0 2024-04-26T01:51:38.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.506+0000 7f6fa1600640 1 -- 172.21.15.1:0/4064411824 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f98003dd0 con 0x7f6fa40749b0 2024-04-26T01:51:38.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.506+0000 7f6fab800640 1 -- 172.21.15.1:0/4064411824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa4102960 con 0x7f6fa40749b0 2024-04-26T01:51:38.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.506+0000 7f6fa1600640 1 -- 172.21.15.1:0/4064411824 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9802fc20 con 0x7f6fa40749b0 2024-04-26T01:51:38.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.507+0000 7f6fab800640 1 -- 172.21.15.1:0/4064411824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa41a6c60 con 0x7f6fa40749b0 2024-04-26T01:51:38.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.508+0000 7f6fab800640 1 -- 172.21.15.1:0/4064411824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa4108bc0 con 0x7f6fa40749b0 2024-04-26T01:51:38.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.510+0000 7f6fa1600640 1 -- 172.21.15.1:0/4064411824 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 138662+0+0 (secure 0 0 0) 0x7f6f980046a0 con 0x7f6fa40749b0 2024-04-26T01:51:38.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.511+0000 7f6fa1600640 1 --2- 172.21.15.1:0/4064411824 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f6f8c0a3bf0 0x7f6f8c0a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:38.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.511+0000 7f6fa1600640 1 -- 172.21.15.1:0/4064411824 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8269+0+0 (secure 0 0 0) 0x7f6f980f1b90 con 0x7f6fa40749b0 2024-04-26T01:51:38.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.512+0000 7f6fa3e00640 1 -- 172.21.15.1:0/4064411824 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f6f8c0a3bf0 msgr2=0x7f6f8c0a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2140737121 2024-04-26T01:51:38.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.512+0000 7f6fa3e00640 1 --2- 172.21.15.1:0/4064411824 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f6f8c0a3bf0 0x7f6f8c0a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:51:38.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.514+0000 7f6fa1600640 1 -- 172.21.15.1:0/4064411824 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f980be580 con 0x7f6fa40749b0 2024-04-26T01:51:38.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:38 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3800991885' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-26T01:51:38.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:38 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3800991885' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-26T01:51:38.630 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:38 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:38.330+0000 7f8dbc288200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:51:38.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.676+0000 7f6fa1600640 1 -- 172.21.15.1:0/4064411824 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f6f980ba4f0 con 0x7f6fa40749b0 2024-04-26T01:51:38.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.676+0000 7f6fa1600640 1 -- 172.21.15.1:0/4064411824 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f6f8c0a3bf0 msgr2=0x7f6f8c0a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:51:38.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:38.676+0000 7f6fa1600640 1 --2- 172.21.15.1:0/4064411824 >> [v2:172.21.15.1:6800/2140737121,v1:172.21.15.1:6801/2140737121] conn(0x7f6f8c0a3bf0 0x7f6f8c0a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:38.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:38 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3800991885' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-26T01:51:38.963 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:38 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:38.655+0000 7f8dbc288200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:51:39.284 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:38 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:38.964+0000 7f8d0ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:51:39.284 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:38 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:38.964+0000 7f8d0ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:51:39.284 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:38 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:38.964+0000 7f8d0ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:51:39.284 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:38 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:38.964+0000 7f8d0ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:51:39.284 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:38 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:38.964+0000 7f8d0ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:51:39.284 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:38 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:38.966+0000 7f8d11c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:51:39.285 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:38 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:38.966+0000 7f8d11c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:51:39.285 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:38 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:38.966+0000 7f8d11c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:51:39.285 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:38 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:38.966+0000 7f8d11c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:51:39.285 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:38 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:38.966+0000 7f8d11c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:51:39.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3800991885' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-26T01:51:39.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:51:39.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: Active manager daemon x restarted 2024-04-26T01:51:39.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: Activating manager daemon x 2024-04-26T01:51:39.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T01:51:39.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: mgrmap e21: x(active, starting, since 0.0160009s), standbys: y, z 2024-04-26T01:51:39.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:51:39.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:51:39.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:51:39.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T01:51:39.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T01:51:39.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T01:51:39.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T01:51:39.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T01:51:39.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T01:51:39.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T01:51:39.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T01:51:39.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:51:39.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T01:51:39.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T01:51:39.569 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T01:51:39.569 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T01:51:39.569 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T01:51:39.569 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T01:51:39.569 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:51:39.569 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:51:39.569 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:51:39.569 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:51:39.569 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:51:39.569 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:51:39.569 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:51:39.569 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:51:39.569 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:51:39.569 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:39.570 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:51:39.570 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:51:39.570 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T01:51:39.570 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T01:51:39.570 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T01:51:39.570 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: Manager daemon x is now available 2024-04-26T01:51:39.570 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:51:39.570 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:51:39.570 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:51:39.570 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:51:39.570 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:39 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:51:39.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3800991885' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-26T01:51:39.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:51:39.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: Active manager daemon x restarted 2024-04-26T01:51:39.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: Activating manager daemon x 2024-04-26T01:51:39.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T01:51:39.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: mgrmap e21: x(active, starting, since 0.0160009s), standbys: y, z 2024-04-26T01:51:39.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:51:39.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:51:39.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:51:39.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T01:51:39.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T01:51:39.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T01:51:39.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T01:51:39.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T01:51:39.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T01:51:39.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T01:51:39.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T01:51:39.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:51:39.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T01:51:39.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T01:51:39.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T01:51:39.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T01:51:39.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T01:51:39.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T01:51:39.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:51:39.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:51:39.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:51:39.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:51:39.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:51:39.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:51:39.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:51:39.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:51:39.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:51:39.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:39.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:51:39.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:51:39.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T01:51:39.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T01:51:39.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T01:51:39.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: Manager daemon x is now available 2024-04-26T01:51:39.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:51:39.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:51:39.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:51:39.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:51:39.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:51:39.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:39.693+0000 7f6fa1600640 1 -- 172.21.15.1:0/4064411824 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f6f980ef690 con 0x7f6fa40749b0 2024-04-26T01:51:39.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:39.694+0000 7f6fa1600640 1 --2- 172.21.15.1:0/4064411824 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6f8c052240 0x7f6f8c052620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:39.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:39.694+0000 7f6fa1600640 1 -- 172.21.15.1:0/4064411824 --> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f6f8c045420 con 0x7f6f8c052240 2024-04-26T01:51:39.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:39.705+0000 7f6fa3e00640 1 --2- 172.21.15.1:0/4064411824 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6f8c052240 0x7f6f8c052620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:39.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:39.718+0000 7f6fa3e00640 1 --2- 172.21.15.1:0/4064411824 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6f8c052240 0x7f6f8c052620 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4103220 tx=0x7f6f90005f90 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:39.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3800991885' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-26T01:51:39.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:51:39.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: Active manager daemon x restarted 2024-04-26T01:51:39.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: Activating manager daemon x 2024-04-26T01:51:39.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T01:51:39.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: mgrmap e21: x(active, starting, since 0.0160009s), standbys: y, z 2024-04-26T01:51:39.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:51:39.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:51:39.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:51:39.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T01:51:39.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T01:51:39.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T01:51:39.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T01:51:39.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T01:51:39.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T01:51:39.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T01:51:39.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T01:51:39.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:51:39.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T01:51:39.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T01:51:39.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T01:51:39.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T01:51:39.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T01:51:39.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T01:51:39.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:51:39.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:51:39.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:51:39.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:51:39.736 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:51:39.736 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:51:39.736 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:51:39.736 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:51:39.736 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:51:39.736 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:39.736 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:51:39.736 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:51:39.736 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T01:51:39.736 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T01:51:39.736 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T01:51:39.736 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: Manager daemon x is now available 2024-04-26T01:51:39.737 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:51:39.737 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:51:39.737 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:51:39.737 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:51:39.737 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:39 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:51:39.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:39.753+0000 7f6fa1600640 1 -- 172.21.15.1:0/4064411824 <== mgr.25087 v2:172.21.15.1:6800/275906565 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f6f8c045420 con 0x7f6f8c052240 2024-04-26T01:51:39.751 INFO:teuthology.orchestra.run.smithi001.stdout:{} 2024-04-26T01:51:39.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:39.756+0000 7f6fab800640 1 -- 172.21.15.1:0/4064411824 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6f8c052240 msgr2=0x7f6f8c052620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:39.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:39.756+0000 7f6fab800640 1 --2- 172.21.15.1:0/4064411824 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6f8c052240 0x7f6f8c052620 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4103220 tx=0x7f6f90005f90 comp rx=0 tx=0).stop 2024-04-26T01:51:39.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:39.756+0000 7f6fab800640 1 -- 172.21.15.1:0/4064411824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fa40749b0 msgr2=0x7f6fa41052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:39.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:39.756+0000 7f6fab800640 1 --2- 172.21.15.1:0/4064411824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fa40749b0 0x7f6fa41052a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6f98006010 tx=0x7f6f980317a0 comp rx=0 tx=0).stop 2024-04-26T01:51:39.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:39.756+0000 7f6fa9800640 1 -- 172.21.15.1:0/4064411824 reap_dead start 2024-04-26T01:51:39.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:39.756+0000 7f6fab800640 1 -- 172.21.15.1:0/4064411824 shutdown_connections 2024-04-26T01:51:39.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:39.756+0000 7f6fab800640 1 -- 172.21.15.1:0/4064411824 >> 172.21.15.1:0/4064411824 conn(0x7f6fa40fd540 msgr2=0x7f6fa4105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:39.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:39.756+0000 7f6fab800640 1 -- 172.21.15.1:0/4064411824 shutdown_connections 2024-04-26T01:51:39.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:39.756+0000 7f6fab800640 1 -- 172.21.15.1:0/4064411824 wait complete. 2024-04-26T01:51:39.771 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-26T01:51:39.782 INFO:teuthology.run_tasks:Running task exec... 2024-04-26T01:51:39.810 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T01:51:39.810 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-04-26T01:51:39.810 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-26T01:51:40.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.015+0000 7f8675400640 1 -- 172.21.15.1:0/3844427806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86701031b0 msgr2=0x7f8670103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:40.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.015+0000 7f8675400640 1 --2- 172.21.15.1:0/3844427806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86701031b0 0x7f8670103630 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8658009900 tx=0x7f865802a410 comp rx=0 tx=0).stop 2024-04-26T01:51:40.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.015+0000 7f8675400640 1 -- 172.21.15.1:0/3844427806 shutdown_connections 2024-04-26T01:51:40.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.015+0000 7f8675400640 1 --2- 172.21.15.1:0/3844427806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8670103b70 0x7f867010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.015+0000 7f8675400640 1 --2- 172.21.15.1:0/3844427806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86701031b0 0x7f8670103630 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.015+0000 7f8675400640 1 --2- 172.21.15.1:0/3844427806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8670101fc0 0x7f86701023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.015+0000 7f8675400640 1 -- 172.21.15.1:0/3844427806 >> 172.21.15.1:0/3844427806 conn(0x7f86700fd530 msgr2=0x7f86700ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:40.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.015+0000 7f8675400640 1 -- 172.21.15.1:0/3844427806 shutdown_connections 2024-04-26T01:51:40.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.015+0000 7f8675400640 1 -- 172.21.15.1:0/3844427806 wait complete. 2024-04-26T01:51:40.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.016+0000 7f8675400640 1 Processor -- start 2024-04-26T01:51:40.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.016+0000 7f8675400640 1 -- start start 2024-04-26T01:51:40.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.017+0000 7f8675400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8670101fc0 0x7f8670075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:40.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.017+0000 7f8675400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86701031b0 0x7f8670074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:40.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.017+0000 7f8675400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8670103b70 0x7f8670074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:40.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.017+0000 7f8675400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f867010db00 con 0x7f8670101fc0 2024-04-26T01:51:40.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.017+0000 7f8675400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f867010d980 con 0x7f86701031b0 2024-04-26T01:51:40.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.017+0000 7f8675400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f867010dc80 con 0x7f8670103b70 2024-04-26T01:51:40.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.017+0000 7f866e800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8670101fc0 0x7f8670075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:40.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.017+0000 7f866f200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8670103b70 0x7f8670074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:40.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.017+0000 7f866f200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8670103b70 0x7f8670074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:52432/0 (socket says 172.21.15.1:52432) 2024-04-26T01:51:40.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.017+0000 7f8665e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86701031b0 0x7f8670074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:40.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.017+0000 7f866f200640 1 -- 172.21.15.1:0/3691390050 learned_addr learned my addr 172.21.15.1:0/3691390050 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:40.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.017+0000 7f866f200640 1 -- 172.21.15.1:0/3691390050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86701031b0 msgr2=0x7f8670074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:40.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.017+0000 7f866f200640 1 --2- 172.21.15.1:0/3691390050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86701031b0 0x7f8670074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.017+0000 7f866f200640 1 -- 172.21.15.1:0/3691390050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8670101fc0 msgr2=0x7f8670075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:40.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.017+0000 7f866f200640 1 --2- 172.21.15.1:0/3691390050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8670101fc0 0x7f8670075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.018+0000 7f866f200640 1 -- 172.21.15.1:0/3691390050 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8670075110 con 0x7f8670103b70 2024-04-26T01:51:40.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.018+0000 7f866f200640 1 --2- 172.21.15.1:0/3691390050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8670103b70 0x7f8670074880 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f866800e970 tx=0x7f866800ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:40.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.018+0000 7f8667e00640 1 -- 172.21.15.1:0/3691390050 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866800ccc0 con 0x7f8670103b70 2024-04-26T01:51:40.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.018+0000 7f8667e00640 1 -- 172.21.15.1:0/3691390050 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f866800ce60 con 0x7f8670103b70 2024-04-26T01:51:40.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.018+0000 7f8667e00640 1 -- 172.21.15.1:0/3691390050 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668010430 con 0x7f8670103b70 2024-04-26T01:51:40.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.018+0000 7f8675400640 1 -- 172.21.15.1:0/3691390050 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86701a2300 con 0x7f8670103b70 2024-04-26T01:51:40.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.019+0000 7f8675400640 1 -- 172.21.15.1:0/3691390050 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8670197fc0 con 0x7f8670103b70 2024-04-26T01:51:40.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.020+0000 7f8675400640 1 -- 172.21.15.1:0/3691390050 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8670109350 con 0x7f8670103b70 2024-04-26T01:51:40.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.022+0000 7f8667e00640 1 -- 172.21.15.1:0/3691390050 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f8668004f90 con 0x7f8670103b70 2024-04-26T01:51:40.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.023+0000 7f8667e00640 1 --2- 172.21.15.1:0/3691390050 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f86540a3a10 0x7f86540a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:40.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.023+0000 7f8667e00640 1 -- 172.21.15.1:0/3691390050 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f86680d0510 con 0x7f8670103b70 2024-04-26T01:51:40.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.024+0000 7f866e800640 1 --2- 172.21.15.1:0/3691390050 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f86540a3a10 0x7f86540a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:40.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.025+0000 7f866e800640 1 --2- 172.21.15.1:0/3691390050 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f86540a3a10 0x7f86540a5ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f86600099a0 tx=0x7f8660009500 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:40.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.027+0000 7f8667e00640 1 -- 172.21.15.1:0/3691390050 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f866809d080 con 0x7f8670103b70 2024-04-26T01:51:40.131 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:39.736+0000 7f8d0ae00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:51:40.131 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:39.736+0000 7f8d0ae00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:51:40.131 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:39.736+0000 7f8d0ae00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:51:40.131 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:39.736+0000 7f8d0ae00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:51:40.131 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:51:39 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:51:39.736+0000 7f8d0ae00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:51:40.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.262+0000 7f8675400640 1 -- 172.21.15.1:0/3691390050 --> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f8670062390 con 0x7f86540a3a10 2024-04-26T01:51:40.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.403+0000 7f8667e00640 1 -- 172.21.15.1:0/3691390050 <== mgr.25087 v2:172.21.15.1:6800/275906565 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8670062390 con 0x7f86540a3a10 2024-04-26T01:51:40.401 INFO:teuthology.orchestra.run.smithi001.stdout:Schedule set for path / 2024-04-26T01:51:40.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.406+0000 7f8675400640 1 -- 172.21.15.1:0/3691390050 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f86540a3a10 msgr2=0x7f86540a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:40.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.406+0000 7f8675400640 1 --2- 172.21.15.1:0/3691390050 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f86540a3a10 0x7f86540a5ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f86600099a0 tx=0x7f8660009500 comp rx=0 tx=0).stop 2024-04-26T01:51:40.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.406+0000 7f8675400640 1 -- 172.21.15.1:0/3691390050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8670103b70 msgr2=0x7f8670074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:40.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.406+0000 7f8675400640 1 --2- 172.21.15.1:0/3691390050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8670103b70 0x7f8670074880 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f866800e970 tx=0x7f866800ed70 comp rx=0 tx=0).stop 2024-04-26T01:51:40.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.406+0000 7f8675400640 1 -- 172.21.15.1:0/3691390050 shutdown_connections 2024-04-26T01:51:40.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.406+0000 7f8675400640 1 --2- 172.21.15.1:0/3691390050 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f86540a3a10 0x7f86540a5ed0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.406+0000 7f8675400640 1 --2- 172.21.15.1:0/3691390050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8670103b70 0x7f8670074880 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.406+0000 7f8675400640 1 --2- 172.21.15.1:0/3691390050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86701031b0 0x7f8670074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.406+0000 7f8675400640 1 --2- 172.21.15.1:0/3691390050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8670101fc0 0x7f8670075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.406+0000 7f8675400640 1 -- 172.21.15.1:0/3691390050 >> 172.21.15.1:0/3691390050 conn(0x7f86700fd530 msgr2=0x7f8670106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:40.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.406+0000 7f8675400640 1 -- 172.21.15.1:0/3691390050 shutdown_connections 2024-04-26T01:51:40.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.406+0000 7f8675400640 1 -- 172.21.15.1:0/3691390050 wait complete. 2024-04-26T01:51:40.432 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-26T01:51:40.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.623+0000 7ffa80200640 1 -- 172.21.15.1:0/3287534554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa780784f0 msgr2=0x7ffa78078d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:40.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.623+0000 7ffa80200640 1 --2- 172.21.15.1:0/3287534554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa780784f0 0x7ffa78078d90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009be0 tx=0x7ffa7002a6c0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.623+0000 7ffa80200640 1 -- 172.21.15.1:0/3287534554 shutdown_connections 2024-04-26T01:51:40.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.623+0000 7ffa80200640 1 --2- 172.21.15.1:0/3287534554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa780784f0 0x7ffa78078d90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.623+0000 7ffa80200640 1 --2- 172.21.15.1:0/3287534554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa78077b50 0x7ffa78077fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.623+0000 7ffa80200640 1 --2- 172.21.15.1:0/3287534554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa78076910 0x7ffa78076d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.623+0000 7ffa80200640 1 -- 172.21.15.1:0/3287534554 >> 172.21.15.1:0/3287534554 conn(0x7ffa780ff770 msgr2=0x7ffa78101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:40.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.624+0000 7ffa80200640 1 -- 172.21.15.1:0/3287534554 shutdown_connections 2024-04-26T01:51:40.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.624+0000 7ffa80200640 1 -- 172.21.15.1:0/3287534554 wait complete. 2024-04-26T01:51:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.625+0000 7ffa80200640 1 Processor -- start 2024-04-26T01:51:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.625+0000 7ffa80200640 1 -- start start 2024-04-26T01:51:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.625+0000 7ffa80200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa78076910 0x7ffa7819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.625+0000 7ffa80200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa78077b50 0x7ffa7819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.625+0000 7ffa80200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa780784f0 0x7ffa781a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.625+0000 7ffa80200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffa78114120 con 0x7ffa78077b50 2024-04-26T01:51:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.625+0000 7ffa80200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffa78113fa0 con 0x7ffa780784f0 2024-04-26T01:51:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.625+0000 7ffa80200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffa781142a0 con 0x7ffa78076910 2024-04-26T01:51:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.625+0000 7ffa7d800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa78076910 0x7ffa7819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.625+0000 7ffa7d800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa78076910 0x7ffa7819e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:52446/0 (socket says 172.21.15.1:52446) 2024-04-26T01:51:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.625+0000 7ffa7d800640 1 -- 172.21.15.1:0/1793601956 learned_addr learned my addr 172.21.15.1:0/1793601956 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.625+0000 7ffa7e200640 1 --2- 172.21.15.1:0/1793601956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa780784f0 0x7ffa781a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.625+0000 7ffa7ce00640 1 --2- 172.21.15.1:0/1793601956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa78077b50 0x7ffa7819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.626+0000 7ffa7d800640 1 -- 172.21.15.1:0/1793601956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa780784f0 msgr2=0x7ffa781a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.626+0000 7ffa7d800640 1 --2- 172.21.15.1:0/1793601956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa780784f0 0x7ffa781a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.626+0000 7ffa7d800640 1 -- 172.21.15.1:0/1793601956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa78077b50 msgr2=0x7ffa7819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.626+0000 7ffa7d800640 1 --2- 172.21.15.1:0/1793601956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa78077b50 0x7ffa7819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.626+0000 7ffa7d800640 1 -- 172.21.15.1:0/1793601956 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa781a63a0 con 0x7ffa78076910 2024-04-26T01:51:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.626+0000 7ffa7d800640 1 --2- 172.21.15.1:0/1793601956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa78076910 0x7ffa7819e780 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00e9f0 tx=0x7ffa6c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.626+0000 7ffa76000640 1 -- 172.21.15.1:0/1793601956 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c00ccc0 con 0x7ffa78076910 2024-04-26T01:51:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.626+0000 7ffa80200640 1 -- 172.21.15.1:0/1793601956 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa781a6690 con 0x7ffa78076910 2024-04-26T01:51:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.626+0000 7ffa76000640 1 -- 172.21.15.1:0/1793601956 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa6c0108e0 con 0x7ffa78076910 2024-04-26T01:51:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.626+0000 7ffa76000640 1 -- 172.21.15.1:0/1793601956 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c018880 con 0x7ffa78076910 2024-04-26T01:51:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.627+0000 7ffa80200640 1 -- 172.21.15.1:0/1793601956 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa781a6c30 con 0x7ffa78076910 2024-04-26T01:51:40.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.628+0000 7ffa80200640 1 -- 172.21.15.1:0/1793601956 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa78110d10 con 0x7ffa78076910 2024-04-26T01:51:40.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.630+0000 7ffa76000640 1 -- 172.21.15.1:0/1793601956 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7ffa6c010430 con 0x7ffa78076910 2024-04-26T01:51:40.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.631+0000 7ffa76000640 1 --2- 172.21.15.1:0/1793601956 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ffa600a3990 0x7ffa600a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:40.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.631+0000 7ffa76000640 1 -- 172.21.15.1:0/1793601956 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7ffa6c0d0630 con 0x7ffa78076910 2024-04-26T01:51:40.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.631+0000 7ffa7ce00640 1 --2- 172.21.15.1:0/1793601956 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ffa600a3990 0x7ffa600a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:40.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.632+0000 7ffa7ce00640 1 --2- 172.21.15.1:0/1793601956 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ffa600a3990 0x7ffa600a5e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ffa7819fc90 tx=0x7ffa64009260 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:40.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.633+0000 7ffa76000640 1 -- 172.21.15.1:0/1793601956 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa6c09d1a0 con 0x7ffa78076910 2024-04-26T01:51:40.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.707+0000 7ffa76000640 1 -- 172.21.15.1:0/1793601956 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mgrmap(e 23) ==== 138816+0+0 (secure 0 0 0) 0x7ffa6c099110 con 0x7ffa78076910 2024-04-26T01:51:40.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.874+0000 7ffa80200640 1 -- 172.21.15.1:0/1793601956 --> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7ffa780624f0 con 0x7ffa600a3990 2024-04-26T01:51:40.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.965+0000 7ffa76000640 1 -- 172.21.15.1:0/1793601956 <== mgr.25087 v2:172.21.15.1:6800/275906565 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7ffa780624f0 con 0x7ffa600a3990 2024-04-26T01:51:40.963 INFO:teuthology.orchestra.run.smithi001.stdout:Retention added to path / 2024-04-26T01:51:40.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.967+0000 7ffa80200640 1 -- 172.21.15.1:0/1793601956 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ffa600a3990 msgr2=0x7ffa600a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:40.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.967+0000 7ffa80200640 1 --2- 172.21.15.1:0/1793601956 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ffa600a3990 0x7ffa600a5e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ffa7819fc90 tx=0x7ffa64009260 comp rx=0 tx=0).stop 2024-04-26T01:51:40.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.967+0000 7ffa80200640 1 -- 172.21.15.1:0/1793601956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa78076910 msgr2=0x7ffa7819e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:40.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.967+0000 7ffa80200640 1 --2- 172.21.15.1:0/1793601956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa78076910 0x7ffa7819e780 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00e9f0 tx=0x7ffa6c00edf0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.967+0000 7ffa80200640 1 -- 172.21.15.1:0/1793601956 shutdown_connections 2024-04-26T01:51:40.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.967+0000 7ffa80200640 1 --2- 172.21.15.1:0/1793601956 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ffa600a3990 0x7ffa600a5e50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.967+0000 7ffa80200640 1 --2- 172.21.15.1:0/1793601956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa780784f0 0x7ffa781a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.967+0000 7ffa80200640 1 --2- 172.21.15.1:0/1793601956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa78077b50 0x7ffa7819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.967+0000 7ffa80200640 1 --2- 172.21.15.1:0/1793601956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa78076910 0x7ffa7819e780 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:40.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.967+0000 7ffa80200640 1 -- 172.21.15.1:0/1793601956 >> 172.21.15.1:0/1793601956 conn(0x7ffa780ff770 msgr2=0x7ffa78100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:40.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.968+0000 7ffa80200640 1 -- 172.21.15.1:0/1793601956 shutdown_connections 2024-04-26T01:51:40.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:40.968+0000 7ffa80200640 1 -- 172.21.15.1:0/1793601956 wait complete. 2024-04-26T01:51:40.980 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:40 smithi001 ceph-mon[27822]: [26/Apr/2024:01:51:39] ENGINE Bus STARTING 2024-04-26T01:51:40.980 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:40 smithi001 ceph-mon[27822]: [26/Apr/2024:01:51:39] ENGINE Serving on https://172.21.15.1:7150 2024-04-26T01:51:40.981 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:40 smithi001 ceph-mon[27822]: [26/Apr/2024:01:51:39] ENGINE Serving on http://172.21.15.1:8765 2024-04-26T01:51:40.981 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:40 smithi001 ceph-mon[27822]: [26/Apr/2024:01:51:39] ENGINE Bus STARTED 2024-04-26T01:51:40.981 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:40 smithi001 ceph-mon[27822]: mgrmap e22: x(active, since 1.03979s), standbys: y, z 2024-04-26T01:51:40.981 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:40 smithi001 ceph-mon[27822]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:40.981 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:40 smithi001 ceph-mon[27822]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:51:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:40 smithi097 ceph-mon[32853]: [26/Apr/2024:01:51:39] ENGINE Bus STARTING 2024-04-26T01:51:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:40 smithi097 ceph-mon[32853]: [26/Apr/2024:01:51:39] ENGINE Serving on https://172.21.15.1:7150 2024-04-26T01:51:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:40 smithi097 ceph-mon[32853]: [26/Apr/2024:01:51:39] ENGINE Serving on http://172.21.15.1:8765 2024-04-26T01:51:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:40 smithi097 ceph-mon[32853]: [26/Apr/2024:01:51:39] ENGINE Bus STARTED 2024-04-26T01:51:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:40 smithi097 ceph-mon[32853]: mgrmap e22: x(active, since 1.03979s), standbys: y, z 2024-04-26T01:51:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:40 smithi097 ceph-mon[32853]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:40 smithi097 ceph-mon[32853]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:51:40.984 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-26T01:51:41.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:40 smithi173 ceph-mon[32553]: [26/Apr/2024:01:51:39] ENGINE Bus STARTING 2024-04-26T01:51:41.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:40 smithi173 ceph-mon[32553]: [26/Apr/2024:01:51:39] ENGINE Serving on https://172.21.15.1:7150 2024-04-26T01:51:41.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:40 smithi173 ceph-mon[32553]: [26/Apr/2024:01:51:39] ENGINE Serving on http://172.21.15.1:8765 2024-04-26T01:51:41.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:40 smithi173 ceph-mon[32553]: [26/Apr/2024:01:51:39] ENGINE Bus STARTED 2024-04-26T01:51:41.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:40 smithi173 ceph-mon[32553]: mgrmap e22: x(active, since 1.03979s), standbys: y, z 2024-04-26T01:51:41.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:40 smithi173 ceph-mon[32553]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:41.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:40 smithi173 ceph-mon[32553]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:51:41.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.158+0000 7f7fea000640 1 -- 172.21.15.1:0/1395715168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7fe4076910 msgr2=0x7f7fe4076d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:41.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.158+0000 7f7fea000640 1 --2- 172.21.15.1:0/1395715168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7fe4076910 0x7f7fe4076d30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009860 tx=0x7f7fd402a340 comp rx=0 tx=0).stop 2024-04-26T01:51:41.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.158+0000 7f7fea000640 1 -- 172.21.15.1:0/1395715168 shutdown_connections 2024-04-26T01:51:41.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.158+0000 7f7fea000640 1 --2- 172.21.15.1:0/1395715168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fe40784f0 0x7f7fe4078d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.158+0000 7f7fea000640 1 --2- 172.21.15.1:0/1395715168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fe4077b50 0x7f7fe4077fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.158+0000 7f7fea000640 1 --2- 172.21.15.1:0/1395715168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7fe4076910 0x7f7fe4076d30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.158+0000 7f7fea000640 1 -- 172.21.15.1:0/1395715168 >> 172.21.15.1:0/1395715168 conn(0x7f7fe40ff5c0 msgr2=0x7f7fe41019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:41.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.158+0000 7f7fea000640 1 -- 172.21.15.1:0/1395715168 shutdown_connections 2024-04-26T01:51:41.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.158+0000 7f7fea000640 1 -- 172.21.15.1:0/1395715168 wait complete. 2024-04-26T01:51:41.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.160+0000 7f7fea000640 1 Processor -- start 2024-04-26T01:51:41.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.160+0000 7f7fea000640 1 -- start start 2024-04-26T01:51:41.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.160+0000 7f7fea000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fe4076910 0x7f7fe419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:41.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.161+0000 7f7fea000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7fe4077b50 0x7f7fe419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:41.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.161+0000 7f7fe8c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fe4076910 0x7f7fe419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:41.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.161+0000 7f7fea000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fe40784f0 0x7f7fe41a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:41.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.161+0000 7f7fea000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe4113f70 con 0x7f7fe4077b50 2024-04-26T01:51:41.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.161+0000 7f7fea000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe4113df0 con 0x7f7fe4076910 2024-04-26T01:51:41.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.161+0000 7f7fea000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe41140f0 con 0x7f7fe40784f0 2024-04-26T01:51:41.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.161+0000 7f7fe8c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fe4076910 0x7f7fe419e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:35038/0 (socket says 172.21.15.1:35038) 2024-04-26T01:51:41.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.161+0000 7f7fe8c00640 1 -- 172.21.15.1:0/1450631648 learned_addr learned my addr 172.21.15.1:0/1450631648 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:41.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.161+0000 7f7fe3e00640 1 --2- 172.21.15.1:0/1450631648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7fe4077b50 0x7f7fe419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:41.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.161+0000 7f7fe9600640 1 --2- 172.21.15.1:0/1450631648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fe40784f0 0x7f7fe41a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:41.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.161+0000 7f7fe8c00640 1 -- 172.21.15.1:0/1450631648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fe40784f0 msgr2=0x7f7fe41a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:41.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.161+0000 7f7fe8c00640 1 --2- 172.21.15.1:0/1450631648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fe40784f0 0x7f7fe41a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.161+0000 7f7fe8c00640 1 -- 172.21.15.1:0/1450631648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7fe4077b50 msgr2=0x7f7fe419eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:41.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.161+0000 7f7fe8c00640 1 --2- 172.21.15.1:0/1450631648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7fe4077b50 0x7f7fe419eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.161+0000 7f7fe8c00640 1 -- 172.21.15.1:0/1450631648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe41a6210 con 0x7f7fe4076910 2024-04-26T01:51:41.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.161+0000 7f7fe9600640 1 --2- 172.21.15.1:0/1450631648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fe40784f0 0x7f7fe41a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:51:41.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.162+0000 7f7fe8c00640 1 --2- 172.21.15.1:0/1450631648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fe4076910 0x7f7fe419e5f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40057c0 tx=0x7f7fd4032da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:41.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.162+0000 7f7fe1600640 1 -- 172.21.15.1:0/1450631648 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4037040 con 0x7f7fe4076910 2024-04-26T01:51:41.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.162+0000 7f7fe1600640 1 -- 172.21.15.1:0/1450631648 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fd4030bd0 con 0x7f7fe4076910 2024-04-26T01:51:41.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.162+0000 7f7fea000640 1 -- 172.21.15.1:0/1450631648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe41a64a0 con 0x7f7fe4076910 2024-04-26T01:51:41.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.162+0000 7f7fe1600640 1 -- 172.21.15.1:0/1450631648 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4005280 con 0x7f7fe4076910 2024-04-26T01:51:41.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.162+0000 7f7fea000640 1 -- 172.21.15.1:0/1450631648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe41a6930 con 0x7f7fe4076910 2024-04-26T01:51:41.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.163+0000 7f7fea000640 1 -- 172.21.15.1:0/1450631648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fe41109a0 con 0x7f7fe4076910 2024-04-26T01:51:41.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.165+0000 7f7fe1600640 1 -- 172.21.15.1:0/1450631648 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 23) ==== 138816+0+0 (secure 0 0 0) 0x7f7fd4043020 con 0x7f7fe4076910 2024-04-26T01:51:41.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.167+0000 7f7fe1600640 1 --2- 172.21.15.1:0/1450631648 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7fc80a3d90 0x7f7fc80a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:41.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.168+0000 7f7fe1600640 1 -- 172.21.15.1:0/1450631648 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f7fd40f00a0 con 0x7f7fe4076910 2024-04-26T01:51:41.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.168+0000 7f7fe3e00640 1 --2- 172.21.15.1:0/1450631648 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7fc80a3d90 0x7f7fc80a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:41.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.169+0000 7f7fe3e00640 1 --2- 172.21.15.1:0/1450631648 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7fc80a3d90 0x7f7fc80a6250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc0095f0 tx=0x7f7fcc005f90 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:41.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.171+0000 7f7fe1600640 1 -- 172.21.15.1:0/1450631648 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7fd40bc680 con 0x7f7fe4076910 2024-04-26T01:51:41.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.413+0000 7f7fea000640 1 -- 172.21.15.1:0/1450631648 --> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f7fe4062390 con 0x7f7fc80a3d90 2024-04-26T01:51:41.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.443+0000 7f7fe1600640 1 -- 172.21.15.1:0/1450631648 <== mgr.25087 v2:172.21.15.1:6800/275906565 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f7fe4062390 con 0x7f7fc80a3d90 2024-04-26T01:51:41.441 INFO:teuthology.orchestra.run.smithi001.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-26T00:00:00", "created": "2024-04-26T01:51:40", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-26T01:51:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.445+0000 7f7fea000640 1 -- 172.21.15.1:0/1450631648 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7fc80a3d90 msgr2=0x7f7fc80a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.445+0000 7f7fea000640 1 --2- 172.21.15.1:0/1450631648 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7fc80a3d90 0x7f7fc80a6250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc0095f0 tx=0x7f7fcc005f90 comp rx=0 tx=0).stop 2024-04-26T01:51:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.445+0000 7f7fea000640 1 -- 172.21.15.1:0/1450631648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fe4076910 msgr2=0x7f7fe419e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.445+0000 7f7fea000640 1 --2- 172.21.15.1:0/1450631648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fe4076910 0x7f7fe419e5f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40057c0 tx=0x7f7fd4032da0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.445+0000 7f7fea000640 1 -- 172.21.15.1:0/1450631648 shutdown_connections 2024-04-26T01:51:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.446+0000 7f7fea000640 1 --2- 172.21.15.1:0/1450631648 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7fc80a3d90 0x7f7fc80a6250 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.446+0000 7f7fea000640 1 --2- 172.21.15.1:0/1450631648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fe40784f0 0x7f7fe41a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.446+0000 7f7fea000640 1 --2- 172.21.15.1:0/1450631648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7fe4077b50 0x7f7fe419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.446+0000 7f7fea000640 1 --2- 172.21.15.1:0/1450631648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fe4076910 0x7f7fe419e5f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.446+0000 7f7fea000640 1 -- 172.21.15.1:0/1450631648 >> 172.21.15.1:0/1450631648 conn(0x7f7fe40ff5c0 msgr2=0x7f7fe41010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:41.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.446+0000 7f7fea000640 1 -- 172.21.15.1:0/1450631648 shutdown_connections 2024-04-26T01:51:41.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.446+0000 7f7fea000640 1 -- 172.21.15.1:0/1450631648 wait complete. 2024-04-26T01:51:41.461 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-26T01:51:41.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.648+0000 7f5535200640 1 -- 172.21.15.1:0/2517393970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5530075f90 msgr2=0x7f55300743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:41.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.648+0000 7f5535200640 1 --2- 172.21.15.1:0/2517393970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5530075f90 0x7f55300743e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5520009900 tx=0x7f552002a3f0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.648+0000 7f5535200640 1 -- 172.21.15.1:0/2517393970 shutdown_connections 2024-04-26T01:51:41.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.648+0000 7f5535200640 1 --2- 172.21.15.1:0/2517393970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f553010e3e0 0x7f55301107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.648+0000 7f5535200640 1 --2- 172.21.15.1:0/2517393970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55300749b0 0x7f5530074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.648+0000 7f5535200640 1 --2- 172.21.15.1:0/2517393970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5530075f90 0x7f55300743e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.648+0000 7f5535200640 1 -- 172.21.15.1:0/2517393970 >> 172.21.15.1:0/2517393970 conn(0x7f55300fd380 msgr2=0x7f55300ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:41.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.648+0000 7f5535200640 1 -- 172.21.15.1:0/2517393970 shutdown_connections 2024-04-26T01:51:41.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.648+0000 7f5535200640 1 -- 172.21.15.1:0/2517393970 wait complete. 2024-04-26T01:51:41.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.649+0000 7f5535200640 1 Processor -- start 2024-04-26T01:51:41.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.650+0000 7f5535200640 1 -- start start 2024-04-26T01:51:41.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.650+0000 7f5535200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55300749b0 0x7f553019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:41.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.650+0000 7f5535200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5530075f90 0x7f553019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:41.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.650+0000 7f5535200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f553010e3e0 0x7f55301a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:41.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.650+0000 7f5535200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5530113f60 con 0x7f5530075f90 2024-04-26T01:51:41.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.650+0000 7f5535200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5530113de0 con 0x7f55300749b0 2024-04-26T01:51:41.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.650+0000 7f5535200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f55301140e0 con 0x7f553010e3e0 2024-04-26T01:51:41.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.650+0000 7f552f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55300749b0 0x7f553019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:41.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.650+0000 7f5527e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5530075f90 0x7f553019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:41.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.650+0000 7f552f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55300749b0 0x7f553019e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:35058/0 (socket says 172.21.15.1:35058) 2024-04-26T01:51:41.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.650+0000 7f552f400640 1 -- 172.21.15.1:0/3041291861 learned_addr learned my addr 172.21.15.1:0/3041291861 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:41.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.650+0000 7f552fe00640 1 --2- 172.21.15.1:0/3041291861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f553010e3e0 0x7f55301a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:41.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.651+0000 7f552f400640 1 -- 172.21.15.1:0/3041291861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f553010e3e0 msgr2=0x7f55301a5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:41.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.651+0000 7f552f400640 1 --2- 172.21.15.1:0/3041291861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f553010e3e0 0x7f55301a5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.651+0000 7f552f400640 1 -- 172.21.15.1:0/3041291861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5530075f90 msgr2=0x7f553019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:41.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.651+0000 7f552f400640 1 --2- 172.21.15.1:0/3041291861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5530075f90 0x7f553019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.651+0000 7f552f400640 1 -- 172.21.15.1:0/3041291861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55301a6240 con 0x7f55300749b0 2024-04-26T01:51:41.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.651+0000 7f5527e00640 1 --2- 172.21.15.1:0/3041291861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5530075f90 0x7f553019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:41.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.651+0000 7f552f400640 1 --2- 172.21.15.1:0/3041291861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55300749b0 0x7f553019e620 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5520031490 tx=0x7f5520003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:41.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.651+0000 7f552cc00640 1 -- 172.21.15.1:0/3041291861 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552003f070 con 0x7f55300749b0 2024-04-26T01:51:41.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.651+0000 7f5535200640 1 -- 172.21.15.1:0/3041291861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55301a64d0 con 0x7f55300749b0 2024-04-26T01:51:41.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.651+0000 7f552cc00640 1 -- 172.21.15.1:0/3041291861 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5520031bf0 con 0x7f55300749b0 2024-04-26T01:51:41.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.651+0000 7f552cc00640 1 -- 172.21.15.1:0/3041291861 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520035030 con 0x7f55300749b0 2024-04-26T01:51:41.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.652+0000 7f5535200640 1 -- 172.21.15.1:0/3041291861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55301a69c0 con 0x7f55300749b0 2024-04-26T01:51:41.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.653+0000 7f5535200640 1 -- 172.21.15.1:0/3041291861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55301045b0 con 0x7f55300749b0 2024-04-26T01:51:41.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.655+0000 7f552cc00640 1 -- 172.21.15.1:0/3041291861 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 23) ==== 138816+0+0 (secure 0 0 0) 0x7f5520042070 con 0x7f55300749b0 2024-04-26T01:51:41.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.656+0000 7f552cc00640 1 --2- 172.21.15.1:0/3041291861 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f55140a3d90 0x7f55140a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:41.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.656+0000 7f552cc00640 1 -- 172.21.15.1:0/3041291861 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f55200f1100 con 0x7f55300749b0 2024-04-26T01:51:41.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.656+0000 7f5527e00640 1 --2- 172.21.15.1:0/3041291861 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f55140a3d90 0x7f55140a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:41.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.657+0000 7f5527e00640 1 --2- 172.21.15.1:0/3041291861 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f55140a3d90 0x7f55140a6250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f553019fb30 tx=0x7f5518005f90 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:41.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.660+0000 7f552cc00640 1 -- 172.21.15.1:0/3041291861 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f55200bcf80 con 0x7f55300749b0 2024-04-26T01:51:41.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:41 smithi001 ceph-mon[27822]: from='client.25162 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:41.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:41 smithi001 ceph-mon[27822]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:51:41.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:41 smithi001 ceph-mon[27822]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-26T01:51:41.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:41 smithi001 ceph-mon[27822]: from='client.25168 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:41.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:41 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:41.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:41 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:41.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:41 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:41.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:41 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:41.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.919+0000 7f5535200640 1 -- 172.21.15.1:0/3041291861 --> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5530062390 con 0x7f55140a3d90 2024-04-26T01:51:41.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.934+0000 7f552cc00640 1 -- 172.21.15.1:0/3041291861 <== mgr.25087 v2:172.21.15.1:6800/275906565 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5530062390 con 0x7f55140a3d90 2024-04-26T01:51:41.932 INFO:teuthology.orchestra.run.smithi001.stdout:/ 1m 6m3h 2024-04-26T01:51:41.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.937+0000 7f5535200640 1 -- 172.21.15.1:0/3041291861 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f55140a3d90 msgr2=0x7f55140a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:41.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.937+0000 7f5535200640 1 --2- 172.21.15.1:0/3041291861 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f55140a3d90 0x7f55140a6250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f553019fb30 tx=0x7f5518005f90 comp rx=0 tx=0).stop 2024-04-26T01:51:41.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.937+0000 7f5535200640 1 -- 172.21.15.1:0/3041291861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55300749b0 msgr2=0x7f553019e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:41.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.937+0000 7f5535200640 1 --2- 172.21.15.1:0/3041291861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55300749b0 0x7f553019e620 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5520031490 tx=0x7f5520003c10 comp rx=0 tx=0).stop 2024-04-26T01:51:41.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.937+0000 7f5535200640 1 -- 172.21.15.1:0/3041291861 shutdown_connections 2024-04-26T01:51:41.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.937+0000 7f5535200640 1 --2- 172.21.15.1:0/3041291861 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f55140a3d90 0x7f55140a6250 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.937+0000 7f5535200640 1 --2- 172.21.15.1:0/3041291861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f553010e3e0 0x7f55301a5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.937+0000 7f5535200640 1 --2- 172.21.15.1:0/3041291861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5530075f90 0x7f553019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.937+0000 7f5535200640 1 --2- 172.21.15.1:0/3041291861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55300749b0 0x7f553019e620 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:41.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.937+0000 7f5535200640 1 -- 172.21.15.1:0/3041291861 >> 172.21.15.1:0/3041291861 conn(0x7f55300fd380 msgr2=0x7f5530101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:41.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.937+0000 7f5535200640 1 -- 172.21.15.1:0/3041291861 shutdown_connections 2024-04-26T01:51:41.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:41.937+0000 7f5535200640 1 -- 172.21.15.1:0/3041291861 wait complete. 2024-04-26T01:51:41.955 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-26T01:51:41.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:41 smithi097 ceph-mon[32853]: from='client.25162 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:41.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:41 smithi097 ceph-mon[32853]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:51:41.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:41 smithi097 ceph-mon[32853]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-26T01:51:41.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:41 smithi097 ceph-mon[32853]: from='client.25168 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:41.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:41 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:41.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:41 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:41.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:41 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:41.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:41 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:42.028 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-26T01:51:42.039 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-26T01:51:42.048 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-04-26T01:51:42.048 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-26T01:51:42.052 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-26T01:51:42.064 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T01:51:42.064 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:51:42.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:41 smithi173 ceph-mon[32553]: from='client.25162 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:42.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:41 smithi173 ceph-mon[32553]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:51:42.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:41 smithi173 ceph-mon[32553]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-26T01:51:42.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:41 smithi173 ceph-mon[32553]: from='client.25168 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:42.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:41 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:42.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:41 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:42.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:41 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:42.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:41 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:42.485 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:43.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='client.25157 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:43.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='client.25163 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:43.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:43.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:43.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:43.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:43.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:43.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:43.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-26T01:51:43.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: Unable to set osd_memory_target on smithi097 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-26T01:51:43.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: Adjusting osd_memory_target on smithi173 to 176.2M 2024-04-26T01:51:43.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:42 smithi097 ceph-mon[32853]: Unable to set osd_memory_target on smithi173 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-26T01:51:43.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='client.25157 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:43.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='client.25163 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:43.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:43.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:43.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:43.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:43.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:43.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:43.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.319 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.319 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.319 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.319 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-26T01:51:43.319 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.319 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.319 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: Unable to set osd_memory_target on smithi097 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-26T01:51:43.319 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: Adjusting osd_memory_target on smithi173 to 176.2M 2024-04-26T01:51:43.319 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:42 smithi173 ceph-mon[32553]: Unable to set osd_memory_target on smithi173 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-26T01:51:43.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='client.25157 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:43.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='client.25163 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:43.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:43.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:43.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:43.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:43.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:43.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:43.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-26T01:51:43.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.383 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:43.383 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: Unable to set osd_memory_target on smithi097 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-26T01:51:43.383 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: Adjusting osd_memory_target on smithi173 to 176.2M 2024-04-26T01:51:43.383 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:42 smithi001 ceph-mon[27822]: Unable to set osd_memory_target on smithi173 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-26T01:51:44.218 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:44 smithi001 ceph-mon[27822]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:51:44.218 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:44 smithi001 ceph-mon[27822]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-26T01:51:44.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:44 smithi173 ceph-mon[32553]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:51:44.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:44 smithi173 ceph-mon[32553]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-26T01:51:44.434 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:44 smithi097 ceph-mon[32853]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:51:44.435 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:44 smithi097 ceph-mon[32853]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-26T01:51:44.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.693+0000 7fc7cf200700 1 -- 172.21.15.1:0/670793838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d0102970 msgr2=0x7fc7d0102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:44.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.693+0000 7fc7cf200700 1 --2- 172.21.15.1:0/670793838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d0102970 0x7fc7d0102df0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc009990 tx=0x7fc7bc009ca0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.693+0000 7fc7cf200700 1 -- 172.21.15.1:0/670793838 shutdown_connections 2024-04-26T01:51:44.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.693+0000 7fc7cf200700 1 --2- 172.21.15.1:0/670793838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7d0103330 0x7fc7d010f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.693+0000 7fc7cf200700 1 --2- 172.21.15.1:0/670793838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7d0108970 0x7fc7d0108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.693+0000 7fc7cf200700 1 --2- 172.21.15.1:0/670793838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d0102970 0x7fc7d0102df0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.693+0000 7fc7cf200700 1 -- 172.21.15.1:0/670793838 >> 172.21.15.1:0/670793838 conn(0x7fc7d00fe4c0 msgr2=0x7fc7d01008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:44.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.694+0000 7fc7cf200700 1 -- 172.21.15.1:0/670793838 shutdown_connections 2024-04-26T01:51:44.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.694+0000 7fc7cf200700 1 -- 172.21.15.1:0/670793838 wait complete. 2024-04-26T01:51:44.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.695+0000 7fc7cf200700 1 Processor -- start 2024-04-26T01:51:44.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.695+0000 7fc7cf200700 1 -- start start 2024-04-26T01:51:44.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.696+0000 7fc7cf200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7d0102970 0x7fc7d019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:44.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.696+0000 7fc7cf200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7d0103330 0x7fc7d019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:44.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.696+0000 7fc7cf200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d0108970 0x7fc7d019fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:44.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.696+0000 7fc7cf200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d0112c00 con 0x7fc7d0103330 2024-04-26T01:51:44.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.696+0000 7fc7cf200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d0112a80 con 0x7fc7d0108970 2024-04-26T01:51:44.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.696+0000 7fc7cf200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d0112d80 con 0x7fc7d0102970 2024-04-26T01:51:44.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.696+0000 7fc7cde00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7d0102970 0x7fc7d019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:44.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.696+0000 7fc7cde00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7d0102970 0x7fc7d019b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:48298/0 (socket says 172.21.15.1:48298) 2024-04-26T01:51:44.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.696+0000 7fc7cde00700 1 -- 172.21.15.1:0/3338909704 learned_addr learned my addr 172.21.15.1:0/3338909704 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:44.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.696+0000 7fc7cd400700 1 --2- 172.21.15.1:0/3338909704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7d0103330 0x7fc7d019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:44.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.697+0000 7fc7ce800700 1 --2- 172.21.15.1:0/3338909704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d0108970 0x7fc7d019fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:44.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.697+0000 7fc7cde00700 1 -- 172.21.15.1:0/3338909704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d0108970 msgr2=0x7fc7d019fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:44.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.697+0000 7fc7cde00700 1 --2- 172.21.15.1:0/3338909704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d0108970 0x7fc7d019fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.697+0000 7fc7cde00700 1 -- 172.21.15.1:0/3338909704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7d0103330 msgr2=0x7fc7d019b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:44.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.697+0000 7fc7cde00700 1 --2- 172.21.15.1:0/3338909704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7d0103330 0x7fc7d019b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.697+0000 7fc7cde00700 1 -- 172.21.15.1:0/3338909704 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d01a02c0 con 0x7fc7d0102970 2024-04-26T01:51:44.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.697+0000 7fc7ce800700 1 --2- 172.21.15.1:0/3338909704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d0108970 0x7fc7d019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:51:44.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.697+0000 7fc7cde00700 1 --2- 172.21.15.1:0/3338909704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7d0102970 0x7fc7d019b2b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000e9c0 tx=0x7fc7c000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:44.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.698+0000 7fc7c6a00700 1 -- 172.21.15.1:0/3338909704 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c000cbd0 con 0x7fc7d0102970 2024-04-26T01:51:44.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.698+0000 7fc7cf200700 1 -- 172.21.15.1:0/3338909704 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7d01a05b0 con 0x7fc7d0102970 2024-04-26T01:51:44.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.698+0000 7fc7c6a00700 1 -- 172.21.15.1:0/3338909704 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7c000cd70 con 0x7fc7d0102970 2024-04-26T01:51:44.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.698+0000 7fc7c6a00700 1 -- 172.21.15.1:0/3338909704 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0005610 con 0x7fc7d0102970 2024-04-26T01:51:44.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.698+0000 7fc7cf200700 1 -- 172.21.15.1:0/3338909704 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7d01a7e30 con 0x7fc7d0102970 2024-04-26T01:51:44.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.699+0000 7fc7cf200700 1 -- 172.21.15.1:0/3338909704 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7d0069730 con 0x7fc7d0102970 2024-04-26T01:51:44.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.701+0000 7fc7c6a00700 1 -- 172.21.15.1:0/3338909704 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fc7c0004750 con 0x7fc7d0102970 2024-04-26T01:51:44.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.703+0000 7fc7c6a00700 1 --2- 172.21.15.1:0/3338909704 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc7b40a3f50 0x7fc7b40a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:44.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.703+0000 7fc7c6a00700 1 -- 172.21.15.1:0/3338909704 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7fc7c00d0cf0 con 0x7fc7d0102970 2024-04-26T01:51:44.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.703+0000 7fc7cd400700 1 --2- 172.21.15.1:0/3338909704 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc7b40a3f50 0x7fc7b40a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:44.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.704+0000 7fc7cd400700 1 --2- 172.21.15.1:0/3338909704 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc7b40a3f50 0x7fc7b40a6410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc7d010f5b0 tx=0x7fc7bc005a90 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:44.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.707+0000 7fc7c6a00700 1 -- 172.21.15.1:0/3338909704 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7c009d0e0 con 0x7fc7d0102970 2024-04-26T01:51:44.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.959+0000 7fc7cf200700 1 -- 172.21.15.1:0/3338909704 --> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc7d010e2a0 con 0x7fc7b40a3f50 2024-04-26T01:51:44.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.973+0000 7fc7c6a00700 1 -- 172.21.15.1:0/3338909704 <== mgr.25087 v2:172.21.15.1:6800/275906565 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257769 (secure 0 0 0) 0x7fc7bc066f00 con 0x7fc7b40a3f50 2024-04-26T01:51:44.977 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:51:44.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.979+0000 7fc7cf200700 1 -- 172.21.15.1:0/3338909704 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc7b40a3f50 msgr2=0x7fc7b40a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:44.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.979+0000 7fc7cf200700 1 --2- 172.21.15.1:0/3338909704 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc7b40a3f50 0x7fc7b40a6410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc7d010f5b0 tx=0x7fc7bc005a90 comp rx=0 tx=0).stop 2024-04-26T01:51:44.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.979+0000 7fc7cf200700 1 -- 172.21.15.1:0/3338909704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7d0102970 msgr2=0x7fc7d019b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:44.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.979+0000 7fc7cf200700 1 --2- 172.21.15.1:0/3338909704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7d0102970 0x7fc7d019b2b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000e9c0 tx=0x7fc7c000ecd0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.979+0000 7fc7cf200700 1 -- 172.21.15.1:0/3338909704 shutdown_connections 2024-04-26T01:51:44.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.979+0000 7fc7cf200700 1 --2- 172.21.15.1:0/3338909704 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc7b40a3f50 0x7fc7b40a6410 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.979+0000 7fc7cf200700 1 --2- 172.21.15.1:0/3338909704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d0108970 0x7fc7d019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.979+0000 7fc7cf200700 1 --2- 172.21.15.1:0/3338909704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7d0102970 0x7fc7d019b2b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.979+0000 7fc7cf200700 1 --2- 172.21.15.1:0/3338909704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7d0103330 0x7fc7d019b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:44.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.979+0000 7fc7cf200700 1 -- 172.21.15.1:0/3338909704 >> 172.21.15.1:0/3338909704 conn(0x7fc7d00fe4c0 msgr2=0x7fc7d010db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:44.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.979+0000 7fc7cf200700 1 -- 172.21.15.1:0/3338909704 shutdown_connections 2024-04-26T01:51:44.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:44.979+0000 7fc7cf200700 1 -- 172.21.15.1:0/3338909704 wait complete. 2024-04-26T01:51:44.980 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-04-26T01:51:45.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:45.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:45.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: Adjusting osd_memory_target on smithi001 to 177.5M 2024-04-26T01:51:45.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: Unable to set osd_memory_target on smithi001 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-26T01:51:45.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:45.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:51:45.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: Updating smithi001:/etc/ceph/ceph.conf 2024-04-26T01:51:45.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-26T01:51:45.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-26T01:51:45.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: Updating smithi097:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:51:45.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: Updating smithi173:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:51:45.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:51:45.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:51:45.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:51:45.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: Updating smithi001:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:51:45.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: from='client.25189 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:51:45.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: Updating smithi097:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:51:45.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:45 smithi001 ceph-mon[27822]: Updating smithi173:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:51:45.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:45.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:45.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: Adjusting osd_memory_target on smithi001 to 177.5M 2024-04-26T01:51:45.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: Unable to set osd_memory_target on smithi001 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-26T01:51:45.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:45.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:51:45.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: Updating smithi001:/etc/ceph/ceph.conf 2024-04-26T01:51:45.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-26T01:51:45.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-26T01:51:45.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: Updating smithi097:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:51:45.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: Updating smithi173:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:51:45.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:51:45.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:51:45.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:51:45.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: Updating smithi001:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:51:45.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: from='client.25189 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:51:45.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: Updating smithi097:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:51:45.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:45 smithi097 ceph-mon[32853]: Updating smithi173:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:51:45.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:45.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:45.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:45.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: Adjusting osd_memory_target on smithi001 to 177.5M 2024-04-26T01:51:45.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: Unable to set osd_memory_target on smithi001 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-26T01:51:45.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:45.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:51:45.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: Updating smithi001:/etc/ceph/ceph.conf 2024-04-26T01:51:45.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-26T01:51:45.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-26T01:51:45.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: Updating smithi097:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:51:45.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: Updating smithi173:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:51:45.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:51:45.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:51:45.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:51:45.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: Updating smithi001:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:51:45.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: from='client.25189 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:51:45.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: Updating smithi097:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:51:45.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:45 smithi173 ceph-mon[32553]: Updating smithi173:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:51:45.786 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-26T01:51:44.691289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188104,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1638,"num_read_kb":979,"num_write":477,"num_write_kb":4379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364776,"kb_used_data":15588,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124577048,"statfs":{"total":1151940427776,"available":1151566897152,"internally_reserved":0,"allocated":15962112,"data_stored":9401790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.951388"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581382+0000","last_change":"2024-04-26T01:47:27.140418+0000","last_active":"2024-04-26T01:51:39.581382+0000","last_peered":"2024-04-26T01:51:39.581382+0000","last_clean":"2024-04-26T01:51:39.581382+0000","last_became_active":"2024-04-26T01:47:27.140017+0000","last_became_peered":"2024-04-26T01:47:27.140017+0000","last_unstale":"2024-04-26T01:51:39.581382+0000","last_undegraded":"2024-04-26T01:51:39.581382+0000","last_fullsized":"2024-04-26T01:51:39.581382+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:46:08.621222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.226726+0000","last_change":"2024-04-26T01:47:27.138746+0000","last_active":"2024-04-26T01:51:39.226726+0000","last_peered":"2024-04-26T01:51:39.226726+0000","last_clean":"2024-04-26T01:51:39.226726+0000","last_became_active":"2024-04-26T01:47:27.136163+0000","last_became_peered":"2024-04-26T01:47:27.136163+0000","last_unstale":"2024-04-26T01:51:39.226726+0000","last_undegraded":"2024-04-26T01:51:39.226726+0000","last_fullsized":"2024-04-26T01:51:39.226726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:01:36.195332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.226469+0000","last_change":"2024-04-26T01:47:27.127017+0000","last_active":"2024-04-26T01:51:39.226469+0000","last_peered":"2024-04-26T01:51:39.226469+0000","last_clean":"2024-04-26T01:51:39.226469+0000","last_became_active":"2024-04-26T01:47:27.126610+0000","last_became_peered":"2024-04-26T01:47:27.126610+0000","last_unstale":"2024-04-26T01:51:39.226469+0000","last_undegraded":"2024-04-26T01:51:39.226469+0000","last_fullsized":"2024-04-26T01:51:39.226469+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:05:12.784486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737519+0000","last_change":"2024-04-26T01:47:27.117640+0000","last_active":"2024-04-26T01:51:38.737519+0000","last_peered":"2024-04-26T01:51:38.737519+0000","last_clean":"2024-04-26T01:51:38.737519+0000","last_became_active":"2024-04-26T01:47:27.117266+0000","last_became_peered":"2024-04-26T01:47:27.117266+0000","last_unstale":"2024-04-26T01:51:38.737519+0000","last_undegraded":"2024-04-26T01:51:38.737519+0000","last_fullsized":"2024-04-26T01:51:38.737519+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:26:57.824560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.760037+0000","last_change":"2024-04-26T01:47:27.128185+0000","last_active":"2024-04-26T01:51:38.760037+0000","last_peered":"2024-04-26T01:51:38.760037+0000","last_clean":"2024-04-26T01:51:38.760037+0000","last_became_active":"2024-04-26T01:47:27.127794+0000","last_became_peered":"2024-04-26T01:47:27.127794+0000","last_unstale":"2024-04-26T01:51:38.760037+0000","last_undegraded":"2024-04-26T01:51:38.760037+0000","last_fullsized":"2024-04-26T01:51:38.760037+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:09:52.774007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672330+0000","last_change":"2024-04-26T01:48:22.748601+0000","last_active":"2024-04-26T01:51:38.672330+0000","last_peered":"2024-04-26T01:51:38.672330+0000","last_clean":"2024-04-26T01:51:38.672330+0000","last_became_active":"2024-04-26T01:48:22.748229+0000","last_became_peered":"2024-04-26T01:48:22.748229+0000","last_unstale":"2024-04-26T01:51:38.672330+0000","last_undegraded":"2024-04-26T01:51:38.672330+0000","last_fullsized":"2024-04-26T01:51:38.672330+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:46:30.433063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.972514+0000","last_change":"2024-04-26T01:47:27.123786+0000","last_active":"2024-04-26T01:51:38.972514+0000","last_peered":"2024-04-26T01:51:38.972514+0000","last_clean":"2024-04-26T01:51:38.972514+0000","last_became_active":"2024-04-26T01:47:27.123473+0000","last_became_peered":"2024-04-26T01:47:27.123473+0000","last_unstale":"2024-04-26T01:51:38.972514+0000","last_undegraded":"2024-04-26T01:51:38.972514+0000","last_fullsized":"2024-04-26T01:51:38.972514+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:48:55.031147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737899+0000","last_change":"2024-04-26T01:47:27.118756+0000","last_active":"2024-04-26T01:51:38.737899+0000","last_peered":"2024-04-26T01:51:38.737899+0000","last_clean":"2024-04-26T01:51:38.737899+0000","last_became_active":"2024-04-26T01:47:27.118406+0000","last_became_peered":"2024-04-26T01:47:27.118406+0000","last_unstale":"2024-04-26T01:51:38.737899+0000","last_undegraded":"2024-04-26T01:51:38.737899+0000","last_fullsized":"2024-04-26T01:51:38.737899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:05.793682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.980594+0000","last_change":"2024-04-26T01:47:27.117622+0000","last_active":"2024-04-26T01:51:38.980594+0000","last_peered":"2024-04-26T01:51:38.980594+0000","last_clean":"2024-04-26T01:51:38.980594+0000","last_became_active":"2024-04-26T01:47:27.115206+0000","last_became_peered":"2024-04-26T01:47:27.115206+0000","last_unstale":"2024-04-26T01:51:38.980594+0000","last_undegraded":"2024-04-26T01:51:38.980594+0000","last_fullsized":"2024-04-26T01:51:38.980594+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:13:17.866630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672921+0000","last_change":"2024-04-26T01:47:27.119524+0000","last_active":"2024-04-26T01:51:38.672921+0000","last_peered":"2024-04-26T01:51:38.672921+0000","last_clean":"2024-04-26T01:51:38.672921+0000","last_became_active":"2024-04-26T01:47:27.119008+0000","last_became_peered":"2024-04-26T01:47:27.119008+0000","last_unstale":"2024-04-26T01:51:38.672921+0000","last_undegraded":"2024-04-26T01:51:38.672921+0000","last_fullsized":"2024-04-26T01:51:38.672921+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:37:31.894486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973282+0000","last_change":"2024-04-26T01:47:27.122361+0000","last_active":"2024-04-26T01:51:38.973282+0000","last_peered":"2024-04-26T01:51:38.973282+0000","last_clean":"2024-04-26T01:51:38.973282+0000","last_became_active":"2024-04-26T01:47:27.121748+0000","last_became_peered":"2024-04-26T01:47:27.121748+0000","last_unstale":"2024-04-26T01:51:38.973282+0000","last_undegraded":"2024-04-26T01:51:38.973282+0000","last_fullsized":"2024-04-26T01:51:38.973282+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:18.680909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737482+0000","last_change":"2024-04-26T01:47:27.116833+0000","last_active":"2024-04-26T01:51:38.737482+0000","last_peered":"2024-04-26T01:51:38.737482+0000","last_clean":"2024-04-26T01:51:38.737482+0000","last_became_active":"2024-04-26T01:47:27.116508+0000","last_became_peered":"2024-04-26T01:47:27.116508+0000","last_unstale":"2024-04-26T01:51:38.737482+0000","last_undegraded":"2024-04-26T01:51:38.737482+0000","last_fullsized":"2024-04-26T01:51:38.737482+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:27:45.864521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.671761+0000","last_change":"2024-04-26T01:47:27.125342+0000","last_active":"2024-04-26T01:51:38.671761+0000","last_peered":"2024-04-26T01:51:38.671761+0000","last_clean":"2024-04-26T01:51:38.671761+0000","last_became_active":"2024-04-26T01:47:27.124862+0000","last_became_peered":"2024-04-26T01:47:27.124862+0000","last_unstale":"2024-04-26T01:51:38.671761+0000","last_undegraded":"2024-04-26T01:51:38.671761+0000","last_fullsized":"2024-04-26T01:51:38.671761+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:48.654711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.227404+0000","last_change":"2024-04-26T01:47:27.127365+0000","last_active":"2024-04-26T01:51:39.227404+0000","last_peered":"2024-04-26T01:51:39.227404+0000","last_clean":"2024-04-26T01:51:39.227404+0000","last_became_active":"2024-04-26T01:47:27.126868+0000","last_became_peered":"2024-04-26T01:47:27.126868+0000","last_unstale":"2024-04-26T01:51:39.227404+0000","last_undegraded":"2024-04-26T01:51:39.227404+0000","last_fullsized":"2024-04-26T01:51:39.227404+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:07:46.400333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.226295+0000","last_change":"2024-04-26T01:47:27.135584+0000","last_active":"2024-04-26T01:51:39.226295+0000","last_peered":"2024-04-26T01:51:39.226295+0000","last_clean":"2024-04-26T01:51:39.226295+0000","last_became_active":"2024-04-26T01:47:27.134849+0000","last_became_peered":"2024-04-26T01:47:27.134849+0000","last_unstale":"2024-04-26T01:51:39.226295+0000","last_undegraded":"2024-04-26T01:51:39.226295+0000","last_fullsized":"2024-04-26T01:51:39.226295+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:57:48.821595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.980000+0000","last_change":"2024-04-26T01:47:27.136182+0000","last_active":"2024-04-26T01:51:38.980000+0000","last_peered":"2024-04-26T01:51:38.980000+0000","last_clean":"2024-04-26T01:51:38.980000+0000","last_became_active":"2024-04-26T01:47:27.135787+0000","last_became_peered":"2024-04-26T01:47:27.135787+0000","last_unstale":"2024-04-26T01:51:38.980000+0000","last_undegraded":"2024-04-26T01:51:38.980000+0000","last_fullsized":"2024-04-26T01:51:38.980000+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:38:14.275898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581398+0000","last_change":"2024-04-26T01:47:27.140700+0000","last_active":"2024-04-26T01:51:39.581398+0000","last_peered":"2024-04-26T01:51:39.581398+0000","last_clean":"2024-04-26T01:51:39.581398+0000","last_became_active":"2024-04-26T01:47:27.140025+0000","last_became_peered":"2024-04-26T01:47:27.140025+0000","last_unstale":"2024-04-26T01:51:39.581398+0000","last_undegraded":"2024-04-26T01:51:39.581398+0000","last_fullsized":"2024-04-26T01:51:39.581398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:50.465236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.794694+0000","last_change":"2024-04-26T01:47:27.117256+0000","last_active":"2024-04-26T01:51:38.794694+0000","last_peered":"2024-04-26T01:51:38.794694+0000","last_clean":"2024-04-26T01:51:38.794694+0000","last_became_active":"2024-04-26T01:47:27.116825+0000","last_became_peered":"2024-04-26T01:47:27.116825+0000","last_unstale":"2024-04-26T01:51:38.794694+0000","last_undegraded":"2024-04-26T01:51:38.794694+0000","last_fullsized":"2024-04-26T01:51:38.794694+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:23.313712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.977034+0000","last_change":"2024-04-26T01:47:27.130233+0000","last_active":"2024-04-26T01:51:38.977034+0000","last_peered":"2024-04-26T01:51:38.977034+0000","last_clean":"2024-04-26T01:51:38.977034+0000","last_became_active":"2024-04-26T01:47:27.129852+0000","last_became_peered":"2024-04-26T01:47:27.129852+0000","last_unstale":"2024-04-26T01:51:38.977034+0000","last_undegraded":"2024-04-26T01:51:38.977034+0000","last_fullsized":"2024-04-26T01:51:38.977034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:49:01.407264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.794816+0000","last_change":"2024-04-26T01:47:23.068477+0000","last_active":"2024-04-26T01:51:38.794816+0000","last_peered":"2024-04-26T01:51:38.794816+0000","last_clean":"2024-04-26T01:51:38.794816+0000","last_became_active":"2024-04-26T01:47:23.068061+0000","last_became_peered":"2024-04-26T01:47:23.068061+0000","last_unstale":"2024-04-26T01:51:38.794816+0000","last_undegraded":"2024-04-26T01:51:38.794816+0000","last_fullsized":"2024-04-26T01:51:38.794816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:02.153724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.738398+0000","last_change":"2024-04-26T01:47:27.130385+0000","last_active":"2024-04-26T01:51:38.738398+0000","last_peered":"2024-04-26T01:51:38.738398+0000","last_clean":"2024-04-26T01:51:38.738398+0000","last_became_active":"2024-04-26T01:47:27.129912+0000","last_became_peered":"2024-04-26T01:47:27.129912+0000","last_unstale":"2024-04-26T01:51:38.738398+0000","last_undegraded":"2024-04-26T01:51:38.738398+0000","last_fullsized":"2024-04-26T01:51:38.738398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:26:03.010544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.738555+0000","last_change":"2024-04-26T01:47:23.110806+0000","last_active":"2024-04-26T01:51:38.738555+0000","last_peered":"2024-04-26T01:51:38.738555+0000","last_clean":"2024-04-26T01:51:38.738555+0000","last_became_active":"2024-04-26T01:47:23.110441+0000","last_became_peered":"2024-04-26T01:47:23.110441+0000","last_unstale":"2024-04-26T01:51:38.738555+0000","last_undegraded":"2024-04-26T01:51:38.738555+0000","last_fullsized":"2024-04-26T01:51:38.738555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:39.239172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.227010+0000","last_change":"2024-04-26T01:47:27.138628+0000","last_active":"2024-04-26T01:51:39.227010+0000","last_peered":"2024-04-26T01:51:39.227010+0000","last_clean":"2024-04-26T01:51:39.227010+0000","last_became_active":"2024-04-26T01:47:27.130483+0000","last_became_peered":"2024-04-26T01:47:27.130483+0000","last_unstale":"2024-04-26T01:51:39.227010+0000","last_undegraded":"2024-04-26T01:51:39.227010+0000","last_fullsized":"2024-04-26T01:51:39.227010+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:40:41.242856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.760376+0000","last_change":"2024-04-26T01:47:23.060066+0000","last_active":"2024-04-26T01:51:38.760376+0000","last_peered":"2024-04-26T01:51:38.760376+0000","last_clean":"2024-04-26T01:51:38.760376+0000","last_became_active":"2024-04-26T01:47:23.059526+0000","last_became_peered":"2024-04-26T01:47:23.059526+0000","last_unstale":"2024-04-26T01:51:38.760376+0000","last_undegraded":"2024-04-26T01:51:38.760376+0000","last_fullsized":"2024-04-26T01:51:38.760376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:28:22.623695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.980142+0000","last_change":"2024-04-26T01:47:27.128343+0000","last_active":"2024-04-26T01:51:38.980142+0000","last_peered":"2024-04-26T01:51:38.980142+0000","last_clean":"2024-04-26T01:51:38.980142+0000","last_became_active":"2024-04-26T01:47:27.127920+0000","last_became_peered":"2024-04-26T01:47:27.127920+0000","last_unstale":"2024-04-26T01:51:38.980142+0000","last_undegraded":"2024-04-26T01:51:38.980142+0000","last_fullsized":"2024-04-26T01:51:38.980142+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:06.725033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.760152+0000","last_change":"2024-04-26T01:47:23.057783+0000","last_active":"2024-04-26T01:51:38.760152+0000","last_peered":"2024-04-26T01:51:38.760152+0000","last_clean":"2024-04-26T01:51:38.760152+0000","last_became_active":"2024-04-26T01:47:23.057391+0000","last_became_peered":"2024-04-26T01:47:23.057391+0000","last_unstale":"2024-04-26T01:51:38.760152+0000","last_undegraded":"2024-04-26T01:51:38.760152+0000","last_fullsized":"2024-04-26T01:51:38.760152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:10:10.050787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672824+0000","last_change":"2024-04-26T01:47:27.116458+0000","last_active":"2024-04-26T01:51:38.672824+0000","last_peered":"2024-04-26T01:51:38.672824+0000","last_clean":"2024-04-26T01:51:38.672824+0000","last_became_active":"2024-04-26T01:47:27.116084+0000","last_became_peered":"2024-04-26T01:47:27.116084+0000","last_unstale":"2024-04-26T01:51:38.672824+0000","last_undegraded":"2024-04-26T01:51:38.672824+0000","last_fullsized":"2024-04-26T01:51:38.672824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:06:34.517478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"93'6","reported_seq":268,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.602959+0000","last_change":"2024-04-26T01:47:23.060657+0000","last_active":"2024-04-26T01:51:39.602959+0000","last_peered":"2024-04-26T01:51:39.602959+0000","last_clean":"2024-04-26T01:51:39.602959+0000","last_became_active":"2024-04-26T01:47:23.060294+0000","last_became_peered":"2024-04-26T01:47:23.060294+0000","last_unstale":"2024-04-26T01:51:39.602959+0000","last_undegraded":"2024-04-26T01:51:39.602959+0000","last_fullsized":"2024-04-26T01:51:39.602959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:33.979717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":233,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.759940+0000","last_change":"2024-04-26T01:47:27.120587+0000","last_active":"2024-04-26T01:51:38.759940+0000","last_peered":"2024-04-26T01:51:38.759940+0000","last_clean":"2024-04-26T01:51:38.759940+0000","last_became_active":"2024-04-26T01:47:27.120236+0000","last_became_peered":"2024-04-26T01:47:27.120236+0000","last_unstale":"2024-04-26T01:51:38.759940+0000","last_undegraded":"2024-04-26T01:51:38.759940+0000","last_fullsized":"2024-04-26T01:51:38.759940+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:30:38.488741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672250+0000","last_change":"2024-04-26T01:47:23.069395+0000","last_active":"2024-04-26T01:51:38.672250+0000","last_peered":"2024-04-26T01:51:38.672250+0000","last_clean":"2024-04-26T01:51:38.672250+0000","last_became_active":"2024-04-26T01:47:23.069102+0000","last_became_peered":"2024-04-26T01:47:23.069102+0000","last_unstale":"2024-04-26T01:51:38.672250+0000","last_undegraded":"2024-04-26T01:51:38.672250+0000","last_fullsized":"2024-04-26T01:51:38.672250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:45.162604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672487+0000","last_change":"2024-04-26T01:47:27.123859+0000","last_active":"2024-04-26T01:51:38.672487+0000","last_peered":"2024-04-26T01:51:38.672487+0000","last_clean":"2024-04-26T01:51:38.672487+0000","last_became_active":"2024-04-26T01:47:27.123438+0000","last_became_peered":"2024-04-26T01:47:27.123438+0000","last_unstale":"2024-04-26T01:51:38.672487+0000","last_undegraded":"2024-04-26T01:51:38.672487+0000","last_fullsized":"2024-04-26T01:51:38.672487+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:33:53.542809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.671891+0000","last_change":"2024-04-26T01:47:23.071544+0000","last_active":"2024-04-26T01:51:38.671891+0000","last_peered":"2024-04-26T01:51:38.671891+0000","last_clean":"2024-04-26T01:51:38.671891+0000","last_became_active":"2024-04-26T01:47:23.071191+0000","last_became_peered":"2024-04-26T01:47:23.071191+0000","last_unstale":"2024-04-26T01:51:38.671891+0000","last_undegraded":"2024-04-26T01:51:38.671891+0000","last_fullsized":"2024-04-26T01:51:38.671891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:58.889561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737808+0000","last_change":"2024-04-26T01:47:27.131887+0000","last_active":"2024-04-26T01:51:38.737808+0000","last_peered":"2024-04-26T01:51:38.737808+0000","last_clean":"2024-04-26T01:51:38.737808+0000","last_became_active":"2024-04-26T01:47:27.127973+0000","last_became_peered":"2024-04-26T01:47:27.127973+0000","last_unstale":"2024-04-26T01:51:38.737808+0000","last_undegraded":"2024-04-26T01:51:38.737808+0000","last_fullsized":"2024-04-26T01:51:38.737808+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:54:35.305779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672192+0000","last_change":"2024-04-26T01:47:23.061353+0000","last_active":"2024-04-26T01:51:38.672192+0000","last_peered":"2024-04-26T01:51:38.672192+0000","last_clean":"2024-04-26T01:51:38.672192+0000","last_became_active":"2024-04-26T01:47:23.061037+0000","last_became_peered":"2024-04-26T01:47:23.061037+0000","last_unstale":"2024-04-26T01:51:38.672192+0000","last_undegraded":"2024-04-26T01:51:38.672192+0000","last_fullsized":"2024-04-26T01:51:38.672192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:13:59.617969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.794335+0000","last_change":"2024-04-26T01:47:27.122303+0000","last_active":"2024-04-26T01:51:38.794335+0000","last_peered":"2024-04-26T01:51:38.794335+0000","last_clean":"2024-04-26T01:51:38.794335+0000","last_became_active":"2024-04-26T01:47:27.121919+0000","last_became_peered":"2024-04-26T01:47:27.121919+0000","last_unstale":"2024-04-26T01:51:38.794335+0000","last_undegraded":"2024-04-26T01:51:38.794335+0000","last_fullsized":"2024-04-26T01:51:38.794335+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:21.676663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"93'4","reported_seq":263,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.976626+0000","last_change":"2024-04-26T01:47:23.057961+0000","last_active":"2024-04-26T01:51:38.976626+0000","last_peered":"2024-04-26T01:51:38.976626+0000","last_clean":"2024-04-26T01:51:38.976626+0000","last_became_active":"2024-04-26T01:47:23.057529+0000","last_became_peered":"2024-04-26T01:47:23.057529+0000","last_unstale":"2024-04-26T01:51:38.976626+0000","last_undegraded":"2024-04-26T01:51:38.976626+0000","last_fullsized":"2024-04-26T01:51:38.976626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:14.420265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":230,"num_read_kb":230,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.980525+0000","last_change":"2024-04-26T01:47:27.120083+0000","last_active":"2024-04-26T01:51:38.980525+0000","last_peered":"2024-04-26T01:51:38.980525+0000","last_clean":"2024-04-26T01:51:38.980525+0000","last_became_active":"2024-04-26T01:47:27.118410+0000","last_became_peered":"2024-04-26T01:47:27.118410+0000","last_unstale":"2024-04-26T01:51:38.980525+0000","last_undegraded":"2024-04-26T01:51:38.980525+0000","last_fullsized":"2024-04-26T01:51:38.980525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:28.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973354+0000","last_change":"2024-04-26T01:47:23.059221+0000","last_active":"2024-04-26T01:51:38.973354+0000","last_peered":"2024-04-26T01:51:38.973354+0000","last_clean":"2024-04-26T01:51:38.973354+0000","last_became_active":"2024-04-26T01:47:23.058881+0000","last_became_peered":"2024-04-26T01:47:23.058881+0000","last_unstale":"2024-04-26T01:51:38.973354+0000","last_undegraded":"2024-04-26T01:51:38.973354+0000","last_fullsized":"2024-04-26T01:51:38.973354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:30.433696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.226796+0000","last_change":"2024-04-26T01:47:27.136763+0000","last_active":"2024-04-26T01:51:39.226796+0000","last_peered":"2024-04-26T01:51:39.226796+0000","last_clean":"2024-04-26T01:51:39.226796+0000","last_became_active":"2024-04-26T01:47:27.133642+0000","last_became_peered":"2024-04-26T01:47:27.133642+0000","last_unstale":"2024-04-26T01:51:39.226796+0000","last_undegraded":"2024-04-26T01:51:39.226796+0000","last_fullsized":"2024-04-26T01:51:39.226796+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:05.901139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672938+0000","last_change":"2024-04-26T01:47:23.108883+0000","last_active":"2024-04-26T01:51:38.672938+0000","last_peered":"2024-04-26T01:51:38.672938+0000","last_clean":"2024-04-26T01:51:38.672938+0000","last_became_active":"2024-04-26T01:47:23.108499+0000","last_became_peered":"2024-04-26T01:47:23.108499+0000","last_unstale":"2024-04-26T01:51:38.672938+0000","last_undegraded":"2024-04-26T01:51:38.672938+0000","last_fullsized":"2024-04-26T01:51:38.672938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:14:17.555416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.980570+0000","last_change":"2024-04-26T01:48:22.410651+0000","last_active":"2024-04-26T01:51:38.980570+0000","last_peered":"2024-04-26T01:51:38.980570+0000","last_clean":"2024-04-26T01:51:38.980570+0000","last_became_active":"2024-04-26T01:48:22.410359+0000","last_became_peered":"2024-04-26T01:48:22.410359+0000","last_unstale":"2024-04-26T01:51:38.980570+0000","last_undegraded":"2024-04-26T01:51:38.980570+0000","last_fullsized":"2024-04-26T01:51:38.980570+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:53:20.686915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"93'36","reported_seq":537,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.615379+0000","last_change":"2024-04-26T01:47:23.054893+0000","last_active":"2024-04-26T01:51:39.615379+0000","last_peered":"2024-04-26T01:51:39.615379+0000","last_clean":"2024-04-26T01:51:39.615379+0000","last_became_active":"2024-04-26T01:47:23.054558+0000","last_became_peered":"2024-04-26T01:47:23.054558+0000","last_unstale":"2024-04-26T01:51:39.615379+0000","last_undegraded":"2024-04-26T01:51:39.615379+0000","last_fullsized":"2024-04-26T01:51:39.615379+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:59.356841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67061,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":472,"num_read_kb":302,"num_write":37,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.760179+0000","last_change":"2024-04-26T01:47:27.116934+0000","last_active":"2024-04-26T01:51:38.760179+0000","last_peered":"2024-04-26T01:51:38.760179+0000","last_clean":"2024-04-26T01:51:38.760179+0000","last_became_active":"2024-04-26T01:47:27.116527+0000","last_became_peered":"2024-04-26T01:47:27.116527+0000","last_unstale":"2024-04-26T01:51:38.760179+0000","last_undegraded":"2024-04-26T01:51:38.760179+0000","last_fullsized":"2024-04-26T01:51:38.760179+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:09:45.820533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.976702+0000","last_change":"2024-04-26T01:47:23.059032+0000","last_active":"2024-04-26T01:51:38.976702+0000","last_peered":"2024-04-26T01:51:38.976702+0000","last_clean":"2024-04-26T01:51:38.976702+0000","last_became_active":"2024-04-26T01:47:23.057992+0000","last_became_peered":"2024-04-26T01:47:23.057992+0000","last_unstale":"2024-04-26T01:51:38.976702+0000","last_undegraded":"2024-04-26T01:51:38.976702+0000","last_fullsized":"2024-04-26T01:51:38.976702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:46:53.164925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.673476+0000","last_change":"2024-04-26T01:47:27.134574+0000","last_active":"2024-04-26T01:51:38.673476+0000","last_peered":"2024-04-26T01:51:38.673476+0000","last_clean":"2024-04-26T01:51:38.673476+0000","last_became_active":"2024-04-26T01:47:27.134184+0000","last_became_peered":"2024-04-26T01:47:27.134184+0000","last_unstale":"2024-04-26T01:51:38.673476+0000","last_undegraded":"2024-04-26T01:51:38.673476+0000","last_fullsized":"2024-04-26T01:51:38.673476+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:42:26.428014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672306+0000","last_change":"2024-04-26T01:47:23.062649+0000","last_active":"2024-04-26T01:51:38.672306+0000","last_peered":"2024-04-26T01:51:38.672306+0000","last_clean":"2024-04-26T01:51:38.672306+0000","last_became_active":"2024-04-26T01:47:23.062282+0000","last_became_peered":"2024-04-26T01:47:23.062282+0000","last_unstale":"2024-04-26T01:51:38.672306+0000","last_undegraded":"2024-04-26T01:51:38.672306+0000","last_fullsized":"2024-04-26T01:51:38.672306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:38.699105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973219+0000","last_change":"2024-04-26T01:47:27.124720+0000","last_active":"2024-04-26T01:51:38.973219+0000","last_peered":"2024-04-26T01:51:38.973219+0000","last_clean":"2024-04-26T01:51:38.973219+0000","last_became_active":"2024-04-26T01:47:27.124378+0000","last_became_peered":"2024-04-26T01:47:27.124378+0000","last_unstale":"2024-04-26T01:51:38.973219+0000","last_undegraded":"2024-04-26T01:51:38.973219+0000","last_fullsized":"2024-04-26T01:51:38.973219+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:45.462741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"93'8","reported_seq":271,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973319+0000","last_change":"2024-04-26T01:47:23.056305+0000","last_active":"2024-04-26T01:51:38.973319+0000","last_peered":"2024-04-26T01:51:38.973319+0000","last_clean":"2024-04-26T01:51:38.973319+0000","last_became_active":"2024-04-26T01:47:23.055905+0000","last_became_peered":"2024-04-26T01:47:23.055905+0000","last_unstale":"2024-04-26T01:51:38.973319+0000","last_undegraded":"2024-04-26T01:51:38.973319+0000","last_fullsized":"2024-04-26T01:51:38.973319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:00:14.609673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2212,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":235,"num_read_kb":6,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.972618+0000","last_change":"2024-04-26T01:47:27.119880+0000","last_active":"2024-04-26T01:51:38.972618+0000","last_peered":"2024-04-26T01:51:38.972618+0000","last_clean":"2024-04-26T01:51:38.972618+0000","last_became_active":"2024-04-26T01:47:27.119483+0000","last_became_peered":"2024-04-26T01:47:27.119483+0000","last_unstale":"2024-04-26T01:51:38.972618+0000","last_undegraded":"2024-04-26T01:51:38.972618+0000","last_fullsized":"2024-04-26T01:51:38.972618+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:15.523150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":69,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.759939+0000","last_change":"2024-04-26T01:48:22.758774+0000","last_active":"2024-04-26T01:51:38.759939+0000","last_peered":"2024-04-26T01:51:38.759939+0000","last_clean":"2024-04-26T01:51:38.759939+0000","last_became_active":"2024-04-26T01:48:22.747354+0000","last_became_peered":"2024-04-26T01:48:22.747354+0000","last_unstale":"2024-04-26T01:51:38.759939+0000","last_undegraded":"2024-04-26T01:51:38.759939+0000","last_fullsized":"2024-04-26T01:51:38.759939+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:23:15.886774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672382+0000","last_change":"2024-04-26T01:47:27.113508+0000","last_active":"2024-04-26T01:51:38.672382+0000","last_peered":"2024-04-26T01:51:38.672382+0000","last_clean":"2024-04-26T01:51:38.672382+0000","last_became_active":"2024-04-26T01:47:27.113066+0000","last_became_peered":"2024-04-26T01:47:27.113066+0000","last_unstale":"2024-04-26T01:51:38.672382+0000","last_undegraded":"2024-04-26T01:51:38.672382+0000","last_fullsized":"2024-04-26T01:51:38.672382+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:37.525486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"93'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.760368+0000","last_change":"2024-04-26T01:47:23.059644+0000","last_active":"2024-04-26T01:51:38.760368+0000","last_peered":"2024-04-26T01:51:38.760368+0000","last_clean":"2024-04-26T01:51:38.760368+0000","last_became_active":"2024-04-26T01:47:23.058966+0000","last_became_peered":"2024-04-26T01:47:23.058966+0000","last_unstale":"2024-04-26T01:51:38.760368+0000","last_undegraded":"2024-04-26T01:51:38.760368+0000","last_fullsized":"2024-04-26T01:51:38.760368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:57:36.475935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581235+0000","last_change":"2024-04-26T01:48:22.749354+0000","last_active":"2024-04-26T01:51:39.581235+0000","last_peered":"2024-04-26T01:51:39.581235+0000","last_clean":"2024-04-26T01:51:39.581235+0000","last_became_active":"2024-04-26T01:48:22.748978+0000","last_became_peered":"2024-04-26T01:48:22.748978+0000","last_unstale":"2024-04-26T01:51:39.581235+0000","last_undegraded":"2024-04-26T01:51:39.581235+0000","last_fullsized":"2024-04-26T01:51:39.581235+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:51:16.087091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.794533+0000","last_change":"2024-04-26T01:47:23.110360+0000","last_active":"2024-04-26T01:51:38.794533+0000","last_peered":"2024-04-26T01:51:38.794533+0000","last_clean":"2024-04-26T01:51:38.794533+0000","last_became_active":"2024-04-26T01:47:23.109744+0000","last_became_peered":"2024-04-26T01:47:23.109744+0000","last_unstale":"2024-04-26T01:51:38.794533+0000","last_undegraded":"2024-04-26T01:51:38.794533+0000","last_fullsized":"2024-04-26T01:51:38.794533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:40:13.658396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672730+0000","last_change":"2024-04-26T01:47:27.125399+0000","last_active":"2024-04-26T01:51:38.672730+0000","last_peered":"2024-04-26T01:51:38.672730+0000","last_clean":"2024-04-26T01:51:38.672730+0000","last_became_active":"2024-04-26T01:47:27.124871+0000","last_became_peered":"2024-04-26T01:47:27.124871+0000","last_unstale":"2024-04-26T01:51:38.672730+0000","last_undegraded":"2024-04-26T01:51:38.672730+0000","last_fullsized":"2024-04-26T01:51:38.672730+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:28:48.285206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973098+0000","last_change":"2024-04-26T01:47:23.067862+0000","last_active":"2024-04-26T01:51:38.973098+0000","last_peered":"2024-04-26T01:51:38.973098+0000","last_clean":"2024-04-26T01:51:38.973098+0000","last_became_active":"2024-04-26T01:47:23.067526+0000","last_became_peered":"2024-04-26T01:47:23.067526+0000","last_unstale":"2024-04-26T01:51:38.973098+0000","last_undegraded":"2024-04-26T01:51:38.973098+0000","last_fullsized":"2024-04-26T01:51:38.973098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:22.668405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672467+0000","last_change":"2024-04-26T01:48:22.748548+0000","last_active":"2024-04-26T01:51:38.672467+0000","last_peered":"2024-04-26T01:51:38.672467+0000","last_clean":"2024-04-26T01:51:38.672467+0000","last_became_active":"2024-04-26T01:48:22.748182+0000","last_became_peered":"2024-04-26T01:48:22.748182+0000","last_unstale":"2024-04-26T01:51:38.672467+0000","last_undegraded":"2024-04-26T01:51:38.672467+0000","last_fullsized":"2024-04-26T01:51:38.672467+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:20:27.596827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672474+0000","last_change":"2024-04-26T01:47:27.118307+0000","last_active":"2024-04-26T01:51:38.672474+0000","last_peered":"2024-04-26T01:51:38.672474+0000","last_clean":"2024-04-26T01:51:38.672474+0000","last_became_active":"2024-04-26T01:47:27.117986+0000","last_became_peered":"2024-04-26T01:47:27.117986+0000","last_unstale":"2024-04-26T01:51:38.672474+0000","last_undegraded":"2024-04-26T01:51:38.672474+0000","last_fullsized":"2024-04-26T01:51:38.672474+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:40.667143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":265,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.735079+0000","last_change":"2024-04-26T01:47:23.109973+0000","last_active":"2024-04-26T01:51:38.735079+0000","last_peered":"2024-04-26T01:51:38.735079+0000","last_clean":"2024-04-26T01:51:38.735079+0000","last_became_active":"2024-04-26T01:47:23.109446+0000","last_became_peered":"2024-04-26T01:47:23.109446+0000","last_unstale":"2024-04-26T01:51:38.735079+0000","last_undegraded":"2024-04-26T01:51:38.735079+0000","last_fullsized":"2024-04-26T01:51:38.735079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:49:00.684164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":229,"num_read_kb":229,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.738353+0000","last_change":"2024-04-26T01:47:27.131502+0000","last_active":"2024-04-26T01:51:38.738353+0000","last_peered":"2024-04-26T01:51:38.738353+0000","last_clean":"2024-04-26T01:51:38.738353+0000","last_became_active":"2024-04-26T01:47:27.127697+0000","last_became_peered":"2024-04-26T01:47:27.127697+0000","last_unstale":"2024-04-26T01:51:38.738353+0000","last_undegraded":"2024-04-26T01:51:38.738353+0000","last_fullsized":"2024-04-26T01:51:38.738353+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:39:15.458189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737630+0000","last_change":"2024-04-26T01:47:23.066596+0000","last_active":"2024-04-26T01:51:38.737630+0000","last_peered":"2024-04-26T01:51:38.737630+0000","last_clean":"2024-04-26T01:51:38.737630+0000","last_became_active":"2024-04-26T01:47:23.066216+0000","last_became_peered":"2024-04-26T01:47:23.066216+0000","last_unstale":"2024-04-26T01:51:38.737630+0000","last_undegraded":"2024-04-26T01:51:38.737630+0000","last_fullsized":"2024-04-26T01:51:38.737630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:08:31.544831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":363,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.972458+0000","last_change":"2024-04-26T01:45:54.834501+0000","last_active":"2024-04-26T01:51:38.972458+0000","last_peered":"2024-04-26T01:51:38.972458+0000","last_clean":"2024-04-26T01:51:38.972458+0000","last_became_active":"2024-04-26T01:45:54.833868+0000","last_became_peered":"2024-04-26T01:45:54.833868+0000","last_unstale":"2024-04-26T01:51:38.972458+0000","last_undegraded":"2024-04-26T01:51:38.972458+0000","last_fullsized":"2024-04-26T01:51:38.972458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:42:05.228956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:42:05.228956+0000","last_clean_scrub_stamp":"2024-04-26T01:42:05.228956+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:06:58.383315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.671645+0000","last_change":"2024-04-26T01:47:27.126475+0000","last_active":"2024-04-26T01:51:38.671645+0000","last_peered":"2024-04-26T01:51:38.671645+0000","last_clean":"2024-04-26T01:51:38.671645+0000","last_became_active":"2024-04-26T01:47:27.123760+0000","last_became_peered":"2024-04-26T01:47:27.123760+0000","last_unstale":"2024-04-26T01:51:38.671645+0000","last_undegraded":"2024-04-26T01:51:38.671645+0000","last_fullsized":"2024-04-26T01:51:38.671645+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:38:27.473909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.972839+0000","last_change":"2024-04-26T01:47:23.059810+0000","last_active":"2024-04-26T01:51:38.972839+0000","last_peered":"2024-04-26T01:51:38.972839+0000","last_clean":"2024-04-26T01:51:38.972839+0000","last_became_active":"2024-04-26T01:47:23.059504+0000","last_became_peered":"2024-04-26T01:47:23.059504+0000","last_unstale":"2024-04-26T01:51:38.972839+0000","last_undegraded":"2024-04-26T01:51:38.972839+0000","last_fullsized":"2024-04-26T01:51:38.972839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:50:51.220951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.760303+0000","last_change":"2024-04-26T01:47:23.109460+0000","last_active":"2024-04-26T01:51:38.760303+0000","last_peered":"2024-04-26T01:51:38.760303+0000","last_clean":"2024-04-26T01:51:38.760303+0000","last_became_active":"2024-04-26T01:47:23.109090+0000","last_became_peered":"2024-04-26T01:47:23.109090+0000","last_unstale":"2024-04-26T01:51:38.760303+0000","last_undegraded":"2024-04-26T01:51:38.760303+0000","last_fullsized":"2024-04-26T01:51:38.760303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:56.821312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672774+0000","last_change":"2024-04-26T01:47:27.129537+0000","last_active":"2024-04-26T01:51:38.672774+0000","last_peered":"2024-04-26T01:51:38.672774+0000","last_clean":"2024-04-26T01:51:38.672774+0000","last_became_active":"2024-04-26T01:47:27.128124+0000","last_became_peered":"2024-04-26T01:47:27.128124+0000","last_unstale":"2024-04-26T01:51:38.672774+0000","last_undegraded":"2024-04-26T01:51:38.672774+0000","last_fullsized":"2024-04-26T01:51:38.672774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:14:26.262197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"93'47","reported_seq":99,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.926236+0000","last_change":"2024-04-26T01:47:23.064511+0000","last_active":"2024-04-26T01:51:38.926236+0000","last_peered":"2024-04-26T01:51:38.926236+0000","last_clean":"2024-04-26T01:51:38.926236+0000","last_became_active":"2024-04-26T01:47:23.064080+0000","last_became_peered":"2024-04-26T01:47:23.064080+0000","last_unstale":"2024-04-26T01:51:38.926236+0000","last_undegraded":"2024-04-26T01:51:38.926236+0000","last_fullsized":"2024-04-26T01:51:38.926236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:37.562644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.738619+0000","last_change":"2024-04-26T01:47:27.124057+0000","last_active":"2024-04-26T01:51:38.738619+0000","last_peered":"2024-04-26T01:51:38.738619+0000","last_clean":"2024-04-26T01:51:38.738619+0000","last_became_active":"2024-04-26T01:47:27.123636+0000","last_became_peered":"2024-04-26T01:47:27.123636+0000","last_unstale":"2024-04-26T01:51:38.738619+0000","last_undegraded":"2024-04-26T01:51:38.738619+0000","last_fullsized":"2024-04-26T01:51:38.738619+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:56:55.691697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737498+0000","last_change":"2024-04-26T01:47:23.136571+0000","last_active":"2024-04-26T01:51:38.737498+0000","last_peered":"2024-04-26T01:51:38.737498+0000","last_clean":"2024-04-26T01:51:38.737498+0000","last_became_active":"2024-04-26T01:47:23.136205+0000","last_became_peered":"2024-04-26T01:47:23.136205+0000","last_unstale":"2024-04-26T01:51:38.737498+0000","last_undegraded":"2024-04-26T01:51:38.737498+0000","last_fullsized":"2024-04-26T01:51:38.737498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:24.418797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.226818+0000","last_change":"2024-04-26T01:48:22.747570+0000","last_active":"2024-04-26T01:51:39.226818+0000","last_peered":"2024-04-26T01:51:39.226818+0000","last_clean":"2024-04-26T01:51:39.226818+0000","last_became_active":"2024-04-26T01:48:22.747147+0000","last_became_peered":"2024-04-26T01:48:22.747147+0000","last_unstale":"2024-04-26T01:51:39.226818+0000","last_undegraded":"2024-04-26T01:51:39.226818+0000","last_fullsized":"2024-04-26T01:51:39.226818+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:37.034388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.980000+0000","last_change":"2024-04-26T01:48:22.414610+0000","last_active":"2024-04-26T01:51:38.980000+0000","last_peered":"2024-04-26T01:51:38.980000+0000","last_clean":"2024-04-26T01:51:38.980000+0000","last_became_active":"2024-04-26T01:48:22.410275+0000","last_became_peered":"2024-04-26T01:48:22.410275+0000","last_unstale":"2024-04-26T01:51:38.980000+0000","last_undegraded":"2024-04-26T01:51:38.980000+0000","last_fullsized":"2024-04-26T01:51:38.980000+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:20:48.131988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.759917+0000","last_change":"2024-04-26T01:47:27.115498+0000","last_active":"2024-04-26T01:51:38.759917+0000","last_peered":"2024-04-26T01:51:38.759917+0000","last_clean":"2024-04-26T01:51:38.759917+0000","last_became_active":"2024-04-26T01:47:27.115134+0000","last_became_peered":"2024-04-26T01:47:27.115134+0000","last_unstale":"2024-04-26T01:51:38.759917+0000","last_undegraded":"2024-04-26T01:51:38.759917+0000","last_fullsized":"2024-04-26T01:51:38.759917+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:20.795224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737941+0000","last_change":"2024-04-26T01:47:23.065366+0000","last_active":"2024-04-26T01:51:38.737941+0000","last_peered":"2024-04-26T01:51:38.737941+0000","last_clean":"2024-04-26T01:51:38.737941+0000","last_became_active":"2024-04-26T01:47:23.063216+0000","last_became_peered":"2024-04-26T01:47:23.063216+0000","last_unstale":"2024-04-26T01:51:38.737941+0000","last_undegraded":"2024-04-26T01:51:38.737941+0000","last_fullsized":"2024-04-26T01:51:38.737941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:06:05.261907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.580975+0000","last_change":"2024-04-26T01:47:27.127938+0000","last_active":"2024-04-26T01:51:39.580975+0000","last_peered":"2024-04-26T01:51:39.580975+0000","last_clean":"2024-04-26T01:51:39.580975+0000","last_became_active":"2024-04-26T01:47:27.127553+0000","last_became_peered":"2024-04-26T01:47:27.127553+0000","last_unstale":"2024-04-26T01:51:39.580975+0000","last_undegraded":"2024-04-26T01:51:39.580975+0000","last_fullsized":"2024-04-26T01:51:39.580975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:17:51.115349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"93'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.976687+0000","last_change":"2024-04-26T01:47:23.060475+0000","last_active":"2024-04-26T01:51:38.976687+0000","last_peered":"2024-04-26T01:51:38.976687+0000","last_clean":"2024-04-26T01:51:38.976687+0000","last_became_active":"2024-04-26T01:47:23.060098+0000","last_became_peered":"2024-04-26T01:47:23.060098+0000","last_unstale":"2024-04-26T01:51:38.976687+0000","last_undegraded":"2024-04-26T01:51:38.976687+0000","last_fullsized":"2024-04-26T01:51:38.976687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:49.009455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.738288+0000","last_change":"2024-04-26T01:47:27.137491+0000","last_active":"2024-04-26T01:51:38.738288+0000","last_peered":"2024-04-26T01:51:38.738288+0000","last_clean":"2024-04-26T01:51:38.738288+0000","last_became_active":"2024-04-26T01:47:27.137059+0000","last_became_peered":"2024-04-26T01:47:27.137059+0000","last_unstale":"2024-04-26T01:51:38.738288+0000","last_undegraded":"2024-04-26T01:51:38.738288+0000","last_fullsized":"2024-04-26T01:51:38.738288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:20:25.004074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581004+0000","last_change":"2024-04-26T01:48:22.405071+0000","last_active":"2024-04-26T01:51:39.581004+0000","last_peered":"2024-04-26T01:51:39.581004+0000","last_clean":"2024-04-26T01:51:39.581004+0000","last_became_active":"2024-04-26T01:48:22.401282+0000","last_became_peered":"2024-04-26T01:48:22.401282+0000","last_unstale":"2024-04-26T01:51:39.581004+0000","last_undegraded":"2024-04-26T01:51:39.581004+0000","last_fullsized":"2024-04-26T01:51:39.581004+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:24:53.628014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973029+0000","last_change":"2024-04-26T01:47:27.128253+0000","last_active":"2024-04-26T01:51:38.973029+0000","last_peered":"2024-04-26T01:51:38.973029+0000","last_clean":"2024-04-26T01:51:38.973029+0000","last_became_active":"2024-04-26T01:47:27.127869+0000","last_became_peered":"2024-04-26T01:47:27.127869+0000","last_unstale":"2024-04-26T01:51:38.973029+0000","last_undegraded":"2024-04-26T01:51:38.973029+0000","last_fullsized":"2024-04-26T01:51:38.973029+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:11:23.476938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.976946+0000","last_change":"2024-04-26T01:47:23.136862+0000","last_active":"2024-04-26T01:51:38.976946+0000","last_peered":"2024-04-26T01:51:38.976946+0000","last_clean":"2024-04-26T01:51:38.976946+0000","last_became_active":"2024-04-26T01:47:23.136380+0000","last_became_peered":"2024-04-26T01:47:23.136380+0000","last_unstale":"2024-04-26T01:51:38.976946+0000","last_undegraded":"2024-04-26T01:51:38.976946+0000","last_fullsized":"2024-04-26T01:51:38.976946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:39:18.103559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973500+0000","last_change":"2024-04-26T01:47:27.113363+0000","last_active":"2024-04-26T01:51:38.973500+0000","last_peered":"2024-04-26T01:51:38.973500+0000","last_clean":"2024-04-26T01:51:38.973500+0000","last_became_active":"2024-04-26T01:47:27.113040+0000","last_became_peered":"2024-04-26T01:47:27.113040+0000","last_unstale":"2024-04-26T01:51:38.973500+0000","last_undegraded":"2024-04-26T01:51:38.973500+0000","last_fullsized":"2024-04-26T01:51:38.973500+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:42:16.428697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'39","reported_seq":76,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.923549+0000","last_change":"2024-04-26T01:47:23.067257+0000","last_active":"2024-04-26T01:51:38.923549+0000","last_peered":"2024-04-26T01:51:38.923549+0000","last_clean":"2024-04-26T01:51:38.923549+0000","last_became_active":"2024-04-26T01:47:23.066858+0000","last_became_peered":"2024-04-26T01:47:23.066858+0000","last_unstale":"2024-04-26T01:51:38.923549+0000","last_undegraded":"2024-04-26T01:51:38.923549+0000","last_fullsized":"2024-04-26T01:51:38.923549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:22:07.848074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973271+0000","last_change":"2024-04-26T01:47:27.125885+0000","last_active":"2024-04-26T01:51:38.973271+0000","last_peered":"2024-04-26T01:51:38.973271+0000","last_clean":"2024-04-26T01:51:38.973271+0000","last_became_active":"2024-04-26T01:47:27.125524+0000","last_became_peered":"2024-04-26T01:47:27.125524+0000","last_unstale":"2024-04-26T01:51:38.973271+0000","last_undegraded":"2024-04-26T01:51:38.973271+0000","last_fullsized":"2024-04-26T01:51:38.973271+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:39.282023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672262+0000","last_change":"2024-04-26T01:47:23.071051+0000","last_active":"2024-04-26T01:51:38.672262+0000","last_peered":"2024-04-26T01:51:38.672262+0000","last_clean":"2024-04-26T01:51:38.672262+0000","last_became_active":"2024-04-26T01:47:23.069524+0000","last_became_peered":"2024-04-26T01:47:23.069524+0000","last_unstale":"2024-04-26T01:51:38.672262+0000","last_undegraded":"2024-04-26T01:51:38.672262+0000","last_fullsized":"2024-04-26T01:51:38.672262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:24:57.643100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973498+0000","last_change":"2024-04-26T01:47:27.114325+0000","last_active":"2024-04-26T01:51:38.973498+0000","last_peered":"2024-04-26T01:51:38.973498+0000","last_clean":"2024-04-26T01:51:38.973498+0000","last_became_active":"2024-04-26T01:47:27.113952+0000","last_became_peered":"2024-04-26T01:47:27.113952+0000","last_unstale":"2024-04-26T01:51:38.973498+0000","last_undegraded":"2024-04-26T01:51:38.973498+0000","last_fullsized":"2024-04-26T01:51:38.973498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:39:19.167175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581077+0000","last_change":"2024-04-26T01:47:27.118709+0000","last_active":"2024-04-26T01:51:39.581077+0000","last_peered":"2024-04-26T01:51:39.581077+0000","last_clean":"2024-04-26T01:51:39.581077+0000","last_became_active":"2024-04-26T01:47:27.118341+0000","last_became_peered":"2024-04-26T01:47:27.118341+0000","last_unstale":"2024-04-26T01:51:39.581077+0000","last_undegraded":"2024-04-26T01:51:39.581077+0000","last_fullsized":"2024-04-26T01:51:39.581077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:13.736180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.976759+0000","last_change":"2024-04-26T01:47:27.119239+0000","last_active":"2024-04-26T01:51:38.976759+0000","last_peered":"2024-04-26T01:51:38.976759+0000","last_clean":"2024-04-26T01:51:38.976759+0000","last_became_active":"2024-04-26T01:47:27.118877+0000","last_became_peered":"2024-04-26T01:47:27.118877+0000","last_unstale":"2024-04-26T01:51:38.976759+0000","last_undegraded":"2024-04-26T01:51:38.976759+0000","last_fullsized":"2024-04-26T01:51:38.976759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:04:10.033517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.976632+0000","last_change":"2024-04-26T01:47:27.142848+0000","last_active":"2024-04-26T01:51:38.976632+0000","last_peered":"2024-04-26T01:51:38.976632+0000","last_clean":"2024-04-26T01:51:38.976632+0000","last_became_active":"2024-04-26T01:47:27.142580+0000","last_became_peered":"2024-04-26T01:47:27.142580+0000","last_unstale":"2024-04-26T01:51:38.976632+0000","last_undegraded":"2024-04-26T01:51:38.976632+0000","last_fullsized":"2024-04-26T01:51:38.976632+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:15:46.923922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.227735+0000","last_change":"2024-04-26T01:47:27.134817+0000","last_active":"2024-04-26T01:51:39.227735+0000","last_peered":"2024-04-26T01:51:39.227735+0000","last_clean":"2024-04-26T01:51:39.227735+0000","last_became_active":"2024-04-26T01:47:27.134024+0000","last_became_peered":"2024-04-26T01:47:27.134024+0000","last_unstale":"2024-04-26T01:51:39.227735+0000","last_undegraded":"2024-04-26T01:51:39.227735+0000","last_fullsized":"2024-04-26T01:51:39.227735+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:56.746427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672094+0000","last_change":"2024-04-26T01:47:27.143137+0000","last_active":"2024-04-26T01:51:38.672094+0000","last_peered":"2024-04-26T01:51:38.672094+0000","last_clean":"2024-04-26T01:51:38.672094+0000","last_became_active":"2024-04-26T01:47:27.142625+0000","last_became_peered":"2024-04-26T01:47:27.142625+0000","last_unstale":"2024-04-26T01:51:38.672094+0000","last_undegraded":"2024-04-26T01:51:38.672094+0000","last_fullsized":"2024-04-26T01:51:38.672094+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:42:50.468382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581192+0000","last_change":"2024-04-26T01:48:22.749582+0000","last_active":"2024-04-26T01:51:39.581192+0000","last_peered":"2024-04-26T01:51:39.581192+0000","last_clean":"2024-04-26T01:51:39.581192+0000","last_became_active":"2024-04-26T01:48:22.749191+0000","last_became_peered":"2024-04-26T01:48:22.749191+0000","last_unstale":"2024-04-26T01:51:39.581192+0000","last_undegraded":"2024-04-26T01:51:39.581192+0000","last_fullsized":"2024-04-26T01:51:39.581192+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:14.614842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.977078+0000","last_change":"2024-04-26T01:47:27.123964+0000","last_active":"2024-04-26T01:51:38.977078+0000","last_peered":"2024-04-26T01:51:38.977078+0000","last_clean":"2024-04-26T01:51:38.977078+0000","last_became_active":"2024-04-26T01:47:27.123552+0000","last_became_peered":"2024-04-26T01:47:27.123552+0000","last_unstale":"2024-04-26T01:51:38.977078+0000","last_undegraded":"2024-04-26T01:51:38.977078+0000","last_fullsized":"2024-04-26T01:51:38.977078+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:33:12.566105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.738482+0000","last_change":"2024-04-26T01:48:22.412621+0000","last_active":"2024-04-26T01:51:38.738482+0000","last_peered":"2024-04-26T01:51:38.738482+0000","last_clean":"2024-04-26T01:51:38.738482+0000","last_became_active":"2024-04-26T01:48:22.411430+0000","last_became_peered":"2024-04-26T01:48:22.411430+0000","last_unstale":"2024-04-26T01:51:38.738482+0000","last_undegraded":"2024-04-26T01:51:38.738482+0000","last_fullsized":"2024-04-26T01:51:38.738482+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:34.152338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672255+0000","last_change":"2024-04-26T01:47:27.131437+0000","last_active":"2024-04-26T01:51:38.672255+0000","last_peered":"2024-04-26T01:51:38.672255+0000","last_clean":"2024-04-26T01:51:38.672255+0000","last_became_active":"2024-04-26T01:47:27.131138+0000","last_became_peered":"2024-04-26T01:47:27.131138+0000","last_unstale":"2024-04-26T01:51:38.672255+0000","last_undegraded":"2024-04-26T01:51:38.672255+0000","last_fullsized":"2024-04-26T01:51:38.672255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:55:39.132972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737796+0000","last_change":"2024-04-26T01:47:27.120928+0000","last_active":"2024-04-26T01:51:38.737796+0000","last_peered":"2024-04-26T01:51:38.737796+0000","last_clean":"2024-04-26T01:51:38.737796+0000","last_became_active":"2024-04-26T01:47:27.120012+0000","last_became_peered":"2024-04-26T01:47:27.120012+0000","last_unstale":"2024-04-26T01:51:38.737796+0000","last_undegraded":"2024-04-26T01:51:38.737796+0000","last_fullsized":"2024-04-26T01:51:38.737796+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:19:44.209876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.980215+0000","last_change":"2024-04-26T01:47:27.138766+0000","last_active":"2024-04-26T01:51:38.980215+0000","last_peered":"2024-04-26T01:51:38.980215+0000","last_clean":"2024-04-26T01:51:38.980215+0000","last_became_active":"2024-04-26T01:47:27.138413+0000","last_became_peered":"2024-04-26T01:47:27.138413+0000","last_unstale":"2024-04-26T01:51:38.980215+0000","last_undegraded":"2024-04-26T01:51:38.980215+0000","last_fullsized":"2024-04-26T01:51:38.980215+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:51:13.117307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.671912+0000","last_change":"2024-04-26T01:47:27.141338+0000","last_active":"2024-04-26T01:51:38.671912+0000","last_peered":"2024-04-26T01:51:38.671912+0000","last_clean":"2024-04-26T01:51:38.671912+0000","last_became_active":"2024-04-26T01:47:27.140825+0000","last_became_peered":"2024-04-26T01:47:27.140825+0000","last_unstale":"2024-04-26T01:51:38.671912+0000","last_undegraded":"2024-04-26T01:51:38.671912+0000","last_fullsized":"2024-04-26T01:51:38.671912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:12:08.680876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581041+0000","last_change":"2024-04-26T01:47:27.114919+0000","last_active":"2024-04-26T01:51:39.581041+0000","last_peered":"2024-04-26T01:51:39.581041+0000","last_clean":"2024-04-26T01:51:39.581041+0000","last_became_active":"2024-04-26T01:47:27.114528+0000","last_became_peered":"2024-04-26T01:47:27.114528+0000","last_unstale":"2024-04-26T01:51:39.581041+0000","last_undegraded":"2024-04-26T01:51:39.581041+0000","last_fullsized":"2024-04-26T01:51:39.581041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:15:25.620566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597502,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1502,"num_read_kb":860,"num_write":294,"num_write_kb":1439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1792506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318337,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":490347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579974,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1724416,"data_stored":1152561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743500,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28424,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716728,"statfs":{"total":95995035648,"available":95965929472,"internally_reserved":0,"allocated":1437696,"data_stored":892784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005140,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36744,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708408,"statfs":{"total":95995035648,"available":95957409792,"internally_reserved":0,"allocated":1437696,"data_stored":892704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299479,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1638400,"data_stored":1093042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463005,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":498957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1101824,"data_stored":565811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738495,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":500228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1118208,"data_stored":567440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182510,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1024084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920872,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":501130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724645,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37080,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708072,"statfs":{"total":95995035648,"available":95957065728,"internally_reserved":0,"allocated":1785856,"data_stored":1222702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":201863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:51:45.791 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:51:46.274 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:46.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:46 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:46.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:46 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:46.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:46 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:46.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:46 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:46.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:46 smithi097 ceph-mon[32853]: Updating smithi001:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:51:46.758 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:46 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:46.758 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:46 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:46.758 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:46 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:46.758 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:46 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:46.758 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:46 smithi001 ceph-mon[27822]: Updating smithi001:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:51:46.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:46 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:46.816 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:46 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:46.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:46 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:46.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:46 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:46.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:46 smithi173 ceph-mon[32553]: Updating smithi001:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:51:48.260 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:48 smithi001 ceph-mon[27822]: Updating smithi001:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:51:48.260 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:48 smithi001 ceph-mon[27822]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 133 KiB/s wr, 77 op/s 2024-04-26T01:51:48.260 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:48 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:48.260 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:48 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:48.260 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:48 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:48.260 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:48 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:51:48.260 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:48 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:48.260 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:48 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:51:48.260 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:48 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:48.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:48 smithi173 ceph-mon[32553]: Updating smithi001:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:51:48.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:48 smithi173 ceph-mon[32553]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 133 KiB/s wr, 77 op/s 2024-04-26T01:51:48.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:48 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:48.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:48 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:48.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:48 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:48.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:48 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:51:48.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:48 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:48.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:48 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:51:48.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:48 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:48 smithi097 ceph-mon[32853]: Updating smithi001:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:51:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:48 smithi097 ceph-mon[32853]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 133 KiB/s wr, 77 op/s 2024-04-26T01:51:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:48 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:48 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:48.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:48 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:48.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:48 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:51:48.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:48 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:48.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:48 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:51:48.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:48 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:51:48.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.545+0000 7f3c18000700 1 -- 172.21.15.1:0/1538368108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c1006ac90 msgr2=0x7f3c1006b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:48.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.545+0000 7f3c18000700 1 --2- 172.21.15.1:0/1538368108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c1006ac90 0x7f3c1006b0f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009990 tx=0x7f3bfc009ca0 comp rx=0 tx=0).stop 2024-04-26T01:51:48.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.545+0000 7f3c18000700 1 -- 172.21.15.1:0/1538368108 shutdown_connections 2024-04-26T01:51:48.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.545+0000 7f3c18000700 1 --2- 172.21.15.1:0/1538368108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c10105480 0x7f3c10105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:48.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.545+0000 7f3c18000700 1 --2- 172.21.15.1:0/1538368108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c10107970 0x7f3c1006a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:48.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.545+0000 7f3c18000700 1 --2- 172.21.15.1:0/1538368108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c1006ac90 0x7f3c1006b0f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:48.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.545+0000 7f3c18000700 1 -- 172.21.15.1:0/1538368108 >> 172.21.15.1:0/1538368108 conn(0x7f3c100788f0 msgr2=0x7f3c10078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:48.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.545+0000 7f3c18000700 1 -- 172.21.15.1:0/1538368108 shutdown_connections 2024-04-26T01:51:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.545+0000 7f3c18000700 1 -- 172.21.15.1:0/1538368108 wait complete. 2024-04-26T01:51:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.546+0000 7f3c18000700 1 Processor -- start 2024-04-26T01:51:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.546+0000 7f3c18000700 1 -- start start 2024-04-26T01:51:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.547+0000 7f3c18000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c1006ac90 0x7f3c101a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.547+0000 7f3c18000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c10105480 0x7f3c101a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.547+0000 7f3c18000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c10107970 0x7f3c1019f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.547+0000 7f3c18000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3c10119210 con 0x7f3c1006ac90 2024-04-26T01:51:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.547+0000 7f3c18000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3c10119090 con 0x7f3c10105480 2024-04-26T01:51:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.547+0000 7f3c18000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3c10119390 con 0x7f3c10107970 2024-04-26T01:51:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.547+0000 7f3c15600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c1006ac90 0x7f3c101a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.547+0000 7f3c15600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c1006ac90 0x7f3c101a5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34470/0 (socket says 172.21.15.1:34470) 2024-04-26T01:51:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.547+0000 7f3c15600700 1 -- 172.21.15.1:0/4201790366 learned_addr learned my addr 172.21.15.1:0/4201790366 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.547+0000 7f3c16000700 1 --2- 172.21.15.1:0/4201790366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c10107970 0x7f3c1019f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.547+0000 7f3c14c00700 1 --2- 172.21.15.1:0/4201790366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c10105480 0x7f3c101a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:48.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.548+0000 7f3c15600700 1 -- 172.21.15.1:0/4201790366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c10107970 msgr2=0x7f3c1019f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:48.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.548+0000 7f3c15600700 1 --2- 172.21.15.1:0/4201790366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c10107970 0x7f3c1019f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:48.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.548+0000 7f3c15600700 1 -- 172.21.15.1:0/4201790366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c10105480 msgr2=0x7f3c101a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:48.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.548+0000 7f3c15600700 1 --2- 172.21.15.1:0/4201790366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c10105480 0x7f3c101a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:48.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.548+0000 7f3c15600700 1 -- 172.21.15.1:0/4201790366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1019fd30 con 0x7f3c1006ac90 2024-04-26T01:51:48.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.548+0000 7f3c15600700 1 --2- 172.21.15.1:0/4201790366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c1006ac90 0x7f3c101a5410 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400d950 tx=0x7f3c0400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:48.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.548+0000 7f3c0e000700 1 -- 172.21.15.1:0/4201790366 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04016070 con 0x7f3c1006ac90 2024-04-26T01:51:48.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.548+0000 7f3c0e000700 1 -- 172.21.15.1:0/4201790366 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c0400fd40 con 0x7f3c1006ac90 2024-04-26T01:51:48.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.548+0000 7f3c18000700 1 -- 172.21.15.1:0/4201790366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c101a0020 con 0x7f3c1006ac90 2024-04-26T01:51:48.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.548+0000 7f3c0e000700 1 -- 172.21.15.1:0/4201790366 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0400e3b0 con 0x7f3c1006ac90 2024-04-26T01:51:48.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.549+0000 7f3c18000700 1 -- 172.21.15.1:0/4201790366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c101ac2e0 con 0x7f3c1006ac90 2024-04-26T01:51:48.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.550+0000 7f3c18000700 1 -- 172.21.15.1:0/4201790366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c10069730 con 0x7f3c1006ac90 2024-04-26T01:51:48.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.552+0000 7f3c0e000700 1 -- 172.21.15.1:0/4201790366 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f3c04019070 con 0x7f3c1006ac90 2024-04-26T01:51:48.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.553+0000 7f3c0e000700 1 --2- 172.21.15.1:0/4201790366 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f3bf80a3ed0 0x7f3bf80a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:48.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.553+0000 7f3c0e000700 1 -- 172.21.15.1:0/4201790366 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f3c040d0550 con 0x7f3c1006ac90 2024-04-26T01:51:48.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.553+0000 7f3c14c00700 1 --2- 172.21.15.1:0/4201790366 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f3bf80a3ed0 0x7f3bf80a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:48.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.554+0000 7f3c14c00700 1 --2- 172.21.15.1:0/4201790366 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f3bf80a3ed0 0x7f3bf80a6390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3c1006b440 tx=0x7f3bfc00b540 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:48.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.558+0000 7f3c0e000700 1 -- 172.21.15.1:0/4201790366 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c0409c940 con 0x7f3c1006ac90 2024-04-26T01:51:48.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.786+0000 7f3c18000700 1 -- 172.21.15.1:0/4201790366 --> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3c10104b70 con 0x7f3bf80a3ed0 2024-04-26T01:51:48.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.809+0000 7f3c0e000700 1 -- 172.21.15.1:0/4201790366 <== mgr.25087 v2:172.21.15.1:6800/275906565 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257771 (secure 0 0 0) 0x7f3c10104b70 con 0x7f3bf80a3ed0 2024-04-26T01:51:48.812 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:51:48.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.816+0000 7f3c18000700 1 -- 172.21.15.1:0/4201790366 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f3bf80a3ed0 msgr2=0x7f3bf80a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:48.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.816+0000 7f3c18000700 1 --2- 172.21.15.1:0/4201790366 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f3bf80a3ed0 0x7f3bf80a6390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3c1006b440 tx=0x7f3bfc00b540 comp rx=0 tx=0).stop 2024-04-26T01:51:48.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.816+0000 7f3c18000700 1 -- 172.21.15.1:0/4201790366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c1006ac90 msgr2=0x7f3c101a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:48.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.816+0000 7f3c18000700 1 --2- 172.21.15.1:0/4201790366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c1006ac90 0x7f3c101a5410 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400d950 tx=0x7f3c0400dc60 comp rx=0 tx=0).stop 2024-04-26T01:51:48.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.816+0000 7f3c18000700 1 -- 172.21.15.1:0/4201790366 shutdown_connections 2024-04-26T01:51:48.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.816+0000 7f3c18000700 1 --2- 172.21.15.1:0/4201790366 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f3bf80a3ed0 0x7f3bf80a6390 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:48.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.816+0000 7f3c18000700 1 --2- 172.21.15.1:0/4201790366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c10107970 0x7f3c1019f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:48.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.816+0000 7f3c18000700 1 --2- 172.21.15.1:0/4201790366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c1006ac90 0x7f3c101a5410 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:48.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.816+0000 7f3c18000700 1 --2- 172.21.15.1:0/4201790366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c10105480 0x7f3c101a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:48.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.816+0000 7f3c18000700 1 -- 172.21.15.1:0/4201790366 >> 172.21.15.1:0/4201790366 conn(0x7f3c100788f0 msgr2=0x7f3c10104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:48.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.816+0000 7f3c18000700 1 -- 172.21.15.1:0/4201790366 shutdown_connections 2024-04-26T01:51:48.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:48.817+0000 7f3c18000700 1 -- 172.21.15.1:0/4201790366 wait complete. 2024-04-26T01:51:48.816 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-04-26T01:51:49.575 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-26T01:51:48.692475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1388429,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":2026,"num_read_kb":1439,"num_write":631,"num_write_kb":5301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":965,"ondisk_log_size":965,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365852,"kb_used_data":16320,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124575972,"statfs":{"total":1151940427776,"available":1151565795328,"internally_reserved":0,"allocated":16711680,"data_stored":10142727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":200325,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":388,"num_read_kb":460,"num_write":154,"num_write_kb":922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.952574"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581382+0000","last_change":"2024-04-26T01:47:27.140418+0000","last_active":"2024-04-26T01:51:39.581382+0000","last_peered":"2024-04-26T01:51:39.581382+0000","last_clean":"2024-04-26T01:51:39.581382+0000","last_became_active":"2024-04-26T01:47:27.140017+0000","last_became_peered":"2024-04-26T01:47:27.140017+0000","last_unstale":"2024-04-26T01:51:39.581382+0000","last_undegraded":"2024-04-26T01:51:39.581382+0000","last_fullsized":"2024-04-26T01:51:39.581382+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:46:08.621222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.226726+0000","last_change":"2024-04-26T01:47:27.138746+0000","last_active":"2024-04-26T01:51:39.226726+0000","last_peered":"2024-04-26T01:51:39.226726+0000","last_clean":"2024-04-26T01:51:39.226726+0000","last_became_active":"2024-04-26T01:47:27.136163+0000","last_became_peered":"2024-04-26T01:47:27.136163+0000","last_unstale":"2024-04-26T01:51:39.226726+0000","last_undegraded":"2024-04-26T01:51:39.226726+0000","last_fullsized":"2024-04-26T01:51:39.226726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:01:36.195332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.226469+0000","last_change":"2024-04-26T01:47:27.127017+0000","last_active":"2024-04-26T01:51:39.226469+0000","last_peered":"2024-04-26T01:51:39.226469+0000","last_clean":"2024-04-26T01:51:39.226469+0000","last_became_active":"2024-04-26T01:47:27.126610+0000","last_became_peered":"2024-04-26T01:47:27.126610+0000","last_unstale":"2024-04-26T01:51:39.226469+0000","last_undegraded":"2024-04-26T01:51:39.226469+0000","last_fullsized":"2024-04-26T01:51:39.226469+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:05:12.784486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737519+0000","last_change":"2024-04-26T01:47:27.117640+0000","last_active":"2024-04-26T01:51:38.737519+0000","last_peered":"2024-04-26T01:51:38.737519+0000","last_clean":"2024-04-26T01:51:38.737519+0000","last_became_active":"2024-04-26T01:47:27.117266+0000","last_became_peered":"2024-04-26T01:47:27.117266+0000","last_unstale":"2024-04-26T01:51:38.737519+0000","last_undegraded":"2024-04-26T01:51:38.737519+0000","last_fullsized":"2024-04-26T01:51:38.737519+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:26:57.824560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.760037+0000","last_change":"2024-04-26T01:47:27.128185+0000","last_active":"2024-04-26T01:51:38.760037+0000","last_peered":"2024-04-26T01:51:38.760037+0000","last_clean":"2024-04-26T01:51:38.760037+0000","last_became_active":"2024-04-26T01:47:27.127794+0000","last_became_peered":"2024-04-26T01:47:27.127794+0000","last_unstale":"2024-04-26T01:51:38.760037+0000","last_undegraded":"2024-04-26T01:51:38.760037+0000","last_fullsized":"2024-04-26T01:51:38.760037+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:09:52.774007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672330+0000","last_change":"2024-04-26T01:48:22.748601+0000","last_active":"2024-04-26T01:51:38.672330+0000","last_peered":"2024-04-26T01:51:38.672330+0000","last_clean":"2024-04-26T01:51:38.672330+0000","last_became_active":"2024-04-26T01:48:22.748229+0000","last_became_peered":"2024-04-26T01:48:22.748229+0000","last_unstale":"2024-04-26T01:51:38.672330+0000","last_undegraded":"2024-04-26T01:51:38.672330+0000","last_fullsized":"2024-04-26T01:51:38.672330+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:46:30.433063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.972514+0000","last_change":"2024-04-26T01:47:27.123786+0000","last_active":"2024-04-26T01:51:38.972514+0000","last_peered":"2024-04-26T01:51:38.972514+0000","last_clean":"2024-04-26T01:51:38.972514+0000","last_became_active":"2024-04-26T01:47:27.123473+0000","last_became_peered":"2024-04-26T01:47:27.123473+0000","last_unstale":"2024-04-26T01:51:38.972514+0000","last_undegraded":"2024-04-26T01:51:38.972514+0000","last_fullsized":"2024-04-26T01:51:38.972514+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:48:55.031147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737899+0000","last_change":"2024-04-26T01:47:27.118756+0000","last_active":"2024-04-26T01:51:38.737899+0000","last_peered":"2024-04-26T01:51:38.737899+0000","last_clean":"2024-04-26T01:51:38.737899+0000","last_became_active":"2024-04-26T01:47:27.118406+0000","last_became_peered":"2024-04-26T01:47:27.118406+0000","last_unstale":"2024-04-26T01:51:38.737899+0000","last_undegraded":"2024-04-26T01:51:38.737899+0000","last_fullsized":"2024-04-26T01:51:38.737899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:05.793682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.980594+0000","last_change":"2024-04-26T01:47:27.117622+0000","last_active":"2024-04-26T01:51:38.980594+0000","last_peered":"2024-04-26T01:51:38.980594+0000","last_clean":"2024-04-26T01:51:38.980594+0000","last_became_active":"2024-04-26T01:47:27.115206+0000","last_became_peered":"2024-04-26T01:47:27.115206+0000","last_unstale":"2024-04-26T01:51:38.980594+0000","last_undegraded":"2024-04-26T01:51:38.980594+0000","last_fullsized":"2024-04-26T01:51:38.980594+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:13:17.866630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672921+0000","last_change":"2024-04-26T01:47:27.119524+0000","last_active":"2024-04-26T01:51:38.672921+0000","last_peered":"2024-04-26T01:51:38.672921+0000","last_clean":"2024-04-26T01:51:38.672921+0000","last_became_active":"2024-04-26T01:47:27.119008+0000","last_became_peered":"2024-04-26T01:47:27.119008+0000","last_unstale":"2024-04-26T01:51:38.672921+0000","last_undegraded":"2024-04-26T01:51:38.672921+0000","last_fullsized":"2024-04-26T01:51:38.672921+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:37:31.894486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973282+0000","last_change":"2024-04-26T01:47:27.122361+0000","last_active":"2024-04-26T01:51:38.973282+0000","last_peered":"2024-04-26T01:51:38.973282+0000","last_clean":"2024-04-26T01:51:38.973282+0000","last_became_active":"2024-04-26T01:47:27.121748+0000","last_became_peered":"2024-04-26T01:47:27.121748+0000","last_unstale":"2024-04-26T01:51:38.973282+0000","last_undegraded":"2024-04-26T01:51:38.973282+0000","last_fullsized":"2024-04-26T01:51:38.973282+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:18.680909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737482+0000","last_change":"2024-04-26T01:47:27.116833+0000","last_active":"2024-04-26T01:51:38.737482+0000","last_peered":"2024-04-26T01:51:38.737482+0000","last_clean":"2024-04-26T01:51:38.737482+0000","last_became_active":"2024-04-26T01:47:27.116508+0000","last_became_peered":"2024-04-26T01:47:27.116508+0000","last_unstale":"2024-04-26T01:51:38.737482+0000","last_undegraded":"2024-04-26T01:51:38.737482+0000","last_fullsized":"2024-04-26T01:51:38.737482+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:27:45.864521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.671761+0000","last_change":"2024-04-26T01:47:27.125342+0000","last_active":"2024-04-26T01:51:38.671761+0000","last_peered":"2024-04-26T01:51:38.671761+0000","last_clean":"2024-04-26T01:51:38.671761+0000","last_became_active":"2024-04-26T01:47:27.124862+0000","last_became_peered":"2024-04-26T01:47:27.124862+0000","last_unstale":"2024-04-26T01:51:38.671761+0000","last_undegraded":"2024-04-26T01:51:38.671761+0000","last_fullsized":"2024-04-26T01:51:38.671761+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:48.654711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.227404+0000","last_change":"2024-04-26T01:47:27.127365+0000","last_active":"2024-04-26T01:51:39.227404+0000","last_peered":"2024-04-26T01:51:39.227404+0000","last_clean":"2024-04-26T01:51:39.227404+0000","last_became_active":"2024-04-26T01:47:27.126868+0000","last_became_peered":"2024-04-26T01:47:27.126868+0000","last_unstale":"2024-04-26T01:51:39.227404+0000","last_undegraded":"2024-04-26T01:51:39.227404+0000","last_fullsized":"2024-04-26T01:51:39.227404+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:07:46.400333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.226295+0000","last_change":"2024-04-26T01:47:27.135584+0000","last_active":"2024-04-26T01:51:39.226295+0000","last_peered":"2024-04-26T01:51:39.226295+0000","last_clean":"2024-04-26T01:51:39.226295+0000","last_became_active":"2024-04-26T01:47:27.134849+0000","last_became_peered":"2024-04-26T01:47:27.134849+0000","last_unstale":"2024-04-26T01:51:39.226295+0000","last_undegraded":"2024-04-26T01:51:39.226295+0000","last_fullsized":"2024-04-26T01:51:39.226295+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:57:48.821595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.980000+0000","last_change":"2024-04-26T01:47:27.136182+0000","last_active":"2024-04-26T01:51:38.980000+0000","last_peered":"2024-04-26T01:51:38.980000+0000","last_clean":"2024-04-26T01:51:38.980000+0000","last_became_active":"2024-04-26T01:47:27.135787+0000","last_became_peered":"2024-04-26T01:47:27.135787+0000","last_unstale":"2024-04-26T01:51:38.980000+0000","last_undegraded":"2024-04-26T01:51:38.980000+0000","last_fullsized":"2024-04-26T01:51:38.980000+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:38:14.275898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581398+0000","last_change":"2024-04-26T01:47:27.140700+0000","last_active":"2024-04-26T01:51:39.581398+0000","last_peered":"2024-04-26T01:51:39.581398+0000","last_clean":"2024-04-26T01:51:39.581398+0000","last_became_active":"2024-04-26T01:47:27.140025+0000","last_became_peered":"2024-04-26T01:47:27.140025+0000","last_unstale":"2024-04-26T01:51:39.581398+0000","last_undegraded":"2024-04-26T01:51:39.581398+0000","last_fullsized":"2024-04-26T01:51:39.581398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:50.465236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.794694+0000","last_change":"2024-04-26T01:47:27.117256+0000","last_active":"2024-04-26T01:51:38.794694+0000","last_peered":"2024-04-26T01:51:38.794694+0000","last_clean":"2024-04-26T01:51:38.794694+0000","last_became_active":"2024-04-26T01:47:27.116825+0000","last_became_peered":"2024-04-26T01:47:27.116825+0000","last_unstale":"2024-04-26T01:51:38.794694+0000","last_undegraded":"2024-04-26T01:51:38.794694+0000","last_fullsized":"2024-04-26T01:51:38.794694+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:23.313712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.977034+0000","last_change":"2024-04-26T01:47:27.130233+0000","last_active":"2024-04-26T01:51:38.977034+0000","last_peered":"2024-04-26T01:51:38.977034+0000","last_clean":"2024-04-26T01:51:38.977034+0000","last_became_active":"2024-04-26T01:47:27.129852+0000","last_became_peered":"2024-04-26T01:47:27.129852+0000","last_unstale":"2024-04-26T01:51:38.977034+0000","last_undegraded":"2024-04-26T01:51:38.977034+0000","last_fullsized":"2024-04-26T01:51:38.977034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:49:01.407264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.794816+0000","last_change":"2024-04-26T01:47:23.068477+0000","last_active":"2024-04-26T01:51:38.794816+0000","last_peered":"2024-04-26T01:51:38.794816+0000","last_clean":"2024-04-26T01:51:38.794816+0000","last_became_active":"2024-04-26T01:47:23.068061+0000","last_became_peered":"2024-04-26T01:47:23.068061+0000","last_unstale":"2024-04-26T01:51:38.794816+0000","last_undegraded":"2024-04-26T01:51:38.794816+0000","last_fullsized":"2024-04-26T01:51:38.794816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:02.153724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.738398+0000","last_change":"2024-04-26T01:47:27.130385+0000","last_active":"2024-04-26T01:51:38.738398+0000","last_peered":"2024-04-26T01:51:38.738398+0000","last_clean":"2024-04-26T01:51:38.738398+0000","last_became_active":"2024-04-26T01:47:27.129912+0000","last_became_peered":"2024-04-26T01:47:27.129912+0000","last_unstale":"2024-04-26T01:51:38.738398+0000","last_undegraded":"2024-04-26T01:51:38.738398+0000","last_fullsized":"2024-04-26T01:51:38.738398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:26:03.010544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.738555+0000","last_change":"2024-04-26T01:47:23.110806+0000","last_active":"2024-04-26T01:51:38.738555+0000","last_peered":"2024-04-26T01:51:38.738555+0000","last_clean":"2024-04-26T01:51:38.738555+0000","last_became_active":"2024-04-26T01:47:23.110441+0000","last_became_peered":"2024-04-26T01:47:23.110441+0000","last_unstale":"2024-04-26T01:51:38.738555+0000","last_undegraded":"2024-04-26T01:51:38.738555+0000","last_fullsized":"2024-04-26T01:51:38.738555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:39.239172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.227010+0000","last_change":"2024-04-26T01:47:27.138628+0000","last_active":"2024-04-26T01:51:39.227010+0000","last_peered":"2024-04-26T01:51:39.227010+0000","last_clean":"2024-04-26T01:51:39.227010+0000","last_became_active":"2024-04-26T01:47:27.130483+0000","last_became_peered":"2024-04-26T01:47:27.130483+0000","last_unstale":"2024-04-26T01:51:39.227010+0000","last_undegraded":"2024-04-26T01:51:39.227010+0000","last_fullsized":"2024-04-26T01:51:39.227010+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:40:41.242856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.760376+0000","last_change":"2024-04-26T01:47:23.060066+0000","last_active":"2024-04-26T01:51:38.760376+0000","last_peered":"2024-04-26T01:51:38.760376+0000","last_clean":"2024-04-26T01:51:38.760376+0000","last_became_active":"2024-04-26T01:47:23.059526+0000","last_became_peered":"2024-04-26T01:47:23.059526+0000","last_unstale":"2024-04-26T01:51:38.760376+0000","last_undegraded":"2024-04-26T01:51:38.760376+0000","last_fullsized":"2024-04-26T01:51:38.760376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:28:22.623695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.980142+0000","last_change":"2024-04-26T01:47:27.128343+0000","last_active":"2024-04-26T01:51:38.980142+0000","last_peered":"2024-04-26T01:51:38.980142+0000","last_clean":"2024-04-26T01:51:38.980142+0000","last_became_active":"2024-04-26T01:47:27.127920+0000","last_became_peered":"2024-04-26T01:47:27.127920+0000","last_unstale":"2024-04-26T01:51:38.980142+0000","last_undegraded":"2024-04-26T01:51:38.980142+0000","last_fullsized":"2024-04-26T01:51:38.980142+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:06.725033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.760152+0000","last_change":"2024-04-26T01:47:23.057783+0000","last_active":"2024-04-26T01:51:38.760152+0000","last_peered":"2024-04-26T01:51:38.760152+0000","last_clean":"2024-04-26T01:51:38.760152+0000","last_became_active":"2024-04-26T01:47:23.057391+0000","last_became_peered":"2024-04-26T01:47:23.057391+0000","last_unstale":"2024-04-26T01:51:38.760152+0000","last_undegraded":"2024-04-26T01:51:38.760152+0000","last_fullsized":"2024-04-26T01:51:38.760152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:10:10.050787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672824+0000","last_change":"2024-04-26T01:47:27.116458+0000","last_active":"2024-04-26T01:51:38.672824+0000","last_peered":"2024-04-26T01:51:38.672824+0000","last_clean":"2024-04-26T01:51:38.672824+0000","last_became_active":"2024-04-26T01:47:27.116084+0000","last_became_peered":"2024-04-26T01:47:27.116084+0000","last_unstale":"2024-04-26T01:51:38.672824+0000","last_undegraded":"2024-04-26T01:51:38.672824+0000","last_fullsized":"2024-04-26T01:51:38.672824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:06:34.517478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"93'7","reported_seq":275,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:43.609347+0000","last_change":"2024-04-26T01:47:23.060657+0000","last_active":"2024-04-26T01:51:43.609347+0000","last_peered":"2024-04-26T01:51:43.609347+0000","last_clean":"2024-04-26T01:51:43.609347+0000","last_became_active":"2024-04-26T01:47:23.060294+0000","last_became_peered":"2024-04-26T01:47:23.060294+0000","last_unstale":"2024-04-26T01:51:43.609347+0000","last_undegraded":"2024-04-26T01:51:43.609347+0000","last_fullsized":"2024-04-26T01:51:43.609347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:33.979717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":239,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.759940+0000","last_change":"2024-04-26T01:47:27.120587+0000","last_active":"2024-04-26T01:51:38.759940+0000","last_peered":"2024-04-26T01:51:38.759940+0000","last_clean":"2024-04-26T01:51:38.759940+0000","last_became_active":"2024-04-26T01:47:27.120236+0000","last_became_peered":"2024-04-26T01:47:27.120236+0000","last_unstale":"2024-04-26T01:51:38.759940+0000","last_undegraded":"2024-04-26T01:51:38.759940+0000","last_fullsized":"2024-04-26T01:51:38.759940+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:30:38.488741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672250+0000","last_change":"2024-04-26T01:47:23.069395+0000","last_active":"2024-04-26T01:51:38.672250+0000","last_peered":"2024-04-26T01:51:38.672250+0000","last_clean":"2024-04-26T01:51:38.672250+0000","last_became_active":"2024-04-26T01:47:23.069102+0000","last_became_peered":"2024-04-26T01:47:23.069102+0000","last_unstale":"2024-04-26T01:51:38.672250+0000","last_undegraded":"2024-04-26T01:51:38.672250+0000","last_fullsized":"2024-04-26T01:51:38.672250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:45.162604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672487+0000","last_change":"2024-04-26T01:47:27.123859+0000","last_active":"2024-04-26T01:51:38.672487+0000","last_peered":"2024-04-26T01:51:38.672487+0000","last_clean":"2024-04-26T01:51:38.672487+0000","last_became_active":"2024-04-26T01:47:27.123438+0000","last_became_peered":"2024-04-26T01:47:27.123438+0000","last_unstale":"2024-04-26T01:51:38.672487+0000","last_undegraded":"2024-04-26T01:51:38.672487+0000","last_fullsized":"2024-04-26T01:51:38.672487+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:33:53.542809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.671891+0000","last_change":"2024-04-26T01:47:23.071544+0000","last_active":"2024-04-26T01:51:38.671891+0000","last_peered":"2024-04-26T01:51:38.671891+0000","last_clean":"2024-04-26T01:51:38.671891+0000","last_became_active":"2024-04-26T01:47:23.071191+0000","last_became_peered":"2024-04-26T01:47:23.071191+0000","last_unstale":"2024-04-26T01:51:38.671891+0000","last_undegraded":"2024-04-26T01:51:38.671891+0000","last_fullsized":"2024-04-26T01:51:38.671891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:58.889561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737808+0000","last_change":"2024-04-26T01:47:27.131887+0000","last_active":"2024-04-26T01:51:38.737808+0000","last_peered":"2024-04-26T01:51:38.737808+0000","last_clean":"2024-04-26T01:51:38.737808+0000","last_became_active":"2024-04-26T01:47:27.127973+0000","last_became_peered":"2024-04-26T01:47:27.127973+0000","last_unstale":"2024-04-26T01:51:38.737808+0000","last_undegraded":"2024-04-26T01:51:38.737808+0000","last_fullsized":"2024-04-26T01:51:38.737808+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:54:35.305779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672192+0000","last_change":"2024-04-26T01:47:23.061353+0000","last_active":"2024-04-26T01:51:38.672192+0000","last_peered":"2024-04-26T01:51:38.672192+0000","last_clean":"2024-04-26T01:51:38.672192+0000","last_became_active":"2024-04-26T01:47:23.061037+0000","last_became_peered":"2024-04-26T01:47:23.061037+0000","last_unstale":"2024-04-26T01:51:38.672192+0000","last_undegraded":"2024-04-26T01:51:38.672192+0000","last_fullsized":"2024-04-26T01:51:38.672192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:13:59.617969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.794335+0000","last_change":"2024-04-26T01:47:27.122303+0000","last_active":"2024-04-26T01:51:38.794335+0000","last_peered":"2024-04-26T01:51:38.794335+0000","last_clean":"2024-04-26T01:51:38.794335+0000","last_became_active":"2024-04-26T01:47:27.121919+0000","last_became_peered":"2024-04-26T01:47:27.121919+0000","last_unstale":"2024-04-26T01:51:38.794335+0000","last_undegraded":"2024-04-26T01:51:38.794335+0000","last_fullsized":"2024-04-26T01:51:38.794335+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:21.676663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"93'4","reported_seq":268,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:43.608595+0000","last_change":"2024-04-26T01:47:23.057961+0000","last_active":"2024-04-26T01:51:43.608595+0000","last_peered":"2024-04-26T01:51:43.608595+0000","last_clean":"2024-04-26T01:51:43.608595+0000","last_became_active":"2024-04-26T01:47:23.057529+0000","last_became_peered":"2024-04-26T01:47:23.057529+0000","last_unstale":"2024-04-26T01:51:43.608595+0000","last_undegraded":"2024-04-26T01:51:43.608595+0000","last_fullsized":"2024-04-26T01:51:43.608595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:14.420265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":235,"num_read_kb":235,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.980525+0000","last_change":"2024-04-26T01:47:27.120083+0000","last_active":"2024-04-26T01:51:38.980525+0000","last_peered":"2024-04-26T01:51:38.980525+0000","last_clean":"2024-04-26T01:51:38.980525+0000","last_became_active":"2024-04-26T01:47:27.118410+0000","last_became_peered":"2024-04-26T01:47:27.118410+0000","last_unstale":"2024-04-26T01:51:38.980525+0000","last_undegraded":"2024-04-26T01:51:38.980525+0000","last_fullsized":"2024-04-26T01:51:38.980525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:28.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973354+0000","last_change":"2024-04-26T01:47:23.059221+0000","last_active":"2024-04-26T01:51:38.973354+0000","last_peered":"2024-04-26T01:51:38.973354+0000","last_clean":"2024-04-26T01:51:38.973354+0000","last_became_active":"2024-04-26T01:47:23.058881+0000","last_became_peered":"2024-04-26T01:47:23.058881+0000","last_unstale":"2024-04-26T01:51:38.973354+0000","last_undegraded":"2024-04-26T01:51:38.973354+0000","last_fullsized":"2024-04-26T01:51:38.973354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:30.433696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.226796+0000","last_change":"2024-04-26T01:47:27.136763+0000","last_active":"2024-04-26T01:51:39.226796+0000","last_peered":"2024-04-26T01:51:39.226796+0000","last_clean":"2024-04-26T01:51:39.226796+0000","last_became_active":"2024-04-26T01:47:27.133642+0000","last_became_peered":"2024-04-26T01:47:27.133642+0000","last_unstale":"2024-04-26T01:51:39.226796+0000","last_undegraded":"2024-04-26T01:51:39.226796+0000","last_fullsized":"2024-04-26T01:51:39.226796+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:05.901139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672938+0000","last_change":"2024-04-26T01:47:23.108883+0000","last_active":"2024-04-26T01:51:38.672938+0000","last_peered":"2024-04-26T01:51:38.672938+0000","last_clean":"2024-04-26T01:51:38.672938+0000","last_became_active":"2024-04-26T01:47:23.108499+0000","last_became_peered":"2024-04-26T01:47:23.108499+0000","last_unstale":"2024-04-26T01:51:38.672938+0000","last_undegraded":"2024-04-26T01:51:38.672938+0000","last_fullsized":"2024-04-26T01:51:38.672938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:14:17.555416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.980570+0000","last_change":"2024-04-26T01:48:22.410651+0000","last_active":"2024-04-26T01:51:38.980570+0000","last_peered":"2024-04-26T01:51:38.980570+0000","last_clean":"2024-04-26T01:51:38.980570+0000","last_became_active":"2024-04-26T01:48:22.410359+0000","last_became_peered":"2024-04-26T01:48:22.410359+0000","last_unstale":"2024-04-26T01:51:38.980570+0000","last_undegraded":"2024-04-26T01:51:38.980570+0000","last_fullsized":"2024-04-26T01:51:38.980570+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:53:20.686915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"93'38","reported_seq":550,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:44.623105+0000","last_change":"2024-04-26T01:47:23.054893+0000","last_active":"2024-04-26T01:51:44.623105+0000","last_peered":"2024-04-26T01:51:44.623105+0000","last_clean":"2024-04-26T01:51:44.623105+0000","last_became_active":"2024-04-26T01:47:23.054558+0000","last_became_peered":"2024-04-26T01:47:23.054558+0000","last_unstale":"2024-04-26T01:51:44.623105+0000","last_undegraded":"2024-04-26T01:51:44.623105+0000","last_fullsized":"2024-04-26T01:51:44.623105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:59.356841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69972,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":483,"num_read_kb":308,"num_write":39,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.760179+0000","last_change":"2024-04-26T01:47:27.116934+0000","last_active":"2024-04-26T01:51:38.760179+0000","last_peered":"2024-04-26T01:51:38.760179+0000","last_clean":"2024-04-26T01:51:38.760179+0000","last_became_active":"2024-04-26T01:47:27.116527+0000","last_became_peered":"2024-04-26T01:47:27.116527+0000","last_unstale":"2024-04-26T01:51:38.760179+0000","last_undegraded":"2024-04-26T01:51:38.760179+0000","last_fullsized":"2024-04-26T01:51:38.760179+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:09:45.820533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.976702+0000","last_change":"2024-04-26T01:47:23.059032+0000","last_active":"2024-04-26T01:51:38.976702+0000","last_peered":"2024-04-26T01:51:38.976702+0000","last_clean":"2024-04-26T01:51:38.976702+0000","last_became_active":"2024-04-26T01:47:23.057992+0000","last_became_peered":"2024-04-26T01:47:23.057992+0000","last_unstale":"2024-04-26T01:51:38.976702+0000","last_undegraded":"2024-04-26T01:51:38.976702+0000","last_fullsized":"2024-04-26T01:51:38.976702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:46:53.164925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.673476+0000","last_change":"2024-04-26T01:47:27.134574+0000","last_active":"2024-04-26T01:51:38.673476+0000","last_peered":"2024-04-26T01:51:38.673476+0000","last_clean":"2024-04-26T01:51:38.673476+0000","last_became_active":"2024-04-26T01:47:27.134184+0000","last_became_peered":"2024-04-26T01:47:27.134184+0000","last_unstale":"2024-04-26T01:51:38.673476+0000","last_undegraded":"2024-04-26T01:51:38.673476+0000","last_fullsized":"2024-04-26T01:51:38.673476+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:42:26.428014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672306+0000","last_change":"2024-04-26T01:47:23.062649+0000","last_active":"2024-04-26T01:51:38.672306+0000","last_peered":"2024-04-26T01:51:38.672306+0000","last_clean":"2024-04-26T01:51:38.672306+0000","last_became_active":"2024-04-26T01:47:23.062282+0000","last_became_peered":"2024-04-26T01:47:23.062282+0000","last_unstale":"2024-04-26T01:51:38.672306+0000","last_undegraded":"2024-04-26T01:51:38.672306+0000","last_fullsized":"2024-04-26T01:51:38.672306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:38.699105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973219+0000","last_change":"2024-04-26T01:47:27.124720+0000","last_active":"2024-04-26T01:51:38.973219+0000","last_peered":"2024-04-26T01:51:38.973219+0000","last_clean":"2024-04-26T01:51:38.973219+0000","last_became_active":"2024-04-26T01:47:27.124378+0000","last_became_peered":"2024-04-26T01:47:27.124378+0000","last_unstale":"2024-04-26T01:51:38.973219+0000","last_undegraded":"2024-04-26T01:51:38.973219+0000","last_fullsized":"2024-04-26T01:51:38.973219+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:45.462741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"93'9","reported_seq":279,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:42.746241+0000","last_change":"2024-04-26T01:47:23.056305+0000","last_active":"2024-04-26T01:51:42.746241+0000","last_peered":"2024-04-26T01:51:42.746241+0000","last_clean":"2024-04-26T01:51:42.746241+0000","last_became_active":"2024-04-26T01:47:23.055905+0000","last_became_peered":"2024-04-26T01:47:23.055905+0000","last_unstale":"2024-04-26T01:51:42.746241+0000","last_undegraded":"2024-04-26T01:51:42.746241+0000","last_fullsized":"2024-04-26T01:51:42.746241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:00:14.609673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2603,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":242,"num_read_kb":8,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.972618+0000","last_change":"2024-04-26T01:47:27.119880+0000","last_active":"2024-04-26T01:51:38.972618+0000","last_peered":"2024-04-26T01:51:38.972618+0000","last_clean":"2024-04-26T01:51:38.972618+0000","last_became_active":"2024-04-26T01:47:27.119483+0000","last_became_peered":"2024-04-26T01:47:27.119483+0000","last_unstale":"2024-04-26T01:51:38.972618+0000","last_undegraded":"2024-04-26T01:51:38.972618+0000","last_fullsized":"2024-04-26T01:51:38.972618+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:15.523150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":69,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.759939+0000","last_change":"2024-04-26T01:48:22.758774+0000","last_active":"2024-04-26T01:51:38.759939+0000","last_peered":"2024-04-26T01:51:38.759939+0000","last_clean":"2024-04-26T01:51:38.759939+0000","last_became_active":"2024-04-26T01:48:22.747354+0000","last_became_peered":"2024-04-26T01:48:22.747354+0000","last_unstale":"2024-04-26T01:51:38.759939+0000","last_undegraded":"2024-04-26T01:51:38.759939+0000","last_fullsized":"2024-04-26T01:51:38.759939+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:23:15.886774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672382+0000","last_change":"2024-04-26T01:47:27.113508+0000","last_active":"2024-04-26T01:51:38.672382+0000","last_peered":"2024-04-26T01:51:38.672382+0000","last_clean":"2024-04-26T01:51:38.672382+0000","last_became_active":"2024-04-26T01:47:27.113066+0000","last_became_peered":"2024-04-26T01:47:27.113066+0000","last_unstale":"2024-04-26T01:51:38.672382+0000","last_undegraded":"2024-04-26T01:51:38.672382+0000","last_fullsized":"2024-04-26T01:51:38.672382+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:37.525486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"93'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.760368+0000","last_change":"2024-04-26T01:47:23.059644+0000","last_active":"2024-04-26T01:51:38.760368+0000","last_peered":"2024-04-26T01:51:38.760368+0000","last_clean":"2024-04-26T01:51:38.760368+0000","last_became_active":"2024-04-26T01:47:23.058966+0000","last_became_peered":"2024-04-26T01:47:23.058966+0000","last_unstale":"2024-04-26T01:51:38.760368+0000","last_undegraded":"2024-04-26T01:51:38.760368+0000","last_fullsized":"2024-04-26T01:51:38.760368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:57:36.475935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581235+0000","last_change":"2024-04-26T01:48:22.749354+0000","last_active":"2024-04-26T01:51:39.581235+0000","last_peered":"2024-04-26T01:51:39.581235+0000","last_clean":"2024-04-26T01:51:39.581235+0000","last_became_active":"2024-04-26T01:48:22.748978+0000","last_became_peered":"2024-04-26T01:48:22.748978+0000","last_unstale":"2024-04-26T01:51:39.581235+0000","last_undegraded":"2024-04-26T01:51:39.581235+0000","last_fullsized":"2024-04-26T01:51:39.581235+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:51:16.087091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.794533+0000","last_change":"2024-04-26T01:47:23.110360+0000","last_active":"2024-04-26T01:51:38.794533+0000","last_peered":"2024-04-26T01:51:38.794533+0000","last_clean":"2024-04-26T01:51:38.794533+0000","last_became_active":"2024-04-26T01:47:23.109744+0000","last_became_peered":"2024-04-26T01:47:23.109744+0000","last_unstale":"2024-04-26T01:51:38.794533+0000","last_undegraded":"2024-04-26T01:51:38.794533+0000","last_fullsized":"2024-04-26T01:51:38.794533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:40:13.658396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672730+0000","last_change":"2024-04-26T01:47:27.125399+0000","last_active":"2024-04-26T01:51:38.672730+0000","last_peered":"2024-04-26T01:51:38.672730+0000","last_clean":"2024-04-26T01:51:38.672730+0000","last_became_active":"2024-04-26T01:47:27.124871+0000","last_became_peered":"2024-04-26T01:47:27.124871+0000","last_unstale":"2024-04-26T01:51:38.672730+0000","last_undegraded":"2024-04-26T01:51:38.672730+0000","last_fullsized":"2024-04-26T01:51:38.672730+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:28:48.285206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973098+0000","last_change":"2024-04-26T01:47:23.067862+0000","last_active":"2024-04-26T01:51:38.973098+0000","last_peered":"2024-04-26T01:51:38.973098+0000","last_clean":"2024-04-26T01:51:38.973098+0000","last_became_active":"2024-04-26T01:47:23.067526+0000","last_became_peered":"2024-04-26T01:47:23.067526+0000","last_unstale":"2024-04-26T01:51:38.973098+0000","last_undegraded":"2024-04-26T01:51:38.973098+0000","last_fullsized":"2024-04-26T01:51:38.973098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:22.668405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672467+0000","last_change":"2024-04-26T01:48:22.748548+0000","last_active":"2024-04-26T01:51:38.672467+0000","last_peered":"2024-04-26T01:51:38.672467+0000","last_clean":"2024-04-26T01:51:38.672467+0000","last_became_active":"2024-04-26T01:48:22.748182+0000","last_became_peered":"2024-04-26T01:48:22.748182+0000","last_unstale":"2024-04-26T01:51:38.672467+0000","last_undegraded":"2024-04-26T01:51:38.672467+0000","last_fullsized":"2024-04-26T01:51:38.672467+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:20:27.596827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672474+0000","last_change":"2024-04-26T01:47:27.118307+0000","last_active":"2024-04-26T01:51:38.672474+0000","last_peered":"2024-04-26T01:51:38.672474+0000","last_clean":"2024-04-26T01:51:38.672474+0000","last_became_active":"2024-04-26T01:47:27.117986+0000","last_became_peered":"2024-04-26T01:47:27.117986+0000","last_unstale":"2024-04-26T01:51:38.672474+0000","last_undegraded":"2024-04-26T01:51:38.672474+0000","last_fullsized":"2024-04-26T01:51:38.672474+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:40.667143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":270,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:42.745383+0000","last_change":"2024-04-26T01:47:23.109973+0000","last_active":"2024-04-26T01:51:42.745383+0000","last_peered":"2024-04-26T01:51:42.745383+0000","last_clean":"2024-04-26T01:51:42.745383+0000","last_became_active":"2024-04-26T01:47:23.109446+0000","last_became_peered":"2024-04-26T01:47:23.109446+0000","last_unstale":"2024-04-26T01:51:42.745383+0000","last_undegraded":"2024-04-26T01:51:42.745383+0000","last_fullsized":"2024-04-26T01:51:42.745383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:49:00.684164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":234,"num_read_kb":234,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.738353+0000","last_change":"2024-04-26T01:47:27.131502+0000","last_active":"2024-04-26T01:51:38.738353+0000","last_peered":"2024-04-26T01:51:38.738353+0000","last_clean":"2024-04-26T01:51:38.738353+0000","last_became_active":"2024-04-26T01:47:27.127697+0000","last_became_peered":"2024-04-26T01:47:27.127697+0000","last_unstale":"2024-04-26T01:51:38.738353+0000","last_undegraded":"2024-04-26T01:51:38.738353+0000","last_fullsized":"2024-04-26T01:51:38.738353+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:39:15.458189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737630+0000","last_change":"2024-04-26T01:47:23.066596+0000","last_active":"2024-04-26T01:51:38.737630+0000","last_peered":"2024-04-26T01:51:38.737630+0000","last_clean":"2024-04-26T01:51:38.737630+0000","last_became_active":"2024-04-26T01:47:23.066216+0000","last_became_peered":"2024-04-26T01:47:23.066216+0000","last_unstale":"2024-04-26T01:51:38.737630+0000","last_undegraded":"2024-04-26T01:51:38.737630+0000","last_fullsized":"2024-04-26T01:51:38.737630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:08:31.544831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"93'148","reported_seq":407,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:40.835064+0000","last_change":"2024-04-26T01:45:54.834501+0000","last_active":"2024-04-26T01:51:40.835064+0000","last_peered":"2024-04-26T01:51:40.835064+0000","last_clean":"2024-04-26T01:51:40.835064+0000","last_became_active":"2024-04-26T01:45:54.833868+0000","last_became_peered":"2024-04-26T01:45:54.833868+0000","last_unstale":"2024-04-26T01:51:40.835064+0000","last_undegraded":"2024-04-26T01:51:40.835064+0000","last_fullsized":"2024-04-26T01:51:40.835064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:42:05.228956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:42:05.228956+0000","last_clean_scrub_stamp":"2024-04-26T01:42:05.228956+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:06:58.383315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.671645+0000","last_change":"2024-04-26T01:47:27.126475+0000","last_active":"2024-04-26T01:51:38.671645+0000","last_peered":"2024-04-26T01:51:38.671645+0000","last_clean":"2024-04-26T01:51:38.671645+0000","last_became_active":"2024-04-26T01:47:27.123760+0000","last_became_peered":"2024-04-26T01:47:27.123760+0000","last_unstale":"2024-04-26T01:51:38.671645+0000","last_undegraded":"2024-04-26T01:51:38.671645+0000","last_fullsized":"2024-04-26T01:51:38.671645+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:38:27.473909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.972839+0000","last_change":"2024-04-26T01:47:23.059810+0000","last_active":"2024-04-26T01:51:38.972839+0000","last_peered":"2024-04-26T01:51:38.972839+0000","last_clean":"2024-04-26T01:51:38.972839+0000","last_became_active":"2024-04-26T01:47:23.059504+0000","last_became_peered":"2024-04-26T01:47:23.059504+0000","last_unstale":"2024-04-26T01:51:38.972839+0000","last_undegraded":"2024-04-26T01:51:38.972839+0000","last_fullsized":"2024-04-26T01:51:38.972839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:50:51.220951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.760303+0000","last_change":"2024-04-26T01:47:23.109460+0000","last_active":"2024-04-26T01:51:38.760303+0000","last_peered":"2024-04-26T01:51:38.760303+0000","last_clean":"2024-04-26T01:51:38.760303+0000","last_became_active":"2024-04-26T01:47:23.109090+0000","last_became_peered":"2024-04-26T01:47:23.109090+0000","last_unstale":"2024-04-26T01:51:38.760303+0000","last_undegraded":"2024-04-26T01:51:38.760303+0000","last_fullsized":"2024-04-26T01:51:38.760303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:56.821312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672774+0000","last_change":"2024-04-26T01:47:27.129537+0000","last_active":"2024-04-26T01:51:38.672774+0000","last_peered":"2024-04-26T01:51:38.672774+0000","last_clean":"2024-04-26T01:51:38.672774+0000","last_became_active":"2024-04-26T01:47:27.128124+0000","last_became_peered":"2024-04-26T01:47:27.128124+0000","last_unstale":"2024-04-26T01:51:38.672774+0000","last_undegraded":"2024-04-26T01:51:38.672774+0000","last_fullsized":"2024-04-26T01:51:38.672774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:14:26.262197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"93'101","reported_seq":223,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:41.934307+0000","last_change":"2024-04-26T01:47:23.064511+0000","last_active":"2024-04-26T01:51:41.934307+0000","last_peered":"2024-04-26T01:51:41.934307+0000","last_clean":"2024-04-26T01:51:41.934307+0000","last_became_active":"2024-04-26T01:47:23.064080+0000","last_became_peered":"2024-04-26T01:47:23.064080+0000","last_unstale":"2024-04-26T01:51:41.934307+0000","last_undegraded":"2024-04-26T01:51:41.934307+0000","last_fullsized":"2024-04-26T01:51:41.934307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:37.562644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":185,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.738658+0000","last_change":"2024-04-26T01:47:27.124057+0000","last_active":"2024-04-26T01:51:38.738658+0000","last_peered":"2024-04-26T01:51:38.738658+0000","last_clean":"2024-04-26T01:51:38.738658+0000","last_became_active":"2024-04-26T01:47:27.123636+0000","last_became_peered":"2024-04-26T01:47:27.123636+0000","last_unstale":"2024-04-26T01:51:38.738658+0000","last_undegraded":"2024-04-26T01:51:38.738658+0000","last_fullsized":"2024-04-26T01:51:38.738658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:56:55.691697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737559+0000","last_change":"2024-04-26T01:47:23.136571+0000","last_active":"2024-04-26T01:51:38.737559+0000","last_peered":"2024-04-26T01:51:38.737559+0000","last_clean":"2024-04-26T01:51:38.737559+0000","last_became_active":"2024-04-26T01:47:23.136205+0000","last_became_peered":"2024-04-26T01:47:23.136205+0000","last_unstale":"2024-04-26T01:51:38.737559+0000","last_undegraded":"2024-04-26T01:51:38.737559+0000","last_fullsized":"2024-04-26T01:51:38.737559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:24.418797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.226818+0000","last_change":"2024-04-26T01:48:22.747570+0000","last_active":"2024-04-26T01:51:39.226818+0000","last_peered":"2024-04-26T01:51:39.226818+0000","last_clean":"2024-04-26T01:51:39.226818+0000","last_became_active":"2024-04-26T01:48:22.747147+0000","last_became_peered":"2024-04-26T01:48:22.747147+0000","last_unstale":"2024-04-26T01:51:39.226818+0000","last_undegraded":"2024-04-26T01:51:39.226818+0000","last_fullsized":"2024-04-26T01:51:39.226818+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:37.034388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.980081+0000","last_change":"2024-04-26T01:48:22.414610+0000","last_active":"2024-04-26T01:51:38.980081+0000","last_peered":"2024-04-26T01:51:38.980081+0000","last_clean":"2024-04-26T01:51:38.980081+0000","last_became_active":"2024-04-26T01:48:22.410275+0000","last_became_peered":"2024-04-26T01:48:22.410275+0000","last_unstale":"2024-04-26T01:51:38.980081+0000","last_undegraded":"2024-04-26T01:51:38.980081+0000","last_fullsized":"2024-04-26T01:51:38.980081+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:20:48.131988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.759917+0000","last_change":"2024-04-26T01:47:27.115498+0000","last_active":"2024-04-26T01:51:38.759917+0000","last_peered":"2024-04-26T01:51:38.759917+0000","last_clean":"2024-04-26T01:51:38.759917+0000","last_became_active":"2024-04-26T01:47:27.115134+0000","last_became_peered":"2024-04-26T01:47:27.115134+0000","last_unstale":"2024-04-26T01:51:38.759917+0000","last_undegraded":"2024-04-26T01:51:38.759917+0000","last_fullsized":"2024-04-26T01:51:38.759917+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:20.795224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.738166+0000","last_change":"2024-04-26T01:47:23.065366+0000","last_active":"2024-04-26T01:51:38.738166+0000","last_peered":"2024-04-26T01:51:38.738166+0000","last_clean":"2024-04-26T01:51:38.738166+0000","last_became_active":"2024-04-26T01:47:23.063216+0000","last_became_peered":"2024-04-26T01:47:23.063216+0000","last_unstale":"2024-04-26T01:51:38.738166+0000","last_undegraded":"2024-04-26T01:51:38.738166+0000","last_fullsized":"2024-04-26T01:51:38.738166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:06:05.261907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581069+0000","last_change":"2024-04-26T01:47:27.127938+0000","last_active":"2024-04-26T01:51:39.581069+0000","last_peered":"2024-04-26T01:51:39.581069+0000","last_clean":"2024-04-26T01:51:39.581069+0000","last_became_active":"2024-04-26T01:47:27.127553+0000","last_became_peered":"2024-04-26T01:47:27.127553+0000","last_unstale":"2024-04-26T01:51:39.581069+0000","last_undegraded":"2024-04-26T01:51:39.581069+0000","last_fullsized":"2024-04-26T01:51:39.581069+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:17:51.115349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"93'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.240037+0000","last_change":"2024-04-26T01:47:23.060475+0000","last_active":"2024-04-26T01:51:39.240037+0000","last_peered":"2024-04-26T01:51:39.240037+0000","last_clean":"2024-04-26T01:51:39.240037+0000","last_became_active":"2024-04-26T01:47:23.060098+0000","last_became_peered":"2024-04-26T01:47:23.060098+0000","last_unstale":"2024-04-26T01:51:39.240037+0000","last_undegraded":"2024-04-26T01:51:39.240037+0000","last_fullsized":"2024-04-26T01:51:39.240037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:49.009455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.738288+0000","last_change":"2024-04-26T01:47:27.137491+0000","last_active":"2024-04-26T01:51:38.738288+0000","last_peered":"2024-04-26T01:51:38.738288+0000","last_clean":"2024-04-26T01:51:38.738288+0000","last_became_active":"2024-04-26T01:47:27.137059+0000","last_became_peered":"2024-04-26T01:47:27.137059+0000","last_unstale":"2024-04-26T01:51:38.738288+0000","last_undegraded":"2024-04-26T01:51:38.738288+0000","last_fullsized":"2024-04-26T01:51:38.738288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:20:25.004074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581004+0000","last_change":"2024-04-26T01:48:22.405071+0000","last_active":"2024-04-26T01:51:39.581004+0000","last_peered":"2024-04-26T01:51:39.581004+0000","last_clean":"2024-04-26T01:51:39.581004+0000","last_became_active":"2024-04-26T01:48:22.401282+0000","last_became_peered":"2024-04-26T01:48:22.401282+0000","last_unstale":"2024-04-26T01:51:39.581004+0000","last_undegraded":"2024-04-26T01:51:39.581004+0000","last_fullsized":"2024-04-26T01:51:39.581004+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:24:53.628014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973029+0000","last_change":"2024-04-26T01:47:27.128253+0000","last_active":"2024-04-26T01:51:38.973029+0000","last_peered":"2024-04-26T01:51:38.973029+0000","last_clean":"2024-04-26T01:51:38.973029+0000","last_became_active":"2024-04-26T01:47:27.127869+0000","last_became_peered":"2024-04-26T01:47:27.127869+0000","last_unstale":"2024-04-26T01:51:38.973029+0000","last_undegraded":"2024-04-26T01:51:38.973029+0000","last_fullsized":"2024-04-26T01:51:38.973029+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:11:23.476938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.976946+0000","last_change":"2024-04-26T01:47:23.136862+0000","last_active":"2024-04-26T01:51:38.976946+0000","last_peered":"2024-04-26T01:51:38.976946+0000","last_clean":"2024-04-26T01:51:38.976946+0000","last_became_active":"2024-04-26T01:47:23.136380+0000","last_became_peered":"2024-04-26T01:47:23.136380+0000","last_unstale":"2024-04-26T01:51:38.976946+0000","last_undegraded":"2024-04-26T01:51:38.976946+0000","last_fullsized":"2024-04-26T01:51:38.976946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:39:18.103559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973500+0000","last_change":"2024-04-26T01:47:27.113363+0000","last_active":"2024-04-26T01:51:38.973500+0000","last_peered":"2024-04-26T01:51:38.973500+0000","last_clean":"2024-04-26T01:51:38.973500+0000","last_became_active":"2024-04-26T01:47:27.113040+0000","last_became_peered":"2024-04-26T01:47:27.113040+0000","last_unstale":"2024-04-26T01:51:38.973500+0000","last_undegraded":"2024-04-26T01:51:38.973500+0000","last_fullsized":"2024-04-26T01:51:38.973500+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:42:16.428697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'65","reported_seq":125,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:41.425620+0000","last_change":"2024-04-26T01:47:23.067257+0000","last_active":"2024-04-26T01:51:41.425620+0000","last_peered":"2024-04-26T01:51:41.425620+0000","last_clean":"2024-04-26T01:51:41.425620+0000","last_became_active":"2024-04-26T01:47:23.066858+0000","last_became_peered":"2024-04-26T01:47:23.066858+0000","last_unstale":"2024-04-26T01:51:41.425620+0000","last_undegraded":"2024-04-26T01:51:41.425620+0000","last_fullsized":"2024-04-26T01:51:41.425620+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:22:07.848074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973271+0000","last_change":"2024-04-26T01:47:27.125885+0000","last_active":"2024-04-26T01:51:38.973271+0000","last_peered":"2024-04-26T01:51:38.973271+0000","last_clean":"2024-04-26T01:51:38.973271+0000","last_became_active":"2024-04-26T01:47:27.125524+0000","last_became_peered":"2024-04-26T01:47:27.125524+0000","last_unstale":"2024-04-26T01:51:38.973271+0000","last_undegraded":"2024-04-26T01:51:38.973271+0000","last_fullsized":"2024-04-26T01:51:38.973271+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:39.282023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672262+0000","last_change":"2024-04-26T01:47:23.071051+0000","last_active":"2024-04-26T01:51:38.672262+0000","last_peered":"2024-04-26T01:51:38.672262+0000","last_clean":"2024-04-26T01:51:38.672262+0000","last_became_active":"2024-04-26T01:47:23.069524+0000","last_became_peered":"2024-04-26T01:47:23.069524+0000","last_unstale":"2024-04-26T01:51:38.672262+0000","last_undegraded":"2024-04-26T01:51:38.672262+0000","last_fullsized":"2024-04-26T01:51:38.672262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:24:57.643100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973498+0000","last_change":"2024-04-26T01:47:27.114325+0000","last_active":"2024-04-26T01:51:38.973498+0000","last_peered":"2024-04-26T01:51:38.973498+0000","last_clean":"2024-04-26T01:51:38.973498+0000","last_became_active":"2024-04-26T01:47:27.113952+0000","last_became_peered":"2024-04-26T01:47:27.113952+0000","last_unstale":"2024-04-26T01:51:38.973498+0000","last_undegraded":"2024-04-26T01:51:38.973498+0000","last_fullsized":"2024-04-26T01:51:38.973498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:39:19.167175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581077+0000","last_change":"2024-04-26T01:47:27.118709+0000","last_active":"2024-04-26T01:51:39.581077+0000","last_peered":"2024-04-26T01:51:39.581077+0000","last_clean":"2024-04-26T01:51:39.581077+0000","last_became_active":"2024-04-26T01:47:27.118341+0000","last_became_peered":"2024-04-26T01:47:27.118341+0000","last_unstale":"2024-04-26T01:51:39.581077+0000","last_undegraded":"2024-04-26T01:51:39.581077+0000","last_fullsized":"2024-04-26T01:51:39.581077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:13.736180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.976759+0000","last_change":"2024-04-26T01:47:27.119239+0000","last_active":"2024-04-26T01:51:38.976759+0000","last_peered":"2024-04-26T01:51:38.976759+0000","last_clean":"2024-04-26T01:51:38.976759+0000","last_became_active":"2024-04-26T01:47:27.118877+0000","last_became_peered":"2024-04-26T01:47:27.118877+0000","last_unstale":"2024-04-26T01:51:38.976759+0000","last_undegraded":"2024-04-26T01:51:38.976759+0000","last_fullsized":"2024-04-26T01:51:38.976759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:04:10.033517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.976632+0000","last_change":"2024-04-26T01:47:27.142848+0000","last_active":"2024-04-26T01:51:38.976632+0000","last_peered":"2024-04-26T01:51:38.976632+0000","last_clean":"2024-04-26T01:51:38.976632+0000","last_became_active":"2024-04-26T01:47:27.142580+0000","last_became_peered":"2024-04-26T01:47:27.142580+0000","last_unstale":"2024-04-26T01:51:38.976632+0000","last_undegraded":"2024-04-26T01:51:38.976632+0000","last_fullsized":"2024-04-26T01:51:38.976632+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:15:46.923922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.227787+0000","last_change":"2024-04-26T01:47:27.134817+0000","last_active":"2024-04-26T01:51:39.227787+0000","last_peered":"2024-04-26T01:51:39.227787+0000","last_clean":"2024-04-26T01:51:39.227787+0000","last_became_active":"2024-04-26T01:47:27.134024+0000","last_became_peered":"2024-04-26T01:47:27.134024+0000","last_unstale":"2024-04-26T01:51:39.227787+0000","last_undegraded":"2024-04-26T01:51:39.227787+0000","last_fullsized":"2024-04-26T01:51:39.227787+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:56.746427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672241+0000","last_change":"2024-04-26T01:47:27.143137+0000","last_active":"2024-04-26T01:51:38.672241+0000","last_peered":"2024-04-26T01:51:38.672241+0000","last_clean":"2024-04-26T01:51:38.672241+0000","last_became_active":"2024-04-26T01:47:27.142625+0000","last_became_peered":"2024-04-26T01:47:27.142625+0000","last_unstale":"2024-04-26T01:51:38.672241+0000","last_undegraded":"2024-04-26T01:51:38.672241+0000","last_fullsized":"2024-04-26T01:51:38.672241+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:42:50.468382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581247+0000","last_change":"2024-04-26T01:48:22.749582+0000","last_active":"2024-04-26T01:51:39.581247+0000","last_peered":"2024-04-26T01:51:39.581247+0000","last_clean":"2024-04-26T01:51:39.581247+0000","last_became_active":"2024-04-26T01:48:22.749191+0000","last_became_peered":"2024-04-26T01:48:22.749191+0000","last_unstale":"2024-04-26T01:51:39.581247+0000","last_undegraded":"2024-04-26T01:51:39.581247+0000","last_fullsized":"2024-04-26T01:51:39.581247+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:14.614842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.977078+0000","last_change":"2024-04-26T01:47:27.123964+0000","last_active":"2024-04-26T01:51:38.977078+0000","last_peered":"2024-04-26T01:51:38.977078+0000","last_clean":"2024-04-26T01:51:38.977078+0000","last_became_active":"2024-04-26T01:47:27.123552+0000","last_became_peered":"2024-04-26T01:47:27.123552+0000","last_unstale":"2024-04-26T01:51:38.977078+0000","last_undegraded":"2024-04-26T01:51:38.977078+0000","last_fullsized":"2024-04-26T01:51:38.977078+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:33:12.566105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.738482+0000","last_change":"2024-04-26T01:48:22.412621+0000","last_active":"2024-04-26T01:51:38.738482+0000","last_peered":"2024-04-26T01:51:38.738482+0000","last_clean":"2024-04-26T01:51:38.738482+0000","last_became_active":"2024-04-26T01:48:22.411430+0000","last_became_peered":"2024-04-26T01:48:22.411430+0000","last_unstale":"2024-04-26T01:51:38.738482+0000","last_undegraded":"2024-04-26T01:51:38.738482+0000","last_fullsized":"2024-04-26T01:51:38.738482+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:34.152338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672255+0000","last_change":"2024-04-26T01:47:27.131437+0000","last_active":"2024-04-26T01:51:38.672255+0000","last_peered":"2024-04-26T01:51:38.672255+0000","last_clean":"2024-04-26T01:51:38.672255+0000","last_became_active":"2024-04-26T01:47:27.131138+0000","last_became_peered":"2024-04-26T01:47:27.131138+0000","last_unstale":"2024-04-26T01:51:38.672255+0000","last_undegraded":"2024-04-26T01:51:38.672255+0000","last_fullsized":"2024-04-26T01:51:38.672255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:55:39.132972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737796+0000","last_change":"2024-04-26T01:47:27.120928+0000","last_active":"2024-04-26T01:51:38.737796+0000","last_peered":"2024-04-26T01:51:38.737796+0000","last_clean":"2024-04-26T01:51:38.737796+0000","last_became_active":"2024-04-26T01:47:27.120012+0000","last_became_peered":"2024-04-26T01:47:27.120012+0000","last_unstale":"2024-04-26T01:51:38.737796+0000","last_undegraded":"2024-04-26T01:51:38.737796+0000","last_fullsized":"2024-04-26T01:51:38.737796+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:19:44.209876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.980215+0000","last_change":"2024-04-26T01:47:27.138766+0000","last_active":"2024-04-26T01:51:38.980215+0000","last_peered":"2024-04-26T01:51:38.980215+0000","last_clean":"2024-04-26T01:51:38.980215+0000","last_became_active":"2024-04-26T01:47:27.138413+0000","last_became_peered":"2024-04-26T01:47:27.138413+0000","last_unstale":"2024-04-26T01:51:38.980215+0000","last_undegraded":"2024-04-26T01:51:38.980215+0000","last_fullsized":"2024-04-26T01:51:38.980215+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:51:13.117307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.671912+0000","last_change":"2024-04-26T01:47:27.141338+0000","last_active":"2024-04-26T01:51:38.671912+0000","last_peered":"2024-04-26T01:51:38.671912+0000","last_clean":"2024-04-26T01:51:38.671912+0000","last_became_active":"2024-04-26T01:47:27.140825+0000","last_became_peered":"2024-04-26T01:47:27.140825+0000","last_unstale":"2024-04-26T01:51:38.671912+0000","last_undegraded":"2024-04-26T01:51:38.671912+0000","last_fullsized":"2024-04-26T01:51:38.671912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:12:08.680876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581041+0000","last_change":"2024-04-26T01:47:27.114919+0000","last_active":"2024-04-26T01:51:39.581041+0000","last_peered":"2024-04-26T01:51:39.581041+0000","last_clean":"2024-04-26T01:51:39.581041+0000","last_became_active":"2024-04-26T01:47:27.114528+0000","last_became_peered":"2024-04-26T01:47:27.114528+0000","last_unstale":"2024-04-26T01:51:39.581041+0000","last_undegraded":"2024-04-26T01:51:39.581041+0000","last_fullsized":"2024-04-26T01:51:39.581041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:15:25.620566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":797827,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1789,"num_read_kb":1109,"num_write":428,"num_write_kb":2361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2617344,"data_stored":2393481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":315,"ondisk_log_size":315,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318338,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":499332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579975,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29152,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716000,"statfs":{"total":95995035648,"available":95965184000,"internally_reserved":0,"allocated":2064384,"data_stored":1488512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743501,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28420,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716732,"statfs":{"total":95995035648,"available":95965933568,"internally_reserved":0,"allocated":1437696,"data_stored":892784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005141,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36740,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708412,"statfs":{"total":95995035648,"available":95957413888,"internally_reserved":0,"allocated":1437696,"data_stored":892704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299480,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1093824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463006,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":498957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1101824,"data_stored":566202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":500228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":568222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182511,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28868,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716284,"statfs":{"total":95995035648,"available":95965474816,"internally_reserved":0,"allocated":1773568,"data_stored":1220716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920873,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":501521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724646,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37392,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707760,"statfs":{"total":95995035648,"available":95956746240,"internally_reserved":0,"allocated":1982464,"data_stored":1419725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":330490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":401797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:51:49.579 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T01:51:49.580 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:51:50.099 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:50.271 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:50 smithi001 ceph-mon[27822]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-04-26T01:51:50.271 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:50 smithi001 ceph-mon[27822]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:51:50.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:50 smithi173 ceph-mon[32553]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-04-26T01:51:50.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:50 smithi173 ceph-mon[32553]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:51:50.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:50 smithi097 ceph-mon[32853]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-04-26T01:51:50.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:50 smithi097 ceph-mon[32853]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:51:52.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.283+0000 7fc9d4c00700 1 -- 172.21.15.1:0/1834547925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9d006b0f0 msgr2=0x7fc9d010f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:52.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.283+0000 7fc9d4c00700 1 --2- 172.21.15.1:0/1834547925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9d006b0f0 0x7fc9d010f900 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800b0a0 tx=0x7fc9c800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.282 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:52 smithi001 ceph-mon[27822]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-04-26T01:51:52.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.283+0000 7fc9d4c00700 1 -- 172.21.15.1:0/1834547925 shutdown_connections 2024-04-26T01:51:52.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.283+0000 7fc9d4c00700 1 --2- 172.21.15.1:0/1834547925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9d006b0f0 0x7fc9d010f900 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.283+0000 7fc9d4c00700 1 --2- 172.21.15.1:0/1834547925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9d0108a30 0x7fc9d0108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.283+0000 7fc9d4c00700 1 --2- 172.21.15.1:0/1834547925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9d006a750 0x7fc9d006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.283+0000 7fc9d4c00700 1 -- 172.21.15.1:0/1834547925 >> 172.21.15.1:0/1834547925 conn(0x7fc9d00788f0 msgr2=0x7fc9d0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:52.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.283+0000 7fc9d4c00700 1 -- 172.21.15.1:0/1834547925 shutdown_connections 2024-04-26T01:51:52.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.283+0000 7fc9d4c00700 1 -- 172.21.15.1:0/1834547925 wait complete. 2024-04-26T01:51:52.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.284+0000 7fc9d4c00700 1 Processor -- start 2024-04-26T01:51:52.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.284+0000 7fc9d4c00700 1 -- start start 2024-04-26T01:51:52.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.285+0000 7fc9d4c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9d006a750 0x7fc9d019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:52.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.285+0000 7fc9d4c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9d006b0f0 0x7fc9d019b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:52.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.285+0000 7fc9d4c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9d0108a30 0x7fc9d019fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:52.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.285+0000 7fc9d4c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0112c20 con 0x7fc9d0108a30 2024-04-26T01:51:52.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.285+0000 7fc9d4c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0112aa0 con 0x7fc9d006a750 2024-04-26T01:51:52.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.285+0000 7fc9d4c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0112da0 con 0x7fc9d006b0f0 2024-04-26T01:51:52.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.285+0000 7fc9cd400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9d006b0f0 0x7fc9d019b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:52.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.285+0000 7fc9cde00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9d006a750 0x7fc9d019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:52.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.285+0000 7fc9cd400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9d006b0f0 0x7fc9d019b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:48336/0 (socket says 172.21.15.1:48336) 2024-04-26T01:51:52.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.285+0000 7fc9cd400700 1 -- 172.21.15.1:0/4103774363 learned_addr learned my addr 172.21.15.1:0/4103774363 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:52.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.286+0000 7fc9ce800700 1 --2- 172.21.15.1:0/4103774363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9d0108a30 0x7fc9d019fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:52.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.286+0000 7fc9cde00700 1 -- 172.21.15.1:0/4103774363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9d006b0f0 msgr2=0x7fc9d019b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:52.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.286+0000 7fc9cde00700 1 --2- 172.21.15.1:0/4103774363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9d006b0f0 0x7fc9d019b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.286+0000 7fc9cde00700 1 -- 172.21.15.1:0/4103774363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9d0108a30 msgr2=0x7fc9d019fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:52.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.286+0000 7fc9cde00700 1 --2- 172.21.15.1:0/4103774363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9d0108a30 0x7fc9d019fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.286+0000 7fc9cde00700 1 -- 172.21.15.1:0/4103774363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d01a0310 con 0x7fc9d006a750 2024-04-26T01:51:52.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.286+0000 7fc9ce800700 1 --2- 172.21.15.1:0/4103774363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9d0108a30 0x7fc9d019fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:52.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.287+0000 7fc9cde00700 1 --2- 172.21.15.1:0/4103774363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9d006a750 0x7fc9d019b350 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000e9f0 tx=0x7fc9c000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:52.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.287+0000 7fc9c6a00700 1 -- 172.21.15.1:0/4103774363 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c000c9c0 con 0x7fc9d006a750 2024-04-26T01:51:52.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.287+0000 7fc9c6a00700 1 -- 172.21.15.1:0/4103774363 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9c000cb60 con 0x7fc9d006a750 2024-04-26T01:51:52.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.287+0000 7fc9d4c00700 1 -- 172.21.15.1:0/4103774363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d01a0600 con 0x7fc9d006a750 2024-04-26T01:51:52.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.287+0000 7fc9d4c00700 1 -- 172.21.15.1:0/4103774363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d01a7e80 con 0x7fc9d006a750 2024-04-26T01:51:52.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.287+0000 7fc9c6a00700 1 -- 172.21.15.1:0/4103774363 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0018680 con 0x7fc9d006a750 2024-04-26T01:51:52.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.289+0000 7fc9d4c00700 1 -- 172.21.15.1:0/4103774363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d0069730 con 0x7fc9d006a750 2024-04-26T01:51:52.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.292+0000 7fc9c6a00700 1 -- 172.21.15.1:0/4103774363 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fc9c0010430 con 0x7fc9d006a750 2024-04-26T01:51:52.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.293+0000 7fc9c6a00700 1 --2- 172.21.15.1:0/4103774363 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc9b40a3f50 0x7fc9b40a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:52.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.294+0000 7fc9c6a00700 1 -- 172.21.15.1:0/4103774363 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7fc9c00d0bb0 con 0x7fc9d006a750 2024-04-26T01:51:52.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.294+0000 7fc9cd400700 1 --2- 172.21.15.1:0/4103774363 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc9b40a3f50 0x7fc9b40a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:52.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.295+0000 7fc9cd400700 1 --2- 172.21.15.1:0/4103774363 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc9b40a3f50 0x7fc9b40a6410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0078ef0 tx=0x7fc9bc009260 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:52.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.298+0000 7fc9c6a00700 1 -- 172.21.15.1:0/4103774363 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc9c009cfa0 con 0x7fc9d006a750 2024-04-26T01:51:52.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:52 smithi173 ceph-mon[32553]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-04-26T01:51:52.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:52 smithi097 ceph-mon[32853]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-04-26T01:51:52.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.524+0000 7fc9d4c00700 1 -- 172.21.15.1:0/4103774363 --> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc9d0062e80 con 0x7fc9b40a3f50 2024-04-26T01:51:52.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.537+0000 7fc9c6a00700 1 -- 172.21.15.1:0/4103774363 <== mgr.25087 v2:172.21.15.1:6800/275906565 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257772 (secure 0 0 0) 0x7fc9d0062e80 con 0x7fc9b40a3f50 2024-04-26T01:51:52.540 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:51:52.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.544+0000 7fc9d4c00700 1 -- 172.21.15.1:0/4103774363 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc9b40a3f50 msgr2=0x7fc9b40a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:52.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.544+0000 7fc9d4c00700 1 --2- 172.21.15.1:0/4103774363 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc9b40a3f50 0x7fc9b40a6410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0078ef0 tx=0x7fc9bc009260 comp rx=0 tx=0).stop 2024-04-26T01:51:52.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.544+0000 7fc9d4c00700 1 -- 172.21.15.1:0/4103774363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9d006a750 msgr2=0x7fc9d019b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:52.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.544+0000 7fc9d4c00700 1 --2- 172.21.15.1:0/4103774363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9d006a750 0x7fc9d019b350 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000e9f0 tx=0x7fc9c000ed00 comp rx=0 tx=0).stop 2024-04-26T01:51:52.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.544+0000 7fc9d4c00700 1 -- 172.21.15.1:0/4103774363 shutdown_connections 2024-04-26T01:51:52.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.544+0000 7fc9d4c00700 1 --2- 172.21.15.1:0/4103774363 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc9b40a3f50 0x7fc9b40a6410 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.544+0000 7fc9d4c00700 1 --2- 172.21.15.1:0/4103774363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9d0108a30 0x7fc9d019fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.544+0000 7fc9d4c00700 1 --2- 172.21.15.1:0/4103774363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9d006a750 0x7fc9d019b350 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.544+0000 7fc9d4c00700 1 --2- 172.21.15.1:0/4103774363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9d006b0f0 0x7fc9d019b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:52.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.544+0000 7fc9d4c00700 1 -- 172.21.15.1:0/4103774363 >> 172.21.15.1:0/4103774363 conn(0x7fc9d00788f0 msgr2=0x7fc9d010df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:52.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.545+0000 7fc9d4c00700 1 -- 172.21.15.1:0/4103774363 shutdown_connections 2024-04-26T01:51:52.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:52.545+0000 7fc9d4c00700 1 -- 172.21.15.1:0/4103774363 wait complete. 2024-04-26T01:51:52.544 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-04-26T01:51:53.484 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-04-26T01:51:50.693152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1388429,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":2057,"num_read_kb":1457,"num_write":631,"num_write_kb":5301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":965,"ondisk_log_size":965,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365728,"kb_used_data":16320,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124576096,"statfs":{"total":1151940427776,"available":1151565922304,"internally_reserved":0,"allocated":16711680,"data_stored":10150287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":200325,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":419,"num_read_kb":478,"num_write":154,"num_write_kb":922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.953251"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581382+0000","last_change":"2024-04-26T01:47:27.140418+0000","last_active":"2024-04-26T01:51:39.581382+0000","last_peered":"2024-04-26T01:51:39.581382+0000","last_clean":"2024-04-26T01:51:39.581382+0000","last_became_active":"2024-04-26T01:47:27.140017+0000","last_became_peered":"2024-04-26T01:47:27.140017+0000","last_unstale":"2024-04-26T01:51:39.581382+0000","last_undegraded":"2024-04-26T01:51:39.581382+0000","last_fullsized":"2024-04-26T01:51:39.581382+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:46:08.621222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.226726+0000","last_change":"2024-04-26T01:47:27.138746+0000","last_active":"2024-04-26T01:51:39.226726+0000","last_peered":"2024-04-26T01:51:39.226726+0000","last_clean":"2024-04-26T01:51:39.226726+0000","last_became_active":"2024-04-26T01:47:27.136163+0000","last_became_peered":"2024-04-26T01:47:27.136163+0000","last_unstale":"2024-04-26T01:51:39.226726+0000","last_undegraded":"2024-04-26T01:51:39.226726+0000","last_fullsized":"2024-04-26T01:51:39.226726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:01:36.195332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.226469+0000","last_change":"2024-04-26T01:47:27.127017+0000","last_active":"2024-04-26T01:51:39.226469+0000","last_peered":"2024-04-26T01:51:39.226469+0000","last_clean":"2024-04-26T01:51:39.226469+0000","last_became_active":"2024-04-26T01:47:27.126610+0000","last_became_peered":"2024-04-26T01:47:27.126610+0000","last_unstale":"2024-04-26T01:51:39.226469+0000","last_undegraded":"2024-04-26T01:51:39.226469+0000","last_fullsized":"2024-04-26T01:51:39.226469+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:05:12.784486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737519+0000","last_change":"2024-04-26T01:47:27.117640+0000","last_active":"2024-04-26T01:51:38.737519+0000","last_peered":"2024-04-26T01:51:38.737519+0000","last_clean":"2024-04-26T01:51:38.737519+0000","last_became_active":"2024-04-26T01:47:27.117266+0000","last_became_peered":"2024-04-26T01:47:27.117266+0000","last_unstale":"2024-04-26T01:51:38.737519+0000","last_undegraded":"2024-04-26T01:51:38.737519+0000","last_fullsized":"2024-04-26T01:51:38.737519+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:26:57.824560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.760037+0000","last_change":"2024-04-26T01:47:27.128185+0000","last_active":"2024-04-26T01:51:38.760037+0000","last_peered":"2024-04-26T01:51:38.760037+0000","last_clean":"2024-04-26T01:51:38.760037+0000","last_became_active":"2024-04-26T01:47:27.127794+0000","last_became_peered":"2024-04-26T01:47:27.127794+0000","last_unstale":"2024-04-26T01:51:38.760037+0000","last_undegraded":"2024-04-26T01:51:38.760037+0000","last_fullsized":"2024-04-26T01:51:38.760037+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:09:52.774007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672330+0000","last_change":"2024-04-26T01:48:22.748601+0000","last_active":"2024-04-26T01:51:38.672330+0000","last_peered":"2024-04-26T01:51:38.672330+0000","last_clean":"2024-04-26T01:51:38.672330+0000","last_became_active":"2024-04-26T01:48:22.748229+0000","last_became_peered":"2024-04-26T01:48:22.748229+0000","last_unstale":"2024-04-26T01:51:38.672330+0000","last_undegraded":"2024-04-26T01:51:38.672330+0000","last_fullsized":"2024-04-26T01:51:38.672330+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:46:30.433063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.972514+0000","last_change":"2024-04-26T01:47:27.123786+0000","last_active":"2024-04-26T01:51:38.972514+0000","last_peered":"2024-04-26T01:51:38.972514+0000","last_clean":"2024-04-26T01:51:38.972514+0000","last_became_active":"2024-04-26T01:47:27.123473+0000","last_became_peered":"2024-04-26T01:47:27.123473+0000","last_unstale":"2024-04-26T01:51:38.972514+0000","last_undegraded":"2024-04-26T01:51:38.972514+0000","last_fullsized":"2024-04-26T01:51:38.972514+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:48:55.031147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737899+0000","last_change":"2024-04-26T01:47:27.118756+0000","last_active":"2024-04-26T01:51:38.737899+0000","last_peered":"2024-04-26T01:51:38.737899+0000","last_clean":"2024-04-26T01:51:38.737899+0000","last_became_active":"2024-04-26T01:47:27.118406+0000","last_became_peered":"2024-04-26T01:47:27.118406+0000","last_unstale":"2024-04-26T01:51:38.737899+0000","last_undegraded":"2024-04-26T01:51:38.737899+0000","last_fullsized":"2024-04-26T01:51:38.737899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:05.793682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.980594+0000","last_change":"2024-04-26T01:47:27.117622+0000","last_active":"2024-04-26T01:51:38.980594+0000","last_peered":"2024-04-26T01:51:38.980594+0000","last_clean":"2024-04-26T01:51:38.980594+0000","last_became_active":"2024-04-26T01:47:27.115206+0000","last_became_peered":"2024-04-26T01:47:27.115206+0000","last_unstale":"2024-04-26T01:51:38.980594+0000","last_undegraded":"2024-04-26T01:51:38.980594+0000","last_fullsized":"2024-04-26T01:51:38.980594+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:13:17.866630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672921+0000","last_change":"2024-04-26T01:47:27.119524+0000","last_active":"2024-04-26T01:51:38.672921+0000","last_peered":"2024-04-26T01:51:38.672921+0000","last_clean":"2024-04-26T01:51:38.672921+0000","last_became_active":"2024-04-26T01:47:27.119008+0000","last_became_peered":"2024-04-26T01:47:27.119008+0000","last_unstale":"2024-04-26T01:51:38.672921+0000","last_undegraded":"2024-04-26T01:51:38.672921+0000","last_fullsized":"2024-04-26T01:51:38.672921+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:37:31.894486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973282+0000","last_change":"2024-04-26T01:47:27.122361+0000","last_active":"2024-04-26T01:51:38.973282+0000","last_peered":"2024-04-26T01:51:38.973282+0000","last_clean":"2024-04-26T01:51:38.973282+0000","last_became_active":"2024-04-26T01:47:27.121748+0000","last_became_peered":"2024-04-26T01:47:27.121748+0000","last_unstale":"2024-04-26T01:51:38.973282+0000","last_undegraded":"2024-04-26T01:51:38.973282+0000","last_fullsized":"2024-04-26T01:51:38.973282+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:18.680909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737482+0000","last_change":"2024-04-26T01:47:27.116833+0000","last_active":"2024-04-26T01:51:38.737482+0000","last_peered":"2024-04-26T01:51:38.737482+0000","last_clean":"2024-04-26T01:51:38.737482+0000","last_became_active":"2024-04-26T01:47:27.116508+0000","last_became_peered":"2024-04-26T01:47:27.116508+0000","last_unstale":"2024-04-26T01:51:38.737482+0000","last_undegraded":"2024-04-26T01:51:38.737482+0000","last_fullsized":"2024-04-26T01:51:38.737482+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:27:45.864521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.671761+0000","last_change":"2024-04-26T01:47:27.125342+0000","last_active":"2024-04-26T01:51:38.671761+0000","last_peered":"2024-04-26T01:51:38.671761+0000","last_clean":"2024-04-26T01:51:38.671761+0000","last_became_active":"2024-04-26T01:47:27.124862+0000","last_became_peered":"2024-04-26T01:47:27.124862+0000","last_unstale":"2024-04-26T01:51:38.671761+0000","last_undegraded":"2024-04-26T01:51:38.671761+0000","last_fullsized":"2024-04-26T01:51:38.671761+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:48.654711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.227404+0000","last_change":"2024-04-26T01:47:27.127365+0000","last_active":"2024-04-26T01:51:39.227404+0000","last_peered":"2024-04-26T01:51:39.227404+0000","last_clean":"2024-04-26T01:51:39.227404+0000","last_became_active":"2024-04-26T01:47:27.126868+0000","last_became_peered":"2024-04-26T01:47:27.126868+0000","last_unstale":"2024-04-26T01:51:39.227404+0000","last_undegraded":"2024-04-26T01:51:39.227404+0000","last_fullsized":"2024-04-26T01:51:39.227404+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:07:46.400333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.226295+0000","last_change":"2024-04-26T01:47:27.135584+0000","last_active":"2024-04-26T01:51:39.226295+0000","last_peered":"2024-04-26T01:51:39.226295+0000","last_clean":"2024-04-26T01:51:39.226295+0000","last_became_active":"2024-04-26T01:47:27.134849+0000","last_became_peered":"2024-04-26T01:47:27.134849+0000","last_unstale":"2024-04-26T01:51:39.226295+0000","last_undegraded":"2024-04-26T01:51:39.226295+0000","last_fullsized":"2024-04-26T01:51:39.226295+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:57:48.821595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.980000+0000","last_change":"2024-04-26T01:47:27.136182+0000","last_active":"2024-04-26T01:51:38.980000+0000","last_peered":"2024-04-26T01:51:38.980000+0000","last_clean":"2024-04-26T01:51:38.980000+0000","last_became_active":"2024-04-26T01:47:27.135787+0000","last_became_peered":"2024-04-26T01:47:27.135787+0000","last_unstale":"2024-04-26T01:51:38.980000+0000","last_undegraded":"2024-04-26T01:51:38.980000+0000","last_fullsized":"2024-04-26T01:51:38.980000+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:38:14.275898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581398+0000","last_change":"2024-04-26T01:47:27.140700+0000","last_active":"2024-04-26T01:51:39.581398+0000","last_peered":"2024-04-26T01:51:39.581398+0000","last_clean":"2024-04-26T01:51:39.581398+0000","last_became_active":"2024-04-26T01:47:27.140025+0000","last_became_peered":"2024-04-26T01:47:27.140025+0000","last_unstale":"2024-04-26T01:51:39.581398+0000","last_undegraded":"2024-04-26T01:51:39.581398+0000","last_fullsized":"2024-04-26T01:51:39.581398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:04:50.465236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.794694+0000","last_change":"2024-04-26T01:47:27.117256+0000","last_active":"2024-04-26T01:51:38.794694+0000","last_peered":"2024-04-26T01:51:38.794694+0000","last_clean":"2024-04-26T01:51:38.794694+0000","last_became_active":"2024-04-26T01:47:27.116825+0000","last_became_peered":"2024-04-26T01:47:27.116825+0000","last_unstale":"2024-04-26T01:51:38.794694+0000","last_undegraded":"2024-04-26T01:51:38.794694+0000","last_fullsized":"2024-04-26T01:51:38.794694+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:23.313712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.977034+0000","last_change":"2024-04-26T01:47:27.130233+0000","last_active":"2024-04-26T01:51:38.977034+0000","last_peered":"2024-04-26T01:51:38.977034+0000","last_clean":"2024-04-26T01:51:38.977034+0000","last_became_active":"2024-04-26T01:47:27.129852+0000","last_became_peered":"2024-04-26T01:47:27.129852+0000","last_unstale":"2024-04-26T01:51:38.977034+0000","last_undegraded":"2024-04-26T01:51:38.977034+0000","last_fullsized":"2024-04-26T01:51:38.977034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:49:01.407264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.794816+0000","last_change":"2024-04-26T01:47:23.068477+0000","last_active":"2024-04-26T01:51:38.794816+0000","last_peered":"2024-04-26T01:51:38.794816+0000","last_clean":"2024-04-26T01:51:38.794816+0000","last_became_active":"2024-04-26T01:47:23.068061+0000","last_became_peered":"2024-04-26T01:47:23.068061+0000","last_unstale":"2024-04-26T01:51:38.794816+0000","last_undegraded":"2024-04-26T01:51:38.794816+0000","last_fullsized":"2024-04-26T01:51:38.794816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:02.153724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.738398+0000","last_change":"2024-04-26T01:47:27.130385+0000","last_active":"2024-04-26T01:51:38.738398+0000","last_peered":"2024-04-26T01:51:38.738398+0000","last_clean":"2024-04-26T01:51:38.738398+0000","last_became_active":"2024-04-26T01:47:27.129912+0000","last_became_peered":"2024-04-26T01:47:27.129912+0000","last_unstale":"2024-04-26T01:51:38.738398+0000","last_undegraded":"2024-04-26T01:51:38.738398+0000","last_fullsized":"2024-04-26T01:51:38.738398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:26:03.010544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.738555+0000","last_change":"2024-04-26T01:47:23.110806+0000","last_active":"2024-04-26T01:51:38.738555+0000","last_peered":"2024-04-26T01:51:38.738555+0000","last_clean":"2024-04-26T01:51:38.738555+0000","last_became_active":"2024-04-26T01:47:23.110441+0000","last_became_peered":"2024-04-26T01:47:23.110441+0000","last_unstale":"2024-04-26T01:51:38.738555+0000","last_undegraded":"2024-04-26T01:51:38.738555+0000","last_fullsized":"2024-04-26T01:51:38.738555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:39.239172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.227010+0000","last_change":"2024-04-26T01:47:27.138628+0000","last_active":"2024-04-26T01:51:39.227010+0000","last_peered":"2024-04-26T01:51:39.227010+0000","last_clean":"2024-04-26T01:51:39.227010+0000","last_became_active":"2024-04-26T01:47:27.130483+0000","last_became_peered":"2024-04-26T01:47:27.130483+0000","last_unstale":"2024-04-26T01:51:39.227010+0000","last_undegraded":"2024-04-26T01:51:39.227010+0000","last_fullsized":"2024-04-26T01:51:39.227010+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:40:41.242856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.760376+0000","last_change":"2024-04-26T01:47:23.060066+0000","last_active":"2024-04-26T01:51:38.760376+0000","last_peered":"2024-04-26T01:51:38.760376+0000","last_clean":"2024-04-26T01:51:38.760376+0000","last_became_active":"2024-04-26T01:47:23.059526+0000","last_became_peered":"2024-04-26T01:47:23.059526+0000","last_unstale":"2024-04-26T01:51:38.760376+0000","last_undegraded":"2024-04-26T01:51:38.760376+0000","last_fullsized":"2024-04-26T01:51:38.760376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:28:22.623695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.980142+0000","last_change":"2024-04-26T01:47:27.128343+0000","last_active":"2024-04-26T01:51:38.980142+0000","last_peered":"2024-04-26T01:51:38.980142+0000","last_clean":"2024-04-26T01:51:38.980142+0000","last_became_active":"2024-04-26T01:47:27.127920+0000","last_became_peered":"2024-04-26T01:47:27.127920+0000","last_unstale":"2024-04-26T01:51:38.980142+0000","last_undegraded":"2024-04-26T01:51:38.980142+0000","last_fullsized":"2024-04-26T01:51:38.980142+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:06.725033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.760152+0000","last_change":"2024-04-26T01:47:23.057783+0000","last_active":"2024-04-26T01:51:38.760152+0000","last_peered":"2024-04-26T01:51:38.760152+0000","last_clean":"2024-04-26T01:51:38.760152+0000","last_became_active":"2024-04-26T01:47:23.057391+0000","last_became_peered":"2024-04-26T01:47:23.057391+0000","last_unstale":"2024-04-26T01:51:38.760152+0000","last_undegraded":"2024-04-26T01:51:38.760152+0000","last_fullsized":"2024-04-26T01:51:38.760152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:10:10.050787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672824+0000","last_change":"2024-04-26T01:47:27.116458+0000","last_active":"2024-04-26T01:51:38.672824+0000","last_peered":"2024-04-26T01:51:38.672824+0000","last_clean":"2024-04-26T01:51:38.672824+0000","last_became_active":"2024-04-26T01:47:27.116084+0000","last_became_peered":"2024-04-26T01:47:27.116084+0000","last_unstale":"2024-04-26T01:51:38.672824+0000","last_undegraded":"2024-04-26T01:51:38.672824+0000","last_fullsized":"2024-04-26T01:51:38.672824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:06:34.517478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"93'7","reported_seq":280,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:48.617225+0000","last_change":"2024-04-26T01:47:23.060657+0000","last_active":"2024-04-26T01:51:48.617225+0000","last_peered":"2024-04-26T01:51:48.617225+0000","last_clean":"2024-04-26T01:51:48.617225+0000","last_became_active":"2024-04-26T01:47:23.060294+0000","last_became_peered":"2024-04-26T01:47:23.060294+0000","last_unstale":"2024-04-26T01:51:48.617225+0000","last_undegraded":"2024-04-26T01:51:48.617225+0000","last_fullsized":"2024-04-26T01:51:48.617225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:33.979717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":244,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.759940+0000","last_change":"2024-04-26T01:47:27.120587+0000","last_active":"2024-04-26T01:51:38.759940+0000","last_peered":"2024-04-26T01:51:38.759940+0000","last_clean":"2024-04-26T01:51:38.759940+0000","last_became_active":"2024-04-26T01:47:27.120236+0000","last_became_peered":"2024-04-26T01:47:27.120236+0000","last_unstale":"2024-04-26T01:51:38.759940+0000","last_undegraded":"2024-04-26T01:51:38.759940+0000","last_fullsized":"2024-04-26T01:51:38.759940+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:30:38.488741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672250+0000","last_change":"2024-04-26T01:47:23.069395+0000","last_active":"2024-04-26T01:51:38.672250+0000","last_peered":"2024-04-26T01:51:38.672250+0000","last_clean":"2024-04-26T01:51:38.672250+0000","last_became_active":"2024-04-26T01:47:23.069102+0000","last_became_peered":"2024-04-26T01:47:23.069102+0000","last_unstale":"2024-04-26T01:51:38.672250+0000","last_undegraded":"2024-04-26T01:51:38.672250+0000","last_fullsized":"2024-04-26T01:51:38.672250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:45.162604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672487+0000","last_change":"2024-04-26T01:47:27.123859+0000","last_active":"2024-04-26T01:51:38.672487+0000","last_peered":"2024-04-26T01:51:38.672487+0000","last_clean":"2024-04-26T01:51:38.672487+0000","last_became_active":"2024-04-26T01:47:27.123438+0000","last_became_peered":"2024-04-26T01:47:27.123438+0000","last_unstale":"2024-04-26T01:51:38.672487+0000","last_undegraded":"2024-04-26T01:51:38.672487+0000","last_fullsized":"2024-04-26T01:51:38.672487+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:33:53.542809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.671891+0000","last_change":"2024-04-26T01:47:23.071544+0000","last_active":"2024-04-26T01:51:38.671891+0000","last_peered":"2024-04-26T01:51:38.671891+0000","last_clean":"2024-04-26T01:51:38.671891+0000","last_became_active":"2024-04-26T01:47:23.071191+0000","last_became_peered":"2024-04-26T01:47:23.071191+0000","last_unstale":"2024-04-26T01:51:38.671891+0000","last_undegraded":"2024-04-26T01:51:38.671891+0000","last_fullsized":"2024-04-26T01:51:38.671891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:58.889561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737808+0000","last_change":"2024-04-26T01:47:27.131887+0000","last_active":"2024-04-26T01:51:38.737808+0000","last_peered":"2024-04-26T01:51:38.737808+0000","last_clean":"2024-04-26T01:51:38.737808+0000","last_became_active":"2024-04-26T01:47:27.127973+0000","last_became_peered":"2024-04-26T01:47:27.127973+0000","last_unstale":"2024-04-26T01:51:38.737808+0000","last_undegraded":"2024-04-26T01:51:38.737808+0000","last_fullsized":"2024-04-26T01:51:38.737808+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:54:35.305779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672192+0000","last_change":"2024-04-26T01:47:23.061353+0000","last_active":"2024-04-26T01:51:38.672192+0000","last_peered":"2024-04-26T01:51:38.672192+0000","last_clean":"2024-04-26T01:51:38.672192+0000","last_became_active":"2024-04-26T01:47:23.061037+0000","last_became_peered":"2024-04-26T01:47:23.061037+0000","last_unstale":"2024-04-26T01:51:38.672192+0000","last_undegraded":"2024-04-26T01:51:38.672192+0000","last_fullsized":"2024-04-26T01:51:38.672192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:13:59.617969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.794335+0000","last_change":"2024-04-26T01:47:27.122303+0000","last_active":"2024-04-26T01:51:38.794335+0000","last_peered":"2024-04-26T01:51:38.794335+0000","last_clean":"2024-04-26T01:51:38.794335+0000","last_became_active":"2024-04-26T01:47:27.121919+0000","last_became_peered":"2024-04-26T01:47:27.121919+0000","last_unstale":"2024-04-26T01:51:38.794335+0000","last_undegraded":"2024-04-26T01:51:38.794335+0000","last_fullsized":"2024-04-26T01:51:38.794335+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:21.676663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"93'4","reported_seq":273,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:48.616553+0000","last_change":"2024-04-26T01:47:23.057961+0000","last_active":"2024-04-26T01:51:48.616553+0000","last_peered":"2024-04-26T01:51:48.616553+0000","last_clean":"2024-04-26T01:51:48.616553+0000","last_became_active":"2024-04-26T01:47:23.057529+0000","last_became_peered":"2024-04-26T01:47:23.057529+0000","last_unstale":"2024-04-26T01:51:48.616553+0000","last_undegraded":"2024-04-26T01:51:48.616553+0000","last_fullsized":"2024-04-26T01:51:48.616553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:14.420265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.980525+0000","last_change":"2024-04-26T01:47:27.120083+0000","last_active":"2024-04-26T01:51:38.980525+0000","last_peered":"2024-04-26T01:51:38.980525+0000","last_clean":"2024-04-26T01:51:38.980525+0000","last_became_active":"2024-04-26T01:47:27.118410+0000","last_became_peered":"2024-04-26T01:47:27.118410+0000","last_unstale":"2024-04-26T01:51:38.980525+0000","last_undegraded":"2024-04-26T01:51:38.980525+0000","last_fullsized":"2024-04-26T01:51:38.980525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:28.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973354+0000","last_change":"2024-04-26T01:47:23.059221+0000","last_active":"2024-04-26T01:51:38.973354+0000","last_peered":"2024-04-26T01:51:38.973354+0000","last_clean":"2024-04-26T01:51:38.973354+0000","last_became_active":"2024-04-26T01:47:23.058881+0000","last_became_peered":"2024-04-26T01:47:23.058881+0000","last_unstale":"2024-04-26T01:51:38.973354+0000","last_undegraded":"2024-04-26T01:51:38.973354+0000","last_fullsized":"2024-04-26T01:51:38.973354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:30.433696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.226796+0000","last_change":"2024-04-26T01:47:27.136763+0000","last_active":"2024-04-26T01:51:39.226796+0000","last_peered":"2024-04-26T01:51:39.226796+0000","last_clean":"2024-04-26T01:51:39.226796+0000","last_became_active":"2024-04-26T01:47:27.133642+0000","last_became_peered":"2024-04-26T01:47:27.133642+0000","last_unstale":"2024-04-26T01:51:39.226796+0000","last_undegraded":"2024-04-26T01:51:39.226796+0000","last_fullsized":"2024-04-26T01:51:39.226796+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:05.901139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672938+0000","last_change":"2024-04-26T01:47:23.108883+0000","last_active":"2024-04-26T01:51:38.672938+0000","last_peered":"2024-04-26T01:51:38.672938+0000","last_clean":"2024-04-26T01:51:38.672938+0000","last_became_active":"2024-04-26T01:47:23.108499+0000","last_became_peered":"2024-04-26T01:47:23.108499+0000","last_unstale":"2024-04-26T01:51:38.672938+0000","last_undegraded":"2024-04-26T01:51:38.672938+0000","last_fullsized":"2024-04-26T01:51:38.672938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:14:17.555416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.980570+0000","last_change":"2024-04-26T01:48:22.410651+0000","last_active":"2024-04-26T01:51:38.980570+0000","last_peered":"2024-04-26T01:51:38.980570+0000","last_clean":"2024-04-26T01:51:38.980570+0000","last_became_active":"2024-04-26T01:48:22.410359+0000","last_became_peered":"2024-04-26T01:48:22.410359+0000","last_unstale":"2024-04-26T01:51:38.980570+0000","last_undegraded":"2024-04-26T01:51:38.980570+0000","last_fullsized":"2024-04-26T01:51:38.980570+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:53:20.686915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"93'38","reported_seq":561,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:49.630288+0000","last_change":"2024-04-26T01:47:23.054893+0000","last_active":"2024-04-26T01:51:49.630288+0000","last_peered":"2024-04-26T01:51:49.630288+0000","last_clean":"2024-04-26T01:51:49.630288+0000","last_became_active":"2024-04-26T01:47:23.054558+0000","last_became_peered":"2024-04-26T01:47:23.054558+0000","last_unstale":"2024-04-26T01:51:49.630288+0000","last_undegraded":"2024-04-26T01:51:49.630288+0000","last_fullsized":"2024-04-26T01:51:49.630288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:20:59.356841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69972,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":494,"num_read_kb":316,"num_write":39,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.760179+0000","last_change":"2024-04-26T01:47:27.116934+0000","last_active":"2024-04-26T01:51:38.760179+0000","last_peered":"2024-04-26T01:51:38.760179+0000","last_clean":"2024-04-26T01:51:38.760179+0000","last_became_active":"2024-04-26T01:47:27.116527+0000","last_became_peered":"2024-04-26T01:47:27.116527+0000","last_unstale":"2024-04-26T01:51:38.760179+0000","last_undegraded":"2024-04-26T01:51:38.760179+0000","last_fullsized":"2024-04-26T01:51:38.760179+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:09:45.820533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.976702+0000","last_change":"2024-04-26T01:47:23.059032+0000","last_active":"2024-04-26T01:51:38.976702+0000","last_peered":"2024-04-26T01:51:38.976702+0000","last_clean":"2024-04-26T01:51:38.976702+0000","last_became_active":"2024-04-26T01:47:23.057992+0000","last_became_peered":"2024-04-26T01:47:23.057992+0000","last_unstale":"2024-04-26T01:51:38.976702+0000","last_undegraded":"2024-04-26T01:51:38.976702+0000","last_fullsized":"2024-04-26T01:51:38.976702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:46:53.164925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.673476+0000","last_change":"2024-04-26T01:47:27.134574+0000","last_active":"2024-04-26T01:51:38.673476+0000","last_peered":"2024-04-26T01:51:38.673476+0000","last_clean":"2024-04-26T01:51:38.673476+0000","last_became_active":"2024-04-26T01:47:27.134184+0000","last_became_peered":"2024-04-26T01:47:27.134184+0000","last_unstale":"2024-04-26T01:51:38.673476+0000","last_undegraded":"2024-04-26T01:51:38.673476+0000","last_fullsized":"2024-04-26T01:51:38.673476+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:42:26.428014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672306+0000","last_change":"2024-04-26T01:47:23.062649+0000","last_active":"2024-04-26T01:51:38.672306+0000","last_peered":"2024-04-26T01:51:38.672306+0000","last_clean":"2024-04-26T01:51:38.672306+0000","last_became_active":"2024-04-26T01:47:23.062282+0000","last_became_peered":"2024-04-26T01:47:23.062282+0000","last_unstale":"2024-04-26T01:51:38.672306+0000","last_undegraded":"2024-04-26T01:51:38.672306+0000","last_fullsized":"2024-04-26T01:51:38.672306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:38.699105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973219+0000","last_change":"2024-04-26T01:47:27.124720+0000","last_active":"2024-04-26T01:51:38.973219+0000","last_peered":"2024-04-26T01:51:38.973219+0000","last_clean":"2024-04-26T01:51:38.973219+0000","last_became_active":"2024-04-26T01:47:27.124378+0000","last_became_peered":"2024-04-26T01:47:27.124378+0000","last_unstale":"2024-04-26T01:51:38.973219+0000","last_undegraded":"2024-04-26T01:51:38.973219+0000","last_fullsized":"2024-04-26T01:51:38.973219+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:45.462741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"93'9","reported_seq":284,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:47.755218+0000","last_change":"2024-04-26T01:47:23.056305+0000","last_active":"2024-04-26T01:51:47.755218+0000","last_peered":"2024-04-26T01:51:47.755218+0000","last_clean":"2024-04-26T01:51:47.755218+0000","last_became_active":"2024-04-26T01:47:23.055905+0000","last_became_peered":"2024-04-26T01:47:23.055905+0000","last_unstale":"2024-04-26T01:51:47.755218+0000","last_undegraded":"2024-04-26T01:51:47.755218+0000","last_fullsized":"2024-04-26T01:51:47.755218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:00:14.609673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2603,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":247,"num_read_kb":8,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.972618+0000","last_change":"2024-04-26T01:47:27.119880+0000","last_active":"2024-04-26T01:51:38.972618+0000","last_peered":"2024-04-26T01:51:38.972618+0000","last_clean":"2024-04-26T01:51:38.972618+0000","last_became_active":"2024-04-26T01:47:27.119483+0000","last_became_peered":"2024-04-26T01:47:27.119483+0000","last_unstale":"2024-04-26T01:51:38.972618+0000","last_undegraded":"2024-04-26T01:51:38.972618+0000","last_fullsized":"2024-04-26T01:51:38.972618+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:15.523150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":69,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.759939+0000","last_change":"2024-04-26T01:48:22.758774+0000","last_active":"2024-04-26T01:51:38.759939+0000","last_peered":"2024-04-26T01:51:38.759939+0000","last_clean":"2024-04-26T01:51:38.759939+0000","last_became_active":"2024-04-26T01:48:22.747354+0000","last_became_peered":"2024-04-26T01:48:22.747354+0000","last_unstale":"2024-04-26T01:51:38.759939+0000","last_undegraded":"2024-04-26T01:51:38.759939+0000","last_fullsized":"2024-04-26T01:51:38.759939+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:23:15.886774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672382+0000","last_change":"2024-04-26T01:47:27.113508+0000","last_active":"2024-04-26T01:51:38.672382+0000","last_peered":"2024-04-26T01:51:38.672382+0000","last_clean":"2024-04-26T01:51:38.672382+0000","last_became_active":"2024-04-26T01:47:27.113066+0000","last_became_peered":"2024-04-26T01:47:27.113066+0000","last_unstale":"2024-04-26T01:51:38.672382+0000","last_undegraded":"2024-04-26T01:51:38.672382+0000","last_fullsized":"2024-04-26T01:51:38.672382+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:35:37.525486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"93'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.760368+0000","last_change":"2024-04-26T01:47:23.059644+0000","last_active":"2024-04-26T01:51:38.760368+0000","last_peered":"2024-04-26T01:51:38.760368+0000","last_clean":"2024-04-26T01:51:38.760368+0000","last_became_active":"2024-04-26T01:47:23.058966+0000","last_became_peered":"2024-04-26T01:47:23.058966+0000","last_unstale":"2024-04-26T01:51:38.760368+0000","last_undegraded":"2024-04-26T01:51:38.760368+0000","last_fullsized":"2024-04-26T01:51:38.760368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:57:36.475935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581235+0000","last_change":"2024-04-26T01:48:22.749354+0000","last_active":"2024-04-26T01:51:39.581235+0000","last_peered":"2024-04-26T01:51:39.581235+0000","last_clean":"2024-04-26T01:51:39.581235+0000","last_became_active":"2024-04-26T01:48:22.748978+0000","last_became_peered":"2024-04-26T01:48:22.748978+0000","last_unstale":"2024-04-26T01:51:39.581235+0000","last_undegraded":"2024-04-26T01:51:39.581235+0000","last_fullsized":"2024-04-26T01:51:39.581235+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:51:16.087091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.794533+0000","last_change":"2024-04-26T01:47:23.110360+0000","last_active":"2024-04-26T01:51:38.794533+0000","last_peered":"2024-04-26T01:51:38.794533+0000","last_clean":"2024-04-26T01:51:38.794533+0000","last_became_active":"2024-04-26T01:47:23.109744+0000","last_became_peered":"2024-04-26T01:47:23.109744+0000","last_unstale":"2024-04-26T01:51:38.794533+0000","last_undegraded":"2024-04-26T01:51:38.794533+0000","last_fullsized":"2024-04-26T01:51:38.794533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:40:13.658396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672730+0000","last_change":"2024-04-26T01:47:27.125399+0000","last_active":"2024-04-26T01:51:38.672730+0000","last_peered":"2024-04-26T01:51:38.672730+0000","last_clean":"2024-04-26T01:51:38.672730+0000","last_became_active":"2024-04-26T01:47:27.124871+0000","last_became_peered":"2024-04-26T01:47:27.124871+0000","last_unstale":"2024-04-26T01:51:38.672730+0000","last_undegraded":"2024-04-26T01:51:38.672730+0000","last_fullsized":"2024-04-26T01:51:38.672730+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:28:48.285206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973098+0000","last_change":"2024-04-26T01:47:23.067862+0000","last_active":"2024-04-26T01:51:38.973098+0000","last_peered":"2024-04-26T01:51:38.973098+0000","last_clean":"2024-04-26T01:51:38.973098+0000","last_became_active":"2024-04-26T01:47:23.067526+0000","last_became_peered":"2024-04-26T01:47:23.067526+0000","last_unstale":"2024-04-26T01:51:38.973098+0000","last_undegraded":"2024-04-26T01:51:38.973098+0000","last_fullsized":"2024-04-26T01:51:38.973098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:22.668405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672467+0000","last_change":"2024-04-26T01:48:22.748548+0000","last_active":"2024-04-26T01:51:38.672467+0000","last_peered":"2024-04-26T01:51:38.672467+0000","last_clean":"2024-04-26T01:51:38.672467+0000","last_became_active":"2024-04-26T01:48:22.748182+0000","last_became_peered":"2024-04-26T01:48:22.748182+0000","last_unstale":"2024-04-26T01:51:38.672467+0000","last_undegraded":"2024-04-26T01:51:38.672467+0000","last_fullsized":"2024-04-26T01:51:38.672467+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:20:27.596827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672474+0000","last_change":"2024-04-26T01:47:27.118307+0000","last_active":"2024-04-26T01:51:38.672474+0000","last_peered":"2024-04-26T01:51:38.672474+0000","last_clean":"2024-04-26T01:51:38.672474+0000","last_became_active":"2024-04-26T01:47:27.117986+0000","last_became_peered":"2024-04-26T01:47:27.117986+0000","last_unstale":"2024-04-26T01:51:38.672474+0000","last_undegraded":"2024-04-26T01:51:38.672474+0000","last_fullsized":"2024-04-26T01:51:38.672474+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:40.667143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":275,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:47.754497+0000","last_change":"2024-04-26T01:47:23.109973+0000","last_active":"2024-04-26T01:51:47.754497+0000","last_peered":"2024-04-26T01:51:47.754497+0000","last_clean":"2024-04-26T01:51:47.754497+0000","last_became_active":"2024-04-26T01:47:23.109446+0000","last_became_peered":"2024-04-26T01:47:23.109446+0000","last_unstale":"2024-04-26T01:51:47.754497+0000","last_undegraded":"2024-04-26T01:51:47.754497+0000","last_fullsized":"2024-04-26T01:51:47.754497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:49:00.684164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":239,"num_read_kb":239,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.738353+0000","last_change":"2024-04-26T01:47:27.131502+0000","last_active":"2024-04-26T01:51:38.738353+0000","last_peered":"2024-04-26T01:51:38.738353+0000","last_clean":"2024-04-26T01:51:38.738353+0000","last_became_active":"2024-04-26T01:47:27.127697+0000","last_became_peered":"2024-04-26T01:47:27.127697+0000","last_unstale":"2024-04-26T01:51:38.738353+0000","last_undegraded":"2024-04-26T01:51:38.738353+0000","last_fullsized":"2024-04-26T01:51:38.738353+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:39:15.458189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737630+0000","last_change":"2024-04-26T01:47:23.066596+0000","last_active":"2024-04-26T01:51:38.737630+0000","last_peered":"2024-04-26T01:51:38.737630+0000","last_clean":"2024-04-26T01:51:38.737630+0000","last_became_active":"2024-04-26T01:47:23.066216+0000","last_became_peered":"2024-04-26T01:47:23.066216+0000","last_unstale":"2024-04-26T01:51:38.737630+0000","last_undegraded":"2024-04-26T01:51:38.737630+0000","last_fullsized":"2024-04-26T01:51:38.737630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:08:31.544831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"93'148","reported_seq":407,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:40.835064+0000","last_change":"2024-04-26T01:45:54.834501+0000","last_active":"2024-04-26T01:51:40.835064+0000","last_peered":"2024-04-26T01:51:40.835064+0000","last_clean":"2024-04-26T01:51:40.835064+0000","last_became_active":"2024-04-26T01:45:54.833868+0000","last_became_peered":"2024-04-26T01:45:54.833868+0000","last_unstale":"2024-04-26T01:51:40.835064+0000","last_undegraded":"2024-04-26T01:51:40.835064+0000","last_fullsized":"2024-04-26T01:51:40.835064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:42:05.228956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:42:05.228956+0000","last_clean_scrub_stamp":"2024-04-26T01:42:05.228956+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:06:58.383315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.671645+0000","last_change":"2024-04-26T01:47:27.126475+0000","last_active":"2024-04-26T01:51:38.671645+0000","last_peered":"2024-04-26T01:51:38.671645+0000","last_clean":"2024-04-26T01:51:38.671645+0000","last_became_active":"2024-04-26T01:47:27.123760+0000","last_became_peered":"2024-04-26T01:47:27.123760+0000","last_unstale":"2024-04-26T01:51:38.671645+0000","last_undegraded":"2024-04-26T01:51:38.671645+0000","last_fullsized":"2024-04-26T01:51:38.671645+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:38:27.473909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.972839+0000","last_change":"2024-04-26T01:47:23.059810+0000","last_active":"2024-04-26T01:51:38.972839+0000","last_peered":"2024-04-26T01:51:38.972839+0000","last_clean":"2024-04-26T01:51:38.972839+0000","last_became_active":"2024-04-26T01:47:23.059504+0000","last_became_peered":"2024-04-26T01:47:23.059504+0000","last_unstale":"2024-04-26T01:51:38.972839+0000","last_undegraded":"2024-04-26T01:51:38.972839+0000","last_fullsized":"2024-04-26T01:51:38.972839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:50:51.220951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.760303+0000","last_change":"2024-04-26T01:47:23.109460+0000","last_active":"2024-04-26T01:51:38.760303+0000","last_peered":"2024-04-26T01:51:38.760303+0000","last_clean":"2024-04-26T01:51:38.760303+0000","last_became_active":"2024-04-26T01:47:23.109090+0000","last_became_peered":"2024-04-26T01:47:23.109090+0000","last_unstale":"2024-04-26T01:51:38.760303+0000","last_undegraded":"2024-04-26T01:51:38.760303+0000","last_fullsized":"2024-04-26T01:51:38.760303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:56.821312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672774+0000","last_change":"2024-04-26T01:47:27.129537+0000","last_active":"2024-04-26T01:51:38.672774+0000","last_peered":"2024-04-26T01:51:38.672774+0000","last_clean":"2024-04-26T01:51:38.672774+0000","last_became_active":"2024-04-26T01:47:27.128124+0000","last_became_peered":"2024-04-26T01:47:27.128124+0000","last_unstale":"2024-04-26T01:51:38.672774+0000","last_undegraded":"2024-04-26T01:51:38.672774+0000","last_fullsized":"2024-04-26T01:51:38.672774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:14:26.262197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"93'101","reported_seq":223,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:41.934307+0000","last_change":"2024-04-26T01:47:23.064511+0000","last_active":"2024-04-26T01:51:41.934307+0000","last_peered":"2024-04-26T01:51:41.934307+0000","last_clean":"2024-04-26T01:51:41.934307+0000","last_became_active":"2024-04-26T01:47:23.064080+0000","last_became_peered":"2024-04-26T01:47:23.064080+0000","last_unstale":"2024-04-26T01:51:41.934307+0000","last_undegraded":"2024-04-26T01:51:41.934307+0000","last_fullsized":"2024-04-26T01:51:41.934307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:37.562644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":185,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.738658+0000","last_change":"2024-04-26T01:47:27.124057+0000","last_active":"2024-04-26T01:51:38.738658+0000","last_peered":"2024-04-26T01:51:38.738658+0000","last_clean":"2024-04-26T01:51:38.738658+0000","last_became_active":"2024-04-26T01:47:27.123636+0000","last_became_peered":"2024-04-26T01:47:27.123636+0000","last_unstale":"2024-04-26T01:51:38.738658+0000","last_undegraded":"2024-04-26T01:51:38.738658+0000","last_fullsized":"2024-04-26T01:51:38.738658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:56:55.691697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737559+0000","last_change":"2024-04-26T01:47:23.136571+0000","last_active":"2024-04-26T01:51:38.737559+0000","last_peered":"2024-04-26T01:51:38.737559+0000","last_clean":"2024-04-26T01:51:38.737559+0000","last_became_active":"2024-04-26T01:47:23.136205+0000","last_became_peered":"2024-04-26T01:47:23.136205+0000","last_unstale":"2024-04-26T01:51:38.737559+0000","last_undegraded":"2024-04-26T01:51:38.737559+0000","last_fullsized":"2024-04-26T01:51:38.737559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:24.418797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.226871+0000","last_change":"2024-04-26T01:48:22.747570+0000","last_active":"2024-04-26T01:51:39.226871+0000","last_peered":"2024-04-26T01:51:39.226871+0000","last_clean":"2024-04-26T01:51:39.226871+0000","last_became_active":"2024-04-26T01:48:22.747147+0000","last_became_peered":"2024-04-26T01:48:22.747147+0000","last_unstale":"2024-04-26T01:51:39.226871+0000","last_undegraded":"2024-04-26T01:51:39.226871+0000","last_fullsized":"2024-04-26T01:51:39.226871+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:37.034388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.980081+0000","last_change":"2024-04-26T01:48:22.414610+0000","last_active":"2024-04-26T01:51:38.980081+0000","last_peered":"2024-04-26T01:51:38.980081+0000","last_clean":"2024-04-26T01:51:38.980081+0000","last_became_active":"2024-04-26T01:48:22.410275+0000","last_became_peered":"2024-04-26T01:48:22.410275+0000","last_unstale":"2024-04-26T01:51:38.980081+0000","last_undegraded":"2024-04-26T01:51:38.980081+0000","last_fullsized":"2024-04-26T01:51:38.980081+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:20:48.131988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.759986+0000","last_change":"2024-04-26T01:47:27.115498+0000","last_active":"2024-04-26T01:51:38.759986+0000","last_peered":"2024-04-26T01:51:38.759986+0000","last_clean":"2024-04-26T01:51:38.759986+0000","last_became_active":"2024-04-26T01:47:27.115134+0000","last_became_peered":"2024-04-26T01:47:27.115134+0000","last_unstale":"2024-04-26T01:51:38.759986+0000","last_undegraded":"2024-04-26T01:51:38.759986+0000","last_fullsized":"2024-04-26T01:51:38.759986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:20.795224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.738166+0000","last_change":"2024-04-26T01:47:23.065366+0000","last_active":"2024-04-26T01:51:38.738166+0000","last_peered":"2024-04-26T01:51:38.738166+0000","last_clean":"2024-04-26T01:51:38.738166+0000","last_became_active":"2024-04-26T01:47:23.063216+0000","last_became_peered":"2024-04-26T01:47:23.063216+0000","last_unstale":"2024-04-26T01:51:38.738166+0000","last_undegraded":"2024-04-26T01:51:38.738166+0000","last_fullsized":"2024-04-26T01:51:38.738166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:06:05.261907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581069+0000","last_change":"2024-04-26T01:47:27.127938+0000","last_active":"2024-04-26T01:51:39.581069+0000","last_peered":"2024-04-26T01:51:39.581069+0000","last_clean":"2024-04-26T01:51:39.581069+0000","last_became_active":"2024-04-26T01:47:27.127553+0000","last_became_peered":"2024-04-26T01:47:27.127553+0000","last_unstale":"2024-04-26T01:51:39.581069+0000","last_undegraded":"2024-04-26T01:51:39.581069+0000","last_fullsized":"2024-04-26T01:51:39.581069+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:17:51.115349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"93'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:40.241511+0000","last_change":"2024-04-26T01:47:23.060475+0000","last_active":"2024-04-26T01:51:40.241511+0000","last_peered":"2024-04-26T01:51:40.241511+0000","last_clean":"2024-04-26T01:51:40.241511+0000","last_became_active":"2024-04-26T01:47:23.060098+0000","last_became_peered":"2024-04-26T01:47:23.060098+0000","last_unstale":"2024-04-26T01:51:40.241511+0000","last_undegraded":"2024-04-26T01:51:40.241511+0000","last_fullsized":"2024-04-26T01:51:40.241511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:49.009455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.738288+0000","last_change":"2024-04-26T01:47:27.137491+0000","last_active":"2024-04-26T01:51:38.738288+0000","last_peered":"2024-04-26T01:51:38.738288+0000","last_clean":"2024-04-26T01:51:38.738288+0000","last_became_active":"2024-04-26T01:47:27.137059+0000","last_became_peered":"2024-04-26T01:47:27.137059+0000","last_unstale":"2024-04-26T01:51:38.738288+0000","last_undegraded":"2024-04-26T01:51:38.738288+0000","last_fullsized":"2024-04-26T01:51:38.738288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:20:25.004074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581004+0000","last_change":"2024-04-26T01:48:22.405071+0000","last_active":"2024-04-26T01:51:39.581004+0000","last_peered":"2024-04-26T01:51:39.581004+0000","last_clean":"2024-04-26T01:51:39.581004+0000","last_became_active":"2024-04-26T01:48:22.401282+0000","last_became_peered":"2024-04-26T01:48:22.401282+0000","last_unstale":"2024-04-26T01:51:39.581004+0000","last_undegraded":"2024-04-26T01:51:39.581004+0000","last_fullsized":"2024-04-26T01:51:39.581004+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:24:53.628014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973087+0000","last_change":"2024-04-26T01:47:27.128253+0000","last_active":"2024-04-26T01:51:38.973087+0000","last_peered":"2024-04-26T01:51:38.973087+0000","last_clean":"2024-04-26T01:51:38.973087+0000","last_became_active":"2024-04-26T01:47:27.127869+0000","last_became_peered":"2024-04-26T01:47:27.127869+0000","last_unstale":"2024-04-26T01:51:38.973087+0000","last_undegraded":"2024-04-26T01:51:38.973087+0000","last_fullsized":"2024-04-26T01:51:38.973087+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:11:23.476938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.976946+0000","last_change":"2024-04-26T01:47:23.136862+0000","last_active":"2024-04-26T01:51:38.976946+0000","last_peered":"2024-04-26T01:51:38.976946+0000","last_clean":"2024-04-26T01:51:38.976946+0000","last_became_active":"2024-04-26T01:47:23.136380+0000","last_became_peered":"2024-04-26T01:47:23.136380+0000","last_unstale":"2024-04-26T01:51:38.976946+0000","last_undegraded":"2024-04-26T01:51:38.976946+0000","last_fullsized":"2024-04-26T01:51:38.976946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:39:18.103559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973500+0000","last_change":"2024-04-26T01:47:27.113363+0000","last_active":"2024-04-26T01:51:38.973500+0000","last_peered":"2024-04-26T01:51:38.973500+0000","last_clean":"2024-04-26T01:51:38.973500+0000","last_became_active":"2024-04-26T01:47:27.113040+0000","last_became_peered":"2024-04-26T01:47:27.113040+0000","last_unstale":"2024-04-26T01:51:38.973500+0000","last_undegraded":"2024-04-26T01:51:38.973500+0000","last_fullsized":"2024-04-26T01:51:38.973500+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:42:16.428697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'65","reported_seq":125,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:41.425620+0000","last_change":"2024-04-26T01:47:23.067257+0000","last_active":"2024-04-26T01:51:41.425620+0000","last_peered":"2024-04-26T01:51:41.425620+0000","last_clean":"2024-04-26T01:51:41.425620+0000","last_became_active":"2024-04-26T01:47:23.066858+0000","last_became_peered":"2024-04-26T01:47:23.066858+0000","last_unstale":"2024-04-26T01:51:41.425620+0000","last_undegraded":"2024-04-26T01:51:41.425620+0000","last_fullsized":"2024-04-26T01:51:41.425620+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:22:07.848074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973271+0000","last_change":"2024-04-26T01:47:27.125885+0000","last_active":"2024-04-26T01:51:38.973271+0000","last_peered":"2024-04-26T01:51:38.973271+0000","last_clean":"2024-04-26T01:51:38.973271+0000","last_became_active":"2024-04-26T01:47:27.125524+0000","last_became_peered":"2024-04-26T01:47:27.125524+0000","last_unstale":"2024-04-26T01:51:38.973271+0000","last_undegraded":"2024-04-26T01:51:38.973271+0000","last_fullsized":"2024-04-26T01:51:38.973271+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:39.282023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672262+0000","last_change":"2024-04-26T01:47:23.071051+0000","last_active":"2024-04-26T01:51:38.672262+0000","last_peered":"2024-04-26T01:51:38.672262+0000","last_clean":"2024-04-26T01:51:38.672262+0000","last_became_active":"2024-04-26T01:47:23.069524+0000","last_became_peered":"2024-04-26T01:47:23.069524+0000","last_unstale":"2024-04-26T01:51:38.672262+0000","last_undegraded":"2024-04-26T01:51:38.672262+0000","last_fullsized":"2024-04-26T01:51:38.672262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:22.039932+0000","last_clean_scrub_stamp":"2024-04-26T01:47:22.039932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:24:57.643100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.973498+0000","last_change":"2024-04-26T01:47:27.114325+0000","last_active":"2024-04-26T01:51:38.973498+0000","last_peered":"2024-04-26T01:51:38.973498+0000","last_clean":"2024-04-26T01:51:38.973498+0000","last_became_active":"2024-04-26T01:47:27.113952+0000","last_became_peered":"2024-04-26T01:47:27.113952+0000","last_unstale":"2024-04-26T01:51:38.973498+0000","last_undegraded":"2024-04-26T01:51:38.973498+0000","last_fullsized":"2024-04-26T01:51:38.973498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:39:19.167175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581077+0000","last_change":"2024-04-26T01:47:27.118709+0000","last_active":"2024-04-26T01:51:39.581077+0000","last_peered":"2024-04-26T01:51:39.581077+0000","last_clean":"2024-04-26T01:51:39.581077+0000","last_became_active":"2024-04-26T01:47:27.118341+0000","last_became_peered":"2024-04-26T01:47:27.118341+0000","last_unstale":"2024-04-26T01:51:39.581077+0000","last_undegraded":"2024-04-26T01:51:39.581077+0000","last_fullsized":"2024-04-26T01:51:39.581077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:13.736180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.976759+0000","last_change":"2024-04-26T01:47:27.119239+0000","last_active":"2024-04-26T01:51:38.976759+0000","last_peered":"2024-04-26T01:51:38.976759+0000","last_clean":"2024-04-26T01:51:38.976759+0000","last_became_active":"2024-04-26T01:47:27.118877+0000","last_became_peered":"2024-04-26T01:47:27.118877+0000","last_unstale":"2024-04-26T01:51:38.976759+0000","last_undegraded":"2024-04-26T01:51:38.976759+0000","last_fullsized":"2024-04-26T01:51:38.976759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:04:10.033517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.976632+0000","last_change":"2024-04-26T01:47:27.142848+0000","last_active":"2024-04-26T01:51:38.976632+0000","last_peered":"2024-04-26T01:51:38.976632+0000","last_clean":"2024-04-26T01:51:38.976632+0000","last_became_active":"2024-04-26T01:47:27.142580+0000","last_became_peered":"2024-04-26T01:47:27.142580+0000","last_unstale":"2024-04-26T01:51:38.976632+0000","last_undegraded":"2024-04-26T01:51:38.976632+0000","last_fullsized":"2024-04-26T01:51:38.976632+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:15:46.923922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.227787+0000","last_change":"2024-04-26T01:47:27.134817+0000","last_active":"2024-04-26T01:51:39.227787+0000","last_peered":"2024-04-26T01:51:39.227787+0000","last_clean":"2024-04-26T01:51:39.227787+0000","last_became_active":"2024-04-26T01:47:27.134024+0000","last_became_peered":"2024-04-26T01:47:27.134024+0000","last_unstale":"2024-04-26T01:51:39.227787+0000","last_undegraded":"2024-04-26T01:51:39.227787+0000","last_fullsized":"2024-04-26T01:51:39.227787+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:39:56.746427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672241+0000","last_change":"2024-04-26T01:47:27.143137+0000","last_active":"2024-04-26T01:51:38.672241+0000","last_peered":"2024-04-26T01:51:38.672241+0000","last_clean":"2024-04-26T01:51:38.672241+0000","last_became_active":"2024-04-26T01:47:27.142625+0000","last_became_peered":"2024-04-26T01:47:27.142625+0000","last_unstale":"2024-04-26T01:51:38.672241+0000","last_undegraded":"2024-04-26T01:51:38.672241+0000","last_fullsized":"2024-04-26T01:51:38.672241+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:42:50.468382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581247+0000","last_change":"2024-04-26T01:48:22.749582+0000","last_active":"2024-04-26T01:51:39.581247+0000","last_peered":"2024-04-26T01:51:39.581247+0000","last_clean":"2024-04-26T01:51:39.581247+0000","last_became_active":"2024-04-26T01:48:22.749191+0000","last_became_peered":"2024-04-26T01:48:22.749191+0000","last_unstale":"2024-04-26T01:51:39.581247+0000","last_undegraded":"2024-04-26T01:51:39.581247+0000","last_fullsized":"2024-04-26T01:51:39.581247+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:14.614842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.977129+0000","last_change":"2024-04-26T01:47:27.123964+0000","last_active":"2024-04-26T01:51:38.977129+0000","last_peered":"2024-04-26T01:51:38.977129+0000","last_clean":"2024-04-26T01:51:38.977129+0000","last_became_active":"2024-04-26T01:47:27.123552+0000","last_became_peered":"2024-04-26T01:47:27.123552+0000","last_unstale":"2024-04-26T01:51:38.977129+0000","last_undegraded":"2024-04-26T01:51:38.977129+0000","last_fullsized":"2024-04-26T01:51:38.977129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:33:12.566105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.738537+0000","last_change":"2024-04-26T01:48:22.412621+0000","last_active":"2024-04-26T01:51:38.738537+0000","last_peered":"2024-04-26T01:51:38.738537+0000","last_clean":"2024-04-26T01:51:38.738537+0000","last_became_active":"2024-04-26T01:48:22.411430+0000","last_became_peered":"2024-04-26T01:48:22.411430+0000","last_unstale":"2024-04-26T01:51:38.738537+0000","last_undegraded":"2024-04-26T01:51:38.738537+0000","last_fullsized":"2024-04-26T01:51:38.738537+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:34.152338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.672319+0000","last_change":"2024-04-26T01:47:27.131437+0000","last_active":"2024-04-26T01:51:38.672319+0000","last_peered":"2024-04-26T01:51:38.672319+0000","last_clean":"2024-04-26T01:51:38.672319+0000","last_became_active":"2024-04-26T01:47:27.131138+0000","last_became_peered":"2024-04-26T01:47:27.131138+0000","last_unstale":"2024-04-26T01:51:38.672319+0000","last_undegraded":"2024-04-26T01:51:38.672319+0000","last_fullsized":"2024-04-26T01:51:38.672319+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:55:39.132972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.737796+0000","last_change":"2024-04-26T01:47:27.120928+0000","last_active":"2024-04-26T01:51:38.737796+0000","last_peered":"2024-04-26T01:51:38.737796+0000","last_clean":"2024-04-26T01:51:38.737796+0000","last_became_active":"2024-04-26T01:47:27.120012+0000","last_became_peered":"2024-04-26T01:47:27.120012+0000","last_unstale":"2024-04-26T01:51:38.737796+0000","last_undegraded":"2024-04-26T01:51:38.737796+0000","last_fullsized":"2024-04-26T01:51:38.737796+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:19:44.209876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.980215+0000","last_change":"2024-04-26T01:47:27.138766+0000","last_active":"2024-04-26T01:51:38.980215+0000","last_peered":"2024-04-26T01:51:38.980215+0000","last_clean":"2024-04-26T01:51:38.980215+0000","last_became_active":"2024-04-26T01:47:27.138413+0000","last_became_peered":"2024-04-26T01:47:27.138413+0000","last_unstale":"2024-04-26T01:51:38.980215+0000","last_undegraded":"2024-04-26T01:51:38.980215+0000","last_fullsized":"2024-04-26T01:51:38.980215+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:51:13.117307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:38.671912+0000","last_change":"2024-04-26T01:47:27.141338+0000","last_active":"2024-04-26T01:51:38.671912+0000","last_peered":"2024-04-26T01:51:38.671912+0000","last_clean":"2024-04-26T01:51:38.671912+0000","last_became_active":"2024-04-26T01:47:27.140825+0000","last_became_peered":"2024-04-26T01:47:27.140825+0000","last_unstale":"2024-04-26T01:51:38.671912+0000","last_undegraded":"2024-04-26T01:51:38.671912+0000","last_fullsized":"2024-04-26T01:51:38.671912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:12:08.680876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T01:51:39.581041+0000","last_change":"2024-04-26T01:47:27.114919+0000","last_active":"2024-04-26T01:51:39.581041+0000","last_peered":"2024-04-26T01:51:39.581041+0000","last_clean":"2024-04-26T01:51:39.581041+0000","last_became_active":"2024-04-26T01:47:27.114528+0000","last_became_peered":"2024-04-26T01:47:27.114528+0000","last_unstale":"2024-04-26T01:51:39.581041+0000","last_undegraded":"2024-04-26T01:51:39.581041+0000","last_fullsized":"2024-04-26T01:51:39.581041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:47:26.066148+0000","last_clean_scrub_stamp":"2024-04-26T01:47:26.066148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:15:25.620566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":797827,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1820,"num_read_kb":1127,"num_write":428,"num_write_kb":2361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2617344,"data_stored":2393481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":315,"ondisk_log_size":315,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318339,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":499332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579976,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29032,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716120,"statfs":{"total":95995035648,"available":95965306880,"internally_reserved":0,"allocated":2064384,"data_stored":1491032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743502,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28420,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716732,"statfs":{"total":95995035648,"available":95965933568,"internally_reserved":0,"allocated":1437696,"data_stored":892784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005142,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36740,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708412,"statfs":{"total":95995035648,"available":95957413888,"internally_reserved":0,"allocated":1437696,"data_stored":892704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299481,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1093824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463007,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":498957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574973,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":568722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738497,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":500228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":570742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182512,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28868,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716284,"statfs":{"total":95995035648,"available":95965474816,"internally_reserved":0,"allocated":1773568,"data_stored":1220716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920874,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":501521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724647,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37392,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707760,"statfs":{"total":95995035648,"available":95956746240,"internally_reserved":0,"allocated":1982464,"data_stored":1419725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":330490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":401797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:51:53.489 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:51:53.961 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:54.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:54 smithi097 ceph-mon[32853]: from='client.25181 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:51:54.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:54 smithi097 ceph-mon[32853]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-26T01:51:54.277 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:54 smithi001 ceph-mon[27822]: from='client.25181 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:51:54.277 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:54 smithi001 ceph-mon[27822]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-26T01:51:54.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:54 smithi173 ceph-mon[32553]: from='client.25181 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:51:54.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:54 smithi173 ceph-mon[32553]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-26T01:51:55.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.289+0000 7f2a76400640 1 -- 172.21.15.1:0/3221129137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a70101ad0 msgr2=0x7f2a70105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.289+0000 7f2a76400640 1 --2- 172.21.15.1:0/3221129137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a70101ad0 0x7f2a70105ac0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009900 tx=0x7f2a5c02a410 comp rx=0 tx=0).stop 2024-04-26T01:51:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.289+0000 7f2a76400640 1 -- 172.21.15.1:0/3221129137 shutdown_connections 2024-04-26T01:51:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.289+0000 7f2a76400640 1 --2- 172.21.15.1:0/3221129137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a701061a0 0x7f2a7010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.289+0000 7f2a76400640 1 --2- 172.21.15.1:0/3221129137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a70101ad0 0x7f2a70105ac0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.289+0000 7f2a76400640 1 --2- 172.21.15.1:0/3221129137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a70101120 0x7f2a70101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.289+0000 7f2a76400640 1 -- 172.21.15.1:0/3221129137 >> 172.21.15.1:0/3221129137 conn(0x7f2a70077fa0 msgr2=0x7f2a700783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.289+0000 7f2a76400640 1 -- 172.21.15.1:0/3221129137 shutdown_connections 2024-04-26T01:51:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.289+0000 7f2a76400640 1 -- 172.21.15.1:0/3221129137 wait complete. 2024-04-26T01:51:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.290+0000 7f2a76400640 1 Processor -- start 2024-04-26T01:51:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.291+0000 7f2a76400640 1 -- start start 2024-04-26T01:51:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.291+0000 7f2a76400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a70101120 0x7f2a701a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.291+0000 7f2a76400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a701061a0 0x7f2a701a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.291+0000 7f2a76400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a7019a7e0 0x7f2a7019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.291+0000 7f2a76400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2a70112210 con 0x7f2a701061a0 2024-04-26T01:51:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.291+0000 7f2a76400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2a70112090 con 0x7f2a7019a7e0 2024-04-26T01:51:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.291+0000 7f2a76400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2a70112390 con 0x7f2a70101120 2024-04-26T01:51:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.291+0000 7f2a6ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a701061a0 0x7f2a701a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.291+0000 7f2a6f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a70101120 0x7f2a701a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.291+0000 7f2a6ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a701061a0 0x7f2a701a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56404/0 (socket says 172.21.15.1:56404) 2024-04-26T01:51:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.291+0000 7f2a6ea00640 1 -- 172.21.15.1:0/274616693 learned_addr learned my addr 172.21.15.1:0/274616693 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.291+0000 7f2a6fe00640 1 --2- 172.21.15.1:0/274616693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a7019a7e0 0x7f2a7019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:55.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.292+0000 7f2a6f400640 1 -- 172.21.15.1:0/274616693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a7019a7e0 msgr2=0x7f2a7019ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:55.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.292+0000 7f2a6f400640 1 --2- 172.21.15.1:0/274616693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a7019a7e0 0x7f2a7019ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.292+0000 7f2a6f400640 1 -- 172.21.15.1:0/274616693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a701061a0 msgr2=0x7f2a701a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:55.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.292+0000 7f2a6f400640 1 --2- 172.21.15.1:0/274616693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a701061a0 0x7f2a701a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.292+0000 7f2a6f400640 1 -- 172.21.15.1:0/274616693 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7019b370 con 0x7f2a70101120 2024-04-26T01:51:55.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.292+0000 7f2a6ea00640 1 --2- 172.21.15.1:0/274616693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a701061a0 0x7f2a701a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.292+0000 7f2a6f400640 1 --2- 172.21.15.1:0/274616693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a70101120 0x7f2a701a0670 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000e9f0 tx=0x7f2a6000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.292+0000 7f2a67e00640 1 -- 172.21.15.1:0/274616693 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6000ccc0 con 0x7f2a70101120 2024-04-26T01:51:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.292+0000 7f2a76400640 1 -- 172.21.15.1:0/274616693 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7019b660 con 0x7f2a70101120 2024-04-26T01:51:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.292+0000 7f2a67e00640 1 -- 172.21.15.1:0/274616693 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a600035f0 con 0x7f2a70101120 2024-04-26T01:51:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.293+0000 7f2a67e00640 1 -- 172.21.15.1:0/274616693 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6000ce60 con 0x7f2a70101120 2024-04-26T01:51:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.293+0000 7f2a76400640 1 -- 172.21.15.1:0/274616693 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a701a7860 con 0x7f2a70101120 2024-04-26T01:51:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.296+0000 7f2a79165740 1 Processor -- start 2024-04-26T01:51:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.296+0000 7f2a67e00640 1 -- 172.21.15.1:0/274616693 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f2a600037a0 con 0x7f2a70101120 2024-04-26T01:51:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.296+0000 7f2a79165740 1 -- start start 2024-04-26T01:51:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.297+0000 7f2a79165740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc5b820 0x558b8fc5bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.297+0000 7f2a79165740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc5c140 0x558b8fc664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.297+0000 7f2a79165740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc66a00 0x558b8fc68df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.297+0000 7f2a79165740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558b8fbc1900 con 0x558b8fc66a00 2024-04-26T01:51:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.297+0000 7f2a79165740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558b8fb67ef0 con 0x558b8fc5c140 2024-04-26T01:51:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.297+0000 7f2a79165740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558b8fbbd8c0 con 0x558b8fc5b820 2024-04-26T01:51:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.297+0000 7f2a6fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc5b820 0x558b8fc5bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.297+0000 7f2a6ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc66a00 0x558b8fc68df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.297+0000 7f2a6fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc5b820 0x558b8fc5bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:60284/0 (socket says 172.21.15.1:60284) 2024-04-26T01:51:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.297+0000 7f2a6fe00640 1 -- 172.21.15.1:0/4244395920 learned_addr learned my addr 172.21.15.1:0/4244395920 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.297+0000 7f2a6f400640 1 --2- 172.21.15.1:0/4244395920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc5c140 0x558b8fc664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.298+0000 7f2a6fe00640 1 -- 172.21.15.1:0/4244395920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc5c140 msgr2=0x558b8fc664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.298+0000 7f2a6fe00640 1 --2- 172.21.15.1:0/4244395920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc5c140 0x558b8fc664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.298+0000 7f2a6fe00640 1 -- 172.21.15.1:0/4244395920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc66a00 msgr2=0x558b8fc68df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.298+0000 7f2a6fe00640 1 --2- 172.21.15.1:0/4244395920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc66a00 0x558b8fc68df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.298+0000 7f2a6fe00640 1 -- 172.21.15.1:0/4244395920 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b8f8d4210 con 0x558b8fc5b820 2024-04-26T01:51:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.298+0000 7f2a6ea00640 1 --2- 172.21.15.1:0/4244395920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc66a00 0x558b8fc68df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.298+0000 7f2a67e00640 1 --2- 172.21.15.1:0/274616693 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2a540a3eb0 0x7f2a540a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.298+0000 7f2a6fe00640 1 --2- 172.21.15.1:0/4244395920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc5b820 0x558b8fc5bc00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009f60 tx=0x7f2a68009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7e4ca795bc1cf81 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.298+0000 7f2a67e00640 1 -- 172.21.15.1:0/274616693 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f2a600d0c50 con 0x7f2a70101120 2024-04-26T01:51:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.298+0000 7f2a64c00640 1 -- 172.21.15.1:0/4244395920 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6801e070 con 0x558b8fc5b820 2024-04-26T01:51:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.298+0000 7f2a6f400640 1 --2- 172.21.15.1:0/274616693 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2a540a3eb0 0x7f2a540a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.298+0000 7f2a64c00640 1 -- 172.21.15.1:0/4244395920 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a68003c30 con 0x558b8fc5b820 2024-04-26T01:51:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.298+0000 7f2a64c00640 1 -- 172.21.15.1:0/4244395920 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800f840 con 0x558b8fc5b820 2024-04-26T01:51:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.299+0000 7f2a79165740 1 -- 172.21.15.1:0/4244395920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc5b820 msgr2=0x558b8fc5bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.299+0000 7f2a6f400640 1 --2- 172.21.15.1:0/274616693 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2a540a3eb0 0x7f2a540a6370 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2a600ce280 tx=0x7f2a600d4750 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.299+0000 7f2a79165740 1 --2- 172.21.15.1:0/4244395920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc5b820 0x558b8fc5bc00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009f60 tx=0x7f2a68009f90 comp rx=0 tx=0).stop 2024-04-26T01:51:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.299+0000 7f2a79165740 1 -- 172.21.15.1:0/4244395920 shutdown_connections 2024-04-26T01:51:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.299+0000 7f2a79165740 1 --2- 172.21.15.1:0/4244395920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc66a00 0x558b8fc68df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.299+0000 7f2a79165740 1 --2- 172.21.15.1:0/4244395920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc5c140 0x558b8fc664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.299+0000 7f2a79165740 1 --2- 172.21.15.1:0/4244395920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc5b820 0x558b8fc5bc00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.299+0000 7f2a79165740 1 -- 172.21.15.1:0/4244395920 >> 172.21.15.1:0/4244395920 conn(0x558b8fc57d30 msgr2=0x558b8fc58110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.299+0000 7f2a79165740 1 -- 172.21.15.1:0/4244395920 shutdown_connections 2024-04-26T01:51:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.299+0000 7f2a79165740 1 -- 172.21.15.1:0/4244395920 wait complete. 2024-04-26T01:51:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.300+0000 7f2a79165740 1 -- start start 2024-04-26T01:51:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.300+0000 7f2a79165740 1 Processor -- start 2024-04-26T01:51:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.300+0000 7f2a79165740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc5b820 0x558b8fc63a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.300+0000 7f2a79165740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc5c140 0x558b8fc63f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.300+0000 7f2a79165740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc66a00 0x558b8fc64490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.301+0000 7f2a79165740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558b8fc69630 con 0x558b8fc66a00 2024-04-26T01:51:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.301+0000 7f2a79165740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558b8fb7b310 con 0x558b8fc5b820 2024-04-26T01:51:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.301+0000 7f2a79165740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558b8fc697b0 con 0x558b8fc5c140 2024-04-26T01:51:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.301+0000 7f2a79165740 10 client.0 _ll_register_callbacks cb 0x558b8fb49cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:51:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.301+0000 7f2a6fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc5b820 0x558b8fc63a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.301+0000 7f2a6fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc5b820 0x558b8fc63a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:56002/0 (socket says 172.21.15.1:56002) 2024-04-26T01:51:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.301+0000 7f2a6fe00640 1 -- 172.21.15.1:0/688922735 learned_addr learned my addr 172.21.15.1:0/688922735 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.301+0000 7f2a6fe00640 1 --2- 172.21.15.1:0/688922735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc66a00 0x558b8fc64490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.301+0000 7f2a6ea00640 1 --2- 172.21.15.1:0/688922735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc5c140 0x558b8fc63f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.301+0000 7f2a6fe00640 1 -- 172.21.15.1:0/688922735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc5c140 msgr2=0x558b8fc63f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.301+0000 7f2a6fe00640 1 --2- 172.21.15.1:0/688922735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc5c140 0x558b8fc63f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.301+0000 7f2a6fe00640 1 -- 172.21.15.1:0/688922735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc66a00 msgr2=0x558b8fc64490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.301+0000 7f2a6fe00640 1 --2- 172.21.15.1:0/688922735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc66a00 0x558b8fc64490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.301+0000 7f2a6fe00640 1 -- 172.21.15.1:0/688922735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b8f9a0620 con 0x558b8fc5b820 2024-04-26T01:51:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.302+0000 7f2a6fe00640 1 --2- 172.21.15.1:0/688922735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc5b820 0x558b8fc63a10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2a68011d60 tx=0x7f2a6800f520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b2bb60a18cc1730 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.302+0000 7f2a59600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:51:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.302+0000 7f2a59600640 1 -- 172.21.15.1:0/688922735 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6801e030 con 0x558b8fc5b820 2024-04-26T01:51:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.302+0000 7f2a79165740 1 -- 172.21.15.1:0/688922735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a68003c30 con 0x558b8fc5b820 2024-04-26T01:51:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.302+0000 7f2a59600640 1 -- 172.21.15.1:0/688922735 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f2a68004490 con 0x558b8fc5b820 2024-04-26T01:51:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.302+0000 7f2a59600640 1 -- 172.21.15.1:0/688922735 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2a68004690 con 0x558b8fc5b820 2024-04-26T01:51:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.302+0000 7f2a79165740 10 client.25193 fetch_fsmap learned FSMap version 26 2024-04-26T01:51:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.302+0000 7f2a79165740 1 -- 172.21.15.1:0/688922735 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558b8fc75920 con 0x558b8fc5b820 2024-04-26T01:51:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.302+0000 7f2a59600640 1 -- 172.21.15.1:0/688922735 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a68003c30 con 0x558b8fc5b820 2024-04-26T01:51:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.303+0000 7f2a59600640 1 -- 172.21.15.1:0/688922735 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2a68004fe0 con 0x558b8fc5b820 2024-04-26T01:51:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.303+0000 7f2a79165740 10 client.25193 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:51:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.303+0000 7f2a79165740 10 client.25193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:51:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.303+0000 7f2a79165740 20 client.25193 populate_metadata read hostname 'smithi001' 2024-04-26T01:51:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.303+0000 7f2a79165740 1 --2- 172.21.15.1:0/688922735 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x558b8fc76d90 0x558b8fc79180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.303+0000 7f2a79165740 4 client.25193 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:51:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.303+0000 7f2a79165740 1 -- 172.21.15.1:0/688922735 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b8fbd58b0 con 0x558b8fc76d90 2024-04-26T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.303+0000 7f2a6ea00640 1 --2- 172.21.15.1:0/688922735 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x558b8fc76d90 0x558b8fc79180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.304+0000 7f2a6ea00640 1 --2- 172.21.15.1:0/688922735 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x558b8fc76d90 0x558b8fc79180 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b2234d218ebbc4e server_cookie=ffb4ab355dd9b664 in_seq=0 out_seq=0 2024-04-26T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.304+0000 7f2a59600640 10 client.25193 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.305+0000 7f2a59600640 1 -- 172.21.15.1:0/688922735 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b8fbd58b0 con 0x558b8fc76d90 2024-04-26T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.305+0000 7f2a59600640 10 client.25193 handle_command_reply: tid=0 2024-04-26T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.305+0000 7f2a79165740 1 client.25193 shutdown 2024-04-26T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.305+0000 7f2a79165740 1 -- 172.21.15.1:0/688922735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc5b820 msgr2=0x558b8fc63a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.305+0000 7f2a79165740 1 --2- 172.21.15.1:0/688922735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc5b820 0x558b8fc63a10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2a68011d60 tx=0x7f2a6800f520 comp rx=0 tx=0).stop 2024-04-26T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.305+0000 7f2a79165740 1 -- 172.21.15.1:0/688922735 shutdown_connections 2024-04-26T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.305+0000 7f2a79165740 1 --2- 172.21.15.1:0/688922735 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x558b8fc76d90 0x558b8fc79180 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.305+0000 7f2a79165740 1 --2- 172.21.15.1:0/688922735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc66a00 0x558b8fc64490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.305+0000 7f2a79165740 1 --2- 172.21.15.1:0/688922735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc5c140 0x558b8fc63f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.305+0000 7f2a79165740 1 --2- 172.21.15.1:0/688922735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc5b820 0x558b8fc63a10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.305+0000 7f2a59600640 0 client.25193 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:51:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.305+0000 7f2a79165740 1 -- 172.21.15.1:0/688922735 >> 172.21.15.1:0/688922735 conn(0x558b8fc57d30 msgr2=0x558b8fc5da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.305+0000 7f2a79165740 1 -- 172.21.15.1:0/688922735 shutdown_connections 2024-04-26T01:51:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.305+0000 7f2a79165740 1 -- 172.21.15.1:0/688922735 wait complete. 2024-04-26T01:51:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.305+0000 7f2a79165740 20 client.25193 trim_cache size 0 max 16384 2024-04-26T01:51:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.322+0000 7f2a79165740 1 Processor -- start 2024-04-26T01:51:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.322+0000 7f2a79165740 1 -- start start 2024-04-26T01:51:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.322+0000 7f2a79165740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc5b820 0x558b8fc661e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.322+0000 7f2a79165740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc5c140 0x558b8fc724f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.322+0000 7f2a79165740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc66a00 0x558b8fc5ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.322+0000 7f2a79165740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701b27d0 con 0x558b8fc5c140 2024-04-26T01:51:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.322+0000 7f2a79165740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701b00a0 con 0x558b8fc66a00 2024-04-26T01:51:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.322+0000 7f2a79165740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701adef0 con 0x558b8fc5b820 2024-04-26T01:51:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.322+0000 7f2a6fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc5b820 0x558b8fc661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.322+0000 7f2a6fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc5b820 0x558b8fc661e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:60294/0 (socket says 172.21.15.1:60294) 2024-04-26T01:51:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.322+0000 7f2a6fe00640 1 -- 172.21.15.1:0/697668864 learned_addr learned my addr 172.21.15.1:0/697668864 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.322+0000 7f2a6ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc5c140 0x558b8fc724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.322+0000 7f2a6fe00640 1 --2- 172.21.15.1:0/697668864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc66a00 0x558b8fc5ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:55.322 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.323+0000 7f2a6fe00640 1 -- 172.21.15.1:0/697668864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc66a00 msgr2=0x558b8fc5ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.323+0000 7f2a6fe00640 1 --2- 172.21.15.1:0/697668864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc66a00 0x558b8fc5ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.323+0000 7f2a6fe00640 1 -- 172.21.15.1:0/697668864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc5c140 msgr2=0x558b8fc724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.323+0000 7f2a6fe00640 1 --2- 172.21.15.1:0/697668864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc5c140 0x558b8fc724f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.323+0000 7f2a6fe00640 1 -- 172.21.15.1:0/697668864 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b8fc61f60 con 0x558b8fc5b820 2024-04-26T01:51:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.323+0000 7f2a6fe00640 1 --2- 172.21.15.1:0/697668864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc5b820 0x558b8fc661e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2a68004a30 tx=0x7f2a68014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ff7bd46e37a6abe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.323+0000 7f2a5be00640 1 -- 172.21.15.1:0/697668864 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6801e070 con 0x558b8fc5b820 2024-04-26T01:51:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.323+0000 7f2a5be00640 1 -- 172.21.15.1:0/697668864 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a6802d070 con 0x558b8fc5b820 2024-04-26T01:51:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.323+0000 7f2a5be00640 1 -- 172.21.15.1:0/697668864 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a680044f0 con 0x558b8fc5b820 2024-04-26T01:51:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.324+0000 7f2a79165740 1 -- 172.21.15.1:0/697668864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc5b820 msgr2=0x558b8fc661e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.324+0000 7f2a79165740 1 --2- 172.21.15.1:0/697668864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc5b820 0x558b8fc661e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2a68004a30 tx=0x7f2a68014040 comp rx=0 tx=0).stop 2024-04-26T01:51:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.324+0000 7f2a79165740 1 -- 172.21.15.1:0/697668864 shutdown_connections 2024-04-26T01:51:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.324+0000 7f2a79165740 1 --2- 172.21.15.1:0/697668864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc66a00 0x558b8fc5ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.324+0000 7f2a79165740 1 --2- 172.21.15.1:0/697668864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc5c140 0x558b8fc724f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.324+0000 7f2a79165740 1 --2- 172.21.15.1:0/697668864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc5b820 0x558b8fc661e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.324+0000 7f2a79165740 1 -- 172.21.15.1:0/697668864 >> 172.21.15.1:0/697668864 conn(0x558b8fc57d30 msgr2=0x558b8fc65ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.324+0000 7f2a79165740 1 -- 172.21.15.1:0/697668864 shutdown_connections 2024-04-26T01:51:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.324+0000 7f2a79165740 1 -- 172.21.15.1:0/697668864 wait complete. 2024-04-26T01:51:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.324+0000 7f2a79165740 1 -- start start 2024-04-26T01:51:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.325+0000 7f2a79165740 1 Processor -- start 2024-04-26T01:51:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.325+0000 7f2a79165740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc5b820 0x558b8fc6b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.325+0000 7f2a79165740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc5c140 0x558b8fc6bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.325+0000 7f2a79165740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc66a00 0x558b8fce3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.325+0000 7f2a79165740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701ae560 con 0x558b8fc5b820 2024-04-26T01:51:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.325+0000 7f2a79165740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701af9d0 con 0x558b8fc5c140 2024-04-26T01:51:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.325+0000 7f2a79165740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701aec30 con 0x558b8fc66a00 2024-04-26T01:51:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.325+0000 7f2a79165740 10 client.0 _ll_register_callbacks cb 0x7f2a68009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:51:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.325+0000 7f2a6fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc5b820 0x558b8fc6b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.325+0000 7f2a6fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc5b820 0x558b8fc6b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56432/0 (socket says 172.21.15.1:56432) 2024-04-26T01:51:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.325+0000 7f2a6fe00640 1 -- 172.21.15.1:0/2699284248 learned_addr learned my addr 172.21.15.1:0/2699284248 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.325+0000 7f2a6ea00640 1 --2- 172.21.15.1:0/2699284248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc5c140 0x558b8fc6bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.326+0000 7f2a6fe00640 1 --2- 172.21.15.1:0/2699284248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc66a00 0x558b8fce3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.326+0000 7f2a6fe00640 1 -- 172.21.15.1:0/2699284248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc66a00 msgr2=0x558b8fce3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.326+0000 7f2a6fe00640 1 --2- 172.21.15.1:0/2699284248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc66a00 0x558b8fce3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.326+0000 7f2a6fe00640 1 -- 172.21.15.1:0/2699284248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc5c140 msgr2=0x558b8fc6bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:51:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.326+0000 7f2a6fe00640 1 --2- 172.21.15.1:0/2699284248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc5c140 0x558b8fc6bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.326+0000 7f2a6fe00640 1 -- 172.21.15.1:0/2699284248 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b8fce3eb0 con 0x558b8fc5b820 2024-04-26T01:51:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.326+0000 7f2a6fe00640 1 --2- 172.21.15.1:0/2699284248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc5b820 0x558b8fc6b9d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2a680048e0 tx=0x7f2a68012fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26023a4b2079e4f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.326+0000 7f2a59600640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:51:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.326+0000 7f2a59600640 1 -- 172.21.15.1:0/2699284248 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6801e070 con 0x558b8fc5b820 2024-04-26T01:51:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.326+0000 7f2a59600640 1 -- 172.21.15.1:0/2699284248 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f2a680051c0 con 0x558b8fc5b820 2024-04-26T01:51:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.326+0000 7f2a79165740 1 -- 172.21.15.1:0/2699284248 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a6802d070 con 0x558b8fc5b820 2024-04-26T01:51:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.327+0000 7f2a59600640 1 -- 172.21.15.1:0/2699284248 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2a68052050 con 0x558b8fc5b820 2024-04-26T01:51:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.327+0000 7f2a79165740 10 client.15081 fetch_fsmap learned FSMap version 26 2024-04-26T01:51:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.327+0000 7f2a79165740 1 -- 172.21.15.1:0/2699284248 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558b8fce5060 con 0x558b8fc5b820 2024-04-26T01:51:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.327+0000 7f2a59600640 1 -- 172.21.15.1:0/2699284248 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a6802d070 con 0x558b8fc5b820 2024-04-26T01:51:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.327+0000 7f2a59600640 1 -- 172.21.15.1:0/2699284248 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2a6804a020 con 0x558b8fc5b820 2024-04-26T01:51:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.327+0000 7f2a79165740 10 client.15081 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:51:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.327+0000 7f2a79165740 10 client.15081 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:51:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.327+0000 7f2a79165740 20 client.15081 populate_metadata read hostname 'smithi001' 2024-04-26T01:51:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.327+0000 7f2a79165740 1 --2- 172.21.15.1:0/2699284248 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x558b8fce5780 0x558b8fce7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.327+0000 7f2a79165740 4 client.15081 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:51:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.327+0000 7f2a79165740 1 -- 172.21.15.1:0/2699284248 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b8fbcf320 con 0x558b8fce5780 2024-04-26T01:51:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.328+0000 7f2a6ea00640 1 --2- 172.21.15.1:0/2699284248 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x558b8fce5780 0x558b8fce7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.328+0000 7f2a6ea00640 1 --2- 172.21.15.1:0/2699284248 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x558b8fce5780 0x558b8fce7ba0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b947c1bf46f98a1 server_cookie=ab9437047101f1fd in_seq=0 out_seq=0 2024-04-26T01:51:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.328+0000 7f2a59600640 10 client.15081 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:51:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.341+0000 7f2a59600640 1 -- 172.21.15.1:0/2699284248 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b8fbcf320 con 0x558b8fce5780 2024-04-26T01:51:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.341+0000 7f2a59600640 10 client.15081 handle_command_reply: tid=0 2024-04-26T01:51:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.341+0000 7f2a79165740 1 client.15081 shutdown 2024-04-26T01:51:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.341+0000 7f2a79165740 1 -- 172.21.15.1:0/2699284248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc5b820 msgr2=0x558b8fc6b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.341+0000 7f2a79165740 1 --2- 172.21.15.1:0/2699284248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc5b820 0x558b8fc6b9d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2a680048e0 tx=0x7f2a68012fe0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.342+0000 7f2a79165740 1 -- 172.21.15.1:0/2699284248 shutdown_connections 2024-04-26T01:51:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.342+0000 7f2a79165740 1 --2- 172.21.15.1:0/2699284248 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x558b8fce5780 0x558b8fce7ba0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.342+0000 7f2a79165740 1 --2- 172.21.15.1:0/2699284248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b8fc66a00 0x558b8fce3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.342+0000 7f2a79165740 1 --2- 172.21.15.1:0/2699284248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b8fc5c140 0x558b8fc6bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.342+0000 7f2a79165740 1 --2- 172.21.15.1:0/2699284248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b8fc5b820 0x558b8fc6b9d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.342+0000 7f2a59600640 0 client.15081 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:51:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.342+0000 7f2a79165740 1 -- 172.21.15.1:0/2699284248 >> 172.21.15.1:0/2699284248 conn(0x558b8fc57d30 msgr2=0x558b8fc714e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.342+0000 7f2a79165740 1 -- 172.21.15.1:0/2699284248 shutdown_connections 2024-04-26T01:51:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.342+0000 7f2a79165740 1 -- 172.21.15.1:0/2699284248 wait complete. 2024-04-26T01:51:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.342+0000 7f2a79165740 20 client.15081 trim_cache size 0 max 16384 2024-04-26T01:51:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:51:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:51:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:51:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:51:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.343+0000 7f2a76400640 1 -- 172.21.15.1:0/274616693 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2a540a3eb0 msgr2=0x7f2a540a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.343+0000 7f2a76400640 1 --2- 172.21.15.1:0/274616693 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2a540a3eb0 0x7f2a540a6370 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2a600ce280 tx=0x7f2a600d4750 comp rx=0 tx=0).stop 2024-04-26T01:51:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.343+0000 7f2a76400640 1 -- 172.21.15.1:0/274616693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a70101120 msgr2=0x7f2a701a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.343+0000 7f2a76400640 1 --2- 172.21.15.1:0/274616693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a70101120 0x7f2a701a0670 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000e9f0 tx=0x7f2a6000edf0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.343+0000 7f2a76400640 1 -- 172.21.15.1:0/274616693 shutdown_connections 2024-04-26T01:51:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.343+0000 7f2a76400640 1 --2- 172.21.15.1:0/274616693 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2a540a3eb0 0x7f2a540a6370 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.343+0000 7f2a76400640 1 --2- 172.21.15.1:0/274616693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a7019a7e0 0x7f2a7019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.343+0000 7f2a76400640 1 --2- 172.21.15.1:0/274616693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a701061a0 0x7f2a701a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.343+0000 7f2a76400640 1 --2- 172.21.15.1:0/274616693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a70101120 0x7f2a701a0670 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.343+0000 7f2a76400640 1 -- 172.21.15.1:0/274616693 >> 172.21.15.1:0/274616693 conn(0x7f2a70077fa0 msgr2=0x7f2a700feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.343+0000 7f2a76400640 1 -- 172.21.15.1:0/274616693 shutdown_connections 2024-04-26T01:51:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:51:55.343+0000 7f2a76400640 1 -- 172.21.15.1:0/274616693 wait complete. 2024-04-26T01:51:56.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.258+0000 7fd6be800700 1 -- 172.21.15.1:0/3850124981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b8109fa0 msgr2=0x7fd6b8111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:56.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.258+0000 7fd6be800700 1 --2- 172.21.15.1:0/3850124981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b8109fa0 0x7fd6b8111b70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000b0d0 tx=0x7fd6b000b3e0 comp rx=0 tx=0).stop 2024-04-26T01:51:56.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.258+0000 7fd6be800700 1 -- 172.21.15.1:0/3850124981 shutdown_connections 2024-04-26T01:51:56.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.258+0000 7fd6be800700 1 --2- 172.21.15.1:0/3850124981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b8109fa0 0x7fd6b8111b70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:56.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.258+0000 7fd6be800700 1 --2- 172.21.15.1:0/3850124981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6b8104ec0 0x7fd6b81052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:56.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.258+0000 7fd6be800700 1 --2- 172.21.15.1:0/3850124981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6b8105870 0x7fd6b81098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:56.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.258+0000 7fd6be800700 1 -- 172.21.15.1:0/3850124981 >> 172.21.15.1:0/3850124981 conn(0x7fd6b8100750 msgr2=0x7fd6b8102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:56.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.259+0000 7fd6be800700 1 -- 172.21.15.1:0/3850124981 shutdown_connections 2024-04-26T01:51:56.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.259+0000 7fd6be800700 1 -- 172.21.15.1:0/3850124981 wait complete. 2024-04-26T01:51:56.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.260+0000 7fd6be800700 1 Processor -- start 2024-04-26T01:51:56.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.260+0000 7fd6be800700 1 -- start start 2024-04-26T01:51:56.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.260+0000 7fd6be800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b8104ec0 0x7fd6b819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:56.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.260+0000 7fd6be800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6b8105870 0x7fd6b819b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:56.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.260+0000 7fd6be800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6b8109fa0 0x7fd6b819fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:56.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.260+0000 7fd6be800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b8114e90 con 0x7fd6b8104ec0 2024-04-26T01:51:56.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.260+0000 7fd6be800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b8114d10 con 0x7fd6b8109fa0 2024-04-26T01:51:56.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.260+0000 7fd6be800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b8115010 con 0x7fd6b8105870 2024-04-26T01:51:56.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.261+0000 7fd6b7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b8104ec0 0x7fd6b819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:56.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.261+0000 7fd6afe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6b8105870 0x7fd6b819b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:56.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.261+0000 7fd6b7e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6b8109fa0 0x7fd6b819fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:56.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.261+0000 7fd6afe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6b8105870 0x7fd6b819b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:60326/0 (socket says 172.21.15.1:60326) 2024-04-26T01:51:56.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.261+0000 7fd6afe00700 1 -- 172.21.15.1:0/1058355572 learned_addr learned my addr 172.21.15.1:0/1058355572 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:51:56.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.261+0000 7fd6b7e00700 1 -- 172.21.15.1:0/1058355572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6b8105870 msgr2=0x7fd6b819b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:56.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.261+0000 7fd6b7e00700 1 --2- 172.21.15.1:0/1058355572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6b8105870 0x7fd6b819b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:56.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.261+0000 7fd6b7e00700 1 -- 172.21.15.1:0/1058355572 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b8104ec0 msgr2=0x7fd6b819b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:56.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.261+0000 7fd6b7e00700 1 --2- 172.21.15.1:0/1058355572 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b8104ec0 0x7fd6b819b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:56.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.261+0000 7fd6b7e00700 1 -- 172.21.15.1:0/1058355572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b81a02e0 con 0x7fd6b8109fa0 2024-04-26T01:51:56.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.261+0000 7fd6afe00700 1 --2- 172.21.15.1:0/1058355572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6b8105870 0x7fd6b819b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:51:56.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.262+0000 7fd6b7e00700 1 --2- 172.21.15.1:0/1058355572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6b8109fa0 0x7fd6b819fba0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000b770 tx=0x7fd6b0007980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:56.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.262+0000 7fd6b4c00700 1 -- 172.21.15.1:0/1058355572 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b001f070 con 0x7fd6b8109fa0 2024-04-26T01:51:56.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.262+0000 7fd6be800700 1 -- 172.21.15.1:0/1058355572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b81a0570 con 0x7fd6b8109fa0 2024-04-26T01:51:56.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.262+0000 7fd6b4c00700 1 -- 172.21.15.1:0/1058355572 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6b0007dc0 con 0x7fd6b8109fa0 2024-04-26T01:51:56.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.262+0000 7fd6b4c00700 1 -- 172.21.15.1:0/1058355572 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b001b430 con 0x7fd6b8109fa0 2024-04-26T01:51:56.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.262+0000 7fd6be800700 1 -- 172.21.15.1:0/1058355572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b81a7e50 con 0x7fd6b8109fa0 2024-04-26T01:51:56.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.263+0000 7fd6be800700 1 -- 172.21.15.1:0/1058355572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b81063a0 con 0x7fd6b8109fa0 2024-04-26T01:51:56.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.265+0000 7fd6b4c00700 1 -- 172.21.15.1:0/1058355572 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7fd6b0022070 con 0x7fd6b8109fa0 2024-04-26T01:51:56.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.267+0000 7fd6b4c00700 1 --2- 172.21.15.1:0/1058355572 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd69c0a3ed0 0x7fd69c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:56.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.267+0000 7fd6b4c00700 1 -- 172.21.15.1:0/1058355572 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7fd6b000f020 con 0x7fd6b8109fa0 2024-04-26T01:51:56.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.267+0000 7fd6b7400700 1 --2- 172.21.15.1:0/1058355572 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd69c0a3ed0 0x7fd69c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:56.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.268+0000 7fd6b7400700 1 --2- 172.21.15.1:0/1058355572 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd69c0a3ed0 0x7fd69c0a6390 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8009c60 tx=0x7fd6a8009260 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:56.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.269+0000 7fd6b4c00700 1 -- 172.21.15.1:0/1058355572 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6b0029020 con 0x7fd6b8109fa0 2024-04-26T01:51:56.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:56 smithi173 ceph-mon[32553]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-26T01:51:56.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:56 smithi001 ceph-mon[27822]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-26T01:51:56.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:56 smithi097 ceph-mon[32853]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-26T01:51:56.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.550+0000 7fd6be800700 1 -- 172.21.15.1:0/1058355572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6b819c630 con 0x7fd6b8109fa0 2024-04-26T01:51:56.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.551+0000 7fd6b4c00700 1 -- 172.21.15.1:0/1058355572 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11841 (secure 0 0 0) 0x7fd6b009e2b0 con 0x7fd6b8109fa0 2024-04-26T01:51:56.550 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:51:56.550 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:51:38.222728+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2952462325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2952462325},{"type":"v1","addr":"172.21.15.173:6835","nonce":2952462325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24499,24514,24518]},"id":1}]} 2024-04-26T01:51:56.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.555+0000 7fd6be800700 1 -- 172.21.15.1:0/1058355572 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd69c0a3ed0 msgr2=0x7fd69c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:56.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.555+0000 7fd6be800700 1 --2- 172.21.15.1:0/1058355572 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd69c0a3ed0 0x7fd69c0a6390 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8009c60 tx=0x7fd6a8009260 comp rx=0 tx=0).stop 2024-04-26T01:51:56.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.555+0000 7fd6be800700 1 -- 172.21.15.1:0/1058355572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6b8109fa0 msgr2=0x7fd6b819fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:56.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.555+0000 7fd6be800700 1 --2- 172.21.15.1:0/1058355572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6b8109fa0 0x7fd6b819fba0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000b770 tx=0x7fd6b0007980 comp rx=0 tx=0).stop 2024-04-26T01:51:56.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.555+0000 7fd6be800700 1 -- 172.21.15.1:0/1058355572 shutdown_connections 2024-04-26T01:51:56.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.555+0000 7fd6be800700 1 --2- 172.21.15.1:0/1058355572 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd69c0a3ed0 0x7fd69c0a6390 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:56.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.555+0000 7fd6be800700 1 --2- 172.21.15.1:0/1058355572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6b8109fa0 0x7fd6b819fba0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:56.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.555+0000 7fd6be800700 1 --2- 172.21.15.1:0/1058355572 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b8104ec0 0x7fd6b819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:56.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.555+0000 7fd6be800700 1 --2- 172.21.15.1:0/1058355572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6b8105870 0x7fd6b819b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:56.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.555+0000 7fd6be800700 1 -- 172.21.15.1:0/1058355572 >> 172.21.15.1:0/1058355572 conn(0x7fd6b8100750 msgr2=0x7fd6b8102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:56.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.555+0000 7fd6be800700 1 -- 172.21.15.1:0/1058355572 shutdown_connections 2024-04-26T01:51:56.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:51:56.555+0000 7fd6be800700 1 -- 172.21.15.1:0/1058355572 wait complete. 2024-04-26T01:51:56.555 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 26 2024-04-26T01:51:57.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:57 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1058355572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:51:57.343 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:57 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1058355572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:51:57.402 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:51:57.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:57 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1058355572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:51:57.887 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:58.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:51:58 smithi173 ceph-mon[32553]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-26T01:51:58.364 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:51:58 smithi001 ceph-mon[27822]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-26T01:51:58.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:51:58 smithi097 ceph-mon[32853]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-26T01:52:00.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.159+0000 7f7522800700 1 -- 172.21.15.1:0/3959929272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f751c10e3b0 msgr2=0x7f751c115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:00.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.159+0000 7f7522800700 1 --2- 172.21.15.1:0/3959929272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f751c10e3b0 0x7f751c115f80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f75140099c0 tx=0x7f7514009cd0 comp rx=0 tx=0).stop 2024-04-26T01:52:00.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.159+0000 7f7522800700 1 -- 172.21.15.1:0/3959929272 shutdown_connections 2024-04-26T01:52:00.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.159+0000 7f7522800700 1 --2- 172.21.15.1:0/3959929272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f751c10e3b0 0x7f751c115f80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:00.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.159+0000 7f7522800700 1 --2- 172.21.15.1:0/3959929272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f751c1092d0 0x7f751c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:00.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.159+0000 7f7522800700 1 --2- 172.21.15.1:0/3959929272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f751c109c80 0x7f751c10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:00.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.159+0000 7f7522800700 1 -- 172.21.15.1:0/3959929272 >> 172.21.15.1:0/3959929272 conn(0x7f751c076210 msgr2=0x7f751c107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:00.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.159+0000 7f7522800700 1 -- 172.21.15.1:0/3959929272 shutdown_connections 2024-04-26T01:52:00.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.159+0000 7f7522800700 1 -- 172.21.15.1:0/3959929272 wait complete. 2024-04-26T01:52:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.160+0000 7f7522800700 1 Processor -- start 2024-04-26T01:52:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.160+0000 7f7522800700 1 -- start start 2024-04-26T01:52:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.161+0000 7f7522800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f751c1092d0 0x7f751c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.161+0000 7f7522800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f751c109c80 0x7f751c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.161+0000 7f7522800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f751c10e3b0 0x7f751c1a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.161+0000 7f7522800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f751c1192c0 con 0x7f751c1092d0 2024-04-26T01:52:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.161+0000 7f7522800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f751c119140 con 0x7f751c109c80 2024-04-26T01:52:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.161+0000 7f7522800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f751c119440 con 0x7f751c10e3b0 2024-04-26T01:52:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.161+0000 7f751b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f751c1092d0 0x7f751c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:00.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.161+0000 7f751b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f751c1092d0 0x7f751c19f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56470/0 (socket says 172.21.15.1:56470) 2024-04-26T01:52:00.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.161+0000 7f751b400700 1 -- 172.21.15.1:0/1944485116 learned_addr learned my addr 172.21.15.1:0/1944485116 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:00.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.161+0000 7f751be00700 1 --2- 172.21.15.1:0/1944485116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f751c10e3b0 0x7f751c1a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:00.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.162+0000 7f751aa00700 1 --2- 172.21.15.1:0/1944485116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f751c109c80 0x7f751c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:00.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.162+0000 7f751b400700 1 -- 172.21.15.1:0/1944485116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f751c10e3b0 msgr2=0x7f751c1a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:00.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.162+0000 7f751b400700 1 --2- 172.21.15.1:0/1944485116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f751c10e3b0 0x7f751c1a4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:00.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.162+0000 7f751b400700 1 -- 172.21.15.1:0/1944485116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f751c109c80 msgr2=0x7f751c19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:00.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.162+0000 7f751b400700 1 --2- 172.21.15.1:0/1944485116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f751c109c80 0x7f751c19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:00.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.162+0000 7f751b400700 1 -- 172.21.15.1:0/1944485116 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751c1a4750 con 0x7f751c1092d0 2024-04-26T01:52:00.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.162+0000 7f751b400700 1 --2- 172.21.15.1:0/1944485116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f751c1092d0 0x7f751c19f790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f750c007f10 tx=0x7f750c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:00.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.163+0000 7f7513e00700 1 -- 172.21.15.1:0/1944485116 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c004db0 con 0x7f751c1092d0 2024-04-26T01:52:00.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.163+0000 7f7513e00700 1 -- 172.21.15.1:0/1944485116 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f750c003a00 con 0x7f751c1092d0 2024-04-26T01:52:00.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.163+0000 7f7513e00700 1 -- 172.21.15.1:0/1944485116 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c0045e0 con 0x7f751c1092d0 2024-04-26T01:52:00.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.163+0000 7f7522800700 1 -- 172.21.15.1:0/1944485116 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751c1a4a40 con 0x7f751c1092d0 2024-04-26T01:52:00.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.163+0000 7f7522800700 1 -- 172.21.15.1:0/1944485116 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f751c1ac2c0 con 0x7f751c1092d0 2024-04-26T01:52:00.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.164+0000 7f7522800700 1 -- 172.21.15.1:0/1944485116 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f751c069730 con 0x7f751c1092d0 2024-04-26T01:52:00.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.166+0000 7f7513e00700 1 -- 172.21.15.1:0/1944485116 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138872+0+0 (secure 0 0 0) 0x7f750c01c020 con 0x7f751c1092d0 2024-04-26T01:52:00.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.167+0000 7f7513e00700 1 --2- 172.21.15.1:0/1944485116 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f75000a3ed0 0x7f75000a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:00.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.168+0000 7f7513e00700 1 -- 172.21.15.1:0/1944485116 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f750c0d0390 con 0x7f751c1092d0 2024-04-26T01:52:00.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.168+0000 7f751aa00700 1 --2- 172.21.15.1:0/1944485116 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f75000a3ed0 0x7f75000a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:00.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.169+0000 7f751aa00700 1 --2- 172.21.15.1:0/1944485116 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f75000a3ed0 0x7f75000a6390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f751c108830 tx=0x7f7508009260 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:00.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.172+0000 7f7513e00700 1 -- 172.21.15.1:0/1944485116 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f750c050ff0 con 0x7f751c1092d0 2024-04-26T01:52:00.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:00 smithi173 ceph-mon[32553]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-26T01:52:00.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:00 smithi001 ceph-mon[27822]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-26T01:52:00.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.460+0000 7f7522800700 1 -- 172.21.15.1:0/1944485116 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f751c02df80 con 0x7f751c1092d0 2024-04-26T01:52:00.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.462+0000 7f7513e00700 1 -- 172.21.15.1:0/1944485116 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11841 (secure 0 0 0) 0x7f750c010070 con 0x7f751c1092d0 2024-04-26T01:52:00.460 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:52:00.460 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:51:38.222728+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2952462325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2952462325},{"type":"v1","addr":"172.21.15.173:6835","nonce":2952462325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24499,24514,24518]},"id":1}]} 2024-04-26T01:52:00.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.464+0000 7f7522800700 1 -- 172.21.15.1:0/1944485116 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f75000a3ed0 msgr2=0x7f75000a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:00.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.464+0000 7f7522800700 1 --2- 172.21.15.1:0/1944485116 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f75000a3ed0 0x7f75000a6390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f751c108830 tx=0x7f7508009260 comp rx=0 tx=0).stop 2024-04-26T01:52:00.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.464+0000 7f7522800700 1 -- 172.21.15.1:0/1944485116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f751c1092d0 msgr2=0x7f751c19f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:00.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.465+0000 7f7522800700 1 --2- 172.21.15.1:0/1944485116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f751c1092d0 0x7f751c19f790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f750c007f10 tx=0x7f750c00d3b0 comp rx=0 tx=0).stop 2024-04-26T01:52:00.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.465+0000 7f7522800700 1 -- 172.21.15.1:0/1944485116 shutdown_connections 2024-04-26T01:52:00.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.465+0000 7f7522800700 1 --2- 172.21.15.1:0/1944485116 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f75000a3ed0 0x7f75000a6390 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:00.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.465+0000 7f7522800700 1 --2- 172.21.15.1:0/1944485116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f751c10e3b0 0x7f751c1a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:00.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.465+0000 7f7522800700 1 --2- 172.21.15.1:0/1944485116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f751c1092d0 0x7f751c19f790 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:00.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.465+0000 7f7522800700 1 --2- 172.21.15.1:0/1944485116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f751c109c80 0x7f751c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:00.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.465+0000 7f7522800700 1 -- 172.21.15.1:0/1944485116 >> 172.21.15.1:0/1944485116 conn(0x7f751c076210 msgr2=0x7f751c0767d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:00.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.465+0000 7f7522800700 1 -- 172.21.15.1:0/1944485116 shutdown_connections 2024-04-26T01:52:00.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:00.465+0000 7f7522800700 1 -- 172.21.15.1:0/1944485116 wait complete. 2024-04-26T01:52:00.465 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 26 2024-04-26T01:52:00.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:00 smithi097 ceph-mon[32853]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-26T01:52:01.054 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:52:00 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:52:00.993+0000 7f8cdd400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:52:01.055 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:52:00 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:52:00.993+0000 7f8cdd400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:52:01.055 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:52:00 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:52:00.993+0000 7f8cdd400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:52:01.055 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:52:00 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:52:00.993+0000 7f8cdd400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:52:01.055 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 01:52:00 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T01:52:00.993+0000 7f8cdd400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:52:01.280 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:52:01.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:01 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1944485116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:52:01.361 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:01 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1944485116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:52:01.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:01 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1944485116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:52:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:02.212 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:02 smithi001 ceph-mon[27822]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-26T01:52:02.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:02 smithi173 ceph-mon[32553]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-26T01:52:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:02 smithi097 ceph-mon[32853]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-26T01:52:03.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.980+0000 7f976d400700 1 -- 172.21.15.1:0/2756494166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9768102ad0 msgr2=0x7f9768102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:03.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.980+0000 7f976d400700 1 --2- 172.21.15.1:0/2756494166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9768102ad0 0x7f9768102f50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9754009990 tx=0x7f9754009ca0 comp rx=0 tx=0).stop 2024-04-26T01:52:03.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.980+0000 7f976d400700 1 -- 172.21.15.1:0/2756494166 shutdown_connections 2024-04-26T01:52:03.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.980+0000 7f976d400700 1 --2- 172.21.15.1:0/2756494166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9768103490 0x7f976810f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:03.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.980+0000 7f976d400700 1 --2- 172.21.15.1:0/2756494166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9768108aa0 0x7f9768108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:03.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.980+0000 7f976d400700 1 --2- 172.21.15.1:0/2756494166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9768102ad0 0x7f9768102f50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:03.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.980+0000 7f976d400700 1 -- 172.21.15.1:0/2756494166 >> 172.21.15.1:0/2756494166 conn(0x7f9768079c30 msgr2=0x7f976807a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:03.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.980+0000 7f976d400700 1 -- 172.21.15.1:0/2756494166 shutdown_connections 2024-04-26T01:52:03.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.980+0000 7f976d400700 1 -- 172.21.15.1:0/2756494166 wait complete. 2024-04-26T01:52:03.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.981+0000 7f976d400700 1 Processor -- start 2024-04-26T01:52:03.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.981+0000 7f976d400700 1 -- start start 2024-04-26T01:52:03.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.982+0000 7f976d400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9768102ad0 0x7f9768074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:03.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.982+0000 7f976d400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9768103490 0x7f9768075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:03.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.982+0000 7f976d400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9768108aa0 0x7f97680757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:03.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.982+0000 7f976d400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9768112c90 con 0x7f9768103490 2024-04-26T01:52:03.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.982+0000 7f976d400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9768112b10 con 0x7f9768108aa0 2024-04-26T01:52:03.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.982+0000 7f976d400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9768112e10 con 0x7f9768102ad0 2024-04-26T01:52:03.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.982+0000 7f9765e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9768103490 0x7f9768075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:03.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.982+0000 7f9766800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9768102ad0 0x7f9768074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:03.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.982+0000 7f9765e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9768103490 0x7f9768075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42132/0 (socket says 172.21.15.1:42132) 2024-04-26T01:52:03.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.982+0000 7f9765e00700 1 -- 172.21.15.1:0/3974315561 learned_addr learned my addr 172.21.15.1:0/3974315561 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:03.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.982+0000 7f9767200700 1 --2- 172.21.15.1:0/3974315561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9768108aa0 0x7f97680757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:03.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.983+0000 7f9765e00700 1 -- 172.21.15.1:0/3974315561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9768102ad0 msgr2=0x7f9768074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:03.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.983+0000 7f9765e00700 1 --2- 172.21.15.1:0/3974315561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9768102ad0 0x7f9768074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:03.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.983+0000 7f9765e00700 1 -- 172.21.15.1:0/3974315561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9768108aa0 msgr2=0x7f97680757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:03.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.983+0000 7f9765e00700 1 --2- 172.21.15.1:0/3974315561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9768108aa0 0x7f97680757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:03.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.983+0000 7f9765e00700 1 -- 172.21.15.1:0/3974315561 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97681a7700 con 0x7f9768103490 2024-04-26T01:52:03.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.983+0000 7f9767200700 1 --2- 172.21.15.1:0/3974315561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9768108aa0 0x7f97680757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:03.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.983+0000 7f9765e00700 1 --2- 172.21.15.1:0/3974315561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9768103490 0x7f9768075280 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f97540057f0 tx=0x7f975400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:03.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.983+0000 7f975f400700 1 -- 172.21.15.1:0/3974315561 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754026070 con 0x7f9768103490 2024-04-26T01:52:03.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.983+0000 7f975f400700 1 -- 172.21.15.1:0/3974315561 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9754021410 con 0x7f9768103490 2024-04-26T01:52:03.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.983+0000 7f976d400700 1 -- 172.21.15.1:0/3974315561 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97681a7990 con 0x7f9768103490 2024-04-26T01:52:03.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.983+0000 7f975f400700 1 -- 172.21.15.1:0/3974315561 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754017650 con 0x7f9768103490 2024-04-26T01:52:03.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.984+0000 7f976d400700 1 -- 172.21.15.1:0/3974315561 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97681a7e70 con 0x7f9768103490 2024-04-26T01:52:03.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.985+0000 7f976d400700 1 -- 172.21.15.1:0/3974315561 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9768069730 con 0x7f9768103490 2024-04-26T01:52:03.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.987+0000 7f975f400700 1 -- 172.21.15.1:0/3974315561 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7f9754029070 con 0x7f9768103490 2024-04-26T01:52:03.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.988+0000 7f975f400700 1 --2- 172.21.15.1:0/3974315561 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f974c0a3f60 0x7f974c0a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.988+0000 7f975f400700 1 -- 172.21.15.1:0/3974315561 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f97540d1700 con 0x7f9768103490 2024-04-26T01:52:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.989+0000 7f9766800700 1 --2- 172.21.15.1:0/3974315561 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f974c0a3f60 0x7f974c0a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.989+0000 7f9766800700 1 --2- 172.21.15.1:0/3974315561 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f974c0a3f60 0x7f974c0a6420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9758009be0 tx=0x7f9758009260 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:03.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:03.993+0000 7f975f400700 1 -- 172.21.15.1:0/3974315561 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f975405dff0 con 0x7f9768103490 2024-04-26T01:52:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:04.225+0000 7f976d400700 1 -- 172.21.15.1:0/3974315561 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f976802df80 con 0x7f9768103490 2024-04-26T01:52:04.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:04.227+0000 7f975f400700 1 -- 172.21.15.1:0/3974315561 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+23017 (secure 0 0 0) 0x7f9754013070 con 0x7f9768103490 2024-04-26T01:52:04.226 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:52:04.226 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":93,"fsid":"77cfcc80-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:38:03.531182+0000","modified":"2024-04-26T01:51:38.660565+0000","last_up_change":"2024-04-26T01:46:16.151275+0000","last_in_change":"2024-04-26T01:45:56.514228+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:42:04.805488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:47:21.957421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:47:25.860260+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb10ae12-09ee-4d6b-bc78-8c6016613b81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6803","nonce":3176633965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6805","nonce":3176633965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6809","nonce":3176633965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6807","nonce":3176633965}]},"public_addr":"172.21.15.1:6803/3176633965","cluster_addr":"172.21.15.1:6805/3176633965","heartbeat_back_addr":"172.21.15.1:6809/3176633965","heartbeat_front_addr":"172.21.15.1:6807/3176633965","state":["exists","up"]},{"osd":1,"uuid":"acd12888-6ff6-40bc-99e7-e5c395e85b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6801","nonce":3755022621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6803","nonce":3755022621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6807","nonce":3755022621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6805","nonce":3755022621}]},"public_addr":"172.21.15.97:6801/3755022621","cluster_addr":"172.21.15.97:6803/3755022621","heartbeat_back_addr":"172.21.15.97:6807/3755022621","heartbeat_front_addr":"172.21.15.97:6805/3755022621","state":["exists","up"]},{"osd":2,"uuid":"79971508-e256-4145-8111-a75666e886fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6801","nonce":734861771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6803","nonce":734861771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6807","nonce":734861771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6805","nonce":734861771}]},"public_addr":"172.21.15.173:6801/734861771","cluster_addr":"172.21.15.173:6803/734861771","heartbeat_back_addr":"172.21.15.173:6807/734861771","heartbeat_front_addr":"172.21.15.173:6805/734861771","state":["exists","up"]},{"osd":3,"uuid":"a8ccc4a9-0f31-446a-8bcb-4691532397c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6811","nonce":3349961961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6813","nonce":3349961961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6817","nonce":3349961961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6815","nonce":3349961961}]},"public_addr":"172.21.15.1:6811/3349961961","cluster_addr":"172.21.15.1:6813/3349961961","heartbeat_back_addr":"172.21.15.1:6817/3349961961","heartbeat_front_addr":"172.21.15.1:6815/3349961961","state":["exists","up"]},{"osd":4,"uuid":"5b724648-4df7-40fc-89d5-55f170cdd806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6809","nonce":2494639397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6811","nonce":2494639397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6815","nonce":2494639397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6813","nonce":2494639397}]},"public_addr":"172.21.15.97:6809/2494639397","cluster_addr":"172.21.15.97:6811/2494639397","heartbeat_back_addr":"172.21.15.97:6815/2494639397","heartbeat_front_addr":"172.21.15.97:6813/2494639397","state":["exists","up"]},{"osd":5,"uuid":"a5a24a9c-3007-4b87-8562-40c98c41ef8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6809","nonce":1026507957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6811","nonce":1026507957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6815","nonce":1026507957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6813","nonce":1026507957}]},"public_addr":"172.21.15.173:6809/1026507957","cluster_addr":"172.21.15.173:6811/1026507957","heartbeat_back_addr":"172.21.15.173:6815/1026507957","heartbeat_front_addr":"172.21.15.173:6813/1026507957","state":["exists","up"]},{"osd":6,"uuid":"97cd2e88-a56c-4b1a-8104-65b23a77986a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6819","nonce":444030587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6821","nonce":444030587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6825","nonce":444030587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6823","nonce":444030587}]},"public_addr":"172.21.15.1:6819/444030587","cluster_addr":"172.21.15.1:6821/444030587","heartbeat_back_addr":"172.21.15.1:6825/444030587","heartbeat_front_addr":"172.21.15.1:6823/444030587","state":["exists","up"]},{"osd":7,"uuid":"afc0c2dd-e112-4358-b3aa-944366d5d383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6817","nonce":2600022531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6819","nonce":2600022531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6823","nonce":2600022531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6821","nonce":2600022531}]},"public_addr":"172.21.15.97:6817/2600022531","cluster_addr":"172.21.15.97:6819/2600022531","heartbeat_back_addr":"172.21.15.97:6823/2600022531","heartbeat_front_addr":"172.21.15.97:6821/2600022531","state":["exists","up"]},{"osd":8,"uuid":"41fa612b-793e-44da-af8b-44d17a9de303","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6817","nonce":1334205527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6819","nonce":1334205527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6823","nonce":1334205527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6821","nonce":1334205527}]},"public_addr":"172.21.15.173:6817/1334205527","cluster_addr":"172.21.15.173:6819/1334205527","heartbeat_back_addr":"172.21.15.173:6823/1334205527","heartbeat_front_addr":"172.21.15.173:6821/1334205527","state":["exists","up"]},{"osd":9,"uuid":"bbbb163b-f8c4-4be5-8f80-82445e8d407f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6827","nonce":2973486839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6829","nonce":2973486839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6833","nonce":2973486839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6831","nonce":2973486839}]},"public_addr":"172.21.15.1:6827/2973486839","cluster_addr":"172.21.15.1:6829/2973486839","heartbeat_back_addr":"172.21.15.1:6833/2973486839","heartbeat_front_addr":"172.21.15.1:6831/2973486839","state":["exists","up"]},{"osd":10,"uuid":"1b447b34-0600-4727-826d-b3d68adfb18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6825","nonce":1693425268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6827","nonce":1693425268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6831","nonce":1693425268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6829","nonce":1693425268}]},"public_addr":"172.21.15.97:6825/1693425268","cluster_addr":"172.21.15.97:6827/1693425268","heartbeat_back_addr":"172.21.15.97:6831/1693425268","heartbeat_front_addr":"172.21.15.97:6829/1693425268","state":["exists","up"]},{"osd":11,"uuid":"f439aae1-f50a-4176-889d-36fa5131541a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6825","nonce":1141524960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6827","nonce":1141524960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6831","nonce":1141524960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6829","nonce":1141524960}]},"public_addr":"172.21.15.173:6825/1141524960","cluster_addr":"172.21.15.173:6827/1141524960","heartbeat_back_addr":"172.21.15.173:6831/1141524960","heartbeat_front_addr":"172.21.15.173:6829/1141524960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:12.955643+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:38.846652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:02.556062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:34.900933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:01.170878+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:24.842573+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:58.304440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:24.868689+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:48.772003+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:22.611681+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:49.470779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:14.163984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1607430337":"2024-04-27T01:51:38.660496+0000","172.21.15.1:0/25191068":"2024-04-27T01:51:38.660496+0000","172.21.15.1:6801/2140737121":"2024-04-27T01:51:38.660496+0000","172.21.15.1:0/304803917":"2024-04-27T01:51:38.660496+0000","172.21.15.1:6801/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6800/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/1000222947":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6800/2140737121":"2024-04-27T01:51:38.660496+0000","172.21.15.1:0/457958597":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6801/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:6800/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/941157221":"2024-04-27T01:51:38.660496+0000","172.21.15.1:0/2715798962":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/1923058728":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/2489038777":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/2869502448":"2024-04-27T01:38:45.974031+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:52:04.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:04.230+0000 7f976d400700 1 -- 172.21.15.1:0/3974315561 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f974c0a3f60 msgr2=0x7f974c0a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:04.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:04.230+0000 7f976d400700 1 --2- 172.21.15.1:0/3974315561 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f974c0a3f60 0x7f974c0a6420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9758009be0 tx=0x7f9758009260 comp rx=0 tx=0).stop 2024-04-26T01:52:04.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:04.230+0000 7f976d400700 1 -- 172.21.15.1:0/3974315561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9768103490 msgr2=0x7f9768075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:04.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:04.230+0000 7f976d400700 1 --2- 172.21.15.1:0/3974315561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9768103490 0x7f9768075280 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f97540057f0 tx=0x7f975400bfd0 comp rx=0 tx=0).stop 2024-04-26T01:52:04.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:04.230+0000 7f976d400700 1 -- 172.21.15.1:0/3974315561 shutdown_connections 2024-04-26T01:52:04.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:04.230+0000 7f976d400700 1 --2- 172.21.15.1:0/3974315561 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f974c0a3f60 0x7f974c0a6420 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:04.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:04.230+0000 7f976d400700 1 --2- 172.21.15.1:0/3974315561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9768108aa0 0x7f97680757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:04.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:04.230+0000 7f976d400700 1 --2- 172.21.15.1:0/3974315561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9768102ad0 0x7f9768074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:04.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:04.230+0000 7f976d400700 1 --2- 172.21.15.1:0/3974315561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9768103490 0x7f9768075280 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:04.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:04.230+0000 7f976d400700 1 -- 172.21.15.1:0/3974315561 >> 172.21.15.1:0/3974315561 conn(0x7f9768079c30 msgr2=0x7f9768100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:04.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:04.230+0000 7f976d400700 1 -- 172.21.15.1:0/3974315561 shutdown_connections 2024-04-26T01:52:04.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:04.231+0000 7f976d400700 1 -- 172.21.15.1:0/3974315561 wait complete. 2024-04-26T01:52:04.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:04 smithi001 ceph-mon[27822]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T01:52:04.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:04 smithi001 ceph-mon[27822]: mgrmap e25: x(active, since 24s), standbys: y, z 2024-04-26T01:52:04.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:04 smithi097 ceph-mon[32853]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T01:52:04.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:04 smithi097 ceph-mon[32853]: mgrmap e25: x(active, since 24s), standbys: y, z 2024-04-26T01:52:04.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:04 smithi173 ceph-mon[32553]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T01:52:04.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:04 smithi173 ceph-mon[32553]: mgrmap e25: x(active, since 24s), standbys: y, z 2024-04-26T01:52:05.118 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-04-26T01:52:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:05 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3974315561' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:52:05.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:05 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3974315561' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:52:05.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:05 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3974315561' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:52:05.604 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:06.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:06 smithi001 ceph-mon[27822]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T01:52:06.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:06 smithi097 ceph-mon[32853]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T01:52:06.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:06 smithi173 ceph-mon[32553]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T01:52:08.293 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:08 smithi001 ceph-mon[27822]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-26T01:52:08.368 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:08 smithi173 ceph-mon[32553]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-26T01:52:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:08 smithi097 ceph-mon[32853]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-26T01:52:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.490+0000 7fdaf9400640 1 -- 172.21.15.1:0/1947604901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaf4102960 msgr2=0x7fdaf41069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.490+0000 7fdaf9400640 1 --2- 172.21.15.1:0/1947604901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaf4102960 0x7fdaf41069a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009900 tx=0x7fdae002a3f0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.490+0000 7fdaf9400640 1 -- 172.21.15.1:0/1947604901 shutdown_connections 2024-04-26T01:52:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.490+0000 7fdaf9400640 1 --2- 172.21.15.1:0/1947604901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf4107080 0x7fdaf410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.490+0000 7fdaf9400640 1 --2- 172.21.15.1:0/1947604901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaf4102960 0x7fdaf41069a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.490+0000 7fdaf9400640 1 --2- 172.21.15.1:0/1947604901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaf4101fb0 0x7fdaf4102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.490+0000 7fdaf9400640 1 -- 172.21.15.1:0/1947604901 >> 172.21.15.1:0/1947604901 conn(0x7fdaf40fd890 msgr2=0x7fdaf40ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.490+0000 7fdaf9400640 1 -- 172.21.15.1:0/1947604901 shutdown_connections 2024-04-26T01:52:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.491+0000 7fdaf9400640 1 -- 172.21.15.1:0/1947604901 wait complete. 2024-04-26T01:52:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.491+0000 7fdaf9400640 1 Processor -- start 2024-04-26T01:52:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.492+0000 7fdaf9400640 1 -- start start 2024-04-26T01:52:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.492+0000 7fdaf9400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaf4101fb0 0x7fdaf41985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.492+0000 7fdaf9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf4107080 0x7fdaf4198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.492+0000 7fdaf9400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaf419ce60 0x7fdaf419d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.492+0000 7fdaf9400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf4111ff0 con 0x7fdaf419ce60 2024-04-26T01:52:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.492+0000 7fdaf9400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf4111e70 con 0x7fdaf4107080 2024-04-26T01:52:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.492+0000 7fdaf9400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf4112170 con 0x7fdaf4101fb0 2024-04-26T01:52:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.492+0000 7fdaf8a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaf419ce60 0x7fdaf419d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.492+0000 7fdaf8a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaf419ce60 0x7fdaf419d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42154/0 (socket says 172.21.15.1:42154) 2024-04-26T01:52:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.492+0000 7fdaf8a00640 1 -- 172.21.15.1:0/467120968 learned_addr learned my addr 172.21.15.1:0/467120968 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.492+0000 7fdaf3e00640 1 --2- 172.21.15.1:0/467120968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaf4101fb0 0x7fdaf41985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.493+0000 7fdaf3400640 1 --2- 172.21.15.1:0/467120968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf4107080 0x7fdaf4198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.493+0000 7fdaf8a00640 1 -- 172.21.15.1:0/467120968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaf4101fb0 msgr2=0x7fdaf41985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.493+0000 7fdaf8a00640 1 --2- 172.21.15.1:0/467120968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaf4101fb0 0x7fdaf41985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.493+0000 7fdaf8a00640 1 -- 172.21.15.1:0/467120968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf4107080 msgr2=0x7fdaf4198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.493+0000 7fdaf8a00640 1 --2- 172.21.15.1:0/467120968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf4107080 0x7fdaf4198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.493+0000 7fdaf8a00640 1 -- 172.21.15.1:0/467120968 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf419d950 con 0x7fdaf419ce60 2024-04-26T01:52:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.493+0000 7fdaf8a00640 1 --2- 172.21.15.1:0/467120968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaf419ce60 0x7fdaf419d310 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00eb10 tx=0x7fdaec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.493+0000 7fdaf0c00640 1 -- 172.21.15.1:0/467120968 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec004b20 con 0x7fdaf419ce60 2024-04-26T01:52:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.493+0000 7fdaf0c00640 1 -- 172.21.15.1:0/467120968 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaec004cc0 con 0x7fdaf419ce60 2024-04-26T01:52:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.494+0000 7fdaf9400640 1 -- 172.21.15.1:0/467120968 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf419dc40 con 0x7fdaf419ce60 2024-04-26T01:52:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.494+0000 7fdaf0c00640 1 -- 172.21.15.1:0/467120968 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec010670 con 0x7fdaf419ce60 2024-04-26T01:52:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.494+0000 7fdaf9400640 1 -- 172.21.15.1:0/467120968 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf41a53b0 con 0x7fdaf419ce60 2024-04-26T01:52:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.497+0000 7fdafd82f740 1 Processor -- start 2024-04-26T01:52:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.497+0000 7fdaf0c00640 1 -- 172.21.15.1:0/467120968 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7fdaec005020 con 0x7fdaf419ce60 2024-04-26T01:52:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.498+0000 7fdafd82f740 1 -- start start 2024-04-26T01:52:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.498+0000 7fdafd82f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce12178b0 0x564ce1217c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.498+0000 7fdafd82f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce12181d0 0x564ce1226a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.498+0000 7fdafd82f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce1226fa0 0x564ce1229390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.498+0000 7fdafd82f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564ce117d900 con 0x564ce12178b0 2024-04-26T01:52:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.498+0000 7fdafd82f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564ce1123ef0 con 0x564ce1226fa0 2024-04-26T01:52:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.498+0000 7fdafd82f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564ce11798c0 con 0x564ce12181d0 2024-04-26T01:52:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.498+0000 7fdaf3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce12181d0 0x564ce1226a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.498+0000 7fdaf3400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce12178b0 0x564ce1217c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.498+0000 7fdaf3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce12181d0 0x564ce1226a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:43582/0 (socket says 172.21.15.1:43582) 2024-04-26T01:52:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.498+0000 7fdaf3e00640 1 -- 172.21.15.1:0/2241003192 learned_addr learned my addr 172.21.15.1:0/2241003192 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.499+0000 7fdaf3400640 1 --2- 172.21.15.1:0/2241003192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce1226fa0 0x564ce1229390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.499+0000 7fdaf3400640 1 -- 172.21.15.1:0/2241003192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce12181d0 msgr2=0x564ce1226a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.499+0000 7fdaf3400640 1 --2- 172.21.15.1:0/2241003192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce12181d0 0x564ce1226a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.499+0000 7fdaf3400640 1 -- 172.21.15.1:0/2241003192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce1226fa0 msgr2=0x564ce1229390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.499+0000 7fdaf3400640 1 --2- 172.21.15.1:0/2241003192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce1226fa0 0x564ce1229390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.499+0000 7fdaf3400640 1 -- 172.21.15.1:0/2241003192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ce0e90210 con 0x564ce12178b0 2024-04-26T01:52:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.499+0000 7fdaf3e00640 1 --2- 172.21.15.1:0/2241003192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce12181d0 0x564ce1226a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.499+0000 7fdaf0c00640 1 --2- 172.21.15.1:0/467120968 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fdad80a3eb0 0x7fdad80a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.499+0000 7fdaf3400640 1 --2- 172.21.15.1:0/2241003192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce12178b0 0x564ce1217c90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdae002a910 tx=0x7fdae0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f339372195c3dfce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.499+0000 7fdaf0c00640 1 -- 172.21.15.1:0/467120968 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7fdaec0d0fe0 con 0x7fdaf419ce60 2024-04-26T01:52:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.499+0000 7fdaf3e00640 1 --2- 172.21.15.1:0/467120968 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fdad80a3eb0 0x7fdad80a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.500+0000 7fdae9200640 1 -- 172.21.15.1:0/2241003192 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0044020 con 0x564ce12178b0 2024-04-26T01:52:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.500+0000 7fdae9200640 1 -- 172.21.15.1:0/2241003192 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdae0004750 con 0x564ce12178b0 2024-04-26T01:52:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.500+0000 7fdae9200640 1 -- 172.21.15.1:0/2241003192 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0035040 con 0x564ce12178b0 2024-04-26T01:52:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.500+0000 7fdafd82f740 1 -- 172.21.15.1:0/2241003192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce12178b0 msgr2=0x564ce1217c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.500+0000 7fdafd82f740 1 --2- 172.21.15.1:0/2241003192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce12178b0 0x564ce1217c90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdae002a910 tx=0x7fdae0005e50 comp rx=0 tx=0).stop 2024-04-26T01:52:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.500+0000 7fdaf3e00640 1 --2- 172.21.15.1:0/467120968 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fdad80a3eb0 0x7fdad80a6370 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x564ce121a5a0 tx=0x7fdae4005f90 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.501+0000 7fdafd82f740 1 -- 172.21.15.1:0/2241003192 shutdown_connections 2024-04-26T01:52:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.501+0000 7fdafd82f740 1 --2- 172.21.15.1:0/2241003192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce1226fa0 0x564ce1229390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.501+0000 7fdafd82f740 1 --2- 172.21.15.1:0/2241003192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce12181d0 0x564ce1226a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.501+0000 7fdafd82f740 1 --2- 172.21.15.1:0/2241003192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce12178b0 0x564ce1217c90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.501+0000 7fdafd82f740 1 -- 172.21.15.1:0/2241003192 >> 172.21.15.1:0/2241003192 conn(0x564ce1213ec0 msgr2=0x564ce12142a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.501+0000 7fdafd82f740 1 -- 172.21.15.1:0/2241003192 shutdown_connections 2024-04-26T01:52:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.501+0000 7fdafd82f740 1 -- 172.21.15.1:0/2241003192 wait complete. 2024-04-26T01:52:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.501+0000 7fdafd82f740 1 -- start start 2024-04-26T01:52:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.501+0000 7fdafd82f740 1 Processor -- start 2024-04-26T01:52:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.502+0000 7fdafd82f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce12178b0 0x564ce1223fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.502+0000 7fdafd82f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce12181d0 0x564ce12244f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.502+0000 7fdafd82f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce1226fa0 0x564ce1224a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.502+0000 7fdafd82f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564ce1229b70 con 0x564ce12178b0 2024-04-26T01:52:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.502+0000 7fdafd82f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564ce1137310 con 0x564ce12181d0 2024-04-26T01:52:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.502+0000 7fdafd82f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564ce1229cf0 con 0x564ce1226fa0 2024-04-26T01:52:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.502+0000 7fdafd82f740 10 client.0 _ll_register_callbacks cb 0x564ce1105cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.502+0000 7fdaf3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce12178b0 0x564ce1223fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.502+0000 7fdaf3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce12178b0 0x564ce1223fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42164/0 (socket says 172.21.15.1:42164) 2024-04-26T01:52:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.502+0000 7fdaf3e00640 1 -- 172.21.15.1:0/757596585 learned_addr learned my addr 172.21.15.1:0/757596585 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.502+0000 7fdaf8a00640 1 --2- 172.21.15.1:0/757596585 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce1226fa0 0x564ce1224a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.502+0000 7fdaf3400640 1 --2- 172.21.15.1:0/757596585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce12181d0 0x564ce12244f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.503+0000 7fdaf3e00640 1 -- 172.21.15.1:0/757596585 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce1226fa0 msgr2=0x564ce1224a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.503+0000 7fdaf3e00640 1 --2- 172.21.15.1:0/757596585 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce1226fa0 0x564ce1224a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.503+0000 7fdaf3e00640 1 -- 172.21.15.1:0/757596585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce12181d0 msgr2=0x564ce12244f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.503+0000 7fdaf3e00640 1 --2- 172.21.15.1:0/757596585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce12181d0 0x564ce12244f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.503+0000 7fdaf3e00640 1 -- 172.21.15.1:0/757596585 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ce0f5c620 con 0x564ce12178b0 2024-04-26T01:52:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.503+0000 7fdaf8a00640 1 --2- 172.21.15.1:0/757596585 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce1226fa0 0x564ce1224a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.503+0000 7fdaf3e00640 1 --2- 172.21.15.1:0/757596585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce12178b0 0x564ce1223fb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdae400fa60 tx=0x7fdae4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c373d22ec40f3b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.503+0000 7fdadd600640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:52:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.503+0000 7fdadd600640 1 -- 172.21.15.1:0/757596585 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae402d060 con 0x564ce12178b0 2024-04-26T01:52:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.503+0000 7fdadd600640 1 -- 172.21.15.1:0/757596585 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7fdae4018030 con 0x564ce12178b0 2024-04-26T01:52:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.503+0000 7fdafd82f740 1 -- 172.21.15.1:0/757596585 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdae0004750 con 0x564ce12178b0 2024-04-26T01:52:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.504+0000 7fdadd600640 1 -- 172.21.15.1:0/757596585 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdae4028470 con 0x564ce12178b0 2024-04-26T01:52:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.504+0000 7fdafd82f740 10 client.15114 fetch_fsmap learned FSMap version 26 2024-04-26T01:52:08.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.504+0000 7fdafd82f740 1 -- 172.21.15.1:0/757596585 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564ce1235f40 con 0x564ce12178b0 2024-04-26T01:52:08.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.504+0000 7fdadd600640 1 -- 172.21.15.1:0/757596585 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdae0004750 con 0x564ce12178b0 2024-04-26T01:52:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.504+0000 7fdadd600640 1 -- 172.21.15.1:0/757596585 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdae401d6e0 con 0x564ce12178b0 2024-04-26T01:52:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.504+0000 7fdafd82f740 10 client.15114 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:52:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.504+0000 7fdafd82f740 10 client.15114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:52:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.504+0000 7fdafd82f740 20 client.15114 populate_metadata read hostname 'smithi001' 2024-04-26T01:52:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.504+0000 7fdafd82f740 1 --2- 172.21.15.1:0/757596585 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ce1237400 0x564ce12397f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.504+0000 7fdafd82f740 4 client.15114 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:52:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.504+0000 7fdafd82f740 1 -- 172.21.15.1:0/757596585 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ce11918b0 con 0x564ce1237400 2024-04-26T01:52:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.505+0000 7fdaf3400640 1 --2- 172.21.15.1:0/757596585 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ce1237400 0x564ce12397f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.505+0000 7fdaf3400640 1 --2- 172.21.15.1:0/757596585 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ce1237400 0x564ce12397f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41d5d6eef4cd7691 server_cookie=bfad5b831df39051 in_seq=0 out_seq=0 2024-04-26T01:52:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.505+0000 7fdadd600640 10 client.15114 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.506+0000 7fdadd600640 1 -- 172.21.15.1:0/757596585 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ce11918b0 con 0x564ce1237400 2024-04-26T01:52:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.506+0000 7fdadd600640 10 client.15114 handle_command_reply: tid=0 2024-04-26T01:52:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.506+0000 7fdafd82f740 1 client.15114 shutdown 2024-04-26T01:52:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.506+0000 7fdafd82f740 1 -- 172.21.15.1:0/757596585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce12178b0 msgr2=0x564ce1223fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.506+0000 7fdafd82f740 1 --2- 172.21.15.1:0/757596585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce12178b0 0x564ce1223fb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdae400fa60 tx=0x7fdae4004730 comp rx=0 tx=0).stop 2024-04-26T01:52:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.506+0000 7fdafd82f740 1 -- 172.21.15.1:0/757596585 shutdown_connections 2024-04-26T01:52:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.506+0000 7fdafd82f740 1 --2- 172.21.15.1:0/757596585 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ce1237400 0x564ce12397f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.506+0000 7fdafd82f740 1 --2- 172.21.15.1:0/757596585 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce1226fa0 0x564ce1224a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.506+0000 7fdafd82f740 1 --2- 172.21.15.1:0/757596585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce12181d0 0x564ce12244f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.506+0000 7fdafd82f740 1 --2- 172.21.15.1:0/757596585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce12178b0 0x564ce1223fb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.506+0000 7fdadd600640 0 client.15114 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.506+0000 7fdafd82f740 1 -- 172.21.15.1:0/757596585 >> 172.21.15.1:0/757596585 conn(0x564ce1213ec0 msgr2=0x564ce1219d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.507+0000 7fdafd82f740 1 -- 172.21.15.1:0/757596585 shutdown_connections 2024-04-26T01:52:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.507+0000 7fdafd82f740 1 -- 172.21.15.1:0/757596585 wait complete. 2024-04-26T01:52:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.507+0000 7fdafd82f740 20 client.15114 trim_cache size 0 max 16384 2024-04-26T01:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.523+0000 7fdafd82f740 1 Processor -- start 2024-04-26T01:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.524+0000 7fdafd82f740 1 -- start start 2024-04-26T01:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.524+0000 7fdafd82f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce12178b0 0x564ce1232a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.524+0000 7fdafd82f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce12181d0 0x564ce1212ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.524+0000 7fdafd82f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce1226fa0 0x564ce123f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.524+0000 7fdafd82f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41adff0 con 0x564ce1226fa0 2024-04-26T01:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.524+0000 7fdafd82f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41ab8c0 con 0x564ce12181d0 2024-04-26T01:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.524+0000 7fdafd82f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41a9710 con 0x564ce12178b0 2024-04-26T01:52:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.524+0000 7fdaf8a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce1226fa0 0x564ce123f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.524+0000 7fdaf8a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce1226fa0 0x564ce123f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42172/0 (socket says 172.21.15.1:42172) 2024-04-26T01:52:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.524+0000 7fdaf8a00640 1 -- 172.21.15.1:0/888080321 learned_addr learned my addr 172.21.15.1:0/888080321 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.524+0000 7fdaf3400640 1 --2- 172.21.15.1:0/888080321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce12181d0 0x564ce1212ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.525+0000 7fdaf3e00640 1 --2- 172.21.15.1:0/888080321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce12178b0 0x564ce1232a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.525+0000 7fdaf8a00640 1 -- 172.21.15.1:0/888080321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce12178b0 msgr2=0x564ce1232a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.525+0000 7fdaf8a00640 1 --2- 172.21.15.1:0/888080321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce12178b0 0x564ce1232a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.525+0000 7fdaf8a00640 1 -- 172.21.15.1:0/888080321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce12181d0 msgr2=0x564ce1212ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.525+0000 7fdaf8a00640 1 --2- 172.21.15.1:0/888080321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce12181d0 0x564ce1212ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.525+0000 7fdaf8a00640 1 -- 172.21.15.1:0/888080321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ce1222500 con 0x564ce1226fa0 2024-04-26T01:52:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.525+0000 7fdaf8a00640 1 --2- 172.21.15.1:0/888080321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce1226fa0 0x564ce123f090 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0996c0 tx=0x7fdaec09c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1123ca9dc507bee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.525+0000 7fdadfe00640 1 -- 172.21.15.1:0/888080321 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec09cd40 con 0x564ce1226fa0 2024-04-26T01:52:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.525+0000 7fdadfe00640 1 -- 172.21.15.1:0/888080321 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaec0a10f0 con 0x564ce1226fa0 2024-04-26T01:52:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.525+0000 7fdadfe00640 1 -- 172.21.15.1:0/888080321 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec0a5780 con 0x564ce1226fa0 2024-04-26T01:52:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.526+0000 7fdafd82f740 1 -- 172.21.15.1:0/888080321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce1226fa0 msgr2=0x564ce123f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.526+0000 7fdafd82f740 1 --2- 172.21.15.1:0/888080321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce1226fa0 0x564ce123f090 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0996c0 tx=0x7fdaec09c880 comp rx=0 tx=0).stop 2024-04-26T01:52:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.526+0000 7fdafd82f740 1 -- 172.21.15.1:0/888080321 shutdown_connections 2024-04-26T01:52:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.526+0000 7fdafd82f740 1 --2- 172.21.15.1:0/888080321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce1226fa0 0x564ce123f090 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.526+0000 7fdafd82f740 1 --2- 172.21.15.1:0/888080321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce12181d0 0x564ce1212ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.526+0000 7fdafd82f740 1 --2- 172.21.15.1:0/888080321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce12178b0 0x564ce1232a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.526+0000 7fdafd82f740 1 -- 172.21.15.1:0/888080321 >> 172.21.15.1:0/888080321 conn(0x564ce1213ec0 msgr2=0x564ce1212790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.526+0000 7fdafd82f740 1 -- 172.21.15.1:0/888080321 shutdown_connections 2024-04-26T01:52:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.526+0000 7fdafd82f740 1 -- 172.21.15.1:0/888080321 wait complete. 2024-04-26T01:52:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.527+0000 7fdafd82f740 1 -- start start 2024-04-26T01:52:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.527+0000 7fdafd82f740 1 Processor -- start 2024-04-26T01:52:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.527+0000 7fdafd82f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce12178b0 0x564ce122ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.527+0000 7fdafd82f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce12181d0 0x564ce122d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.527+0000 7fdafd82f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce1226fa0 0x564ce129c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.527+0000 7fdafd82f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41a9d80 con 0x564ce12181d0 2024-04-26T01:52:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.527+0000 7fdafd82f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41ab1f0 con 0x564ce12178b0 2024-04-26T01:52:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.527+0000 7fdafd82f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41aa450 con 0x564ce1226fa0 2024-04-26T01:52:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.527+0000 7fdafd82f740 10 client.0 _ll_register_callbacks cb 0x7fdae400f820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.528+0000 7fdaf3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce12178b0 0x564ce122ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.528+0000 7fdaf3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce12178b0 0x564ce122ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:55094/0 (socket says 172.21.15.1:55094) 2024-04-26T01:52:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.528+0000 7fdaf3e00640 1 -- 172.21.15.1:0/609510919 learned_addr learned my addr 172.21.15.1:0/609510919 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.528+0000 7fdaf3400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce12181d0 0x564ce122d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.528+0000 7fdaf8a00640 1 --2- 172.21.15.1:0/609510919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce1226fa0 0x564ce129c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.528+0000 7fdaf3e00640 1 -- 172.21.15.1:0/609510919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce1226fa0 msgr2=0x564ce129c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.528+0000 7fdaf3e00640 1 --2- 172.21.15.1:0/609510919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce1226fa0 0x564ce129c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.528+0000 7fdaf3e00640 1 -- 172.21.15.1:0/609510919 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce12181d0 msgr2=0x564ce122d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.528+0000 7fdaf3e00640 1 --2- 172.21.15.1:0/609510919 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce12181d0 0x564ce122d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.528+0000 7fdaf3e00640 1 -- 172.21.15.1:0/609510919 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ce122da90 con 0x564ce12178b0 2024-04-26T01:52:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.528+0000 7fdaf3400640 1 --2- 172.21.15.1:0/609510919 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce12181d0 0x564ce122d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:52:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.528+0000 7fdaf3e00640 1 --2- 172.21.15.1:0/609510919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce12178b0 0x564ce122ccb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdae40092e0 tx=0x7fdae4004620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c3a96ebc42ad33d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.528+0000 7fdadd600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:52:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.529+0000 7fdadd600640 1 -- 172.21.15.1:0/609510919 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae401d8f0 con 0x564ce12178b0 2024-04-26T01:52:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.529+0000 7fdadd600640 1 -- 172.21.15.1:0/609510919 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7fdae4028820 con 0x564ce12178b0 2024-04-26T01:52:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.529+0000 7fdafd82f740 1 -- 172.21.15.1:0/609510919 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaec0a10f0 con 0x564ce12178b0 2024-04-26T01:52:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.529+0000 7fdadd600640 1 -- 172.21.15.1:0/609510919 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdae402d030 con 0x564ce12178b0 2024-04-26T01:52:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.529+0000 7fdafd82f740 10 client.25241 fetch_fsmap learned FSMap version 26 2024-04-26T01:52:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.529+0000 7fdafd82f740 1 -- 172.21.15.1:0/609510919 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564ce129df30 con 0x564ce12178b0 2024-04-26T01:52:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.529+0000 7fdadd600640 1 -- 172.21.15.1:0/609510919 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaec0a10f0 con 0x564ce12178b0 2024-04-26T01:52:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.530+0000 7fdadd600640 1 -- 172.21.15.1:0/609510919 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdae4031c00 con 0x564ce12178b0 2024-04-26T01:52:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.530+0000 7fdafd82f740 10 client.25241 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:52:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.530+0000 7fdafd82f740 10 client.25241 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:52:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.530+0000 7fdafd82f740 20 client.25241 populate_metadata read hostname 'smithi001' 2024-04-26T01:52:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.530+0000 7fdafd82f740 1 --2- 172.21.15.1:0/609510919 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ce129e700 0x564ce12a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.530+0000 7fdafd82f740 4 client.25241 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:52:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.530+0000 7fdafd82f740 1 -- 172.21.15.1:0/609510919 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564ce118b320 con 0x564ce129e700 2024-04-26T01:52:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.530+0000 7fdaf3400640 1 --2- 172.21.15.1:0/609510919 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ce129e700 0x564ce12a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.531+0000 7fdaf3400640 1 --2- 172.21.15.1:0/609510919 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ce129e700 0x564ce12a0b20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=138db13ea8a88d46 server_cookie=4223312ea0584d61 in_seq=0 out_seq=0 2024-04-26T01:52:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.531+0000 7fdadd600640 10 client.25241 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.539+0000 7fdadd600640 1 -- 172.21.15.1:0/609510919 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564ce118b320 con 0x564ce129e700 2024-04-26T01:52:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.539+0000 7fdadd600640 10 client.25241 handle_command_reply: tid=0 2024-04-26T01:52:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.539+0000 7fdafd82f740 1 client.25241 shutdown 2024-04-26T01:52:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.539+0000 7fdafd82f740 1 -- 172.21.15.1:0/609510919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce12178b0 msgr2=0x564ce122ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.539+0000 7fdafd82f740 1 --2- 172.21.15.1:0/609510919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce12178b0 0x564ce122ccb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdae40092e0 tx=0x7fdae4004620 comp rx=0 tx=0).stop 2024-04-26T01:52:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.539+0000 7fdafd82f740 1 -- 172.21.15.1:0/609510919 shutdown_connections 2024-04-26T01:52:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.539+0000 7fdafd82f740 1 --2- 172.21.15.1:0/609510919 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ce129e700 0x564ce12a0b20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.539+0000 7fdafd82f740 1 --2- 172.21.15.1:0/609510919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ce1226fa0 0x564ce129c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.539+0000 7fdafd82f740 1 --2- 172.21.15.1:0/609510919 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ce12181d0 0x564ce122d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.539+0000 7fdafd82f740 1 --2- 172.21.15.1:0/609510919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ce12178b0 0x564ce122ccb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.539+0000 7fdadd600640 0 client.25241 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.539+0000 7fdafd82f740 1 -- 172.21.15.1:0/609510919 >> 172.21.15.1:0/609510919 conn(0x564ce1213ec0 msgr2=0x564ce12247e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.539+0000 7fdafd82f740 1 -- 172.21.15.1:0/609510919 shutdown_connections 2024-04-26T01:52:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.539+0000 7fdafd82f740 1 -- 172.21.15.1:0/609510919 wait complete. 2024-04-26T01:52:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.540+0000 7fdafd82f740 20 client.25241 trim_cache size 0 max 16384 2024-04-26T01:52:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:52:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:52:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:52:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:52:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.540+0000 7fdaf9400640 1 -- 172.21.15.1:0/467120968 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fdad80a3eb0 msgr2=0x7fdad80a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.540+0000 7fdaf9400640 1 --2- 172.21.15.1:0/467120968 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fdad80a3eb0 0x7fdad80a6370 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x564ce121a5a0 tx=0x7fdae4005f90 comp rx=0 tx=0).stop 2024-04-26T01:52:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.540+0000 7fdaf9400640 1 -- 172.21.15.1:0/467120968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaf419ce60 msgr2=0x7fdaf419d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.540+0000 7fdaf9400640 1 --2- 172.21.15.1:0/467120968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaf419ce60 0x7fdaf419d310 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00eb10 tx=0x7fdaec00c2d0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.540+0000 7fdaf9400640 1 -- 172.21.15.1:0/467120968 shutdown_connections 2024-04-26T01:52:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.540+0000 7fdaf9400640 1 --2- 172.21.15.1:0/467120968 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fdad80a3eb0 0x7fdad80a6370 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.540+0000 7fdaf9400640 1 --2- 172.21.15.1:0/467120968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaf419ce60 0x7fdaf419d310 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.540+0000 7fdaf9400640 1 --2- 172.21.15.1:0/467120968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf4107080 0x7fdaf4198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.540+0000 7fdaf9400640 1 --2- 172.21.15.1:0/467120968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaf4101fb0 0x7fdaf41985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.540+0000 7fdaf9400640 1 -- 172.21.15.1:0/467120968 >> 172.21.15.1:0/467120968 conn(0x7fdaf40fd890 msgr2=0x7fdaf40fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.541+0000 7fdaf9400640 1 -- 172.21.15.1:0/467120968 shutdown_connections 2024-04-26T01:52:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:08.541+0000 7fdaf9400640 1 -- 172.21.15.1:0/467120968 wait complete. 2024-04-26T01:52:08.851 INFO:teuthology.orchestra.run.smithi001.stdout:{"run_dir":"/var/run/ceph"} 2024-04-26T01:52:08.852 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-04-26T01:52:08.852 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 3448 2024-04-26T01:52:08.852 DEBUG:tasks.quiescer:Yielding 2024-04-26T01:52:08.852 INFO:teuthology.run_tasks:Running task workunit... 2024-04-26T01:52:08.867 INFO:tasks.workunit:Pulling workunits from ref afa1933b0fdbb6c99c947d1eda34d661d23cd327 2024-04-26T01:52:08.868 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-26T01:52:08.868 INFO:tasks.workunit:timeout=3h 2024-04-26T01:52:08.868 INFO:tasks.workunit:cleanup=True 2024-04-26T01:52:08.868 DEBUG:teuthology.orchestra.run.smithi001:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-26T01:52:08.869 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:52:08.926 INFO:teuthology.orchestra.run.smithi001.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-26T01:52:08.926 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-26T01:52:08.926 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-04-26T01:52:08.926 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:52:08.926 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-26T01:52:08.926 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-04-26 01:48:09.216223352 +0000 2024-04-26T01:52:08.926 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-04-26 01:48:09.216223352 +0000 2024-04-26T01:52:08.926 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-04-26 01:51:37.131553569 +0000 2024-04-26T01:52:08.926 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-04-26 01:48:09.216223352 +0000 2024-04-26T01:52:08.926 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-26T01:52:08.927 DEBUG:teuthology.orchestra.run.smithi001:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-26T01:52:09.057 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout afa1933b0fdbb6c99c947d1eda34d661d23cd327 2024-04-26T01:52:09.113 INFO:tasks.workunit.client.0.smithi001.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-26T01:52:09.323 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:10.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:10 smithi001 ceph-mon[27822]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-26T01:52:10.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:10 smithi097 ceph-mon[32853]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-26T01:52:10.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:10 smithi173 ceph-mon[32553]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-26T01:52:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.600+0000 7f1ac4e00700 1 -- 172.21.15.1:0/97140362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ac0075590 msgr2=0x7f1ac010ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.600+0000 7f1ac4e00700 1 --2- 172.21.15.1:0/97140362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ac0075590 0x7f1ac010ef00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0098f0 tx=0x7f1aac009c00 comp rx=0 tx=0).stop 2024-04-26T01:52:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.600+0000 7f1ac4e00700 1 -- 172.21.15.1:0/97140362 shutdown_connections 2024-04-26T01:52:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.600+0000 7f1ac4e00700 1 --2- 172.21.15.1:0/97140362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ac010f440 0x7f1ac0111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.600+0000 7f1ac4e00700 1 --2- 172.21.15.1:0/97140362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac0074c70 0x7f1ac0075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.600+0000 7f1ac4e00700 1 --2- 172.21.15.1:0/97140362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ac0075590 0x7f1ac010ef00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.601+0000 7f1ac4e00700 1 -- 172.21.15.1:0/97140362 >> 172.21.15.1:0/97140362 conn(0x7f1ac0079ba0 msgr2=0x7f1ac0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.601+0000 7f1ac4e00700 1 -- 172.21.15.1:0/97140362 shutdown_connections 2024-04-26T01:52:11.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.601+0000 7f1ac4e00700 1 -- 172.21.15.1:0/97140362 wait complete. 2024-04-26T01:52:11.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.602+0000 7f1ac4e00700 1 Processor -- start 2024-04-26T01:52:11.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.602+0000 7f1ac4e00700 1 -- start start 2024-04-26T01:52:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.603+0000 7f1ac4e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ac0074c70 0x7f1ac019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.603+0000 7f1ac4e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ac0075590 0x7f1ac019b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.603+0000 7f1ac4e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac010f440 0x7f1ac019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.603+0000 7f1ac4e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac0114e50 con 0x7f1ac010f440 2024-04-26T01:52:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.603+0000 7f1ac4e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac0114cd0 con 0x7f1ac0074c70 2024-04-26T01:52:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.603+0000 7f1ac4e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac0114fd0 con 0x7f1ac0075590 2024-04-26T01:52:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.603+0000 7f1abde00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ac0074c70 0x7f1ac019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.603+0000 7f1abde00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ac0074c70 0x7f1ac019b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:55118/0 (socket says 172.21.15.1:55118) 2024-04-26T01:52:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.603+0000 7f1abde00700 1 -- 172.21.15.1:0/1608734885 learned_addr learned my addr 172.21.15.1:0/1608734885 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.603+0000 7f1abd400700 1 --2- 172.21.15.1:0/1608734885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ac0075590 0x7f1ac019b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.603+0000 7f1abe800700 1 --2- 172.21.15.1:0/1608734885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac010f440 0x7f1ac019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:11.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.604+0000 7f1abd400700 1 -- 172.21.15.1:0/1608734885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ac0074c70 msgr2=0x7f1ac019b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:11.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.604+0000 7f1abd400700 1 --2- 172.21.15.1:0/1608734885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ac0074c70 0x7f1ac019b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:11.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.604+0000 7f1abd400700 1 -- 172.21.15.1:0/1608734885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac010f440 msgr2=0x7f1ac019fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:11.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.604+0000 7f1abd400700 1 --2- 172.21.15.1:0/1608734885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac010f440 0x7f1ac019fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:11.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.604+0000 7f1abd400700 1 -- 172.21.15.1:0/1608734885 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac01a0330 con 0x7f1ac0075590 2024-04-26T01:52:11.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.604+0000 7f1abde00700 1 --2- 172.21.15.1:0/1608734885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ac0074c70 0x7f1ac019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:11.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.604+0000 7f1abd400700 1 --2- 172.21.15.1:0/1608734885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ac0075590 0x7f1ac019b860 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1aac000c00 tx=0x7f1aac00fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:11.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.604+0000 7f1ab6a00700 1 -- 172.21.15.1:0/1608734885 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac005140 con 0x7f1ac0075590 2024-04-26T01:52:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.604+0000 7f1ac4e00700 1 -- 172.21.15.1:0/1608734885 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac01a05c0 con 0x7f1ac0075590 2024-04-26T01:52:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.604+0000 7f1ab6a00700 1 -- 172.21.15.1:0/1608734885 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aac00b560 con 0x7f1ac0075590 2024-04-26T01:52:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.604+0000 7f1ab6a00700 1 -- 172.21.15.1:0/1608734885 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac003a00 con 0x7f1ac0075590 2024-04-26T01:52:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.604+0000 7f1ac4e00700 1 -- 172.21.15.1:0/1608734885 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac01a7ea0 con 0x7f1ac0075590 2024-04-26T01:52:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.605+0000 7f1ac4e00700 1 -- 172.21.15.1:0/1608734885 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac0076360 con 0x7f1ac0075590 2024-04-26T01:52:11.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.607+0000 7f1ab6a00700 1 -- 172.21.15.1:0/1608734885 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7f1aac00b9e0 con 0x7f1ac0075590 2024-04-26T01:52:11.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.609+0000 7f1ab6a00700 1 --2- 172.21.15.1:0/1608734885 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f1aa40a3fe0 0x7f1aa40a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:11.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.609+0000 7f1ab6a00700 1 -- 172.21.15.1:0/1608734885 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f1aac0d19a0 con 0x7f1ac0075590 2024-04-26T01:52:11.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.610+0000 7f1abde00700 1 --2- 172.21.15.1:0/1608734885 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f1aa40a3fe0 0x7f1aa40a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:11.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.610+0000 7f1abde00700 1 --2- 172.21.15.1:0/1608734885 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f1aa40a3fe0 0x7f1aa40a64a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009750 tx=0x7f1ab0009310 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.613+0000 7f1ab6a00700 1 -- 172.21.15.1:0/1608734885 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1aac013030 con 0x7f1ac0075590 2024-04-26T01:52:11.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.900+0000 7f1ac4e00700 1 -- 172.21.15.1:0/1608734885 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ac019c680 con 0x7f1ac0075590 2024-04-26T01:52:11.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.901+0000 7f1ab6a00700 1 -- 172.21.15.1:0/1608734885 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11841 (secure 0 0 0) 0x7f1aac09de90 con 0x7f1ac0075590 2024-04-26T01:52:11.899 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:52:11.899 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:51:38.222728+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2952462325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2952462325},{"type":"v1","addr":"172.21.15.173:6835","nonce":2952462325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24499,24514,24518]},"id":1}]} 2024-04-26T01:52:11.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.904+0000 7f1ac4e00700 1 -- 172.21.15.1:0/1608734885 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f1aa40a3fe0 msgr2=0x7f1aa40a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:11.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.904+0000 7f1ac4e00700 1 --2- 172.21.15.1:0/1608734885 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f1aa40a3fe0 0x7f1aa40a64a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009750 tx=0x7f1ab0009310 comp rx=0 tx=0).stop 2024-04-26T01:52:11.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.904+0000 7f1ac4e00700 1 -- 172.21.15.1:0/1608734885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ac0075590 msgr2=0x7f1ac019b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:11.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.904+0000 7f1ac4e00700 1 --2- 172.21.15.1:0/1608734885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ac0075590 0x7f1ac019b860 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1aac000c00 tx=0x7f1aac00fa70 comp rx=0 tx=0).stop 2024-04-26T01:52:11.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.904+0000 7f1ac4e00700 1 -- 172.21.15.1:0/1608734885 shutdown_connections 2024-04-26T01:52:11.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.904+0000 7f1ac4e00700 1 --2- 172.21.15.1:0/1608734885 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f1aa40a3fe0 0x7f1aa40a64a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:11.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.904+0000 7f1ac4e00700 1 --2- 172.21.15.1:0/1608734885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac010f440 0x7f1ac019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:11.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.904+0000 7f1ac4e00700 1 --2- 172.21.15.1:0/1608734885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ac0074c70 0x7f1ac019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:11.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.904+0000 7f1ac4e00700 1 --2- 172.21.15.1:0/1608734885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ac0075590 0x7f1ac019b860 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:11.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.904+0000 7f1ac4e00700 1 -- 172.21.15.1:0/1608734885 >> 172.21.15.1:0/1608734885 conn(0x7f1ac0079ba0 msgr2=0x7f1ac00ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:11.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.905+0000 7f1ac4e00700 1 -- 172.21.15.1:0/1608734885 shutdown_connections 2024-04-26T01:52:11.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:11.905+0000 7f1ac4e00700 1 -- 172.21.15.1:0/1608734885 wait complete. 2024-04-26T01:52:11.904 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 26 2024-04-26T01:52:12.343 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:12 smithi001 ceph-mon[27822]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 29 op/s 2024-04-26T01:52:12.343 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:12 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1608734885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:52:12.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:12 smithi097 ceph-mon[32853]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 29 op/s 2024-04-26T01:52:12.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:12 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1608734885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:52:12.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:12 smithi173 ceph-mon[32553]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 29 op/s 2024-04-26T01:52:12.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:12 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1608734885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:52:12.876 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T01:47:29.999521+0000', 'modified': '2024-04-26T01:51:38.222728+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24518, 'mds_1': 24514, 'mds_2': 24499}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24464': {'gid': 24464, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6835/763675408', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 763675408}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 763675408}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6837/3607142398', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 3607142398}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 3607142398}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.173:6835/2952462325', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 2952462325}, {'type': 'v1', 'addr': '172.21.15.173:6835', 'nonce': 2952462325}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.173:6837/2502298793', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6836', 'nonce': 2502298793}, {'type': 'v1', 'addr': '172.21.15.173:6837', 'nonce': 2502298793}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24514': {'gid': 24514, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.173:6839/373692430', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6838', 'nonce': 373692430}, {'type': 'v1', 'addr': '172.21.15.173:6839', 'nonce': 373692430}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.97:6839/1315554162', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 1315554162}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 1315554162}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24518, 'qdb_cluster': [24499, 24514, 24518]} 2024-04-26T01:52:12.876 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-26T01:52:12.877 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph tell mds.c status 2024-04-26T01:52:13.344 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:14.367 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:14 smithi001 ceph-mon[27822]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-26T01:52:14.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:14 smithi097 ceph-mon[32853]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-26T01:52:14.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:14 smithi173 ceph-mon[32553]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-26T01:52:15.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.699+0000 7f67ee000700 1 -- 172.21.15.1:0/2738381113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67e8108970 msgr2=0x7f67e8108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:15.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.699+0000 7f67ee000700 1 --2- 172.21.15.1:0/2738381113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67e8108970 0x7f67e8108d50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00b0a0 tx=0x7f67dc00b3b0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.699+0000 7f67ee000700 1 -- 172.21.15.1:0/2738381113 shutdown_connections 2024-04-26T01:52:15.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.699+0000 7f67ee000700 1 --2- 172.21.15.1:0/2738381113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67e8103330 0x7f67e810f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.699+0000 7f67ee000700 1 --2- 172.21.15.1:0/2738381113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67e8108970 0x7f67e8108d50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.699+0000 7f67ee000700 1 --2- 172.21.15.1:0/2738381113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67e8102970 0x7f67e8102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.699+0000 7f67ee000700 1 -- 172.21.15.1:0/2738381113 >> 172.21.15.1:0/2738381113 conn(0x7f67e80fe4c0 msgr2=0x7f67e81008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:15.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.699+0000 7f67ee000700 1 -- 172.21.15.1:0/2738381113 shutdown_connections 2024-04-26T01:52:15.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.699+0000 7f67ee000700 1 -- 172.21.15.1:0/2738381113 wait complete. 2024-04-26T01:52:15.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.701+0000 7f67ee000700 1 Processor -- start 2024-04-26T01:52:15.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.701+0000 7f67ee000700 1 -- start start 2024-04-26T01:52:15.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.701+0000 7f67ee000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67e8102970 0x7f67e8074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:15.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.701+0000 7f67ee000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67e8103330 0x7f67e8075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:15.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.701+0000 7f67ee000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67e8108970 0x7f67e80757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:15.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.701+0000 7f67ee000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f67e8112bd0 con 0x7f67e8103330 2024-04-26T01:52:15.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.701+0000 7f67ee000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f67e8112a50 con 0x7f67e8102970 2024-04-26T01:52:15.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.701+0000 7f67ee000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f67e8112d50 con 0x7f67e8108970 2024-04-26T01:52:15.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.702+0000 7f67e7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67e8103330 0x7f67e8075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:15.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.702+0000 7f67ecc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67e8102970 0x7f67e8074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:15.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.702+0000 7f67ecc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67e8102970 0x7f67e8074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:54488/0 (socket says 172.21.15.1:54488) 2024-04-26T01:52:15.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.702+0000 7f67ecc00700 1 -- 172.21.15.1:0/3586186129 learned_addr learned my addr 172.21.15.1:0/3586186129 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:15.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.702+0000 7f67ed600700 1 --2- 172.21.15.1:0/3586186129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67e8108970 0x7f67e80757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:15.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.702+0000 7f67ecc00700 1 -- 172.21.15.1:0/3586186129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67e8108970 msgr2=0x7f67e80757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:15.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.702+0000 7f67ecc00700 1 --2- 172.21.15.1:0/3586186129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67e8108970 0x7f67e80757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.702+0000 7f67ecc00700 1 -- 172.21.15.1:0/3586186129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67e8103330 msgr2=0x7f67e8075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:15.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.702+0000 7f67ecc00700 1 --2- 172.21.15.1:0/3586186129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67e8103330 0x7f67e8075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.702+0000 7f67ecc00700 1 -- 172.21.15.1:0/3586186129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e81a75a0 con 0x7f67e8102970 2024-04-26T01:52:15.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.702+0000 7f67e7e00700 1 --2- 172.21.15.1:0/3586186129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67e8103330 0x7f67e8075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:15.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.703+0000 7f67ecc00700 1 --2- 172.21.15.1:0/3586186129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67e8102970 0x7f67e8074d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00d010 tx=0x7f67dc004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:15.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.703+0000 7f67e5600700 1 -- 172.21.15.1:0/3586186129 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc021070 con 0x7f67e8102970 2024-04-26T01:52:15.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.703+0000 7f67e5600700 1 -- 172.21.15.1:0/3586186129 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67dc007e30 con 0x7f67e8102970 2024-04-26T01:52:15.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.703+0000 7f67e5600700 1 -- 172.21.15.1:0/3586186129 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc004640 con 0x7f67e8102970 2024-04-26T01:52:15.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.703+0000 7f67ee000700 1 -- 172.21.15.1:0/3586186129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e81a7830 con 0x7f67e8102970 2024-04-26T01:52:15.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.703+0000 7f67ee000700 1 -- 172.21.15.1:0/3586186129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e81a7d30 con 0x7f67e8102970 2024-04-26T01:52:15.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.707+0000 7f67e5600700 1 -- 172.21.15.1:0/3586186129 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7f67dc007980 con 0x7f67e8102970 2024-04-26T01:52:15.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.709+0000 7f67e5600700 1 --2- 172.21.15.1:0/3586186129 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f67d00a3fe0 0x7f67d00a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:15.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.709+0000 7f67e5600700 1 -- 172.21.15.1:0/3586186129 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f67dc00f020 con 0x7f67e8102970 2024-04-26T01:52:15.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.709+0000 7f67f9f92b80 1 Processor -- start 2024-04-26T01:52:15.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.709+0000 7f67f9f92b80 1 -- start start 2024-04-26T01:52:15.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.709+0000 7f67e7e00700 1 --2- 172.21.15.1:0/3586186129 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f67d00a3fe0 0x7f67d00a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:15.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.710+0000 7f67f9f92b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007be3e0 0x5604007be7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:15.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.710+0000 7f67f9f92b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007bed00 0x5604007c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:15.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.710+0000 7f67f9f92b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 0x5604007cc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:15.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.710+0000 7f67f9f92b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560400711e40 con 0x5604007be3e0 2024-04-26T01:52:15.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.710+0000 7f67f9f92b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5604006b86e0 con 0x5604007bed00 2024-04-26T01:52:15.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.710+0000 7f67f9f92b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56040070de00 con 0x5604007c9c40 2024-04-26T01:52:15.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.710+0000 7f67ecc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007be3e0 0x5604007be7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:15.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.710+0000 7f67ecc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007be3e0 0x5604007be7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37926/0 (socket says 172.21.15.1:37926) 2024-04-26T01:52:15.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.710+0000 7f67ecc00700 1 -- 172.21.15.1:0/3976290688 learned_addr learned my addr 172.21.15.1:0/3976290688 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:15.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.710+0000 7f67ed600700 1 --2- 172.21.15.1:0/3976290688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 0x5604007cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:15.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.710+0000 7f67e7e00700 1 --2- 172.21.15.1:0/3976290688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007bed00 0x5604007c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:15.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.711+0000 7f67ecc00700 1 -- 172.21.15.1:0/3976290688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 msgr2=0x5604007cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:15.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.711+0000 7f67ecc00700 1 --2- 172.21.15.1:0/3976290688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 0x5604007cc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.711+0000 7f67ecc00700 1 -- 172.21.15.1:0/3976290688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007bed00 msgr2=0x5604007c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:15.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.711+0000 7f67ecc00700 1 --2- 172.21.15.1:0/3976290688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007bed00 0x5604007c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.711+0000 7f67ecc00700 1 -- 172.21.15.1:0/3976290688 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604006ae630 con 0x5604007be3e0 2024-04-26T01:52:15.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.711+0000 7f67e7e00700 1 --2- 172.21.15.1:0/3586186129 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f67d00a3fe0 0x7f67d00a64a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f67e8100b80 tx=0x7f67d4009260 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:15.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.711+0000 7f67ed600700 1 --2- 172.21.15.1:0/3976290688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 0x5604007cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:52:15.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.711+0000 7f67ecc00700 1 --2- 172.21.15.1:0/3976290688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007be3e0 0x5604007be7c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0062a0 tx=0x7f67dc00b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9054affe8e8c7aa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:15.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.712+0000 7f67da000700 1 -- 172.21.15.1:0/3976290688 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc09dcc0 con 0x5604007be3e0 2024-04-26T01:52:15.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.712+0000 7f67da000700 1 -- 172.21.15.1:0/3976290688 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67dc0a2070 con 0x5604007be3e0 2024-04-26T01:52:15.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.712+0000 7f67da000700 1 -- 172.21.15.1:0/3976290688 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc0074e0 con 0x5604007be3e0 2024-04-26T01:52:15.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.712+0000 7f67f9f92b80 1 -- 172.21.15.1:0/3976290688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007be3e0 msgr2=0x5604007be7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:15.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.712+0000 7f67f9f92b80 1 --2- 172.21.15.1:0/3976290688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007be3e0 0x5604007be7c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0062a0 tx=0x7f67dc00b740 comp rx=0 tx=0).stop 2024-04-26T01:52:15.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.713+0000 7f67f9f92b80 1 -- 172.21.15.1:0/3976290688 shutdown_connections 2024-04-26T01:52:15.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.713+0000 7f67f9f92b80 1 --2- 172.21.15.1:0/3976290688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 0x5604007cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.713+0000 7f67f9f92b80 1 --2- 172.21.15.1:0/3976290688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007be3e0 0x5604007be7c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.713+0000 7f67f9f92b80 1 --2- 172.21.15.1:0/3976290688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007bed00 0x5604007c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.713+0000 7f67f9f92b80 1 -- 172.21.15.1:0/3976290688 >> 172.21.15.1:0/3976290688 conn(0x5604006ed550 msgr2=0x5604007baa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:15.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.713+0000 7f67f9f92b80 1 -- 172.21.15.1:0/3976290688 shutdown_connections 2024-04-26T01:52:15.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.713+0000 7f67f9f92b80 1 -- 172.21.15.1:0/3976290688 wait complete. 2024-04-26T01:52:15.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.714+0000 7f67f9f92b80 1 -- start start 2024-04-26T01:52:15.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.714+0000 7f67f9f92b80 1 Processor -- start 2024-04-26T01:52:15.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.715+0000 7f67f9f92b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007be3e0 0x5604007c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:15.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.715+0000 7f67f9f92b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007bed00 0x5604007c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:15.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.715+0000 7f67f9f92b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 0x5604007c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:15.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.715+0000 7f67f9f92b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5604007cc570 con 0x5604007bed00 2024-04-26T01:52:15.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.715+0000 7f67f9f92b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5604006cb810 con 0x5604007be3e0 2024-04-26T01:52:15.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.715+0000 7f67f9f92b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5604007cc6f0 con 0x5604007c9c40 2024-04-26T01:52:15.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.715+0000 7f67f9f92b80 10 client.0 _ll_register_callbacks cb 0x5604006aa6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:15.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.715+0000 7f67ecc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007be3e0 0x5604007c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:15.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.715+0000 7f67ecc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007be3e0 0x5604007c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:54496/0 (socket says 172.21.15.1:54496) 2024-04-26T01:52:15.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.715+0000 7f67ecc00700 1 -- 172.21.15.1:0/2996151966 learned_addr learned my addr 172.21.15.1:0/2996151966 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:15.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.715+0000 7f67e7e00700 1 --2- 172.21.15.1:0/2996151966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007bed00 0x5604007c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:15.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.715+0000 7f67ed600700 1 --2- 172.21.15.1:0/2996151966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 0x5604007c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:15.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.716+0000 7f67ecc00700 1 -- 172.21.15.1:0/2996151966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 msgr2=0x5604007c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:15.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.716+0000 7f67ecc00700 1 --2- 172.21.15.1:0/2996151966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 0x5604007c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.716+0000 7f67ecc00700 1 -- 172.21.15.1:0/2996151966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007bed00 msgr2=0x5604007c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:15.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.716+0000 7f67ecc00700 1 --2- 172.21.15.1:0/2996151966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007bed00 0x5604007c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.716+0000 7f67ecc00700 1 -- 172.21.15.1:0/2996151966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56040065f880 con 0x5604007be3e0 2024-04-26T01:52:15.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.716+0000 7f67ecc00700 1 --2- 172.21.15.1:0/2996151966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007be3e0 0x5604007c93d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0cf4d0 tx=0x7f67dc00b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97b176b5a90dd9c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:15.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.716+0000 7f67cea00700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:52:15.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.716+0000 7f67cea00700 1 -- 172.21.15.1:0/2996151966 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc0bc020 con 0x5604007be3e0 2024-04-26T01:52:15.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.716+0000 7f67cea00700 1 -- 172.21.15.1:0/2996151966 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f67dc09bfc0 con 0x5604007be3e0 2024-04-26T01:52:15.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.716+0000 7f67f9f92b80 1 -- 172.21.15.1:0/2996151966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67dc0a2070 con 0x5604007be3e0 2024-04-26T01:52:15.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.717+0000 7f67cea00700 1 -- 172.21.15.1:0/2996151966 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f67dc0a68a0 con 0x5604007be3e0 2024-04-26T01:52:15.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.717+0000 7f67f9f92b80 10 client.25259 fetch_fsmap learned FSMap version 26 2024-04-26T01:52:15.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.717+0000 7f67f9f92b80 1 -- 172.21.15.1:0/2996151966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5604006a8aa0 con 0x5604007be3e0 2024-04-26T01:52:15.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.717+0000 7f67cea00700 1 -- 172.21.15.1:0/2996151966 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67dc0a2070 con 0x5604007be3e0 2024-04-26T01:52:15.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.717+0000 7f67cea00700 1 -- 172.21.15.1:0/2996151966 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f67dc011070 con 0x5604007be3e0 2024-04-26T01:52:15.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.718+0000 7f67f9f92b80 10 client.25259 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:52:15.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.718+0000 7f67f9f92b80 10 client.25259 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T01:52:15.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.718+0000 7f67f9f92b80 20 client.25259 populate_metadata read hostname 'smithi001' 2024-04-26T01:52:15.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.718+0000 7f67f9f92b80 1 --2- 172.21.15.1:0/2996151966 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x5604007d90b0 0x5604007da520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:15.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.718+0000 7f67f9f92b80 4 client.25259 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:52:15.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.718+0000 7f67f9f92b80 1 -- 172.21.15.1:0/2996151966 --> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56040071f630 con 0x5604007d90b0 2024-04-26T01:52:15.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.718+0000 7f67e7e00700 1 --2- 172.21.15.1:0/2996151966 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x5604007d90b0 0x5604007da520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:15.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.719+0000 7f67e7e00700 1 --2- 172.21.15.1:0/2996151966 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x5604007d90b0 0x5604007da520 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d67c0368759ae234 server_cookie=591126ae85c1dbe6 in_seq=0 out_seq=0 2024-04-26T01:52:15.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.719+0000 7f67cea00700 10 client.25259 ms_handle_connect on v2:172.21.15.173:6836/2502298793 2024-04-26T01:52:15.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.720+0000 7f67cea00700 1 -- 172.21.15.1:0/2996151966 <== mds.2 v2:172.21.15.173:6836/2502298793 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56040071f630 con 0x5604007d90b0 2024-04-26T01:52:15.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.720+0000 7f67cea00700 10 client.25259 handle_command_reply: tid=0 2024-04-26T01:52:15.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.720+0000 7f67f9f92b80 1 client.25259 shutdown 2024-04-26T01:52:15.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.720+0000 7f67f9f92b80 1 -- 172.21.15.1:0/2996151966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007be3e0 msgr2=0x5604007c93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:15.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.720+0000 7f67f9f92b80 1 --2- 172.21.15.1:0/2996151966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007be3e0 0x5604007c93d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0cf4d0 tx=0x7f67dc00b3b0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.720+0000 7f67f9f92b80 1 -- 172.21.15.1:0/2996151966 shutdown_connections 2024-04-26T01:52:15.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.720+0000 7f67f9f92b80 1 --2- 172.21.15.1:0/2996151966 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x5604007d90b0 0x5604007da520 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.720+0000 7f67f9f92b80 1 --2- 172.21.15.1:0/2996151966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 0x5604007c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.720+0000 7f67f9f92b80 1 --2- 172.21.15.1:0/2996151966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007be3e0 0x5604007c93d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.720+0000 7f67f9f92b80 1 --2- 172.21.15.1:0/2996151966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007bed00 0x5604007c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.720+0000 7f67cea00700 0 client.25259 ms_handle_reset on v2:172.21.15.173:6836/2502298793 2024-04-26T01:52:15.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.720+0000 7f67f9f92b80 1 -- 172.21.15.1:0/2996151966 >> 172.21.15.1:0/2996151966 conn(0x5604006ed550 msgr2=0x5604007ba680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:15.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.721+0000 7f67f9f92b80 1 -- 172.21.15.1:0/2996151966 shutdown_connections 2024-04-26T01:52:15.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.721+0000 7f67f9f92b80 1 -- 172.21.15.1:0/2996151966 wait complete. 2024-04-26T01:52:15.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.721+0000 7f67f9f92b80 20 client.25259 trim_cache size 0 max 16384 2024-04-26T01:52:15.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.759+0000 7f67f9f92b80 1 Processor -- start 2024-04-26T01:52:15.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.759+0000 7f67f9f92b80 1 -- start start 2024-04-26T01:52:15.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.759+0000 7f67f9f92b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007be3e0 0x5604007bab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:15.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.760+0000 7f67f9f92b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007bed00 0x5604007c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:15.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.760+0000 7f67f9f92b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 0x5604007c3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:15.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.760+0000 7f67f9f92b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5604007d8860 con 0x5604007bed00 2024-04-26T01:52:15.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.760+0000 7f67f9f92b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5604007d8f30 con 0x5604007be3e0 2024-04-26T01:52:15.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.760+0000 7f67f9f92b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5604007d9600 con 0x5604007c9c40 2024-04-26T01:52:15.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.760+0000 7f67ed600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 0x5604007c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:15.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.760+0000 7f67ecc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007be3e0 0x5604007bab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:15.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.760+0000 7f67ed600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 0x5604007c3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:49670/0 (socket says 172.21.15.1:49670) 2024-04-26T01:52:15.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.760+0000 7f67ed600700 1 -- 172.21.15.1:0/3146465529 learned_addr learned my addr 172.21.15.1:0/3146465529 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:15.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.760+0000 7f67e7e00700 1 --2- 172.21.15.1:0/3146465529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007bed00 0x5604007c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:15.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.761+0000 7f67ed600700 1 -- 172.21.15.1:0/3146465529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007be3e0 msgr2=0x5604007bab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:15.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.761+0000 7f67ed600700 1 --2- 172.21.15.1:0/3146465529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007be3e0 0x5604007bab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.761+0000 7f67ed600700 1 -- 172.21.15.1:0/3146465529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007bed00 msgr2=0x5604007c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:15.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.761+0000 7f67ed600700 1 --2- 172.21.15.1:0/3146465529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007bed00 0x5604007c6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.761+0000 7f67ed600700 1 -- 172.21.15.1:0/3146465529 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604007c7100 con 0x5604007c9c40 2024-04-26T01:52:15.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.761+0000 7f67ecc00700 1 --2- 172.21.15.1:0/3146465529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007be3e0 0x5604007bab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:15.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.761+0000 7f67ed600700 1 --2- 172.21.15.1:0/3146465529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 0x5604007c3950 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009f20 tx=0x7f67e0012730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3773d9f05003cd33 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.761+0000 7f67d9600700 1 -- 172.21.15.1:0/3146465529 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e001f020 con 0x5604007c9c40 2024-04-26T01:52:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.761+0000 7f67d9600700 1 -- 172.21.15.1:0/3146465529 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67e0004ce0 con 0x5604007c9c40 2024-04-26T01:52:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.761+0000 7f67d9600700 1 -- 172.21.15.1:0/3146465529 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0010690 con 0x5604007c9c40 2024-04-26T01:52:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.762+0000 7f67f9f92b80 1 -- 172.21.15.1:0/3146465529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 msgr2=0x5604007c3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.762+0000 7f67f9f92b80 1 --2- 172.21.15.1:0/3146465529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 0x5604007c3950 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009f20 tx=0x7f67e0012730 comp rx=0 tx=0).stop 2024-04-26T01:52:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.763+0000 7f67f9f92b80 1 -- 172.21.15.1:0/3146465529 shutdown_connections 2024-04-26T01:52:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.763+0000 7f67f9f92b80 1 --2- 172.21.15.1:0/3146465529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 0x5604007c3950 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.763+0000 7f67f9f92b80 1 --2- 172.21.15.1:0/3146465529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007be3e0 0x5604007bab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.763+0000 7f67f9f92b80 1 --2- 172.21.15.1:0/3146465529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007bed00 0x5604007c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.763+0000 7f67f9f92b80 1 -- 172.21.15.1:0/3146465529 >> 172.21.15.1:0/3146465529 conn(0x5604006ed550 msgr2=0x5604007c64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.763+0000 7f67f9f92b80 1 -- 172.21.15.1:0/3146465529 shutdown_connections 2024-04-26T01:52:15.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.763+0000 7f67f9f92b80 1 -- 172.21.15.1:0/3146465529 wait complete. 2024-04-26T01:52:15.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.763+0000 7f67f9f92b80 1 -- start start 2024-04-26T01:52:15.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.763+0000 7f67f9f92b80 1 Processor -- start 2024-04-26T01:52:15.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.764+0000 7f67f9f92b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007be3e0 0x5604008394e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:15.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.764+0000 7f67f9f92b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007bed00 0x560400839a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:15.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.764+0000 7f67f9f92b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 0x5604007dfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:15.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.764+0000 7f67f9f92b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5604007da3a0 con 0x5604007be3e0 2024-04-26T01:52:15.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.764+0000 7f67f9f92b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5604007d9cd0 con 0x5604007bed00 2024-04-26T01:52:15.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.764+0000 7f67f9f92b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5604007daa70 con 0x5604007c9c40 2024-04-26T01:52:15.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.764+0000 7f67f9f92b80 10 client.0 _ll_register_callbacks cb 0x5604006aa6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:15.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.764+0000 7f67ed600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 0x5604007dfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:15.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.764+0000 7f67ecc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007be3e0 0x5604008394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:15.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.764+0000 7f67ed600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 0x5604007dfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:49680/0 (socket says 172.21.15.1:49680) 2024-04-26T01:52:15.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.764+0000 7f67ed600700 1 -- 172.21.15.1:0/928604462 learned_addr learned my addr 172.21.15.1:0/928604462 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:15.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.765+0000 7f67e7e00700 1 --2- 172.21.15.1:0/928604462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007bed00 0x560400839a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:15.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.765+0000 7f67ecc00700 1 -- 172.21.15.1:0/928604462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 msgr2=0x5604007dfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:15.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.765+0000 7f67ecc00700 1 --2- 172.21.15.1:0/928604462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 0x5604007dfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.765+0000 7f67ecc00700 1 -- 172.21.15.1:0/928604462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007bed00 msgr2=0x560400839a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:15.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.765+0000 7f67ecc00700 1 --2- 172.21.15.1:0/928604462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007bed00 0x560400839a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.765+0000 7f67ecc00700 1 -- 172.21.15.1:0/928604462 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604007e0260 con 0x5604007be3e0 2024-04-26T01:52:15.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.766+0000 7f67ecc00700 1 --2- 172.21.15.1:0/928604462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007be3e0 0x5604008394e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0062a0 tx=0x7f67dc004150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd6bbad021d2ddc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:15.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.766+0000 7f67cea00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:52:15.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.766+0000 7f67cea00700 1 -- 172.21.15.1:0/928604462 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc0bc040 con 0x5604007be3e0 2024-04-26T01:52:15.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.766+0000 7f67cea00700 1 -- 172.21.15.1:0/928604462 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f67dc09d110 con 0x5604007be3e0 2024-04-26T01:52:15.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.766+0000 7f67f9f92b80 1 -- 172.21.15.1:0/928604462 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67e0004ce0 con 0x5604007be3e0 2024-04-26T01:52:15.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.766+0000 7f67cea00700 1 -- 172.21.15.1:0/928604462 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f67dc09bd60 con 0x5604007be3e0 2024-04-26T01:52:15.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.766+0000 7f67f9f92b80 10 client.15144 fetch_fsmap learned FSMap version 26 2024-04-26T01:52:15.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.766+0000 7f67f9f92b80 1 -- 172.21.15.1:0/928604462 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5604007e0260 con 0x5604007be3e0 2024-04-26T01:52:15.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.767+0000 7f67cea00700 1 -- 172.21.15.1:0/928604462 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67e0004ce0 con 0x5604007be3e0 2024-04-26T01:52:15.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.767+0000 7f67cea00700 1 -- 172.21.15.1:0/928604462 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f67dc011070 con 0x5604007be3e0 2024-04-26T01:52:15.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.767+0000 7f67f9f92b80 10 client.15144 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:52:15.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.767+0000 7f67f9f92b80 10 client.15144 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T01:52:15.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.767+0000 7f67f9f92b80 20 client.15144 populate_metadata read hostname 'smithi001' 2024-04-26T01:52:15.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.767+0000 7f67f9f92b80 1 --2- 172.21.15.1:0/928604462 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x5604008454e0 0x5604008441f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:15.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.767+0000 7f67f9f92b80 4 client.15144 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T01:52:15.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.767+0000 7f67f9f92b80 1 -- 172.21.15.1:0/928604462 --> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] -- command(tid 0: {"prefix": "status"}) -- 0x5604007196d0 con 0x5604008454e0 2024-04-26T01:52:15.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.768+0000 7f67e7e00700 1 --2- 172.21.15.1:0/928604462 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x5604008454e0 0x5604008441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:15.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.769+0000 7f67e7e00700 1 --2- 172.21.15.1:0/928604462 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x5604008454e0 0x5604008441f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=42b951151e107a1c server_cookie=e1008d65d28788c5 in_seq=0 out_seq=0 2024-04-26T01:52:15.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.769+0000 7f67cea00700 10 client.15144 ms_handle_connect on v2:172.21.15.173:6836/2502298793 2024-04-26T01:52:15.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.769+0000 7f67cea00700 1 -- 172.21.15.1:0/928604462 <== mds.2 v2:172.21.15.173:6836/2502298793 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5604007196d0 con 0x5604008454e0 2024-04-26T01:52:15.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.769+0000 7f67cea00700 10 client.15144 handle_command_reply: tid=0 2024-04-26T01:52:15.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.769+0000 7f67f9f92b80 1 client.15144 shutdown 2024-04-26T01:52:15.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.769+0000 7f67f9f92b80 1 -- 172.21.15.1:0/928604462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007be3e0 msgr2=0x5604008394e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:15.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.769+0000 7f67f9f92b80 1 --2- 172.21.15.1:0/928604462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007be3e0 0x5604008394e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0062a0 tx=0x7f67dc004150 comp rx=0 tx=0).stop 2024-04-26T01:52:15.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.770+0000 7f67f9f92b80 1 -- 172.21.15.1:0/928604462 shutdown_connections 2024-04-26T01:52:15.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.770+0000 7f67f9f92b80 1 --2- 172.21.15.1:0/928604462 >> [v2:172.21.15.173:6836/2502298793,v1:172.21.15.173:6837/2502298793] conn(0x5604008454e0 0x5604008441f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.770+0000 7f67f9f92b80 1 --2- 172.21.15.1:0/928604462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604007c9c40 0x5604007dfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.770+0000 7f67f9f92b80 1 --2- 172.21.15.1:0/928604462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604007be3e0 0x5604008394e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.770+0000 7f67f9f92b80 1 --2- 172.21.15.1:0/928604462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5604007bed00 0x560400839a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.770+0000 7f67cea00700 0 client.15144 ms_handle_reset on v2:172.21.15.173:6836/2502298793 2024-04-26T01:52:15.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.770+0000 7f67f9f92b80 1 -- 172.21.15.1:0/928604462 >> 172.21.15.1:0/928604462 conn(0x5604006ed550 msgr2=0x5604007c7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:15.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.770+0000 7f67f9f92b80 1 -- 172.21.15.1:0/928604462 shutdown_connections 2024-04-26T01:52:15.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.770+0000 7f67f9f92b80 1 -- 172.21.15.1:0/928604462 wait complete. 2024-04-26T01:52:15.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.770+0000 7f67f9f92b80 20 client.15144 trim_cache size 0 max 16384 2024-04-26T01:52:15.769 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T01:52:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "77cfcc80-036d-11ef-bc93-c7b262605968", 2024-04-26T01:52:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 2, 2024-04-26T01:52:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24499, 2024-04-26T01:52:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-04-26T01:52:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-04-26T01:52:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-04-26T01:52:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 268.555648115, 2024-04-26T01:52:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 26, 2024-04-26T01:52:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 93, 2024-04-26T01:52:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 92, 2024-04-26T01:52:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 324.462725842 2024-04-26T01:52:15.770 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T01:52:15.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.771+0000 7f67ee000700 1 -- 172.21.15.1:0/3586186129 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f67d00a3fe0 msgr2=0x7f67d00a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:15.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.771+0000 7f67ee000700 1 --2- 172.21.15.1:0/3586186129 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f67d00a3fe0 0x7f67d00a64a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f67e8100b80 tx=0x7f67d4009260 comp rx=0 tx=0).stop 2024-04-26T01:52:15.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.771+0000 7f67ee000700 1 -- 172.21.15.1:0/3586186129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67e8102970 msgr2=0x7f67e8074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:15.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.771+0000 7f67ee000700 1 --2- 172.21.15.1:0/3586186129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67e8102970 0x7f67e8074d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00d010 tx=0x7f67dc004d00 comp rx=0 tx=0).stop 2024-04-26T01:52:15.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.771+0000 7f67ee000700 1 -- 172.21.15.1:0/3586186129 shutdown_connections 2024-04-26T01:52:15.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.771+0000 7f67ee000700 1 --2- 172.21.15.1:0/3586186129 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f67d00a3fe0 0x7f67d00a64a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.771+0000 7f67ee000700 1 --2- 172.21.15.1:0/3586186129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67e8108970 0x7f67e80757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.771+0000 7f67ee000700 1 --2- 172.21.15.1:0/3586186129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67e8102970 0x7f67e8074d40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.771+0000 7f67ee000700 1 --2- 172.21.15.1:0/3586186129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67e8103330 0x7f67e8075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:15.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.771+0000 7f67ee000700 1 -- 172.21.15.1:0/3586186129 >> 172.21.15.1:0/3586186129 conn(0x7f67e80fe4c0 msgr2=0x7f67e80fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:15.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.772+0000 7f67ee000700 1 -- 172.21.15.1:0/3586186129 shutdown_connections 2024-04-26T01:52:15.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:15.772+0000 7f67ee000700 1 -- 172.21.15.1:0/3586186129 wait complete. 2024-04-26T01:52:16.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:16 smithi001 ceph-mon[27822]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-26T01:52:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:16 smithi097 ceph-mon[32853]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-26T01:52:16.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:16 smithi173 ceph-mon[32553]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-26T01:52:16.706 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-26T01:52:17.152 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:18.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:18 smithi001 ceph-mon[27822]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-26T01:52:18.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:18 smithi097 ceph-mon[32853]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-26T01:52:18.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:18 smithi173 ceph-mon[32553]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-26T01:52:19.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.544+0000 7efd22c00700 1 -- 172.21.15.1:0/1442296789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd1c075730 msgr2=0x7efd1c10ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:19.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.544+0000 7efd22c00700 1 --2- 172.21.15.1:0/1442296789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd1c075730 0x7efd1c10ecd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efd08009990 tx=0x7efd08009ca0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.544+0000 7efd22c00700 1 -- 172.21.15.1:0/1442296789 shutdown_connections 2024-04-26T01:52:19.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.544+0000 7efd22c00700 1 --2- 172.21.15.1:0/1442296789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1c10f210 0x7efd1c111620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.544+0000 7efd22c00700 1 --2- 172.21.15.1:0/1442296789 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd1c074e10 0x7efd1c0751f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.544+0000 7efd22c00700 1 --2- 172.21.15.1:0/1442296789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd1c075730 0x7efd1c10ecd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.544+0000 7efd22c00700 1 -- 172.21.15.1:0/1442296789 >> 172.21.15.1:0/1442296789 conn(0x7efd1c0fe4c0 msgr2=0x7efd1c1008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:19.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.544+0000 7efd22c00700 1 -- 172.21.15.1:0/1442296789 shutdown_connections 2024-04-26T01:52:19.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.544+0000 7efd22c00700 1 -- 172.21.15.1:0/1442296789 wait complete. 2024-04-26T01:52:19.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.545+0000 7efd22c00700 1 Processor -- start 2024-04-26T01:52:19.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.545+0000 7efd22c00700 1 -- start start 2024-04-26T01:52:19.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.546+0000 7efd22c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd1c074e10 0x7efd1c19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:19.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.546+0000 7efd22c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1c075730 0x7efd1c19b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:19.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.546+0000 7efd22c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd1c10f210 0x7efd1c19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:19.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.546+0000 7efd22c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c114db0 con 0x7efd1c10f210 2024-04-26T01:52:19.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.546+0000 7efd22c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c114c30 con 0x7efd1c075730 2024-04-26T01:52:19.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.546+0000 7efd22c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c114f30 con 0x7efd1c074e10 2024-04-26T01:52:19.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.546+0000 7efd21800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd1c074e10 0x7efd1c19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:19.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.546+0000 7efd21800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd1c074e10 0x7efd1c19b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:49688/0 (socket says 172.21.15.1:49688) 2024-04-26T01:52:19.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.546+0000 7efd21800700 1 -- 172.21.15.1:0/852880947 learned_addr learned my addr 172.21.15.1:0/852880947 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:19.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.546+0000 7efd22200700 1 --2- 172.21.15.1:0/852880947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd1c10f210 0x7efd1c19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:19.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.546+0000 7efd20e00700 1 --2- 172.21.15.1:0/852880947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1c075730 0x7efd1c19b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:19.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.547+0000 7efd20e00700 1 -- 172.21.15.1:0/852880947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd1c074e10 msgr2=0x7efd1c19b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:19.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.547+0000 7efd20e00700 1 --2- 172.21.15.1:0/852880947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd1c074e10 0x7efd1c19b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.547+0000 7efd20e00700 1 -- 172.21.15.1:0/852880947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd1c10f210 msgr2=0x7efd1c19fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:19.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.547+0000 7efd20e00700 1 --2- 172.21.15.1:0/852880947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd1c10f210 0x7efd1c19fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.547+0000 7efd20e00700 1 -- 172.21.15.1:0/852880947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c1a0290 con 0x7efd1c075730 2024-04-26T01:52:19.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.547+0000 7efd20e00700 1 --2- 172.21.15.1:0/852880947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1c075730 0x7efd1c19b860 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efd08000c00 tx=0x7efd080047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:19.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.548+0000 7efd1a000700 1 -- 172.21.15.1:0/852880947 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08026070 con 0x7efd1c075730 2024-04-26T01:52:19.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.548+0000 7efd1a000700 1 -- 172.21.15.1:0/852880947 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd08021410 con 0x7efd1c075730 2024-04-26T01:52:19.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.548+0000 7efd1a000700 1 -- 172.21.15.1:0/852880947 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08017650 con 0x7efd1c075730 2024-04-26T01:52:19.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.548+0000 7efd22c00700 1 -- 172.21.15.1:0/852880947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1c1a0520 con 0x7efd1c075730 2024-04-26T01:52:19.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.548+0000 7efd22c00700 1 -- 172.21.15.1:0/852880947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd1c1a7e00 con 0x7efd1c075730 2024-04-26T01:52:19.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.551+0000 7efd1a000700 1 -- 172.21.15.1:0/852880947 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7efd08029070 con 0x7efd1c075730 2024-04-26T01:52:19.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.553+0000 7efd2eb71b80 1 Processor -- start 2024-04-26T01:52:19.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.553+0000 7efd1a000700 1 --2- 172.21.15.1:0/852880947 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efd040a3fe0 0x7efd040a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:19.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.553+0000 7efd2eb71b80 1 -- start start 2024-04-26T01:52:19.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.553+0000 7efd1a000700 1 -- 172.21.15.1:0/852880947 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7efd080d1710 con 0x7efd1c075730 2024-04-26T01:52:19.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.553+0000 7efd22200700 1 --2- 172.21.15.1:0/852880947 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efd040a3fe0 0x7efd040a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:19.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.553+0000 7efd2eb71b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e09c6e53e0 0x55e09c6e57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:19.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.553+0000 7efd2eb71b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6e5d00 0x55e09c6f0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:19.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.554+0000 7efd2eb71b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6f0c40 0x55e09c6f3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:19.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.554+0000 7efd2eb71b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e09c638e40 con 0x55e09c6e5d00 2024-04-26T01:52:19.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.554+0000 7efd2eb71b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e09c5df6e0 con 0x55e09c6f0c40 2024-04-26T01:52:19.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.554+0000 7efd2eb71b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e09c634e00 con 0x55e09c6e53e0 2024-04-26T01:52:19.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.554+0000 7efd20e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6e5d00 0x55e09c6f0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:19.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.554+0000 7efd20e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6e5d00 0x55e09c6f0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37968/0 (socket says 172.21.15.1:37968) 2024-04-26T01:52:19.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.554+0000 7efd20e00700 1 -- 172.21.15.1:0/990563479 learned_addr learned my addr 172.21.15.1:0/990563479 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:19.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.554+0000 7efd22200700 1 --2- 172.21.15.1:0/990563479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6f0c40 0x55e09c6f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:19.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.554+0000 7efd20e00700 1 -- 172.21.15.1:0/990563479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e09c6e53e0 msgr2=0x55e09c6e57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:19.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.554+0000 7efd20e00700 1 --2- 172.21.15.1:0/990563479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e09c6e53e0 0x55e09c6e57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.554+0000 7efd20e00700 1 -- 172.21.15.1:0/990563479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6f0c40 msgr2=0x55e09c6f3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:19.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.554+0000 7efd20e00700 1 --2- 172.21.15.1:0/990563479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6f0c40 0x55e09c6f3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.554+0000 7efd20e00700 1 -- 172.21.15.1:0/990563479 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e09c5d5630 con 0x55e09c6e5d00 2024-04-26T01:52:19.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.555+0000 7efd22200700 1 --2- 172.21.15.1:0/852880947 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efd040a3fe0 0x7efd040a64a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efd1c06b350 tx=0x7efd14011720 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:19.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.555+0000 7efd20e00700 1 --2- 172.21.15.1:0/990563479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6e5d00 0x55e09c6f0700 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efd080098e0 tx=0x7efd0809ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f89237a0acdbbfa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:19.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.555+0000 7efd0ea00700 1 -- 172.21.15.1:0/990563479 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd080a18b0 con 0x55e09c6e5d00 2024-04-26T01:52:19.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.555+0000 7efd0ea00700 1 -- 172.21.15.1:0/990563479 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd080a60e0 con 0x55e09c6e5d00 2024-04-26T01:52:19.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.555+0000 7efd0ea00700 1 -- 172.21.15.1:0/990563479 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd080a5c50 con 0x55e09c6e5d00 2024-04-26T01:52:19.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.556+0000 7efd2eb71b80 1 -- 172.21.15.1:0/990563479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6e5d00 msgr2=0x55e09c6f0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:19.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.556+0000 7efd2eb71b80 1 --2- 172.21.15.1:0/990563479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6e5d00 0x55e09c6f0700 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efd080098e0 tx=0x7efd0809ce60 comp rx=0 tx=0).stop 2024-04-26T01:52:19.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.556+0000 7efd2eb71b80 1 -- 172.21.15.1:0/990563479 shutdown_connections 2024-04-26T01:52:19.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.556+0000 7efd2eb71b80 1 --2- 172.21.15.1:0/990563479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6f0c40 0x55e09c6f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.556+0000 7efd2eb71b80 1 --2- 172.21.15.1:0/990563479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e09c6e53e0 0x55e09c6e57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.556+0000 7efd2eb71b80 1 --2- 172.21.15.1:0/990563479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6e5d00 0x55e09c6f0700 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.556+0000 7efd2eb71b80 1 -- 172.21.15.1:0/990563479 >> 172.21.15.1:0/990563479 conn(0x55e09c614550 msgr2=0x55e09c6e1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:19.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.557+0000 7efd2eb71b80 1 -- 172.21.15.1:0/990563479 shutdown_connections 2024-04-26T01:52:19.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.557+0000 7efd2eb71b80 1 -- 172.21.15.1:0/990563479 wait complete. 2024-04-26T01:52:19.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.557+0000 7efd2eb71b80 1 -- start start 2024-04-26T01:52:19.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.558+0000 7efd2eb71b80 1 Processor -- start 2024-04-26T01:52:19.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.558+0000 7efd2eb71b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e09c6e53e0 0x55e09c6f0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:19.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.558+0000 7efd2eb71b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6e5d00 0x55e09c6ed650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:19.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.558+0000 7efd2eb71b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6f0c40 0x55e09c6edb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:19.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.558+0000 7efd2eb71b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e09c6f3570 con 0x55e09c6e5d00 2024-04-26T01:52:19.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.558+0000 7efd2eb71b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e09c5f2810 con 0x55e09c6f0c40 2024-04-26T01:52:19.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.558+0000 7efd2eb71b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e09c6f36f0 con 0x55e09c6e53e0 2024-04-26T01:52:19.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.559+0000 7efd2eb71b80 10 client.0 _ll_register_callbacks cb 0x55e09c5d16d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:19.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.559+0000 7efd20e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6e5d00 0x55e09c6ed650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:19.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.559+0000 7efd20e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6e5d00 0x55e09c6ed650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37972/0 (socket says 172.21.15.1:37972) 2024-04-26T01:52:19.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.559+0000 7efd20e00700 1 -- 172.21.15.1:0/1941282553 learned_addr learned my addr 172.21.15.1:0/1941282553 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:19.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.559+0000 7efd22200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6f0c40 0x55e09c6edb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:19.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.559+0000 7efd21800700 1 --2- 172.21.15.1:0/1941282553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e09c6e53e0 0x55e09c6f0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:19.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.560+0000 7efd22200700 1 -- 172.21.15.1:0/1941282553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e09c6e53e0 msgr2=0x55e09c6f0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:19.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.560+0000 7efd22200700 1 --2- 172.21.15.1:0/1941282553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e09c6e53e0 0x55e09c6f0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.560+0000 7efd22200700 1 -- 172.21.15.1:0/1941282553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6e5d00 msgr2=0x55e09c6ed650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:19.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.560+0000 7efd22200700 1 --2- 172.21.15.1:0/1941282553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6e5d00 0x55e09c6ed650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.560+0000 7efd22200700 1 -- 172.21.15.1:0/1941282553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e09c586880 con 0x55e09c6f0c40 2024-04-26T01:52:19.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.560+0000 7efd22200700 1 --2- 172.21.15.1:0/1941282553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6f0c40 0x55e09c6edb90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efd14009230 tx=0x7efd140047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10bc09288f0ab206 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:19.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.560+0000 7efd03400700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:52:19.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.560+0000 7efd03400700 1 -- 172.21.15.1:0/1941282553 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14004830 con 0x55e09c6f0c40 2024-04-26T01:52:19.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.560+0000 7efd03400700 1 -- 172.21.15.1:0/1941282553 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7efd140049d0 con 0x55e09c6f0c40 2024-04-26T01:52:19.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.561+0000 7efd2eb71b80 1 -- 172.21.15.1:0/1941282553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd080a60e0 con 0x55e09c6f0c40 2024-04-26T01:52:19.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.561+0000 7efd03400700 1 -- 172.21.15.1:0/1941282553 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efd14004bd0 con 0x55e09c6f0c40 2024-04-26T01:52:19.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.561+0000 7efd2eb71b80 10 client.25274 fetch_fsmap learned FSMap version 26 2024-04-26T01:52:19.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.561+0000 7efd2eb71b80 1 -- 172.21.15.1:0/1941282553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e09c5cfaa0 con 0x55e09c6f0c40 2024-04-26T01:52:19.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.561+0000 7efd03400700 1 -- 172.21.15.1:0/1941282553 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd080a60e0 con 0x55e09c6f0c40 2024-04-26T01:52:19.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.562+0000 7efd03400700 1 -- 172.21.15.1:0/1941282553 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7efd1400e930 con 0x55e09c6f0c40 2024-04-26T01:52:19.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.562+0000 7efd2eb71b80 10 client.25274 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:52:19.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.562+0000 7efd2eb71b80 10 client.25274 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T01:52:19.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.562+0000 7efd2eb71b80 20 client.25274 populate_metadata read hostname 'smithi001' 2024-04-26T01:52:19.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.562+0000 7efd2eb71b80 1 --2- 172.21.15.1:0/1941282553 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x55e09c7001a0 0x55e09c701610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:19.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.562+0000 7efd2eb71b80 4 client.25274 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:52:19.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.562+0000 7efd2eb71b80 1 -- 172.21.15.1:0/1941282553 --> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e09c646630 con 0x55e09c7001a0 2024-04-26T01:52:19.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.562+0000 7efd21800700 1 --2- 172.21.15.1:0/1941282553 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x55e09c7001a0 0x55e09c701610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:19.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.563+0000 7efd21800700 1 --2- 172.21.15.1:0/1941282553 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x55e09c7001a0 0x55e09c701610 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=882c306af98f1727 server_cookie=dc770b42252fbb5b in_seq=0 out_seq=0 2024-04-26T01:52:19.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.563+0000 7efd03400700 10 client.25274 ms_handle_connect on v2:172.21.15.173:6838/373692430 2024-04-26T01:52:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.564+0000 7efd03400700 1 -- 172.21.15.1:0/1941282553 <== mds.1 v2:172.21.15.173:6838/373692430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e09c646630 con 0x55e09c7001a0 2024-04-26T01:52:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.564+0000 7efd03400700 10 client.25274 handle_command_reply: tid=0 2024-04-26T01:52:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.564+0000 7efd2eb71b80 1 client.25274 shutdown 2024-04-26T01:52:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.564+0000 7efd2eb71b80 1 -- 172.21.15.1:0/1941282553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6f0c40 msgr2=0x55e09c6edb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.564+0000 7efd2eb71b80 1 --2- 172.21.15.1:0/1941282553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6f0c40 0x55e09c6edb90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efd14009230 tx=0x7efd140047c0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.564+0000 7efd2eb71b80 1 -- 172.21.15.1:0/1941282553 shutdown_connections 2024-04-26T01:52:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.564+0000 7efd2eb71b80 1 --2- 172.21.15.1:0/1941282553 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x55e09c7001a0 0x55e09c701610 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.564+0000 7efd2eb71b80 1 --2- 172.21.15.1:0/1941282553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6f0c40 0x55e09c6edb90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.564+0000 7efd2eb71b80 1 --2- 172.21.15.1:0/1941282553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e09c6e53e0 0x55e09c6f0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.564+0000 7efd2eb71b80 1 --2- 172.21.15.1:0/1941282553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6e5d00 0x55e09c6ed650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.564+0000 7efd2eb71b80 1 -- 172.21.15.1:0/1941282553 >> 172.21.15.1:0/1941282553 conn(0x55e09c614550 msgr2=0x55e09c6e1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:19.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.565+0000 7efd03400700 0 client.25274 ms_handle_reset on v2:172.21.15.173:6838/373692430 2024-04-26T01:52:19.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.565+0000 7efd2eb71b80 1 -- 172.21.15.1:0/1941282553 shutdown_connections 2024-04-26T01:52:19.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.565+0000 7efd2eb71b80 1 -- 172.21.15.1:0/1941282553 wait complete. 2024-04-26T01:52:19.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.565+0000 7efd2eb71b80 20 client.25274 trim_cache size 0 max 16384 2024-04-26T01:52:19.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.598+0000 7efd2eb71b80 1 Processor -- start 2024-04-26T01:52:19.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.599+0000 7efd2eb71b80 1 -- start start 2024-04-26T01:52:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.599+0000 7efd2eb71b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e09c6e53e0 0x55e09c6e1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.599+0000 7efd2eb71b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6e5d00 0x55e09c70aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.599+0000 7efd2eb71b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6f0c40 0x55e09c70afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.599+0000 7efd2eb71b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e09c6ff950 con 0x55e09c6f0c40 2024-04-26T01:52:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.599+0000 7efd2eb71b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e09c700020 con 0x55e09c6e5d00 2024-04-26T01:52:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.599+0000 7efd2eb71b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e09c7006f0 con 0x55e09c6e53e0 2024-04-26T01:52:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.600+0000 7efd20e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6e5d00 0x55e09c70aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.600+0000 7efd22200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6f0c40 0x55e09c70afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.600+0000 7efd20e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6e5d00 0x55e09c70aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:54564/0 (socket says 172.21.15.1:54564) 2024-04-26T01:52:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.600+0000 7efd20e00700 1 -- 172.21.15.1:0/853688613 learned_addr learned my addr 172.21.15.1:0/853688613 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.600+0000 7efd21800700 1 --2- 172.21.15.1:0/853688613 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e09c6e53e0 0x55e09c6e1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.600+0000 7efd20e00700 1 -- 172.21.15.1:0/853688613 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e09c6e53e0 msgr2=0x55e09c6e1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.600+0000 7efd20e00700 1 --2- 172.21.15.1:0/853688613 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e09c6e53e0 0x55e09c6e1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.600+0000 7efd20e00700 1 -- 172.21.15.1:0/853688613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6f0c40 msgr2=0x55e09c70afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.600+0000 7efd20e00700 1 --2- 172.21.15.1:0/853688613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6f0c40 0x55e09c70afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.600+0000 7efd20e00700 1 -- 172.21.15.1:0/853688613 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e09c6e6fb0 con 0x55e09c6e5d00 2024-04-26T01:52:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.600+0000 7efd22200700 1 --2- 172.21.15.1:0/853688613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6f0c40 0x55e09c70afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.600+0000 7efd20e00700 1 --2- 172.21.15.1:0/853688613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6e5d00 0x55e09c70aa70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efd0809d490 tx=0x7efd0809d140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2b1adf953665f3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.601+0000 7efd0e000700 1 -- 172.21.15.1:0/853688613 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd080a18b0 con 0x55e09c6e5d00 2024-04-26T01:52:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.601+0000 7efd0e000700 1 -- 172.21.15.1:0/853688613 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd080094b0 con 0x55e09c6e5d00 2024-04-26T01:52:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.601+0000 7efd0e000700 1 -- 172.21.15.1:0/853688613 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd080a5eb0 con 0x55e09c6e5d00 2024-04-26T01:52:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.602+0000 7efd2eb71b80 1 -- 172.21.15.1:0/853688613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6e5d00 msgr2=0x55e09c70aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.602+0000 7efd2eb71b80 1 --2- 172.21.15.1:0/853688613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6e5d00 0x55e09c70aa70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efd0809d490 tx=0x7efd0809d140 comp rx=0 tx=0).stop 2024-04-26T01:52:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.602+0000 7efd2eb71b80 1 -- 172.21.15.1:0/853688613 shutdown_connections 2024-04-26T01:52:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.602+0000 7efd2eb71b80 1 --2- 172.21.15.1:0/853688613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6f0c40 0x55e09c70afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.602+0000 7efd2eb71b80 1 --2- 172.21.15.1:0/853688613 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e09c6e53e0 0x55e09c6e1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.602+0000 7efd2eb71b80 1 --2- 172.21.15.1:0/853688613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6e5d00 0x55e09c70aa70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.602+0000 7efd2eb71b80 1 -- 172.21.15.1:0/853688613 >> 172.21.15.1:0/853688613 conn(0x55e09c614550 msgr2=0x55e09c6ed610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.602+0000 7efd2eb71b80 1 -- 172.21.15.1:0/853688613 shutdown_connections 2024-04-26T01:52:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.602+0000 7efd2eb71b80 1 -- 172.21.15.1:0/853688613 wait complete. 2024-04-26T01:52:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.602+0000 7efd2eb71b80 1 -- start start 2024-04-26T01:52:19.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.603+0000 7efd2eb71b80 1 Processor -- start 2024-04-26T01:52:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.603+0000 7efd2eb71b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6e53e0 0x55e09c70af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.603+0000 7efd2eb71b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6e5d00 0x55e09c7042d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.603+0000 7efd2eb71b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e09c6f0c40 0x55e09c7049c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.603+0000 7efd2eb71b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e09c701490 con 0x55e09c6e53e0 2024-04-26T01:52:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.603+0000 7efd2eb71b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e09c700dc0 con 0x55e09c6e5d00 2024-04-26T01:52:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.603+0000 7efd2eb71b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e09c701b60 con 0x55e09c6f0c40 2024-04-26T01:52:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.603+0000 7efd2eb71b80 10 client.0 _ll_register_callbacks cb 0x55e09c5d16d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.604+0000 7efd21800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6e53e0 0x55e09c70af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.604+0000 7efd20e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6e5d00 0x55e09c7042d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.604+0000 7efd20e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6e5d00 0x55e09c7042d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:54566/0 (socket says 172.21.15.1:54566) 2024-04-26T01:52:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.604+0000 7efd20e00700 1 -- 172.21.15.1:0/1354126046 learned_addr learned my addr 172.21.15.1:0/1354126046 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.604+0000 7efd22200700 1 --2- 172.21.15.1:0/1354126046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e09c6f0c40 0x55e09c7049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.604+0000 7efd20e00700 1 -- 172.21.15.1:0/1354126046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e09c6f0c40 msgr2=0x55e09c7049c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.604+0000 7efd20e00700 1 --2- 172.21.15.1:0/1354126046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e09c6f0c40 0x55e09c7049c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.604+0000 7efd20e00700 1 -- 172.21.15.1:0/1354126046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6e53e0 msgr2=0x55e09c70af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.604+0000 7efd20e00700 1 --2- 172.21.15.1:0/1354126046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6e53e0 0x55e09c70af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.604+0000 7efd20e00700 1 -- 172.21.15.1:0/1354126046 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e09c708a90 con 0x55e09c6e5d00 2024-04-26T01:52:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.604+0000 7efd20e00700 1 --2- 172.21.15.1:0/1354126046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6e5d00 0x55e09c7042d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efd0800b2e0 tx=0x7efd080bc400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4113ce55f250ae90 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.604+0000 7efd03400700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:52:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.605+0000 7efd03400700 1 -- 172.21.15.1:0/1354126046 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08083060 con 0x55e09c6e5d00 2024-04-26T01:52:19.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.605+0000 7efd2eb71b80 1 -- 172.21.15.1:0/1354126046 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd080094b0 con 0x55e09c6e5d00 2024-04-26T01:52:19.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.605+0000 7efd03400700 1 -- 172.21.15.1:0/1354126046 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7efd0809d970 con 0x55e09c6e5d00 2024-04-26T01:52:19.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.605+0000 7efd03400700 1 -- 172.21.15.1:0/1354126046 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efd080a18b0 con 0x55e09c6e5d00 2024-04-26T01:52:19.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.605+0000 7efd2eb71b80 10 client.25280 fetch_fsmap learned FSMap version 26 2024-04-26T01:52:19.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.605+0000 7efd2eb71b80 1 -- 172.21.15.1:0/1354126046 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e09c7674a0 con 0x55e09c6e5d00 2024-04-26T01:52:19.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.605+0000 7efd03400700 1 -- 172.21.15.1:0/1354126046 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd080094b0 con 0x55e09c6e5d00 2024-04-26T01:52:19.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.605+0000 7efd03400700 1 -- 172.21.15.1:0/1354126046 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7efd08099250 con 0x55e09c6e5d00 2024-04-26T01:52:19.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.605+0000 7efd2eb71b80 10 client.25280 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:52:19.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.606+0000 7efd2eb71b80 10 client.25280 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T01:52:19.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.606+0000 7efd2eb71b80 20 client.25280 populate_metadata read hostname 'smithi001' 2024-04-26T01:52:19.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.606+0000 7efd2eb71b80 1 --2- 172.21.15.1:0/1354126046 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x55e09c76bb90 0x55e09c76a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:19.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.606+0000 7efd2eb71b80 4 client.25280 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T01:52:19.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.606+0000 7efd2eb71b80 1 -- 172.21.15.1:0/1354126046 --> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] -- command(tid 0: {"prefix": "status"}) -- 0x55e09c6406d0 con 0x55e09c76bb90 2024-04-26T01:52:19.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.606+0000 7efd21800700 1 --2- 172.21.15.1:0/1354126046 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x55e09c76bb90 0x55e09c76a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:19.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.607+0000 7efd21800700 1 --2- 172.21.15.1:0/1354126046 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x55e09c76bb90 0x55e09c76a8a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b36521429b494fe server_cookie=232ebdf916b9fc5d in_seq=0 out_seq=0 2024-04-26T01:52:19.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.607+0000 7efd03400700 10 client.25280 ms_handle_connect on v2:172.21.15.173:6838/373692430 2024-04-26T01:52:19.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.607+0000 7efd03400700 1 -- 172.21.15.1:0/1354126046 <== mds.1 v2:172.21.15.173:6838/373692430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e09c6406d0 con 0x55e09c76bb90 2024-04-26T01:52:19.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.607+0000 7efd03400700 10 client.25280 handle_command_reply: tid=0 2024-04-26T01:52:19.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.607+0000 7efd2eb71b80 1 client.25280 shutdown 2024-04-26T01:52:19.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.608+0000 7efd2eb71b80 1 -- 172.21.15.1:0/1354126046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6e5d00 msgr2=0x55e09c7042d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:19.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.608+0000 7efd2eb71b80 1 --2- 172.21.15.1:0/1354126046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6e5d00 0x55e09c7042d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efd0800b2e0 tx=0x7efd080bc400 comp rx=0 tx=0).stop 2024-04-26T01:52:19.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.608+0000 7efd2eb71b80 1 -- 172.21.15.1:0/1354126046 shutdown_connections 2024-04-26T01:52:19.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.608+0000 7efd2eb71b80 1 --2- 172.21.15.1:0/1354126046 >> [v2:172.21.15.173:6838/373692430,v1:172.21.15.173:6839/373692430] conn(0x55e09c76bb90 0x55e09c76a8a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.608+0000 7efd2eb71b80 1 --2- 172.21.15.1:0/1354126046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e09c6f0c40 0x55e09c7049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.608+0000 7efd2eb71b80 1 --2- 172.21.15.1:0/1354126046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e09c6e53e0 0x55e09c70af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.608+0000 7efd2eb71b80 1 --2- 172.21.15.1:0/1354126046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e09c6e5d00 0x55e09c7042d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.608+0000 7efd2eb71b80 1 -- 172.21.15.1:0/1354126046 >> 172.21.15.1:0/1354126046 conn(0x55e09c614550 msgr2=0x55e09c6eee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:19.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.608+0000 7efd03400700 0 client.25280 ms_handle_reset on v2:172.21.15.173:6838/373692430 2024-04-26T01:52:19.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.608+0000 7efd2eb71b80 1 -- 172.21.15.1:0/1354126046 shutdown_connections 2024-04-26T01:52:19.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.608+0000 7efd2eb71b80 1 -- 172.21.15.1:0/1354126046 wait complete. 2024-04-26T01:52:19.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.608+0000 7efd2eb71b80 20 client.25280 trim_cache size 0 max 16384 2024-04-26T01:52:19.609 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T01:52:19.609 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "77cfcc80-036d-11ef-bc93-c7b262605968", 2024-04-26T01:52:19.610 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 1, 2024-04-26T01:52:19.610 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24514, 2024-04-26T01:52:19.610 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-04-26T01:52:19.610 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-04-26T01:52:19.610 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-04-26T01:52:19.610 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 273.407487145, 2024-04-26T01:52:19.610 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 26, 2024-04-26T01:52:19.610 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 93, 2024-04-26T01:52:19.610 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 92, 2024-04-26T01:52:19.610 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 322.74917065300002 2024-04-26T01:52:19.610 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T01:52:19.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.609+0000 7efd22c00700 1 -- 172.21.15.1:0/852880947 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efd040a3fe0 msgr2=0x7efd040a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:19.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.609+0000 7efd22c00700 1 --2- 172.21.15.1:0/852880947 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efd040a3fe0 0x7efd040a64a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efd1c06b350 tx=0x7efd14011720 comp rx=0 tx=0).stop 2024-04-26T01:52:19.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.609+0000 7efd22c00700 1 -- 172.21.15.1:0/852880947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1c075730 msgr2=0x7efd1c19b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:19.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.609+0000 7efd22c00700 1 --2- 172.21.15.1:0/852880947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1c075730 0x7efd1c19b860 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efd08000c00 tx=0x7efd080047b0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.609+0000 7efd22c00700 1 -- 172.21.15.1:0/852880947 shutdown_connections 2024-04-26T01:52:19.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.609+0000 7efd22c00700 1 --2- 172.21.15.1:0/852880947 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efd040a3fe0 0x7efd040a64a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.609+0000 7efd22c00700 1 --2- 172.21.15.1:0/852880947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd1c10f210 0x7efd1c19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.609+0000 7efd22c00700 1 --2- 172.21.15.1:0/852880947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd1c074e10 0x7efd1c19b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.609+0000 7efd22c00700 1 --2- 172.21.15.1:0/852880947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1c075730 0x7efd1c19b860 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.609+0000 7efd22c00700 1 -- 172.21.15.1:0/852880947 >> 172.21.15.1:0/852880947 conn(0x7efd1c0fe4c0 msgr2=0x7efd1c0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:19.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.609+0000 7efd22c00700 1 -- 172.21.15.1:0/852880947 shutdown_connections 2024-04-26T01:52:19.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:19.609+0000 7efd22c00700 1 -- 172.21.15.1:0/852880947 wait complete. 2024-04-26T01:52:20.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:20 smithi001 ceph-mon[27822]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-04-26T01:52:20.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:20 smithi097 ceph-mon[32853]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-04-26T01:52:20.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:20 smithi173 ceph-mon[32553]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-04-26T01:52:20.698 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-26T01:52:21.161 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.691+0000 7fdea7400640 1 -- 172.21.15.1:0/1728717793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdea01043c0 msgr2=0x7fdea01047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.691+0000 7fdea7400640 1 --2- 172.21.15.1:0/1728717793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdea01043c0 0x7fdea01047a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fde94009900 tx=0x7fde9402a410 comp rx=0 tx=0).stop 2024-04-26T01:52:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.691+0000 7fdea7400640 1 -- 172.21.15.1:0/1728717793 shutdown_connections 2024-04-26T01:52:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.691+0000 7fdea7400640 1 --2- 172.21.15.1:0/1728717793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdea0109490 0x7fdea0111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.691+0000 7fdea7400640 1 --2- 172.21.15.1:0/1728717793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdea0104d70 0x7fdea0108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.691+0000 7fdea7400640 1 --2- 172.21.15.1:0/1728717793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdea01043c0 0x7fdea01047a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.691+0000 7fdea7400640 1 -- 172.21.15.1:0/1728717793 >> 172.21.15.1:0/1728717793 conn(0x7fdea00ffc20 msgr2=0x7fdea0102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.692+0000 7fdea7400640 1 -- 172.21.15.1:0/1728717793 shutdown_connections 2024-04-26T01:52:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.692+0000 7fdea7400640 1 -- 172.21.15.1:0/1728717793 wait complete. 2024-04-26T01:52:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.693+0000 7fdea7400640 1 Processor -- start 2024-04-26T01:52:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.693+0000 7fdea7400640 1 -- start start 2024-04-26T01:52:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.693+0000 7fdea7400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdea01043c0 0x7fdea01a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.693+0000 7fdea7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdea0109490 0x7fdea01a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.693+0000 7fdea7400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdea019a800 0x7fdea019acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.694+0000 7fdea7400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdea0114420 con 0x7fdea019a800 2024-04-26T01:52:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.694+0000 7fdea7400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdea01142a0 con 0x7fdea0109490 2024-04-26T01:52:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.694+0000 7fdea7400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdea01145a0 con 0x7fdea01043c0 2024-04-26T01:52:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.694+0000 7fdea5400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdea019a800 0x7fdea019acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.694+0000 7fdea4a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdea01043c0 0x7fdea01a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.694+0000 7fde9fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdea0109490 0x7fdea01a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.694+0000 7fdea5400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdea019a800 0x7fdea019acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38004/0 (socket says 172.21.15.1:38004) 2024-04-26T01:52:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.694+0000 7fdea5400640 1 -- 172.21.15.1:0/1401003764 learned_addr learned my addr 172.21.15.1:0/1401003764 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.695+0000 7fde9fe00640 1 -- 172.21.15.1:0/1401003764 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdea01043c0 msgr2=0x7fdea01a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.695+0000 7fde9fe00640 1 --2- 172.21.15.1:0/1401003764 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdea01043c0 0x7fdea01a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.695+0000 7fde9fe00640 1 -- 172.21.15.1:0/1401003764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdea019a800 msgr2=0x7fdea019acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.695+0000 7fde9fe00640 1 --2- 172.21.15.1:0/1401003764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdea019a800 0x7fdea019acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.695+0000 7fde9fe00640 1 -- 172.21.15.1:0/1401003764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea019b390 con 0x7fdea0109490 2024-04-26T01:52:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.695+0000 7fde9fe00640 1 --2- 172.21.15.1:0/1401003764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdea0109490 0x7fdea01a0bd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00e9c0 tx=0x7fde8c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.695+0000 7fde9d600640 1 -- 172.21.15.1:0/1401003764 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c00ccc0 con 0x7fdea0109490 2024-04-26T01:52:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.695+0000 7fde9d600640 1 -- 172.21.15.1:0/1401003764 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde8c0108e0 con 0x7fdea0109490 2024-04-26T01:52:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.695+0000 7fdea7400640 1 -- 172.21.15.1:0/1401003764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea019b680 con 0x7fdea0109490 2024-04-26T01:52:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.695+0000 7fde9d600640 1 -- 172.21.15.1:0/1401003764 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c0187c0 con 0x7fdea0109490 2024-04-26T01:52:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.696+0000 7fdea7400640 1 -- 172.21.15.1:0/1401003764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea01a78b0 con 0x7fdea0109490 2024-04-26T01:52:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.699+0000 7fdeaa171740 1 Processor -- start 2024-04-26T01:52:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.700+0000 7fdeaa171740 1 -- start start 2024-04-26T01:52:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.700+0000 7fdeaa171740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48ea9820 0x564d48ea9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.700+0000 7fde9d600640 1 -- 172.21.15.1:0/1401003764 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7fde8c010430 con 0x7fdea0109490 2024-04-26T01:52:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.700+0000 7fdeaa171740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eaa140 0x564d48eb44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.700+0000 7fdeaa171740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48eb4a00 0x564d48eb6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.700+0000 7fdeaa171740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564d48e0f900 con 0x564d48eaa140 2024-04-26T01:52:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.700+0000 7fdeaa171740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564d48db5ef0 con 0x564d48ea9820 2024-04-26T01:52:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.700+0000 7fdeaa171740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564d48e0b8c0 con 0x564d48eb4a00 2024-04-26T01:52:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.700+0000 7fdea4a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eaa140 0x564d48eb44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.700+0000 7fdea4a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eaa140 0x564d48eb44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38006/0 (socket says 172.21.15.1:38006) 2024-04-26T01:52:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.700+0000 7fde9fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48eb4a00 0x564d48eb6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.700+0000 7fdea4a00640 1 -- 172.21.15.1:0/645259708 learned_addr learned my addr 172.21.15.1:0/645259708 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.701+0000 7fdea5400640 1 --2- 172.21.15.1:0/645259708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48ea9820 0x564d48ea9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.701+0000 7fdea4a00640 1 -- 172.21.15.1:0/645259708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48eb4a00 msgr2=0x564d48eb6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.701+0000 7fdea4a00640 1 --2- 172.21.15.1:0/645259708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48eb4a00 0x564d48eb6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.701+0000 7fdea4a00640 1 -- 172.21.15.1:0/645259708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48ea9820 msgr2=0x564d48ea9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.701+0000 7fdea4a00640 1 --2- 172.21.15.1:0/645259708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48ea9820 0x564d48ea9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.701+0000 7fdea4a00640 1 -- 172.21.15.1:0/645259708 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d48b22210 con 0x564d48eaa140 2024-04-26T01:52:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.701+0000 7fdea5400640 1 --2- 172.21.15.1:0/645259708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48ea9820 0x564d48ea9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.701+0000 7fdea4a00640 1 --2- 172.21.15.1:0/645259708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eaa140 0x564d48eb44c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fde94009230 tx=0x7fde94003720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1efa5d85160d4d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.702+0000 7fde91c00640 1 -- 172.21.15.1:0/645259708 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94004940 con 0x564d48eaa140 2024-04-26T01:52:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.702+0000 7fde91c00640 1 -- 172.21.15.1:0/645259708 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde940318f0 con 0x564d48eaa140 2024-04-26T01:52:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.702+0000 7fde91c00640 1 -- 172.21.15.1:0/645259708 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9403d8f0 con 0x564d48eaa140 2024-04-26T01:52:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.703+0000 7fdeaa171740 1 -- 172.21.15.1:0/645259708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eaa140 msgr2=0x564d48eb44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.703+0000 7fdeaa171740 1 --2- 172.21.15.1:0/645259708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eaa140 0x564d48eb44c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fde94009230 tx=0x7fde94003720 comp rx=0 tx=0).stop 2024-04-26T01:52:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.703+0000 7fde9d600640 1 --2- 172.21.15.1:0/1401003764 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fde880a3f40 0x7fde880a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.703+0000 7fde9d600640 1 -- 172.21.15.1:0/1401003764 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7fde8c0d0d80 con 0x7fdea0109490 2024-04-26T01:52:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.703+0000 7fdeaa171740 1 -- 172.21.15.1:0/645259708 shutdown_connections 2024-04-26T01:52:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.703+0000 7fdeaa171740 1 --2- 172.21.15.1:0/645259708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48eb4a00 0x564d48eb6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.703+0000 7fdeaa171740 1 --2- 172.21.15.1:0/645259708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eaa140 0x564d48eb44c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.703+0000 7fdeaa171740 1 --2- 172.21.15.1:0/645259708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48ea9820 0x564d48ea9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.703+0000 7fdeaa171740 1 -- 172.21.15.1:0/645259708 >> 172.21.15.1:0/645259708 conn(0x564d48ea5d30 msgr2=0x564d48ea6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.704+0000 7fdea5400640 1 --2- 172.21.15.1:0/1401003764 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fde880a3f40 0x7fde880a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.704+0000 7fdeaa171740 1 -- 172.21.15.1:0/645259708 shutdown_connections 2024-04-26T01:52:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.704+0000 7fdeaa171740 1 -- 172.21.15.1:0/645259708 wait complete. 2024-04-26T01:52:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.705+0000 7fdea5400640 1 --2- 172.21.15.1:0/1401003764 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fde880a3f40 0x7fde880a6400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fde98009370 tx=0x7fde98009760 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.705+0000 7fdeaa171740 1 -- start start 2024-04-26T01:52:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.705+0000 7fdeaa171740 1 Processor -- start 2024-04-26T01:52:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.706+0000 7fdeaa171740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48ea9820 0x564d48eb1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.706+0000 7fdeaa171740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48eaa140 0x564d48eb1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.706+0000 7fdea4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48ea9820 0x564d48eb1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.706+0000 7fdea4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48ea9820 0x564d48eb1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:54584/0 (socket says 172.21.15.1:54584) 2024-04-26T01:52:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.706+0000 7fdea4a00640 1 -- 172.21.15.1:0/62666001 learned_addr learned my addr 172.21.15.1:0/62666001 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.707+0000 7fdeaa171740 1 --2- 172.21.15.1:0/62666001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eb4a00 0x564d48eb2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.707+0000 7fde9fe00640 1 --2- 172.21.15.1:0/62666001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48eaa140 0x564d48eb1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.707+0000 7fdeaa171740 1 -- 172.21.15.1:0/62666001 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564d48eb75d0 con 0x564d48eb4a00 2024-04-26T01:52:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.707+0000 7fdeaa171740 1 -- 172.21.15.1:0/62666001 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564d48dc9310 con 0x564d48ea9820 2024-04-26T01:52:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.707+0000 7fdeaa171740 1 -- 172.21.15.1:0/62666001 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564d48eb7750 con 0x564d48eaa140 2024-04-26T01:52:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.707+0000 7fdeaa171740 10 client.0 _ll_register_callbacks cb 0x564d48d97cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.707+0000 7fdea5400640 1 --2- 172.21.15.1:0/62666001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eb4a00 0x564d48eb2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.708+0000 7fde9fe00640 1 -- 172.21.15.1:0/62666001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48ea9820 msgr2=0x564d48eb1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.708+0000 7fde9fe00640 1 --2- 172.21.15.1:0/62666001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48ea9820 0x564d48eb1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.708+0000 7fde9fe00640 1 -- 172.21.15.1:0/62666001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eb4a00 msgr2=0x564d48eb2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.708+0000 7fde9fe00640 1 --2- 172.21.15.1:0/62666001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eb4a00 0x564d48eb2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.708+0000 7fde9fe00640 1 -- 172.21.15.1:0/62666001 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d48bee620 con 0x564d48eaa140 2024-04-26T01:52:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.708+0000 7fde9fe00640 1 --2- 172.21.15.1:0/62666001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48eaa140 0x564d48eb1ed0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fde8c0ce290 tx=0x7fde8c010720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecefd06f34fed242 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.708+0000 7fde86000640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:52:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.708+0000 7fde86000640 1 -- 172.21.15.1:0/62666001 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c0d2040 con 0x564d48eaa140 2024-04-26T01:52:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.708+0000 7fdeaa171740 1 -- 172.21.15.1:0/62666001 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde940318f0 con 0x564d48eaa140 2024-04-26T01:52:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.708+0000 7fde86000640 1 -- 172.21.15.1:0/62666001 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7fde8c010a90 con 0x564d48eaa140 2024-04-26T01:52:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.709+0000 7fde86000640 1 -- 172.21.15.1:0/62666001 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fde8c0a5750 con 0x564d48eaa140 2024-04-26T01:52:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.709+0000 7fdeaa171740 10 client.25306 fetch_fsmap learned FSMap version 26 2024-04-26T01:52:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.709+0000 7fdeaa171740 1 -- 172.21.15.1:0/62666001 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564d48ecc090 con 0x564d48eaa140 2024-04-26T01:52:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.709+0000 7fde86000640 1 -- 172.21.15.1:0/62666001 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde940318f0 con 0x564d48eaa140 2024-04-26T01:52:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.709+0000 7fde86000640 1 -- 172.21.15.1:0/62666001 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fde8c0d4020 con 0x564d48eaa140 2024-04-26T01:52:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.709+0000 7fdeaa171740 10 client.25306 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:52:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.709+0000 7fdeaa171740 10 client.25306 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:52:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.709+0000 7fdeaa171740 20 client.25306 populate_metadata read hostname 'smithi001' 2024-04-26T01:52:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.709+0000 7fdeaa171740 1 --2- 172.21.15.1:0/62666001 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564d48ecd500 0x564d48ecf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.709+0000 7fdeaa171740 4 client.25306 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:52:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.709+0000 7fdeaa171740 1 -- 172.21.15.1:0/62666001 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d48e238b0 con 0x564d48ecd500 2024-04-26T01:52:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.710+0000 7fdea4a00640 1 --2- 172.21.15.1:0/62666001 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564d48ecd500 0x564d48ecf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.710+0000 7fdea4a00640 1 --2- 172.21.15.1:0/62666001 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564d48ecd500 0x564d48ecf8f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb2a402f59123d1e server_cookie=c086149ac3505a21 in_seq=0 out_seq=0 2024-04-26T01:52:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.710+0000 7fde86000640 10 client.25306 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.711+0000 7fde86000640 1 -- 172.21.15.1:0/62666001 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d48e238b0 con 0x564d48ecd500 2024-04-26T01:52:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.711+0000 7fde86000640 10 client.25306 handle_command_reply: tid=0 2024-04-26T01:52:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.711+0000 7fdeaa171740 1 client.25306 shutdown 2024-04-26T01:52:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.712+0000 7fdeaa171740 1 -- 172.21.15.1:0/62666001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48eaa140 msgr2=0x564d48eb1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.712+0000 7fdeaa171740 1 --2- 172.21.15.1:0/62666001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48eaa140 0x564d48eb1ed0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fde8c0ce290 tx=0x7fde8c010720 comp rx=0 tx=0).stop 2024-04-26T01:52:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.712+0000 7fdeaa171740 1 -- 172.21.15.1:0/62666001 shutdown_connections 2024-04-26T01:52:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.712+0000 7fdeaa171740 1 --2- 172.21.15.1:0/62666001 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564d48ecd500 0x564d48ecf8f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.712+0000 7fdeaa171740 1 --2- 172.21.15.1:0/62666001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eb4a00 0x564d48eb2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.712+0000 7fdeaa171740 1 --2- 172.21.15.1:0/62666001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48eaa140 0x564d48eb1ed0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.712+0000 7fdeaa171740 1 --2- 172.21.15.1:0/62666001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48ea9820 0x564d48eb1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.712+0000 7fde86000640 0 client.25306 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.712+0000 7fdeaa171740 1 -- 172.21.15.1:0/62666001 >> 172.21.15.1:0/62666001 conn(0x564d48ea5d30 msgr2=0x564d48eabab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.712+0000 7fdeaa171740 1 -- 172.21.15.1:0/62666001 shutdown_connections 2024-04-26T01:52:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.712+0000 7fdeaa171740 1 -- 172.21.15.1:0/62666001 wait complete. 2024-04-26T01:52:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.712+0000 7fdeaa171740 20 client.25306 trim_cache size 0 max 16384 2024-04-26T01:52:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.730+0000 7fdeaa171740 1 Processor -- start 2024-04-26T01:52:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.730+0000 7fdeaa171740 1 -- start start 2024-04-26T01:52:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.731+0000 7fdeaa171740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48ea9820 0x564d48ed51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.731+0000 7fdeaa171740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eaa140 0x564d48ed56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.731+0000 7fdeaa171740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48eb4a00 0x564d48ea4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.731+0000 7fdeaa171740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdea01ae340 con 0x564d48eaa140 2024-04-26T01:52:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.731+0000 7fdeaa171740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdea01abc10 con 0x564d48eb4a00 2024-04-26T01:52:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.731+0000 7fdeaa171740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdea01a9a80 con 0x564d48ea9820 2024-04-26T01:52:21.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.731+0000 7fdea4a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48ea9820 0x564d48ed51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:21.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.731+0000 7fdea5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48eb4a00 0x564d48ea4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:21.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.731+0000 7fdea4a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48ea9820 0x564d48ed51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:49756/0 (socket says 172.21.15.1:49756) 2024-04-26T01:52:21.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.731+0000 7fdea5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48eb4a00 0x564d48ea4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:54590/0 (socket says 172.21.15.1:54590) 2024-04-26T01:52:21.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.731+0000 7fdea4a00640 1 -- 172.21.15.1:0/687680212 learned_addr learned my addr 172.21.15.1:0/687680212 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.731+0000 7fde9fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eaa140 0x564d48ed56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.732+0000 7fdea5400640 1 -- 172.21.15.1:0/687680212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48ea9820 msgr2=0x564d48ed51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.732+0000 7fdea5400640 1 --2- 172.21.15.1:0/687680212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48ea9820 0x564d48ed51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.732+0000 7fdea5400640 1 -- 172.21.15.1:0/687680212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eaa140 msgr2=0x564d48ed56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.732+0000 7fdea5400640 1 --2- 172.21.15.1:0/687680212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eaa140 0x564d48ed56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.732+0000 7fdea5400640 1 -- 172.21.15.1:0/687680212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d48eafee0 con 0x564d48eb4a00 2024-04-26T01:52:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.732+0000 7fdea4a00640 1 --2- 172.21.15.1:0/687680212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48ea9820 0x564d48ed51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.732+0000 7fdea5400640 1 --2- 172.21.15.1:0/687680212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48eb4a00 0x564d48ea4600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fde98005f00 tx=0x7fde98019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99b3aaab5e478147 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.733+0000 7fde91200640 1 -- 172.21.15.1:0/687680212 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9801e970 con 0x564d48eb4a00 2024-04-26T01:52:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.733+0000 7fde91200640 1 -- 172.21.15.1:0/687680212 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde9801c3f0 con 0x564d48eb4a00 2024-04-26T01:52:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.733+0000 7fde91200640 1 -- 172.21.15.1:0/687680212 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9801b5f0 con 0x564d48eb4a00 2024-04-26T01:52:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.733+0000 7fdeaa171740 1 -- 172.21.15.1:0/687680212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48eb4a00 msgr2=0x564d48ea4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.733+0000 7fdeaa171740 1 --2- 172.21.15.1:0/687680212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48eb4a00 0x564d48ea4600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fde98005f00 tx=0x7fde98019000 comp rx=0 tx=0).stop 2024-04-26T01:52:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.734+0000 7fdeaa171740 1 -- 172.21.15.1:0/687680212 shutdown_connections 2024-04-26T01:52:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.734+0000 7fdeaa171740 1 --2- 172.21.15.1:0/687680212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48eb4a00 0x564d48ea4600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.734+0000 7fdeaa171740 1 --2- 172.21.15.1:0/687680212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eaa140 0x564d48ed56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.734+0000 7fdeaa171740 1 --2- 172.21.15.1:0/687680212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48ea9820 0x564d48ed51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.734+0000 7fdeaa171740 1 -- 172.21.15.1:0/687680212 >> 172.21.15.1:0/687680212 conn(0x564d48ea5d30 msgr2=0x564d48eacc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.734+0000 7fdeaa171740 1 -- 172.21.15.1:0/687680212 shutdown_connections 2024-04-26T01:52:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.734+0000 7fdeaa171740 1 -- 172.21.15.1:0/687680212 wait complete. 2024-04-26T01:52:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.734+0000 7fdeaa171740 1 -- start start 2024-04-26T01:52:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.734+0000 7fdeaa171740 1 Processor -- start 2024-04-26T01:52:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.735+0000 7fdeaa171740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48ea9820 0x564d48ec5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.735+0000 7fdeaa171740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eaa140 0x564d48ec2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.735+0000 7fdeaa171740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48eb4a00 0x564d48ec3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.735+0000 7fdeaa171740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdea01aa0d0 con 0x564d48eaa140 2024-04-26T01:52:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.735+0000 7fdeaa171740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdea01ab540 con 0x564d48eb4a00 2024-04-26T01:52:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.735+0000 7fdeaa171740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdea01aa7a0 con 0x564d48ea9820 2024-04-26T01:52:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.735+0000 7fdeaa171740 10 client.0 _ll_register_callbacks cb 0x7fde8c09c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.735+0000 7fdea5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48eb4a00 0x564d48ec3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.736+0000 7fdea5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48eb4a00 0x564d48ec3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:54596/0 (socket says 172.21.15.1:54596) 2024-04-26T01:52:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.736+0000 7fdea5400640 1 -- 172.21.15.1:0/3446264237 learned_addr learned my addr 172.21.15.1:0/3446264237 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.736+0000 7fde9fe00640 1 --2- 172.21.15.1:0/3446264237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eaa140 0x564d48ec2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.736+0000 7fdea4a00640 1 --2- 172.21.15.1:0/3446264237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48ea9820 0x564d48ec5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.736+0000 7fdea5400640 1 -- 172.21.15.1:0/3446264237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48ea9820 msgr2=0x564d48ec5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.736+0000 7fdea5400640 1 --2- 172.21.15.1:0/3446264237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48ea9820 0x564d48ec5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.736+0000 7fdea5400640 1 -- 172.21.15.1:0/3446264237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eaa140 msgr2=0x564d48ec2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.736+0000 7fdea5400640 1 --2- 172.21.15.1:0/3446264237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eaa140 0x564d48ec2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.736+0000 7fdea5400640 1 -- 172.21.15.1:0/3446264237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d48f32a30 con 0x564d48eb4a00 2024-04-26T01:52:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.736+0000 7fde9fe00640 1 --2- 172.21.15.1:0/3446264237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eaa140 0x564d48ec2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:52:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.736+0000 7fdea5400640 1 --2- 172.21.15.1:0/3446264237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48eb4a00 0x564d48ec3250 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fde98018040 tx=0x7fde98010fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72263ebc112a13ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.736+0000 7fde86000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:52:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.737+0000 7fde86000640 1 -- 172.21.15.1:0/3446264237 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde98036060 con 0x564d48eb4a00 2024-04-26T01:52:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.737+0000 7fde86000640 1 -- 172.21.15.1:0/3446264237 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7fde9801c5a0 con 0x564d48eb4a00 2024-04-26T01:52:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.737+0000 7fdeaa171740 1 -- 172.21.15.1:0/3446264237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde9801c3f0 con 0x564d48eb4a00 2024-04-26T01:52:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.737+0000 7fde86000640 1 -- 172.21.15.1:0/3446264237 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fde9801c7a0 con 0x564d48eb4a00 2024-04-26T01:52:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.737+0000 7fdeaa171740 10 client.25298 fetch_fsmap learned FSMap version 26 2024-04-26T01:52:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.737+0000 7fdeaa171740 1 -- 172.21.15.1:0/3446264237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564d48f33be0 con 0x564d48eb4a00 2024-04-26T01:52:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.738+0000 7fde86000640 1 -- 172.21.15.1:0/3446264237 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde9801c3f0 con 0x564d48eb4a00 2024-04-26T01:52:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.738+0000 7fde86000640 1 -- 172.21.15.1:0/3446264237 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fde98005040 con 0x564d48eb4a00 2024-04-26T01:52:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.738+0000 7fdeaa171740 10 client.25298 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:52:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.738+0000 7fdeaa171740 10 client.25298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:52:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.738+0000 7fdeaa171740 20 client.25298 populate_metadata read hostname 'smithi001' 2024-04-26T01:52:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.738+0000 7fdeaa171740 1 --2- 172.21.15.1:0/3446264237 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564d48f341f0 0x564d48f36610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.738+0000 7fdeaa171740 4 client.25298 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:52:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.738+0000 7fdeaa171740 1 -- 172.21.15.1:0/3446264237 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564d48e1d320 con 0x564d48f341f0 2024-04-26T01:52:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.738+0000 7fdea4a00640 1 --2- 172.21.15.1:0/3446264237 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564d48f341f0 0x564d48f36610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.739+0000 7fdea4a00640 1 --2- 172.21.15.1:0/3446264237 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564d48f341f0 0x564d48f36610 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f65fd4b44193c2f server_cookie=e6fff8514212f97f in_seq=0 out_seq=0 2024-04-26T01:52:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.739+0000 7fde86000640 10 client.25298 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.749+0000 7fde86000640 1 -- 172.21.15.1:0/3446264237 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564d48e1d320 con 0x564d48f341f0 2024-04-26T01:52:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.749+0000 7fde86000640 10 client.25298 handle_command_reply: tid=0 2024-04-26T01:52:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.749+0000 7fdeaa171740 1 client.25298 shutdown 2024-04-26T01:52:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.749+0000 7fdeaa171740 1 -- 172.21.15.1:0/3446264237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48eb4a00 msgr2=0x564d48ec3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.749+0000 7fdeaa171740 1 --2- 172.21.15.1:0/3446264237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48eb4a00 0x564d48ec3250 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fde98018040 tx=0x7fde98010fb0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.749+0000 7fdeaa171740 1 -- 172.21.15.1:0/3446264237 shutdown_connections 2024-04-26T01:52:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.749+0000 7fdeaa171740 1 --2- 172.21.15.1:0/3446264237 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564d48f341f0 0x564d48f36610 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.749+0000 7fdeaa171740 1 --2- 172.21.15.1:0/3446264237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d48eb4a00 0x564d48ec3250 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.749+0000 7fdeaa171740 1 --2- 172.21.15.1:0/3446264237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d48eaa140 0x564d48ec2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.749+0000 7fde86000640 0 client.25298 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.749+0000 7fdeaa171740 1 --2- 172.21.15.1:0/3446264237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564d48ea9820 0x564d48ec5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.749+0000 7fdeaa171740 1 -- 172.21.15.1:0/3446264237 >> 172.21.15.1:0/3446264237 conn(0x564d48ea5d30 msgr2=0x564d48eb1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.750+0000 7fdeaa171740 1 -- 172.21.15.1:0/3446264237 shutdown_connections 2024-04-26T01:52:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.750+0000 7fdeaa171740 1 -- 172.21.15.1:0/3446264237 wait complete. 2024-04-26T01:52:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.750+0000 7fdeaa171740 20 client.25298 trim_cache size 0 max 16384 2024-04-26T01:52:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:52:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:52:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:52:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:52:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.751+0000 7fdea7400640 1 -- 172.21.15.1:0/1401003764 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fde880a3f40 msgr2=0x7fde880a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.751+0000 7fdea7400640 1 --2- 172.21.15.1:0/1401003764 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fde880a3f40 0x7fde880a6400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fde98009370 tx=0x7fde98009760 comp rx=0 tx=0).stop 2024-04-26T01:52:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.751+0000 7fdea7400640 1 -- 172.21.15.1:0/1401003764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdea0109490 msgr2=0x7fdea01a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.751+0000 7fdea7400640 1 --2- 172.21.15.1:0/1401003764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdea0109490 0x7fdea01a0bd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00e9c0 tx=0x7fde8c00ee80 comp rx=0 tx=0).stop 2024-04-26T01:52:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.751+0000 7fdea7400640 1 -- 172.21.15.1:0/1401003764 shutdown_connections 2024-04-26T01:52:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.751+0000 7fdea7400640 1 --2- 172.21.15.1:0/1401003764 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fde880a3f40 0x7fde880a6400 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.751+0000 7fdea7400640 1 --2- 172.21.15.1:0/1401003764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdea019a800 0x7fdea019acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.751+0000 7fdea7400640 1 --2- 172.21.15.1:0/1401003764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdea0109490 0x7fdea01a0bd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.751+0000 7fdea7400640 1 --2- 172.21.15.1:0/1401003764 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdea01043c0 0x7fdea01a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.751+0000 7fdea7400640 1 -- 172.21.15.1:0/1401003764 >> 172.21.15.1:0/1401003764 conn(0x7fdea00ffc20 msgr2=0x7fdea01010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.751+0000 7fdea7400640 1 -- 172.21.15.1:0/1401003764 shutdown_connections 2024-04-26T01:52:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:21.751+0000 7fdea7400640 1 -- 172.21.15.1:0/1401003764 wait complete. 2024-04-26T01:52:22.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:22 smithi001 ceph-mon[27822]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-26T01:52:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:22 smithi097 ceph-mon[32853]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-26T01:52:22.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:22 smithi173 ceph-mon[32553]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-26T01:52:23.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.508+0000 7f4449400700 1 -- 172.21.15.1:0/3001076449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f444406b0f0 msgr2=0x7f444410f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:23.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.508+0000 7f4449400700 1 --2- 172.21.15.1:0/3001076449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f444406b0f0 0x7f444410f900 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f443c00b0a0 tx=0x7f443c00b3b0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.508+0000 7f4449400700 1 -- 172.21.15.1:0/3001076449 shutdown_connections 2024-04-26T01:52:23.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.508+0000 7f4449400700 1 --2- 172.21.15.1:0/3001076449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f444406b0f0 0x7f444410f900 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.508+0000 7f4449400700 1 --2- 172.21.15.1:0/3001076449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4444108a30 0x7f4444108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.508+0000 7f4449400700 1 --2- 172.21.15.1:0/3001076449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f444406a750 0x7f444406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.508+0000 7f4449400700 1 -- 172.21.15.1:0/3001076449 >> 172.21.15.1:0/3001076449 conn(0x7f44440788f0 msgr2=0x7f4444078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:23.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.508+0000 7f4449400700 1 -- 172.21.15.1:0/3001076449 shutdown_connections 2024-04-26T01:52:23.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.508+0000 7f4449400700 1 -- 172.21.15.1:0/3001076449 wait complete. 2024-04-26T01:52:23.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.510+0000 7f4449400700 1 Processor -- start 2024-04-26T01:52:23.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.510+0000 7f4449400700 1 -- start start 2024-04-26T01:52:23.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.510+0000 7f4449400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f444406a750 0x7f444419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:23.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.510+0000 7f4449400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f444406b0f0 0x7f444419b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:23.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.510+0000 7f4449400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4444108a30 0x7f444419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:23.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.510+0000 7f4449400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4444112c10 con 0x7f444406a750 2024-04-26T01:52:23.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.510+0000 7f4449400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4444112a90 con 0x7f444406b0f0 2024-04-26T01:52:23.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.510+0000 7f4449400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4444112d90 con 0x7f4444108a30 2024-04-26T01:52:23.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.511+0000 7f4442800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f444406a750 0x7f444419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:23.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.511+0000 7f4442800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f444406a750 0x7f444419b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57580/0 (socket says 172.21.15.1:57580) 2024-04-26T01:52:23.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.511+0000 7f4442800700 1 -- 172.21.15.1:0/3632528943 learned_addr learned my addr 172.21.15.1:0/3632528943 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:23.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.511+0000 7f4441e00700 1 --2- 172.21.15.1:0/3632528943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f444406b0f0 0x7f444419b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:23.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.511+0000 7f4443200700 1 --2- 172.21.15.1:0/3632528943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4444108a30 0x7f444419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:23.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.511+0000 7f4441e00700 1 -- 172.21.15.1:0/3632528943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4444108a30 msgr2=0x7f444419fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:23.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.511+0000 7f4441e00700 1 --2- 172.21.15.1:0/3632528943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4444108a30 0x7f444419fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.511+0000 7f4441e00700 1 -- 172.21.15.1:0/3632528943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f444406a750 msgr2=0x7f444419b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:23.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.511+0000 7f4441e00700 1 --2- 172.21.15.1:0/3632528943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f444406a750 0x7f444419b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.511+0000 7f4441e00700 1 -- 172.21.15.1:0/3632528943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44441a02e0 con 0x7f444406b0f0 2024-04-26T01:52:23.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.512+0000 7f4441e00700 1 --2- 172.21.15.1:0/3632528943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f444406b0f0 0x7f444419b860 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f443000e9f0 tx=0x7f443000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:23.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.512+0000 7f443b400700 1 -- 172.21.15.1:0/3632528943 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443000c9c0 con 0x7f444406b0f0 2024-04-26T01:52:23.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.512+0000 7f4449400700 1 -- 172.21.15.1:0/3632528943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44441a05d0 con 0x7f444406b0f0 2024-04-26T01:52:23.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.512+0000 7f4449400700 1 -- 172.21.15.1:0/3632528943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44441a7e50 con 0x7f444406b0f0 2024-04-26T01:52:23.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.512+0000 7f443b400700 1 -- 172.21.15.1:0/3632528943 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f443000cb60 con 0x7f444406b0f0 2024-04-26T01:52:23.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.512+0000 7f443b400700 1 -- 172.21.15.1:0/3632528943 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430018680 con 0x7f444406b0f0 2024-04-26T01:52:23.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.516+0000 7f443b400700 1 -- 172.21.15.1:0/3632528943 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7f4430018820 con 0x7f444406b0f0 2024-04-26T01:52:23.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.518+0000 7f4453db0b80 1 Processor -- start 2024-04-26T01:52:23.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.518+0000 7f4453db0b80 1 -- start start 2024-04-26T01:52:23.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.518+0000 7f443b400700 1 --2- 172.21.15.1:0/3632528943 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f44280a3fe0 0x7f44280a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:23.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.518+0000 7f4453db0b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06c52f0 0x564ad06c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:23.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.518+0000 7f4453db0b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06c5c10 0x564ad06d0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:23.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.518+0000 7f4453db0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06d0ab0 0x564ad06d2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:23.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.518+0000 7f4453db0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564ad0618e40 con 0x564ad06d0ab0 2024-04-26T01:52:23.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.518+0000 7f4453db0b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564ad05bf6e0 con 0x564ad06c52f0 2024-04-26T01:52:23.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.518+0000 7f4453db0b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564ad0614e00 con 0x564ad06c5c10 2024-04-26T01:52:23.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.518+0000 7f443b400700 1 -- 172.21.15.1:0/3632528943 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f44300d0aa0 con 0x7f444406b0f0 2024-04-26T01:52:23.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.519+0000 7f4441e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06c5c10 0x564ad06d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:23.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.519+0000 7f4442800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06c52f0 0x564ad06c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:23.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.519+0000 7f4441e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06c5c10 0x564ad06d0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:52376/0 (socket says 172.21.15.1:52376) 2024-04-26T01:52:23.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.519+0000 7f4441e00700 1 -- 172.21.15.1:0/1146245391 learned_addr learned my addr 172.21.15.1:0/1146245391 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:23.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.519+0000 7f4443200700 1 --2- 172.21.15.1:0/1146245391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06d0ab0 0x564ad06d2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:23.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.519+0000 7f4441e00700 1 -- 172.21.15.1:0/1146245391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06c52f0 msgr2=0x564ad06c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:23.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.519+0000 7f4441e00700 1 --2- 172.21.15.1:0/1146245391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06c52f0 0x564ad06c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.519+0000 7f4441e00700 1 -- 172.21.15.1:0/1146245391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06d0ab0 msgr2=0x564ad06d2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:23.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.519+0000 7f4441e00700 1 --2- 172.21.15.1:0/1146245391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06d0ab0 0x564ad06d2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.519+0000 7f4441e00700 1 -- 172.21.15.1:0/1146245391 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ad05b5630 con 0x564ad06c5c10 2024-04-26T01:52:23.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.520+0000 7f4443200700 1 --2- 172.21.15.1:0/1146245391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06d0ab0 0x564ad06d2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:52:23.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.520+0000 7f4443200700 1 --2- 172.21.15.1:0/3632528943 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f44280a3fe0 0x7f44280a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:23.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.520+0000 7f4441e00700 1 --2- 172.21.15.1:0/1146245391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06c5c10 0x564ad06d0570 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f443000c5b0 tx=0x7f443009cf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed1bc0da3db9125d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:23.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.520+0000 7f442fe00700 1 -- 172.21.15.1:0/1146245391 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44300a5470 con 0x564ad06c5c10 2024-04-26T01:52:23.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.520+0000 7f442fe00700 1 -- 172.21.15.1:0/1146245391 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f443009ab00 con 0x564ad06c5c10 2024-04-26T01:52:23.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.520+0000 7f442fe00700 1 -- 172.21.15.1:0/1146245391 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44300a5240 con 0x564ad06c5c10 2024-04-26T01:52:23.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.521+0000 7f4453db0b80 1 -- 172.21.15.1:0/1146245391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06c5c10 msgr2=0x564ad06d0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:23.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.521+0000 7f4443200700 1 --2- 172.21.15.1:0/3632528943 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f44280a3fe0 0x7f44280a64a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f443c00b780 tx=0x7f443c006210 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:23.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.521+0000 7f4453db0b80 1 --2- 172.21.15.1:0/1146245391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06c5c10 0x564ad06d0570 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f443000c5b0 tx=0x7f443009cf00 comp rx=0 tx=0).stop 2024-04-26T01:52:23.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.521+0000 7f4453db0b80 1 -- 172.21.15.1:0/1146245391 shutdown_connections 2024-04-26T01:52:23.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.521+0000 7f4453db0b80 1 --2- 172.21.15.1:0/1146245391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06d0ab0 0x564ad06d2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.521+0000 7f4453db0b80 1 --2- 172.21.15.1:0/1146245391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06c52f0 0x564ad06c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.521+0000 7f4453db0b80 1 --2- 172.21.15.1:0/1146245391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06c5c10 0x564ad06d0570 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.521+0000 7f4453db0b80 1 -- 172.21.15.1:0/1146245391 >> 172.21.15.1:0/1146245391 conn(0x564ad05f4550 msgr2=0x564ad06c1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:23.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.521+0000 7f4453db0b80 1 -- 172.21.15.1:0/1146245391 shutdown_connections 2024-04-26T01:52:23.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.522+0000 7f4453db0b80 1 -- 172.21.15.1:0/1146245391 wait complete. 2024-04-26T01:52:23.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.522+0000 7f4453db0b80 1 -- start start 2024-04-26T01:52:23.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.523+0000 7f4453db0b80 1 Processor -- start 2024-04-26T01:52:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.523+0000 7f4453db0b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06c52f0 0x564ad06d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.523+0000 7f4453db0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06c5c10 0x564ad06cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.523+0000 7f4453db0b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06d0ab0 0x564ad06cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.523+0000 7f4453db0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564ad06dbdb0 con 0x564ad06c5c10 2024-04-26T01:52:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.523+0000 7f4453db0b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564ad05d2810 con 0x564ad06d0ab0 2024-04-26T01:52:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.523+0000 7f4453db0b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564ad06dbf30 con 0x564ad06c52f0 2024-04-26T01:52:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.523+0000 7f4453db0b80 10 client.0 _ll_register_callbacks cb 0x564ad05b16d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.524+0000 7f4443200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06d0ab0 0x564ad06cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.524+0000 7f4443200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06d0ab0 0x564ad06cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:40714/0 (socket says 172.21.15.1:40714) 2024-04-26T01:52:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.524+0000 7f4443200700 1 -- 172.21.15.1:0/3012172947 learned_addr learned my addr 172.21.15.1:0/3012172947 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.524+0000 7f4442800700 1 --2- 172.21.15.1:0/3012172947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06c52f0 0x564ad06d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.524+0000 7f4441e00700 1 --2- 172.21.15.1:0/3012172947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06c5c10 0x564ad06cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.524+0000 7f4443200700 1 -- 172.21.15.1:0/3012172947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06c52f0 msgr2=0x564ad06d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.524+0000 7f4443200700 1 --2- 172.21.15.1:0/3012172947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06c52f0 0x564ad06d0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.524+0000 7f4443200700 1 -- 172.21.15.1:0/3012172947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06c5c10 msgr2=0x564ad06cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.524+0000 7f4443200700 1 --2- 172.21.15.1:0/3012172947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06c5c10 0x564ad06cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.524+0000 7f4443200700 1 -- 172.21.15.1:0/3012172947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ad0566880 con 0x564ad06d0ab0 2024-04-26T01:52:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.525+0000 7f4443200700 1 --2- 172.21.15.1:0/3012172947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06d0ab0 0x564ad06cdab0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f443c015000 tx=0x7f443c00b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=141ed3bf97448a25 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:23.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.525+0000 7f442cc00700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:52:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.525+0000 7f442cc00700 1 -- 172.21.15.1:0/3012172947 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c02c030 con 0x564ad06d0ab0 2024-04-26T01:52:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.525+0000 7f442cc00700 1 -- 172.21.15.1:0/3012172947 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f443c007930 con 0x564ad06d0ab0 2024-04-26T01:52:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.525+0000 7f4453db0b80 1 -- 172.21.15.1:0/3012172947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f443009ab00 con 0x564ad06d0ab0 2024-04-26T01:52:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.526+0000 7f442cc00700 1 -- 172.21.15.1:0/3012172947 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f443c007b50 con 0x564ad06d0ab0 2024-04-26T01:52:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.526+0000 7f4453db0b80 10 client.25310 fetch_fsmap learned FSMap version 26 2024-04-26T01:52:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.526+0000 7f4453db0b80 1 -- 172.21.15.1:0/3012172947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564ad05afaa0 con 0x564ad06d0ab0 2024-04-26T01:52:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.526+0000 7f442cc00700 1 -- 172.21.15.1:0/3012172947 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f443009ab00 con 0x564ad06d0ab0 2024-04-26T01:52:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.526+0000 7f442cc00700 1 -- 172.21.15.1:0/3012172947 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f443c025070 con 0x564ad06d0ab0 2024-04-26T01:52:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.527+0000 7f4453db0b80 10 client.25310 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:52:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.527+0000 7f4453db0b80 10 client.25310 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T01:52:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.527+0000 7f4453db0b80 20 client.25310 populate_metadata read hostname 'smithi001' 2024-04-26T01:52:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.527+0000 7f4453db0b80 1 --2- 172.21.15.1:0/3012172947 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ad06e8b90 0x564ad06ea000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:23.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.527+0000 7f4453db0b80 4 client.25310 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:52:23.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.527+0000 7f4453db0b80 1 -- 172.21.15.1:0/3012172947 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ad0626630 con 0x564ad06e8b90 2024-04-26T01:52:23.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.527+0000 7f4442800700 1 --2- 172.21.15.1:0/3012172947 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ad06e8b90 0x564ad06ea000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:23.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.528+0000 7f4442800700 1 --2- 172.21.15.1:0/3012172947 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ad06e8b90 0x564ad06ea000 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c25d58abf369c2d server_cookie=6e34937642a9bb65 in_seq=0 out_seq=0 2024-04-26T01:52:23.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.528+0000 7f442cc00700 10 client.25310 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:23.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.529+0000 7f442cc00700 1 -- 172.21.15.1:0/3012172947 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ad0626630 con 0x564ad06e8b90 2024-04-26T01:52:23.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.529+0000 7f442cc00700 10 client.25310 handle_command_reply: tid=0 2024-04-26T01:52:23.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.529+0000 7f4453db0b80 1 client.25310 shutdown 2024-04-26T01:52:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.529+0000 7f4453db0b80 1 -- 172.21.15.1:0/3012172947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06d0ab0 msgr2=0x564ad06cdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.529+0000 7f4453db0b80 1 --2- 172.21.15.1:0/3012172947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06d0ab0 0x564ad06cdab0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f443c015000 tx=0x7f443c00b3b0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.529+0000 7f4453db0b80 1 -- 172.21.15.1:0/3012172947 shutdown_connections 2024-04-26T01:52:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.529+0000 7f4453db0b80 1 --2- 172.21.15.1:0/3012172947 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ad06e8b90 0x564ad06ea000 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.529+0000 7f4453db0b80 1 --2- 172.21.15.1:0/3012172947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06d0ab0 0x564ad06cdab0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.529+0000 7f4453db0b80 1 --2- 172.21.15.1:0/3012172947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06c52f0 0x564ad06d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.529+0000 7f4453db0b80 1 --2- 172.21.15.1:0/3012172947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06c5c10 0x564ad06cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.529+0000 7f442cc00700 0 client.25310 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.529+0000 7f4453db0b80 1 -- 172.21.15.1:0/3012172947 >> 172.21.15.1:0/3012172947 conn(0x564ad05f4550 msgr2=0x564ad06c0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.530+0000 7f4453db0b80 1 -- 172.21.15.1:0/3012172947 shutdown_connections 2024-04-26T01:52:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.530+0000 7f4453db0b80 1 -- 172.21.15.1:0/3012172947 wait complete. 2024-04-26T01:52:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.530+0000 7f4453db0b80 20 client.25310 trim_cache size 0 max 16384 2024-04-26T01:52:23.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.552+0000 7f4453db0b80 1 Processor -- start 2024-04-26T01:52:23.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.553+0000 7f4453db0b80 1 -- start start 2024-04-26T01:52:23.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.553+0000 7f4453db0b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06c52f0 0x564ad06c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:23.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.553+0000 7f4453db0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06c5c10 0x564ad06f3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:23.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.553+0000 7f4453db0b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06d0ab0 0x564ad06f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:23.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.553+0000 7f4453db0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564ad06e8340 con 0x564ad06c5c10 2024-04-26T01:52:23.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.553+0000 7f4453db0b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564ad06e8a10 con 0x564ad06c52f0 2024-04-26T01:52:23.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.553+0000 7f4453db0b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564ad06e90e0 con 0x564ad06d0ab0 2024-04-26T01:52:23.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.553+0000 7f4442800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06c52f0 0x564ad06c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:23.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.553+0000 7f4443200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06d0ab0 0x564ad06f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:23.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.554+0000 7f4442800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06c52f0 0x564ad06c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:40720/0 (socket says 172.21.15.1:40720) 2024-04-26T01:52:23.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.554+0000 7f4442800700 1 -- 172.21.15.1:0/172346660 learned_addr learned my addr 172.21.15.1:0/172346660 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:23.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.554+0000 7f4443200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06d0ab0 0x564ad06f39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:52388/0 (socket says 172.21.15.1:52388) 2024-04-26T01:52:23.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.554+0000 7f4441e00700 1 --2- 172.21.15.1:0/172346660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06c5c10 0x564ad06f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:23.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.554+0000 7f4442800700 1 -- 172.21.15.1:0/172346660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06d0ab0 msgr2=0x564ad06f39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:23.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.554+0000 7f4442800700 1 --2- 172.21.15.1:0/172346660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06d0ab0 0x564ad06f39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.554+0000 7f4442800700 1 -- 172.21.15.1:0/172346660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06c5c10 msgr2=0x564ad06f3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:23.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.554+0000 7f4442800700 1 --2- 172.21.15.1:0/172346660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06c5c10 0x564ad06f3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.554+0000 7f4442800700 1 -- 172.21.15.1:0/172346660 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ad06d31e0 con 0x564ad06c52f0 2024-04-26T01:52:23.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.554+0000 7f4441e00700 1 --2- 172.21.15.1:0/172346660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06c5c10 0x564ad06f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:52:23.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.554+0000 7f4442800700 1 --2- 172.21.15.1:0/172346660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06c52f0 0x564ad06c8a20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4434005bf0 tx=0x7f4434009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=489faa21323761a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:23.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.555+0000 7f442f400700 1 -- 172.21.15.1:0/172346660 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434004a30 con 0x564ad06c52f0 2024-04-26T01:52:23.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.555+0000 7f442f400700 1 -- 172.21.15.1:0/172346660 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4434022030 con 0x564ad06c52f0 2024-04-26T01:52:23.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.555+0000 7f442f400700 1 -- 172.21.15.1:0/172346660 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434003970 con 0x564ad06c52f0 2024-04-26T01:52:23.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.556+0000 7f4453db0b80 1 -- 172.21.15.1:0/172346660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06c52f0 msgr2=0x564ad06c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:23.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.556+0000 7f4453db0b80 1 --2- 172.21.15.1:0/172346660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06c52f0 0x564ad06c8a20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4434005bf0 tx=0x7f4434009260 comp rx=0 tx=0).stop 2024-04-26T01:52:23.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.556+0000 7f4453db0b80 1 -- 172.21.15.1:0/172346660 shutdown_connections 2024-04-26T01:52:23.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.556+0000 7f4453db0b80 1 --2- 172.21.15.1:0/172346660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06d0ab0 0x564ad06f39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.556+0000 7f4453db0b80 1 --2- 172.21.15.1:0/172346660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06c52f0 0x564ad06c8a20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.556+0000 7f4453db0b80 1 --2- 172.21.15.1:0/172346660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06c5c10 0x564ad06f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.556+0000 7f4453db0b80 1 -- 172.21.15.1:0/172346660 >> 172.21.15.1:0/172346660 conn(0x564ad05f4550 msgr2=0x564ad06c8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:23.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.556+0000 7f4453db0b80 1 -- 172.21.15.1:0/172346660 shutdown_connections 2024-04-26T01:52:23.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.556+0000 7f4453db0b80 1 -- 172.21.15.1:0/172346660 wait complete. 2024-04-26T01:52:23.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.556+0000 7f4453db0b80 1 -- start start 2024-04-26T01:52:23.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.557+0000 7f4453db0b80 1 Processor -- start 2024-04-26T01:52:23.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.557+0000 7f4453db0b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06c52f0 0x564ad06edf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:23.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.557+0000 7f4453db0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06c5c10 0x564ad06ee450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:23.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.557+0000 7f4453db0b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06d0ab0 0x564ad06ee990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:23.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.557+0000 7f4453db0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564ad06e9e80 con 0x564ad06c5c10 2024-04-26T01:52:23.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.557+0000 7f4453db0b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564ad06e97b0 con 0x564ad06c52f0 2024-04-26T01:52:23.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.557+0000 7f4453db0b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564ad06ea550 con 0x564ad06d0ab0 2024-04-26T01:52:23.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.557+0000 7f4453db0b80 10 client.0 _ll_register_callbacks cb 0x564ad05b16d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:23.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.557+0000 7f4441e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06c5c10 0x564ad06ee450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:23.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.558+0000 7f4441e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06c5c10 0x564ad06ee450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57618/0 (socket says 172.21.15.1:57618) 2024-04-26T01:52:23.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.558+0000 7f4441e00700 1 -- 172.21.15.1:0/3871768113 learned_addr learned my addr 172.21.15.1:0/3871768113 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:23.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.558+0000 7f4442800700 1 --2- 172.21.15.1:0/3871768113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06c52f0 0x564ad06edf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:23.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.558+0000 7f4443200700 1 --2- 172.21.15.1:0/3871768113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06d0ab0 0x564ad06ee990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:23.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.558+0000 7f4441e00700 1 -- 172.21.15.1:0/3871768113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06d0ab0 msgr2=0x564ad06ee990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:23.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.558+0000 7f4441e00700 1 --2- 172.21.15.1:0/3871768113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06d0ab0 0x564ad06ee990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.558+0000 7f4441e00700 1 -- 172.21.15.1:0/3871768113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06c52f0 msgr2=0x564ad06edf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:23.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.558+0000 7f4441e00700 1 --2- 172.21.15.1:0/3871768113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06c52f0 0x564ad06edf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.558+0000 7f4441e00700 1 -- 172.21.15.1:0/3871768113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ad074fe30 con 0x564ad06c5c10 2024-04-26T01:52:23.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.558+0000 7f4442800700 1 --2- 172.21.15.1:0/3871768113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06c52f0 0x564ad06edf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:23.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.558+0000 7f4441e00700 1 --2- 172.21.15.1:0/3871768113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06c5c10 0x564ad06ee450 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f443000e9c0 tx=0x7f4430009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14c65f55848320 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:23.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.558+0000 7f442cc00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:52:23.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.559+0000 7f442cc00700 1 -- 172.21.15.1:0/3871768113 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44300a5470 con 0x564ad06c5c10 2024-04-26T01:52:23.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.559+0000 7f442cc00700 1 -- 172.21.15.1:0/3871768113 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f44300a4fe0 con 0x564ad06c5c10 2024-04-26T01:52:23.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.559+0000 7f4453db0b80 1 -- 172.21.15.1:0/3871768113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4434022030 con 0x564ad06c5c10 2024-04-26T01:52:23.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.559+0000 7f442cc00700 1 -- 172.21.15.1:0/3871768113 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f44300a51e0 con 0x564ad06c5c10 2024-04-26T01:52:23.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.559+0000 7f4453db0b80 10 client.15192 fetch_fsmap learned FSMap version 26 2024-04-26T01:52:23.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.559+0000 7f4453db0b80 1 -- 172.21.15.1:0/3871768113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564ad0751130 con 0x564ad06c5c10 2024-04-26T01:52:23.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.560+0000 7f442cc00700 1 -- 172.21.15.1:0/3871768113 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4434022030 con 0x564ad06c5c10 2024-04-26T01:52:23.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.560+0000 7f442cc00700 1 -- 172.21.15.1:0/3871768113 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4430098550 con 0x564ad06c5c10 2024-04-26T01:52:23.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.560+0000 7f4453db0b80 10 client.15192 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:52:23.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.560+0000 7f4453db0b80 10 client.15192 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T01:52:23.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.560+0000 7f4453db0b80 20 client.15192 populate_metadata read hostname 'smithi001' 2024-04-26T01:52:23.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.560+0000 7f4453db0b80 1 --2- 172.21.15.1:0/3871768113 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ad0755890 0x564ad07545a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:23.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.560+0000 7f4453db0b80 4 client.15192 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T01:52:23.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.560+0000 7f4453db0b80 1 -- 172.21.15.1:0/3871768113 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "status"}) -- 0x564ad06206d0 con 0x564ad0755890 2024-04-26T01:52:23.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.560+0000 7f4442800700 1 --2- 172.21.15.1:0/3871768113 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ad0755890 0x564ad07545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:23.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.561+0000 7f442cc00700 10 client.15192 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:23.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.561+0000 7f4442800700 1 --2- 172.21.15.1:0/3871768113 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ad0755890 0x564ad07545a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dfc37b5bb6a9d05 server_cookie=2c0f09ba6e53efe7 in_seq=0 out_seq=0 2024-04-26T01:52:23.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.562+0000 7f442cc00700 1 -- 172.21.15.1:0/3871768113 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564ad06206d0 con 0x564ad0755890 2024-04-26T01:52:23.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.562+0000 7f442cc00700 10 client.15192 handle_command_reply: tid=0 2024-04-26T01:52:23.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.562+0000 7f4453db0b80 1 client.15192 shutdown 2024-04-26T01:52:23.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.562+0000 7f4453db0b80 1 -- 172.21.15.1:0/3871768113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06c5c10 msgr2=0x564ad06ee450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:23.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.562+0000 7f4453db0b80 1 --2- 172.21.15.1:0/3871768113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06c5c10 0x564ad06ee450 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f443000e9c0 tx=0x7f4430009790 comp rx=0 tx=0).stop 2024-04-26T01:52:23.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.562+0000 7f4453db0b80 1 -- 172.21.15.1:0/3871768113 shutdown_connections 2024-04-26T01:52:23.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.562+0000 7f4453db0b80 1 --2- 172.21.15.1:0/3871768113 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ad0755890 0x564ad07545a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.562+0000 7f4453db0b80 1 --2- 172.21.15.1:0/3871768113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ad06d0ab0 0x564ad06ee990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.562+0000 7f4453db0b80 1 --2- 172.21.15.1:0/3871768113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ad06c52f0 0x564ad06edf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.562+0000 7f4453db0b80 1 --2- 172.21.15.1:0/3871768113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ad06c5c10 0x564ad06ee450 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.562+0000 7f4453db0b80 1 -- 172.21.15.1:0/3871768113 >> 172.21.15.1:0/3871768113 conn(0x564ad05f4550 msgr2=0x564ad06cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:23.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.562+0000 7f442cc00700 0 client.15192 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:23.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.562+0000 7f4453db0b80 1 -- 172.21.15.1:0/3871768113 shutdown_connections 2024-04-26T01:52:23.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.562+0000 7f4453db0b80 1 -- 172.21.15.1:0/3871768113 wait complete. 2024-04-26T01:52:23.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.563+0000 7f4453db0b80 20 client.15192 trim_cache size 0 max 16384 2024-04-26T01:52:23.565 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T01:52:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "77cfcc80-036d-11ef-bc93-c7b262605968", 2024-04-26T01:52:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-04-26T01:52:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24518, 2024-04-26T01:52:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-04-26T01:52:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-04-26T01:52:23.566 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-04-26T01:52:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 293.443819281, 2024-04-26T01:52:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 26, 2024-04-26T01:52:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 93, 2024-04-26T01:52:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 92, 2024-04-26T01:52:23.567 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 313.93145553199997 2024-04-26T01:52:23.567 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T01:52:23.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.563+0000 7f4449400700 1 -- 172.21.15.1:0/3632528943 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f44280a3fe0 msgr2=0x7f44280a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:23.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.563+0000 7f4449400700 1 --2- 172.21.15.1:0/3632528943 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f44280a3fe0 0x7f44280a64a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f443c00b780 tx=0x7f443c006210 comp rx=0 tx=0).stop 2024-04-26T01:52:23.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.563+0000 7f4449400700 1 -- 172.21.15.1:0/3632528943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f444406b0f0 msgr2=0x7f444419b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:23.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.563+0000 7f4449400700 1 --2- 172.21.15.1:0/3632528943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f444406b0f0 0x7f444419b860 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f443000e9f0 tx=0x7f443000ed00 comp rx=0 tx=0).stop 2024-04-26T01:52:23.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.564+0000 7f4449400700 1 -- 172.21.15.1:0/3632528943 shutdown_connections 2024-04-26T01:52:23.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.564+0000 7f4449400700 1 --2- 172.21.15.1:0/3632528943 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f44280a3fe0 0x7f44280a64a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.564+0000 7f4449400700 1 --2- 172.21.15.1:0/3632528943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4444108a30 0x7f444419fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.564+0000 7f4449400700 1 --2- 172.21.15.1:0/3632528943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f444406a750 0x7f444419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.564+0000 7f4449400700 1 --2- 172.21.15.1:0/3632528943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f444406b0f0 0x7f444419b860 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:23.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.564+0000 7f4449400700 1 -- 172.21.15.1:0/3632528943 >> 172.21.15.1:0/3632528943 conn(0x7f44440788f0 msgr2=0x7f444410df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:23.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.564+0000 7f4449400700 1 -- 172.21.15.1:0/3632528943 shutdown_connections 2024-04-26T01:52:23.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:23.564+0000 7f4449400700 1 -- 172.21.15.1:0/3632528943 wait complete. 2024-04-26T01:52:24.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:24 smithi001 ceph-mon[27822]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T01:52:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:24 smithi097 ceph-mon[32853]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T01:52:24.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:24 smithi173 ceph-mon[32553]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T01:52:24.647 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-26T01:52:26.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:26 smithi097 ceph-mon[32853]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T01:52:26.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:26 smithi173 ceph-mon[32553]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T01:52:26.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:26 smithi001 ceph-mon[27822]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T01:52:28.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:28 smithi097 ceph-mon[32853]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-26T01:52:28.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:28 smithi173 ceph-mon[32553]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-26T01:52:28.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:28 smithi001 ceph-mon[27822]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-26T01:52:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:30 smithi097 ceph-mon[32853]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T01:52:30.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:30 smithi173 ceph-mon[32553]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T01:52:30.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:30 smithi001 ceph-mon[27822]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T01:52:32.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:32 smithi097 ceph-mon[32853]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-26T01:52:32.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:32 smithi173 ceph-mon[32553]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-26T01:52:32.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:32 smithi001 ceph-mon[27822]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-26T01:52:34.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:34 smithi097 ceph-mon[32853]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T01:52:34.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:34 smithi173 ceph-mon[32553]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T01:52:34.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:34 smithi001 ceph-mon[27822]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T01:52:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.900+0000 7f9575000640 1 -- 172.21.15.1:0/2009979461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9570101640 msgr2=0x7f9570105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.900+0000 7f9575000640 1 --2- 172.21.15.1:0/2009979461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9570101640 0x7f9570105b00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9558009900 tx=0x7f955802a480 comp rx=0 tx=0).stop 2024-04-26T01:52:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.900+0000 7f9575000640 1 -- 172.21.15.1:0/2009979461 shutdown_connections 2024-04-26T01:52:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.900+0000 7f9575000640 1 --2- 172.21.15.1:0/2009979461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9570106120 0x7f95701065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.900+0000 7f9575000640 1 --2- 172.21.15.1:0/2009979461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9570101640 0x7f9570105b00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.900+0000 7f9575000640 1 --2- 172.21.15.1:0/2009979461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9570100d20 0x7f9570101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.900+0000 7f9575000640 1 -- 172.21.15.1:0/2009979461 >> 172.21.15.1:0/2009979461 conn(0x7f95700fc730 msgr2=0x7f95700feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.902+0000 7f9575000640 1 -- 172.21.15.1:0/2009979461 shutdown_connections 2024-04-26T01:52:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.902+0000 7f9575000640 1 -- 172.21.15.1:0/2009979461 wait complete. 2024-04-26T01:52:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.903+0000 7f9575000640 1 Processor -- start 2024-04-26T01:52:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.903+0000 7f9575000640 1 -- start start 2024-04-26T01:52:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.904+0000 7f9575000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9570100d20 0x7f95701a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.904+0000 7f9575000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9570101640 0x7f95701a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.904+0000 7f9575000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9570106120 0x7f957019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.904+0000 7f9575000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f95701185e0 con 0x7f9570106120 2024-04-26T01:52:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.904+0000 7f9575000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9570118460 con 0x7f9570101640 2024-04-26T01:52:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.904+0000 7f9575000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9570118760 con 0x7f9570100d20 2024-04-26T01:52:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.904+0000 7f956f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9570100d20 0x7f95701a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.904+0000 7f9567e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9570101640 0x7f95701a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.904+0000 7f9567e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9570101640 0x7f95701a4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:54416/0 (socket says 172.21.15.1:54416) 2024-04-26T01:52:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.904+0000 7f9567e00640 1 -- 172.21.15.1:0/3555867231 learned_addr learned my addr 172.21.15.1:0/3555867231 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.905+0000 7f956fe00640 1 --2- 172.21.15.1:0/3555867231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9570106120 0x7f957019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.905+0000 7f9567e00640 1 -- 172.21.15.1:0/3555867231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9570100d20 msgr2=0x7f95701a47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.905+0000 7f9567e00640 1 --2- 172.21.15.1:0/3555867231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9570100d20 0x7f95701a47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.905+0000 7f9567e00640 1 -- 172.21.15.1:0/3555867231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9570106120 msgr2=0x7f957019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.905+0000 7f9567e00640 1 --2- 172.21.15.1:0/3555867231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9570106120 0x7f957019e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.905+0000 7f9567e00640 1 -- 172.21.15.1:0/3555867231 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957019f0e0 con 0x7f9570101640 2024-04-26T01:52:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.905+0000 7f956fe00640 1 --2- 172.21.15.1:0/3555867231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9570106120 0x7f957019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:52:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.906+0000 7f9567e00640 1 --2- 172.21.15.1:0/3555867231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9570101640 0x7f95701a4cf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9558005ab0 tx=0x7f9558032c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.906+0000 7f956cc00640 1 -- 172.21.15.1:0/3555867231 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955803f030 con 0x7f9570101640 2024-04-26T01:52:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.906+0000 7f956cc00640 1 -- 172.21.15.1:0/3555867231 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9558003aa0 con 0x7f9570101640 2024-04-26T01:52:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.906+0000 7f956cc00640 1 -- 172.21.15.1:0/3555867231 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558035070 con 0x7f9570101640 2024-04-26T01:52:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.906+0000 7f9575000640 1 -- 172.21.15.1:0/3555867231 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957019f370 con 0x7f9570101640 2024-04-26T01:52:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.906+0000 7f9575000640 1 -- 172.21.15.1:0/3555867231 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95701ab670 con 0x7f9570101640 2024-04-26T01:52:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.910+0000 7f956cc00640 1 -- 172.21.15.1:0/3555867231 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7f9558042030 con 0x7f9570101640 2024-04-26T01:52:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.911+0000 7f9579416740 1 Processor -- start 2024-04-26T01:52:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.912+0000 7f9579416740 1 -- start start 2024-04-26T01:52:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.912+0000 7f9579416740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563fd42509b0 0x563fd4250d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.912+0000 7f9579416740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd42512d0 0x563fd425b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.912+0000 7f9579416740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd425bb90 0x563fd425df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.912+0000 7f9579416740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563fd41b6900 con 0x563fd42509b0 2024-04-26T01:52:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.912+0000 7f9579416740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563fd415cef0 con 0x563fd42512d0 2024-04-26T01:52:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.912+0000 7f9579416740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563fd41b28c0 con 0x563fd425bb90 2024-04-26T01:52:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.912+0000 7f956cc00640 1 --2- 172.21.15.1:0/3555867231 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f95500a3fb0 0x7f95500a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.913+0000 7f956cc00640 1 -- 172.21.15.1:0/3555867231 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f95580f0e60 con 0x7f9570101640 2024-04-26T01:52:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.913+0000 7f9567e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd425bb90 0x563fd425df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.913+0000 7f9567e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd425bb90 0x563fd425df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:43460/0 (socket says 172.21.15.1:43460) 2024-04-26T01:52:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.913+0000 7f9567e00640 1 -- 172.21.15.1:0/4162385790 learned_addr learned my addr 172.21.15.1:0/4162385790 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.913+0000 7f9567e00640 1 -- 172.21.15.1:0/4162385790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd42512d0 msgr2=0x563fd425b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.913+0000 7f9567e00640 1 --2- 172.21.15.1:0/4162385790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd42512d0 0x563fd425b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.913+0000 7f9567e00640 1 -- 172.21.15.1:0/4162385790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563fd42509b0 msgr2=0x563fd4250d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.913+0000 7f9567e00640 1 --2- 172.21.15.1:0/4162385790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563fd42509b0 0x563fd4250d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.913+0000 7f956fe00640 1 --2- 172.21.15.1:0/3555867231 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f95500a3fb0 0x7f95500a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.913+0000 7f9567e00640 1 -- 172.21.15.1:0/4162385790 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fd3ec9210 con 0x563fd425bb90 2024-04-26T01:52:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.914+0000 7f9567e00640 1 --2- 172.21.15.1:0/4162385790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd425bb90 0x563fd425df80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f95580b9020 tx=0x7f95580bc7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2aff4fe993b24566 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.914+0000 7f9564c00640 1 -- 172.21.15.1:0/4162385790 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95580bcbc0 con 0x563fd425bb90 2024-04-26T01:52:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.914+0000 7f9564c00640 1 -- 172.21.15.1:0/4162385790 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95580c0f70 con 0x563fd425bb90 2024-04-26T01:52:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.914+0000 7f9564c00640 1 -- 172.21.15.1:0/4162385790 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95580e4d00 con 0x563fd425bb90 2024-04-26T01:52:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.915+0000 7f956fe00640 1 --2- 172.21.15.1:0/3555867231 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f95500a3fb0 0x7f95500a6470 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9568005c40 tx=0x7f9568009540 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.915+0000 7f9579416740 1 -- 172.21.15.1:0/4162385790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd425bb90 msgr2=0x563fd425df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.915+0000 7f9579416740 1 --2- 172.21.15.1:0/4162385790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd425bb90 0x563fd425df80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f95580b9020 tx=0x7f95580bc7c0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.916+0000 7f9579416740 1 -- 172.21.15.1:0/4162385790 shutdown_connections 2024-04-26T01:52:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.916+0000 7f9579416740 1 --2- 172.21.15.1:0/4162385790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd425bb90 0x563fd425df80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.916+0000 7f9579416740 1 --2- 172.21.15.1:0/4162385790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd42512d0 0x563fd425b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.916+0000 7f9579416740 1 --2- 172.21.15.1:0/4162385790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563fd42509b0 0x563fd4250d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.916+0000 7f9579416740 1 -- 172.21.15.1:0/4162385790 >> 172.21.15.1:0/4162385790 conn(0x563fd424cec0 msgr2=0x563fd424d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.916+0000 7f9579416740 1 -- 172.21.15.1:0/4162385790 shutdown_connections 2024-04-26T01:52:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.916+0000 7f9579416740 1 -- 172.21.15.1:0/4162385790 wait complete. 2024-04-26T01:52:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.917+0000 7f9579416740 1 -- start start 2024-04-26T01:52:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.917+0000 7f9579416740 1 Processor -- start 2024-04-26T01:52:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.918+0000 7f9579416740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd42509b0 0x563fd4258b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.918+0000 7f9579416740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563fd42512d0 0x563fd4259050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.918+0000 7f9579416740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd425bb90 0x563fd4259590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.918+0000 7f9579416740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563fd425e580 con 0x563fd42512d0 2024-04-26T01:52:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.918+0000 7f9579416740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563fd4170310 con 0x563fd42509b0 2024-04-26T01:52:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.918+0000 7f9579416740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563fd425e700 con 0x563fd425bb90 2024-04-26T01:52:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.918+0000 7f9579416740 10 client.0 _ll_register_callbacks cb 0x563fd413ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.918+0000 7f956f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd42509b0 0x563fd4258b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.918+0000 7f956f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd42509b0 0x563fd4258b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:54432/0 (socket says 172.21.15.1:54432) 2024-04-26T01:52:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.918+0000 7f956f400640 1 -- 172.21.15.1:0/3646144483 learned_addr learned my addr 172.21.15.1:0/3646144483 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.918+0000 7f956fe00640 1 --2- 172.21.15.1:0/3646144483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd425bb90 0x563fd4259590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.919+0000 7f956f400640 1 -- 172.21.15.1:0/3646144483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd425bb90 msgr2=0x563fd4259590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.919+0000 7f956f400640 1 --2- 172.21.15.1:0/3646144483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd425bb90 0x563fd4259590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.919+0000 7f956f400640 1 -- 172.21.15.1:0/3646144483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563fd42512d0 msgr2=0x563fd4259050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.919+0000 7f9567e00640 1 --2- 172.21.15.1:0/3646144483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563fd42512d0 0x563fd4259050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.919+0000 7f956f400640 1 --2- 172.21.15.1:0/3646144483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563fd42512d0 0x563fd4259050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.919+0000 7f956f400640 1 -- 172.21.15.1:0/3646144483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fd3f95620 con 0x563fd42509b0 2024-04-26T01:52:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.919+0000 7f9567e00640 1 --2- 172.21.15.1:0/3646144483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563fd42512d0 0x563fd4259050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:52:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.919+0000 7f956f400640 1 --2- 172.21.15.1:0/3646144483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd42509b0 0x563fd4258b10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9560011660 tx=0x7f9560011b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53f382a1e03e058a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.919+0000 7f955d600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:52:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.919+0000 7f955d600640 1 -- 172.21.15.1:0/3646144483 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560004250 con 0x563fd42509b0 2024-04-26T01:52:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.920+0000 7f955d600640 1 -- 172.21.15.1:0/3646144483 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f95600043f0 con 0x563fd42509b0 2024-04-26T01:52:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.920+0000 7f9579416740 1 -- 172.21.15.1:0/3646144483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95580c0f70 con 0x563fd42509b0 2024-04-26T01:52:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.920+0000 7f955d600640 1 -- 172.21.15.1:0/3646144483 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9560011e60 con 0x563fd42509b0 2024-04-26T01:52:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.920+0000 7f9579416740 10 client.25325 fetch_fsmap learned FSMap version 26 2024-04-26T01:52:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.920+0000 7f9579416740 1 -- 172.21.15.1:0/3646144483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563fd426a750 con 0x563fd42509b0 2024-04-26T01:52:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.921+0000 7f955d600640 1 -- 172.21.15.1:0/3646144483 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95580c0f70 con 0x563fd42509b0 2024-04-26T01:52:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.921+0000 7f955d600640 1 -- 172.21.15.1:0/3646144483 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9560003ae0 con 0x563fd42509b0 2024-04-26T01:52:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.921+0000 7f9579416740 10 client.25325 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:52:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.921+0000 7f9579416740 10 client.25325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:52:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.921+0000 7f9579416740 20 client.25325 populate_metadata read hostname 'smithi001' 2024-04-26T01:52:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.921+0000 7f9579416740 1 --2- 172.21.15.1:0/3646144483 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563fd426bbc0 0x563fd426dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.921+0000 7f9579416740 4 client.25325 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:52:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.921+0000 7f9579416740 1 -- 172.21.15.1:0/3646144483 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fd41ca8b0 con 0x563fd426bbc0 2024-04-26T01:52:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.922+0000 7f9567e00640 1 --2- 172.21.15.1:0/3646144483 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563fd426bbc0 0x563fd426dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.922+0000 7f9567e00640 1 --2- 172.21.15.1:0/3646144483 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563fd426bbc0 0x563fd426dff0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d3afac7c1624c74 server_cookie=199c3a3a58f52055 in_seq=0 out_seq=0 2024-04-26T01:52:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.922+0000 7f955d600640 10 client.25325 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.923+0000 7f955d600640 1 -- 172.21.15.1:0/3646144483 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563fd41ca8b0 con 0x563fd426bbc0 2024-04-26T01:52:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.923+0000 7f955d600640 10 client.25325 handle_command_reply: tid=0 2024-04-26T01:52:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.923+0000 7f9579416740 1 client.25325 shutdown 2024-04-26T01:52:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.923+0000 7f9579416740 1 -- 172.21.15.1:0/3646144483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd42509b0 msgr2=0x563fd4258b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.923+0000 7f9579416740 1 --2- 172.21.15.1:0/3646144483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd42509b0 0x563fd4258b10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9560011660 tx=0x7f9560011b20 comp rx=0 tx=0).stop 2024-04-26T01:52:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.924+0000 7f9579416740 1 -- 172.21.15.1:0/3646144483 shutdown_connections 2024-04-26T01:52:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.924+0000 7f9579416740 1 --2- 172.21.15.1:0/3646144483 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563fd426bbc0 0x563fd426dff0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.924+0000 7f9579416740 1 --2- 172.21.15.1:0/3646144483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd425bb90 0x563fd4259590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.924+0000 7f9579416740 1 --2- 172.21.15.1:0/3646144483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563fd42512d0 0x563fd4259050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.924+0000 7f9579416740 1 --2- 172.21.15.1:0/3646144483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd42509b0 0x563fd4258b10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.924+0000 7f9579416740 1 -- 172.21.15.1:0/3646144483 >> 172.21.15.1:0/3646144483 conn(0x563fd424cec0 msgr2=0x563fd4252bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.924+0000 7f955d600640 0 client.25325 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.924+0000 7f9579416740 1 -- 172.21.15.1:0/3646144483 shutdown_connections 2024-04-26T01:52:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.924+0000 7f9579416740 1 -- 172.21.15.1:0/3646144483 wait complete. 2024-04-26T01:52:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.924+0000 7f9579416740 20 client.25325 trim_cache size 0 max 16384 2024-04-26T01:52:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.950+0000 7f9579416740 1 Processor -- start 2024-04-26T01:52:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.951+0000 7f9579416740 1 -- start start 2024-04-26T01:52:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.951+0000 7f9579416740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563fd42509b0 0x563fd4267380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.951+0000 7f9579416740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd42512d0 0x563fd4273900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.951+0000 7f9579416740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd425bb90 0x563fd4273e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.951+0000 7f9579416740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f95701b69a0 con 0x563fd42509b0 2024-04-26T01:52:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.951+0000 7f9567e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd42512d0 0x563fd4273900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.951+0000 7f9579416740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f95701b4270 con 0x563fd425bb90 2024-04-26T01:52:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.951+0000 7f9579416740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f95701b20c0 con 0x563fd42512d0 2024-04-26T01:52:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.952+0000 7f9567e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd42512d0 0x563fd4273900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:43492/0 (socket says 172.21.15.1:43492) 2024-04-26T01:52:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.952+0000 7f9567e00640 1 -- 172.21.15.1:0/3296815417 learned_addr learned my addr 172.21.15.1:0/3296815417 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.952+0000 7f956fe00640 1 --2- 172.21.15.1:0/3296815417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd425bb90 0x563fd4273e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.952+0000 7f956f400640 1 --2- 172.21.15.1:0/3296815417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563fd42509b0 0x563fd4267380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.953+0000 7f9567e00640 1 -- 172.21.15.1:0/3296815417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd425bb90 msgr2=0x563fd4273e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.953+0000 7f9567e00640 1 --2- 172.21.15.1:0/3296815417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd425bb90 0x563fd4273e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.953+0000 7f9567e00640 1 -- 172.21.15.1:0/3296815417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563fd42509b0 msgr2=0x563fd4267380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.953+0000 7f9567e00640 1 --2- 172.21.15.1:0/3296815417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563fd42509b0 0x563fd4267380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.953+0000 7f9567e00640 1 -- 172.21.15.1:0/3296815417 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fd4247400 con 0x563fd42512d0 2024-04-26T01:52:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.953+0000 7f956fe00640 1 --2- 172.21.15.1:0/3296815417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd425bb90 0x563fd4273e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:52:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.953+0000 7f9567e00640 1 --2- 172.21.15.1:0/3296815417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd42512d0 0x563fd4273900 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f95580bcb50 tx=0x7f95580e4040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=204ab2220ebf8840 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.953+0000 7f955fe00640 1 -- 172.21.15.1:0/3296815417 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95580c0f70 con 0x563fd42512d0 2024-04-26T01:52:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.953+0000 7f955fe00640 1 -- 172.21.15.1:0/3296815417 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95580bcbc0 con 0x563fd42512d0 2024-04-26T01:52:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.953+0000 7f955fe00640 1 -- 172.21.15.1:0/3296815417 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558003c50 con 0x563fd42512d0 2024-04-26T01:52:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.954+0000 7f9579416740 1 -- 172.21.15.1:0/3296815417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd42512d0 msgr2=0x563fd4273900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.954+0000 7f9579416740 1 --2- 172.21.15.1:0/3296815417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd42512d0 0x563fd4273900 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f95580bcb50 tx=0x7f95580e4040 comp rx=0 tx=0).stop 2024-04-26T01:52:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.954+0000 7f9579416740 1 -- 172.21.15.1:0/3296815417 shutdown_connections 2024-04-26T01:52:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.954+0000 7f9579416740 1 --2- 172.21.15.1:0/3296815417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd425bb90 0x563fd4273e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.954+0000 7f9579416740 1 --2- 172.21.15.1:0/3296815417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd42512d0 0x563fd4273900 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.954+0000 7f9579416740 1 --2- 172.21.15.1:0/3296815417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563fd42509b0 0x563fd4267380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.954+0000 7f9579416740 1 -- 172.21.15.1:0/3296815417 >> 172.21.15.1:0/3296815417 conn(0x563fd424cec0 msgr2=0x563fd4253e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.954+0000 7f9579416740 1 -- 172.21.15.1:0/3296815417 shutdown_connections 2024-04-26T01:52:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.954+0000 7f9579416740 1 -- 172.21.15.1:0/3296815417 wait complete. 2024-04-26T01:52:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.955+0000 7f9579416740 1 -- start start 2024-04-26T01:52:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.955+0000 7f9579416740 1 Processor -- start 2024-04-26T01:52:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.956+0000 7f9579416740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd42509b0 0x563fd42639d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.956+0000 7f9579416740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563fd42512d0 0x563fd4262530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.956+0000 7f9579416740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd425bb90 0x563fd4262a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.956+0000 7f9579416740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f95701b2730 con 0x563fd42512d0 2024-04-26T01:52:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.956+0000 7f9579416740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f95701b3ba0 con 0x563fd425bb90 2024-04-26T01:52:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.956+0000 7f9579416740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f95701b2e00 con 0x563fd42509b0 2024-04-26T01:52:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.956+0000 7f9579416740 10 client.0 _ll_register_callbacks cb 0x7f9560009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.956+0000 7f956fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd425bb90 0x563fd4262a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.956+0000 7f9567e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563fd42512d0 0x563fd4262530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.956+0000 7f956fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd425bb90 0x563fd4262a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:54438/0 (socket says 172.21.15.1:54438) 2024-04-26T01:52:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.956+0000 7f956fe00640 1 -- 172.21.15.1:0/1769063968 learned_addr learned my addr 172.21.15.1:0/1769063968 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.957+0000 7f956f400640 1 --2- 172.21.15.1:0/1769063968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd42509b0 0x563fd42639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.957+0000 7f956f400640 1 -- 172.21.15.1:0/1769063968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd425bb90 msgr2=0x563fd4262a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.957+0000 7f956f400640 1 --2- 172.21.15.1:0/1769063968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd425bb90 0x563fd4262a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.957+0000 7f956f400640 1 -- 172.21.15.1:0/1769063968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563fd42512d0 msgr2=0x563fd4262530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.957+0000 7f956f400640 1 --2- 172.21.15.1:0/1769063968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563fd42512d0 0x563fd4262530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.957+0000 7f956f400640 1 -- 172.21.15.1:0/1769063968 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fd4263030 con 0x563fd42509b0 2024-04-26T01:52:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.957+0000 7f956f400640 1 --2- 172.21.15.1:0/1769063968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd42509b0 0x563fd42639d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9560003f00 tx=0x7f956000f7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c70fb512943c94c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.957+0000 7f955d600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:52:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.958+0000 7f955d600640 1 -- 172.21.15.1:0/1769063968 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560032060 con 0x563fd42509b0 2024-04-26T01:52:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.958+0000 7f955d600640 1 -- 172.21.15.1:0/1769063968 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f956000f8c0 con 0x563fd42509b0 2024-04-26T01:52:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.958+0000 7f9579416740 1 -- 172.21.15.1:0/1769063968 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95580bcbc0 con 0x563fd42509b0 2024-04-26T01:52:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.959+0000 7f955d600640 1 -- 172.21.15.1:0/1769063968 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9560028080 con 0x563fd42509b0 2024-04-26T01:52:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.959+0000 7f9579416740 10 client.25348 fetch_fsmap learned FSMap version 26 2024-04-26T01:52:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.959+0000 7f9579416740 1 -- 172.21.15.1:0/1769063968 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563fd42d1f40 con 0x563fd42509b0 2024-04-26T01:52:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.959+0000 7f955d600640 1 -- 172.21.15.1:0/1769063968 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95580bcbc0 con 0x563fd42509b0 2024-04-26T01:52:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.959+0000 7f955d600640 1 -- 172.21.15.1:0/1769063968 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f95600035f0 con 0x563fd42509b0 2024-04-26T01:52:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.959+0000 7f9579416740 10 client.25348 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:52:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.959+0000 7f9579416740 10 client.25348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:52:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.959+0000 7f9579416740 20 client.25348 populate_metadata read hostname 'smithi001' 2024-04-26T01:52:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.959+0000 7f9579416740 1 --2- 172.21.15.1:0/1769063968 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563fd42d2650 0x563fd42d4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.959+0000 7f9579416740 4 client.25348 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:52:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.959+0000 7f9579416740 1 -- 172.21.15.1:0/1769063968 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563fd41c4320 con 0x563fd42d2650 2024-04-26T01:52:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.960+0000 7f9567e00640 1 --2- 172.21.15.1:0/1769063968 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563fd42d2650 0x563fd42d4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.961+0000 7f9567e00640 1 --2- 172.21.15.1:0/1769063968 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563fd42d2650 0x563fd42d4a70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1df10b01004f11a7 server_cookie=8013eb9bb754fc5e in_seq=0 out_seq=0 2024-04-26T01:52:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.961+0000 7f955d600640 10 client.25348 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.965+0000 7f955d600640 1 -- 172.21.15.1:0/1769063968 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563fd41c4320 con 0x563fd42d2650 2024-04-26T01:52:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.965+0000 7f955d600640 10 client.25348 handle_command_reply: tid=0 2024-04-26T01:52:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.965+0000 7f9579416740 1 client.25348 shutdown 2024-04-26T01:52:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.966+0000 7f9579416740 1 -- 172.21.15.1:0/1769063968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd42509b0 msgr2=0x563fd42639d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.966+0000 7f9579416740 1 --2- 172.21.15.1:0/1769063968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd42509b0 0x563fd42639d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9560003f00 tx=0x7f956000f7a0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.966+0000 7f9579416740 1 -- 172.21.15.1:0/1769063968 shutdown_connections 2024-04-26T01:52:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.966+0000 7f9579416740 1 --2- 172.21.15.1:0/1769063968 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563fd42d2650 0x563fd42d4a70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.966+0000 7f9579416740 1 --2- 172.21.15.1:0/1769063968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563fd425bb90 0x563fd4262a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.966+0000 7f9579416740 1 --2- 172.21.15.1:0/1769063968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563fd42512d0 0x563fd4262530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.966+0000 7f9579416740 1 --2- 172.21.15.1:0/1769063968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fd42509b0 0x563fd42639d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.966+0000 7f955d600640 0 client.25348 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.966+0000 7f9579416740 1 -- 172.21.15.1:0/1769063968 >> 172.21.15.1:0/1769063968 conn(0x563fd424cec0 msgr2=0x563fd424b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.966+0000 7f9579416740 1 -- 172.21.15.1:0/1769063968 shutdown_connections 2024-04-26T01:52:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.966+0000 7f9579416740 1 -- 172.21.15.1:0/1769063968 wait complete. 2024-04-26T01:52:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.966+0000 7f9579416740 20 client.25348 trim_cache size 0 max 16384 2024-04-26T01:52:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:52:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:52:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:52:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:52:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.967+0000 7f9575000640 1 -- 172.21.15.1:0/3555867231 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f95500a3fb0 msgr2=0x7f95500a6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.967+0000 7f9575000640 1 --2- 172.21.15.1:0/3555867231 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f95500a3fb0 0x7f95500a6470 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9568005c40 tx=0x7f9568009540 comp rx=0 tx=0).stop 2024-04-26T01:52:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.967+0000 7f9575000640 1 -- 172.21.15.1:0/3555867231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9570101640 msgr2=0x7f95701a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.967+0000 7f9575000640 1 --2- 172.21.15.1:0/3555867231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9570101640 0x7f95701a4cf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9558005ab0 tx=0x7f9558032c00 comp rx=0 tx=0).stop 2024-04-26T01:52:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.968+0000 7f9575000640 1 -- 172.21.15.1:0/3555867231 shutdown_connections 2024-04-26T01:52:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.968+0000 7f9575000640 1 --2- 172.21.15.1:0/3555867231 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f95500a3fb0 0x7f95500a6470 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.968+0000 7f9575000640 1 --2- 172.21.15.1:0/3555867231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9570106120 0x7f957019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.968+0000 7f9575000640 1 --2- 172.21.15.1:0/3555867231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9570101640 0x7f95701a4cf0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.968+0000 7f9575000640 1 --2- 172.21.15.1:0/3555867231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9570100d20 0x7f95701a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.968+0000 7f9575000640 1 -- 172.21.15.1:0/3555867231 >> 172.21.15.1:0/3555867231 conn(0x7f95700fc730 msgr2=0x7f95700fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.968+0000 7f9575000640 1 -- 172.21.15.1:0/3555867231 shutdown_connections 2024-04-26T01:52:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:34.968+0000 7f9575000640 1 -- 172.21.15.1:0/3555867231 wait complete. 2024-04-26T01:52:36.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:36 smithi097 ceph-mon[32853]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T01:52:36.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:36 smithi173 ceph-mon[32553]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T01:52:36.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:36 smithi001 ceph-mon[27822]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-26T01:52:38.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:38 smithi097 ceph-mon[32853]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-26T01:52:38.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:38 smithi173 ceph-mon[32553]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-26T01:52:38.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:38 smithi001 ceph-mon[27822]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-26T01:52:39.647 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.6 2024-04-26T01:52:39.648 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:52:40.064 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:40.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:40 smithi001 ceph-mon[27822]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:52:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:40 smithi097 ceph-mon[32853]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:52:40.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:40 smithi173 ceph-mon[32553]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T01:52:42.232 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:42 smithi001 ceph-mon[27822]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T01:52:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.466+0000 7f4ad5e00700 1 -- 172.21.15.1:0/1559826402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad006a750 msgr2=0x7f4ad006abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.466+0000 7f4ad5e00700 1 --2- 172.21.15.1:0/1559826402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad006a750 0x7f4ad006abb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0098f0 tx=0x7f4abc009c00 comp rx=0 tx=0).stop 2024-04-26T01:52:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.466+0000 7f4ad5e00700 1 -- 172.21.15.1:0/1559826402 shutdown_connections 2024-04-26T01:52:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.466+0000 7f4ad5e00700 1 --2- 172.21.15.1:0/1559826402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ad006b0f0 0x7f4ad010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.466+0000 7f4ad5e00700 1 --2- 172.21.15.1:0/1559826402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ad0108a30 0x7f4ad0108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.466+0000 7f4ad5e00700 1 --2- 172.21.15.1:0/1559826402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad006a750 0x7f4ad006abb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.466+0000 7f4ad5e00700 1 -- 172.21.15.1:0/1559826402 >> 172.21.15.1:0/1559826402 conn(0x7f4ad00788f0 msgr2=0x7f4ad0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:42.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.466+0000 7f4ad5e00700 1 -- 172.21.15.1:0/1559826402 shutdown_connections 2024-04-26T01:52:42.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.466+0000 7f4ad5e00700 1 -- 172.21.15.1:0/1559826402 wait complete. 2024-04-26T01:52:42.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.468+0000 7f4ad5e00700 1 Processor -- start 2024-04-26T01:52:42.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.468+0000 7f4ad5e00700 1 -- start start 2024-04-26T01:52:42.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.469+0000 7f4ad5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ad006a750 0x7f4ad00742f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:42.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.469+0000 7f4ad5e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad006b0f0 0x7f4ad006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:42.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.469+0000 7f4ad5e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ad0108a30 0x7f4ad006d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:42.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.469+0000 7f4ad5e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad0112c20 con 0x7f4ad006a750 2024-04-26T01:52:42.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.469+0000 7f4ad5e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad0112aa0 con 0x7f4ad006b0f0 2024-04-26T01:52:42.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.469+0000 7f4ad5e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad0112da0 con 0x7f4ad0108a30 2024-04-26T01:52:42.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.469+0000 7f4acea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ad006a750 0x7f4ad00742f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:42.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.469+0000 7f4acea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ad006a750 0x7f4ad00742f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34854/0 (socket says 172.21.15.1:34854) 2024-04-26T01:52:42.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.469+0000 7f4acea00700 1 -- 172.21.15.1:0/587342826 learned_addr learned my addr 172.21.15.1:0/587342826 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:42.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.469+0000 7f4ace000700 1 --2- 172.21.15.1:0/587342826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad006b0f0 0x7f4ad006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:42.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.469+0000 7f4acf400700 1 --2- 172.21.15.1:0/587342826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ad0108a30 0x7f4ad006d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:42.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.470+0000 7f4acea00700 1 -- 172.21.15.1:0/587342826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ad0108a30 msgr2=0x7f4ad006d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:42.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.470+0000 7f4acea00700 1 --2- 172.21.15.1:0/587342826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ad0108a30 0x7f4ad006d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:42.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.470+0000 7f4acea00700 1 -- 172.21.15.1:0/587342826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad006b0f0 msgr2=0x7f4ad006d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:42.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.470+0000 7f4acea00700 1 --2- 172.21.15.1:0/587342826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad006b0f0 0x7f4ad006d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:42.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.470+0000 7f4acea00700 1 -- 172.21.15.1:0/587342826 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad006dff0 con 0x7f4ad006a750 2024-04-26T01:52:42.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.470+0000 7f4acea00700 1 --2- 172.21.15.1:0/587342826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ad006a750 0x7f4ad00742f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0012db0 tx=0x7f4ac000c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:42.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.470+0000 7f4ad4a00700 1 -- 172.21.15.1:0/587342826 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac000cdd0 con 0x7f4ad006a750 2024-04-26T01:52:42.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.471+0000 7f4ad5e00700 1 -- 172.21.15.1:0/587342826 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad0071e20 con 0x7f4ad006a750 2024-04-26T01:52:42.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.471+0000 7f4ad5e00700 1 -- 172.21.15.1:0/587342826 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad0072380 con 0x7f4ad006a750 2024-04-26T01:52:42.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.471+0000 7f4ad4a00700 1 -- 172.21.15.1:0/587342826 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ac001a8c0 con 0x7f4ad006a750 2024-04-26T01:52:42.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.471+0000 7f4ad4a00700 1 -- 172.21.15.1:0/587342826 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0019900 con 0x7f4ad006a750 2024-04-26T01:52:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.473+0000 7f4ad5e00700 1 -- 172.21.15.1:0/587342826 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ab0005330 con 0x7f4ad006a750 2024-04-26T01:52:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.476+0000 7f4ad4a00700 1 -- 172.21.15.1:0/587342826 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7f4ac001a410 con 0x7f4ad006a750 2024-04-26T01:52:42.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.478+0000 7f4ad4a00700 1 --2- 172.21.15.1:0/587342826 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f4ab40a3fe0 0x7f4ab40a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:42.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.478+0000 7f4ad4a00700 1 -- 172.21.15.1:0/587342826 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f4ac00d45a0 con 0x7f4ad006a750 2024-04-26T01:52:42.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.479+0000 7f4ace000700 1 --2- 172.21.15.1:0/587342826 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f4ab40a3fe0 0x7f4ab40a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:42.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.480+0000 7f4ace000700 1 --2- 172.21.15.1:0/587342826 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f4ab40a3fe0 0x7f4ab40a64a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4ad006ebd0 tx=0x7f4abc019040 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:42.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:42 smithi097 ceph-mon[32853]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T01:52:42.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:42.487+0000 7f4ad4a00700 1 -- 172.21.15.1:0/587342826 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ac00a0a00 con 0x7f4ad006a750 2024-04-26T01:52:42.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:42 smithi173 ceph-mon[32553]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T01:52:43.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:43.021+0000 7f4ad5e00700 1 -- 172.21.15.1:0/587342826 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ab00058f0 con 0x7f4ad006a750 2024-04-26T01:52:43.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:43.023+0000 7f4ad4a00700 1 -- 172.21.15.1:0/587342826 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11841 (secure 0 0 0) 0x7f4ac00a0300 con 0x7f4ad006a750 2024-04-26T01:52:43.021 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:52:43.021 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:51:38.222728+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2952462325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2952462325},{"type":"v1","addr":"172.21.15.173:6835","nonce":2952462325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24499,24514,24518]},"id":1}]} 2024-04-26T01:52:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:43.026+0000 7f4ad5e00700 1 -- 172.21.15.1:0/587342826 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f4ab40a3fe0 msgr2=0x7f4ab40a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:43.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:43.026+0000 7f4ad5e00700 1 --2- 172.21.15.1:0/587342826 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f4ab40a3fe0 0x7f4ab40a64a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4ad006ebd0 tx=0x7f4abc019040 comp rx=0 tx=0).stop 2024-04-26T01:52:43.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:43.026+0000 7f4ad5e00700 1 -- 172.21.15.1:0/587342826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ad006a750 msgr2=0x7f4ad00742f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:43.026+0000 7f4ad5e00700 1 --2- 172.21.15.1:0/587342826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ad006a750 0x7f4ad00742f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0012db0 tx=0x7f4ac000c5b0 comp rx=0 tx=0).stop 2024-04-26T01:52:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:43.026+0000 7f4ad5e00700 1 -- 172.21.15.1:0/587342826 shutdown_connections 2024-04-26T01:52:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:43.026+0000 7f4ad5e00700 1 --2- 172.21.15.1:0/587342826 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f4ab40a3fe0 0x7f4ab40a64a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:43.026+0000 7f4ad5e00700 1 --2- 172.21.15.1:0/587342826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ad0108a30 0x7f4ad006d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:43.026+0000 7f4ad5e00700 1 --2- 172.21.15.1:0/587342826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ad006a750 0x7f4ad00742f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:43.026+0000 7f4ad5e00700 1 --2- 172.21.15.1:0/587342826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ad006b0f0 0x7f4ad006d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:43.026+0000 7f4ad5e00700 1 -- 172.21.15.1:0/587342826 >> 172.21.15.1:0/587342826 conn(0x7f4ad00788f0 msgr2=0x7f4ad00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:43.027+0000 7f4ad5e00700 1 -- 172.21.15.1:0/587342826 shutdown_connections 2024-04-26T01:52:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:43.027+0000 7f4ad5e00700 1 -- 172.21.15.1:0/587342826 wait complete. 2024-04-26T01:52:43.027 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 26 2024-04-26T01:52:43.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:43 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/587342826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:52:43.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:43 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/587342826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:52:43.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:43 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/587342826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:52:43.899 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24518 quiesce db / --timeout 60 --expiration 85.6 --await-for 42.0' 2024-04-26T01:52:43.899 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24518 quiesce db / --timeout 60 --expiration 85.6 --await-for 42.0 2024-04-26T01:52:44.390 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:44.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:44 smithi097 ceph-mon[32853]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T01:52:44.548 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:44 smithi001 ceph-mon[27822]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T01:52:44.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:44 smithi173 ceph-mon[32553]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T01:52:45.438 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:45 smithi001 ceph-mon[27822]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T01:52:45.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:45 smithi097 ceph-mon[32853]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T01:52:45.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:45 smithi173 ceph-mon[32553]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T01:52:46.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.708+0000 7efcdfe00700 1 -- 172.21.15.1:0/1749041458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efce010f2a0 msgr2=0x7efce0111690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:46.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.708+0000 7efcdfe00700 1 --2- 172.21.15.1:0/1749041458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efce010f2a0 0x7efce0111690 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efcd800b0a0 tx=0x7efcd800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.708+0000 7efcdfe00700 1 -- 172.21.15.1:0/1749041458 shutdown_connections 2024-04-26T01:52:46.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.708+0000 7efcdfe00700 1 --2- 172.21.15.1:0/1749041458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efce010f2a0 0x7efce0111690 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.708+0000 7efcdfe00700 1 --2- 172.21.15.1:0/1749041458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efce0074fe0 0x7efce00753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.708+0000 7efcdfe00700 1 --2- 172.21.15.1:0/1749041458 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efce0075900 0x7efce010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.708+0000 7efcdfe00700 1 -- 172.21.15.1:0/1749041458 >> 172.21.15.1:0/1749041458 conn(0x7efce00fe3a0 msgr2=0x7efce01007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:46.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.709+0000 7efcdfe00700 1 -- 172.21.15.1:0/1749041458 shutdown_connections 2024-04-26T01:52:46.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.709+0000 7efcdfe00700 1 -- 172.21.15.1:0/1749041458 wait complete. 2024-04-26T01:52:46.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.710+0000 7efcdfe00700 1 Processor -- start 2024-04-26T01:52:46.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.710+0000 7efcdfe00700 1 -- start start 2024-04-26T01:52:46.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.711+0000 7efcdfe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efce0074fe0 0x7efce019b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:46.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.711+0000 7efcdfe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efce0075900 0x7efce019b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:46.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.711+0000 7efcdfe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efce010f2a0 0x7efce019fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:46.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.711+0000 7efcdfe00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efce0114d70 con 0x7efce010f2a0 2024-04-26T01:52:46.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.711+0000 7efcdfe00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efce0114bf0 con 0x7efce0075900 2024-04-26T01:52:46.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.711+0000 7efcdfe00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efce0114ef0 con 0x7efce0074fe0 2024-04-26T01:52:46.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.711+0000 7efcde000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efce0075900 0x7efce019b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:46.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.711+0000 7efcdea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efce0074fe0 0x7efce019b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:46.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.711+0000 7efcde000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efce0075900 0x7efce019b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:57600/0 (socket says 172.21.15.1:57600) 2024-04-26T01:52:46.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.711+0000 7efcdea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efce0074fe0 0x7efce019b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:50552/0 (socket says 172.21.15.1:50552) 2024-04-26T01:52:46.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.711+0000 7efcde000700 1 -- 172.21.15.1:0/2298840108 learned_addr learned my addr 172.21.15.1:0/2298840108 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:46.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.712+0000 7efcdf400700 1 --2- 172.21.15.1:0/2298840108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efce010f2a0 0x7efce019fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:46.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.712+0000 7efcdea00700 1 -- 172.21.15.1:0/2298840108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efce0075900 msgr2=0x7efce019b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:46.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.712+0000 7efcdea00700 1 --2- 172.21.15.1:0/2298840108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efce0075900 0x7efce019b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.712+0000 7efcdea00700 1 -- 172.21.15.1:0/2298840108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efce010f2a0 msgr2=0x7efce019fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:46.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.712+0000 7efcdea00700 1 --2- 172.21.15.1:0/2298840108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efce010f2a0 0x7efce019fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.712+0000 7efcdea00700 1 -- 172.21.15.1:0/2298840108 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce01a0220 con 0x7efce0074fe0 2024-04-26T01:52:46.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.712+0000 7efcdea00700 1 --2- 172.21.15.1:0/2298840108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efce0074fe0 0x7efce019b260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efcd000e9f0 tx=0x7efcd000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:46.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.713+0000 7efcd7400700 1 -- 172.21.15.1:0/2298840108 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd000cb70 con 0x7efce0074fe0 2024-04-26T01:52:46.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.713+0000 7efcd7400700 1 -- 172.21.15.1:0/2298840108 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcd000ee90 con 0x7efce0074fe0 2024-04-26T01:52:46.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.713+0000 7efcd7400700 1 -- 172.21.15.1:0/2298840108 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0003d40 con 0x7efce0074fe0 2024-04-26T01:52:46.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.713+0000 7efcdfe00700 1 -- 172.21.15.1:0/2298840108 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce01a0510 con 0x7efce0074fe0 2024-04-26T01:52:46.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.713+0000 7efcdfe00700 1 -- 172.21.15.1:0/2298840108 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce01a7d90 con 0x7efce0074fe0 2024-04-26T01:52:46.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.717+0000 7efcd7400700 1 -- 172.21.15.1:0/2298840108 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7efcd0003ee0 con 0x7efce0074fe0 2024-04-26T01:52:46.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.718+0000 7efceff96b80 1 Processor -- start 2024-04-26T01:52:46.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.719+0000 7efceff96b80 1 -- start start 2024-04-26T01:52:46.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.719+0000 7efceff96b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d897ab80 0x5588d897af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:46.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.719+0000 7efceff96b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d897b4a0 0x5588d8985e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:46.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.719+0000 7efceff96b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d8986350 0x5588d8988740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:46.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.719+0000 7efceff96b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5588d88ce020 con 0x5588d8986350 2024-04-26T01:52:46.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.719+0000 7efceff96b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5588d88748c0 con 0x5588d897b4a0 2024-04-26T01:52:46.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.719+0000 7efceff96b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5588d88c9fe0 con 0x5588d897ab80 2024-04-26T01:52:46.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.719+0000 7efcd7400700 1 --2- 172.21.15.1:0/2298840108 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efcc40a4000 0x7efcc40a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:46.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.720+0000 7efcdf400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d897ab80 0x5588d897af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:46.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.720+0000 7efcdea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d897b4a0 0x5588d8985e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:46.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.720+0000 7efcdea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d897b4a0 0x5588d8985e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:57610/0 (socket says 172.21.15.1:57610) 2024-04-26T01:52:46.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.720+0000 7efcdea00700 1 -- 172.21.15.1:0/3021029994 learned_addr learned my addr 172.21.15.1:0/3021029994 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:46.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.720+0000 7efcde000700 1 --2- 172.21.15.1:0/3021029994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d8986350 0x5588d8988740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.720+0000 7efcd7400700 1 -- 172.21.15.1:0/2298840108 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7efcd00d0ba0 con 0x7efce0074fe0 2024-04-26T01:52:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.721+0000 7efcde000700 1 -- 172.21.15.1:0/3021029994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d897ab80 msgr2=0x5588d897af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.721+0000 7efcde000700 1 --2- 172.21.15.1:0/3021029994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d897ab80 0x5588d897af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.721+0000 7efcde000700 1 -- 172.21.15.1:0/3021029994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d897b4a0 msgr2=0x5588d8985e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.721+0000 7efcde000700 1 --2- 172.21.15.1:0/3021029994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d897b4a0 0x5588d8985e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.721+0000 7efcde000700 1 -- 172.21.15.1:0/3021029994 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588d886a810 con 0x5588d8986350 2024-04-26T01:52:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.721+0000 7efcdf400700 1 --2- 172.21.15.1:0/3021029994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d897ab80 0x5588d897af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:52:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.721+0000 7efcdf400700 1 --2- 172.21.15.1:0/2298840108 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efcc40a4000 0x7efcc40a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.721+0000 7efcde000700 1 --2- 172.21.15.1:0/3021029994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d8986350 0x5588d8988740 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efccc005950 tx=0x7efccc009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbdaafed354137a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.722+0000 7efccbe00700 1 -- 172.21.15.1:0/3021029994 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc011d10 con 0x5588d8986350 2024-04-26T01:52:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.722+0000 7efccbe00700 1 -- 172.21.15.1:0/3021029994 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efccc0044d0 con 0x5588d8986350 2024-04-26T01:52:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.722+0000 7efccbe00700 1 -- 172.21.15.1:0/3021029994 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc005280 con 0x5588d8986350 2024-04-26T01:52:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.722+0000 7efceff96b80 1 -- 172.21.15.1:0/3021029994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d8986350 msgr2=0x5588d8988740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.722+0000 7efcdf400700 1 --2- 172.21.15.1:0/2298840108 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efcc40a4000 0x7efcc40a64c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x5588d8977ec0 tx=0x7efcd80099e0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.722+0000 7efceff96b80 1 --2- 172.21.15.1:0/3021029994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d8986350 0x5588d8988740 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efccc005950 tx=0x7efccc009f70 comp rx=0 tx=0).stop 2024-04-26T01:52:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.723+0000 7efceff96b80 1 -- 172.21.15.1:0/3021029994 shutdown_connections 2024-04-26T01:52:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.723+0000 7efceff96b80 1 --2- 172.21.15.1:0/3021029994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d8986350 0x5588d8988740 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.723+0000 7efceff96b80 1 --2- 172.21.15.1:0/3021029994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d897ab80 0x5588d897af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.723+0000 7efceff96b80 1 --2- 172.21.15.1:0/3021029994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d897b4a0 0x5588d8985e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.723+0000 7efceff96b80 1 -- 172.21.15.1:0/3021029994 >> 172.21.15.1:0/3021029994 conn(0x5588d88a9730 msgr2=0x5588d8976fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:46.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.723+0000 7efceff96b80 1 -- 172.21.15.1:0/3021029994 shutdown_connections 2024-04-26T01:52:46.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.724+0000 7efceff96b80 1 -- 172.21.15.1:0/3021029994 wait complete. 2024-04-26T01:52:46.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.724+0000 7efceff96b80 1 -- start start 2024-04-26T01:52:46.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.724+0000 7efceff96b80 1 Processor -- start 2024-04-26T01:52:46.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.725+0000 7efceff96b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d897ab80 0x5588d8985ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:46.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.725+0000 7efceff96b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d897b4a0 0x5588d8982de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.725+0000 7efceff96b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d8986350 0x5588d8983320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.725+0000 7efceff96b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5588d8988ed0 con 0x5588d897ab80 2024-04-26T01:52:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.725+0000 7efceff96b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5588d88879f0 con 0x5588d897b4a0 2024-04-26T01:52:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.725+0000 7efceff96b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5588d8989050 con 0x5588d8986350 2024-04-26T01:52:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.725+0000 7efceff96b80 10 client.0 _ll_register_callbacks cb 0x5588d88668b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.725+0000 7efcdea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d897ab80 0x5588d8985ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.725+0000 7efcdea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d897ab80 0x5588d8985ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55600/0 (socket says 172.21.15.1:55600) 2024-04-26T01:52:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.725+0000 7efcdea00700 1 -- 172.21.15.1:0/1905986541 learned_addr learned my addr 172.21.15.1:0/1905986541 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.726+0000 7efcdf400700 1 --2- 172.21.15.1:0/1905986541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d8986350 0x5588d8983320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.726+0000 7efcde000700 1 --2- 172.21.15.1:0/1905986541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d897b4a0 0x5588d8982de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.726+0000 7efcdea00700 1 -- 172.21.15.1:0/1905986541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d8986350 msgr2=0x5588d8983320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.726+0000 7efcdea00700 1 --2- 172.21.15.1:0/1905986541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d8986350 0x5588d8983320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.726+0000 7efcdea00700 1 -- 172.21.15.1:0/1905986541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d897b4a0 msgr2=0x5588d8982de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.726+0000 7efcdea00700 1 --2- 172.21.15.1:0/1905986541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d897b4a0 0x5588d8982de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.726+0000 7efcdea00700 1 -- 172.21.15.1:0/1905986541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588d881ba60 con 0x5588d897ab80 2024-04-26T01:52:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.726+0000 7efcde000700 1 --2- 172.21.15.1:0/1905986541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d897b4a0 0x5588d8982de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:52:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.727+0000 7efcdea00700 1 --2- 172.21.15.1:0/1905986541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d897ab80 0x5588d8985ae0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efcd0000c00 tx=0x7efcd00a0cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3662b25139c7fb25 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.727+0000 7efcc8c00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:52:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.727+0000 7efcc8c00700 1 -- 172.21.15.1:0/1905986541 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd009cd60 con 0x5588d897ab80 2024-04-26T01:52:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.727+0000 7efcc8c00700 1 -- 172.21.15.1:0/1905986541 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7efcd0087070 con 0x5588d897ab80 2024-04-26T01:52:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.727+0000 7efceff96b80 1 -- 172.21.15.1:0/1905986541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efccc0044d0 con 0x5588d897ab80 2024-04-26T01:52:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.727+0000 7efcc8c00700 1 -- 172.21.15.1:0/1905986541 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efcd009acc0 con 0x5588d897ab80 2024-04-26T01:52:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.728+0000 7efceff96b80 10 client.15222 fetch_fsmap learned FSMap version 26 2024-04-26T01:52:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.728+0000 7efceff96b80 1 -- 172.21.15.1:0/1905986541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5588d8864c80 con 0x5588d897ab80 2024-04-26T01:52:46.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.728+0000 7efcc8c00700 1 -- 172.21.15.1:0/1905986541 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efccc0044d0 con 0x5588d897ab80 2024-04-26T01:52:46.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.728+0000 7efcc8c00700 1 -- 172.21.15.1:0/1905986541 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7efcd00041d0 con 0x5588d897ab80 2024-04-26T01:52:46.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.728+0000 7efceff96b80 10 client.15222 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:52:46.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.728+0000 7efceff96b80 10 client.15222 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:52:46.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.728+0000 7efceff96b80 20 client.15222 populate_metadata read hostname 'smithi001' 2024-04-26T01:52:46.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.728+0000 7efceff96b80 1 --2- 172.21.15.1:0/1905986541 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5588d8995c30 0x5588d8998080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:46.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.728+0000 7efceff96b80 4 client.15222 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:52:46.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.728+0000 7efceff96b80 1 -- 172.21.15.1:0/1905986541 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588d88db810 con 0x5588d8995c30 2024-04-26T01:52:46.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.729+0000 7efcde000700 1 --2- 172.21.15.1:0/1905986541 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5588d8995c30 0x5588d8998080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:46.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.732+0000 7efcde000700 1 --2- 172.21.15.1:0/1905986541 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5588d8995c30 0x5588d8998080 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f98d3b2b681dabf server_cookie=62a147ea8a6d76b3 in_seq=0 out_seq=0 2024-04-26T01:52:46.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.732+0000 7efcc8c00700 10 client.15222 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:46.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.733+0000 7efcc8c00700 1 -- 172.21.15.1:0/1905986541 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5588d88db810 con 0x5588d8995c30 2024-04-26T01:52:46.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.733+0000 7efcc8c00700 10 client.15222 handle_command_reply: tid=0 2024-04-26T01:52:46.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.733+0000 7efceff96b80 1 client.15222 shutdown 2024-04-26T01:52:46.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.734+0000 7efceff96b80 1 -- 172.21.15.1:0/1905986541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d897ab80 msgr2=0x5588d8985ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:46.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.734+0000 7efceff96b80 1 --2- 172.21.15.1:0/1905986541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d897ab80 0x5588d8985ae0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efcd0000c00 tx=0x7efcd00a0cb0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.734+0000 7efceff96b80 1 -- 172.21.15.1:0/1905986541 shutdown_connections 2024-04-26T01:52:46.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.734+0000 7efceff96b80 1 --2- 172.21.15.1:0/1905986541 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5588d8995c30 0x5588d8998080 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.734+0000 7efceff96b80 1 --2- 172.21.15.1:0/1905986541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d8986350 0x5588d8983320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.734+0000 7efceff96b80 1 --2- 172.21.15.1:0/1905986541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d897ab80 0x5588d8985ae0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.734+0000 7efceff96b80 1 --2- 172.21.15.1:0/1905986541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d897b4a0 0x5588d8982de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.734+0000 7efceff96b80 1 -- 172.21.15.1:0/1905986541 >> 172.21.15.1:0/1905986541 conn(0x5588d88a9730 msgr2=0x5588d897cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:46.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.734+0000 7efcc8c00700 0 client.15222 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:46.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.734+0000 7efceff96b80 1 -- 172.21.15.1:0/1905986541 shutdown_connections 2024-04-26T01:52:46.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.734+0000 7efceff96b80 1 -- 172.21.15.1:0/1905986541 wait complete. 2024-04-26T01:52:46.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.735+0000 7efceff96b80 20 client.15222 trim_cache size 0 max 16384 2024-04-26T01:52:46.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.755+0000 7efceff96b80 1 Processor -- start 2024-04-26T01:52:46.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.755+0000 7efceff96b80 1 -- start start 2024-04-26T01:52:46.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.756+0000 7efceff96b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d897ab80 0x5588d89704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:46.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.756+0000 7efceff96b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d897b4a0 0x5588d8991d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:46.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.756+0000 7efceff96b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d8986350 0x5588d899d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:46.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.756+0000 7efceff96b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efce01b1c90 con 0x5588d8986350 2024-04-26T01:52:46.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.756+0000 7efceff96b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efce01af520 con 0x5588d897b4a0 2024-04-26T01:52:46.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.756+0000 7efceff96b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efce01ad370 con 0x5588d897ab80 2024-04-26T01:52:46.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.756+0000 7efcde000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d897b4a0 0x5588d8991d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:46.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.756+0000 7efcde000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d897b4a0 0x5588d8991d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:57614/0 (socket says 172.21.15.1:57614) 2024-04-26T01:52:46.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.756+0000 7efcde000700 1 -- 172.21.15.1:0/1837491564 learned_addr learned my addr 172.21.15.1:0/1837491564 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:46.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.756+0000 7efcdea00700 1 --2- 172.21.15.1:0/1837491564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d897ab80 0x5588d89704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:46.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.757+0000 7efcdea00700 1 -- 172.21.15.1:0/1837491564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d897b4a0 msgr2=0x5588d8991d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:46.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.757+0000 7efcdea00700 1 --2- 172.21.15.1:0/1837491564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d897b4a0 0x5588d8991d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.757+0000 7efcdea00700 1 -- 172.21.15.1:0/1837491564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d8986350 msgr2=0x5588d899d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:46.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.757+0000 7efcdea00700 1 --2- 172.21.15.1:0/1837491564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d8986350 0x5588d899d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.757+0000 7efcdea00700 1 -- 172.21.15.1:0/1837491564 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588d897bba0 con 0x5588d897ab80 2024-04-26T01:52:46.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.758+0000 7efcdea00700 1 --2- 172.21.15.1:0/1837491564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d897ab80 0x5588d89704d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efcd009c890 tx=0x7efcd0009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48cd1208fb8322e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:46.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.758+0000 7efccb400700 1 -- 172.21.15.1:0/1837491564 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd009c6d0 con 0x5588d897ab80 2024-04-26T01:52:46.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.758+0000 7efccb400700 1 -- 172.21.15.1:0/1837491564 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcd00041d0 con 0x5588d897ab80 2024-04-26T01:52:46.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.758+0000 7efccb400700 1 -- 172.21.15.1:0/1837491564 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0098ba0 con 0x5588d897ab80 2024-04-26T01:52:46.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.759+0000 7efceff96b80 1 -- 172.21.15.1:0/1837491564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d897ab80 msgr2=0x5588d89704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:46.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.759+0000 7efceff96b80 1 --2- 172.21.15.1:0/1837491564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d897ab80 0x5588d89704d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efcd009c890 tx=0x7efcd0009610 comp rx=0 tx=0).stop 2024-04-26T01:52:46.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.759+0000 7efceff96b80 1 -- 172.21.15.1:0/1837491564 shutdown_connections 2024-04-26T01:52:46.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.759+0000 7efceff96b80 1 --2- 172.21.15.1:0/1837491564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d8986350 0x5588d899d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.759+0000 7efceff96b80 1 --2- 172.21.15.1:0/1837491564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d897ab80 0x5588d89704d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.759+0000 7efceff96b80 1 --2- 172.21.15.1:0/1837491564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d897b4a0 0x5588d8991d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.759+0000 7efceff96b80 1 -- 172.21.15.1:0/1837491564 >> 172.21.15.1:0/1837491564 conn(0x5588d88a9730 msgr2=0x5588d897e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:46.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.760+0000 7efceff96b80 1 -- 172.21.15.1:0/1837491564 shutdown_connections 2024-04-26T01:52:46.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.760+0000 7efceff96b80 1 -- 172.21.15.1:0/1837491564 wait complete. 2024-04-26T01:52:46.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.761+0000 7efceff96b80 1 -- start start 2024-04-26T01:52:46.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.761+0000 7efceff96b80 1 Processor -- start 2024-04-26T01:52:46.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.761+0000 7efceff96b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d897ab80 0x5588d898e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:46.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.762+0000 7efceff96b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d897b4a0 0x5588d898b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:46.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.762+0000 7efceff96b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d8986350 0x5588d898b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:46.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.762+0000 7efceff96b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efce01ad9e0 con 0x5588d897ab80 2024-04-26T01:52:46.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.762+0000 7efceff96b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efce01aee50 con 0x5588d8986350 2024-04-26T01:52:46.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.762+0000 7efceff96b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efce01ae0b0 con 0x5588d897b4a0 2024-04-26T01:52:46.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.762+0000 7efceff96b80 10 client.0 _ll_register_callbacks cb 0x5588d88668b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.762+0000 7efcdea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d897ab80 0x5588d898e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.762+0000 7efcdea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d897ab80 0x5588d898e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55614/0 (socket says 172.21.15.1:55614) 2024-04-26T01:52:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.762+0000 7efcdea00700 1 -- 172.21.15.1:0/1550793337 learned_addr learned my addr 172.21.15.1:0/1550793337 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.762+0000 7efcde000700 1 --2- 172.21.15.1:0/1550793337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d897b4a0 0x5588d898b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.762+0000 7efcdf400700 1 --2- 172.21.15.1:0/1550793337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d8986350 0x5588d898b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.763+0000 7efcde000700 1 -- 172.21.15.1:0/1550793337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d8986350 msgr2=0x5588d898b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.763+0000 7efcde000700 1 --2- 172.21.15.1:0/1550793337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d8986350 0x5588d898b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.763+0000 7efcde000700 1 -- 172.21.15.1:0/1550793337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d897ab80 msgr2=0x5588d898e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.763+0000 7efcde000700 1 --2- 172.21.15.1:0/1550793337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d897ab80 0x5588d898e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.763+0000 7efcde000700 1 -- 172.21.15.1:0/1550793337 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588d898bdf0 con 0x5588d897b4a0 2024-04-26T01:52:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.764+0000 7efcde000700 1 --2- 172.21.15.1:0/1550793337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d897b4a0 0x5588d898b270 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efccc011720 tx=0x7efccc011ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=368689fb9b80506d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.764+0000 7efcc8c00700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:52:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.764+0000 7efcc8c00700 1 -- 172.21.15.1:0/1550793337 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc01d040 con 0x5588d897b4a0 2024-04-26T01:52:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.764+0000 7efcc8c00700 1 -- 172.21.15.1:0/1550793337 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7efccc0050d0 con 0x5588d897b4a0 2024-04-26T01:52:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.765+0000 7efceff96b80 1 -- 172.21.15.1:0/1550793337 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcd00041d0 con 0x5588d897b4a0 2024-04-26T01:52:46.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.765+0000 7efcc8c00700 1 -- 172.21.15.1:0/1550793337 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efccc039050 con 0x5588d897b4a0 2024-04-26T01:52:46.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.765+0000 7efceff96b80 10 client.25369 fetch_fsmap learned FSMap version 26 2024-04-26T01:52:46.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.765+0000 7efceff96b80 1 -- 172.21.15.1:0/1550793337 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5588d8a05830 con 0x5588d897b4a0 2024-04-26T01:52:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.766+0000 7efcc8c00700 1 -- 172.21.15.1:0/1550793337 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcd00041d0 con 0x5588d897b4a0 2024-04-26T01:52:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.766+0000 7efcc8c00700 1 -- 172.21.15.1:0/1550793337 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7efccc024430 con 0x5588d897b4a0 2024-04-26T01:52:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.766+0000 7efceff96b80 10 client.25369 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:52:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.766+0000 7efceff96b80 10 client.25369 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:52:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.766+0000 7efceff96b80 20 client.25369 populate_metadata read hostname 'smithi001' 2024-04-26T01:52:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.766+0000 7efceff96b80 1 --2- 172.21.15.1:0/1550793337 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5588d8a05dd0 0x5588d8a081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.766+0000 7efceff96b80 4 client.25369 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.6, "await_for": 42.0}] 2024-04-26T01:52:46.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.766+0000 7efceff96b80 1 -- 172.21.15.1:0/1550793337 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.6, "await_for": 42.0}) -- 0x5588d88d58b0 con 0x5588d8a05dd0 2024-04-26T01:52:46.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.767+0000 7efcdea00700 1 --2- 172.21.15.1:0/1550793337 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5588d8a05dd0 0x5588d8a081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:46.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.772+0000 7efcdea00700 1 --2- 172.21.15.1:0/1550793337 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5588d8a05dd0 0x5588d8a081f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71c63892d2415071 server_cookie=8bfab197bf5a376b in_seq=0 out_seq=0 2024-04-26T01:52:46.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.772+0000 7efcc8c00700 10 client.25369 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:46.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.782+0000 7efcc8c00700 1 -- 172.21.15.1:0/1550793337 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x5588d88d58b0 con 0x5588d8a05dd0 2024-04-26T01:52:46.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.782+0000 7efcc8c00700 10 client.25369 handle_command_reply: tid=0 2024-04-26T01:52:46.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.782+0000 7efceff96b80 1 client.25369 shutdown 2024-04-26T01:52:46.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.782+0000 7efceff96b80 1 -- 172.21.15.1:0/1550793337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d897b4a0 msgr2=0x5588d898b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:46.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.782+0000 7efceff96b80 1 --2- 172.21.15.1:0/1550793337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d897b4a0 0x5588d898b270 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efccc011720 tx=0x7efccc011ed0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.783+0000 7efceff96b80 1 -- 172.21.15.1:0/1550793337 shutdown_connections 2024-04-26T01:52:46.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.783+0000 7efceff96b80 1 --2- 172.21.15.1:0/1550793337 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5588d8a05dd0 0x5588d8a081f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.783+0000 7efceff96b80 1 --2- 172.21.15.1:0/1550793337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5588d8986350 0x5588d898b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.783+0000 7efceff96b80 1 --2- 172.21.15.1:0/1550793337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588d897ab80 0x5588d898e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.783+0000 7efceff96b80 1 --2- 172.21.15.1:0/1550793337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588d897b4a0 0x5588d898b270 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.783+0000 7efcc8c00700 0 client.25369 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:46.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.783+0000 7efceff96b80 1 -- 172.21.15.1:0/1550793337 >> 172.21.15.1:0/1550793337 conn(0x5588d88a9730 msgr2=0x5588d89816d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:46.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.783+0000 7efceff96b80 1 -- 172.21.15.1:0/1550793337 shutdown_connections 2024-04-26T01:52:46.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.783+0000 7efceff96b80 1 -- 172.21.15.1:0/1550793337 wait complete. 2024-04-26T01:52:46.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.784+0000 7efceff96b80 20 client.25369 trim_cache size 0 max 16384 2024-04-26T01:52:46.783 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T01:52:46.783 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 26, 2024-04-26T01:52:46.783 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24518, 2024-04-26T01:52:46.783 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 2, 2024-04-26T01:52:46.783 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-04-26T01:52:46.783 INFO:teuthology.orchestra.run.smithi001.stdout: "4a66b69e": { 2024-04-26T01:52:46.783 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 2, 2024-04-26T01:52:46.783 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-04-26T01:52:46.784 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:52:46.784 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-04-26T01:52:46.784 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:52:46.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T01:52:46.784 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-04-26T01:52:46.784 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 85.6, 2024-04-26T01:52:46.784 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-04-26T01:52:46.784 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-04-26T01:52:46.784 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-04-26T01:52:46.784 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:52:46.784 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-04-26T01:52:46.784 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:52:46.784 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:52:46.784 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:52:46.784 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:52:46.785 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:52:46.785 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:52:46.785 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T01:52:46.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.785+0000 7efcdfe00700 1 -- 172.21.15.1:0/2298840108 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efcc40a4000 msgr2=0x7efcc40a64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:46.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.785+0000 7efcdfe00700 1 --2- 172.21.15.1:0/2298840108 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efcc40a4000 0x7efcc40a64c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x5588d8977ec0 tx=0x7efcd80099e0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.785+0000 7efcdfe00700 1 -- 172.21.15.1:0/2298840108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efce0074fe0 msgr2=0x7efce019b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:46.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.785+0000 7efcdfe00700 1 --2- 172.21.15.1:0/2298840108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efce0074fe0 0x7efce019b260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efcd000e9f0 tx=0x7efcd000ed00 comp rx=0 tx=0).stop 2024-04-26T01:52:46.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.785+0000 7efcdfe00700 1 -- 172.21.15.1:0/2298840108 shutdown_connections 2024-04-26T01:52:46.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.785+0000 7efcdfe00700 1 --2- 172.21.15.1:0/2298840108 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efcc40a4000 0x7efcc40a64c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.785+0000 7efcdfe00700 1 --2- 172.21.15.1:0/2298840108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efce010f2a0 0x7efce019fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.785+0000 7efcdfe00700 1 --2- 172.21.15.1:0/2298840108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efce0074fe0 0x7efce019b260 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.785+0000 7efcdfe00700 1 --2- 172.21.15.1:0/2298840108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efce0075900 0x7efce019b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:46.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.785+0000 7efcdfe00700 1 -- 172.21.15.1:0/2298840108 >> 172.21.15.1:0/2298840108 conn(0x7efce00fe3a0 msgr2=0x7efce00ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:46.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.786+0000 7efcdfe00700 1 -- 172.21.15.1:0/2298840108 shutdown_connections 2024-04-26T01:52:46.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:52:46.786+0000 7efcdfe00700 1 -- 172.21.15.1:0/2298840108 wait complete. 2024-04-26T01:52:47.714 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4a66b69e', quiesce took 8.1 seconds. Will release after: 25.6 2024-04-26T01:52:48.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:47 smithi173 ceph-mon[32553]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-04-26T01:52:48.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:47 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:52:48.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:47 smithi001 ceph-mon[27822]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-04-26T01:52:48.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:47 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:52:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.159+0000 7ff6e9a00640 1 -- 172.21.15.1:0/1361371613 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6e410c350 msgr2=0x7ff6e410e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.159+0000 7ff6e9a00640 1 --2- 172.21.15.1:0/1361371613 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6e410c350 0x7ff6e410e7f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009be0 tx=0x7ff6dc02a750 comp rx=0 tx=0).stop 2024-04-26T01:52:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.159+0000 7ff6e9a00640 1 -- 172.21.15.1:0/1361371613 shutdown_connections 2024-04-26T01:52:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.159+0000 7ff6e9a00640 1 --2- 172.21.15.1:0/1361371613 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6e410c350 0x7ff6e410e7f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.159+0000 7ff6e9a00640 1 --2- 172.21.15.1:0/1361371613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6e4103850 0x7ff6e410bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.159+0000 7ff6e9a00640 1 --2- 172.21.15.1:0/1361371613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6e4102f30 0x7ff6e4103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.159+0000 7ff6e9a00640 1 -- 172.21.15.1:0/1361371613 >> 172.21.15.1:0/1361371613 conn(0x7ff6e40fc730 msgr2=0x7ff6e40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.159+0000 7ff6e9a00640 1 -- 172.21.15.1:0/1361371613 shutdown_connections 2024-04-26T01:52:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.159+0000 7ff6e9a00640 1 -- 172.21.15.1:0/1361371613 wait complete. 2024-04-26T01:52:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.160+0000 7ff6e9a00640 1 Processor -- start 2024-04-26T01:52:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.161+0000 7ff6e9a00640 1 -- start start 2024-04-26T01:52:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.161+0000 7ff6e9a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6e4102f30 0x7ff6e4198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.161+0000 7ff6e9a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6e4103850 0x7ff6e4198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.161+0000 7ff6e9a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6e410c350 0x7ff6e419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.161+0000 7ff6e9a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e4111fc0 con 0x7ff6e4103850 2024-04-26T01:52:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.161+0000 7ff6e9a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e4111e40 con 0x7ff6e410c350 2024-04-26T01:52:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.161+0000 7ff6e9a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e4112140 con 0x7ff6e4102f30 2024-04-26T01:52:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.161+0000 7ff6e3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6e4102f30 0x7ff6e4198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.161+0000 7ff6e9000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6e410c350 0x7ff6e419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.162+0000 7ff6e9000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6e410c350 0x7ff6e419cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:57632/0 (socket says 172.21.15.1:57632) 2024-04-26T01:52:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.162+0000 7ff6e9000640 1 -- 172.21.15.1:0/4201290940 learned_addr learned my addr 172.21.15.1:0/4201290940 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.162+0000 7ff6e3400640 1 --2- 172.21.15.1:0/4201290940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6e4103850 0x7ff6e4198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.162+0000 7ff6e9000640 1 -- 172.21.15.1:0/4201290940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6e4102f30 msgr2=0x7ff6e4198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.162+0000 7ff6e9000640 1 --2- 172.21.15.1:0/4201290940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6e4102f30 0x7ff6e4198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.162+0000 7ff6e9000640 1 -- 172.21.15.1:0/4201290940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6e4103850 msgr2=0x7ff6e4198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.162+0000 7ff6e9000640 1 --2- 172.21.15.1:0/4201290940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6e4103850 0x7ff6e4198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.162+0000 7ff6e9000640 1 -- 172.21.15.1:0/4201290940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e419d500 con 0x7ff6e410c350 2024-04-26T01:52:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.162+0000 7ff6e3400640 1 --2- 172.21.15.1:0/4201290940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6e4103850 0x7ff6e4198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:52:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.162+0000 7ff6e9000640 1 --2- 172.21.15.1:0/4201290940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6e410c350 0x7ff6e419cdd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009b30 tx=0x7ff6dc003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.163+0000 7ff6e0c00640 1 -- 172.21.15.1:0/4201290940 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc041020 con 0x7ff6e410c350 2024-04-26T01:52:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.163+0000 7ff6e0c00640 1 -- 172.21.15.1:0/4201290940 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6dc0318f0 con 0x7ff6e410c350 2024-04-26T01:52:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.163+0000 7ff6e0c00640 1 -- 172.21.15.1:0/4201290940 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc036040 con 0x7ff6e410c350 2024-04-26T01:52:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.163+0000 7ff6e9a00640 1 -- 172.21.15.1:0/4201290940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e419d730 con 0x7ff6e410c350 2024-04-26T01:52:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.163+0000 7ff6e9a00640 1 -- 172.21.15.1:0/4201290940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e4100ab0 con 0x7ff6e410c350 2024-04-26T01:52:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.167+0000 7ff6e0c00640 1 -- 172.21.15.1:0/4201290940 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7ff6dc044020 con 0x7ff6e410c350 2024-04-26T01:52:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.167+0000 7ff6ede3c740 1 Processor -- start 2024-04-26T01:52:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.168+0000 7ff6ede3c740 1 -- start start 2024-04-26T01:52:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.168+0000 7ff6ede3c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65da8f0 0x5607e65dacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.169+0000 7ff6ede3c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65db210 0x5607e65ede60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.169+0000 7ff6ede3c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65ee3a0 0x5607e65f0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.169+0000 7ff6ede3c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5607e6540900 con 0x5607e65da8f0 2024-04-26T01:52:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.169+0000 7ff6ede3c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5607e64e6ef0 con 0x5607e65db210 2024-04-26T01:52:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.169+0000 7ff6ede3c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5607e653c8c0 con 0x5607e65ee3a0 2024-04-26T01:52:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.169+0000 7ff6e3400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65da8f0 0x5607e65dacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.169+0000 7ff6e3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65db210 0x5607e65ede60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.169+0000 7ff6e3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65db210 0x5607e65ede60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:57644/0 (socket says 172.21.15.1:57644) 2024-04-26T01:52:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.169+0000 7ff6e3e00640 1 -- 172.21.15.1:0/3418439653 learned_addr learned my addr 172.21.15.1:0/3418439653 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.169+0000 7ff6e0c00640 1 --2- 172.21.15.1:0/4201290940 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ff6c80a3f30 0x7ff6c80a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.170+0000 7ff6e3e00640 1 -- 172.21.15.1:0/3418439653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65ee3a0 msgr2=0x5607e65f0790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T01:52:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.170+0000 7ff6e3e00640 1 --2- 172.21.15.1:0/3418439653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65ee3a0 0x5607e65f0790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.170+0000 7ff6e3e00640 1 -- 172.21.15.1:0/3418439653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65da8f0 msgr2=0x5607e65dacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.170+0000 7ff6e3e00640 1 --2- 172.21.15.1:0/3418439653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65da8f0 0x5607e65dacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.170+0000 7ff6e3e00640 1 -- 172.21.15.1:0/3418439653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607e6253210 con 0x5607e65db210 2024-04-26T01:52:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.170+0000 7ff6e0c00640 1 -- 172.21.15.1:0/4201290940 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7ff6dc0f1100 con 0x7ff6e410c350 2024-04-26T01:52:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.170+0000 7ff6e3400640 1 --2- 172.21.15.1:0/3418439653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65da8f0 0x5607e65dacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:52:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.170+0000 7ff6e9000640 1 --2- 172.21.15.1:0/4201290940 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ff6c80a3f30 0x7ff6c80a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.170+0000 7ff6e3e00640 1 --2- 172.21.15.1:0/3418439653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65db210 0x5607e65ede60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009cc0 tx=0x7ff6d4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a134a0770755f9f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.171+0000 7ff6d9600640 1 -- 172.21.15.1:0/3418439653 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4005220 con 0x5607e65db210 2024-04-26T01:52:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.171+0000 7ff6d9600640 1 -- 172.21.15.1:0/3418439653 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6d4004c50 con 0x5607e65db210 2024-04-26T01:52:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.171+0000 7ff6d9600640 1 -- 172.21.15.1:0/3418439653 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4019810 con 0x5607e65db210 2024-04-26T01:52:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.171+0000 7ff6ede3c740 1 -- 172.21.15.1:0/3418439653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65db210 msgr2=0x5607e65ede60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.171+0000 7ff6e9000640 1 --2- 172.21.15.1:0/4201290940 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ff6c80a3f30 0x7ff6c80a63f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0ee670 tx=0x7ff6dc003f40 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.171+0000 7ff6ede3c740 1 --2- 172.21.15.1:0/3418439653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65db210 0x5607e65ede60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009cc0 tx=0x7ff6d4011810 comp rx=0 tx=0).stop 2024-04-26T01:52:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.172+0000 7ff6ede3c740 1 -- 172.21.15.1:0/3418439653 shutdown_connections 2024-04-26T01:52:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.172+0000 7ff6ede3c740 1 --2- 172.21.15.1:0/3418439653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65ee3a0 0x5607e65f0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.172+0000 7ff6ede3c740 1 --2- 172.21.15.1:0/3418439653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65db210 0x5607e65ede60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.172+0000 7ff6ede3c740 1 --2- 172.21.15.1:0/3418439653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65da8f0 0x5607e65dacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.172+0000 7ff6ede3c740 1 -- 172.21.15.1:0/3418439653 >> 172.21.15.1:0/3418439653 conn(0x5607e65d6ec0 msgr2=0x5607e65d72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.172+0000 7ff6ede3c740 1 -- 172.21.15.1:0/3418439653 shutdown_connections 2024-04-26T01:52:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.172+0000 7ff6ede3c740 1 -- 172.21.15.1:0/3418439653 wait complete. 2024-04-26T01:52:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.172+0000 7ff6ede3c740 1 -- start start 2024-04-26T01:52:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.173+0000 7ff6ede3c740 1 Processor -- start 2024-04-26T01:52:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.173+0000 7ff6ede3c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65da8f0 0x5607e65eb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.173+0000 7ff6ede3c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65db210 0x5607e65eb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.173+0000 7ff6ede3c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65ee3a0 0x5607e65ebd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.173+0000 7ff6ede3c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5607e65f1010 con 0x5607e65db210 2024-04-26T01:52:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.173+0000 7ff6ede3c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5607e64fa310 con 0x5607e65da8f0 2024-04-26T01:52:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.173+0000 7ff6ede3c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5607e65f1190 con 0x5607e65ee3a0 2024-04-26T01:52:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.173+0000 7ff6ede3c740 10 client.0 _ll_register_callbacks cb 0x5607e64c8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.173+0000 7ff6e3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65da8f0 0x5607e65eb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.173+0000 7ff6e3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65da8f0 0x5607e65eb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:57646/0 (socket says 172.21.15.1:57646) 2024-04-26T01:52:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.173+0000 7ff6e3400640 1 -- 172.21.15.1:0/2357258531 learned_addr learned my addr 172.21.15.1:0/2357258531 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.173+0000 7ff6e3e00640 1 --2- 172.21.15.1:0/2357258531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65db210 0x5607e65eb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.173+0000 7ff6e3400640 1 --2- 172.21.15.1:0/2357258531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65ee3a0 0x5607e65ebd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.174+0000 7ff6e3400640 1 -- 172.21.15.1:0/2357258531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65ee3a0 msgr2=0x5607e65ebd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.174+0000 7ff6e3400640 1 --2- 172.21.15.1:0/2357258531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65ee3a0 0x5607e65ebd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.174+0000 7ff6e3400640 1 -- 172.21.15.1:0/2357258531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65db210 msgr2=0x5607e65eb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.174+0000 7ff6e3400640 1 --2- 172.21.15.1:0/2357258531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65db210 0x5607e65eb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.174+0000 7ff6e3400640 1 -- 172.21.15.1:0/2357258531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607e631f620 con 0x5607e65da8f0 2024-04-26T01:52:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.174+0000 7ff6e3400640 1 --2- 172.21.15.1:0/2357258531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65da8f0 0x5607e65eb2d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0013cd0 tx=0x7ff6d0009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41bf00ebbeb0109c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.174+0000 7ff6ce000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:52:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.174+0000 7ff6ce000640 1 -- 172.21.15.1:0/2357258531 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0011c10 con 0x5607e65da8f0 2024-04-26T01:52:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.174+0000 7ff6ce000640 1 -- 172.21.15.1:0/2357258531 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7ff6d0011db0 con 0x5607e65da8f0 2024-04-26T01:52:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.174+0000 7ff6ede3c740 1 -- 172.21.15.1:0/2357258531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6d4004c50 con 0x5607e65da8f0 2024-04-26T01:52:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.175+0000 7ff6ce000640 1 -- 172.21.15.1:0/2357258531 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff6d0025050 con 0x5607e65da8f0 2024-04-26T01:52:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.175+0000 7ff6ede3c740 10 client.25364 fetch_fsmap learned FSMap version 26 2024-04-26T01:52:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.175+0000 7ff6ede3c740 1 -- 172.21.15.1:0/2357258531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5607e65e0320 con 0x5607e65da8f0 2024-04-26T01:52:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.175+0000 7ff6ce000640 1 -- 172.21.15.1:0/2357258531 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6d4004c50 con 0x5607e65da8f0 2024-04-26T01:52:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.175+0000 7ff6ce000640 1 -- 172.21.15.1:0/2357258531 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff6d000f040 con 0x5607e65da8f0 2024-04-26T01:52:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.176+0000 7ff6ede3c740 10 client.25364 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:52:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.176+0000 7ff6ede3c740 10 client.25364 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:52:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.176+0000 7ff6ede3c740 20 client.25364 populate_metadata read hostname 'smithi001' 2024-04-26T01:52:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.176+0000 7ff6ede3c740 1 --2- 172.21.15.1:0/2357258531 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5607e65e1830 0x5607e65e3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.176+0000 7ff6ede3c740 4 client.25364 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:52:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.176+0000 7ff6ede3c740 1 -- 172.21.15.1:0/2357258531 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607e65548b0 con 0x5607e65e1830 2024-04-26T01:52:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.176+0000 7ff6e3e00640 1 --2- 172.21.15.1:0/2357258531 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5607e65e1830 0x5607e65e3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.177+0000 7ff6e3e00640 1 --2- 172.21.15.1:0/2357258531 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5607e65e1830 0x5607e65e3c20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19525d9977b4f67f server_cookie=5666930576e3af50 in_seq=0 out_seq=0 2024-04-26T01:52:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.177+0000 7ff6ce000640 10 client.25364 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.177+0000 7ff6ce000640 1 -- 172.21.15.1:0/2357258531 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5607e65548b0 con 0x5607e65e1830 2024-04-26T01:52:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.177+0000 7ff6ce000640 10 client.25364 handle_command_reply: tid=0 2024-04-26T01:52:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.177+0000 7ff6ede3c740 1 client.25364 shutdown 2024-04-26T01:52:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.178+0000 7ff6ede3c740 1 -- 172.21.15.1:0/2357258531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65da8f0 msgr2=0x5607e65eb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.178+0000 7ff6ede3c740 1 --2- 172.21.15.1:0/2357258531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65da8f0 0x5607e65eb2d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0013cd0 tx=0x7ff6d0009590 comp rx=0 tx=0).stop 2024-04-26T01:52:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.178+0000 7ff6ede3c740 1 -- 172.21.15.1:0/2357258531 shutdown_connections 2024-04-26T01:52:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.178+0000 7ff6ede3c740 1 --2- 172.21.15.1:0/2357258531 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5607e65e1830 0x5607e65e3c20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.178+0000 7ff6ede3c740 1 --2- 172.21.15.1:0/2357258531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65ee3a0 0x5607e65ebd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.178+0000 7ff6ede3c740 1 --2- 172.21.15.1:0/2357258531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65db210 0x5607e65eb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.178+0000 7ff6ede3c740 1 --2- 172.21.15.1:0/2357258531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65da8f0 0x5607e65eb2d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.178+0000 7ff6ede3c740 1 -- 172.21.15.1:0/2357258531 >> 172.21.15.1:0/2357258531 conn(0x5607e65d6ec0 msgr2=0x5607e65dcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.178+0000 7ff6ce000640 0 client.25364 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.178+0000 7ff6ede3c740 1 -- 172.21.15.1:0/2357258531 shutdown_connections 2024-04-26T01:52:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.178+0000 7ff6ede3c740 1 -- 172.21.15.1:0/2357258531 wait complete. 2024-04-26T01:52:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.178+0000 7ff6ede3c740 20 client.25364 trim_cache size 0 max 16384 2024-04-26T01:52:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.209+0000 7ff6ede3c740 1 Processor -- start 2024-04-26T01:52:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.209+0000 7ff6ede3c740 1 -- start start 2024-04-26T01:52:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.210+0000 7ff6ede3c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65da8f0 0x5607e65f9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.210+0000 7ff6ede3c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65db210 0x5607e65ea430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.210+0000 7ff6ede3c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65ee3a0 0x5607e6610550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.210+0000 7ff6ede3c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e41ae1c0 con 0x5607e65db210 2024-04-26T01:52:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.210+0000 7ff6ede3c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e41aba90 con 0x5607e65ee3a0 2024-04-26T01:52:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.210+0000 7ff6ede3c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e41a98e0 con 0x5607e65da8f0 2024-04-26T01:52:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.210+0000 7ff6e3400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65da8f0 0x5607e65f9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.210+0000 7ff6e3400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65da8f0 0x5607e65f9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:50626/0 (socket says 172.21.15.1:50626) 2024-04-26T01:52:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.210+0000 7ff6e3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65db210 0x5607e65ea430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.210+0000 7ff6e3400640 1 -- 172.21.15.1:0/1125299615 learned_addr learned my addr 172.21.15.1:0/1125299615 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.210+0000 7ff6e3400640 1 --2- 172.21.15.1:0/1125299615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65ee3a0 0x5607e6610550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.211+0000 7ff6e3400640 1 -- 172.21.15.1:0/1125299615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65ee3a0 msgr2=0x5607e6610550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.211+0000 7ff6e3400640 1 --2- 172.21.15.1:0/1125299615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65ee3a0 0x5607e6610550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.211+0000 7ff6e3400640 1 -- 172.21.15.1:0/1125299615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65db210 msgr2=0x5607e65ea430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.211+0000 7ff6e3400640 1 --2- 172.21.15.1:0/1125299615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65db210 0x5607e65ea430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.211+0000 7ff6e3400640 1 -- 172.21.15.1:0/1125299615 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607e65f0b00 con 0x5607e65da8f0 2024-04-26T01:52:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.211+0000 7ff6e3400640 1 --2- 172.21.15.1:0/1125299615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65da8f0 0x5607e65f9f60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0013fd0 tx=0x7ff6d0013990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c66eafee000900d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.211+0000 7ff6d8c00640 1 -- 172.21.15.1:0/1125299615 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0018040 con 0x5607e65da8f0 2024-04-26T01:52:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.211+0000 7ff6d8c00640 1 -- 172.21.15.1:0/1125299615 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6d0004490 con 0x5607e65da8f0 2024-04-26T01:52:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.211+0000 7ff6d8c00640 1 -- 172.21.15.1:0/1125299615 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d00035f0 con 0x5607e65da8f0 2024-04-26T01:52:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.212+0000 7ff6ede3c740 1 -- 172.21.15.1:0/1125299615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65da8f0 msgr2=0x5607e65f9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.212+0000 7ff6ede3c740 1 --2- 172.21.15.1:0/1125299615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65da8f0 0x5607e65f9f60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0013fd0 tx=0x7ff6d0013990 comp rx=0 tx=0).stop 2024-04-26T01:52:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.213+0000 7ff6ede3c740 1 -- 172.21.15.1:0/1125299615 shutdown_connections 2024-04-26T01:52:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.213+0000 7ff6ede3c740 1 --2- 172.21.15.1:0/1125299615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65ee3a0 0x5607e6610550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.213+0000 7ff6ede3c740 1 --2- 172.21.15.1:0/1125299615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65db210 0x5607e65ea430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.213+0000 7ff6ede3c740 1 --2- 172.21.15.1:0/1125299615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65da8f0 0x5607e65f9f60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.213+0000 7ff6ede3c740 1 -- 172.21.15.1:0/1125299615 >> 172.21.15.1:0/1125299615 conn(0x5607e65d6ec0 msgr2=0x5607e65dc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.213+0000 7ff6ede3c740 1 -- 172.21.15.1:0/1125299615 shutdown_connections 2024-04-26T01:52:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.213+0000 7ff6ede3c740 1 -- 172.21.15.1:0/1125299615 wait complete. 2024-04-26T01:52:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.213+0000 7ff6ede3c740 1 -- start start 2024-04-26T01:52:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.214+0000 7ff6ede3c740 1 Processor -- start 2024-04-26T01:52:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.214+0000 7ff6ede3c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65da8f0 0x5607e65f6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.214+0000 7ff6ede3c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65db210 0x5607e65f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.214+0000 7ff6ede3c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65ee3a0 0x5607e65e44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.214+0000 7ff6ede3c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e41a9f50 con 0x5607e65db210 2024-04-26T01:52:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.214+0000 7ff6ede3c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e41ab3c0 con 0x5607e65ee3a0 2024-04-26T01:52:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.214+0000 7ff6ede3c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e41aa620 con 0x5607e65da8f0 2024-04-26T01:52:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.214+0000 7ff6ede3c740 10 client.0 _ll_register_callbacks cb 0x7ff6d00138e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:52:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.215+0000 7ff6e3400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65da8f0 0x5607e65f6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.215+0000 7ff6e3400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65da8f0 0x5607e65f6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:50632/0 (socket says 172.21.15.1:50632) 2024-04-26T01:52:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.215+0000 7ff6e3400640 1 -- 172.21.15.1:0/816033558 learned_addr learned my addr 172.21.15.1:0/816033558 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:52:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.215+0000 7ff6e3400640 1 --2- 172.21.15.1:0/816033558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65ee3a0 0x5607e65e44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.215+0000 7ff6e3e00640 1 --2- 172.21.15.1:0/816033558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65db210 0x5607e65f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.215+0000 7ff6e3400640 1 -- 172.21.15.1:0/816033558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65ee3a0 msgr2=0x5607e65e44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.215+0000 7ff6e3400640 1 --2- 172.21.15.1:0/816033558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65ee3a0 0x5607e65e44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.215+0000 7ff6e3400640 1 -- 172.21.15.1:0/816033558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65db210 msgr2=0x5607e65f69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:52:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.215+0000 7ff6e3400640 1 --2- 172.21.15.1:0/816033558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65db210 0x5607e65f69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.215+0000 7ff6e3400640 1 -- 172.21.15.1:0/816033558 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607e65e4a10 con 0x5607e65da8f0 2024-04-26T01:52:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.216+0000 7ff6e3e00640 1 --2- 172.21.15.1:0/816033558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65db210 0x5607e65f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:52:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.216+0000 7ff6e3400640 1 --2- 172.21.15.1:0/816033558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65da8f0 0x5607e65f6470 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff6d00099f0 tx=0x7ff6d00093f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dba07346977ab79 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.216+0000 7ff6ce000640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:52:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.216+0000 7ff6ce000640 1 -- 172.21.15.1:0/816033558 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0005460 con 0x5607e65da8f0 2024-04-26T01:52:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.216+0000 7ff6ce000640 1 -- 172.21.15.1:0/816033558 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7ff6d00035f0 con 0x5607e65da8f0 2024-04-26T01:52:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.216+0000 7ff6ede3c740 1 -- 172.21.15.1:0/816033558 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6d0004490 con 0x5607e65da8f0 2024-04-26T01:52:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.217+0000 7ff6ce000640 1 -- 172.21.15.1:0/816033558 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff6d00037f0 con 0x5607e65da8f0 2024-04-26T01:52:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.217+0000 7ff6ede3c740 10 client.25384 fetch_fsmap learned FSMap version 26 2024-04-26T01:52:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.217+0000 7ff6ede3c740 1 -- 172.21.15.1:0/816033558 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5607e666c600 con 0x5607e65da8f0 2024-04-26T01:52:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.217+0000 7ff6ce000640 1 -- 172.21.15.1:0/816033558 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6d0004490 con 0x5607e65da8f0 2024-04-26T01:52:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.217+0000 7ff6ce000640 1 -- 172.21.15.1:0/816033558 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff6d0003990 con 0x5607e65da8f0 2024-04-26T01:52:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.217+0000 7ff6ede3c740 10 client.25384 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:52:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.217+0000 7ff6ede3c740 10 client.25384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:52:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.218+0000 7ff6ede3c740 20 client.25384 populate_metadata read hostname 'smithi001' 2024-04-26T01:52:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.218+0000 7ff6ede3c740 1 --2- 172.21.15.1:0/816033558 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5607e666ccc0 0x5607e666f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.218+0000 7ff6ede3c740 4 client.25384 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:52:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.218+0000 7ff6ede3c740 1 -- 172.21.15.1:0/816033558 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5607e654e320 con 0x5607e666ccc0 2024-04-26T01:52:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.218+0000 7ff6e3e00640 1 --2- 172.21.15.1:0/816033558 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5607e666ccc0 0x5607e666f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.219+0000 7ff6e3e00640 1 --2- 172.21.15.1:0/816033558 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5607e666ccc0 0x5607e666f0e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb35a3e8906427ed server_cookie=3a8938dd23c54ce in_seq=0 out_seq=0 2024-04-26T01:52:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.219+0000 7ff6ce000640 10 client.25384 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.227+0000 7ff6ce000640 1 -- 172.21.15.1:0/816033558 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5607e654e320 con 0x5607e666ccc0 2024-04-26T01:52:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.227+0000 7ff6ce000640 10 client.25384 handle_command_reply: tid=0 2024-04-26T01:52:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.227+0000 7ff6ede3c740 1 client.25384 shutdown 2024-04-26T01:52:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.228+0000 7ff6ede3c740 1 -- 172.21.15.1:0/816033558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65da8f0 msgr2=0x5607e65f6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.228+0000 7ff6ede3c740 1 --2- 172.21.15.1:0/816033558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65da8f0 0x5607e65f6470 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff6d00099f0 tx=0x7ff6d00093f0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.228+0000 7ff6ede3c740 1 -- 172.21.15.1:0/816033558 shutdown_connections 2024-04-26T01:52:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.228+0000 7ff6ede3c740 1 --2- 172.21.15.1:0/816033558 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5607e666ccc0 0x5607e666f0e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.228+0000 7ff6ede3c740 1 --2- 172.21.15.1:0/816033558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607e65ee3a0 0x5607e65e44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.228+0000 7ff6ede3c740 1 --2- 172.21.15.1:0/816033558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607e65db210 0x5607e65f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.228+0000 7ff6ede3c740 1 --2- 172.21.15.1:0/816033558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5607e65da8f0 0x5607e65f6470 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.228+0000 7ff6ce000640 0 client.25384 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:52:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.228+0000 7ff6ede3c740 1 -- 172.21.15.1:0/816033558 >> 172.21.15.1:0/816033558 conn(0x5607e65d6ec0 msgr2=0x5607e65dcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.229+0000 7ff6ede3c740 1 -- 172.21.15.1:0/816033558 shutdown_connections 2024-04-26T01:52:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.229+0000 7ff6ede3c740 1 -- 172.21.15.1:0/816033558 wait complete. 2024-04-26T01:52:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.229+0000 7ff6ede3c740 20 client.25384 trim_cache size 0 max 16384 2024-04-26T01:52:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:52:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:52:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:52:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:52:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.230+0000 7ff6e9a00640 1 -- 172.21.15.1:0/4201290940 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ff6c80a3f30 msgr2=0x7ff6c80a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.230+0000 7ff6e9a00640 1 --2- 172.21.15.1:0/4201290940 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ff6c80a3f30 0x7ff6c80a63f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0ee670 tx=0x7ff6dc003f40 comp rx=0 tx=0).stop 2024-04-26T01:52:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.230+0000 7ff6e9a00640 1 -- 172.21.15.1:0/4201290940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6e410c350 msgr2=0x7ff6e419cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.230+0000 7ff6e9a00640 1 --2- 172.21.15.1:0/4201290940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6e410c350 0x7ff6e419cdd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009b30 tx=0x7ff6dc003ba0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.230+0000 7ff6e9a00640 1 -- 172.21.15.1:0/4201290940 shutdown_connections 2024-04-26T01:52:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.230+0000 7ff6e9a00640 1 --2- 172.21.15.1:0/4201290940 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ff6c80a3f30 0x7ff6c80a63f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.230+0000 7ff6e9a00640 1 --2- 172.21.15.1:0/4201290940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6e410c350 0x7ff6e419cdd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.230+0000 7ff6e9a00640 1 --2- 172.21.15.1:0/4201290940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6e4103850 0x7ff6e4198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.230+0000 7ff6e9a00640 1 --2- 172.21.15.1:0/4201290940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6e4102f30 0x7ff6e4198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.230+0000 7ff6e9a00640 1 -- 172.21.15.1:0/4201290940 >> 172.21.15.1:0/4201290940 conn(0x7ff6e40fc730 msgr2=0x7ff6e40fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.231+0000 7ff6e9a00640 1 -- 172.21.15.1:0/4201290940 shutdown_connections 2024-04-26T01:52:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:52:48.231+0000 7ff6e9a00640 1 -- 172.21.15.1:0/4201290940 wait complete. 2024-04-26T01:52:48.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:47 smithi097 ceph-mon[32853]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-04-26T01:52:48.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:47 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:52:50.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:49 smithi173 ceph-mon[32553]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-26T01:52:50.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:49 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:50.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:49 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:50.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:49 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:52:50.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:49 smithi001 ceph-mon[27822]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-26T01:52:50.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:49 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:50.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:49 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:50.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:49 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:52:50.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:49 smithi097 ceph-mon[32853]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-26T01:52:50.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:49 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:50.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:49 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:50.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:49 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:52:52.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:51 smithi173 ceph-mon[32553]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-26T01:52:52.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:51 smithi001 ceph-mon[27822]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-26T01:52:52.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:51 smithi097 ceph-mon[32853]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-26T01:52:54.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:53 smithi173 ceph-mon[32553]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-26T01:52:54.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:53 smithi001 ceph-mon[27822]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-26T01:52:54.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:53 smithi097 ceph-mon[32853]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-26T01:52:56.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:55 smithi173 ceph-mon[32553]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-26T01:52:56.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:55 smithi001 ceph-mon[27822]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-26T01:52:56.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:55 smithi097 ceph-mon[32853]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-26T01:52:58.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:57 smithi173 ceph-mon[32553]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-26T01:52:58.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:57 smithi001 ceph-mon[27822]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-26T01:52:58.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:57 smithi097 ceph-mon[32853]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-26T01:53:00.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:52:59 smithi173 ceph-mon[32553]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-26T01:53:00.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:52:59 smithi001 ceph-mon[27822]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-26T01:53:00.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:52:59 smithi097 ceph-mon[32853]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-26T01:53:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.366+0000 7f93ece00640 1 -- 172.21.15.1:0/2154623838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e810e620 msgr2=0x7f93e8110a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.366+0000 7f93ece00640 1 --2- 172.21.15.1:0/2154623838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e810e620 0x7f93e8110a10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009ca0 tx=0x7f93e002a950 comp rx=0 tx=0).stop 2024-04-26T01:53:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.367+0000 7f93ece00640 1 -- 172.21.15.1:0/2154623838 shutdown_connections 2024-04-26T01:53:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.367+0000 7f93ece00640 1 --2- 172.21.15.1:0/2154623838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e810e620 0x7f93e8110a10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.367+0000 7f93ece00640 1 --2- 172.21.15.1:0/2154623838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93e8074c20 0x7f93e810e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.367+0000 7f93ece00640 1 --2- 172.21.15.1:0/2154623838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8074300 0x7f93e80746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.367+0000 7f93ece00640 1 -- 172.21.15.1:0/2154623838 >> 172.21.15.1:0/2154623838 conn(0x7f93e80fd890 msgr2=0x7f93e80ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.367+0000 7f93ece00640 1 -- 172.21.15.1:0/2154623838 shutdown_connections 2024-04-26T01:53:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.367+0000 7f93ece00640 1 -- 172.21.15.1:0/2154623838 wait complete. 2024-04-26T01:53:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.368+0000 7f93ece00640 1 Processor -- start 2024-04-26T01:53:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.368+0000 7f93ece00640 1 -- start start 2024-04-26T01:53:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.369+0000 7f93ece00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e8074300 0x7f93e819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.369+0000 7f93ece00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8074c20 0x7f93e819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.369+0000 7f93ece00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93e810e620 0x7f93e819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.369+0000 7f93ece00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f93e81141b0 con 0x7f93e8074300 2024-04-26T01:53:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.369+0000 7f93ece00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f93e8114030 con 0x7f93e810e620 2024-04-26T01:53:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.369+0000 7f93ece00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f93e8114330 con 0x7f93e8074c20 2024-04-26T01:53:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.369+0000 7f93e7400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e8074300 0x7f93e819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.369+0000 7f93e7400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e8074300 0x7f93e819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47216/0 (socket says 172.21.15.1:47216) 2024-04-26T01:53:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.369+0000 7f93e7400640 1 -- 172.21.15.1:0/3234095046 learned_addr learned my addr 172.21.15.1:0/3234095046 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.369+0000 7f93e6a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8074c20 0x7f93e819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.369+0000 7f93e7e00640 1 --2- 172.21.15.1:0/3234095046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93e810e620 0x7f93e819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.370+0000 7f93e7400640 1 -- 172.21.15.1:0/3234095046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8074c20 msgr2=0x7f93e819acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.370+0000 7f93e7400640 1 --2- 172.21.15.1:0/3234095046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8074c20 0x7f93e819acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.370+0000 7f93e7400640 1 -- 172.21.15.1:0/3234095046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93e810e620 msgr2=0x7f93e819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.370+0000 7f93e7400640 1 --2- 172.21.15.1:0/3234095046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93e810e620 0x7f93e819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.370+0000 7f93e7400640 1 -- 172.21.15.1:0/3234095046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e819f710 con 0x7f93e8074300 2024-04-26T01:53:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.370+0000 7f93e7400640 1 --2- 172.21.15.1:0/3234095046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e8074300 0x7f93e819a770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f93d800ea60 tx=0x7f93d800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.371+0000 7f93dfe00640 1 -- 172.21.15.1:0/3234095046 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d800cde0 con 0x7f93e8074300 2024-04-26T01:53:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.371+0000 7f93dfe00640 1 -- 172.21.15.1:0/3234095046 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93d80035f0 con 0x7f93e8074300 2024-04-26T01:53:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.371+0000 7f93dfe00640 1 -- 172.21.15.1:0/3234095046 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d8010690 con 0x7f93e8074300 2024-04-26T01:53:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.371+0000 7f93ece00640 1 -- 172.21.15.1:0/3234095046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e819fa00 con 0x7f93e8074300 2024-04-26T01:53:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.371+0000 7f93ece00640 1 -- 172.21.15.1:0/3234095046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e81a7290 con 0x7f93e8074300 2024-04-26T01:53:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.375+0000 7f93dfe00640 1 -- 172.21.15.1:0/3234095046 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) ==== 138929+0+0 (secure 0 0 0) 0x7f93d80037a0 con 0x7f93e8074300 2024-04-26T01:53:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.376+0000 7f93f123b740 1 Processor -- start 2024-04-26T01:53:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.376+0000 7f93f123b740 1 -- start start 2024-04-26T01:53:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.376+0000 7f93f123b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f985029b0 0x559f98502d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.376+0000 7f93dfe00640 1 --2- 172.21.15.1:0/3234095046 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f93cc0a3f30 0x7f93cc0a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.376+0000 7f93f123b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985032d0 0x559f9850d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.376+0000 7f93f123b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f9850db90 0x559f9850ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.376+0000 7f93f123b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559f98468900 con 0x559f985029b0 2024-04-26T01:53:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.376+0000 7f93f123b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559f9840eef0 con 0x559f9850db90 2024-04-26T01:53:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.376+0000 7f93f123b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559f984648c0 con 0x559f985032d0 2024-04-26T01:53:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.377+0000 7f93dfe00640 1 -- 172.21.15.1:0/3234095046 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f93d80d0d00 con 0x7f93e8074300 2024-04-26T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.377+0000 7f93e7e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f985029b0 0x559f98502d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.377+0000 7f93e7e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f985029b0 0x559f98502d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47226/0 (socket says 172.21.15.1:47226) 2024-04-26T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.377+0000 7f93e7e00640 1 -- 172.21.15.1:0/239346408 learned_addr learned my addr 172.21.15.1:0/239346408 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.377+0000 7f93e7400640 1 --2- 172.21.15.1:0/3234095046 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f93cc0a3f30 0x7f93cc0a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.377+0000 7f93e7e00640 1 --2- 172.21.15.1:0/239346408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f9850db90 0x559f9850ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.378+0000 7f93e6a00640 1 --2- 172.21.15.1:0/239346408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985032d0 0x559f9850d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.378+0000 7f93e7e00640 1 -- 172.21.15.1:0/239346408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985032d0 msgr2=0x559f9850d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.378+0000 7f93e7e00640 1 --2- 172.21.15.1:0/239346408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985032d0 0x559f9850d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.378+0000 7f93e7e00640 1 -- 172.21.15.1:0/239346408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f9850db90 msgr2=0x559f9850ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.378+0000 7f93e7e00640 1 --2- 172.21.15.1:0/239346408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f9850db90 0x559f9850ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.378+0000 7f93e7e00640 1 -- 172.21.15.1:0/239346408 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f9817b210 con 0x559f985029b0 2024-04-26T01:53:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.378+0000 7f93e7e00640 1 --2- 172.21.15.1:0/239346408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f985029b0 0x559f98502d90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x559f98500050 tx=0x7f93e0009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbe575735b764ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.378+0000 7f93e7400640 1 --2- 172.21.15.1:0/3234095046 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f93cc0a3f30 0x7f93cc0a63f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f93d80ce270 tx=0x7f93d800ef90 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.378+0000 7f93dcc00640 1 -- 172.21.15.1:0/239346408 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0034040 con 0x559f985029b0 2024-04-26T01:53:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.378+0000 7f93dcc00640 1 -- 172.21.15.1:0/239346408 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93e0003e20 con 0x559f985029b0 2024-04-26T01:53:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.378+0000 7f93dcc00640 1 -- 172.21.15.1:0/239346408 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0004290 con 0x559f985029b0 2024-04-26T01:53:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.379+0000 7f93f123b740 1 -- 172.21.15.1:0/239346408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f985029b0 msgr2=0x559f98502d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.379+0000 7f93f123b740 1 --2- 172.21.15.1:0/239346408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f985029b0 0x559f98502d90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x559f98500050 tx=0x7f93e0009360 comp rx=0 tx=0).stop 2024-04-26T01:53:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.380+0000 7f93f123b740 1 -- 172.21.15.1:0/239346408 shutdown_connections 2024-04-26T01:53:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.380+0000 7f93f123b740 1 --2- 172.21.15.1:0/239346408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f9850db90 0x559f9850ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.380+0000 7f93f123b740 1 --2- 172.21.15.1:0/239346408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985032d0 0x559f9850d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.380+0000 7f93f123b740 1 --2- 172.21.15.1:0/239346408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f985029b0 0x559f98502d90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.380+0000 7f93f123b740 1 -- 172.21.15.1:0/239346408 >> 172.21.15.1:0/239346408 conn(0x559f984feec0 msgr2=0x559f984ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.380+0000 7f93f123b740 1 -- 172.21.15.1:0/239346408 shutdown_connections 2024-04-26T01:53:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.380+0000 7f93f123b740 1 -- 172.21.15.1:0/239346408 wait complete. 2024-04-26T01:53:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.380+0000 7f93f123b740 1 -- start start 2024-04-26T01:53:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.381+0000 7f93f123b740 1 Processor -- start 2024-04-26T01:53:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.381+0000 7f93f123b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f985029b0 0x559f9850abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.381+0000 7f93f123b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985032d0 0x559f9850b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.381+0000 7f93f123b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f9850db90 0x559f9850b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.381+0000 7f93f123b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559f98518f70 con 0x559f985029b0 2024-04-26T01:53:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.381+0000 7f93f123b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559f98422310 con 0x559f9850db90 2024-04-26T01:53:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.381+0000 7f93f123b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559f985190f0 con 0x559f985032d0 2024-04-26T01:53:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.381+0000 7f93f123b740 10 client.0 _ll_register_callbacks cb 0x559f983f0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:53:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.382+0000 7f93e7e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f985029b0 0x559f9850abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.382+0000 7f93e7e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f985029b0 0x559f9850abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47236/0 (socket says 172.21.15.1:47236) 2024-04-26T01:53:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.382+0000 7f93e7e00640 1 -- 172.21.15.1:0/380127500 learned_addr learned my addr 172.21.15.1:0/380127500 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.382+0000 7f93e6a00640 1 --2- 172.21.15.1:0/380127500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985032d0 0x559f9850b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.382+0000 7f93e7e00640 1 --2- 172.21.15.1:0/380127500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f9850db90 0x559f9850b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.382+0000 7f93e6a00640 1 -- 172.21.15.1:0/380127500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f9850db90 msgr2=0x559f9850b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.382+0000 7f93e6a00640 1 --2- 172.21.15.1:0/380127500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f9850db90 0x559f9850b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.382+0000 7f93e6a00640 1 -- 172.21.15.1:0/380127500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f985029b0 msgr2=0x559f9850abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.382+0000 7f93e6a00640 1 --2- 172.21.15.1:0/380127500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f985029b0 0x559f9850abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.382+0000 7f93e6a00640 1 -- 172.21.15.1:0/380127500 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f98247620 con 0x559f985032d0 2024-04-26T01:53:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.382+0000 7f93e7e00640 1 --2- 172.21.15.1:0/380127500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f9850db90 0x559f9850b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:53:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.383+0000 7f93e6a00640 1 --2- 172.21.15.1:0/380127500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985032d0 0x559f9850b120 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f93e810fab0 tx=0x7f93d4012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2a469d4c5b26199 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.383+0000 7f93d1600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:53:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.383+0000 7f93d1600640 1 -- 172.21.15.1:0/380127500 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4004050 con 0x559f985032d0 2024-04-26T01:53:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.383+0000 7f93f123b740 1 -- 172.21.15.1:0/380127500 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93e0003e20 con 0x559f985032d0 2024-04-26T01:53:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.383+0000 7f93d1600640 1 -- 172.21.15.1:0/380127500 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f93d40041f0 con 0x559f985032d0 2024-04-26T01:53:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.383+0000 7f93d1600640 1 -- 172.21.15.1:0/380127500 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f93d40043f0 con 0x559f985032d0 2024-04-26T01:53:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.383+0000 7f93f123b740 10 client.25396 fetch_fsmap learned FSMap version 26 2024-04-26T01:53:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.383+0000 7f93f123b740 1 -- 172.21.15.1:0/380127500 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559f98525340 con 0x559f985032d0 2024-04-26T01:53:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.384+0000 7f93d1600640 1 -- 172.21.15.1:0/380127500 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93e0003e20 con 0x559f985032d0 2024-04-26T01:53:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.384+0000 7f93d1600640 1 -- 172.21.15.1:0/380127500 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f93d4003ac0 con 0x559f985032d0 2024-04-26T01:53:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.384+0000 7f93f123b740 10 client.25396 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:53:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.384+0000 7f93f123b740 10 client.25396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:53:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.384+0000 7f93f123b740 20 client.25396 populate_metadata read hostname 'smithi001' 2024-04-26T01:53:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.384+0000 7f93f123b740 1 --2- 172.21.15.1:0/380127500 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559f985267b0 0x559f98528ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.384+0000 7f93f123b740 4 client.25396 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:53:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.384+0000 7f93f123b740 1 -- 172.21.15.1:0/380127500 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f9847c8b0 con 0x559f985267b0 2024-04-26T01:53:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.384+0000 7f93e7e00640 1 --2- 172.21.15.1:0/380127500 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559f985267b0 0x559f98528ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.385+0000 7f93d1600640 10 client.25396 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.385+0000 7f93e7e00640 1 --2- 172.21.15.1:0/380127500 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559f985267b0 0x559f98528ba0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb2812c01b30ba50 server_cookie=9a4e29f71a1431de in_seq=0 out_seq=0 2024-04-26T01:53:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.386+0000 7f93d1600640 1 -- 172.21.15.1:0/380127500 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f9847c8b0 con 0x559f985267b0 2024-04-26T01:53:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.386+0000 7f93d1600640 10 client.25396 handle_command_reply: tid=0 2024-04-26T01:53:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.386+0000 7f93f123b740 1 client.25396 shutdown 2024-04-26T01:53:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.386+0000 7f93f123b740 1 -- 172.21.15.1:0/380127500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985032d0 msgr2=0x559f9850b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.386+0000 7f93f123b740 1 --2- 172.21.15.1:0/380127500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985032d0 0x559f9850b120 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f93e810fab0 tx=0x7f93d4012b60 comp rx=0 tx=0).stop 2024-04-26T01:53:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.386+0000 7f93f123b740 1 -- 172.21.15.1:0/380127500 shutdown_connections 2024-04-26T01:53:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.386+0000 7f93f123b740 1 --2- 172.21.15.1:0/380127500 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559f985267b0 0x559f98528ba0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.386+0000 7f93f123b740 1 --2- 172.21.15.1:0/380127500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f9850db90 0x559f9850b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.386+0000 7f93f123b740 1 --2- 172.21.15.1:0/380127500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985032d0 0x559f9850b120 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.386+0000 7f93f123b740 1 --2- 172.21.15.1:0/380127500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f985029b0 0x559f9850abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.386+0000 7f93f123b740 1 -- 172.21.15.1:0/380127500 >> 172.21.15.1:0/380127500 conn(0x559f984feec0 msgr2=0x559f98504c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.386+0000 7f93d1600640 0 client.25396 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.386+0000 7f93f123b740 1 -- 172.21.15.1:0/380127500 shutdown_connections 2024-04-26T01:53:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.386+0000 7f93f123b740 1 -- 172.21.15.1:0/380127500 wait complete. 2024-04-26T01:53:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.387+0000 7f93f123b740 20 client.25396 trim_cache size 0 max 16384 2024-04-26T01:53:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.402+0000 7f93f123b740 1 Processor -- start 2024-04-26T01:53:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.403+0000 7f93f123b740 1 -- start start 2024-04-26T01:53:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.403+0000 7f93f123b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985029b0 0x559f98521e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.403+0000 7f93f123b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f985032d0 0x559f984fe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.403+0000 7f93f123b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f9850db90 0x559f9852e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.403+0000 7f93f123b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f93e81ae220 con 0x559f985032d0 2024-04-26T01:53:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.403+0000 7f93f123b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f93e81abaf0 con 0x559f9850db90 2024-04-26T01:53:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.403+0000 7f93f123b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f93e81a9940 con 0x559f985029b0 2024-04-26T01:53:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.403+0000 7f93e6a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f985032d0 0x559f984fe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.403+0000 7f93e6a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f985032d0 0x559f984fe7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47246/0 (socket says 172.21.15.1:47246) 2024-04-26T01:53:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.403+0000 7f93e6a00640 1 -- 172.21.15.1:0/3342857395 learned_addr learned my addr 172.21.15.1:0/3342857395 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.403+0000 7f93e7e00640 1 --2- 172.21.15.1:0/3342857395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f9850db90 0x559f9852e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.404+0000 7f93e7e00640 1 --2- 172.21.15.1:0/3342857395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985029b0 0x559f98521e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.404+0000 7f93e6a00640 1 -- 172.21.15.1:0/3342857395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985029b0 msgr2=0x559f98521e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.404+0000 7f93e6a00640 1 --2- 172.21.15.1:0/3342857395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985029b0 0x559f98521e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.404+0000 7f93e6a00640 1 -- 172.21.15.1:0/3342857395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f9850db90 msgr2=0x559f9852e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.404+0000 7f93e6a00640 1 --2- 172.21.15.1:0/3342857395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f9850db90 0x559f9852e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.404+0000 7f93e6a00640 1 -- 172.21.15.1:0/3342857395 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f984f9490 con 0x559f985032d0 2024-04-26T01:53:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.404+0000 7f93e6a00640 1 --2- 172.21.15.1:0/3342857395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f985032d0 0x559f984fe7a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f93d4003f10 tx=0x7f93d40257e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d6b9492ee39de2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.404+0000 7f93d3e00640 1 -- 172.21.15.1:0/3342857395 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4003fe0 con 0x559f985032d0 2024-04-26T01:53:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.405+0000 7f93d3e00640 1 -- 172.21.15.1:0/3342857395 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93d4004180 con 0x559f985032d0 2024-04-26T01:53:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.405+0000 7f93d3e00640 1 -- 172.21.15.1:0/3342857395 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d40035f0 con 0x559f985032d0 2024-04-26T01:53:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.405+0000 7f93f123b740 1 -- 172.21.15.1:0/3342857395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f985032d0 msgr2=0x559f984fe7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.405+0000 7f93f123b740 1 --2- 172.21.15.1:0/3342857395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f985032d0 0x559f984fe7a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f93d4003f10 tx=0x7f93d40257e0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.406+0000 7f93f123b740 1 -- 172.21.15.1:0/3342857395 shutdown_connections 2024-04-26T01:53:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.406+0000 7f93f123b740 1 --2- 172.21.15.1:0/3342857395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f9850db90 0x559f9852e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.406+0000 7f93f123b740 1 --2- 172.21.15.1:0/3342857395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f985032d0 0x559f984fe7a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.406+0000 7f93f123b740 1 --2- 172.21.15.1:0/3342857395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985029b0 0x559f98521e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.406+0000 7f93f123b740 1 -- 172.21.15.1:0/3342857395 >> 172.21.15.1:0/3342857395 conn(0x559f984feec0 msgr2=0x559f9850cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.406+0000 7f93f123b740 1 -- 172.21.15.1:0/3342857395 shutdown_connections 2024-04-26T01:53:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.406+0000 7f93f123b740 1 -- 172.21.15.1:0/3342857395 wait complete. 2024-04-26T01:53:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.406+0000 7f93f123b740 1 -- start start 2024-04-26T01:53:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.406+0000 7f93f123b740 1 Processor -- start 2024-04-26T01:53:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.407+0000 7f93f123b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985029b0 0x559f9851bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.407+0000 7f93f123b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f985032d0 0x559f9851c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.407+0000 7f93f123b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f9850db90 0x559f9851ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.407+0000 7f93f123b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f93e81a9fb0 con 0x559f9850db90 2024-04-26T01:53:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.407+0000 7f93f123b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f93e81ab420 con 0x559f985032d0 2024-04-26T01:53:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.407+0000 7f93f123b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f93e81aa680 con 0x559f985029b0 2024-04-26T01:53:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.407+0000 7f93f123b740 10 client.0 _ll_register_callbacks cb 0x7f93d4009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:53:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.407+0000 7f93e7e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985029b0 0x559f9851bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.407+0000 7f93e7e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985029b0 0x559f9851bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:60290/0 (socket says 172.21.15.1:60290) 2024-04-26T01:53:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.407+0000 7f93e7e00640 1 -- 172.21.15.1:0/4104569906 learned_addr learned my addr 172.21.15.1:0/4104569906 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.407+0000 7f93e7e00640 1 --2- 172.21.15.1:0/4104569906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f9850db90 0x559f9851ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.408+0000 7f93e6a00640 1 --2- 172.21.15.1:0/4104569906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f985032d0 0x559f9851c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.408+0000 7f93e7e00640 1 -- 172.21.15.1:0/4104569906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f985032d0 msgr2=0x559f9851c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.408+0000 7f93e7e00640 1 --2- 172.21.15.1:0/4104569906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f985032d0 0x559f9851c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.408+0000 7f93e7e00640 1 -- 172.21.15.1:0/4104569906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f9850db90 msgr2=0x559f9851ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.408+0000 7f93e7e00640 1 --2- 172.21.15.1:0/4104569906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f9850db90 0x559f9851ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.408+0000 7f93e7e00640 1 -- 172.21.15.1:0/4104569906 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f9858bf20 con 0x559f985029b0 2024-04-26T01:53:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.408+0000 7f93e7e00640 1 --2- 172.21.15.1:0/4104569906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985029b0 0x559f9851bf80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f93e0004ad0 tx=0x7f93d4009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea1899eec1e54cb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.408+0000 7f93d1600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:53:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.408+0000 7f93d1600640 1 -- 172.21.15.1:0/4104569906 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0034040 con 0x559f985029b0 2024-04-26T01:53:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.408+0000 7f93d1600640 1 -- 172.21.15.1:0/4104569906 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f93e0003b70 con 0x559f985029b0 2024-04-26T01:53:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.408+0000 7f93f123b740 1 -- 172.21.15.1:0/4104569906 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93d4004180 con 0x559f985029b0 2024-04-26T01:53:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.409+0000 7f93d1600640 1 -- 172.21.15.1:0/4104569906 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f93e0003d70 con 0x559f985029b0 2024-04-26T01:53:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.409+0000 7f93f123b740 10 client.25402 fetch_fsmap learned FSMap version 26 2024-04-26T01:53:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.409+0000 7f93f123b740 1 -- 172.21.15.1:0/4104569906 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559f9858d1d0 con 0x559f985029b0 2024-04-26T01:53:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.409+0000 7f93d1600640 1 -- 172.21.15.1:0/4104569906 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93e005e020 con 0x559f985029b0 2024-04-26T01:53:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.409+0000 7f93d1600640 1 -- 172.21.15.1:0/4104569906 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f93e0056280 con 0x559f985029b0 2024-04-26T01:53:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.409+0000 7f93f123b740 10 client.25402 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T01:53:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.410+0000 7f93f123b740 10 client.25402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:53:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.410+0000 7f93f123b740 20 client.25402 populate_metadata read hostname 'smithi001' 2024-04-26T01:53:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.410+0000 7f93f123b740 1 --2- 172.21.15.1:0/4104569906 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559f9858d4b0 0x559f9858f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.410+0000 7f93f123b740 4 client.25402 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:53:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.410+0000 7f93f123b740 1 -- 172.21.15.1:0/4104569906 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f98476320 con 0x559f9858d4b0 2024-04-26T01:53:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.410+0000 7f93e6a00640 1 --2- 172.21.15.1:0/4104569906 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559f9858d4b0 0x559f9858f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.411+0000 7f93d1600640 10 client.25402 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:01.411+0000 7f93e6a00640 1 --2- 172.21.15.1:0/4104569906 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559f9858d4b0 0x559f9858f8a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9256e098c20d98e1 server_cookie=677b455b7e219157 in_seq=0 out_seq=0 2024-04-26T01:53:02.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:01 smithi173 ceph-mon[32553]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-26T01:53:02.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:01 smithi001 ceph-mon[27822]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-26T01:53:02.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:01 smithi097 ceph-mon[32853]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-26T01:53:04.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:03 smithi173 ceph-mon[32553]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 85 B/s wr, 4 op/s 2024-04-26T01:53:04.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:03 smithi001 ceph-mon[27822]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 85 B/s wr, 4 op/s 2024-04-26T01:53:04.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:03 smithi097 ceph-mon[32853]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 85 B/s wr, 4 op/s 2024-04-26T01:53:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.644+0000 7f93d1600640 1 -- 172.21.15.1:0/4104569906 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f98476320 con 0x559f9858d4b0 2024-04-26T01:53:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.644+0000 7f93d1600640 10 client.25402 handle_command_reply: tid=0 2024-04-26T01:53:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.644+0000 7f93f123b740 1 client.25402 shutdown 2024-04-26T01:53:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.645+0000 7f93f123b740 1 -- 172.21.15.1:0/4104569906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985029b0 msgr2=0x559f9851bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.645+0000 7f93f123b740 1 --2- 172.21.15.1:0/4104569906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985029b0 0x559f9851bf80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f93e0004ad0 tx=0x7f93d4009ca0 comp rx=0 tx=0).stop 2024-04-26T01:53:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.645+0000 7f93f123b740 1 -- 172.21.15.1:0/4104569906 shutdown_connections 2024-04-26T01:53:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.645+0000 7f93f123b740 1 --2- 172.21.15.1:0/4104569906 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559f9858d4b0 0x559f9858f8a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.645+0000 7f93f123b740 1 --2- 172.21.15.1:0/4104569906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f9850db90 0x559f9851ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.645+0000 7f93f123b740 1 --2- 172.21.15.1:0/4104569906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f985032d0 0x559f9851c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.645+0000 7f93f123b740 1 --2- 172.21.15.1:0/4104569906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f985029b0 0x559f9851bf80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.645+0000 7f93f123b740 1 -- 172.21.15.1:0/4104569906 >> 172.21.15.1:0/4104569906 conn(0x559f984feec0 msgr2=0x559f98504360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.645+0000 7f93d1600640 0 client.25402 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.645+0000 7f93f123b740 1 -- 172.21.15.1:0/4104569906 shutdown_connections 2024-04-26T01:53:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.645+0000 7f93f123b740 1 -- 172.21.15.1:0/4104569906 wait complete. 2024-04-26T01:53:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.645+0000 7f93f123b740 20 client.25402 trim_cache size 0 max 16384 2024-04-26T01:53:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:53:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:53:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:53:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:53:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.646+0000 7f93ece00640 1 -- 172.21.15.1:0/3234095046 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f93cc0a3f30 msgr2=0x7f93cc0a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.646+0000 7f93ece00640 1 --2- 172.21.15.1:0/3234095046 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f93cc0a3f30 0x7f93cc0a63f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f93d80ce270 tx=0x7f93d800ef90 comp rx=0 tx=0).stop 2024-04-26T01:53:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.646+0000 7f93ece00640 1 -- 172.21.15.1:0/3234095046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e8074300 msgr2=0x7f93e819a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.646+0000 7f93ece00640 1 --2- 172.21.15.1:0/3234095046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e8074300 0x7f93e819a770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f93d800ea60 tx=0x7f93d800ef20 comp rx=0 tx=0).stop 2024-04-26T01:53:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.647+0000 7f93ece00640 1 -- 172.21.15.1:0/3234095046 shutdown_connections 2024-04-26T01:53:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.647+0000 7f93ece00640 1 --2- 172.21.15.1:0/3234095046 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f93cc0a3f30 0x7f93cc0a63f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.647+0000 7f93ece00640 1 --2- 172.21.15.1:0/3234095046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93e810e620 0x7f93e819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.647+0000 7f93ece00640 1 --2- 172.21.15.1:0/3234095046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8074c20 0x7f93e819acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.647+0000 7f93ece00640 1 --2- 172.21.15.1:0/3234095046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e8074300 0x7f93e819a770 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.647+0000 7f93ece00640 1 -- 172.21.15.1:0/3234095046 >> 172.21.15.1:0/3234095046 conn(0x7f93e80fd890 msgr2=0x7f93e80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.647+0000 7f93ece00640 1 -- 172.21.15.1:0/3234095046 shutdown_connections 2024-04-26T01:53:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:04.647+0000 7f93ece00640 1 -- 172.21.15.1:0/3234095046 wait complete. 2024-04-26T01:53:05.067 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: *** Caught signal (Segmentation fault) ** 2024-04-26T01:53:05.068 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: in thread 7f5b833bc700 thread_name:md_log_replay 2024-04-26T01:53:05.068 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-26T01:53:05.068 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5b9477dd20] 2024-04-26T01:53:05.068 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 2: /usr/bin/ceph-mds(+0x54938f) [0x56511ac6838f] 2024-04-26T01:53:05.068 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x56511ac6bb23] 2024-04-26T01:53:05.068 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 4: (MDLog::_replay_thread()+0x617) [0x56511acc6647] 2024-04-26T01:53:05.068 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 5: (MDLog::ReplayThread::entry()+0x11) [0x56511accfd31] 2024-04-26T01:53:05.068 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f5b947731ca] 2024-04-26T01:53:05.068 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 7: clone() 2024-04-26T01:53:05.068 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 2024-04-26T01:53:04.756+0000 7f5b833bc700 -1 *** Caught signal (Segmentation fault) ** 2024-04-26T01:53:05.069 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: in thread 7f5b833bc700 thread_name:md_log_replay 2024-04-26T01:53:05.069 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 2024-04-26T01:53:05.069 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-26T01:53:05.069 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5b9477dd20] 2024-04-26T01:53:05.069 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 2: /usr/bin/ceph-mds(+0x54938f) [0x56511ac6838f] 2024-04-26T01:53:05.069 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x56511ac6bb23] 2024-04-26T01:53:05.069 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 4: (MDLog::_replay_thread()+0x617) [0x56511acc6647] 2024-04-26T01:53:05.069 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 5: (MDLog::ReplayThread::entry()+0x11) [0x56511accfd31] 2024-04-26T01:53:05.069 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f5b947731ca] 2024-04-26T01:53:05.069 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 7: clone() 2024-04-26T01:53:05.069 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T01:53:05.069 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 2024-04-26T01:53:05.069 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 0> 2024-04-26T01:53:04.756+0000 7f5b833bc700 -1 *** Caught signal (Segmentation fault) ** 2024-04-26T01:53:05.069 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: in thread 7f5b833bc700 thread_name:md_log_replay 2024-04-26T01:53:05.070 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 2024-04-26T01:53:05.070 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-26T01:53:05.070 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5b9477dd20] 2024-04-26T01:53:05.070 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 2: /usr/bin/ceph-mds(+0x54938f) [0x56511ac6838f] 2024-04-26T01:53:05.070 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x56511ac6bb23] 2024-04-26T01:53:05.070 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 4: (MDLog::_replay_thread()+0x617) [0x56511acc6647] 2024-04-26T01:53:05.070 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 5: (MDLog::ReplayThread::entry()+0x11) [0x56511accfd31] 2024-04-26T01:53:05.070 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f5b947731ca] 2024-04-26T01:53:05.070 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 7: clone() 2024-04-26T01:53:05.070 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T01:53:05.070 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 2024-04-26T01:53:05.070 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 0> 2024-04-26T01:53:04.756+0000 7f5b833bc700 -1 *** Caught signal (Segmentation fault) ** 2024-04-26T01:53:05.070 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: in thread 7f5b833bc700 thread_name:md_log_replay 2024-04-26T01:53:05.070 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 2024-04-26T01:53:05.070 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-26T01:53:05.071 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5b9477dd20] 2024-04-26T01:53:05.071 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 2: /usr/bin/ceph-mds(+0x54938f) [0x56511ac6838f] 2024-04-26T01:53:05.071 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x56511ac6bb23] 2024-04-26T01:53:05.071 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 4: (MDLog::_replay_thread()+0x617) [0x56511acc6647] 2024-04-26T01:53:05.071 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 5: (MDLog::ReplayThread::entry()+0x11) [0x56511accfd31] 2024-04-26T01:53:05.071 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f5b947731ca] 2024-04-26T01:53:05.071 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 7: clone() 2024-04-26T01:53:05.071 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T01:53:05.071 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[62812]: 2024-04-26T01:53:05.071 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:04 smithi173 podman[67145]: 2024-04-26 01:53:04.820313014 +0000 UTC m=+0.022500725 container died a584bc72024dd8819b3d444a54554a3b74d0dd4e9fdf5ed1f6690e11fac600a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-26T01:53:05.805 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:05 smithi173 podman[67145]: 2024-04-26 01:53:05.452567499 +0000 UTC m=+0.654755188 container remove a584bc72024dd8819b3d444a54554a3b74d0dd4e9fdf5ed1f6690e11fac600a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T01:53:05.806 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:05 smithi173 systemd[1]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.f.service: Main process exited, code=exited, status=139/n/a 2024-04-26T01:53:05.806 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:05 smithi173 systemd[1]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-04-26T01:53:05.806 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:05 smithi173 systemd[1]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.f.service: Consumed 1.156s CPU time. 2024-04-26T01:53:06.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:05 smithi173 ceph-mon[32553]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 85 B/s wr, 4 op/s 2024-04-26T01:53:06.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:05 smithi001 ceph-mon[27822]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 85 B/s wr, 4 op/s 2024-04-26T01:53:06.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:05 smithi097 ceph-mon[32853]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 85 B/s wr, 4 op/s 2024-04-26T01:53:06.490 INFO:tasks.workunit.client.0.smithi001.stderr:Updating files: 62% (7807/12492) Updating files: 63% (7870/12492) Updating files: 64% (7995/12492) Updating files: 65% (8120/12492) Updating files: 66% (8245/12492) Updating files: 67% (8370/12492) Updating files: 68% (8495/12492) Updating files: 69% (8620/12492) Updating files: 70% (8745/12492) Updating files: 71% (8870/12492) Updating files: 72% (8995/12492) Updating files: 73% (9120/12492) Updating files: 74% (9245/12492) Updating files: 75% (9369/12492) Updating files: 76% (9494/12492) Updating files: 77% (9619/12492) Updating files: 78% (9744/12492) Updating files: 79% (9869/12492) Updating files: 80% (9994/12492) Updating files: 81% (10119/12492) Updating files: 82% (10244/12492) Updating files: 83% (10369/12492) Updating files: 84% (10494/12492) Updating files: 85% (10619/12492) Updating files: 86% (10744/12492) Updating files: 87% (10869/12492) Updating files: 88% (10993/12492) Updating files: 89% (11118/12492) Updating files: 90% (11243/12492) Updating files: 91% (11368/12492) Updating files: 92% (11493/12492) Updating files: 93% (11618/12492) Updating files: 94% (11743/12492) Updating files: 95% (11868/12492) Updating files: 96% (11993/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-26T01:53:06.911 INFO:tasks.workunit.client.0.smithi001.stderr:Note: switching to 'afa1933b0fdbb6c99c947d1eda34d661d23cd327'. 2024-04-26T01:53:06.911 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-04-26T01:53:06.912 INFO:tasks.workunit.client.0.smithi001.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-26T01:53:06.912 INFO:tasks.workunit.client.0.smithi001.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-26T01:53:06.912 INFO:tasks.workunit.client.0.smithi001.stderr:state without impacting any branches by switching back to a branch. 2024-04-26T01:53:06.912 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-04-26T01:53:06.912 INFO:tasks.workunit.client.0.smithi001.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-26T01:53:06.912 INFO:tasks.workunit.client.0.smithi001.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-26T01:53:06.912 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-04-26T01:53:06.912 INFO:tasks.workunit.client.0.smithi001.stderr: git switch -c 2024-04-26T01:53:06.912 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-04-26T01:53:06.912 INFO:tasks.workunit.client.0.smithi001.stderr:Or undo this operation with: 2024-04-26T01:53:06.912 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-04-26T01:53:06.912 INFO:tasks.workunit.client.0.smithi001.stderr: git switch - 2024-04-26T01:53:06.912 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-04-26T01:53:06.912 INFO:tasks.workunit.client.0.smithi001.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-26T01:53:06.912 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-04-26T01:53:06.912 INFO:tasks.workunit.client.0.smithi001.stderr:HEAD is now at afa1933b0fd qa/suites/fs: add quiescer to the fs suite 2024-04-26T01:53:06.915 DEBUG:teuthology.orchestra.run.smithi001:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-26T01:53:07.026 INFO:tasks.workunit.client.0.smithi001.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-26T01:53:07.034 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T01:53:07.034 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-26T01:53:07.922 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-26T01:53:08.032 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-26T01:53:08.066 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:07 smithi173 ceph-mon[32553]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T01:53:08.122 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T01:53:08.128 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T01:53:08.128 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-26T01:53:08.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:07 smithi001 ceph-mon[27822]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T01:53:08.220 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T01:53:08.231 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-04-26T01:53:08.231 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-26T01:53:08.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:07 smithi097 ceph-mon[32853]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T01:53:08.286 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-04-26T01:53:08.287 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-04-26T01:53:08.287 DEBUG:teuthology.orchestra.run.smithi001:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afa1933b0fdbb6c99c947d1eda34d661d23cd327 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-04-26T01:53:10.067 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:09 smithi173 ceph-mon[32553]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T01:53:10.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:09 smithi001 ceph-mon[27822]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T01:53:10.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:09 smithi097 ceph-mon[32853]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T01:53:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:11 smithi001 ceph-mon[27822]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-26T01:53:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:11 smithi001 ceph-mon[27822]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-26T01:53:12.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:11 smithi097 ceph-mon[32853]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-26T01:53:12.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:11 smithi097 ceph-mon[32853]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-26T01:53:12.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:11 smithi173 ceph-mon[32553]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-26T01:53:12.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:11 smithi173 ceph-mon[32553]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-26T01:53:13.316 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:53:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:14.102 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:13 smithi001 ceph-mon[27822]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-26T01:53:14.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:13 smithi097 ceph-mon[32853]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-26T01:53:14.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:13 smithi173 ceph-mon[32553]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-26T01:53:16.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:15 smithi173 ceph-mon[32553]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-26T01:53:16.076 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:15 smithi173 systemd[1]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.f.service: Scheduled restart job, restart counter is at 1. 2024-04-26T01:53:16.076 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:15 smithi173 systemd[1]: Stopped Ceph mds.f for 77cfcc80-036d-11ef-bc93-c7b262605968. 2024-04-26T01:53:16.076 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:15 smithi173 systemd[1]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.f.service: Consumed 1.156s CPU time. 2024-04-26T01:53:16.076 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:15 smithi173 systemd[1]: Starting Ceph mds.f for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T01:53:16.076 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:16 smithi173 podman[67249]: 2024-04-26 01:53:15.969624796 +0000 UTC m=+0.018008051 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:53:16.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.090+0000 7f2e1ce00700 1 -- 172.21.15.1:0/2761362164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e18109940 msgr2=0x7f2e18115de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:16.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.090+0000 7f2e1ce00700 1 --2- 172.21.15.1:0/2761362164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e18109940 0x7f2e18115de0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000b0a0 tx=0x7f2e1000b3b0 comp rx=0 tx=0).stop 2024-04-26T01:53:16.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.091+0000 7f2e1ce00700 1 -- 172.21.15.1:0/2761362164 shutdown_connections 2024-04-26T01:53:16.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.091+0000 7f2e1ce00700 1 --2- 172.21.15.1:0/2761362164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e18109940 0x7f2e18115de0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:16.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.091+0000 7f2e1ce00700 1 --2- 172.21.15.1:0/2761362164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e1810f030 0x7f2e1810f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:16.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.091+0000 7f2e1ce00700 1 --2- 172.21.15.1:0/2761362164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e18108f80 0x7f2e18109400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:16.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.091+0000 7f2e1ce00700 1 -- 172.21.15.1:0/2761362164 >> 172.21.15.1:0/2761362164 conn(0x7f2e180787d0 msgr2=0x7f2e18107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:16.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.091+0000 7f2e1ce00700 1 -- 172.21.15.1:0/2761362164 shutdown_connections 2024-04-26T01:53:16.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.091+0000 7f2e1ce00700 1 -- 172.21.15.1:0/2761362164 wait complete. 2024-04-26T01:53:16.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.092+0000 7f2e1ce00700 1 Processor -- start 2024-04-26T01:53:16.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.092+0000 7f2e1ce00700 1 -- start start 2024-04-26T01:53:16.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.093+0000 7f2e1ce00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e18108f80 0x7f2e1819f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:16.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.093+0000 7f2e1ce00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e18109940 0x7f2e1819fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:16.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.093+0000 7f2e1ce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e1810f030 0x7f2e181a3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:16.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.093+0000 7f2e1ce00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2e18119140 con 0x7f2e1810f030 2024-04-26T01:53:16.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.093+0000 7f2e1ce00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2e18118fc0 con 0x7f2e18108f80 2024-04-26T01:53:16.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.093+0000 7f2e1ce00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2e181192c0 con 0x7f2e18109940 2024-04-26T01:53:16.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.093+0000 7f2e16a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e18109940 0x7f2e1819fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:16.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.093+0000 7f2e16a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e18109940 0x7f2e1819fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:47402/0 (socket says 172.21.15.1:47402) 2024-04-26T01:53:16.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.093+0000 7f2e16a00700 1 -- 172.21.15.1:0/2871522637 learned_addr learned my addr 172.21.15.1:0/2871522637 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:16.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.093+0000 7f2e17400700 1 --2- 172.21.15.1:0/2871522637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e18108f80 0x7f2e1819f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:16.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.093+0000 7f2e17e00700 1 --2- 172.21.15.1:0/2871522637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e1810f030 0x7f2e181a3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:16.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.094+0000 7f2e16a00700 1 -- 172.21.15.1:0/2871522637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e18108f80 msgr2=0x7f2e1819f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:16.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.094+0000 7f2e16a00700 1 --2- 172.21.15.1:0/2871522637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e18108f80 0x7f2e1819f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:16.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.094+0000 7f2e16a00700 1 -- 172.21.15.1:0/2871522637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e1810f030 msgr2=0x7f2e181a3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:16.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.094+0000 7f2e16a00700 1 --2- 172.21.15.1:0/2871522637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e1810f030 0x7f2e181a3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:16.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.094+0000 7f2e16a00700 1 -- 172.21.15.1:0/2871522637 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e181a4560 con 0x7f2e18109940 2024-04-26T01:53:16.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.094+0000 7f2e17400700 1 --2- 172.21.15.1:0/2871522637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e18108f80 0x7f2e1819f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:16.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.094+0000 7f2e16a00700 1 --2- 172.21.15.1:0/2871522637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e18109940 0x7f2e1819fa90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400d980 tx=0x7f2e0400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:16.093 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:15 smithi001 ceph-mon[27822]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-26T01:53:16.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.095+0000 7f2e0fe00700 1 -- 172.21.15.1:0/2871522637 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04016070 con 0x7f2e18109940 2024-04-26T01:53:16.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.095+0000 7f2e0fe00700 1 -- 172.21.15.1:0/2871522637 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e04004830 con 0x7f2e18109940 2024-04-26T01:53:16.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.095+0000 7f2e1ce00700 1 -- 172.21.15.1:0/2871522637 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e181a4850 con 0x7f2e18109940 2024-04-26T01:53:16.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.095+0000 7f2e0fe00700 1 -- 172.21.15.1:0/2871522637 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04005540 con 0x7f2e18109940 2024-04-26T01:53:16.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.095+0000 7f2e1ce00700 1 -- 172.21.15.1:0/2871522637 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e181ac0d0 con 0x7f2e18109940 2024-04-26T01:53:16.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.096+0000 7f2e1ce00700 1 -- 172.21.15.1:0/2871522637 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e18069730 con 0x7f2e18109940 2024-04-26T01:53:16.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.098+0000 7f2e0fe00700 1 -- 172.21.15.1:0/2871522637 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f2e04004220 con 0x7f2e18109940 2024-04-26T01:53:16.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.099+0000 7f2e0fe00700 1 --2- 172.21.15.1:0/2871522637 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2dfc0a3db0 0x7f2dfc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:16.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.099+0000 7f2e0fe00700 1 -- 172.21.15.1:0/2871522637 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8527+0+0 (secure 0 0 0) 0x7f2e040d06a0 con 0x7f2e18109940 2024-04-26T01:53:16.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.100+0000 7f2e17400700 1 --2- 172.21.15.1:0/2871522637 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2dfc0a3db0 0x7f2dfc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:16.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.100+0000 7f2e17400700 1 --2- 172.21.15.1:0/2871522637 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2dfc0a3db0 0x7f2dfc0a6270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e08009fd0 tx=0x7f2e08009260 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:16.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.104+0000 7f2e0fe00700 1 -- 172.21.15.1:0/2871522637 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e0401f020 con 0x7f2e18109940 2024-04-26T01:53:16.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:15 smithi097 ceph-mon[32853]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-26T01:53:16.328 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:16 smithi173 podman[67249]: 2024-04-26 01:53:16.078270116 +0000 UTC m=+0.126653344 container create 9b8b0befaba88f43fddc553b78bf0af47e9db628f09c22f73cbec527f02e2ba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-26T01:53:16.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.386+0000 7f2e1ce00700 1 -- 172.21.15.1:0/2871522637 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e1802df80 con 0x7f2e18109940 2024-04-26T01:53:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.387+0000 7f2e0fe00700 1 -- 172.21.15.1:0/2871522637 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11841 (secure 0 0 0) 0x7f2e04022350 con 0x7f2e18109940 2024-04-26T01:53:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:53:16.386 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:51:38.222728+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2952462325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2952462325},{"type":"v1","addr":"172.21.15.173:6835","nonce":2952462325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24499,24514,24518]},"id":1}]} 2024-04-26T01:53:16.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.390+0000 7f2e1ce00700 1 -- 172.21.15.1:0/2871522637 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2dfc0a3db0 msgr2=0x7f2dfc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:16.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.390+0000 7f2e1ce00700 1 --2- 172.21.15.1:0/2871522637 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2dfc0a3db0 0x7f2dfc0a6270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e08009fd0 tx=0x7f2e08009260 comp rx=0 tx=0).stop 2024-04-26T01:53:16.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.390+0000 7f2e1ce00700 1 -- 172.21.15.1:0/2871522637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e18109940 msgr2=0x7f2e1819fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:16.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.390+0000 7f2e1ce00700 1 --2- 172.21.15.1:0/2871522637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e18109940 0x7f2e1819fa90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400d980 tx=0x7f2e0400dc90 comp rx=0 tx=0).stop 2024-04-26T01:53:16.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.391+0000 7f2e1ce00700 1 -- 172.21.15.1:0/2871522637 shutdown_connections 2024-04-26T01:53:16.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.391+0000 7f2e1ce00700 1 --2- 172.21.15.1:0/2871522637 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2dfc0a3db0 0x7f2dfc0a6270 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:16.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.391+0000 7f2e1ce00700 1 --2- 172.21.15.1:0/2871522637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e1810f030 0x7f2e181a3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:16.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.391+0000 7f2e1ce00700 1 --2- 172.21.15.1:0/2871522637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e18108f80 0x7f2e1819f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:16.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.391+0000 7f2e1ce00700 1 --2- 172.21.15.1:0/2871522637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e18109940 0x7f2e1819fa90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:16.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.391+0000 7f2e1ce00700 1 -- 172.21.15.1:0/2871522637 >> 172.21.15.1:0/2871522637 conn(0x7f2e180787d0 msgr2=0x7f2e180798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:16.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.391+0000 7f2e1ce00700 1 -- 172.21.15.1:0/2871522637 shutdown_connections 2024-04-26T01:53:16.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:16.391+0000 7f2e1ce00700 1 -- 172.21.15.1:0/2871522637 wait complete. 2024-04-26T01:53:16.390 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 26 2024-04-26T01:53:16.817 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:16 smithi173 podman[67249]: 2024-04-26 01:53:16.331179788 +0000 UTC m=+0.379563016 container init 9b8b0befaba88f43fddc553b78bf0af47e9db628f09c22f73cbec527f02e2ba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-26T01:53:16.817 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:16 smithi173 podman[67249]: 2024-04-26 01:53:16.333797536 +0000 UTC m=+0.382180777 container start 9b8b0befaba88f43fddc553b78bf0af47e9db628f09c22f73cbec527f02e2ba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-26T01:53:16.817 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:16 smithi173 ceph-mds[67268]: -- 172.21.15.173:0/77176874 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e8994d1a40 con 0x55e8994a4800 2024-04-26T01:53:16.817 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:16 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[67264]: starting mds.f at 2024-04-26T01:53:16.817 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:16 smithi173 bash[67249]: 9b8b0befaba88f43fddc553b78bf0af47e9db628f09c22f73cbec527f02e2ba9 2024-04-26T01:53:16.817 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 01:53:16 smithi173 systemd[1]: Started Ceph mds.f for 77cfcc80-036d-11ef-bc93-c7b262605968. 2024-04-26T01:53:17.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:16 smithi001 ceph-mon[27822]: daemon mds.f restarted 2024-04-26T01:53:17.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:16 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2871522637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:17.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:16 smithi097 ceph-mon[32853]: daemon mds.f restarted 2024-04-26T01:53:17.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:16 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/2871522637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:17.259 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24518 quiesce db --set-id 4a66b69e --release --await' 2024-04-26T01:53:17.259 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24518 quiesce db --set-id 4a66b69e --release --await 2024-04-26T01:53:17.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:16 smithi173 ceph-mon[32553]: daemon mds.f restarted 2024-04-26T01:53:17.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:16 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/2871522637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:17.691 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.788+0000 7fbbef200640 1 -- 172.21.15.1:0/2594628708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbf01014e0 msgr2=0x7fbbf010e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.788+0000 7fbbef200640 1 --2- 172.21.15.1:0/2594628708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbf01014e0 0x7fbbf010e1a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc009900 tx=0x7fbbdc02a3f0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.788+0000 7fbbef200640 1 -- 172.21.15.1:0/2594628708 shutdown_connections 2024-04-26T01:53:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.788+0000 7fbbef200640 1 --2- 172.21.15.1:0/2594628708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbf010e6e0 0x7fbbf0110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.788+0000 7fbbef200640 1 --2- 172.21.15.1:0/2594628708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbf01014e0 0x7fbbf010e1a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.788+0000 7fbbef200640 1 --2- 172.21.15.1:0/2594628708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbf0100b30 0x7fbbf0100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.788+0000 7fbbef200640 1 -- 172.21.15.1:0/2594628708 >> 172.21.15.1:0/2594628708 conn(0x7fbbf00fc730 msgr2=0x7fbbf00feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.788+0000 7fbbef200640 1 -- 172.21.15.1:0/2594628708 shutdown_connections 2024-04-26T01:53:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.788+0000 7fbbef200640 1 -- 172.21.15.1:0/2594628708 wait complete. 2024-04-26T01:53:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.789+0000 7fbbef200640 1 Processor -- start 2024-04-26T01:53:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.790+0000 7fbbef200640 1 -- start start 2024-04-26T01:53:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.790+0000 7fbbef200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbf0100b30 0x7fbbf01a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.790+0000 7fbbef200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbf01014e0 0x7fbbf01a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.790+0000 7fbbef200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbf010e6e0 0x7fbbf019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.790+0000 7fbbef200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf0114270 con 0x7fbbf0100b30 2024-04-26T01:53:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.790+0000 7fbbef200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01140f0 con 0x7fbbf01014e0 2024-04-26T01:53:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.790+0000 7fbbef200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01143f0 con 0x7fbbf010e6e0 2024-04-26T01:53:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.790+0000 7fbbede00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbf0100b30 0x7fbbf01a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.790+0000 7fbbede00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbf0100b30 0x7fbbf01a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41024/0 (socket says 172.21.15.1:41024) 2024-04-26T01:53:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.790+0000 7fbbed400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbf01014e0 0x7fbbf01a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.790+0000 7fbbede00640 1 -- 172.21.15.1:0/3426493838 learned_addr learned my addr 172.21.15.1:0/3426493838 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.790+0000 7fbbee800640 1 --2- 172.21.15.1:0/3426493838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbf010e6e0 0x7fbbf019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.791+0000 7fbbede00640 1 -- 172.21.15.1:0/3426493838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbf010e6e0 msgr2=0x7fbbf019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.791+0000 7fbbede00640 1 --2- 172.21.15.1:0/3426493838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbf010e6e0 0x7fbbf019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.791+0000 7fbbede00640 1 -- 172.21.15.1:0/3426493838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbf01014e0 msgr2=0x7fbbf01a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.791+0000 7fbbede00640 1 --2- 172.21.15.1:0/3426493838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbf01014e0 0x7fbbf01a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.791+0000 7fbbede00640 1 -- 172.21.15.1:0/3426493838 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf019ad90 con 0x7fbbf0100b30 2024-04-26T01:53:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.791+0000 7fbbed400640 1 --2- 172.21.15.1:0/3426493838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbf01014e0 0x7fbbf01a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.791+0000 7fbbede00640 1 --2- 172.21.15.1:0/3426493838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbf0100b30 0x7fbbf01a0460 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000e9f0 tx=0x7fbbe000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.791+0000 7fbbe6a00640 1 -- 172.21.15.1:0/3426493838 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe000cd80 con 0x7fbbf0100b30 2024-04-26T01:53:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.791+0000 7fbbe6a00640 1 -- 172.21.15.1:0/3426493838 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe00108e0 con 0x7fbbf0100b30 2024-04-26T01:53:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.791+0000 7fbbef200640 1 -- 172.21.15.1:0/3426493838 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf019b080 con 0x7fbbf0100b30 2024-04-26T01:53:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.791+0000 7fbbe6a00640 1 -- 172.21.15.1:0/3426493838 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0018980 con 0x7fbbf0100b30 2024-04-26T01:53:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.791+0000 7fbbef200640 1 -- 172.21.15.1:0/3426493838 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf01a7320 con 0x7fbbf0100b30 2024-04-26T01:53:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.795+0000 7fbbe6a00640 1 -- 172.21.15.1:0/3426493838 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fbbe0010430 con 0x7fbbf0100b30 2024-04-26T01:53:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.795+0000 7fbbf7c00740 1 Processor -- start 2024-04-26T01:53:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.795+0000 7fbbf7c00740 1 -- start start 2024-04-26T01:53:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.795+0000 7fbbf7c00740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e69b0 0x5615395e6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.795+0000 7fbbf7c00740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395e72d0 0x5615395f1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.795+0000 7fbbf7c00740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615395f1b90 0x5615395f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.795+0000 7fbbf7c00740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56153954c900 con 0x5615395e69b0 2024-04-26T01:53:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.795+0000 7fbbf7c00740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5615394f2ef0 con 0x5615395f1b90 2024-04-26T01:53:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.795+0000 7fbbf7c00740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5615395488c0 con 0x5615395e72d0 2024-04-26T01:53:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.796+0000 7fbbee800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e69b0 0x5615395e6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.796+0000 7fbbee800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e69b0 0x5615395e6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41028/0 (socket says 172.21.15.1:41028) 2024-04-26T01:53:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.796+0000 7fbbee800640 1 -- 172.21.15.1:0/1294980668 learned_addr learned my addr 172.21.15.1:0/1294980668 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.796+0000 7fbbede00640 1 --2- 172.21.15.1:0/1294980668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395e72d0 0x5615395f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.796+0000 7fbbed400640 1 --2- 172.21.15.1:0/1294980668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615395f1b90 0x5615395f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.796+0000 7fbbee800640 1 -- 172.21.15.1:0/1294980668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395e72d0 msgr2=0x5615395f1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.796+0000 7fbbee800640 1 --2- 172.21.15.1:0/1294980668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395e72d0 0x5615395f1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.796+0000 7fbbee800640 1 -- 172.21.15.1:0/1294980668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615395f1b90 msgr2=0x5615395f3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.796+0000 7fbbee800640 1 --2- 172.21.15.1:0/1294980668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615395f1b90 0x5615395f3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.796+0000 7fbbee800640 1 -- 172.21.15.1:0/1294980668 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56153925f210 con 0x5615395e69b0 2024-04-26T01:53:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.797+0000 7fbbee800640 1 --2- 172.21.15.1:0/1294980668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e69b0 0x5615395e6d90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009f10 tx=0x7fbbe8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4dc63f70ee8cfc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.797+0000 7fbbe6a00640 1 --2- 172.21.15.1:0/3426493838 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fbbd40a3d80 0x7fbbd40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.797+0000 7fbbe6a00640 1 -- 172.21.15.1:0/3426493838 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fbbe00d0b30 con 0x7fbbf0100b30 2024-04-26T01:53:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.797+0000 7fbbdb400640 1 -- 172.21.15.1:0/1294980668 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8016070 con 0x5615395e69b0 2024-04-26T01:53:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.797+0000 7fbbede00640 1 --2- 172.21.15.1:0/3426493838 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fbbd40a3d80 0x7fbbd40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.797+0000 7fbbdb400640 1 -- 172.21.15.1:0/1294980668 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe8012900 con 0x5615395e69b0 2024-04-26T01:53:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.797+0000 7fbbdb400640 1 -- 172.21.15.1:0/1294980668 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe80118f0 con 0x5615395e69b0 2024-04-26T01:53:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.798+0000 7fbbf7c00740 1 -- 172.21.15.1:0/1294980668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e69b0 msgr2=0x5615395e6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.798+0000 7fbbede00640 1 --2- 172.21.15.1:0/3426493838 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fbbd40a3d80 0x7fbbd40a6240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x5615395e7f50 tx=0x7fbbe00d45e0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.798+0000 7fbbf7c00740 1 --2- 172.21.15.1:0/1294980668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e69b0 0x5615395e6d90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009f10 tx=0x7fbbe8010740 comp rx=0 tx=0).stop 2024-04-26T01:53:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.798+0000 7fbbf7c00740 1 -- 172.21.15.1:0/1294980668 shutdown_connections 2024-04-26T01:53:17.797 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.798+0000 7fbbf7c00740 1 --2- 172.21.15.1:0/1294980668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615395f1b90 0x5615395f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.798 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.798+0000 7fbbf7c00740 1 --2- 172.21.15.1:0/1294980668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395e72d0 0x5615395f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.798 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.798+0000 7fbbf7c00740 1 --2- 172.21.15.1:0/1294980668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e69b0 0x5615395e6d90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.798 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.798+0000 7fbbf7c00740 1 -- 172.21.15.1:0/1294980668 >> 172.21.15.1:0/1294980668 conn(0x5615395e2ec0 msgr2=0x5615395e32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:17.798 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.798+0000 7fbbf7c00740 1 -- 172.21.15.1:0/1294980668 shutdown_connections 2024-04-26T01:53:17.798 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.798+0000 7fbbf7c00740 1 -- 172.21.15.1:0/1294980668 wait complete. 2024-04-26T01:53:17.798 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.799+0000 7fbbf7c00740 1 -- start start 2024-04-26T01:53:17.798 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.799+0000 7fbbf7c00740 1 Processor -- start 2024-04-26T01:53:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.799+0000 7fbbf7c00740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e69b0 0x5615395eeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.799+0000 7fbbf7c00740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395e72d0 0x5615395ef090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.799+0000 7fbbf7c00740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615395f1b90 0x5615395ef5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.799+0000 7fbbf7c00740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5615395f47c0 con 0x5615395e69b0 2024-04-26T01:53:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.799+0000 7fbbf7c00740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561539506310 con 0x5615395f1b90 2024-04-26T01:53:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.799+0000 7fbbf7c00740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5615395f4940 con 0x5615395e72d0 2024-04-26T01:53:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.799+0000 7fbbf7c00740 10 client.0 _ll_register_callbacks cb 0x5615394d4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:53:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.799+0000 7fbbee800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e69b0 0x5615395eeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.800+0000 7fbbee800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e69b0 0x5615395eeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41038/0 (socket says 172.21.15.1:41038) 2024-04-26T01:53:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.800+0000 7fbbee800640 1 -- 172.21.15.1:0/744849172 learned_addr learned my addr 172.21.15.1:0/744849172 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.800+0000 7fbbed400640 1 --2- 172.21.15.1:0/744849172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395e72d0 0x5615395ef090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.800+0000 7fbbee800640 1 --2- 172.21.15.1:0/744849172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615395f1b90 0x5615395ef5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.800+0000 7fbbed400640 1 -- 172.21.15.1:0/744849172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615395f1b90 msgr2=0x5615395ef5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.800+0000 7fbbed400640 1 --2- 172.21.15.1:0/744849172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615395f1b90 0x5615395ef5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.800+0000 7fbbed400640 1 -- 172.21.15.1:0/744849172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e69b0 msgr2=0x5615395eeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.800+0000 7fbbed400640 1 --2- 172.21.15.1:0/744849172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e69b0 0x5615395eeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.800+0000 7fbbed400640 1 -- 172.21.15.1:0/744849172 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56153932b620 con 0x5615395e72d0 2024-04-26T01:53:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.800+0000 7fbbee800640 1 --2- 172.21.15.1:0/744849172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e69b0 0x5615395eeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.800+0000 7fbbed400640 1 --2- 172.21.15.1:0/744849172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395e72d0 0x5615395ef090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc02a910 tx=0x7fbbdc005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39ff0aaa462f1393 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.800+0000 7fbbcfe00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:53:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.801+0000 7fbbcfe00640 1 -- 172.21.15.1:0/744849172 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc035040 con 0x5615395e72d0 2024-04-26T01:53:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.801+0000 7fbbf7c00740 1 -- 172.21.15.1:0/744849172 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbe8012900 con 0x5615395e72d0 2024-04-26T01:53:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.801+0000 7fbbcfe00640 1 -- 172.21.15.1:0/744849172 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fbbdc004970 con 0x5615395e72d0 2024-04-26T01:53:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.801+0000 7fbbcfe00640 1 -- 172.21.15.1:0/744849172 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbbdc004b70 con 0x5615395e72d0 2024-04-26T01:53:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.801+0000 7fbbf7c00740 10 client.25426 fetch_fsmap learned FSMap version 28 2024-04-26T01:53:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.801+0000 7fbbf7c00740 1 -- 172.21.15.1:0/744849172 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561539600aa0 con 0x5615395e72d0 2024-04-26T01:53:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.801+0000 7fbbcfe00640 1 -- 172.21.15.1:0/744849172 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe8012900 con 0x5615395e72d0 2024-04-26T01:53:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.802+0000 7fbbcfe00640 1 -- 172.21.15.1:0/744849172 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fbbdc0314a0 con 0x5615395e72d0 2024-04-26T01:53:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.802+0000 7fbbf7c00740 10 client.25426 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:53:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.802+0000 7fbbf7c00740 10 client.25426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:53:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.802+0000 7fbbf7c00740 20 client.25426 populate_metadata read hostname 'smithi001' 2024-04-26T01:53:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.802+0000 7fbbf7c00740 1 --2- 172.21.15.1:0/744849172 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561539601f10 0x561539604300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.802+0000 7fbbf7c00740 4 client.25426 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:53:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.802+0000 7fbbf7c00740 1 -- 172.21.15.1:0/744849172 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615395608b0 con 0x561539601f10 2024-04-26T01:53:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.802+0000 7fbbee800640 1 --2- 172.21.15.1:0/744849172 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561539601f10 0x561539604300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.803+0000 7fbbee800640 1 --2- 172.21.15.1:0/744849172 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561539601f10 0x561539604300 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cad4040f1f634db server_cookie=2e7be6add6dc6b47 in_seq=0 out_seq=0 2024-04-26T01:53:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.803+0000 7fbbcfe00640 10 client.25426 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.803+0000 7fbbcfe00640 1 -- 172.21.15.1:0/744849172 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5615395608b0 con 0x561539601f10 2024-04-26T01:53:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.803+0000 7fbbcfe00640 10 client.25426 handle_command_reply: tid=0 2024-04-26T01:53:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.803+0000 7fbbf7c00740 1 client.25426 shutdown 2024-04-26T01:53:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.803+0000 7fbbf7c00740 1 -- 172.21.15.1:0/744849172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395e72d0 msgr2=0x5615395ef090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.803+0000 7fbbf7c00740 1 --2- 172.21.15.1:0/744849172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395e72d0 0x5615395ef090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc02a910 tx=0x7fbbdc005270 comp rx=0 tx=0).stop 2024-04-26T01:53:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.804+0000 7fbbf7c00740 1 -- 172.21.15.1:0/744849172 shutdown_connections 2024-04-26T01:53:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.804+0000 7fbbf7c00740 1 --2- 172.21.15.1:0/744849172 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561539601f10 0x561539604300 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.804+0000 7fbbf7c00740 1 --2- 172.21.15.1:0/744849172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615395f1b90 0x5615395ef5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.804+0000 7fbbf7c00740 1 --2- 172.21.15.1:0/744849172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395e72d0 0x5615395ef090 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.804+0000 7fbbf7c00740 1 --2- 172.21.15.1:0/744849172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e69b0 0x5615395eeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.804+0000 7fbbcfe00640 0 client.25426 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.804+0000 7fbbf7c00740 1 -- 172.21.15.1:0/744849172 >> 172.21.15.1:0/744849172 conn(0x5615395e2ec0 msgr2=0x5615395e8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.804+0000 7fbbf7c00740 1 -- 172.21.15.1:0/744849172 shutdown_connections 2024-04-26T01:53:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.804+0000 7fbbf7c00740 1 -- 172.21.15.1:0/744849172 wait complete. 2024-04-26T01:53:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.804+0000 7fbbf7c00740 20 client.25426 trim_cache size 0 max 16384 2024-04-26T01:53:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.820+0000 7fbbf7c00740 1 Processor -- start 2024-04-26T01:53:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.820+0000 7fbbf7c00740 1 -- start start 2024-04-26T01:53:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.820+0000 7fbbf7c00740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615395e69b0 0x5615395fd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.820+0000 7fbbf7c00740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e72d0 0x5615395e27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.820+0000 7fbbf7c00740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395f1b90 0x5615395ef220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.820+0000 7fbbf7c00740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01b2620 con 0x5615395e72d0 2024-04-26T01:53:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.820+0000 7fbbf7c00740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01afef0 con 0x5615395e69b0 2024-04-26T01:53:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.820+0000 7fbbf7c00740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01add40 con 0x5615395f1b90 2024-04-26T01:53:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.821+0000 7fbbee800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615395e69b0 0x5615395fd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.821+0000 7fbbed400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e72d0 0x5615395e27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.821+0000 7fbbee800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615395e69b0 0x5615395fd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:54930/0 (socket says 172.21.15.1:54930) 2024-04-26T01:53:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.821+0000 7fbbee800640 1 -- 172.21.15.1:0/3132272925 learned_addr learned my addr 172.21.15.1:0/3132272925 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.821+0000 7fbbee800640 1 --2- 172.21.15.1:0/3132272925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395f1b90 0x5615395ef220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.821+0000 7fbbed400640 1 -- 172.21.15.1:0/3132272925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395f1b90 msgr2=0x5615395ef220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.821+0000 7fbbed400640 1 --2- 172.21.15.1:0/3132272925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395f1b90 0x5615395ef220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.821+0000 7fbbed400640 1 -- 172.21.15.1:0/3132272925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615395e69b0 msgr2=0x5615395fd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.821+0000 7fbbed400640 1 --2- 172.21.15.1:0/3132272925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615395e69b0 0x5615395fd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.821+0000 7fbbed400640 1 -- 172.21.15.1:0/3132272925 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615395f42f0 con 0x5615395e72d0 2024-04-26T01:53:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.821+0000 7fbbee800640 1 --2- 172.21.15.1:0/3132272925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395f1b90 0x5615395ef220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:53:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.821+0000 7fbbed400640 1 --2- 172.21.15.1:0/3132272925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e72d0 0x5615395e27a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc031440 tx=0x7fbbdc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92353fa002f5a5c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.821+0000 7fbbdaa00640 1 -- 172.21.15.1:0/3132272925 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc035030 con 0x5615395e72d0 2024-04-26T01:53:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.822+0000 7fbbdaa00640 1 -- 172.21.15.1:0/3132272925 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbdc041030 con 0x5615395e72d0 2024-04-26T01:53:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.822+0000 7fbbdaa00640 1 -- 172.21.15.1:0/3132272925 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc0046e0 con 0x5615395e72d0 2024-04-26T01:53:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.822+0000 7fbbf7c00740 1 -- 172.21.15.1:0/3132272925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e72d0 msgr2=0x5615395e27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.822+0000 7fbbf7c00740 1 --2- 172.21.15.1:0/3132272925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e72d0 0x5615395e27a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc031440 tx=0x7fbbdc000bc0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.822+0000 7fbbf7c00740 1 -- 172.21.15.1:0/3132272925 shutdown_connections 2024-04-26T01:53:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.822+0000 7fbbf7c00740 1 --2- 172.21.15.1:0/3132272925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395f1b90 0x5615395ef220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.822+0000 7fbbf7c00740 1 --2- 172.21.15.1:0/3132272925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e72d0 0x5615395e27a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.822+0000 7fbbf7c00740 1 --2- 172.21.15.1:0/3132272925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615395e69b0 0x5615395fd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.822+0000 7fbbf7c00740 1 -- 172.21.15.1:0/3132272925 >> 172.21.15.1:0/3132272925 conn(0x5615395e2ec0 msgr2=0x5615395eeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.822+0000 7fbbf7c00740 1 -- 172.21.15.1:0/3132272925 shutdown_connections 2024-04-26T01:53:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.822+0000 7fbbf7c00740 1 -- 172.21.15.1:0/3132272925 wait complete. 2024-04-26T01:53:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.823+0000 7fbbf7c00740 1 -- start start 2024-04-26T01:53:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.823+0000 7fbbf7c00740 1 Processor -- start 2024-04-26T01:53:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.823+0000 7fbbf7c00740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395e69b0 0x5615395f9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.824+0000 7fbbf7c00740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e72d0 0x5615395f71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.824+0000 7fbbf7c00740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615395f1b90 0x5615395f7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.824+0000 7fbbf7c00740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01ae3b0 con 0x5615395e72d0 2024-04-26T01:53:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.824+0000 7fbbf7c00740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01af820 con 0x5615395f1b90 2024-04-26T01:53:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.824+0000 7fbbf7c00740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01aea80 con 0x5615395e69b0 2024-04-26T01:53:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.824+0000 7fbbf7c00740 10 client.0 _ll_register_callbacks cb 0x7fbbdc004480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:53:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.824+0000 7fbbee800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395e69b0 0x5615395f9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.824+0000 7fbbed400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e72d0 0x5615395f71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.824+0000 7fbbee800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395e69b0 0x5615395f9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:47462/0 (socket says 172.21.15.1:47462) 2024-04-26T01:53:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.824+0000 7fbbee800640 1 -- 172.21.15.1:0/2547331156 learned_addr learned my addr 172.21.15.1:0/2547331156 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.824+0000 7fbbee800640 1 --2- 172.21.15.1:0/2547331156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615395f1b90 0x5615395f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.824+0000 7fbbed400640 1 -- 172.21.15.1:0/2547331156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395e69b0 msgr2=0x5615395f9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.824+0000 7fbbed400640 1 --2- 172.21.15.1:0/2547331156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395e69b0 0x5615395f9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.824+0000 7fbbed400640 1 -- 172.21.15.1:0/2547331156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615395f1b90 msgr2=0x5615395f7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.824+0000 7fbbed400640 1 --2- 172.21.15.1:0/2547331156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615395f1b90 0x5615395f7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.824+0000 7fbbed400640 1 -- 172.21.15.1:0/2547331156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615395f7df0 con 0x5615395e72d0 2024-04-26T01:53:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.824+0000 7fbbee800640 1 --2- 172.21.15.1:0/2547331156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395e69b0 0x5615395f9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.825+0000 7fbbed400640 1 --2- 172.21.15.1:0/2547331156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e72d0 0x5615395f71c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x5615395ee840 tx=0x7fbbdc005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d94c927cff24f9e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.825+0000 7fbbcfe00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:53:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.825+0000 7fbbcfe00640 1 -- 172.21.15.1:0/2547331156 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc035070 con 0x5615395e72d0 2024-04-26T01:53:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.825+0000 7fbbcfe00640 1 -- 172.21.15.1:0/2547331156 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fbbdc0048e0 con 0x5615395e72d0 2024-04-26T01:53:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.825+0000 7fbbf7c00740 1 -- 172.21.15.1:0/2547331156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbdc041030 con 0x5615395e72d0 2024-04-26T01:53:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.825+0000 7fbbcfe00640 1 -- 172.21.15.1:0/2547331156 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbbdc004d10 con 0x5615395e72d0 2024-04-26T01:53:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.825+0000 7fbbf7c00740 10 client.15279 fetch_fsmap learned FSMap version 28 2024-04-26T01:53:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.825+0000 7fbbf7c00740 1 -- 172.21.15.1:0/2547331156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5615396682a0 con 0x5615395e72d0 2024-04-26T01:53:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.826+0000 7fbbcfe00640 1 -- 172.21.15.1:0/2547331156 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbdc041030 con 0x5615395e72d0 2024-04-26T01:53:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.826+0000 7fbbcfe00640 1 -- 172.21.15.1:0/2547331156 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7fbbdc04f480 con 0x5615395e72d0 2024-04-26T01:53:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.826+0000 7fbbf7c00740 10 client.15279 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:53:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.826+0000 7fbbf7c00740 10 client.15279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:53:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.826+0000 7fbbf7c00740 20 client.15279 populate_metadata read hostname 'smithi001' 2024-04-26T01:53:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.826+0000 7fbbf7c00740 1 --2- 172.21.15.1:0/2547331156 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561539668950 0x56153966ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.826+0000 7fbbf7c00740 4 client.15279 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:53:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.826+0000 7fbbf7c00740 1 -- 172.21.15.1:0/2547331156 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56153955a320 con 0x561539668950 2024-04-26T01:53:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.826+0000 7fbbee800640 1 --2- 172.21.15.1:0/2547331156 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561539668950 0x56153966ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.827+0000 7fbbee800640 1 --2- 172.21.15.1:0/2547331156 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561539668950 0x56153966ad70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e98a8fbee9a00be7 server_cookie=903cbf3ec76ee4b5 in_seq=0 out_seq=0 2024-04-26T01:53:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.827+0000 7fbbcfe00640 10 client.15279 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.833+0000 7fbbcfe00640 1 -- 172.21.15.1:0/2547331156 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56153955a320 con 0x561539668950 2024-04-26T01:53:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.833+0000 7fbbcfe00640 10 client.15279 handle_command_reply: tid=0 2024-04-26T01:53:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.833+0000 7fbbf7c00740 1 client.15279 shutdown 2024-04-26T01:53:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.833+0000 7fbbf7c00740 1 -- 172.21.15.1:0/2547331156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e72d0 msgr2=0x5615395f71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.833+0000 7fbbf7c00740 1 --2- 172.21.15.1:0/2547331156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e72d0 0x5615395f71c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x5615395ee840 tx=0x7fbbdc005d90 comp rx=0 tx=0).stop 2024-04-26T01:53:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.833+0000 7fbbf7c00740 1 -- 172.21.15.1:0/2547331156 shutdown_connections 2024-04-26T01:53:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.833+0000 7fbbf7c00740 1 --2- 172.21.15.1:0/2547331156 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561539668950 0x56153966ad70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.833+0000 7fbbf7c00740 1 --2- 172.21.15.1:0/2547331156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615395f1b90 0x5615395f7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.833+0000 7fbbf7c00740 1 --2- 172.21.15.1:0/2547331156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615395e72d0 0x5615395f71c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.833+0000 7fbbf7c00740 1 --2- 172.21.15.1:0/2547331156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615395e69b0 0x5615395f9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.833+0000 7fbbf7c00740 1 -- 172.21.15.1:0/2547331156 >> 172.21.15.1:0/2547331156 conn(0x5615395e2ec0 msgr2=0x5615395e8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.833+0000 7fbbcfe00640 0 client.15279 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.833+0000 7fbbf7c00740 1 -- 172.21.15.1:0/2547331156 shutdown_connections 2024-04-26T01:53:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.833+0000 7fbbf7c00740 1 -- 172.21.15.1:0/2547331156 wait complete. 2024-04-26T01:53:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.833+0000 7fbbf7c00740 20 client.15279 trim_cache size 0 max 16384 2024-04-26T01:53:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:53:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:53:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:53:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:53:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.834+0000 7fbbef200640 1 -- 172.21.15.1:0/3426493838 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fbbd40a3d80 msgr2=0x7fbbd40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.834+0000 7fbbef200640 1 --2- 172.21.15.1:0/3426493838 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fbbd40a3d80 0x7fbbd40a6240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x5615395e7f50 tx=0x7fbbe00d45e0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.834+0000 7fbbef200640 1 -- 172.21.15.1:0/3426493838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbf0100b30 msgr2=0x7fbbf01a0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.834+0000 7fbbef200640 1 --2- 172.21.15.1:0/3426493838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbf0100b30 0x7fbbf01a0460 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000e9f0 tx=0x7fbbe000eeb0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.834+0000 7fbbef200640 1 -- 172.21.15.1:0/3426493838 shutdown_connections 2024-04-26T01:53:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.834+0000 7fbbef200640 1 --2- 172.21.15.1:0/3426493838 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fbbd40a3d80 0x7fbbd40a6240 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.834+0000 7fbbef200640 1 --2- 172.21.15.1:0/3426493838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbf010e6e0 0x7fbbf019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.834+0000 7fbbef200640 1 --2- 172.21.15.1:0/3426493838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbf01014e0 0x7fbbf01a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.834+0000 7fbbef200640 1 --2- 172.21.15.1:0/3426493838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbf0100b30 0x7fbbf01a0460 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.834+0000 7fbbef200640 1 -- 172.21.15.1:0/3426493838 >> 172.21.15.1:0/3426493838 conn(0x7fbbf00fc730 msgr2=0x7fbbf00feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.834+0000 7fbbef200640 1 -- 172.21.15.1:0/3426493838 shutdown_connections 2024-04-26T01:53:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:17.834+0000 7fbbef200640 1 -- 172.21.15.1:0/3426493838 wait complete. 2024-04-26T01:53:18.108 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:17 smithi001 ceph-mon[27822]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-26T01:53:18.108 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:17 smithi001 ceph-mon[27822]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T01:53:18.108 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:17 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:53:18.108 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:17 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.173:6834/453427322,v1:172.21.15.173:6835/453427322] up:boot 2024-04-26T01:53:18.108 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:17 smithi001 ceph-mon[27822]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T01:53:18.108 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:17 smithi001 ceph-mon[27822]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:53:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:17 smithi097 ceph-mon[32853]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-26T01:53:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:17 smithi097 ceph-mon[32853]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T01:53:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:17 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:53:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:17 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.173:6834/453427322,v1:172.21.15.173:6835/453427322] up:boot 2024-04-26T01:53:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:17 smithi097 ceph-mon[32853]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T01:53:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:17 smithi097 ceph-mon[32853]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:53:18.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:17 smithi173 ceph-mon[32553]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-26T01:53:18.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:17 smithi173 ceph-mon[32553]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T01:53:18.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:17 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:53:18.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:17 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.173:6834/453427322,v1:172.21.15.173:6835/453427322] up:boot 2024-04-26T01:53:18.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:17 smithi173 ceph-mon[32553]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T01:53:18.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:17 smithi173 ceph-mon[32553]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:53:20.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:19 smithi001 ceph-mon[27822]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-04-26T01:53:20.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.145+0000 7f0d7cc00700 1 -- 172.21.15.1:0/2669815459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d781034b0 msgr2=0x7f0d7810f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:20.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.145+0000 7f0d7cc00700 1 --2- 172.21.15.1:0/2669815459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d781034b0 0x7f0d7810f920 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009fc0 tx=0x7f0d7000b330 comp rx=0 tx=0).stop 2024-04-26T01:53:20.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.146+0000 7f0d7cc00700 1 -- 172.21.15.1:0/2669815459 shutdown_connections 2024-04-26T01:53:20.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.146+0000 7f0d7cc00700 1 --2- 172.21.15.1:0/2669815459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d781034b0 0x7f0d7810f920 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.146+0000 7f0d7cc00700 1 --2- 172.21.15.1:0/2669815459 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d78107940 0x7f0d78107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.146+0000 7f0d7cc00700 1 --2- 172.21.15.1:0/2669815459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d78102af0 0x7f0d78102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.146+0000 7f0d7cc00700 1 -- 172.21.15.1:0/2669815459 >> 172.21.15.1:0/2669815459 conn(0x7f0d78079ba0 msgr2=0x7f0d78079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:20.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.146+0000 7f0d7cc00700 1 -- 172.21.15.1:0/2669815459 shutdown_connections 2024-04-26T01:53:20.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.146+0000 7f0d7cc00700 1 -- 172.21.15.1:0/2669815459 wait complete. 2024-04-26T01:53:20.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.148+0000 7f0d7cc00700 1 Processor -- start 2024-04-26T01:53:20.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.148+0000 7f0d7cc00700 1 -- start start 2024-04-26T01:53:20.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.149+0000 7f0d7cc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d78102af0 0x7f0d78199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:20.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.149+0000 7f0d7cc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d781034b0 0x7f0d781995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:20.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.149+0000 7f0d7cc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d78107940 0x7f0d7819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:20.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.149+0000 7f0d7cc00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0d78112c40 con 0x7f0d78107940 2024-04-26T01:53:20.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.149+0000 7f0d7cc00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0d78112ac0 con 0x7f0d781034b0 2024-04-26T01:53:20.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.149+0000 7f0d7cc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0d78112dc0 con 0x7f0d78102af0 2024-04-26T01:53:20.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.149+0000 7f0d76800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d78107940 0x7f0d7819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:20.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.149+0000 7f0d75400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d781034b0 0x7f0d781995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:20.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.149+0000 7f0d75400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d781034b0 0x7f0d781995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:54950/0 (socket says 172.21.15.1:54950) 2024-04-26T01:53:20.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.149+0000 7f0d76800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d78107940 0x7f0d7819d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41062/0 (socket says 172.21.15.1:41062) 2024-04-26T01:53:20.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.149+0000 7f0d75400700 1 -- 172.21.15.1:0/3790084648 learned_addr learned my addr 172.21.15.1:0/3790084648 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:20.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.149+0000 7f0d75e00700 1 --2- 172.21.15.1:0/3790084648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d78102af0 0x7f0d78199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:20.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.150+0000 7f0d75400700 1 -- 172.21.15.1:0/3790084648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d78102af0 msgr2=0x7f0d78199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:20.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.150+0000 7f0d75400700 1 --2- 172.21.15.1:0/3790084648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d78102af0 0x7f0d78199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.150+0000 7f0d75400700 1 -- 172.21.15.1:0/3790084648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d78107940 msgr2=0x7f0d7819d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:20.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.150+0000 7f0d75400700 1 --2- 172.21.15.1:0/3790084648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d78107940 0x7f0d7819d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.150+0000 7f0d75400700 1 -- 172.21.15.1:0/3790084648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7819e0a0 con 0x7f0d781034b0 2024-04-26T01:53:20.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.150+0000 7f0d75400700 1 --2- 172.21.15.1:0/3790084648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d781034b0 0x7f0d781995d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400e9a0 tx=0x7f0d6400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:20.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.150+0000 7f0d6ea00700 1 -- 172.21.15.1:0/3790084648 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6400ca10 con 0x7f0d781034b0 2024-04-26T01:53:20.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.151+0000 7f0d6ea00700 1 -- 172.21.15.1:0/3790084648 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d6400cbb0 con 0x7f0d781034b0 2024-04-26T01:53:20.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.151+0000 7f0d6ea00700 1 -- 172.21.15.1:0/3790084648 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64018680 con 0x7f0d781034b0 2024-04-26T01:53:20.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.151+0000 7f0d7cc00700 1 -- 172.21.15.1:0/3790084648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7819e390 con 0x7f0d781034b0 2024-04-26T01:53:20.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.151+0000 7f0d7cc00700 1 -- 172.21.15.1:0/3790084648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d781a5c10 con 0x7f0d781034b0 2024-04-26T01:53:20.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.155+0000 7f0d6ea00700 1 -- 172.21.15.1:0/3790084648 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f0d64010430 con 0x7f0d781034b0 2024-04-26T01:53:20.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.156+0000 7f0d874fcb80 1 Processor -- start 2024-04-26T01:53:20.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.156+0000 7f0d874fcb80 1 -- start start 2024-04-26T01:53:20.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.156+0000 7f0d874fcb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72ec150 0x55f1d72ec530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:20.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.156+0000 7f0d874fcb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72eca70 0x55f1d72f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:20.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.156+0000 7f0d874fcb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72f7900 0x55f1d72f9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:20.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.156+0000 7f0d874fcb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f1d723af90 con 0x55f1d72f7900 2024-04-26T01:53:20.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.156+0000 7f0d874fcb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f1d71e1830 con 0x55f1d72eca70 2024-04-26T01:53:20.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.156+0000 7f0d874fcb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f1d7236f50 con 0x55f1d72ec150 2024-04-26T01:53:20.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.157+0000 7f0d76800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72ec150 0x55f1d72ec530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:20.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.157+0000 7f0d75400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72f7900 0x55f1d72f9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:20.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.157+0000 7f0d76800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72ec150 0x55f1d72ec530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:47494/0 (socket says 172.21.15.1:47494) 2024-04-26T01:53:20.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.157+0000 7f0d76800700 1 -- 172.21.15.1:0/3719811906 learned_addr learned my addr 172.21.15.1:0/3719811906 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:20.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.157+0000 7f0d75e00700 1 --2- 172.21.15.1:0/3719811906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72eca70 0x55f1d72f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:20.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.157+0000 7f0d75e00700 1 -- 172.21.15.1:0/3719811906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72ec150 msgr2=0x55f1d72ec530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:20.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.157+0000 7f0d75e00700 1 --2- 172.21.15.1:0/3719811906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72ec150 0x55f1d72ec530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.157+0000 7f0d75e00700 1 -- 172.21.15.1:0/3719811906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72f7900 msgr2=0x55f1d72f9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:20.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.157+0000 7f0d75e00700 1 --2- 172.21.15.1:0/3719811906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72f7900 0x55f1d72f9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.157+0000 7f0d75e00700 1 -- 172.21.15.1:0/3719811906 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1d71d7780 con 0x55f1d72eca70 2024-04-26T01:53:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.157+0000 7f0d76800700 1 --2- 172.21.15.1:0/3719811906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72ec150 0x55f1d72ec530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.157+0000 7f0d6ea00700 1 --2- 172.21.15.1:0/3790084648 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0d5c0a3e30 0x7f0d5c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.158+0000 7f0d75e00700 1 --2- 172.21.15.1:0/3719811906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72eca70 0x55f1d72f73c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009e70 tx=0x7f0d68011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ae4aa28492e53a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.158+0000 7f0d6ea00700 1 -- 172.21.15.1:0/3790084648 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f0d640d09a0 con 0x7f0d781034b0 2024-04-26T01:53:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.158+0000 7f0d76800700 1 --2- 172.21.15.1:0/3790084648 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0d5c0a3e30 0x7f0d5c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.158+0000 7f0d63400700 1 -- 172.21.15.1:0/3719811906 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68004250 con 0x55f1d72eca70 2024-04-26T01:53:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.158+0000 7f0d63400700 1 -- 172.21.15.1:0/3719811906 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d680043f0 con 0x55f1d72eca70 2024-04-26T01:53:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.158+0000 7f0d63400700 1 -- 172.21.15.1:0/3719811906 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d680197d0 con 0x55f1d72eca70 2024-04-26T01:53:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.159+0000 7f0d874fcb80 1 -- 172.21.15.1:0/3719811906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72eca70 msgr2=0x55f1d72f73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.159+0000 7f0d76800700 1 --2- 172.21.15.1:0/3790084648 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0d5c0a3e30 0x7f0d5c0a62f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0d70007fd0 tx=0x7f0d70009f10 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.159+0000 7f0d874fcb80 1 --2- 172.21.15.1:0/3719811906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72eca70 0x55f1d72f73c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009e70 tx=0x7f0d68011720 comp rx=0 tx=0).stop 2024-04-26T01:53:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.159+0000 7f0d874fcb80 1 -- 172.21.15.1:0/3719811906 shutdown_connections 2024-04-26T01:53:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.159+0000 7f0d874fcb80 1 --2- 172.21.15.1:0/3719811906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72f7900 0x55f1d72f9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.159+0000 7f0d874fcb80 1 --2- 172.21.15.1:0/3719811906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72ec150 0x55f1d72ec530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.159+0000 7f0d874fcb80 1 --2- 172.21.15.1:0/3719811906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72eca70 0x55f1d72f73c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.159+0000 7f0d874fcb80 1 -- 172.21.15.1:0/3719811906 >> 172.21.15.1:0/3719811906 conn(0x55f1d7217050 msgr2=0x55f1d72e4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:20.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.159+0000 7f0d874fcb80 1 -- 172.21.15.1:0/3719811906 shutdown_connections 2024-04-26T01:53:20.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.159+0000 7f0d874fcb80 1 -- 172.21.15.1:0/3719811906 wait complete. 2024-04-26T01:53:20.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.160+0000 7f0d874fcb80 1 -- start start 2024-04-26T01:53:20.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.160+0000 7f0d874fcb80 1 Processor -- start 2024-04-26T01:53:20.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.161+0000 7f0d874fcb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72ec150 0x55f1d72f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:20.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.161+0000 7f0d874fcb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72eca70 0x55f1d72f42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:20.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.161+0000 7f0d874fcb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72f7900 0x55f1d72f4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:20.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.161+0000 7f0d874fcb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f1d72fa350 con 0x55f1d72ec150 2024-04-26T01:53:20.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.161+0000 7f0d874fcb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f1d71f4960 con 0x55f1d72f7900 2024-04-26T01:53:20.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.161+0000 7f0d874fcb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f1d72fa4d0 con 0x55f1d72eca70 2024-04-26T01:53:20.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.161+0000 7f0d874fcb80 10 client.0 _ll_register_callbacks cb 0x55f1d71d3820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:53:20.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.161+0000 7f0d75e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72ec150 0x55f1d72f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:20.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.161+0000 7f0d76800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72f7900 0x55f1d72f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:20.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.161+0000 7f0d75e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72ec150 0x55f1d72f6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41080/0 (socket says 172.21.15.1:41080) 2024-04-26T01:53:20.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.161+0000 7f0d75e00700 1 -- 172.21.15.1:0/3587987315 learned_addr learned my addr 172.21.15.1:0/3587987315 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:20.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.161+0000 7f0d75400700 1 --2- 172.21.15.1:0/3587987315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72eca70 0x55f1d72f42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:20.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.162+0000 7f0d75e00700 1 -- 172.21.15.1:0/3587987315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72eca70 msgr2=0x55f1d72f42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:20.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.162+0000 7f0d75e00700 1 --2- 172.21.15.1:0/3587987315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72eca70 0x55f1d72f42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.162+0000 7f0d75e00700 1 -- 172.21.15.1:0/3587987315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72f7900 msgr2=0x55f1d72f4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:20.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.162+0000 7f0d75e00700 1 --2- 172.21.15.1:0/3587987315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72f7900 0x55f1d72f4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.162+0000 7f0d75e00700 1 -- 172.21.15.1:0/3587987315 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1d71889d0 con 0x55f1d72ec150 2024-04-26T01:53:20.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.162+0000 7f0d76800700 1 --2- 172.21.15.1:0/3587987315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72f7900 0x55f1d72f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:20.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.162+0000 7f0d75e00700 1 --2- 172.21.15.1:0/3587987315 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72ec150 0x55f1d72f6f60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0d68006010 tx=0x7f0d68005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35f2f276ea851a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:20.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.162+0000 7f0d57e00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:53:20.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.162+0000 7f0d57e00700 1 -- 172.21.15.1:0/3587987315 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68027020 con 0x55f1d72ec150 2024-04-26T01:53:20.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.162+0000 7f0d57e00700 1 -- 172.21.15.1:0/3587987315 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f0d68003890 con 0x55f1d72ec150 2024-04-26T01:53:20.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.162+0000 7f0d874fcb80 1 -- 172.21.15.1:0/3587987315 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d680043f0 con 0x55f1d72ec150 2024-04-26T01:53:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.163+0000 7f0d57e00700 1 -- 172.21.15.1:0/3587987315 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d68032050 con 0x55f1d72ec150 2024-04-26T01:53:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.163+0000 7f0d874fcb80 10 client.15291 fetch_fsmap learned FSMap version 28 2024-04-26T01:53:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.163+0000 7f0d874fcb80 1 -- 172.21.15.1:0/3587987315 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f1d71d1bf0 con 0x55f1d72ec150 2024-04-26T01:53:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.163+0000 7f0d57e00700 1 -- 172.21.15.1:0/3587987315 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d680043f0 con 0x55f1d72ec150 2024-04-26T01:53:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.163+0000 7f0d57e00700 1 -- 172.21.15.1:0/3587987315 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f0d68018070 con 0x55f1d72ec150 2024-04-26T01:53:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.163+0000 7f0d874fcb80 10 client.15291 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:53:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.163+0000 7f0d874fcb80 10 client.15291 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:53:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.163+0000 7f0d874fcb80 20 client.15291 populate_metadata read hostname 'smithi001' 2024-04-26T01:53:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.163+0000 7f0d874fcb80 1 --2- 172.21.15.1:0/3587987315 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55f1d7306fe0 0x55f1d73093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.164+0000 7f0d874fcb80 4 client.15291 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:53:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.164+0000 7f0d874fcb80 1 -- 172.21.15.1:0/3587987315 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1d7248780 con 0x55f1d7306fe0 2024-04-26T01:53:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.164+0000 7f0d75400700 1 --2- 172.21.15.1:0/3587987315 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55f1d7306fe0 0x55f1d73093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.164+0000 7f0d75400700 1 --2- 172.21.15.1:0/3587987315 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55f1d7306fe0 0x55f1d73093d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fbc8a506747f172 server_cookie=e984d6697a21b8aa in_seq=0 out_seq=0 2024-04-26T01:53:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.164+0000 7f0d57e00700 10 client.15291 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.165+0000 7f0d57e00700 1 -- 172.21.15.1:0/3587987315 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f1d7248780 con 0x55f1d7306fe0 2024-04-26T01:53:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.165+0000 7f0d57e00700 10 client.15291 handle_command_reply: tid=0 2024-04-26T01:53:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.165+0000 7f0d874fcb80 1 client.15291 shutdown 2024-04-26T01:53:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.165+0000 7f0d874fcb80 1 -- 172.21.15.1:0/3587987315 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72ec150 msgr2=0x55f1d72f6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.165+0000 7f0d874fcb80 1 --2- 172.21.15.1:0/3587987315 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72ec150 0x55f1d72f6f60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0d68006010 tx=0x7f0d68005760 comp rx=0 tx=0).stop 2024-04-26T01:53:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.166+0000 7f0d874fcb80 1 -- 172.21.15.1:0/3587987315 shutdown_connections 2024-04-26T01:53:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.166+0000 7f0d874fcb80 1 --2- 172.21.15.1:0/3587987315 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55f1d7306fe0 0x55f1d73093d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.166+0000 7f0d874fcb80 1 --2- 172.21.15.1:0/3587987315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72f7900 0x55f1d72f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.166+0000 7f0d57e00700 0 client.15291 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:20.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.166+0000 7f0d874fcb80 1 --2- 172.21.15.1:0/3587987315 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72ec150 0x55f1d72f6f60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.166+0000 7f0d874fcb80 1 --2- 172.21.15.1:0/3587987315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72eca70 0x55f1d72f42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.166+0000 7f0d874fcb80 1 -- 172.21.15.1:0/3587987315 >> 172.21.15.1:0/3587987315 conn(0x55f1d7217050 msgr2=0x55f1d72e3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:20.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.166+0000 7f0d874fcb80 1 -- 172.21.15.1:0/3587987315 shutdown_connections 2024-04-26T01:53:20.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.166+0000 7f0d874fcb80 1 -- 172.21.15.1:0/3587987315 wait complete. 2024-04-26T01:53:20.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.166+0000 7f0d874fcb80 20 client.15291 trim_cache size 0 max 16384 2024-04-26T01:53:20.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.188+0000 7f0d874fcb80 1 Processor -- start 2024-04-26T01:53:20.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.188+0000 7f0d874fcb80 1 -- start start 2024-04-26T01:53:20.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.188+0000 7f0d874fcb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72ec150 0x55f1d72e2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:20.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.188+0000 7f0d874fcb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72eca70 0x55f1d7303310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:20.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.188+0000 7f0d874fcb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72f7900 0x55f1d72f6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:20.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.188+0000 7f0d874fcb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0d781ada90 con 0x55f1d72eca70 2024-04-26T01:53:20.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.188+0000 7f0d874fcb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0d781ab320 con 0x55f1d72f7900 2024-04-26T01:53:20.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.188+0000 7f0d874fcb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0d781a9170 con 0x55f1d72ec150 2024-04-26T01:53:20.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.189+0000 7f0d75400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72eca70 0x55f1d7303310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:20.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.189+0000 7f0d75e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72ec150 0x55f1d72e2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:20.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.189+0000 7f0d75e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72ec150 0x55f1d72e2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:47502/0 (socket says 172.21.15.1:47502) 2024-04-26T01:53:20.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.189+0000 7f0d75e00700 1 -- 172.21.15.1:0/3444118531 learned_addr learned my addr 172.21.15.1:0/3444118531 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:20.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.189+0000 7f0d75e00700 1 -- 172.21.15.1:0/3444118531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72f7900 msgr2=0x55f1d72f6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:20.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.189+0000 7f0d76800700 1 --2- 172.21.15.1:0/3444118531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72f7900 0x55f1d72f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:20.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.189+0000 7f0d75e00700 1 --2- 172.21.15.1:0/3444118531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72f7900 0x55f1d72f6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.189+0000 7f0d75e00700 1 -- 172.21.15.1:0/3444118531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72eca70 msgr2=0x55f1d7303310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:20.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.189+0000 7f0d75e00700 1 --2- 172.21.15.1:0/3444118531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72eca70 0x55f1d7303310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.189+0000 7f0d75e00700 1 -- 172.21.15.1:0/3444118531 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1d72e42d0 con 0x55f1d72ec150 2024-04-26T01:53:20.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.189+0000 7f0d75400700 1 --2- 172.21.15.1:0/3444118531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72eca70 0x55f1d7303310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:20.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.189+0000 7f0d76800700 1 --2- 172.21.15.1:0/3444118531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72f7900 0x55f1d72f6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:53:20.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.189+0000 7f0d75e00700 1 --2- 172.21.15.1:0/3444118531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72ec150 0x55f1d72e2960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0d6801ae60 tx=0x7f0d680099f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b5cbf2adea3a7be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:20.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.190+0000 7f0d62a00700 1 -- 172.21.15.1:0/3444118531 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68027040 con 0x55f1d72ec150 2024-04-26T01:53:20.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.190+0000 7f0d62a00700 1 -- 172.21.15.1:0/3444118531 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d68023450 con 0x55f1d72ec150 2024-04-26T01:53:20.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.190+0000 7f0d62a00700 1 -- 172.21.15.1:0/3444118531 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d680238f0 con 0x55f1d72ec150 2024-04-26T01:53:20.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.190+0000 7f0d874fcb80 1 -- 172.21.15.1:0/3444118531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72ec150 msgr2=0x55f1d72e2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:20.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.190+0000 7f0d874fcb80 1 --2- 172.21.15.1:0/3444118531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72ec150 0x55f1d72e2960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0d6801ae60 tx=0x7f0d680099f0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.191+0000 7f0d874fcb80 1 -- 172.21.15.1:0/3444118531 shutdown_connections 2024-04-26T01:53:20.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.191+0000 7f0d874fcb80 1 --2- 172.21.15.1:0/3444118531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72f7900 0x55f1d72f6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.191+0000 7f0d874fcb80 1 --2- 172.21.15.1:0/3444118531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72ec150 0x55f1d72e2960 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.191+0000 7f0d874fcb80 1 --2- 172.21.15.1:0/3444118531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72eca70 0x55f1d7303310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.191+0000 7f0d874fcb80 1 -- 172.21.15.1:0/3444118531 >> 172.21.15.1:0/3444118531 conn(0x55f1d7217050 msgr2=0x55f1d72ef950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:20.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.191+0000 7f0d874fcb80 1 -- 172.21.15.1:0/3444118531 shutdown_connections 2024-04-26T01:53:20.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.191+0000 7f0d874fcb80 1 -- 172.21.15.1:0/3444118531 wait complete. 2024-04-26T01:53:20.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.191+0000 7f0d874fcb80 1 -- start start 2024-04-26T01:53:20.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.191+0000 7f0d874fcb80 1 Processor -- start 2024-04-26T01:53:20.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.192+0000 7f0d874fcb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72ec150 0x55f1d72e5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:20.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.192+0000 7f0d874fcb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72eca70 0x55f1d72e6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:20.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.192+0000 7f0d874fcb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72f7900 0x55f1d72e67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:20.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.192+0000 7f0d874fcb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0d781a97e0 con 0x55f1d72f7900 2024-04-26T01:53:20.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.192+0000 7f0d874fcb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0d781aac50 con 0x55f1d72eca70 2024-04-26T01:53:20.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.192+0000 7f0d874fcb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0d781a9eb0 con 0x55f1d72ec150 2024-04-26T01:53:20.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.192+0000 7f0d874fcb80 10 client.0 _ll_register_callbacks cb 0x55f1d71d3820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:53:20.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.192+0000 7f0d75e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72ec150 0x55f1d72e5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:20.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.192+0000 7f0d76800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72f7900 0x55f1d72e67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:20.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.192+0000 7f0d75e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72ec150 0x55f1d72e5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:47518/0 (socket says 172.21.15.1:47518) 2024-04-26T01:53:20.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.192+0000 7f0d75e00700 1 -- 172.21.15.1:0/4257345517 learned_addr learned my addr 172.21.15.1:0/4257345517 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:20.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.192+0000 7f0d75400700 1 --2- 172.21.15.1:0/4257345517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72eca70 0x55f1d72e6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.193+0000 7f0d75e00700 1 -- 172.21.15.1:0/4257345517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72eca70 msgr2=0x55f1d72e6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.193+0000 7f0d75e00700 1 --2- 172.21.15.1:0/4257345517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72eca70 0x55f1d72e6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.193+0000 7f0d75e00700 1 -- 172.21.15.1:0/4257345517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72f7900 msgr2=0x55f1d72e67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.193+0000 7f0d75e00700 1 --2- 172.21.15.1:0/4257345517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72f7900 0x55f1d72e67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.193+0000 7f0d75e00700 1 -- 172.21.15.1:0/4257345517 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1d72ff810 con 0x55f1d72ec150 2024-04-26T01:53:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.193+0000 7f0d76800700 1 --2- 172.21.15.1:0/4257345517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72f7900 0x55f1d72e67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.193+0000 7f0d57e00700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:53:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.193+0000 7f0d75e00700 1 --2- 172.21.15.1:0/4257345517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72ec150 0x55f1d72e5d30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0d68011760 tx=0x7f0d68005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d650db74d3afbb25 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.193+0000 7f0d57e00700 1 -- 172.21.15.1:0/4257345517 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68027040 con 0x55f1d72ec150 2024-04-26T01:53:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.193+0000 7f0d57e00700 1 -- 172.21.15.1:0/4257345517 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f0d6800f960 con 0x55f1d72ec150 2024-04-26T01:53:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.193+0000 7f0d874fcb80 1 -- 172.21.15.1:0/4257345517 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d68023450 con 0x55f1d72ec150 2024-04-26T01:53:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.194+0000 7f0d57e00700 1 -- 172.21.15.1:0/4257345517 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d68044050 con 0x55f1d72ec150 2024-04-26T01:53:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.194+0000 7f0d874fcb80 10 client.25447 fetch_fsmap learned FSMap version 28 2024-04-26T01:53:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.194+0000 7f0d874fcb80 1 -- 172.21.15.1:0/4257345517 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f1d73721a0 con 0x55f1d72ec150 2024-04-26T01:53:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.194+0000 7f0d57e00700 1 -- 172.21.15.1:0/4257345517 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d68023450 con 0x55f1d72ec150 2024-04-26T01:53:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.194+0000 7f0d57e00700 1 -- 172.21.15.1:0/4257345517 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8338+0+0 (secure 0 0 0) 0x7f0d68018070 con 0x55f1d72ec150 2024-04-26T01:53:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.194+0000 7f0d874fcb80 10 client.25447 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T01:53:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.194+0000 7f0d874fcb80 10 client.25447 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:53:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.194+0000 7f0d874fcb80 20 client.25447 populate_metadata read hostname 'smithi001' 2024-04-26T01:53:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.194+0000 7f0d874fcb80 1 --2- 172.21.15.1:0/4257345517 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55f1d7372790 0x55f1d7374b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.194+0000 7f0d874fcb80 4 client.25447 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4a66b69e", "release": true, "await": true}] 2024-04-26T01:53:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.194+0000 7f0d874fcb80 1 -- 172.21.15.1:0/4257345517 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4a66b69e", "release": true, "await": true}) -- 0x55f1d7242820 con 0x55f1d7372790 2024-04-26T01:53:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.195+0000 7f0d75400700 1 --2- 172.21.15.1:0/4257345517 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55f1d7372790 0x55f1d7374b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.195+0000 7f0d75400700 1 --2- 172.21.15.1:0/4257345517 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55f1d7372790 0x55f1d7374b80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca3b040ba8246dc0 server_cookie=cc2ddd787d9a1042 in_seq=0 out_seq=0 2024-04-26T01:53:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.195+0000 7f0d57e00700 10 client.25447 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:20.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.199+0000 7f0d57e00700 1 -- 172.21.15.1:0/4257345517 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55f1d7242820 con 0x55f1d7372790 2024-04-26T01:53:20.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.199+0000 7f0d57e00700 10 client.25447 handle_command_reply: tid=0 2024-04-26T01:53:20.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.199+0000 7f0d874fcb80 1 client.25447 shutdown 2024-04-26T01:53:20.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.199+0000 7f0d874fcb80 1 -- 172.21.15.1:0/4257345517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72ec150 msgr2=0x55f1d72e5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:20.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.199+0000 7f0d874fcb80 1 --2- 172.21.15.1:0/4257345517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72ec150 0x55f1d72e5d30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0d68011760 tx=0x7f0d68005ef0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.199+0000 7f0d874fcb80 1 -- 172.21.15.1:0/4257345517 shutdown_connections 2024-04-26T01:53:20.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.199+0000 7f0d874fcb80 1 --2- 172.21.15.1:0/4257345517 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55f1d7372790 0x55f1d7374b80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.199+0000 7f0d874fcb80 1 --2- 172.21.15.1:0/4257345517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d72f7900 0x55f1d72e67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.199+0000 7f0d874fcb80 1 --2- 172.21.15.1:0/4257345517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1d72ec150 0x55f1d72e5d30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.199+0000 7f0d874fcb80 1 --2- 172.21.15.1:0/4257345517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f1d72eca70 0x55f1d72e6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.199+0000 7f0d57e00700 0 client.25447 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:20.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.199+0000 7f0d874fcb80 1 -- 172.21.15.1:0/4257345517 >> 172.21.15.1:0/4257345517 conn(0x55f1d7217050 msgr2=0x55f1d72f6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:20.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.199+0000 7f0d874fcb80 1 -- 172.21.15.1:0/4257345517 shutdown_connections 2024-04-26T01:53:20.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.200+0000 7f0d874fcb80 1 -- 172.21.15.1:0/4257345517 wait complete. 2024-04-26T01:53:20.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.200+0000 7f0d874fcb80 20 client.25447 trim_cache size 0 max 16384 2024-04-26T01:53:20.199 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T01:53:20.199 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 28, 2024-04-26T01:53:20.199 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24518, 2024-04-26T01:53:20.199 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 4, 2024-04-26T01:53:20.199 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-04-26T01:53:20.199 INFO:teuthology.orchestra.run.smithi001.stdout: "4a66b69e": { 2024-04-26T01:53:20.200 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 4, 2024-04-26T01:53:20.200 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-04-26T01:53:20.200 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:53:20.200 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-04-26T01:53:20.200 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:53:20.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T01:53:20.200 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-04-26T01:53:20.200 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 85.6, 2024-04-26T01:53:20.200 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-04-26T01:53:20.200 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-04-26T01:53:20.200 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-04-26T01:53:20.200 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:53:20.200 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-04-26T01:53:20.201 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:53:20.201 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:53:20.201 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:53:20.201 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:53:20.201 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:53:20.201 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:53:20.201 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T01:53:20.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.200+0000 7f0d7cc00700 1 -- 172.21.15.1:0/3790084648 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0d5c0a3e30 msgr2=0x7f0d5c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:20.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.200+0000 7f0d7cc00700 1 --2- 172.21.15.1:0/3790084648 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0d5c0a3e30 0x7f0d5c0a62f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0d70007fd0 tx=0x7f0d70009f10 comp rx=0 tx=0).stop 2024-04-26T01:53:20.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.201+0000 7f0d7cc00700 1 -- 172.21.15.1:0/3790084648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d781034b0 msgr2=0x7f0d781995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:20.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.201+0000 7f0d7cc00700 1 --2- 172.21.15.1:0/3790084648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d781034b0 0x7f0d781995d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400e9a0 tx=0x7f0d6400ecb0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.201+0000 7f0d7cc00700 1 -- 172.21.15.1:0/3790084648 shutdown_connections 2024-04-26T01:53:20.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.201+0000 7f0d7cc00700 1 --2- 172.21.15.1:0/3790084648 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0d5c0a3e30 0x7f0d5c0a62f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.201+0000 7f0d7cc00700 1 --2- 172.21.15.1:0/3790084648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d78107940 0x7f0d7819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.201+0000 7f0d7cc00700 1 --2- 172.21.15.1:0/3790084648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d78102af0 0x7f0d78199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.201+0000 7f0d7cc00700 1 --2- 172.21.15.1:0/3790084648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d781034b0 0x7f0d781995d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:20.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.201+0000 7f0d7cc00700 1 -- 172.21.15.1:0/3790084648 >> 172.21.15.1:0/3790084648 conn(0x7f0d78079ba0 msgr2=0x7f0d78100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:20.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.201+0000 7f0d7cc00700 1 -- 172.21.15.1:0/3790084648 shutdown_connections 2024-04-26T01:53:20.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:53:20.201+0000 7f0d7cc00700 1 -- 172.21.15.1:0/3790084648 wait complete. 2024-04-26T01:53:20.210 INFO:tasks.workunit.client.0.smithi001.stderr:+ set -e 2024-04-26T01:53:20.210 INFO:tasks.workunit.client.0.smithi001.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-04-26T01:53:20.215 INFO:tasks.workunit.client.0.smithi001.stderr:Cloning into 'xfstests-dev'... 2024-04-26T01:53:20.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:19 smithi097 ceph-mon[32853]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-04-26T01:53:20.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:19 smithi173 ceph-mon[32553]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-04-26T01:53:21.194 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4a66b69e', seconds elapsed: 41.5 2024-04-26T01:53:22.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:21 smithi001 ceph-mon[27822]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-26T01:53:22.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:21 smithi097 ceph-mon[32853]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-26T01:53:22.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:21 smithi173 ceph-mon[32553]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-26T01:53:24.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:23 smithi001 ceph-mon[27822]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-26T01:53:24.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:23 smithi097 ceph-mon[32853]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-26T01:53:24.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:23 smithi173 ceph-mon[32553]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-26T01:53:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:25 smithi001 ceph-mon[27822]: pgmap v57: 97 pgs: 97 active+clean; 13 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 6 op/s 2024-04-26T01:53:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:25 smithi001 ceph-mon[27822]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:53:26.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:25 smithi097 ceph-mon[32853]: pgmap v57: 97 pgs: 97 active+clean; 13 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 6 op/s 2024-04-26T01:53:26.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:25 smithi097 ceph-mon[32853]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:53:26.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:25 smithi173 ceph-mon[32553]: pgmap v57: 97 pgs: 97 active+clean; 13 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 6 op/s 2024-04-26T01:53:26.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:25 smithi173 ceph-mon[32553]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:53:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:27 smithi097 ceph-mon[32853]: pgmap v58: 97 pgs: 97 active+clean; 15 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 11 op/s 2024-04-26T01:53:28.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:27 smithi173 ceph-mon[32553]: pgmap v58: 97 pgs: 97 active+clean; 15 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 11 op/s 2024-04-26T01:53:28.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:27 smithi001 ceph-mon[27822]: pgmap v58: 97 pgs: 97 active+clean; 15 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 11 op/s 2024-04-26T01:53:30.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:29 smithi097 ceph-mon[32853]: pgmap v59: 97 pgs: 97 active+clean; 15 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.2 MiB/s wr, 9 op/s 2024-04-26T01:53:30.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:29 smithi173 ceph-mon[32553]: pgmap v59: 97 pgs: 97 active+clean; 15 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.2 MiB/s wr, 9 op/s 2024-04-26T01:53:30.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:29 smithi001 ceph-mon[27822]: pgmap v59: 97 pgs: 97 active+clean; 15 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.2 MiB/s wr, 9 op/s 2024-04-26T01:53:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.987+0000 7f1d59800640 1 -- 172.21.15.1:0/1278334452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d5410c360 msgr2=0x7f1d5410e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.987+0000 7f1d59800640 1 --2- 172.21.15.1:0/1278334452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d5410c360 0x7f1d5410e7e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009ca0 tx=0x7f1d3c02a950 comp rx=0 tx=0).stop 2024-04-26T01:53:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.987+0000 7f1d59800640 1 -- 172.21.15.1:0/1278334452 shutdown_connections 2024-04-26T01:53:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.987+0000 7f1d59800640 1 --2- 172.21.15.1:0/1278334452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d5410c360 0x7f1d5410e7e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.987+0000 7f1d59800640 1 --2- 172.21.15.1:0/1278334452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d541038b0 0x7f1d5410bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.987+0000 7f1d59800640 1 --2- 172.21.15.1:0/1278334452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d54102f90 0x7f1d54103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.987+0000 7f1d59800640 1 -- 172.21.15.1:0/1278334452 >> 172.21.15.1:0/1278334452 conn(0x7f1d540fc730 msgr2=0x7f1d540feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.987+0000 7f1d59800640 1 -- 172.21.15.1:0/1278334452 shutdown_connections 2024-04-26T01:53:30.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.987+0000 7f1d59800640 1 -- 172.21.15.1:0/1278334452 wait complete. 2024-04-26T01:53:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.988+0000 7f1d59800640 1 Processor -- start 2024-04-26T01:53:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.988+0000 7f1d59800640 1 -- start start 2024-04-26T01:53:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.989+0000 7f1d59800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d54102f90 0x7f1d54100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.989+0000 7f1d59800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d541038b0 0x7f1d54101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.989+0000 7f1d59800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d5410c360 0x7f1d54101660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.989+0000 7f1d59800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1d54112000 con 0x7f1d54102f90 2024-04-26T01:53:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.989+0000 7f1d59800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1d54111e80 con 0x7f1d5410c360 2024-04-26T01:53:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.989+0000 7f1d59800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1d54112180 con 0x7f1d541038b0 2024-04-26T01:53:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.989+0000 7f1d53e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d54102f90 0x7f1d54100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.989+0000 7f1d53e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d54102f90 0x7f1d54100be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42990/0 (socket says 172.21.15.1:42990) 2024-04-26T01:53:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.989+0000 7f1d53e00640 1 -- 172.21.15.1:0/2024731998 learned_addr learned my addr 172.21.15.1:0/2024731998 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.989+0000 7f1d53400640 1 --2- 172.21.15.1:0/2024731998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d541038b0 0x7f1d54101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.989+0000 7f1d58e00640 1 --2- 172.21.15.1:0/2024731998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d5410c360 0x7f1d54101660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.990+0000 7f1d53e00640 1 -- 172.21.15.1:0/2024731998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d541038b0 msgr2=0x7f1d54101120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.990+0000 7f1d53e00640 1 --2- 172.21.15.1:0/2024731998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d541038b0 0x7f1d54101120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.990+0000 7f1d53e00640 1 -- 172.21.15.1:0/2024731998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d5410c360 msgr2=0x7f1d54101660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.990+0000 7f1d53e00640 1 --2- 172.21.15.1:0/2024731998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d5410c360 0x7f1d54101660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.990+0000 7f1d53e00640 1 -- 172.21.15.1:0/2024731998 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d541a6a00 con 0x7f1d54102f90 2024-04-26T01:53:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.990+0000 7f1d53e00640 1 --2- 172.21.15.1:0/2024731998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d54102f90 0x7f1d54100be0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00e9f0 tx=0x7f1d4c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.990+0000 7f1d50c00640 1 -- 172.21.15.1:0/2024731998 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c00cd80 con 0x7f1d54102f90 2024-04-26T01:53:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.990+0000 7f1d59800640 1 -- 172.21.15.1:0/2024731998 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d541a6cf0 con 0x7f1d54102f90 2024-04-26T01:53:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.990+0000 7f1d50c00640 1 -- 172.21.15.1:0/2024731998 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d4c0108e0 con 0x7f1d54102f90 2024-04-26T01:53:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.990+0000 7f1d50c00640 1 -- 172.21.15.1:0/2024731998 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c018980 con 0x7f1d54102f90 2024-04-26T01:53:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.990+0000 7f1d59800640 1 -- 172.21.15.1:0/2024731998 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d541a7230 con 0x7f1d54102f90 2024-04-26T01:53:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.994+0000 7f1d5dda8740 1 Processor -- start 2024-04-26T01:53:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.994+0000 7f1d50c00640 1 -- 172.21.15.1:0/2024731998 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f1d4c010430 con 0x7f1d54102f90 2024-04-26T01:53:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.994+0000 7f1d5dda8740 1 -- start start 2024-04-26T01:53:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.994+0000 7f1d5dda8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f738419b0 0x559f73841d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.994+0000 7f1d5dda8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f738422d0 0x559f7384c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.994+0000 7f1d5dda8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f7384cb90 0x559f7384ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.994+0000 7f1d5dda8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559f737a7900 con 0x559f738422d0 2024-04-26T01:53:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.994+0000 7f1d5dda8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559f7374def0 con 0x559f738419b0 2024-04-26T01:53:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.994+0000 7f1d5dda8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559f737a38c0 con 0x559f7384cb90 2024-04-26T01:53:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.995+0000 7f1d58e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f738419b0 0x559f73841d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.995+0000 7f1d58e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f738419b0 0x559f73841d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:33446/0 (socket says 172.21.15.1:33446) 2024-04-26T01:53:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.995+0000 7f1d58e00640 1 -- 172.21.15.1:0/1721145831 learned_addr learned my addr 172.21.15.1:0/1721145831 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.995+0000 7f1d53e00640 1 --2- 172.21.15.1:0/1721145831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f738422d0 0x559f7384c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.995+0000 7f1d53400640 1 --2- 172.21.15.1:0/1721145831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f7384cb90 0x559f7384ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.995+0000 7f1d58e00640 1 -- 172.21.15.1:0/1721145831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f7384cb90 msgr2=0x559f7384ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.995+0000 7f1d58e00640 1 --2- 172.21.15.1:0/1721145831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f7384cb90 0x559f7384ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.995+0000 7f1d58e00640 1 -- 172.21.15.1:0/1721145831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f738422d0 msgr2=0x559f7384c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.995+0000 7f1d58e00640 1 --2- 172.21.15.1:0/1721145831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f738422d0 0x559f7384c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.995+0000 7f1d58e00640 1 -- 172.21.15.1:0/1721145831 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f734ba210 con 0x559f738419b0 2024-04-26T01:53:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.996+0000 7f1d58e00640 1 --2- 172.21.15.1:0/1721145831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f738419b0 0x559f73841d90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009ca0 tx=0x7f1d3c02f450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f28795937ce4f2c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.996+0000 7f1d50c00640 1 --2- 172.21.15.1:0/2024731998 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f1d380a3d00 0x7f1d380a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.996+0000 7f1d49600640 1 -- 172.21.15.1:0/1721145831 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c048070 con 0x559f738419b0 2024-04-26T01:53:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.996+0000 7f1d49600640 1 -- 172.21.15.1:0/1721145831 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d3c003e20 con 0x559f738419b0 2024-04-26T01:53:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.996+0000 7f1d49600640 1 -- 172.21.15.1:0/1721145831 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c034040 con 0x559f738419b0 2024-04-26T01:53:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.996+0000 7f1d50c00640 1 -- 172.21.15.1:0/2024731998 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f1d4c0d0ae0 con 0x7f1d54102f90 2024-04-26T01:53:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.996+0000 7f1d53e00640 1 --2- 172.21.15.1:0/2024731998 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f1d380a3d00 0x7f1d380a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.997+0000 7f1d5dda8740 1 -- 172.21.15.1:0/1721145831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f738419b0 msgr2=0x559f73841d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.997+0000 7f1d5dda8740 1 --2- 172.21.15.1:0/1721145831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f738419b0 0x559f73841d90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009ca0 tx=0x7f1d3c02f450 comp rx=0 tx=0).stop 2024-04-26T01:53:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.997+0000 7f1d5dda8740 1 -- 172.21.15.1:0/1721145831 shutdown_connections 2024-04-26T01:53:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.997+0000 7f1d5dda8740 1 --2- 172.21.15.1:0/1721145831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f7384cb90 0x559f7384ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.997+0000 7f1d53e00640 1 --2- 172.21.15.1:0/2024731998 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f1d380a3d00 0x7f1d380a61c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c0ce2e0 tx=0x7f1d4c018420 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.997+0000 7f1d5dda8740 1 --2- 172.21.15.1:0/1721145831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f738422d0 0x559f7384c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.997+0000 7f1d5dda8740 1 --2- 172.21.15.1:0/1721145831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f738419b0 0x559f73841d90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.997+0000 7f1d5dda8740 1 -- 172.21.15.1:0/1721145831 >> 172.21.15.1:0/1721145831 conn(0x559f7383dec0 msgr2=0x559f7383e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.997+0000 7f1d5dda8740 1 -- 172.21.15.1:0/1721145831 shutdown_connections 2024-04-26T01:53:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.997+0000 7f1d5dda8740 1 -- 172.21.15.1:0/1721145831 wait complete. 2024-04-26T01:53:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.998+0000 7f1d5dda8740 1 -- start start 2024-04-26T01:53:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.998+0000 7f1d5dda8740 1 Processor -- start 2024-04-26T01:53:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.998+0000 7f1d5dda8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f738419b0 0x559f73849ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.998+0000 7f1d5dda8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f738422d0 0x559f7384a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.998+0000 7f1d5dda8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7384cb90 0x559f7384a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.998+0000 7f1d5dda8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559f7384f760 con 0x559f738419b0 2024-04-26T01:53:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.998+0000 7f1d5dda8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559f73761310 con 0x559f7384cb90 2024-04-26T01:53:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.998+0000 7f1d5dda8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559f7384f8e0 con 0x559f738422d0 2024-04-26T01:53:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.999+0000 7f1d5dda8740 10 client.0 _ll_register_callbacks cb 0x559f7372fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:53:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.999+0000 7f1d58e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f738419b0 0x559f73849ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.999+0000 7f1d58e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f738419b0 0x559f73849ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43014/0 (socket says 172.21.15.1:43014) 2024-04-26T01:53:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.999+0000 7f1d58e00640 1 -- 172.21.15.1:0/1382466813 learned_addr learned my addr 172.21.15.1:0/1382466813 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.999+0000 7f1d53400640 1 --2- 172.21.15.1:0/1382466813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f738422d0 0x559f7384a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.999+0000 7f1d58e00640 1 --2- 172.21.15.1:0/1382466813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7384cb90 0x559f7384a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.999+0000 7f1d58e00640 1 -- 172.21.15.1:0/1382466813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f738422d0 msgr2=0x559f7384a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.999+0000 7f1d58e00640 1 --2- 172.21.15.1:0/1382466813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f738422d0 0x559f7384a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.999+0000 7f1d58e00640 1 -- 172.21.15.1:0/1382466813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f738419b0 msgr2=0x559f73849ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.999+0000 7f1d58e00640 1 --2- 172.21.15.1:0/1382466813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f738419b0 0x559f73849ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.999+0000 7f1d58e00640 1 -- 172.21.15.1:0/1382466813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f73586620 con 0x559f7384cb90 2024-04-26T01:53:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:30.999+0000 7f1d53400640 1 --2- 172.21.15.1:0/1382466813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f738422d0 0x559f7384a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.000+0000 7f1d58e00640 1 --2- 172.21.15.1:0/1382466813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7384cb90 0x559f7384a620 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c005e30 tx=0x7f1d3c005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7344697953c1681a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.000+0000 7f1d42000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:53:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.000+0000 7f1d42000640 1 -- 172.21.15.1:0/1382466813 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c048070 con 0x559f7384cb90 2024-04-26T01:53:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.000+0000 7f1d42000640 1 -- 172.21.15.1:0/1382466813 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f1d3c004210 con 0x559f7384cb90 2024-04-26T01:53:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.000+0000 7f1d5dda8740 1 -- 172.21.15.1:0/1382466813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d3c003e20 con 0x559f7384cb90 2024-04-26T01:53:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.000+0000 7f1d42000640 1 -- 172.21.15.1:0/1382466813 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d3c034040 con 0x559f7384cb90 2024-04-26T01:53:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.000+0000 7f1d5dda8740 10 client.25436 fetch_fsmap learned FSMap version 30 2024-04-26T01:53:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.000+0000 7f1d5dda8740 1 -- 172.21.15.1:0/1382466813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559f7385ba50 con 0x559f7384cb90 2024-04-26T01:53:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.000+0000 7f1d42000640 1 -- 172.21.15.1:0/1382466813 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d3c003e20 con 0x559f7384cb90 2024-04-26T01:53:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.001+0000 7f1d42000640 1 -- 172.21.15.1:0/1382466813 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f1d3c0050b0 con 0x559f7384cb90 2024-04-26T01:53:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.001+0000 7f1d5dda8740 10 client.25436 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:53:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.001+0000 7f1d5dda8740 10 client.25436 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:53:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.001+0000 7f1d5dda8740 20 client.25436 populate_metadata read hostname 'smithi001' 2024-04-26T01:53:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.001+0000 7f1d5dda8740 1 --2- 172.21.15.1:0/1382466813 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559f7385cda0 0x559f7385f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.001+0000 7f1d5dda8740 4 client.25436 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:53:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.001+0000 7f1d5dda8740 1 -- 172.21.15.1:0/1382466813 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f737bb8b0 con 0x559f7385cda0 2024-04-26T01:53:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.001+0000 7f1d53400640 1 --2- 172.21.15.1:0/1382466813 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559f7385cda0 0x559f7385f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.002+0000 7f1d53400640 1 --2- 172.21.15.1:0/1382466813 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559f7385cda0 0x559f7385f190 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e0d2dc00e392c09 server_cookie=206c5c2d527e5090 in_seq=0 out_seq=0 2024-04-26T01:53:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.002+0000 7f1d42000640 10 client.25436 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.004+0000 7f1d42000640 1 -- 172.21.15.1:0/1382466813 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f737bb8b0 con 0x559f7385cda0 2024-04-26T01:53:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.004+0000 7f1d42000640 10 client.25436 handle_command_reply: tid=0 2024-04-26T01:53:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.004+0000 7f1d5dda8740 1 client.25436 shutdown 2024-04-26T01:53:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.004+0000 7f1d5dda8740 1 -- 172.21.15.1:0/1382466813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7384cb90 msgr2=0x559f7384a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.004+0000 7f1d5dda8740 1 --2- 172.21.15.1:0/1382466813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7384cb90 0x559f7384a620 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c005e30 tx=0x7f1d3c005d80 comp rx=0 tx=0).stop 2024-04-26T01:53:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.005+0000 7f1d5dda8740 1 -- 172.21.15.1:0/1382466813 shutdown_connections 2024-04-26T01:53:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.005+0000 7f1d5dda8740 1 --2- 172.21.15.1:0/1382466813 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559f7385cda0 0x559f7385f190 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.005+0000 7f1d42000640 0 client.25436 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.005+0000 7f1d5dda8740 1 --2- 172.21.15.1:0/1382466813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7384cb90 0x559f7384a620 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.005+0000 7f1d5dda8740 1 --2- 172.21.15.1:0/1382466813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f738422d0 0x559f7384a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.005+0000 7f1d5dda8740 1 --2- 172.21.15.1:0/1382466813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f738419b0 0x559f73849ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.005+0000 7f1d5dda8740 1 -- 172.21.15.1:0/1382466813 >> 172.21.15.1:0/1382466813 conn(0x559f7383dec0 msgr2=0x559f73843c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.005+0000 7f1d5dda8740 1 -- 172.21.15.1:0/1382466813 shutdown_connections 2024-04-26T01:53:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.005+0000 7f1d5dda8740 1 -- 172.21.15.1:0/1382466813 wait complete. 2024-04-26T01:53:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.005+0000 7f1d5dda8740 20 client.25436 trim_cache size 0 max 16384 2024-04-26T01:53:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.021+0000 7f1d5dda8740 1 Processor -- start 2024-04-26T01:53:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.022+0000 7f1d5dda8740 1 -- start start 2024-04-26T01:53:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.022+0000 7f1d5dda8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f738419b0 0x559f7384a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.022+0000 7f1d5dda8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f738422d0 0x559f7383c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.022+0000 7f1d5dda8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f7384cb90 0x559f7383ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.022+0000 7f1d5dda8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1d541b25a0 con 0x559f7384cb90 2024-04-26T01:53:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.022+0000 7f1d5dda8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1d541afe10 con 0x559f738419b0 2024-04-26T01:53:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.022+0000 7f1d5dda8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1d541adc60 con 0x559f738422d0 2024-04-26T01:53:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.022+0000 7f1d58e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f738419b0 0x559f7384a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.022+0000 7f1d58e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f738419b0 0x559f7384a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:33464/0 (socket says 172.21.15.1:33464) 2024-04-26T01:53:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.022+0000 7f1d58e00640 1 -- 172.21.15.1:0/2529122077 learned_addr learned my addr 172.21.15.1:0/2529122077 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.022+0000 7f1d58e00640 1 --2- 172.21.15.1:0/2529122077 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f7384cb90 0x559f7383ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.022+0000 7f1d53400640 1 --2- 172.21.15.1:0/2529122077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f738422d0 0x559f7383c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.023+0000 7f1d58e00640 1 -- 172.21.15.1:0/2529122077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f738422d0 msgr2=0x559f7383c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.023+0000 7f1d58e00640 1 --2- 172.21.15.1:0/2529122077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f738422d0 0x559f7383c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.023+0000 7f1d58e00640 1 -- 172.21.15.1:0/2529122077 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f7384cb90 msgr2=0x559f7383ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.023+0000 7f1d58e00640 1 --2- 172.21.15.1:0/2529122077 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f7384cb90 0x559f7383ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.023+0000 7f1d58e00640 1 -- 172.21.15.1:0/2529122077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f7384f2f0 con 0x559f738419b0 2024-04-26T01:53:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.023+0000 7f1d58e00640 1 --2- 172.21.15.1:0/2529122077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f738419b0 0x559f7384a270 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0039d0 tx=0x7f1d3c005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=514654c0fbb49cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.023+0000 7f1d48c00640 1 -- 172.21.15.1:0/2529122077 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c048070 con 0x559f738419b0 2024-04-26T01:53:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.023+0000 7f1d48c00640 1 -- 172.21.15.1:0/2529122077 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d3c05f040 con 0x559f738419b0 2024-04-26T01:53:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.023+0000 7f1d48c00640 1 -- 172.21.15.1:0/2529122077 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c003c10 con 0x559f738419b0 2024-04-26T01:53:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.024+0000 7f1d5dda8740 1 -- 172.21.15.1:0/2529122077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f738419b0 msgr2=0x559f7384a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.024+0000 7f1d5dda8740 1 --2- 172.21.15.1:0/2529122077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f738419b0 0x559f7384a270 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0039d0 tx=0x7f1d3c005d80 comp rx=0 tx=0).stop 2024-04-26T01:53:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.024+0000 7f1d5dda8740 1 -- 172.21.15.1:0/2529122077 shutdown_connections 2024-04-26T01:53:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.024+0000 7f1d5dda8740 1 --2- 172.21.15.1:0/2529122077 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f7384cb90 0x559f7383ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.024+0000 7f1d5dda8740 1 --2- 172.21.15.1:0/2529122077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f738422d0 0x559f7383c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.024+0000 7f1d5dda8740 1 --2- 172.21.15.1:0/2529122077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f738419b0 0x559f7384a270 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.024+0000 7f1d5dda8740 1 -- 172.21.15.1:0/2529122077 >> 172.21.15.1:0/2529122077 conn(0x559f7383dec0 msgr2=0x559f73849b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.024+0000 7f1d5dda8740 1 -- 172.21.15.1:0/2529122077 shutdown_connections 2024-04-26T01:53:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.024+0000 7f1d5dda8740 1 -- 172.21.15.1:0/2529122077 wait complete. 2024-04-26T01:53:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.025+0000 7f1d5dda8740 1 -- start start 2024-04-26T01:53:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.025+0000 7f1d5dda8740 1 Processor -- start 2024-04-26T01:53:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.025+0000 7f1d5dda8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f738419b0 0x559f738bb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.025+0000 7f1d5dda8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f738422d0 0x559f738bbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.025+0000 7f1d5dda8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7384cb90 0x559f73861680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.025+0000 7f1d5dda8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1d541ae2d0 con 0x559f738422d0 2024-04-26T01:53:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.025+0000 7f1d5dda8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1d541af740 con 0x559f7384cb90 2024-04-26T01:53:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.025+0000 7f1d5dda8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1d541ae9a0 con 0x559f738419b0 2024-04-26T01:53:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.025+0000 7f1d5dda8740 10 client.0 _ll_register_callbacks cb 0x7f1d3c02fe90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:53:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.025+0000 7f1d58e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f738419b0 0x559f738bb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.026+0000 7f1d58e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f738419b0 0x559f738bb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:55078/0 (socket says 172.21.15.1:55078) 2024-04-26T01:53:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.026+0000 7f1d58e00640 1 -- 172.21.15.1:0/1182185662 learned_addr learned my addr 172.21.15.1:0/1182185662 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.026+0000 7f1d58e00640 1 --2- 172.21.15.1:0/1182185662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7384cb90 0x559f73861680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.026+0000 7f1d53400640 1 --2- 172.21.15.1:0/1182185662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f738422d0 0x559f738bbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.026+0000 7f1d53400640 1 -- 172.21.15.1:0/1182185662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f738419b0 msgr2=0x559f738bb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.026+0000 7f1d53400640 1 --2- 172.21.15.1:0/1182185662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f738419b0 0x559f738bb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.026+0000 7f1d53400640 1 -- 172.21.15.1:0/1182185662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7384cb90 msgr2=0x559f73861680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.026+0000 7f1d53400640 1 --2- 172.21.15.1:0/1182185662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7384cb90 0x559f73861680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.026+0000 7f1d53400640 1 -- 172.21.15.1:0/1182185662 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f73861d70 con 0x559f738422d0 2024-04-26T01:53:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.026+0000 7f1d53400640 1 --2- 172.21.15.1:0/1182185662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f738422d0 0x559f738bbad0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x559f7383f2e0 tx=0x7f1d44013940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c54c1458f670ef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.026+0000 7f1d42000640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:53:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.027+0000 7f1d42000640 1 -- 172.21.15.1:0/1182185662 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d44004220 con 0x559f738422d0 2024-04-26T01:53:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.027+0000 7f1d42000640 1 -- 172.21.15.1:0/1182185662 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f1d440043c0 con 0x559f738422d0 2024-04-26T01:53:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.027+0000 7f1d5dda8740 1 -- 172.21.15.1:0/1182185662 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d3c05f040 con 0x559f738422d0 2024-04-26T01:53:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.027+0000 7f1d42000640 1 -- 172.21.15.1:0/1182185662 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d4401d440 con 0x559f738422d0 2024-04-26T01:53:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.027+0000 7f1d5dda8740 10 client.15315 fetch_fsmap learned FSMap version 30 2024-04-26T01:53:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.027+0000 7f1d5dda8740 1 -- 172.21.15.1:0/1182185662 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559f738c2d70 con 0x559f738422d0 2024-04-26T01:53:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.027+0000 7f1d42000640 1 -- 172.21.15.1:0/1182185662 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d3c05f040 con 0x559f738422d0 2024-04-26T01:53:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.028+0000 7f1d42000640 1 -- 172.21.15.1:0/1182185662 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f1d44003c90 con 0x559f738422d0 2024-04-26T01:53:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.028+0000 7f1d5dda8740 10 client.15315 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:53:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.028+0000 7f1d5dda8740 10 client.15315 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:53:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.028+0000 7f1d5dda8740 20 client.15315 populate_metadata read hostname 'smithi001' 2024-04-26T01:53:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.028+0000 7f1d5dda8740 1 --2- 172.21.15.1:0/1182185662 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559f7385cda0 0x559f738c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.028+0000 7f1d5dda8740 4 client.15315 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:53:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.028+0000 7f1d5dda8740 1 -- 172.21.15.1:0/1182185662 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f737b5320 con 0x559f7385cda0 2024-04-26T01:53:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.028+0000 7f1d58e00640 1 --2- 172.21.15.1:0/1182185662 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559f7385cda0 0x559f738c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.029+0000 7f1d58e00640 1 --2- 172.21.15.1:0/1182185662 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559f7385cda0 0x559f738c5540 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f036133f1b878f7 server_cookie=72ad6d47e511fee in_seq=0 out_seq=0 2024-04-26T01:53:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.029+0000 7f1d42000640 10 client.15315 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:31.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:30 smithi097 ceph-mon[32853]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:53:31.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:30 smithi173 ceph-mon[32553]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:53:31.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:30 smithi001 ceph-mon[27822]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:53:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.782+0000 7f1d42000640 1 -- 172.21.15.1:0/1182185662 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f737b5320 con 0x559f7385cda0 2024-04-26T01:53:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.782+0000 7f1d42000640 10 client.15315 handle_command_reply: tid=0 2024-04-26T01:53:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.782+0000 7f1d5dda8740 1 client.15315 shutdown 2024-04-26T01:53:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.783+0000 7f1d5dda8740 1 -- 172.21.15.1:0/1182185662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f738422d0 msgr2=0x559f738bbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.783+0000 7f1d5dda8740 1 --2- 172.21.15.1:0/1182185662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f738422d0 0x559f738bbad0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x559f7383f2e0 tx=0x7f1d44013940 comp rx=0 tx=0).stop 2024-04-26T01:53:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.783+0000 7f1d5dda8740 1 -- 172.21.15.1:0/1182185662 shutdown_connections 2024-04-26T01:53:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.783+0000 7f1d5dda8740 1 --2- 172.21.15.1:0/1182185662 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559f7385cda0 0x559f738c5540 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.783+0000 7f1d5dda8740 1 --2- 172.21.15.1:0/1182185662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559f7384cb90 0x559f73861680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.783+0000 7f1d5dda8740 1 --2- 172.21.15.1:0/1182185662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f738422d0 0x559f738bbad0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.783+0000 7f1d5dda8740 1 --2- 172.21.15.1:0/1182185662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559f738419b0 0x559f738bb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.783+0000 7f1d42000640 0 client.15315 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.783+0000 7f1d5dda8740 1 -- 172.21.15.1:0/1182185662 >> 172.21.15.1:0/1182185662 conn(0x559f7383dec0 msgr2=0x559f7384a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.783+0000 7f1d5dda8740 1 -- 172.21.15.1:0/1182185662 shutdown_connections 2024-04-26T01:53:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.783+0000 7f1d5dda8740 1 -- 172.21.15.1:0/1182185662 wait complete. 2024-04-26T01:53:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.783+0000 7f1d5dda8740 20 client.15315 trim_cache size 0 max 16384 2024-04-26T01:53:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:53:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:53:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:53:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:53:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.784+0000 7f1d59800640 1 -- 172.21.15.1:0/2024731998 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f1d380a3d00 msgr2=0x7f1d380a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.784+0000 7f1d59800640 1 --2- 172.21.15.1:0/2024731998 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f1d380a3d00 0x7f1d380a61c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c0ce2e0 tx=0x7f1d4c018420 comp rx=0 tx=0).stop 2024-04-26T01:53:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.784+0000 7f1d59800640 1 -- 172.21.15.1:0/2024731998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d54102f90 msgr2=0x7f1d54100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.784+0000 7f1d59800640 1 --2- 172.21.15.1:0/2024731998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d54102f90 0x7f1d54100be0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00e9f0 tx=0x7f1d4c00eeb0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.785+0000 7f1d59800640 1 -- 172.21.15.1:0/2024731998 shutdown_connections 2024-04-26T01:53:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.785+0000 7f1d59800640 1 --2- 172.21.15.1:0/2024731998 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f1d380a3d00 0x7f1d380a61c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.785+0000 7f1d59800640 1 --2- 172.21.15.1:0/2024731998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d5410c360 0x7f1d54101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.785+0000 7f1d59800640 1 --2- 172.21.15.1:0/2024731998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d541038b0 0x7f1d54101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.785+0000 7f1d59800640 1 --2- 172.21.15.1:0/2024731998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d54102f90 0x7f1d54100be0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.785+0000 7f1d59800640 1 -- 172.21.15.1:0/2024731998 >> 172.21.15.1:0/2024731998 conn(0x7f1d540fc730 msgr2=0x7f1d540fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.785+0000 7f1d59800640 1 -- 172.21.15.1:0/2024731998 shutdown_connections 2024-04-26T01:53:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:31.785+0000 7f1d59800640 1 -- 172.21.15.1:0/2024731998 wait complete. 2024-04-26T01:53:32.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:31 smithi097 ceph-mon[32853]: pgmap v60: 97 pgs: 97 active+clean; 17 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 18 op/s 2024-04-26T01:53:32.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:31 smithi173 ceph-mon[32553]: pgmap v60: 97 pgs: 97 active+clean; 17 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 18 op/s 2024-04-26T01:53:32.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:31 smithi001 ceph-mon[27822]: pgmap v60: 97 pgs: 97 active+clean; 17 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 18 op/s 2024-04-26T01:53:34.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:33 smithi097 ceph-mon[32853]: pgmap v61: 97 pgs: 97 active+clean; 17 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 15 op/s 2024-04-26T01:53:34.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:33 smithi173 ceph-mon[32553]: pgmap v61: 97 pgs: 97 active+clean; 17 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 15 op/s 2024-04-26T01:53:34.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:33 smithi001 ceph-mon[27822]: pgmap v61: 97 pgs: 97 active+clean; 17 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 15 op/s 2024-04-26T01:53:36.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:35 smithi097 ceph-mon[32853]: pgmap v62: 97 pgs: 97 active+clean; 17 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-04-26T01:53:36.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:35 smithi173 ceph-mon[32553]: pgmap v62: 97 pgs: 97 active+clean; 17 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-04-26T01:53:36.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:35 smithi001 ceph-mon[27822]: pgmap v62: 97 pgs: 97 active+clean; 17 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-04-26T01:53:38.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:37 smithi097 ceph-mon[32853]: pgmap v63: 97 pgs: 97 active+clean; 16 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 851 KiB/s wr, 368 op/s 2024-04-26T01:53:38.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:37 smithi173 ceph-mon[32553]: pgmap v63: 97 pgs: 97 active+clean; 16 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 851 KiB/s wr, 368 op/s 2024-04-26T01:53:38.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:37 smithi001 ceph-mon[27822]: pgmap v63: 97 pgs: 97 active+clean; 16 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 851 KiB/s wr, 368 op/s 2024-04-26T01:53:40.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:39 smithi097 ceph-mon[32853]: pgmap v64: 97 pgs: 97 active+clean; 16 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 638 KiB/s wr, 362 op/s 2024-04-26T01:53:40.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:39 smithi173 ceph-mon[32553]: pgmap v64: 97 pgs: 97 active+clean; 16 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 638 KiB/s wr, 362 op/s 2024-04-26T01:53:40.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:39 smithi001 ceph-mon[27822]: pgmap v64: 97 pgs: 97 active+clean; 16 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 638 KiB/s wr, 362 op/s 2024-04-26T01:53:42.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:41 smithi097 ceph-mon[32853]: pgmap v65: 97 pgs: 97 active+clean; 17 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 693 KiB/s wr, 367 op/s 2024-04-26T01:53:42.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:41 smithi173 ceph-mon[32553]: pgmap v65: 97 pgs: 97 active+clean; 17 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 693 KiB/s wr, 367 op/s 2024-04-26T01:53:42.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:41 smithi001 ceph-mon[27822]: pgmap v65: 97 pgs: 97 active+clean; 17 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 693 KiB/s wr, 367 op/s 2024-04-26T01:53:43.581 INFO:tasks.workunit.client.0.smithi001.stderr:Updating files: 7% (321/4290) Updating files: 8% (344/4290) Updating files: 9% (387/4290) Updating files: 10% (429/4290) Updating files: 11% (472/4290) Updating files: 11% (476/4290) Updating files: 12% (515/4290) Updating files: 13% (558/4290) Updating files: 14% (601/4290) Updating files: 15% (644/4290) Updating files: 16% (687/4290) Updating files: 17% (730/4290) Updating files: 18% (773/4290) Updating files: 19% (816/4290) Updating files: 20% (858/4290) Updating files: 21% (901/4290) Updating files: 22% (944/4290) Updating files: 23% (987/4290) Updating files: 23% (1012/4290) Updating files: 24% (1030/4290) Updating files: 25% (1073/4290) Updating files: 26% (1116/4290) Updating files: 27% (1159/4290) Updating files: 28% (1202/4290) Updating files: 29% (1245/4290) Updating files: 30% (1287/4290) Updating files: 30% (1288/4290) Updating files: 30% (1320/4290) Updating files: 31% (1330/4290) Updating files: 32% (1373/4290) Updating files: 33% (1416/4290) Updating files: 34% (1459/4290) Updating files: 35% (1502/4290) Updating files: 36% (1545/4290) Updating files: 37% (1588/4290) Updating files: 38% (1631/4290) Updating files: 39% (1674/4290) Updating files: 40% (1716/4290) Updating files: 41% (1759/4290) Updating files: 42% (1802/4290) Updating files: 43% (1845/4290) Updating files: 43% (1875/4290) Updating files: 44% (1888/4290) Updating files: 45% (1931/4290) Updating files: 46% (1974/4290) Updating files: 47% (2017/4290) Updating files: 48% (2060/4290) Updating files: 49% (2103/4290) Updating files: 50% (2145/4290) Updating files: 51% (2188/4290) Updating files: 52% (2231/4290) Updating files: 52% (2259/4290) Updating files: 53% (2274/4290) Updating files: 54% (2317/4290) Updating files: 55% (2360/4290) Updating files: 56% (2403/4290) Updating files: 57% (2446/4290) Updating files: 58% (2489/4290) Updating files: 59% (2532/4290) Updating files: 60% (2574/4290) Updating files: 61% (2617/4290) Updating files: 62% (2660/4290) Updating files: 63% (2703/4290) Updating files: 63% (2743/4290) Updating files: 64% (2746/4290) Updating files: 64% (2784/4290) Updating files: 65% (2789/4290) Updating files: 66% (2832/4290) Updating files: 67% (2875/4290) Updating files: 68% (2918/4290) Updating files: 69% (2961/4290) Updating files: 69% (2973/4290) Updating files: 70% (3003/4290) Updating files: 70% (3022/4290) Updating files: 71% (3046/4290) Updating files: 72% (3089/4290) Updating files: 73% (3132/4290) Updating files: 74% (3175/4290) Updating files: 74% (3190/4290) Updating files: 75% (3218/4290) Updating files: 76% (3261/4290) Updating files: 77% (3304/4290) Updating files: 78% (3347/4290) Updating files: 79% (3390/4290) Updating files: 80% (3432/4290) Updating files: 81% (3475/4290) Updating files: 81% (3497/4290) Updating files: 82% (3518/4290) Updating files: 83% (3561/4290) Updating files: 84% (3604/4290) Updating files: 85% (3647/4290) Updating files: 86% (3690/4290) Updating files: 86% (3727/4290) Updating files: 87% (3733/4290) Updating files: 88% (3776/4290) Updating files: 89% (3819/4290) Updating files: 90% (3861/4290) Updating files: 91% (3904/4290) Updating files: 92% (3947/4290) Updating files: 93% (3990/4290) Updating files: 94% (4033/4290) Updating files: 94% (4071/4290) Updating files: 95% (4076/4290) Updating files: 96% (4119/4290) Updating files: 97% (4162/4290) Updating files: 98% (4205/4290) Updating files: 99% (4248/4290) Updating files: 100% (4290/4290) Updating files: 100% (4290/4290), done. 2024-04-26T01:53:43.603 INFO:tasks.workunit.client.0.smithi001.stderr:+ cd xfstests-dev 2024-04-26T01:53:43.603 INFO:tasks.workunit.client.0.smithi001.stderr:+ make -j4 2024-04-26T01:53:43.677 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize -cfi 2024-04-26T01:53:44.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:43 smithi097 ceph-mon[32853]: pgmap v66: 97 pgs: 97 active+clean; 17 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 494 KiB/s wr, 358 op/s 2024-04-26T01:53:44.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:43 smithi173 ceph-mon[32553]: pgmap v66: 97 pgs: 97 active+clean; 17 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 494 KiB/s wr, 358 op/s 2024-04-26T01:53:44.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:43 smithi001 ceph-mon[27822]: pgmap v66: 97 pgs: 97 active+clean; 17 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 494 KiB/s wr, 358 op/s 2024-04-26T01:53:44.432 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-26T01:53:44.432 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file './config.guess' 2024-04-26T01:53:44.449 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file './config.sub' 2024-04-26T01:53:44.466 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file './install-sh' 2024-04-26T01:53:44.485 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file './ltmain.sh' 2024-04-26T01:53:44.581 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-26T01:53:44.581 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-26T01:53:44.610 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-26T01:53:44.636 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-26T01:53:44.659 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-26T01:53:44.684 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-26T01:53:44.710 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-26T01:53:44.715 INFO:tasks.workunit.client.0.smithi001.stdout:cp include/install-sh . 2024-04-26T01:53:44.725 INFO:tasks.workunit.client.0.smithi001.stdout:aclocal -I m4 2024-04-26T01:53:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.931+0000 7fcaeda00640 1 -- 172.21.15.1:0/2174565685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcae806a630 msgr2=0x7fcae806aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.931+0000 7fcaeda00640 1 --2- 172.21.15.1:0/2174565685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcae806a630 0x7fcae806aa10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcad8009900 tx=0x7fcad802a480 comp rx=0 tx=0).stop 2024-04-26T01:53:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.931+0000 7fcaeda00640 1 -- 172.21.15.1:0/2174565685 shutdown_connections 2024-04-26T01:53:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.931+0000 7fcaeda00640 1 --2- 172.21.15.1:0/2174565685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae810e960 0x7fcae8110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.931+0000 7fcaeda00640 1 --2- 172.21.15.1:0/2174565685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcae806af50 0x7fcae810e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.931+0000 7fcaeda00640 1 --2- 172.21.15.1:0/2174565685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcae806a630 0x7fcae806aa10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.931+0000 7fcaeda00640 1 -- 172.21.15.1:0/2174565685 >> 172.21.15.1:0/2174565685 conn(0x7fcae8077f50 msgr2=0x7fcae8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.931+0000 7fcaeda00640 1 -- 172.21.15.1:0/2174565685 shutdown_connections 2024-04-26T01:53:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.931+0000 7fcaeda00640 1 -- 172.21.15.1:0/2174565685 wait complete. 2024-04-26T01:53:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.932+0000 7fcaeda00640 1 Processor -- start 2024-04-26T01:53:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.933+0000 7fcaeda00640 1 -- start start 2024-04-26T01:53:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.933+0000 7fcaeda00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcae806a630 0x7fcae81a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.933+0000 7fcaeda00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcae806af50 0x7fcae81a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.933+0000 7fcaeda00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae810e960 0x7fcae819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.933+0000 7fcaeda00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcae81143c0 con 0x7fcae806a630 2024-04-26T01:53:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.933+0000 7fcaeda00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcae8114240 con 0x7fcae810e960 2024-04-26T01:53:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.933+0000 7fcaeda00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcae8114540 con 0x7fcae806af50 2024-04-26T01:53:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.934+0000 7fcae6a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcae806a630 0x7fcae81a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.934+0000 7fcade000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcae806af50 0x7fcae81a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.934+0000 7fcae6a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcae806a630 0x7fcae81a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48268/0 (socket says 172.21.15.1:48268) 2024-04-26T01:53:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.934+0000 7fcade000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcae806af50 0x7fcae81a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:55016/0 (socket says 172.21.15.1:55016) 2024-04-26T01:53:44.932 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.934+0000 7fcae6a00640 1 -- 172.21.15.1:0/2383728464 learned_addr learned my addr 172.21.15.1:0/2383728464 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:44.933 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.934+0000 7fcae7400640 1 --2- 172.21.15.1:0/2383728464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae810e960 0x7fcae819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.934+0000 7fcade000640 1 -- 172.21.15.1:0/2383728464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae810e960 msgr2=0x7fcae819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.934+0000 7fcade000640 1 --2- 172.21.15.1:0/2383728464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae810e960 0x7fcae819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.934+0000 7fcade000640 1 -- 172.21.15.1:0/2383728464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcae806a630 msgr2=0x7fcae81a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.934+0000 7fcade000640 1 --2- 172.21.15.1:0/2383728464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcae806a630 0x7fcae81a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.934+0000 7fcade000640 1 -- 172.21.15.1:0/2383728464 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae819b020 con 0x7fcae806af50 2024-04-26T01:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.934+0000 7fcae7400640 1 --2- 172.21.15.1:0/2383728464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae810e960 0x7fcae819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.935+0000 7fcade000640 1 --2- 172.21.15.1:0/2383728464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcae806af50 0x7fcae81a0be0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcad000e9f0 tx=0x7fcad000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.935+0000 7fcadfe00640 1 -- 172.21.15.1:0/2383728464 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad000ccc0 con 0x7fcae806af50 2024-04-26T01:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.935+0000 7fcaeda00640 1 -- 172.21.15.1:0/2383728464 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae819b310 con 0x7fcae806af50 2024-04-26T01:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.935+0000 7fcadfe00640 1 -- 172.21.15.1:0/2383728464 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcad00044d0 con 0x7fcae806af50 2024-04-26T01:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.935+0000 7fcadfe00640 1 -- 172.21.15.1:0/2383728464 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad001c070 con 0x7fcae806af50 2024-04-26T01:53:44.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.935+0000 7fcaeda00640 1 -- 172.21.15.1:0/2383728464 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae81a7620 con 0x7fcae806af50 2024-04-26T01:53:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.938+0000 7fcaf07f0740 1 Processor -- start 2024-04-26T01:53:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.938+0000 7fcadfe00640 1 -- 172.21.15.1:0/2383728464 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fcad0005020 con 0x7fcae806af50 2024-04-26T01:53:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.939+0000 7fcaf07f0740 1 -- start start 2024-04-26T01:53:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.939+0000 7fcaf07f0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab479820 0x5578ab479c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.939+0000 7fcaf07f0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab4844c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.939+0000 7fcaf07f0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578ab484a00 0x5578ab486df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.939+0000 7fcaf07f0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5578ab3df900 con 0x5578ab47a140 2024-04-26T01:53:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.939+0000 7fcaf07f0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5578ab385ef0 con 0x5578ab479820 2024-04-26T01:53:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.939+0000 7fcaf07f0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5578ab3db8c0 con 0x5578ab484a00 2024-04-26T01:53:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.939+0000 7fcae6a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab4844c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.939+0000 7fcade000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578ab484a00 0x5578ab486df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.939+0000 7fcae6a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab4844c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48276/0 (socket says 172.21.15.1:48276) 2024-04-26T01:53:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.939+0000 7fcae6a00640 1 -- 172.21.15.1:0/970395985 learned_addr learned my addr 172.21.15.1:0/970395985 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.939+0000 7fcae7400640 1 --2- 172.21.15.1:0/970395985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab479820 0x5578ab479c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.940+0000 7fcae7400640 1 -- 172.21.15.1:0/970395985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578ab484a00 msgr2=0x5578ab486df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.940+0000 7fcae7400640 1 --2- 172.21.15.1:0/970395985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578ab484a00 0x5578ab486df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.940+0000 7fcae7400640 1 -- 172.21.15.1:0/970395985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 msgr2=0x5578ab4844c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.940+0000 7fcae7400640 1 --2- 172.21.15.1:0/970395985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab4844c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.940+0000 7fcae7400640 1 -- 172.21.15.1:0/970395985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578ab0f2210 con 0x5578ab479820 2024-04-26T01:53:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.940+0000 7fcae6a00640 1 --2- 172.21.15.1:0/970395985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab4844c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.940+0000 7fcae7400640 1 --2- 172.21.15.1:0/970395985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab479820 0x5578ab479c00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcae0005fd0 tx=0x7fcae0011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=813d9c19e2b27bfb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.940+0000 7fcadfe00640 1 --2- 172.21.15.1:0/2383728464 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fcacc0a3d90 0x7fcacc0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.940+0000 7fcad7e00640 1 -- 172.21.15.1:0/970395985 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0004d20 con 0x5578ab479820 2024-04-26T01:53:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.940+0000 7fcadfe00640 1 -- 172.21.15.1:0/2383728464 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fcad00d07d0 con 0x7fcae806af50 2024-04-26T01:53:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.941+0000 7fcad7e00640 1 -- 172.21.15.1:0/970395985 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcae000f9a0 con 0x5578ab479820 2024-04-26T01:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.941+0000 7fcad7e00640 1 -- 172.21.15.1:0/970395985 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0004450 con 0x5578ab479820 2024-04-26T01:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.941+0000 7fcae6a00640 1 --2- 172.21.15.1:0/2383728464 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fcacc0a3d90 0x7fcacc0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.941+0000 7fcaf07f0740 1 -- 172.21.15.1:0/970395985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab479820 msgr2=0x5578ab479c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.941+0000 7fcaf07f0740 1 --2- 172.21.15.1:0/970395985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab479820 0x5578ab479c00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcae0005fd0 tx=0x7fcae0011bf0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.941+0000 7fcae6a00640 1 --2- 172.21.15.1:0/2383728464 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fcacc0a3d90 0x7fcacc0a6250 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x5578ab47adc0 tx=0x7fcad80058d0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.941+0000 7fcaf07f0740 1 -- 172.21.15.1:0/970395985 shutdown_connections 2024-04-26T01:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.941+0000 7fcaf07f0740 1 --2- 172.21.15.1:0/970395985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578ab484a00 0x5578ab486df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.941+0000 7fcaf07f0740 1 --2- 172.21.15.1:0/970395985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab4844c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.942+0000 7fcaf07f0740 1 --2- 172.21.15.1:0/970395985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab479820 0x5578ab479c00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.942+0000 7fcaf07f0740 1 -- 172.21.15.1:0/970395985 >> 172.21.15.1:0/970395985 conn(0x5578ab475d30 msgr2=0x5578ab476110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.942+0000 7fcaf07f0740 1 -- 172.21.15.1:0/970395985 shutdown_connections 2024-04-26T01:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.942+0000 7fcaf07f0740 1 -- 172.21.15.1:0/970395985 wait complete. 2024-04-26T01:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.942+0000 7fcaf07f0740 1 -- start start 2024-04-26T01:53:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.942+0000 7fcaf07f0740 1 Processor -- start 2024-04-26T01:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.943+0000 7fcaf07f0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab479820 0x5578ab481920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.943+0000 7fcaf07f0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab481e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.943+0000 7fcaf07f0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578ab484a00 0x5578ab4823a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.943+0000 7fcaf07f0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5578ab487630 con 0x5578ab47a140 2024-04-26T01:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.943+0000 7fcaf07f0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5578ab399310 con 0x5578ab479820 2024-04-26T01:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.943+0000 7fcaf07f0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5578ab4877b0 con 0x5578ab484a00 2024-04-26T01:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.943+0000 7fcaf07f0740 10 client.0 _ll_register_callbacks cb 0x5578ab367cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.943+0000 7fcae6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab479820 0x5578ab481920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.943+0000 7fcae6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab479820 0x5578ab481920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:48714/0 (socket says 172.21.15.1:48714) 2024-04-26T01:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.943+0000 7fcae6a00640 1 -- 172.21.15.1:0/219566742 learned_addr learned my addr 172.21.15.1:0/219566742 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.943+0000 7fcade000640 1 --2- 172.21.15.1:0/219566742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab481e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.944+0000 7fcae7400640 1 --2- 172.21.15.1:0/219566742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578ab484a00 0x5578ab4823a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.944+0000 7fcae6a00640 1 -- 172.21.15.1:0/219566742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578ab484a00 msgr2=0x5578ab4823a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:44.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.944+0000 7fcae6a00640 1 --2- 172.21.15.1:0/219566742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578ab484a00 0x5578ab4823a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.944+0000 7fcae6a00640 1 -- 172.21.15.1:0/219566742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 msgr2=0x5578ab481e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.944+0000 7fcae6a00640 1 --2- 172.21.15.1:0/219566742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab481e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.944+0000 7fcae6a00640 1 -- 172.21.15.1:0/219566742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578ab1be620 con 0x5578ab479820 2024-04-26T01:53:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.944+0000 7fcae7400640 1 --2- 172.21.15.1:0/219566742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578ab484a00 0x5578ab4823a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:53:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.944+0000 7fcae6a00640 1 --2- 172.21.15.1:0/219566742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab479820 0x5578ab481920 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcad8031720 tx=0x7fcad8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcfe1e472af62638 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.944+0000 7fcad4c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:53:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.944+0000 7fcad4c00640 1 -- 172.21.15.1:0/219566742 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad804e070 con 0x5578ab479820 2024-04-26T01:53:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.945+0000 7fcaf07f0740 1 -- 172.21.15.1:0/219566742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcae000f9a0 con 0x5578ab479820 2024-04-26T01:53:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.945+0000 7fcad4c00640 1 -- 172.21.15.1:0/219566742 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fcad80037f0 con 0x5578ab479820 2024-04-26T01:53:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.948+0000 7fcad4c00640 1 -- 172.21.15.1:0/219566742 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcad805a050 con 0x5578ab479820 2024-04-26T01:53:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.948+0000 7fcaf07f0740 10 client.25454 fetch_fsmap learned FSMap version 30 2024-04-26T01:53:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.948+0000 7fcaf07f0740 1 -- 172.21.15.1:0/219566742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5578ab49c090 con 0x5578ab479820 2024-04-26T01:53:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.948+0000 7fcad4c00640 1 -- 172.21.15.1:0/219566742 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcae000f9a0 con 0x5578ab479820 2024-04-26T01:53:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.949+0000 7fcad4c00640 1 -- 172.21.15.1:0/219566742 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fcad804aac0 con 0x5578ab479820 2024-04-26T01:53:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.949+0000 7fcaf07f0740 10 client.25454 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:53:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.949+0000 7fcaf07f0740 10 client.25454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:53:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.949+0000 7fcaf07f0740 20 client.25454 populate_metadata read hostname 'smithi001' 2024-04-26T01:53:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.949+0000 7fcaf07f0740 1 --2- 172.21.15.1:0/219566742 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5578ab49d4f0 0x5578ab49f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.949+0000 7fcaf07f0740 4 client.25454 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:53:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.949+0000 7fcaf07f0740 1 -- 172.21.15.1:0/219566742 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578ab3f38b0 con 0x5578ab49d4f0 2024-04-26T01:53:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.949+0000 7fcade000640 1 --2- 172.21.15.1:0/219566742 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5578ab49d4f0 0x5578ab49f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.949+0000 7fcade000640 1 --2- 172.21.15.1:0/219566742 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5578ab49d4f0 0x5578ab49f900 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b7b59295048c2cf server_cookie=9354a094e4a50cbd in_seq=0 out_seq=0 2024-04-26T01:53:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.950+0000 7fcad4c00640 10 client.25454 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.950+0000 7fcad4c00640 1 -- 172.21.15.1:0/219566742 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578ab3f38b0 con 0x5578ab49d4f0 2024-04-26T01:53:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.950+0000 7fcad4c00640 10 client.25454 handle_command_reply: tid=0 2024-04-26T01:53:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.950+0000 7fcaf07f0740 1 client.25454 shutdown 2024-04-26T01:53:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.951+0000 7fcaf07f0740 1 -- 172.21.15.1:0/219566742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab479820 msgr2=0x5578ab481920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.951+0000 7fcaf07f0740 1 --2- 172.21.15.1:0/219566742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab479820 0x5578ab481920 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcad8031720 tx=0x7fcad8009260 comp rx=0 tx=0).stop 2024-04-26T01:53:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.951+0000 7fcaf07f0740 1 -- 172.21.15.1:0/219566742 shutdown_connections 2024-04-26T01:53:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.951+0000 7fcaf07f0740 1 --2- 172.21.15.1:0/219566742 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5578ab49d4f0 0x5578ab49f900 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.951+0000 7fcaf07f0740 1 --2- 172.21.15.1:0/219566742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578ab484a00 0x5578ab4823a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.951+0000 7fcaf07f0740 1 --2- 172.21.15.1:0/219566742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab481e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.951+0000 7fcaf07f0740 1 --2- 172.21.15.1:0/219566742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab479820 0x5578ab481920 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.951+0000 7fcaf07f0740 1 -- 172.21.15.1:0/219566742 >> 172.21.15.1:0/219566742 conn(0x5578ab475d30 msgr2=0x5578ab47bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.951+0000 7fcad4c00640 0 client.25454 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.951+0000 7fcaf07f0740 1 -- 172.21.15.1:0/219566742 shutdown_connections 2024-04-26T01:53:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.951+0000 7fcaf07f0740 1 -- 172.21.15.1:0/219566742 wait complete. 2024-04-26T01:53:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.951+0000 7fcaf07f0740 20 client.25454 trim_cache size 0 max 16384 2024-04-26T01:53:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.967+0000 7fcaf07f0740 1 Processor -- start 2024-04-26T01:53:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.967+0000 7fcaf07f0740 1 -- start start 2024-04-26T01:53:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.967+0000 7fcaf07f0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578ab479820 0x5578ab498d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.967+0000 7fcaf07f0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab474d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.967+0000 7fcaf07f0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab484a00 0x5578ab4a5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.967+0000 7fcaf07f0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcae81ae320 con 0x5578ab47a140 2024-04-26T01:53:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.967+0000 7fcaf07f0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcae81abb90 con 0x5578ab484a00 2024-04-26T01:53:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.968+0000 7fcaf07f0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcae81a9a00 con 0x5578ab479820 2024-04-26T01:53:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.968+0000 7fcade000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab474d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.968+0000 7fcade000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab474d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48308/0 (socket says 172.21.15.1:48308) 2024-04-26T01:53:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.968+0000 7fcade000640 1 -- 172.21.15.1:0/689370371 learned_addr learned my addr 172.21.15.1:0/689370371 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.968+0000 7fcae7400640 1 --2- 172.21.15.1:0/689370371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab484a00 0x5578ab4a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.968+0000 7fcae6a00640 1 --2- 172.21.15.1:0/689370371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578ab479820 0x5578ab498d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.968+0000 7fcade000640 1 -- 172.21.15.1:0/689370371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578ab479820 msgr2=0x5578ab498d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.968+0000 7fcade000640 1 --2- 172.21.15.1:0/689370371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578ab479820 0x5578ab498d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.968+0000 7fcade000640 1 -- 172.21.15.1:0/689370371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab484a00 msgr2=0x5578ab4a5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.968+0000 7fcade000640 1 --2- 172.21.15.1:0/689370371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab484a00 0x5578ab4a5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.968+0000 7fcade000640 1 -- 172.21.15.1:0/689370371 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578ab47fe90 con 0x5578ab47a140 2024-04-26T01:53:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.969+0000 7fcade000640 1 --2- 172.21.15.1:0/689370371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab474d10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcad00990e0 tx=0x7fcad00d2c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c510524050f43e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.969+0000 7fcad7400640 1 -- 172.21.15.1:0/689370371 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad009c070 con 0x5578ab47a140 2024-04-26T01:53:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.969+0000 7fcad7400640 1 -- 172.21.15.1:0/689370371 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcad009ce10 con 0x5578ab47a140 2024-04-26T01:53:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.969+0000 7fcad7400640 1 -- 172.21.15.1:0/689370371 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad0098d80 con 0x5578ab47a140 2024-04-26T01:53:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.969+0000 7fcaf07f0740 1 -- 172.21.15.1:0/689370371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 msgr2=0x5578ab474d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.969+0000 7fcaf07f0740 1 --2- 172.21.15.1:0/689370371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab474d10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcad00990e0 tx=0x7fcad00d2c20 comp rx=0 tx=0).stop 2024-04-26T01:53:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.970+0000 7fcaf07f0740 1 -- 172.21.15.1:0/689370371 shutdown_connections 2024-04-26T01:53:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.970+0000 7fcaf07f0740 1 --2- 172.21.15.1:0/689370371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab484a00 0x5578ab4a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.970+0000 7fcaf07f0740 1 --2- 172.21.15.1:0/689370371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab474d10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.970+0000 7fcaf07f0740 1 --2- 172.21.15.1:0/689370371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578ab479820 0x5578ab498d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.970+0000 7fcaf07f0740 1 -- 172.21.15.1:0/689370371 >> 172.21.15.1:0/689370371 conn(0x5578ab475d30 msgr2=0x5578ab474600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.970+0000 7fcaf07f0740 1 -- 172.21.15.1:0/689370371 shutdown_connections 2024-04-26T01:53:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.970+0000 7fcaf07f0740 1 -- 172.21.15.1:0/689370371 wait complete. 2024-04-26T01:53:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.970+0000 7fcaf07f0740 1 -- start start 2024-04-26T01:53:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.970+0000 7fcaf07f0740 1 Processor -- start 2024-04-26T01:53:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.971+0000 7fcaf07f0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578ab479820 0x5578ab495720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.971+0000 7fcaf07f0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab492ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.971+0000 7fcaf07f0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab484a00 0x5578ab4930c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.971+0000 7fcaf07f0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcae81aa050 con 0x5578ab47a140 2024-04-26T01:53:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.971+0000 7fcaf07f0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcae81ab4c0 con 0x5578ab484a00 2024-04-26T01:53:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.971+0000 7fcaf07f0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcae81aa720 con 0x5578ab479820 2024-04-26T01:53:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.971+0000 7fcaf07f0740 10 client.0 _ll_register_callbacks cb 0x7fcad8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:53:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.971+0000 7fcade000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab492ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.971+0000 7fcade000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab492ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48310/0 (socket says 172.21.15.1:48310) 2024-04-26T01:53:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.971+0000 7fcade000640 1 -- 172.21.15.1:0/1578639648 learned_addr learned my addr 172.21.15.1:0/1578639648 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.971+0000 7fcae6a00640 1 --2- 172.21.15.1:0/1578639648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578ab479820 0x5578ab495720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.971+0000 7fcae7400640 1 --2- 172.21.15.1:0/1578639648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab484a00 0x5578ab4930c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.972+0000 7fcade000640 1 -- 172.21.15.1:0/1578639648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578ab479820 msgr2=0x5578ab495720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.972+0000 7fcade000640 1 --2- 172.21.15.1:0/1578639648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578ab479820 0x5578ab495720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.972+0000 7fcade000640 1 -- 172.21.15.1:0/1578639648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab484a00 msgr2=0x5578ab4930c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.972+0000 7fcade000640 1 --2- 172.21.15.1:0/1578639648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab484a00 0x5578ab4930c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.972+0000 7fcade000640 1 -- 172.21.15.1:0/1578639648 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578ab4938c0 con 0x5578ab47a140 2024-04-26T01:53:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.972+0000 7fcade000640 1 --2- 172.21.15.1:0/1578639648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab492ab0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcad00a4e50 tx=0x7fcad0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed55e6970fd7b23b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.972+0000 7fcad4c00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:53:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.972+0000 7fcad4c00640 1 -- 172.21.15.1:0/1578639648 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad009c070 con 0x5578ab47a140 2024-04-26T01:53:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.972+0000 7fcad4c00640 1 -- 172.21.15.1:0/1578639648 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fcad00ce140 con 0x5578ab47a140 2024-04-26T01:53:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.972+0000 7fcaf07f0740 1 -- 172.21.15.1:0/1578639648 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcad009ce10 con 0x5578ab47a140 2024-04-26T01:53:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.972+0000 7fcad4c00640 1 -- 172.21.15.1:0/1578639648 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcad00a0ac0 con 0x5578ab47a140 2024-04-26T01:53:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.973+0000 7fcaf07f0740 10 client.15333 fetch_fsmap learned FSMap version 30 2024-04-26T01:53:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.973+0000 7fcaf07f0740 1 -- 172.21.15.1:0/1578639648 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5578ab503ba0 con 0x5578ab47a140 2024-04-26T01:53:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.973+0000 7fcad4c00640 1 -- 172.21.15.1:0/1578639648 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcad009ce10 con 0x5578ab47a140 2024-04-26T01:53:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.973+0000 7fcad4c00640 1 -- 172.21.15.1:0/1578639648 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fcad00a4ef0 con 0x5578ab47a140 2024-04-26T01:53:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.973+0000 7fcaf07f0740 10 client.15333 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:53:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.973+0000 7fcaf07f0740 10 client.15333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:53:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.973+0000 7fcaf07f0740 20 client.15333 populate_metadata read hostname 'smithi001' 2024-04-26T01:53:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.973+0000 7fcaf07f0740 1 --2- 172.21.15.1:0/1578639648 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5578ab504350 0x5578ab506770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.973+0000 7fcaf07f0740 4 client.15333 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:53:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.973+0000 7fcaf07f0740 1 -- 172.21.15.1:0/1578639648 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5578ab3ed320 con 0x5578ab504350 2024-04-26T01:53:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.974+0000 7fcae6a00640 1 --2- 172.21.15.1:0/1578639648 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5578ab504350 0x5578ab506770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.974+0000 7fcae6a00640 1 --2- 172.21.15.1:0/1578639648 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5578ab504350 0x5578ab506770 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd1e16cc10611263 server_cookie=e8784c58c1db1bac in_seq=0 out_seq=0 2024-04-26T01:53:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:44.974+0000 7fcad4c00640 10 client.15333 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.465+0000 7fcad4c00640 1 -- 172.21.15.1:0/1578639648 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5578ab3ed320 con 0x5578ab504350 2024-04-26T01:53:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.465+0000 7fcad4c00640 10 client.15333 handle_command_reply: tid=0 2024-04-26T01:53:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.465+0000 7fcaf07f0740 1 client.15333 shutdown 2024-04-26T01:53:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.465+0000 7fcaf07f0740 1 -- 172.21.15.1:0/1578639648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 msgr2=0x5578ab492ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.465+0000 7fcaf07f0740 1 --2- 172.21.15.1:0/1578639648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab492ab0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcad00a4e50 tx=0x7fcad0005c50 comp rx=0 tx=0).stop 2024-04-26T01:53:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.465+0000 7fcaf07f0740 1 -- 172.21.15.1:0/1578639648 shutdown_connections 2024-04-26T01:53:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.465+0000 7fcaf07f0740 1 --2- 172.21.15.1:0/1578639648 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5578ab504350 0x5578ab506770 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.465+0000 7fcaf07f0740 1 --2- 172.21.15.1:0/1578639648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5578ab484a00 0x5578ab4930c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.465+0000 7fcad4c00640 0 client.15333 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.465+0000 7fcaf07f0740 1 --2- 172.21.15.1:0/1578639648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578ab47a140 0x5578ab492ab0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.465+0000 7fcaf07f0740 1 --2- 172.21.15.1:0/1578639648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5578ab479820 0x5578ab495720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.466+0000 7fcaf07f0740 1 -- 172.21.15.1:0/1578639648 >> 172.21.15.1:0/1578639648 conn(0x5578ab475d30 msgr2=0x5578ab474e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.466+0000 7fcaf07f0740 1 -- 172.21.15.1:0/1578639648 shutdown_connections 2024-04-26T01:53:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.466+0000 7fcaf07f0740 1 -- 172.21.15.1:0/1578639648 wait complete. 2024-04-26T01:53:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.466+0000 7fcaf07f0740 20 client.15333 trim_cache size 0 max 16384 2024-04-26T01:53:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:53:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:53:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:53:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:53:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.467+0000 7fcaeda00640 1 -- 172.21.15.1:0/2383728464 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fcacc0a3d90 msgr2=0x7fcacc0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.467+0000 7fcaeda00640 1 --2- 172.21.15.1:0/2383728464 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fcacc0a3d90 0x7fcacc0a6250 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x5578ab47adc0 tx=0x7fcad80058d0 comp rx=0 tx=0).stop 2024-04-26T01:53:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.467+0000 7fcaeda00640 1 -- 172.21.15.1:0/2383728464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcae806af50 msgr2=0x7fcae81a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.467+0000 7fcaeda00640 1 --2- 172.21.15.1:0/2383728464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcae806af50 0x7fcae81a0be0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcad000e9f0 tx=0x7fcad000eeb0 comp rx=0 tx=0).stop 2024-04-26T01:53:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.467+0000 7fcaeda00640 1 -- 172.21.15.1:0/2383728464 shutdown_connections 2024-04-26T01:53:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.467+0000 7fcaeda00640 1 --2- 172.21.15.1:0/2383728464 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fcacc0a3d90 0x7fcacc0a6250 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.467+0000 7fcaeda00640 1 --2- 172.21.15.1:0/2383728464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae810e960 0x7fcae819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.467+0000 7fcaeda00640 1 --2- 172.21.15.1:0/2383728464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcae806af50 0x7fcae81a0be0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.467+0000 7fcaeda00640 1 --2- 172.21.15.1:0/2383728464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcae806a630 0x7fcae81a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.467+0000 7fcaeda00640 1 -- 172.21.15.1:0/2383728464 >> 172.21.15.1:0/2383728464 conn(0x7fcae8077f50 msgr2=0x7fcae810ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.467+0000 7fcaeda00640 1 -- 172.21.15.1:0/2383728464 shutdown_connections 2024-04-26T01:53:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:45.467+0000 7fcaeda00640 1 -- 172.21.15.1:0/2383728464 wait complete. 2024-04-26T01:53:46.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:45 smithi097 ceph-mon[32853]: pgmap v67: 97 pgs: 97 active+clean; 24 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 1.1 MiB/s wr, 442 op/s 2024-04-26T01:53:46.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:45 smithi173 ceph-mon[32553]: pgmap v67: 97 pgs: 97 active+clean; 24 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 1.1 MiB/s wr, 442 op/s 2024-04-26T01:53:46.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:45 smithi001 ceph-mon[27822]: pgmap v67: 97 pgs: 97 active+clean; 24 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 1.1 MiB/s wr, 442 op/s 2024-04-26T01:53:47.346 INFO:tasks.workunit.client.0.smithi001.stdout:autoheader 2024-04-26T01:53:47.903 INFO:tasks.workunit.client.0.smithi001.stdout:autoconf 2024-04-26T01:53:48.208 INFO:tasks.workunit.client.0.smithi001.stdout:./configure \ 2024-04-26T01:53:48.208 INFO:tasks.workunit.client.0.smithi001.stdout: --libexecdir=/usr/lib \ 2024-04-26T01:53:48.208 INFO:tasks.workunit.client.0.smithi001.stdout: --exec_prefix=/var/lib 2024-04-26T01:53:48.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:47 smithi097 ceph-mon[32853]: pgmap v68: 97 pgs: 97 active+clean; 26 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.4 MiB/s wr, 354 op/s 2024-04-26T01:53:48.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:47 smithi173 ceph-mon[32553]: pgmap v68: 97 pgs: 97 active+clean; 26 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.4 MiB/s wr, 354 op/s 2024-04-26T01:53:48.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:47 smithi001 ceph-mon[27822]: pgmap v68: 97 pgs: 97 active+clean; 26 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 1.4 MiB/s wr, 354 op/s 2024-04-26T01:53:48.611 INFO:tasks.workunit.client.0.smithi001.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-26T01:53:48.611 INFO:tasks.workunit.client.0.smithi001.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-26T01:53:48.615 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to print strings... printf 2024-04-26T01:53:48.615 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gcc... gcc 2024-04-26T01:53:48.802 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether the C compiler works... yes 2024-04-26T01:53:48.802 INFO:tasks.workunit.client.0.smithi001.stdout:checking for C compiler default output file name... a.out 2024-04-26T01:53:48.889 INFO:tasks.workunit.client.0.smithi001.stdout:checking for suffix of executables... 2024-04-26T01:53:48.985 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether we are cross compiling... no 2024-04-26T01:53:49.064 INFO:tasks.workunit.client.0.smithi001.stdout:checking for suffix of object files... o 2024-04-26T01:53:49.129 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether we are using the GNU C compiler... yes 2024-04-26T01:53:49.193 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether gcc accepts -g... yes 2024-04-26T01:53:49.312 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-26T01:53:49.324 INFO:tasks.workunit.client.0.smithi001.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-04-26T01:53:49.333 INFO:tasks.workunit.client.0.smithi001.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-26T01:53:49.337 INFO:tasks.workunit.client.0.smithi001.stdout:checking for egrep... /usr/bin/grep -E 2024-04-26T01:53:49.340 INFO:tasks.workunit.client.0.smithi001.stdout:checking for fgrep... /usr/bin/grep -F 2024-04-26T01:53:49.350 INFO:tasks.workunit.client.0.smithi001.stdout:checking for ld used by gcc... /usr/bin/ld 2024-04-26T01:53:49.355 INFO:tasks.workunit.client.0.smithi001.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-04-26T01:53:49.410 INFO:tasks.workunit.client.0.smithi001.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-04-26T01:53:50.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:49 smithi097 ceph-mon[32853]: pgmap v69: 97 pgs: 97 active+clean; 26 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 1013 KiB/s wr, 133 op/s 2024-04-26T01:53:50.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:49 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:53:50.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:49 smithi173 ceph-mon[32553]: pgmap v69: 97 pgs: 97 active+clean; 26 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 1013 KiB/s wr, 133 op/s 2024-04-26T01:53:50.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:49 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:53:50.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:49 smithi001 ceph-mon[27822]: pgmap v69: 97 pgs: 97 active+clean; 26 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 1013 KiB/s wr, 133 op/s 2024-04-26T01:53:50.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:49 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:53:50.484 INFO:tasks.workunit.client.0.smithi001.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-04-26T01:53:50.484 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether ln -s works... yes 2024-04-26T01:53:50.504 INFO:tasks.workunit.client.0.smithi001.stdout:checking the maximum length of command line arguments... 1572864 2024-04-26T01:53:50.512 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-26T01:53:50.513 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-26T01:53:50.513 INFO:tasks.workunit.client.0.smithi001.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-04-26T01:53:50.513 INFO:tasks.workunit.client.0.smithi001.stdout:checking for objdump... objdump 2024-04-26T01:53:50.513 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to recognize dependent libraries... pass_all 2024-04-26T01:53:50.514 INFO:tasks.workunit.client.0.smithi001.stdout:checking for dlltool... no 2024-04-26T01:53:50.515 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-26T01:53:50.515 INFO:tasks.workunit.client.0.smithi001.stdout:checking for ar... ar 2024-04-26T01:53:50.675 INFO:tasks.workunit.client.0.smithi001.stdout:checking for archiver @FILE support... @ 2024-04-26T01:53:50.675 INFO:tasks.workunit.client.0.smithi001.stdout:checking for strip... strip 2024-04-26T01:53:50.676 INFO:tasks.workunit.client.0.smithi001.stdout:checking for ranlib... ranlib 2024-04-26T01:53:50.676 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gawk... gawk 2024-04-26T01:53:50.905 INFO:tasks.workunit.client.0.smithi001.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-04-26T01:53:50.909 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sysroot... no 2024-04-26T01:53:50.933 INFO:tasks.workunit.client.0.smithi001.stdout:checking for a working dd... /usr/bin/dd 2024-04-26T01:53:50.955 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-04-26T01:53:51.027 INFO:tasks.workunit.client.0.smithi001.stdout:checking for mt... no 2024-04-26T01:53:51.054 INFO:tasks.workunit.client.0.smithi001.stdout:checking if : is a manifest tool... no 2024-04-26T01:53:51.204 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to run the C preprocessor... gcc -E 2024-04-26T01:53:51.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:50 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:51.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:50 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:53:51.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:50 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:53:51.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:50 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:51.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:50 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:53:51.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:50 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:53:51.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:50 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:51.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:50 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:53:51.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:50 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:53:51.610 INFO:tasks.workunit.client.0.smithi001.stdout:checking for ANSI C header files... yes 2024-04-26T01:53:51.695 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/types.h... yes 2024-04-26T01:53:51.793 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/stat.h... yes 2024-04-26T01:53:51.889 INFO:tasks.workunit.client.0.smithi001.stdout:checking for stdlib.h... yes 2024-04-26T01:53:51.987 INFO:tasks.workunit.client.0.smithi001.stdout:checking for string.h... yes 2024-04-26T01:53:52.083 INFO:tasks.workunit.client.0.smithi001.stdout:checking for memory.h... yes 2024-04-26T01:53:52.178 INFO:tasks.workunit.client.0.smithi001.stdout:checking for strings.h... yes 2024-04-26T01:53:52.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:51 smithi097 ceph-mon[32853]: pgmap v70: 97 pgs: 97 active+clean; 31 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 1.9 MiB/s wr, 788 op/s 2024-04-26T01:53:52.274 INFO:tasks.workunit.client.0.smithi001.stdout:checking for inttypes.h... yes 2024-04-26T01:53:52.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:51 smithi173 ceph-mon[32553]: pgmap v70: 97 pgs: 97 active+clean; 31 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 1.9 MiB/s wr, 788 op/s 2024-04-26T01:53:52.373 INFO:tasks.workunit.client.0.smithi001.stdout:checking for stdint.h... yes 2024-04-26T01:53:52.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:51 smithi001 ceph-mon[27822]: pgmap v70: 97 pgs: 97 active+clean; 31 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 1.9 MiB/s wr, 788 op/s 2024-04-26T01:53:52.478 INFO:tasks.workunit.client.0.smithi001.stdout:checking for unistd.h... yes 2024-04-26T01:53:52.587 INFO:tasks.workunit.client.0.smithi001.stdout:checking for dlfcn.h... yes 2024-04-26T01:53:52.603 INFO:tasks.workunit.client.0.smithi001.stdout:checking for objdir... .libs 2024-04-26T01:53:52.895 INFO:tasks.workunit.client.0.smithi001.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-26T01:53:52.895 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-26T01:53:52.982 INFO:tasks.workunit.client.0.smithi001.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-26T01:53:53.073 INFO:tasks.workunit.client.0.smithi001.stdout:checking if gcc static flag -static works... no 2024-04-26T01:53:53.218 INFO:tasks.workunit.client.0.smithi001.stdout:checking if gcc supports -c -o file.o... yes 2024-04-26T01:53:53.218 INFO:tasks.workunit.client.0.smithi001.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-26T01:53:53.245 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-26T01:53:53.343 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether -lc should be explicitly linked in... no 2024-04-26T01:53:53.556 INFO:tasks.workunit.client.0.smithi001.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-26T01:53:53.556 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to hardcode library paths into programs... immediate 2024-04-26T01:53:53.561 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether stripping libraries is possible... yes 2024-04-26T01:53:53.561 INFO:tasks.workunit.client.0.smithi001.stdout:checking if libtool supports shared libraries... yes 2024-04-26T01:53:53.562 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether to build shared libraries... yes 2024-04-26T01:53:53.562 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether to build static libraries... yes 2024-04-26T01:53:53.584 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gcc... (cached) gcc 2024-04-26T01:53:53.665 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-26T01:53:53.665 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-26T01:53:53.665 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-26T01:53:53.666 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gmake... /usr/bin/gmake 2024-04-26T01:53:53.666 INFO:tasks.workunit.client.0.smithi001.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-04-26T01:53:53.667 INFO:tasks.workunit.client.0.smithi001.stdout:checking for tar... /bin/tar 2024-04-26T01:53:53.667 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gzip... /bin/gzip 2024-04-26T01:53:53.707 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether gcc -MM is supported... yes 2024-04-26T01:53:53.708 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sort... /bin/sort 2024-04-26T01:53:53.708 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether ln -s works... yes 2024-04-26T01:53:53.708 INFO:tasks.workunit.client.0.smithi001.stdout:checking for rpm... /bin/rpm 2024-04-26T01:53:53.720 INFO:tasks.workunit.client.0.smithi001.stdout:checking for rpmbuild... no 2024-04-26T01:53:53.831 INFO:tasks.workunit.client.0.smithi001.stdout:checking assert.h usability... yes 2024-04-26T01:53:53.870 INFO:tasks.workunit.client.0.smithi001.stdout:checking assert.h presence... yes 2024-04-26T01:53:53.870 INFO:tasks.workunit.client.0.smithi001.stdout:checking for assert.h... yes 2024-04-26T01:53:54.000 INFO:tasks.workunit.client.0.smithi001.stdout:checking bstring.h usability... no 2024-04-26T01:53:54.067 INFO:tasks.workunit.client.0.smithi001.stdout:checking bstring.h presence... no 2024-04-26T01:53:54.067 INFO:tasks.workunit.client.0.smithi001.stdout:checking for bstring.h... no 2024-04-26T01:53:54.165 INFO:tasks.workunit.client.0.smithi001.stdout:checking libgen.h usability... yes 2024-04-26T01:53:54.206 INFO:tasks.workunit.client.0.smithi001.stdout:checking libgen.h presence... yes 2024-04-26T01:53:54.206 INFO:tasks.workunit.client.0.smithi001.stdout:checking for libgen.h... yes 2024-04-26T01:53:54.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:53 smithi097 ceph-mon[32853]: pgmap v71: 97 pgs: 97 active+clean; 31 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 1.8 MiB/s wr, 783 op/s 2024-04-26T01:53:54.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:53 smithi173 ceph-mon[32553]: pgmap v71: 97 pgs: 97 active+clean; 31 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 1.8 MiB/s wr, 783 op/s 2024-04-26T01:53:54.319 INFO:tasks.workunit.client.0.smithi001.stdout:checking dirent.h usability... yes 2024-04-26T01:53:54.360 INFO:tasks.workunit.client.0.smithi001.stdout:checking dirent.h presence... yes 2024-04-26T01:53:54.361 INFO:tasks.workunit.client.0.smithi001.stdout:checking for dirent.h... yes 2024-04-26T01:53:54.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:53 smithi001 ceph-mon[27822]: pgmap v71: 97 pgs: 97 active+clean; 31 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 1.8 MiB/s wr, 783 op/s 2024-04-26T01:53:54.474 INFO:tasks.workunit.client.0.smithi001.stdout:checking errno.h usability... yes 2024-04-26T01:53:54.513 INFO:tasks.workunit.client.0.smithi001.stdout:checking errno.h presence... yes 2024-04-26T01:53:54.514 INFO:tasks.workunit.client.0.smithi001.stdout:checking for errno.h... yes 2024-04-26T01:53:54.626 INFO:tasks.workunit.client.0.smithi001.stdout:checking malloc.h usability... yes 2024-04-26T01:53:54.671 INFO:tasks.workunit.client.0.smithi001.stdout:checking malloc.h presence... yes 2024-04-26T01:53:54.672 INFO:tasks.workunit.client.0.smithi001.stdout:checking for malloc.h... yes 2024-04-26T01:53:54.787 INFO:tasks.workunit.client.0.smithi001.stdout:checking uuid.h usability... no 2024-04-26T01:53:54.851 INFO:tasks.workunit.client.0.smithi001.stdout:checking uuid.h presence... no 2024-04-26T01:53:54.851 INFO:tasks.workunit.client.0.smithi001.stdout:checking for uuid.h... no 2024-04-26T01:53:54.962 INFO:tasks.workunit.client.0.smithi001.stdout:checking uuid/uuid.h usability... yes 2024-04-26T01:53:55.007 INFO:tasks.workunit.client.0.smithi001.stdout:checking uuid/uuid.h presence... yes 2024-04-26T01:53:55.007 INFO:tasks.workunit.client.0.smithi001.stdout:checking for uuid/uuid.h... yes 2024-04-26T01:53:55.126 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/uuid.h usability... no 2024-04-26T01:53:55.188 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/uuid.h presence... no 2024-04-26T01:53:55.188 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/uuid.h... no 2024-04-26T01:53:55.291 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/file.h usability... yes 2024-04-26T01:53:55.337 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/file.h presence... yes 2024-04-26T01:53:55.338 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/file.h... yes 2024-04-26T01:53:55.462 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/fcntl.h usability... yes 2024-04-26T01:53:55.504 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/fcntl.h presence... yes 2024-04-26T01:53:55.504 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/fcntl.h... yes 2024-04-26T01:53:55.622 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/syssgi.h usability... no 2024-04-26T01:53:55.689 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/syssgi.h presence... no 2024-04-26T01:53:55.689 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/syssgi.h... no 2024-04-26T01:53:55.825 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/param.h usability... yes 2024-04-26T01:53:55.875 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/param.h presence... yes 2024-04-26T01:53:55.875 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/param.h... yes 2024-04-26T01:53:55.912 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/stat.h... (cached) yes 2024-04-26T01:53:56.021 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/statvfs.h usability... yes 2024-04-26T01:53:56.063 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/statvfs.h presence... yes 2024-04-26T01:53:56.063 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/statvfs.h... yes 2024-04-26T01:53:56.176 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/time.h usability... yes 2024-04-26T01:53:56.217 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/time.h presence... yes 2024-04-26T01:53:56.218 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/time.h... yes 2024-04-26T01:53:56.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:55 smithi097 ceph-mon[32853]: pgmap v72: 97 pgs: 97 active+clean; 31 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 1.8 MiB/s wr, 784 op/s 2024-04-26T01:53:56.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:55 smithi173 ceph-mon[32553]: pgmap v72: 97 pgs: 97 active+clean; 31 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 1.8 MiB/s wr, 784 op/s 2024-04-26T01:53:56.334 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/ioctl.h usability... yes 2024-04-26T01:53:56.375 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/ioctl.h presence... yes 2024-04-26T01:53:56.375 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/ioctl.h... yes 2024-04-26T01:53:56.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:55 smithi001 ceph-mon[27822]: pgmap v72: 97 pgs: 97 active+clean; 31 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 1.8 MiB/s wr, 784 op/s 2024-04-26T01:53:56.502 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/wait.h usability... yes 2024-04-26T01:53:56.547 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/wait.h presence... yes 2024-04-26T01:53:56.547 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/wait.h... yes 2024-04-26T01:53:56.585 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/types.h... (cached) yes 2024-04-26T01:53:56.605 INFO:tasks.workunit.client.0.smithi001.stdout:checking for strings.h... (cached) yes 2024-04-26T01:53:56.701 INFO:tasks.workunit.client.0.smithi001.stdout:checking err.h usability... yes 2024-04-26T01:53:56.743 INFO:tasks.workunit.client.0.smithi001.stdout:checking err.h presence... yes 2024-04-26T01:53:56.743 INFO:tasks.workunit.client.0.smithi001.stdout:checking for err.h... yes 2024-04-26T01:53:56.860 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/falloc.h usability... yes 2024-04-26T01:53:56.898 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/falloc.h presence... yes 2024-04-26T01:53:56.898 INFO:tasks.workunit.client.0.smithi001.stdout:checking for linux/falloc.h... yes 2024-04-26T01:53:57.021 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-26T01:53:57.085 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-26T01:53:57.085 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-26T01:53:57.180 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-26T01:53:57.241 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-26T01:53:57.241 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-26T01:53:57.346 INFO:tasks.workunit.client.0.smithi001.stdout:checking xfs/platform_defs.h usability... no 2024-04-26T01:53:57.413 INFO:tasks.workunit.client.0.smithi001.stdout:checking xfs/platform_defs.h presence... no 2024-04-26T01:53:57.413 INFO:tasks.workunit.client.0.smithi001.stdout:checking for xfs/platform_defs.h... no 2024-04-26T01:53:57.513 INFO:tasks.workunit.client.0.smithi001.stdout:checking btrfs/ioctl.h usability... no 2024-04-26T01:53:57.579 INFO:tasks.workunit.client.0.smithi001.stdout:checking btrfs/ioctl.h presence... no 2024-04-26T01:53:57.579 INFO:tasks.workunit.client.0.smithi001.stdout:checking for btrfs/ioctl.h... no 2024-04-26T01:53:57.721 INFO:tasks.workunit.client.0.smithi001.stdout:checking cifs/ioctl.h usability... no 2024-04-26T01:53:57.784 INFO:tasks.workunit.client.0.smithi001.stdout:checking cifs/ioctl.h presence... no 2024-04-26T01:53:57.784 INFO:tasks.workunit.client.0.smithi001.stdout:checking for cifs/ioctl.h... no 2024-04-26T01:53:57.890 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/mman.h usability... yes 2024-04-26T01:53:57.931 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/mman.h presence... yes 2024-04-26T01:53:57.931 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/mman.h... yes 2024-04-26T01:53:58.048 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/ext4.h usability... no 2024-04-26T01:53:58.117 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/ext4.h presence... no 2024-04-26T01:53:58.117 INFO:tasks.workunit.client.0.smithi001.stdout:checking for linux/ext4.h... no 2024-04-26T01:53:58.190 INFO:tasks.workunit.client.0.smithi001.stdout:checking for xfs/xfs_log_format.h... no 2024-04-26T01:53:58.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:57 smithi097 ceph-mon[32853]: pgmap v73: 97 pgs: 97 active+clean; 32 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.3 MiB/s wr, 708 op/s 2024-04-26T01:53:58.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:57 smithi173 ceph-mon[32553]: pgmap v73: 97 pgs: 97 active+clean; 32 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.3 MiB/s wr, 708 op/s 2024-04-26T01:53:58.318 INFO:tasks.workunit.client.0.smithi001.stdout:checking for uuid_compare... no 2024-04-26T01:53:58.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:57 smithi001 ceph-mon[27822]: pgmap v73: 97 pgs: 97 active+clean; 32 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.3 MiB/s wr, 708 op/s 2024-04-26T01:53:58.413 INFO:tasks.workunit.client.0.smithi001.stdout:checking for uuid_compare in -luuid... yes 2024-04-26T01:53:58.528 INFO:tasks.workunit.client.0.smithi001.stdout:checking for xfs/xfs.h... yes 2024-04-26T01:53:58.600 INFO:tasks.workunit.client.0.smithi001.stdout:checking for xfs/libxfs.h... no 2024-04-26T01:53:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.616+0000 7f7d38000640 1 -- 172.21.15.1:0/1475895832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d301021b0 msgr2=0x7f7d30102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.616+0000 7f7d38000640 1 --2- 172.21.15.1:0/1475895832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d301021b0 0x7f7d30102590 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009900 tx=0x7f7d2402a410 comp rx=0 tx=0).stop 2024-04-26T01:53:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.617+0000 7f7d38000640 1 -- 172.21.15.1:0/1475895832 shutdown_connections 2024-04-26T01:53:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.617+0000 7f7d38000640 1 --2- 172.21.15.1:0/1475895832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d30107280 0x7f7d3010ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.617+0000 7f7d38000640 1 --2- 172.21.15.1:0/1475895832 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d30102b60 0x7f7d30106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.617+0000 7f7d38000640 1 --2- 172.21.15.1:0/1475895832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d301021b0 0x7f7d30102590 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.617+0000 7f7d38000640 1 -- 172.21.15.1:0/1475895832 >> 172.21.15.1:0/1475895832 conn(0x7f7d300fda10 msgr2=0x7f7d300ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.617+0000 7f7d38000640 1 -- 172.21.15.1:0/1475895832 shutdown_connections 2024-04-26T01:53:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.617+0000 7f7d38000640 1 -- 172.21.15.1:0/1475895832 wait complete. 2024-04-26T01:53:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.618+0000 7f7d38000640 1 Processor -- start 2024-04-26T01:53:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.618+0000 7f7d38000640 1 -- start start 2024-04-26T01:53:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.618+0000 7f7d38000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d301021b0 0x7f7d301a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.618+0000 7f7d38000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d30107280 0x7f7d301a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.618+0000 7f7d38000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d3019a820 0x7f7d3019acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.618+0000 7f7d38000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7d30112240 con 0x7f7d301021b0 2024-04-26T01:53:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.618+0000 7f7d38000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301120c0 con 0x7f7d30107280 2024-04-26T01:53:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.618+0000 7f7d38000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301123c0 con 0x7f7d3019a820 2024-04-26T01:53:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.619+0000 7f7d35600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d301021b0 0x7f7d301a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.619+0000 7f7d35600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d301021b0 0x7f7d301a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52390/0 (socket says 172.21.15.1:52390) 2024-04-26T01:53:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.619+0000 7f7d34c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d30107280 0x7f7d301a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.619+0000 7f7d35600640 1 -- 172.21.15.1:0/806228455 learned_addr learned my addr 172.21.15.1:0/806228455 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.619+0000 7f7d36000640 1 --2- 172.21.15.1:0/806228455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d3019a820 0x7f7d3019acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.619+0000 7f7d35600640 1 -- 172.21.15.1:0/806228455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d3019a820 msgr2=0x7f7d3019acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.619+0000 7f7d35600640 1 --2- 172.21.15.1:0/806228455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d3019a820 0x7f7d3019acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.619+0000 7f7d35600640 1 -- 172.21.15.1:0/806228455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d30107280 msgr2=0x7f7d301a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.619+0000 7f7d35600640 1 --2- 172.21.15.1:0/806228455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d30107280 0x7f7d301a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.619+0000 7f7d35600640 1 -- 172.21.15.1:0/806228455 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3019b3b0 con 0x7f7d301021b0 2024-04-26T01:53:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.619+0000 7f7d34c00640 1 --2- 172.21.15.1:0/806228455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d30107280 0x7f7d301a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.619+0000 7f7d36000640 1 --2- 172.21.15.1:0/806228455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d3019a820 0x7f7d3019acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.620+0000 7f7d35600640 1 --2- 172.21.15.1:0/806228455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d301021b0 0x7f7d301a06b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7d24005ab0 tx=0x7f7d24003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.620+0000 7f7d2e000640 1 -- 172.21.15.1:0/806228455 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2403f070 con 0x7f7d301021b0 2024-04-26T01:53:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.620+0000 7f7d2e000640 1 -- 172.21.15.1:0/806228455 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d24031b60 con 0x7f7d301021b0 2024-04-26T01:53:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.620+0000 7f7d38000640 1 -- 172.21.15.1:0/806228455 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3019b640 con 0x7f7d301021b0 2024-04-26T01:53:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.620+0000 7f7d2e000640 1 -- 172.21.15.1:0/806228455 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24035030 con 0x7f7d301021b0 2024-04-26T01:53:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.620+0000 7f7d38000640 1 -- 172.21.15.1:0/806228455 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d301a77f0 con 0x7f7d301021b0 2024-04-26T01:53:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.625+0000 7f7d3ae15740 1 Processor -- start 2024-04-26T01:53:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.625+0000 7f7d3ae15740 1 -- start start 2024-04-26T01:53:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.625+0000 7f7d2e000640 1 -- 172.21.15.1:0/806228455 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f7d24042070 con 0x7f7d301021b0 2024-04-26T01:53:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.625+0000 7f7d3ae15740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c31820 0x564943c31c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.625+0000 7f7d3ae15740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c32140 0x564943c3c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.625+0000 7f7d3ae15740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564943c3ca00 0x564943c3edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.625+0000 7f7d3ae15740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564943b97900 con 0x564943c32140 2024-04-26T01:53:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.625+0000 7f7d3ae15740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564943b3def0 con 0x564943c31820 2024-04-26T01:53:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.625+0000 7f7d3ae15740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564943b938c0 con 0x564943c3ca00 2024-04-26T01:53:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.626+0000 7f7d35600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c32140 0x564943c3c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.626+0000 7f7d36000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c31820 0x564943c31c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.626+0000 7f7d36000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c31820 0x564943c31c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:56166/0 (socket says 172.21.15.1:56166) 2024-04-26T01:53:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.626+0000 7f7d36000640 1 -- 172.21.15.1:0/112305289 learned_addr learned my addr 172.21.15.1:0/112305289 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.626+0000 7f7d34c00640 1 --2- 172.21.15.1:0/112305289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564943c3ca00 0x564943c3edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.626+0000 7f7d36000640 1 -- 172.21.15.1:0/112305289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564943c3ca00 msgr2=0x564943c3edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.626+0000 7f7d36000640 1 --2- 172.21.15.1:0/112305289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564943c3ca00 0x564943c3edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.626+0000 7f7d36000640 1 -- 172.21.15.1:0/112305289 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c32140 msgr2=0x564943c3c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.626+0000 7f7d36000640 1 --2- 172.21.15.1:0/112305289 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c32140 0x564943c3c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.626+0000 7f7d36000640 1 -- 172.21.15.1:0/112305289 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649438aa210 con 0x564943c31820 2024-04-26T01:53:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.627+0000 7f7d36000640 1 --2- 172.21.15.1:0/112305289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c31820 0x564943c31c00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7d28011790 tx=0x7f7d28011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bff1b9f02e1ad83 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.627+0000 7f7d22600640 1 -- 172.21.15.1:0/112305289 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28003c30 con 0x564943c31820 2024-04-26T01:53:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.627+0000 7f7d22600640 1 -- 172.21.15.1:0/112305289 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d2800fe50 con 0x564943c31820 2024-04-26T01:53:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.627+0000 7f7d22600640 1 -- 172.21.15.1:0/112305289 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d280193e0 con 0x564943c31820 2024-04-26T01:53:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.627+0000 7f7d2e000640 1 --2- 172.21.15.1:0/806228455 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7d180a3d10 0x7f7d180a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.627+0000 7f7d2e000640 1 -- 172.21.15.1:0/806228455 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f7d240f0f20 con 0x7f7d301021b0 2024-04-26T01:53:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.627+0000 7f7d35600640 1 --2- 172.21.15.1:0/806228455 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7d180a3d10 0x7f7d180a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.628+0000 7f7d3ae15740 1 -- 172.21.15.1:0/112305289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c31820 msgr2=0x564943c31c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.628+0000 7f7d3ae15740 1 --2- 172.21.15.1:0/112305289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c31820 0x564943c31c00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7d28011790 tx=0x7f7d28011c50 comp rx=0 tx=0).stop 2024-04-26T01:53:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.628+0000 7f7d35600640 1 --2- 172.21.15.1:0/806228455 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7d180a3d10 0x7f7d180a61d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x564943c32dc0 tx=0x7f7d240f49d0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.628+0000 7f7d3ae15740 1 -- 172.21.15.1:0/112305289 shutdown_connections 2024-04-26T01:53:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.628+0000 7f7d3ae15740 1 --2- 172.21.15.1:0/112305289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564943c3ca00 0x564943c3edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.628+0000 7f7d3ae15740 1 --2- 172.21.15.1:0/112305289 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c32140 0x564943c3c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.628+0000 7f7d3ae15740 1 --2- 172.21.15.1:0/112305289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c31820 0x564943c31c00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.628+0000 7f7d3ae15740 1 -- 172.21.15.1:0/112305289 >> 172.21.15.1:0/112305289 conn(0x564943c2dd30 msgr2=0x564943c2e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.628+0000 7f7d3ae15740 1 -- 172.21.15.1:0/112305289 shutdown_connections 2024-04-26T01:53:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.628+0000 7f7d3ae15740 1 -- 172.21.15.1:0/112305289 wait complete. 2024-04-26T01:53:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.629+0000 7f7d3ae15740 1 -- start start 2024-04-26T01:53:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.629+0000 7f7d3ae15740 1 Processor -- start 2024-04-26T01:53:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.630+0000 7f7d3ae15740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c31820 0x564943c399c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.630+0000 7f7d3ae15740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c32140 0x564943c39f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.630+0000 7f7d3ae15740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564943c3ca00 0x564943c3a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.630+0000 7f7d3ae15740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564943c3f630 con 0x564943c31820 2024-04-26T01:53:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.630+0000 7f7d3ae15740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564943b51310 con 0x564943c32140 2024-04-26T01:53:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.630+0000 7f7d3ae15740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564943c3f7b0 con 0x564943c3ca00 2024-04-26T01:53:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.630+0000 7f7d3ae15740 10 client.0 _ll_register_callbacks cb 0x564943b1fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:53:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.630+0000 7f7d36000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c31820 0x564943c399c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.630+0000 7f7d34c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c32140 0x564943c39f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.630+0000 7f7d36000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c31820 0x564943c399c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52394/0 (socket says 172.21.15.1:52394) 2024-04-26T01:53:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.630+0000 7f7d36000640 1 -- 172.21.15.1:0/3318762879 learned_addr learned my addr 172.21.15.1:0/3318762879 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.631+0000 7f7d36000640 1 --2- 172.21.15.1:0/3318762879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564943c3ca00 0x564943c3a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.631+0000 7f7d34c00640 1 -- 172.21.15.1:0/3318762879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564943c3ca00 msgr2=0x564943c3a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.631+0000 7f7d34c00640 1 --2- 172.21.15.1:0/3318762879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564943c3ca00 0x564943c3a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.631+0000 7f7d34c00640 1 -- 172.21.15.1:0/3318762879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c31820 msgr2=0x564943c399c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.631+0000 7f7d34c00640 1 --2- 172.21.15.1:0/3318762879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c31820 0x564943c399c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.631+0000 7f7d34c00640 1 -- 172.21.15.1:0/3318762879 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564943976620 con 0x564943c32140 2024-04-26T01:53:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.631+0000 7f7d36000640 1 --2- 172.21.15.1:0/3318762879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c31820 0x564943c399c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.631+0000 7f7d34c00640 1 --2- 172.21.15.1:0/3318762879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c32140 0x564943c39f00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7d300fe310 tx=0x7f7d1c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7814e70ee8493ac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.631+0000 7f7d16a00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:53:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.631+0000 7f7d16a00640 1 -- 172.21.15.1:0/3318762879 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c012110 con 0x564943c32140 2024-04-26T01:53:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.631+0000 7f7d3ae15740 1 -- 172.21.15.1:0/3318762879 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d2800fe50 con 0x564943c32140 2024-04-26T01:53:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.631+0000 7f7d16a00640 1 -- 172.21.15.1:0/3318762879 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f7d1c004050 con 0x564943c32140 2024-04-26T01:53:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.632+0000 7f7d16a00640 1 -- 172.21.15.1:0/3318762879 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7d1c0122b0 con 0x564943c32140 2024-04-26T01:53:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.632+0000 7f7d3ae15740 10 client.25472 fetch_fsmap learned FSMap version 30 2024-04-26T01:53:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.632+0000 7f7d3ae15740 1 -- 172.21.15.1:0/3318762879 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564943c4b960 con 0x564943c32140 2024-04-26T01:53:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.632+0000 7f7d16a00640 1 -- 172.21.15.1:0/3318762879 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d2800fe50 con 0x564943c32140 2024-04-26T01:53:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.632+0000 7f7d16a00640 1 -- 172.21.15.1:0/3318762879 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7d1c010430 con 0x564943c32140 2024-04-26T01:53:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.633+0000 7f7d3ae15740 10 client.25472 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:53:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.633+0000 7f7d3ae15740 10 client.25472 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:53:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.633+0000 7f7d3ae15740 20 client.25472 populate_metadata read hostname 'smithi001' 2024-04-26T01:53:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.633+0000 7f7d3ae15740 1 --2- 172.21.15.1:0/3318762879 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564943c4ce20 0x564943c4f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.633+0000 7f7d3ae15740 4 client.25472 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:53:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.633+0000 7f7d3ae15740 1 -- 172.21.15.1:0/3318762879 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564943bab8b0 con 0x564943c4ce20 2024-04-26T01:53:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.633+0000 7f7d36000640 1 --2- 172.21.15.1:0/3318762879 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564943c4ce20 0x564943c4f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.633+0000 7f7d36000640 1 --2- 172.21.15.1:0/3318762879 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564943c4ce20 0x564943c4f280 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef3e8fa6ab9b812b server_cookie=e821b2d7d845b909 in_seq=0 out_seq=0 2024-04-26T01:53:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.633+0000 7f7d16a00640 10 client.25472 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.634+0000 7f7d16a00640 1 -- 172.21.15.1:0/3318762879 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564943bab8b0 con 0x564943c4ce20 2024-04-26T01:53:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.634+0000 7f7d16a00640 10 client.25472 handle_command_reply: tid=0 2024-04-26T01:53:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.634+0000 7f7d3ae15740 1 client.25472 shutdown 2024-04-26T01:53:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.634+0000 7f7d3ae15740 1 -- 172.21.15.1:0/3318762879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c32140 msgr2=0x564943c39f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.634+0000 7f7d3ae15740 1 --2- 172.21.15.1:0/3318762879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c32140 0x564943c39f00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7d300fe310 tx=0x7f7d1c00f790 comp rx=0 tx=0).stop 2024-04-26T01:53:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.635+0000 7f7d3ae15740 1 -- 172.21.15.1:0/3318762879 shutdown_connections 2024-04-26T01:53:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.635+0000 7f7d3ae15740 1 --2- 172.21.15.1:0/3318762879 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564943c4ce20 0x564943c4f280 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.635+0000 7f7d3ae15740 1 --2- 172.21.15.1:0/3318762879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564943c3ca00 0x564943c3a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.635+0000 7f7d3ae15740 1 --2- 172.21.15.1:0/3318762879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c32140 0x564943c39f00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.635+0000 7f7d3ae15740 1 --2- 172.21.15.1:0/3318762879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c31820 0x564943c399c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.635+0000 7f7d3ae15740 1 -- 172.21.15.1:0/3318762879 >> 172.21.15.1:0/3318762879 conn(0x564943c2dd30 msgr2=0x564943c33ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.635+0000 7f7d16a00640 0 client.25472 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.635+0000 7f7d3ae15740 1 -- 172.21.15.1:0/3318762879 shutdown_connections 2024-04-26T01:53:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.635+0000 7f7d3ae15740 1 -- 172.21.15.1:0/3318762879 wait complete. 2024-04-26T01:53:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.635+0000 7f7d3ae15740 20 client.25472 trim_cache size 0 max 16384 2024-04-26T01:53:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.652+0000 7f7d3ae15740 1 Processor -- start 2024-04-26T01:53:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.652+0000 7f7d3ae15740 1 -- start start 2024-04-26T01:53:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.652+0000 7f7d3ae15740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c31820 0x564943c484f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.652+0000 7f7d3ae15740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564943c32140 0x564943c3a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.652+0000 7f7d3ae15740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c3ca00 0x564943c54b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.652+0000 7f7d3ae15740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301b2760 con 0x564943c3ca00 2024-04-26T01:53:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.652+0000 7f7d3ae15740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301affd0 con 0x564943c31820 2024-04-26T01:53:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.652+0000 7f7d3ae15740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301ade20 con 0x564943c32140 2024-04-26T01:53:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.652+0000 7f7d36000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c31820 0x564943c484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.652+0000 7f7d36000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c31820 0x564943c484f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:56184/0 (socket says 172.21.15.1:56184) 2024-04-26T01:53:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.652+0000 7f7d36000640 1 -- 172.21.15.1:0/2462329317 learned_addr learned my addr 172.21.15.1:0/2462329317 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.652+0000 7f7d34c00640 1 --2- 172.21.15.1:0/2462329317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564943c32140 0x564943c3a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.652+0000 7f7d36000640 1 --2- 172.21.15.1:0/2462329317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c3ca00 0x564943c54b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.653+0000 7f7d36000640 1 -- 172.21.15.1:0/2462329317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564943c32140 msgr2=0x564943c3a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.653+0000 7f7d36000640 1 --2- 172.21.15.1:0/2462329317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564943c32140 0x564943c3a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.653+0000 7f7d36000640 1 -- 172.21.15.1:0/2462329317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c3ca00 msgr2=0x564943c54b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.653+0000 7f7d36000640 1 --2- 172.21.15.1:0/2462329317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c3ca00 0x564943c54b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.653+0000 7f7d36000640 1 -- 172.21.15.1:0/2462329317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564943c38970 con 0x564943c31820 2024-04-26T01:53:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.653+0000 7f7d36000640 1 --2- 172.21.15.1:0/2462329317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c31820 0x564943c484f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009eb0 tx=0x7f7d28009e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91a335a49da30ffb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.653+0000 7f7d21c00640 1 -- 172.21.15.1:0/2462329317 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2801f040 con 0x564943c31820 2024-04-26T01:53:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.653+0000 7f7d21c00640 1 -- 172.21.15.1:0/2462329317 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d28004500 con 0x564943c31820 2024-04-26T01:53:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.653+0000 7f7d21c00640 1 -- 172.21.15.1:0/2462329317 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2802e3f0 con 0x564943c31820 2024-04-26T01:53:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.654+0000 7f7d3ae15740 1 -- 172.21.15.1:0/2462329317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c31820 msgr2=0x564943c484f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.654+0000 7f7d3ae15740 1 --2- 172.21.15.1:0/2462329317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c31820 0x564943c484f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009eb0 tx=0x7f7d28009e00 comp rx=0 tx=0).stop 2024-04-26T01:53:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.654+0000 7f7d3ae15740 1 -- 172.21.15.1:0/2462329317 shutdown_connections 2024-04-26T01:53:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.654+0000 7f7d3ae15740 1 --2- 172.21.15.1:0/2462329317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c3ca00 0x564943c54b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.654+0000 7f7d3ae15740 1 --2- 172.21.15.1:0/2462329317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564943c32140 0x564943c3a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.654+0000 7f7d3ae15740 1 --2- 172.21.15.1:0/2462329317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c31820 0x564943c484f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.654+0000 7f7d3ae15740 1 -- 172.21.15.1:0/2462329317 >> 172.21.15.1:0/2462329317 conn(0x564943c2dd30 msgr2=0x564943c2e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.654+0000 7f7d3ae15740 1 -- 172.21.15.1:0/2462329317 shutdown_connections 2024-04-26T01:53:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.654+0000 7f7d3ae15740 1 -- 172.21.15.1:0/2462329317 wait complete. 2024-04-26T01:53:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.655+0000 7f7d3ae15740 1 -- start start 2024-04-26T01:53:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.655+0000 7f7d3ae15740 1 Processor -- start 2024-04-26T01:53:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.655+0000 7f7d3ae15740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c31820 0x564943c42330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.655+0000 7f7d3ae15740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564943c32140 0x564943c42870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.655+0000 7f7d3ae15740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c3ca00 0x564943c42db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.655+0000 7f7d3ae15740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301ae490 con 0x564943c31820 2024-04-26T01:53:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.655+0000 7f7d3ae15740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301af900 con 0x564943c3ca00 2024-04-26T01:53:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.655+0000 7f7d3ae15740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301aeb60 con 0x564943c32140 2024-04-26T01:53:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.655+0000 7f7d3ae15740 10 client.0 _ll_register_callbacks cb 0x7f7d1c012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:53:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.656+0000 7f7d36000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c31820 0x564943c42330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.656+0000 7f7d36000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c31820 0x564943c42330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52408/0 (socket says 172.21.15.1:52408) 2024-04-26T01:53:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.656+0000 7f7d36000640 1 -- 172.21.15.1:0/2885330247 learned_addr learned my addr 172.21.15.1:0/2885330247 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:53:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.656+0000 7f7d34c00640 1 --2- 172.21.15.1:0/2885330247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564943c32140 0x564943c42870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.656+0000 7f7d36000640 1 --2- 172.21.15.1:0/2885330247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c3ca00 0x564943c42db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.656+0000 7f7d36000640 1 -- 172.21.15.1:0/2885330247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564943c32140 msgr2=0x564943c42870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.656+0000 7f7d36000640 1 --2- 172.21.15.1:0/2885330247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564943c32140 0x564943c42870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.656+0000 7f7d36000640 1 -- 172.21.15.1:0/2885330247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c3ca00 msgr2=0x564943c42db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:53:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.656+0000 7f7d36000640 1 --2- 172.21.15.1:0/2885330247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c3ca00 0x564943c42db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.656+0000 7f7d36000640 1 -- 172.21.15.1:0/2885330247 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564943c31f70 con 0x564943c31820 2024-04-26T01:53:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.656+0000 7f7d36000640 1 --2- 172.21.15.1:0/2885330247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c31820 0x564943c42330 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7d28010010 tx=0x7f7d28011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45067b8ec51af017 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.656+0000 7f7d16a00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:53:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.657+0000 7f7d16a00640 1 -- 172.21.15.1:0/2885330247 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28032040 con 0x564943c31820 2024-04-26T01:53:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.657+0000 7f7d16a00640 1 -- 172.21.15.1:0/2885330247 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f7d28004050 con 0x564943c31820 2024-04-26T01:53:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.657+0000 7f7d3ae15740 1 -- 172.21.15.1:0/2885330247 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d28004500 con 0x564943c31820 2024-04-26T01:53:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.657+0000 7f7d16a00640 1 -- 172.21.15.1:0/2885330247 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7d28040050 con 0x564943c31820 2024-04-26T01:53:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.657+0000 7f7d3ae15740 10 client.15351 fetch_fsmap learned FSMap version 30 2024-04-26T01:53:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.657+0000 7f7d3ae15740 1 -- 172.21.15.1:0/2885330247 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564943cb3430 con 0x564943c31820 2024-04-26T01:53:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.657+0000 7f7d16a00640 1 -- 172.21.15.1:0/2885330247 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d28004500 con 0x564943c31820 2024-04-26T01:53:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.658+0000 7f7d16a00640 1 -- 172.21.15.1:0/2885330247 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7d28018070 con 0x564943c31820 2024-04-26T01:53:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.658+0000 7f7d3ae15740 10 client.15351 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:53:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.658+0000 7f7d3ae15740 10 client.15351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:53:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.658+0000 7f7d3ae15740 20 client.15351 populate_metadata read hostname 'smithi001' 2024-04-26T01:53:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.658+0000 7f7d3ae15740 1 --2- 172.21.15.1:0/2885330247 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564943cb3860 0x564943cb5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.658+0000 7f7d3ae15740 4 client.15351 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:53:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.658+0000 7f7d3ae15740 1 -- 172.21.15.1:0/2885330247 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564943ba5320 con 0x564943cb3860 2024-04-26T01:53:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.658+0000 7f7d34c00640 1 --2- 172.21.15.1:0/2885330247 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564943cb3860 0x564943cb5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.659+0000 7f7d34c00640 1 --2- 172.21.15.1:0/2885330247 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564943cb3860 0x564943cb5c80 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c135f596764aa69 server_cookie=941e791be8cab992 in_seq=0 out_seq=0 2024-04-26T01:53:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:58.659+0000 7f7d16a00640 10 client.15351 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:58.666 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-26T01:53:58.744 INFO:tasks.workunit.client.0.smithi001.stdout:checking for xfs/xqm.h... yes 2024-04-26T01:53:58.867 INFO:tasks.workunit.client.0.smithi001.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-26T01:53:58.951 INFO:tasks.workunit.client.0.smithi001.stdout:checking xfs/handle.h usability... yes 2024-04-26T01:53:58.993 INFO:tasks.workunit.client.0.smithi001.stdout:checking xfs/handle.h presence... yes 2024-04-26T01:53:58.993 INFO:tasks.workunit.client.0.smithi001.stdout:checking for xfs/handle.h... yes 2024-04-26T01:53:59.106 INFO:tasks.workunit.client.0.smithi001.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-26T01:53:59.192 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/acl.h usability... yes 2024-04-26T01:53:59.234 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/acl.h presence... yes 2024-04-26T01:53:59.234 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/acl.h... yes 2024-04-26T01:53:59.322 INFO:tasks.workunit.client.0.smithi001.stdout:checking acl/libacl.h usability... yes 2024-04-26T01:53:59.365 INFO:tasks.workunit.client.0.smithi001.stdout:checking acl/libacl.h presence... yes 2024-04-26T01:53:59.365 INFO:tasks.workunit.client.0.smithi001.stdout:checking for acl/libacl.h... yes 2024-04-26T01:53:59.468 INFO:tasks.workunit.client.0.smithi001.stdout:checking for acl_init in -lacl... yes 2024-04-26T01:53:59.712 INFO:tasks.workunit.client.0.smithi001.stdout:checking gdbm-ndbm.h usability... no 2024-04-26T01:53:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.737+0000 7f7d16a00640 1 -- 172.21.15.1:0/2885330247 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564943ba5320 con 0x564943cb3860 2024-04-26T01:53:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.737+0000 7f7d16a00640 10 client.15351 handle_command_reply: tid=0 2024-04-26T01:53:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.737+0000 7f7d3ae15740 1 client.15351 shutdown 2024-04-26T01:53:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.737+0000 7f7d3ae15740 1 -- 172.21.15.1:0/2885330247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c31820 msgr2=0x564943c42330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.737+0000 7f7d3ae15740 1 --2- 172.21.15.1:0/2885330247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c31820 0x564943c42330 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7d28010010 tx=0x7f7d28011c50 comp rx=0 tx=0).stop 2024-04-26T01:53:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.738+0000 7f7d3ae15740 1 -- 172.21.15.1:0/2885330247 shutdown_connections 2024-04-26T01:53:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.738+0000 7f7d3ae15740 1 --2- 172.21.15.1:0/2885330247 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564943cb3860 0x564943cb5c80 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.738+0000 7f7d3ae15740 1 --2- 172.21.15.1:0/2885330247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564943c3ca00 0x564943c42db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.738+0000 7f7d3ae15740 1 --2- 172.21.15.1:0/2885330247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564943c32140 0x564943c42870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.738+0000 7f7d3ae15740 1 --2- 172.21.15.1:0/2885330247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564943c31820 0x564943c42330 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.738+0000 7f7d3ae15740 1 -- 172.21.15.1:0/2885330247 >> 172.21.15.1:0/2885330247 conn(0x564943c2dd30 msgr2=0x564943c33a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.738+0000 7f7d16a00640 0 client.15351 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:53:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.738+0000 7f7d3ae15740 1 -- 172.21.15.1:0/2885330247 shutdown_connections 2024-04-26T01:53:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.738+0000 7f7d3ae15740 1 -- 172.21.15.1:0/2885330247 wait complete. 2024-04-26T01:53:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.738+0000 7f7d3ae15740 20 client.15351 trim_cache size 0 max 16384 2024-04-26T01:53:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:53:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:53:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:53:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:53:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.739+0000 7f7d38000640 1 -- 172.21.15.1:0/806228455 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7d180a3d10 msgr2=0x7f7d180a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.739+0000 7f7d38000640 1 --2- 172.21.15.1:0/806228455 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7d180a3d10 0x7f7d180a61d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x564943c32dc0 tx=0x7f7d240f49d0 comp rx=0 tx=0).stop 2024-04-26T01:53:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.739+0000 7f7d38000640 1 -- 172.21.15.1:0/806228455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d301021b0 msgr2=0x7f7d301a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.739+0000 7f7d38000640 1 --2- 172.21.15.1:0/806228455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d301021b0 0x7f7d301a06b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7d24005ab0 tx=0x7f7d24003c10 comp rx=0 tx=0).stop 2024-04-26T01:53:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.739+0000 7f7d38000640 1 -- 172.21.15.1:0/806228455 shutdown_connections 2024-04-26T01:53:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.739+0000 7f7d38000640 1 --2- 172.21.15.1:0/806228455 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7d180a3d10 0x7f7d180a61d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.739+0000 7f7d38000640 1 --2- 172.21.15.1:0/806228455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d3019a820 0x7f7d3019acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.739+0000 7f7d38000640 1 --2- 172.21.15.1:0/806228455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d30107280 0x7f7d301a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.739+0000 7f7d38000640 1 --2- 172.21.15.1:0/806228455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d301021b0 0x7f7d301a06b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.739+0000 7f7d38000640 1 -- 172.21.15.1:0/806228455 >> 172.21.15.1:0/806228455 conn(0x7f7d300fda10 msgr2=0x7f7d300ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.740+0000 7f7d38000640 1 -- 172.21.15.1:0/806228455 shutdown_connections 2024-04-26T01:53:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:53:59.740+0000 7f7d38000640 1 -- 172.21.15.1:0/806228455 wait complete. 2024-04-26T01:53:59.764 INFO:tasks.workunit.client.0.smithi001.stdout:checking gdbm-ndbm.h presence... no 2024-04-26T01:53:59.764 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gdbm-ndbm.h... no 2024-04-26T01:53:59.832 INFO:tasks.workunit.client.0.smithi001.stdout:checking gdbm/ndbm.h usability... no 2024-04-26T01:53:59.890 INFO:tasks.workunit.client.0.smithi001.stdout:checking gdbm/ndbm.h presence... no 2024-04-26T01:53:59.890 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gdbm/ndbm.h... no 2024-04-26T01:53:59.976 INFO:tasks.workunit.client.0.smithi001.stdout:checking ndbm.h usability... no 2024-04-26T01:54:00.035 INFO:tasks.workunit.client.0.smithi001.stdout:checking ndbm.h presence... no 2024-04-26T01:54:00.036 INFO:tasks.workunit.client.0.smithi001.stdout:checking for ndbm.h... no 2024-04-26T01:54:00.113 INFO:tasks.workunit.client.0.smithi001.stdout:checking libaio.h usability... yes 2024-04-26T01:54:00.156 INFO:tasks.workunit.client.0.smithi001.stdout:checking libaio.h presence... yes 2024-04-26T01:54:00.156 INFO:tasks.workunit.client.0.smithi001.stdout:checking for libaio.h... yes 2024-04-26T01:54:00.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:53:59 smithi097 ceph-mon[32853]: pgmap v74: 97 pgs: 97 active+clean; 32 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.0 MiB/s wr, 663 op/s 2024-04-26T01:54:00.256 INFO:tasks.workunit.client.0.smithi001.stdout:checking liburing.h usability... no 2024-04-26T01:54:00.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:53:59 smithi173 ceph-mon[32553]: pgmap v74: 97 pgs: 97 active+clean; 32 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.0 MiB/s wr, 663 op/s 2024-04-26T01:54:00.333 INFO:tasks.workunit.client.0.smithi001.stdout:checking liburing.h presence... no 2024-04-26T01:54:00.333 INFO:tasks.workunit.client.0.smithi001.stdout:checking for liburing.h... no 2024-04-26T01:54:00.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:53:59 smithi001 ceph-mon[27822]: pgmap v74: 97 pgs: 97 active+clean; 32 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.0 MiB/s wr, 663 op/s 2024-04-26T01:54:00.417 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/capability.h usability... no 2024-04-26T01:54:00.474 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/capability.h presence... no 2024-04-26T01:54:00.474 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/capability.h... no 2024-04-26T01:54:00.550 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/fiemap.h usability... yes 2024-04-26T01:54:00.589 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/fiemap.h presence... yes 2024-04-26T01:54:00.589 INFO:tasks.workunit.client.0.smithi001.stdout:checking for linux/fiemap.h... yes 2024-04-26T01:54:00.691 INFO:tasks.workunit.client.0.smithi001.stdout:checking for fallocate... yes 2024-04-26T01:54:00.802 INFO:tasks.workunit.client.0.smithi001.stdout:checking for open_by_handle_at... yes 2024-04-26T01:54:00.891 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/fs.h usability... yes 2024-04-26T01:54:00.931 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/fs.h presence... yes 2024-04-26T01:54:00.931 INFO:tasks.workunit.client.0.smithi001.stdout:checking for linux/fs.h... yes 2024-04-26T01:54:01.060 INFO:tasks.workunit.client.0.smithi001.stdout:checking btrfsutil.h usability... no 2024-04-26T01:54:01.127 INFO:tasks.workunit.client.0.smithi001.stdout:checking btrfsutil.h presence... no 2024-04-26T01:54:01.127 INFO:tasks.workunit.client.0.smithi001.stdout:checking for btrfsutil.h... no 2024-04-26T01:54:01.234 INFO:tasks.workunit.client.0.smithi001.stdout:checking for copy_file_range... yes 2024-04-26T01:54:01.341 INFO:tasks.workunit.client.0.smithi001.stdout:checking for SEEK_DATA... yes 2024-04-26T01:54:01.456 INFO:tasks.workunit.client.0.smithi001.stdout:checking for BMV_OF_SHARED... yes 2024-04-26T01:54:01.564 INFO:tasks.workunit.client.0.smithi001.stdout:checking for nftw... yes 2024-04-26T01:54:01.676 INFO:tasks.workunit.client.0.smithi001.stdout:checking for RLIMIT_NOFILE... yes 2024-04-26T01:54:01.777 INFO:tasks.workunit.client.0.smithi001.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-04-26T01:54:01.890 INFO:tasks.workunit.client.0.smithi001.stdout:checking for FICLONE... yes 2024-04-26T01:54:02.000 INFO:tasks.workunit.client.0.smithi001.stdout:checking for renameat2... yes 2024-04-26T01:54:02.106 INFO:tasks.workunit.client.0.smithi001.stdout:checking for reallocarray... yes 2024-04-26T01:54:02.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:01 smithi097 ceph-mon[32853]: pgmap v75: 97 pgs: 97 active+clean; 29 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.3 MiB/s wr, 762 op/s 2024-04-26T01:54:02.262 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct mount_attr... yes 2024-04-26T01:54:02.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:01 smithi173 ceph-mon[32553]: pgmap v75: 97 pgs: 97 active+clean; 29 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.3 MiB/s wr, 762 op/s 2024-04-26T01:54:02.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:01 smithi001 ceph-mon[27822]: pgmap v75: 97 pgs: 97 active+clean; 29 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.3 MiB/s wr, 762 op/s 2024-04-26T01:54:02.429 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_qgroup_limit... yes 2024-04-26T01:54:02.577 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-04-26T01:54:02.721 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-04-26T01:54:02.871 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-04-26T01:54:03.018 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-04-26T01:54:03.168 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-04-26T01:54:03.315 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-04-26T01:54:03.416 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/btrfs.h usability... yes 2024-04-26T01:54:03.456 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/btrfs.h presence... yes 2024-04-26T01:54:03.456 INFO:tasks.workunit.client.0.smithi001.stdout:checking for linux/btrfs.h... yes 2024-04-26T01:54:03.569 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/btrfs_tree.h usability... yes 2024-04-26T01:54:03.609 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/btrfs_tree.h presence... yes 2024-04-26T01:54:03.610 INFO:tasks.workunit.client.0.smithi001.stdout:checking for linux/btrfs_tree.h... yes 2024-04-26T01:54:03.700 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-04-26T01:54:03.779 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-04-26T01:54:03.824 INFO:tasks.workunit.client.0.smithi001.stdout:configure: creating ./config.status 2024-04-26T01:54:04.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:03 smithi173 ceph-mon[32553]: pgmap v76: 97 pgs: 97 active+clean; 29 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 361 KiB/s wr, 107 op/s 2024-04-26T01:54:04.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:03 smithi001 ceph-mon[27822]: pgmap v76: 97 pgs: 97 active+clean; 29 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 361 KiB/s wr, 107 op/s 2024-04-26T01:54:04.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:03 smithi097 ceph-mon[32853]: pgmap v76: 97 pgs: 97 active+clean; 29 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 361 KiB/s wr, 107 op/s 2024-04-26T01:54:05.192 INFO:tasks.workunit.client.0.smithi001.stdout:config.status: creating include/builddefs 2024-04-26T01:54:05.244 INFO:tasks.workunit.client.0.smithi001.stdout:config.status: creating include/config.h 2024-04-26T01:54:05.284 INFO:tasks.workunit.client.0.smithi001.stdout:config.status: executing libtool commands 2024-04-26T01:54:05.457 INFO:tasks.workunit.client.0.smithi001.stdout:Building include 2024-04-26T01:54:05.457 INFO:tasks.workunit.client.0.smithi001.stdout:Building lib 2024-04-26T01:54:05.458 INFO:tasks.workunit.client.0.smithi001.stdout:Building tests 2024-04-26T01:54:05.469 INFO:tasks.workunit.client.0.smithi001.stdout: [LTDEP] 2024-04-26T01:54:05.478 INFO:tasks.workunit.client.0.smithi001.stdout:Building btrfs 2024-04-26T01:54:05.479 INFO:tasks.workunit.client.0.smithi001.stdout:Building ceph 2024-04-26T01:54:05.481 INFO:tasks.workunit.client.0.smithi001.stdout:Building cifs 2024-04-26T01:54:05.501 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ceph/group.list 2024-04-26T01:54:05.501 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/cifs/group.list 2024-04-26T01:54:05.524 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/btrfs/group.list 2024-04-26T01:54:05.549 INFO:tasks.workunit.client.0.smithi001.stdout:Building ext4 2024-04-26T01:54:05.554 INFO:tasks.workunit.client.0.smithi001.stdout:Building f2fs 2024-04-26T01:54:05.570 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/f2fs/group.list 2024-04-26T01:54:05.577 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ext4/group.list 2024-04-26T01:54:05.630 INFO:tasks.workunit.client.0.smithi001.stdout:Building generic 2024-04-26T01:54:05.714 INFO:tasks.workunit.client.0.smithi001.stdout:Building nfs 2024-04-26T01:54:05.728 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/nfs/group.list 2024-04-26T01:54:05.758 INFO:tasks.workunit.client.0.smithi001.stdout:Building ocfs2 2024-04-26T01:54:05.774 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ocfs2/group.list 2024-04-26T01:54:05.776 INFO:tasks.workunit.client.0.smithi001.stdout:Building overlay 2024-04-26T01:54:05.812 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/overlay/group.list 2024-04-26T01:54:05.825 INFO:tasks.workunit.client.0.smithi001.stdout:Building perf 2024-04-26T01:54:05.841 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/perf/group.list 2024-04-26T01:54:05.841 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/generic/group.list 2024-04-26T01:54:05.878 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] dataascii.lo 2024-04-26T01:54:05.892 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] databin.lo 2024-04-26T01:54:05.974 INFO:tasks.workunit.client.0.smithi001.stdout:Building selftest 2024-04-26T01:54:05.987 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/selftest/group.list 2024-04-26T01:54:06.030 INFO:tasks.workunit.client.0.smithi001.stdout:Building shared 2024-04-26T01:54:06.043 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/shared/group.list 2024-04-26T01:54:06.101 INFO:tasks.workunit.client.0.smithi001.stdout:Building tmpfs 2024-04-26T01:54:06.115 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/tmpfs/group.list 2024-04-26T01:54:06.138 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] datapid.lo 2024-04-26T01:54:06.162 INFO:tasks.workunit.client.0.smithi001.stdout:Building udf 2024-04-26T01:54:06.178 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/udf/group.list 2024-04-26T01:54:06.190 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] file_lock.lo 2024-04-26T01:54:06.233 INFO:tasks.workunit.client.0.smithi001.stdout:Building xfs 2024-04-26T01:54:06.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:06 smithi173 ceph-mon[32553]: pgmap v77: 97 pgs: 97 active+clean; 26 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 385 KiB/s wr, 126 op/s 2024-04-26T01:54:06.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:06 smithi001 ceph-mon[27822]: pgmap v77: 97 pgs: 97 active+clean; 26 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 385 KiB/s wr, 126 op/s 2024-04-26T01:54:06.479 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] forker.lo 2024-04-26T01:54:06.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:06 smithi097 ceph-mon[32853]: pgmap v77: 97 pgs: 97 active+clean; 26 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 385 KiB/s wr, 126 op/s 2024-04-26T01:54:07.506 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] pattern.lo 2024-04-26T01:54:07.573 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] open_flags.lo 2024-04-26T01:54:07.610 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] random_range.lo 2024-04-26T01:54:07.724 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/xfs/group.list 2024-04-26T01:54:07.758 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] string_to_tokens.lo 2024-04-26T01:54:07.926 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] str_to_bytes.lo 2024-04-26T01:54:08.018 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] tlibio.lo 2024-04-26T01:54:08.039 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] write_log.lo 2024-04-26T01:54:08.159 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] random.lo 2024-04-26T01:54:08.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:08 smithi173 ceph-mon[32553]: pgmap v78: 97 pgs: 97 active+clean; 27 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 509 KiB/s wr, 132 op/s 2024-04-26T01:54:08.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:08 smithi001 ceph-mon[27822]: pgmap v78: 97 pgs: 97 active+clean; 27 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 509 KiB/s wr, 132 op/s 2024-04-26T01:54:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:08 smithi097 ceph-mon[32853]: pgmap v78: 97 pgs: 97 active+clean; 27 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 509 KiB/s wr, 132 op/s 2024-04-26T01:54:08.502 INFO:tasks.workunit.client.0.smithi001.stdout: [LD] 2024-04-26T01:54:08.718 INFO:tasks.workunit.client.0.smithi001.stdout:Building ltp 2024-04-26T01:54:08.718 INFO:tasks.workunit.client.0.smithi001.stdout:Building src 2024-04-26T01:54:08.718 INFO:tasks.workunit.client.0.smithi001.stdout:Building m4 2024-04-26T01:54:08.719 INFO:tasks.workunit.client.0.smithi001.stdout:Building common 2024-04-26T01:54:08.731 INFO:tasks.workunit.client.0.smithi001.stdout: [DEP] 2024-04-26T01:54:08.738 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-26T01:54:08.738 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-26T01:54:08.739 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:54:08.752 INFO:tasks.workunit.client.0.smithi001.stdout: [DEP] 2024-04-26T01:54:09.111 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] doio 2024-04-26T01:54:09.112 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fsstress 2024-04-26T01:54:09.114 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fsx 2024-04-26T01:54:09.203 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:10.156 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] growfiles 2024-04-26T01:54:10.158 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] iogen 2024-04-26T01:54:10.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:10 smithi173 ceph-mon[32553]: pgmap v79: 97 pgs: 97 active+clean; 27 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 408 KiB/s wr, 124 op/s 2024-04-26T01:54:10.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:10 smithi001 ceph-mon[27822]: pgmap v79: 97 pgs: 97 active+clean; 27 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 408 KiB/s wr, 124 op/s 2024-04-26T01:54:10.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:10 smithi097 ceph-mon[32853]: pgmap v79: 97 pgs: 97 active+clean; 27 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 408 KiB/s wr, 124 op/s 2024-04-26T01:54:10.714 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-stress 2024-04-26T01:54:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.614+0000 7fd811800700 1 -- 172.21.15.1:0/2065299573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd80c102960 msgr2=0x7fd80c102de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.614+0000 7fd811800700 1 --2- 172.21.15.1:0/2065299573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd80c102960 0x7fd80c102de0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009990 tx=0x7fd7f8009ca0 comp rx=0 tx=0).stop 2024-04-26T01:54:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.615+0000 7fd811800700 1 -- 172.21.15.1:0/2065299573 shutdown_connections 2024-04-26T01:54:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.615+0000 7fd811800700 1 --2- 172.21.15.1:0/2065299573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd80c103320 0x7fd80c10f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.615+0000 7fd811800700 1 --2- 172.21.15.1:0/2065299573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd80c108960 0x7fd80c108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.615+0000 7fd811800700 1 --2- 172.21.15.1:0/2065299573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd80c102960 0x7fd80c102de0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.615+0000 7fd811800700 1 -- 172.21.15.1:0/2065299573 >> 172.21.15.1:0/2065299573 conn(0x7fd80c0fe490 msgr2=0x7fd80c1008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.615+0000 7fd811800700 1 -- 172.21.15.1:0/2065299573 shutdown_connections 2024-04-26T01:54:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.615+0000 7fd811800700 1 -- 172.21.15.1:0/2065299573 wait complete. 2024-04-26T01:54:11.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.617+0000 7fd811800700 1 Processor -- start 2024-04-26T01:54:11.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.617+0000 7fd811800700 1 -- start start 2024-04-26T01:54:11.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.618+0000 7fd811800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd80c102960 0x7fd80c19b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:11.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.618+0000 7fd811800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd80c103320 0x7fd80c19b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:11.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.618+0000 7fd811800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd80c108960 0x7fd80c19faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:11.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.618+0000 7fd811800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c112ba0 con 0x7fd80c103320 2024-04-26T01:54:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.618+0000 7fd811800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c112a20 con 0x7fd80c102960 2024-04-26T01:54:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.618+0000 7fd811800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c112d20 con 0x7fd80c108960 2024-04-26T01:54:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.618+0000 7fd80b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd80c103320 0x7fd80c19b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.618+0000 7fd80b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd80c103320 0x7fd80c19b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42532/0 (socket says 172.21.15.1:42532) 2024-04-26T01:54:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.618+0000 7fd80b400700 1 -- 172.21.15.1:0/1663696176 learned_addr learned my addr 172.21.15.1:0/1663696176 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.618+0000 7fd80be00700 1 --2- 172.21.15.1:0/1663696176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd80c102960 0x7fd80c19b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:11.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.618+0000 7fd810e00700 1 --2- 172.21.15.1:0/1663696176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd80c108960 0x7fd80c19faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.619+0000 7fd80b400700 1 -- 172.21.15.1:0/1663696176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd80c108960 msgr2=0x7fd80c19faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.619+0000 7fd80b400700 1 --2- 172.21.15.1:0/1663696176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd80c108960 0x7fd80c19faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.619+0000 7fd80b400700 1 -- 172.21.15.1:0/1663696176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd80c102960 msgr2=0x7fd80c19b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.619+0000 7fd80b400700 1 --2- 172.21.15.1:0/1663696176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd80c102960 0x7fd80c19b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.619+0000 7fd80b400700 1 -- 172.21.15.1:0/1663696176 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80c1a01e0 con 0x7fd80c103320 2024-04-26T01:54:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.619+0000 7fd810e00700 1 --2- 172.21.15.1:0/1663696176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd80c108960 0x7fd80c19faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:54:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.619+0000 7fd80b400700 1 --2- 172.21.15.1:0/1663696176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd80c103320 0x7fd80c19b710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8000c00 tx=0x7fd7f800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.620+0000 7fd808c00700 1 -- 172.21.15.1:0/1663696176 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8027070 con 0x7fd80c103320 2024-04-26T01:54:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.620+0000 7fd808c00700 1 -- 172.21.15.1:0/1663696176 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7f8022470 con 0x7fd80c103320 2024-04-26T01:54:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.620+0000 7fd811800700 1 -- 172.21.15.1:0/1663696176 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80c1a0470 con 0x7fd80c103320 2024-04-26T01:54:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.620+0000 7fd811800700 1 -- 172.21.15.1:0/1663696176 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd80c1a7d50 con 0x7fd80c103320 2024-04-26T01:54:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.620+0000 7fd808c00700 1 -- 172.21.15.1:0/1663696176 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8017650 con 0x7fd80c103320 2024-04-26T01:54:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.621+0000 7fd811800700 1 -- 172.21.15.1:0/1663696176 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7ec005330 con 0x7fd80c103320 2024-04-26T01:54:11.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.624+0000 7fd808c00700 1 -- 172.21.15.1:0/1663696176 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fd7f802a070 con 0x7fd80c103320 2024-04-26T01:54:11.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.626+0000 7fd808c00700 1 --2- 172.21.15.1:0/1663696176 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd7f00a3db0 0x7fd7f00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:11.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.627+0000 7fd808c00700 1 -- 172.21.15.1:0/1663696176 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fd7f80d1530 con 0x7fd80c103320 2024-04-26T01:54:11.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.627+0000 7fd80be00700 1 --2- 172.21.15.1:0/1663696176 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd7f00a3db0 0x7fd7f00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:11.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.628+0000 7fd80be00700 1 --2- 172.21.15.1:0/1663696176 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd7f00a3db0 0x7fd7f00a6270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009be0 tx=0x7fd7fc009260 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:11.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.631+0000 7fd808c00700 1 -- 172.21.15.1:0/1663696176 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd7f8013070 con 0x7fd80c103320 2024-04-26T01:54:11.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.915+0000 7fd811800700 1 -- 172.21.15.1:0/1663696176 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7ec0058f0 con 0x7fd80c103320 2024-04-26T01:54:11.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.917+0000 7fd808c00700 1 -- 172.21.15.1:0/1663696176 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11841 (secure 0 0 0) 0x7fd7f809dd20 con 0x7fd80c103320 2024-04-26T01:54:11.917 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:54:11.917 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:53:29.647029+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25414":{"gid":25414,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/453427322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":453427322},{"type":"v1","addr":"172.21.15.173:6835","nonce":453427322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24518,24499,24514]},"id":1}]} 2024-04-26T01:54:11.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.920+0000 7fd811800700 1 -- 172.21.15.1:0/1663696176 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd7f00a3db0 msgr2=0x7fd7f00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:11.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.920+0000 7fd811800700 1 --2- 172.21.15.1:0/1663696176 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd7f00a3db0 0x7fd7f00a6270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009be0 tx=0x7fd7fc009260 comp rx=0 tx=0).stop 2024-04-26T01:54:11.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.920+0000 7fd811800700 1 -- 172.21.15.1:0/1663696176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd80c103320 msgr2=0x7fd80c19b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:11.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.920+0000 7fd811800700 1 --2- 172.21.15.1:0/1663696176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd80c103320 0x7fd80c19b710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8000c00 tx=0x7fd7f800bfd0 comp rx=0 tx=0).stop 2024-04-26T01:54:11.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.920+0000 7fd811800700 1 -- 172.21.15.1:0/1663696176 shutdown_connections 2024-04-26T01:54:11.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.920+0000 7fd811800700 1 --2- 172.21.15.1:0/1663696176 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd7f00a3db0 0x7fd7f00a6270 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:11.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.920+0000 7fd811800700 1 --2- 172.21.15.1:0/1663696176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd80c108960 0x7fd80c19faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:11.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.920+0000 7fd811800700 1 --2- 172.21.15.1:0/1663696176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd80c102960 0x7fd80c19b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:11.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.920+0000 7fd811800700 1 --2- 172.21.15.1:0/1663696176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd80c103320 0x7fd80c19b710 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:11.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.920+0000 7fd811800700 1 -- 172.21.15.1:0/1663696176 >> 172.21.15.1:0/1663696176 conn(0x7fd80c0fe490 msgr2=0x7fd80c10df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:11.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.920+0000 7fd811800700 1 -- 172.21.15.1:0/1663696176 shutdown_connections 2024-04-26T01:54:11.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:11.921+0000 7fd811800700 1 -- 172.21.15.1:0/1663696176 wait complete. 2024-04-26T01:54:11.921 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 30 2024-04-26T01:54:12.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:12 smithi173 ceph-mon[32553]: pgmap v80: 97 pgs: 97 active+clean; 37 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 1.5 MiB/s wr, 282 op/s 2024-04-26T01:54:12.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:12 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1663696176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:54:12.326 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:12 smithi001 ceph-mon[27822]: pgmap v80: 97 pgs: 97 active+clean; 37 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 1.5 MiB/s wr, 282 op/s 2024-04-26T01:54:12.326 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:12 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1663696176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:54:12.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:12 smithi097 ceph-mon[32853]: pgmap v80: 97 pgs: 97 active+clean; 37 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 1.5 MiB/s wr, 282 op/s 2024-04-26T01:54:12.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:12 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1663696176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:54:12.847 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24518 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-04-26T01:54:12.847 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24518 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-04-26T01:54:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.886+0000 7fe227200640 1 -- 172.21.15.1:0/1858479481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2201043c0 msgr2=0x7fe2201047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.886+0000 7fe227200640 1 --2- 172.21.15.1:0/1858479481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2201043c0 0x7fe2201047a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe210009860 tx=0x7fe21002a340 comp rx=0 tx=0).stop 2024-04-26T01:54:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.886+0000 7fe227200640 1 -- 172.21.15.1:0/1858479481 shutdown_connections 2024-04-26T01:54:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.886+0000 7fe227200640 1 --2- 172.21.15.1:0/1858479481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe220109490 0x7fe220111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.886+0000 7fe227200640 1 --2- 172.21.15.1:0/1858479481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe220104d70 0x7fe220108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.886+0000 7fe227200640 1 --2- 172.21.15.1:0/1858479481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2201043c0 0x7fe2201047a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.886+0000 7fe227200640 1 -- 172.21.15.1:0/1858479481 >> 172.21.15.1:0/1858479481 conn(0x7fe2200ffc40 msgr2=0x7fe220102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.886+0000 7fe227200640 1 -- 172.21.15.1:0/1858479481 shutdown_connections 2024-04-26T01:54:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.886+0000 7fe227200640 1 -- 172.21.15.1:0/1858479481 wait complete. 2024-04-26T01:54:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.887+0000 7fe227200640 1 Processor -- start 2024-04-26T01:54:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.888+0000 7fe227200640 1 -- start start 2024-04-26T01:54:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.888+0000 7fe227200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2201043c0 0x7fe2201a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.888+0000 7fe227200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe220109490 0x7fe2201a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.888+0000 7fe227200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22019a7f0 0x7fe22019aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.888+0000 7fe227200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201143f0 con 0x7fe2201043c0 2024-04-26T01:54:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.888+0000 7fe227200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe220114270 con 0x7fe22019a7f0 2024-04-26T01:54:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.888+0000 7fe227200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe220114570 con 0x7fe220109490 2024-04-26T01:54:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.889+0000 7fe21fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2201043c0 0x7fe2201a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.889+0000 7fe225200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22019a7f0 0x7fe22019aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.889+0000 7fe217e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe220109490 0x7fe2201a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.889+0000 7fe217e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe220109490 0x7fe2201a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:48528/0 (socket says 172.21.15.1:48528) 2024-04-26T01:54:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.889+0000 7fe217e00640 1 -- 172.21.15.1:0/871447921 learned_addr learned my addr 172.21.15.1:0/871447921 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.889+0000 7fe225200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22019a7f0 0x7fe22019aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:45966/0 (socket says 172.21.15.1:45966) 2024-04-26T01:54:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.889+0000 7fe225200640 1 -- 172.21.15.1:0/871447921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe220109490 msgr2=0x7fe2201a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.889+0000 7fe225200640 1 --2- 172.21.15.1:0/871447921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe220109490 0x7fe2201a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.889+0000 7fe225200640 1 -- 172.21.15.1:0/871447921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2201043c0 msgr2=0x7fe2201a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.889+0000 7fe225200640 1 --2- 172.21.15.1:0/871447921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2201043c0 0x7fe2201a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.889+0000 7fe225200640 1 -- 172.21.15.1:0/871447921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22019b380 con 0x7fe22019a7f0 2024-04-26T01:54:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.889+0000 7fe225200640 1 --2- 172.21.15.1:0/871447921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22019a7f0 0x7fe22019aca0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe21800da10 tx=0x7fe21800ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.890+0000 7fe21d600640 1 -- 172.21.15.1:0/871447921 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21800bd70 con 0x7fe22019a7f0 2024-04-26T01:54:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.890+0000 7fe21d600640 1 -- 172.21.15.1:0/871447921 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe218003e20 con 0x7fe22019a7f0 2024-04-26T01:54:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.890+0000 7fe227200640 1 -- 172.21.15.1:0/871447921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22019b670 con 0x7fe22019a7f0 2024-04-26T01:54:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.890+0000 7fe21d600640 1 -- 172.21.15.1:0/871447921 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2180106c0 con 0x7fe22019a7f0 2024-04-26T01:54:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.890+0000 7fe227200640 1 -- 172.21.15.1:0/871447921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2201a7800 con 0x7fe22019a7f0 2024-04-26T01:54:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.894+0000 7fe22a01e740 1 Processor -- start 2024-04-26T01:54:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.894+0000 7fe21d600640 1 -- 172.21.15.1:0/871447921 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fe218004270 con 0x7fe22019a7f0 2024-04-26T01:54:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.895+0000 7fe22a01e740 1 -- start start 2024-04-26T01:54:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.895+0000 7fe22a01e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bcbf820 0x55bc8bcbfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.895+0000 7fe22a01e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 0x55bc8bcca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.895+0000 7fe22a01e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bccaa00 0x55bc8bcccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.895+0000 7fe22a01e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bc8bc25900 con 0x55bc8bcbf820 2024-04-26T01:54:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.895+0000 7fe22a01e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bc8bbcbef0 con 0x55bc8bcc0140 2024-04-26T01:54:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.895+0000 7fe22a01e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bc8bc218c0 con 0x55bc8bccaa00 2024-04-26T01:54:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.895+0000 7fe217e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bcbf820 0x55bc8bcbfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.896+0000 7fe217e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bcbf820 0x55bc8bcbfc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56200/0 (socket says 172.21.15.1:56200) 2024-04-26T01:54:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.896+0000 7fe217e00640 1 -- 172.21.15.1:0/1119203018 learned_addr learned my addr 172.21.15.1:0/1119203018 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.896+0000 7fe21fe00640 1 --2- 172.21.15.1:0/1119203018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 0x55bc8bcca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.896+0000 7fe217e00640 1 --2- 172.21.15.1:0/1119203018 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bccaa00 0x55bc8bcccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.896+0000 7fe21fe00640 1 -- 172.21.15.1:0/1119203018 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bccaa00 msgr2=0x55bc8bcccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.896+0000 7fe21fe00640 1 --2- 172.21.15.1:0/1119203018 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bccaa00 0x55bc8bcccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.896+0000 7fe21fe00640 1 -- 172.21.15.1:0/1119203018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bcbf820 msgr2=0x55bc8bcbfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.896+0000 7fe21fe00640 1 --2- 172.21.15.1:0/1119203018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bcbf820 0x55bc8bcbfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.896+0000 7fe21fe00640 1 -- 172.21.15.1:0/1119203018 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc8b938210 con 0x55bc8bcc0140 2024-04-26T01:54:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.896+0000 7fe217e00640 1 --2- 172.21.15.1:0/1119203018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bcbf820 0x55bc8bcbfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.897+0000 7fe21fe00640 1 --2- 172.21.15.1:0/1119203018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 0x55bc8bcca4c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe210035040 tx=0x7fe210005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89f6d338af7150e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.897+0000 7fe215600640 1 -- 172.21.15.1:0/1119203018 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210041020 con 0x55bc8bcc0140 2024-04-26T01:54:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.897+0000 7fe215600640 1 -- 172.21.15.1:0/1119203018 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe210030450 con 0x55bc8bcc0140 2024-04-26T01:54:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.897+0000 7fe215600640 1 -- 172.21.15.1:0/1119203018 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210037040 con 0x55bc8bcc0140 2024-04-26T01:54:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.897+0000 7fe21d600640 1 --2- 172.21.15.1:0/871447921 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe2040a3d10 0x7fe2040a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.897+0000 7fe21d600640 1 -- 172.21.15.1:0/871447921 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fe2180d0ba0 con 0x7fe22019a7f0 2024-04-26T01:54:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.897+0000 7fe217e00640 1 --2- 172.21.15.1:0/871447921 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe2040a3d10 0x7fe2040a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.898+0000 7fe22a01e740 1 -- 172.21.15.1:0/1119203018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 msgr2=0x55bc8bcca4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.898+0000 7fe22a01e740 1 --2- 172.21.15.1:0/1119203018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 0x55bc8bcca4c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe210035040 tx=0x7fe210005d90 comp rx=0 tx=0).stop 2024-04-26T01:54:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.898+0000 7fe217e00640 1 --2- 172.21.15.1:0/871447921 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe2040a3d10 0x7fe2040a61d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe208009700 tx=0x7fe208015040 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.898+0000 7fe22a01e740 1 -- 172.21.15.1:0/1119203018 shutdown_connections 2024-04-26T01:54:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.898+0000 7fe22a01e740 1 --2- 172.21.15.1:0/1119203018 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bccaa00 0x55bc8bcccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.898+0000 7fe22a01e740 1 --2- 172.21.15.1:0/1119203018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 0x55bc8bcca4c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.898+0000 7fe22a01e740 1 --2- 172.21.15.1:0/1119203018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bcbf820 0x55bc8bcbfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.898+0000 7fe22a01e740 1 -- 172.21.15.1:0/1119203018 >> 172.21.15.1:0/1119203018 conn(0x55bc8bcbbd30 msgr2=0x55bc8bcbc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.898+0000 7fe22a01e740 1 -- 172.21.15.1:0/1119203018 shutdown_connections 2024-04-26T01:54:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.898+0000 7fe22a01e740 1 -- 172.21.15.1:0/1119203018 wait complete. 2024-04-26T01:54:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.899+0000 7fe22a01e740 1 -- start start 2024-04-26T01:54:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.899+0000 7fe22a01e740 1 Processor -- start 2024-04-26T01:54:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.900+0000 7fe22a01e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bcbf820 0x55bc8bcc7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.900+0000 7fe22a01e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 0x55bc8bcc7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.900+0000 7fe22a01e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bccaa00 0x55bc8bcc8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.900+0000 7fe22a01e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bc8bccd5d0 con 0x55bc8bccaa00 2024-04-26T01:54:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.900+0000 7fe22a01e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bc8bbdf310 con 0x55bc8bcc0140 2024-04-26T01:54:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.900+0000 7fe22a01e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bc8bccd750 con 0x55bc8bcbf820 2024-04-26T01:54:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.900+0000 7fe22a01e740 10 client.0 _ll_register_callbacks cb 0x55bc8bbadcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.900+0000 7fe21fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bcbf820 0x55bc8bcc7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.900+0000 7fe217e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 0x55bc8bcc7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.900+0000 7fe225200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bccaa00 0x55bc8bcc8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.900+0000 7fe21fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bcbf820 0x55bc8bcc7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:48552/0 (socket says 172.21.15.1:48552) 2024-04-26T01:54:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.900+0000 7fe21fe00640 1 -- 172.21.15.1:0/165889292 learned_addr learned my addr 172.21.15.1:0/165889292 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.900+0000 7fe21fe00640 1 -- 172.21.15.1:0/165889292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 msgr2=0x55bc8bcc7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.900+0000 7fe21fe00640 1 --2- 172.21.15.1:0/165889292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 0x55bc8bcc7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.900+0000 7fe21fe00640 1 -- 172.21.15.1:0/165889292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bccaa00 msgr2=0x55bc8bcc8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.900+0000 7fe21fe00640 1 --2- 172.21.15.1:0/165889292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bccaa00 0x55bc8bcc8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.900+0000 7fe21fe00640 1 -- 172.21.15.1:0/165889292 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc8ba04620 con 0x55bc8bcbf820 2024-04-26T01:54:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.901+0000 7fe21fe00640 1 --2- 172.21.15.1:0/165889292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bcbf820 0x55bc8bcc7980 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe210009780 tx=0x7fe210005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aa765cbbd7ce917 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.901+0000 7fe20e000640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:54:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.901+0000 7fe20e000640 1 -- 172.21.15.1:0/165889292 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210041030 con 0x55bc8bcbf820 2024-04-26T01:54:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.901+0000 7fe20e000640 1 -- 172.21.15.1:0/165889292 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fe21003d480 con 0x55bc8bcbf820 2024-04-26T01:54:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.901+0000 7fe22a01e740 1 -- 172.21.15.1:0/165889292 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe210030450 con 0x55bc8bcbf820 2024-04-26T01:54:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.902+0000 7fe20e000640 1 -- 172.21.15.1:0/165889292 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe210058050 con 0x55bc8bcbf820 2024-04-26T01:54:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.902+0000 7fe22a01e740 10 client.25519 fetch_fsmap learned FSMap version 30 2024-04-26T01:54:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.902+0000 7fe22a01e740 1 -- 172.21.15.1:0/165889292 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bc8bccd5d0 con 0x55bc8bcbf820 2024-04-26T01:54:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.902+0000 7fe20e000640 1 -- 172.21.15.1:0/165889292 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe210030450 con 0x55bc8bcbf820 2024-04-26T01:54:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.902+0000 7fe20e000640 1 -- 172.21.15.1:0/165889292 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fe210046890 con 0x55bc8bcbf820 2024-04-26T01:54:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.903+0000 7fe22a01e740 10 client.25519 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:54:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.903+0000 7fe22a01e740 10 client.25519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:54:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.903+0000 7fe22a01e740 20 client.25519 populate_metadata read hostname 'smithi001' 2024-04-26T01:54:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.903+0000 7fe22a01e740 1 --2- 172.21.15.1:0/165889292 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55bc8bcda9e0 0x55bc8bcdcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.903+0000 7fe22a01e740 4 client.25519 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:54:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.903+0000 7fe22a01e740 1 -- 172.21.15.1:0/165889292 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc8bc398b0 con 0x55bc8bcda9e0 2024-04-26T01:54:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.903+0000 7fe217e00640 1 --2- 172.21.15.1:0/165889292 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55bc8bcda9e0 0x55bc8bcdcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.904+0000 7fe217e00640 1 --2- 172.21.15.1:0/165889292 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55bc8bcda9e0 0x55bc8bcdcdd0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=246c50847483bc76 server_cookie=9ca5bc1c172d61f7 in_seq=0 out_seq=0 2024-04-26T01:54:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.904+0000 7fe20e000640 10 client.25519 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.904+0000 7fe20e000640 1 -- 172.21.15.1:0/165889292 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc8bc398b0 con 0x55bc8bcda9e0 2024-04-26T01:54:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.904+0000 7fe20e000640 10 client.25519 handle_command_reply: tid=0 2024-04-26T01:54:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.905+0000 7fe22a01e740 1 client.25519 shutdown 2024-04-26T01:54:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.905+0000 7fe22a01e740 1 -- 172.21.15.1:0/165889292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bcbf820 msgr2=0x55bc8bcc7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.905+0000 7fe22a01e740 1 --2- 172.21.15.1:0/165889292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bcbf820 0x55bc8bcc7980 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe210009780 tx=0x7fe210005f90 comp rx=0 tx=0).stop 2024-04-26T01:54:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.905+0000 7fe22a01e740 1 -- 172.21.15.1:0/165889292 shutdown_connections 2024-04-26T01:54:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.905+0000 7fe22a01e740 1 --2- 172.21.15.1:0/165889292 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55bc8bcda9e0 0x55bc8bcdcdd0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.905+0000 7fe22a01e740 1 --2- 172.21.15.1:0/165889292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bccaa00 0x55bc8bcc8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.905+0000 7fe22a01e740 1 --2- 172.21.15.1:0/165889292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 0x55bc8bcc7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.905+0000 7fe22a01e740 1 --2- 172.21.15.1:0/165889292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bcbf820 0x55bc8bcc7980 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.905+0000 7fe20e000640 0 client.25519 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.905+0000 7fe22a01e740 1 -- 172.21.15.1:0/165889292 >> 172.21.15.1:0/165889292 conn(0x55bc8bcbbd30 msgr2=0x55bc8bcc1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.905+0000 7fe22a01e740 1 -- 172.21.15.1:0/165889292 shutdown_connections 2024-04-26T01:54:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.905+0000 7fe22a01e740 1 -- 172.21.15.1:0/165889292 wait complete. 2024-04-26T01:54:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.905+0000 7fe22a01e740 20 client.25519 trim_cache size 0 max 16384 2024-04-26T01:54:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.922+0000 7fe22a01e740 1 Processor -- start 2024-04-26T01:54:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.922+0000 7fe22a01e740 1 -- start start 2024-04-26T01:54:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.922+0000 7fe22a01e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bcbf820 0x55bc8bcd6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.922+0000 7fe22a01e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 0x55bc8bce2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.922+0000 7fe22a01e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bccaa00 0x55bc8bce2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.922+0000 7fe22a01e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201b27b0 con 0x55bc8bcbf820 2024-04-26T01:54:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.922+0000 7fe22a01e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201b0020 con 0x55bc8bcc0140 2024-04-26T01:54:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.922+0000 7fe22a01e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201ade70 con 0x55bc8bccaa00 2024-04-26T01:54:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.922+0000 7fe21fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bcbf820 0x55bc8bcd6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.923+0000 7fe217e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 0x55bc8bce2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.923+0000 7fe217e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 0x55bc8bce2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:45990/0 (socket says 172.21.15.1:45990) 2024-04-26T01:54:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.923+0000 7fe217e00640 1 -- 172.21.15.1:0/230847546 learned_addr learned my addr 172.21.15.1:0/230847546 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.923+0000 7fe21fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bcbf820 0x55bc8bcd6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56218/0 (socket says 172.21.15.1:56218) 2024-04-26T01:54:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.923+0000 7fe225200640 1 --2- 172.21.15.1:0/230847546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bccaa00 0x55bc8bce2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.923+0000 7fe217e00640 1 -- 172.21.15.1:0/230847546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bccaa00 msgr2=0x55bc8bce2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.923+0000 7fe217e00640 1 --2- 172.21.15.1:0/230847546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bccaa00 0x55bc8bce2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.923+0000 7fe217e00640 1 -- 172.21.15.1:0/230847546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bcbf820 msgr2=0x55bc8bcd6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.923+0000 7fe217e00640 1 --2- 172.21.15.1:0/230847546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bcbf820 0x55bc8bcd6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.923+0000 7fe217e00640 1 -- 172.21.15.1:0/230847546 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc8bcdd3b0 con 0x55bc8bcc0140 2024-04-26T01:54:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.923+0000 7fe217e00640 1 --2- 172.21.15.1:0/230847546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 0x55bc8bce2640 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55bc8bcbd150 tx=0x7fe2080046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b64b3c38b1935ba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.924+0000 7fe214c00640 1 -- 172.21.15.1:0/230847546 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208013e60 con 0x55bc8bcc0140 2024-04-26T01:54:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.924+0000 7fe214c00640 1 -- 172.21.15.1:0/230847546 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe208032070 con 0x55bc8bcc0140 2024-04-26T01:54:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.924+0000 7fe214c00640 1 -- 172.21.15.1:0/230847546 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208010690 con 0x55bc8bcc0140 2024-04-26T01:54:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.924+0000 7fe22a01e740 1 -- 172.21.15.1:0/230847546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 msgr2=0x55bc8bce2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.924+0000 7fe22a01e740 1 --2- 172.21.15.1:0/230847546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 0x55bc8bce2640 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55bc8bcbd150 tx=0x7fe2080046c0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.925+0000 7fe22a01e740 1 -- 172.21.15.1:0/230847546 shutdown_connections 2024-04-26T01:54:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.925+0000 7fe22a01e740 1 --2- 172.21.15.1:0/230847546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bccaa00 0x55bc8bce2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.925+0000 7fe22a01e740 1 --2- 172.21.15.1:0/230847546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 0x55bc8bce2640 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.925+0000 7fe22a01e740 1 --2- 172.21.15.1:0/230847546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bcbf820 0x55bc8bcd6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.925+0000 7fe22a01e740 1 -- 172.21.15.1:0/230847546 >> 172.21.15.1:0/230847546 conn(0x55bc8bcbbd30 msgr2=0x55bc8bcc1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.925+0000 7fe22a01e740 1 -- 172.21.15.1:0/230847546 shutdown_connections 2024-04-26T01:54:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.925+0000 7fe22a01e740 1 -- 172.21.15.1:0/230847546 wait complete. 2024-04-26T01:54:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.925+0000 7fe22a01e740 1 -- start start 2024-04-26T01:54:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.925+0000 7fe22a01e740 1 Processor -- start 2024-04-26T01:54:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.926+0000 7fe22a01e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bcbf820 0x55bc8bcd2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.926+0000 7fe22a01e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 0x55bc8bcd01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.926+0000 7fe22a01e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bccaa00 0x55bc8bcd0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.926+0000 7fe22a01e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201ae4e0 con 0x55bc8bcbf820 2024-04-26T01:54:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.926+0000 7fe22a01e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201af950 con 0x55bc8bcc0140 2024-04-26T01:54:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.926+0000 7fe22a01e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201aebb0 con 0x55bc8bccaa00 2024-04-26T01:54:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.926+0000 7fe22a01e740 10 client.0 _ll_register_callbacks cb 0x7fe2100096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.926+0000 7fe21fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bcbf820 0x55bc8bcd2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.927+0000 7fe21fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bcbf820 0x55bc8bcd2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56226/0 (socket says 172.21.15.1:56226) 2024-04-26T01:54:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.927+0000 7fe21fe00640 1 -- 172.21.15.1:0/901566410 learned_addr learned my addr 172.21.15.1:0/901566410 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.927+0000 7fe217e00640 1 --2- 172.21.15.1:0/901566410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 0x55bc8bcd01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.927+0000 7fe225200640 1 --2- 172.21.15.1:0/901566410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bccaa00 0x55bc8bcd0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.927+0000 7fe21fe00640 1 -- 172.21.15.1:0/901566410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bccaa00 msgr2=0x55bc8bcd0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.927+0000 7fe21fe00640 1 --2- 172.21.15.1:0/901566410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bccaa00 0x55bc8bcd0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.927+0000 7fe21fe00640 1 -- 172.21.15.1:0/901566410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 msgr2=0x55bc8bcd01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.927+0000 7fe21fe00640 1 --2- 172.21.15.1:0/901566410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 0x55bc8bcd01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.927+0000 7fe21fe00640 1 -- 172.21.15.1:0/901566410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc8bcd0f50 con 0x55bc8bcbf820 2024-04-26T01:54:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.927+0000 7fe21fe00640 1 --2- 172.21.15.1:0/901566410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bcbf820 0x55bc8bcd2de0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe2100057c0 tx=0x7fe210009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12d8b8b45961fe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.927+0000 7fe20e000640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:54:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.928+0000 7fe20e000640 1 -- 172.21.15.1:0/901566410 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210041070 con 0x55bc8bcbf820 2024-04-26T01:54:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.928+0000 7fe20e000640 1 -- 172.21.15.1:0/901566410 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fe210004450 con 0x55bc8bcbf820 2024-04-26T01:54:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.928+0000 7fe22a01e740 1 -- 172.21.15.1:0/901566410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe208032070 con 0x55bc8bcbf820 2024-04-26T01:54:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.928+0000 7fe20e000640 1 -- 172.21.15.1:0/901566410 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe210062050 con 0x55bc8bcbf820 2024-04-26T01:54:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.928+0000 7fe22a01e740 10 client.15375 fetch_fsmap learned FSMap version 30 2024-04-26T01:54:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.928+0000 7fe22a01e740 1 -- 172.21.15.1:0/901566410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bc8bd40f60 con 0x55bc8bcbf820 2024-04-26T01:54:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.928+0000 7fe20e000640 1 -- 172.21.15.1:0/901566410 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe208032070 con 0x55bc8bcbf820 2024-04-26T01:54:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.929+0000 7fe20e000640 1 -- 172.21.15.1:0/901566410 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fe2100046c0 con 0x55bc8bcbf820 2024-04-26T01:54:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.929+0000 7fe22a01e740 10 client.15375 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:54:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.929+0000 7fe22a01e740 10 client.15375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:54:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.929+0000 7fe22a01e740 20 client.15375 populate_metadata read hostname 'smithi001' 2024-04-26T01:54:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.929+0000 7fe22a01e740 1 --2- 172.21.15.1:0/901566410 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55bc8bd416d0 0x55bc8bd43af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.929+0000 7fe22a01e740 4 client.15375 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:54:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.929+0000 7fe22a01e740 1 -- 172.21.15.1:0/901566410 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bc8bc33320 con 0x55bc8bd416d0 2024-04-26T01:54:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.929+0000 7fe217e00640 1 --2- 172.21.15.1:0/901566410 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55bc8bd416d0 0x55bc8bd43af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.930+0000 7fe217e00640 1 --2- 172.21.15.1:0/901566410 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55bc8bd416d0 0x55bc8bd43af0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e015a6cddaa117e4 server_cookie=941f0caf3479dcb6 in_seq=0 out_seq=0 2024-04-26T01:54:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:12.930+0000 7fe20e000640 10 client.15375 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.055+0000 7fe20e000640 1 -- 172.21.15.1:0/901566410 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bc8bc33320 con 0x55bc8bd416d0 2024-04-26T01:54:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.055+0000 7fe20e000640 10 client.15375 handle_command_reply: tid=0 2024-04-26T01:54:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.055+0000 7fe22a01e740 1 client.15375 shutdown 2024-04-26T01:54:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.055+0000 7fe22a01e740 1 -- 172.21.15.1:0/901566410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bcbf820 msgr2=0x55bc8bcd2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.055+0000 7fe22a01e740 1 --2- 172.21.15.1:0/901566410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bcbf820 0x55bc8bcd2de0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe2100057c0 tx=0x7fe210009450 comp rx=0 tx=0).stop 2024-04-26T01:54:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.056+0000 7fe22a01e740 1 -- 172.21.15.1:0/901566410 shutdown_connections 2024-04-26T01:54:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.056+0000 7fe22a01e740 1 --2- 172.21.15.1:0/901566410 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55bc8bd416d0 0x55bc8bd43af0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.056+0000 7fe22a01e740 1 --2- 172.21.15.1:0/901566410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bc8bccaa00 0x55bc8bcd0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.056+0000 7fe22a01e740 1 --2- 172.21.15.1:0/901566410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bc8bcc0140 0x55bc8bcd01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.056+0000 7fe22a01e740 1 --2- 172.21.15.1:0/901566410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc8bcbf820 0x55bc8bcd2de0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.056+0000 7fe22a01e740 1 -- 172.21.15.1:0/901566410 >> 172.21.15.1:0/901566410 conn(0x55bc8bcbbd30 msgr2=0x55bc8bcc11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.056+0000 7fe20e000640 0 client.15375 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.056+0000 7fe22a01e740 1 -- 172.21.15.1:0/901566410 shutdown_connections 2024-04-26T01:54:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.057+0000 7fe22a01e740 1 -- 172.21.15.1:0/901566410 wait complete. 2024-04-26T01:54:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.057+0000 7fe22a01e740 20 client.15375 trim_cache size 0 max 16384 2024-04-26T01:54:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:54:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:54:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:54:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:54:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.058+0000 7fe227200640 1 -- 172.21.15.1:0/871447921 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe2040a3d10 msgr2=0x7fe2040a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.058+0000 7fe227200640 1 --2- 172.21.15.1:0/871447921 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe2040a3d10 0x7fe2040a61d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe208009700 tx=0x7fe208015040 comp rx=0 tx=0).stop 2024-04-26T01:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.058+0000 7fe227200640 1 -- 172.21.15.1:0/871447921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22019a7f0 msgr2=0x7fe22019aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.058+0000 7fe227200640 1 --2- 172.21.15.1:0/871447921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22019a7f0 0x7fe22019aca0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe21800da10 tx=0x7fe21800ded0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.058+0000 7fe227200640 1 -- 172.21.15.1:0/871447921 shutdown_connections 2024-04-26T01:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.058+0000 7fe227200640 1 --2- 172.21.15.1:0/871447921 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe2040a3d10 0x7fe2040a61d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.058+0000 7fe227200640 1 --2- 172.21.15.1:0/871447921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22019a7f0 0x7fe22019aca0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.058+0000 7fe227200640 1 --2- 172.21.15.1:0/871447921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe220109490 0x7fe2201a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.058+0000 7fe227200640 1 --2- 172.21.15.1:0/871447921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2201043c0 0x7fe2201a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.058+0000 7fe227200640 1 -- 172.21.15.1:0/871447921 >> 172.21.15.1:0/871447921 conn(0x7fe2200ffc40 msgr2=0x7fe220101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.058+0000 7fe227200640 1 -- 172.21.15.1:0/871447921 shutdown_connections 2024-04-26T01:54:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:13.058+0000 7fe227200640 1 -- 172.21.15.1:0/871447921 wait complete. 2024-04-26T01:54:13.104 INFO:tasks.workunit.client.0.smithi001.stdout: [DEP] 2024-04-26T01:54:13.104 INFO:tasks.workunit.client.0.smithi001.stdout: [DEP] 2024-04-26T01:54:13.105 INFO:tasks.workunit.client.0.smithi001.stdout: [DEP] 2024-04-26T01:54:13.105 INFO:tasks.workunit.client.0.smithi001.stdout: [DEP] 2024-04-26T01:54:13.351 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:13.640 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] dirstress 2024-04-26T01:54:13.642 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fill 2024-04-26T01:54:13.642 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] getpagesize 2024-04-26T01:54:13.643 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fill2 2024-04-26T01:54:13.958 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] holes 2024-04-26T01:54:13.977 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] lstat64 2024-04-26T01:54:14.073 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] nametest 2024-04-26T01:54:14.101 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] permname 2024-04-26T01:54:14.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:14 smithi001 ceph-mon[27822]: pgmap v81: 97 pgs: 97 active+clean; 37 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.2 MiB/s wr, 184 op/s 2024-04-26T01:54:14.316 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:14 smithi173 ceph-mon[32553]: pgmap v81: 97 pgs: 97 active+clean; 37 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.2 MiB/s wr, 184 op/s 2024-04-26T01:54:14.328 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] randholes 2024-04-26T01:54:14.331 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] runas 2024-04-26T01:54:14.483 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] truncfile 2024-04-26T01:54:14.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:14 smithi097 ceph-mon[32853]: pgmap v81: 97 pgs: 97 active+clean; 37 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.2 MiB/s wr, 184 op/s 2024-04-26T01:54:14.487 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] usemem 2024-04-26T01:54:14.679 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] mmapcat 2024-04-26T01:54:14.776 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] append_reader 2024-04-26T01:54:14.787 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] append_writer 2024-04-26T01:54:14.818 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] dirperf 2024-04-26T01:54:14.961 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] metaperf 2024-04-26T01:54:15.079 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] devzero 2024-04-26T01:54:15.088 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] feature 2024-04-26T01:54:15.258 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] alloc 2024-04-26T01:54:15.424 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fault 2024-04-26T01:54:15.465 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fstest 2024-04-26T01:54:15.532 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_access_root 2024-04-26T01:54:15.645 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] godown 2024-04-26T01:54:15.734 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] resvtest 2024-04-26T01:54:15.827 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] writemod 2024-04-26T01:54:15.890 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] writev_on_pagefault 2024-04-26T01:54:15.999 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] makeextents 2024-04-26T01:54:16.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.038+0000 7f2c90200700 1 -- 172.21.15.1:0/2995040569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88109fc0 msgr2=0x7f2c88111b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:16.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.038+0000 7f2c90200700 1 --2- 172.21.15.1:0/2995040569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88109fc0 0x7f2c88111b90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000b0a0 tx=0x7f2c8000b3b0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.039+0000 7f2c90200700 1 -- 172.21.15.1:0/2995040569 shutdown_connections 2024-04-26T01:54:16.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.039+0000 7f2c90200700 1 --2- 172.21.15.1:0/2995040569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88109fc0 0x7f2c88111b90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.039+0000 7f2c90200700 1 --2- 172.21.15.1:0/2995040569 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c88104ee0 0x7f2c881052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.039+0000 7f2c90200700 1 --2- 172.21.15.1:0/2995040569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c88105890 0x7f2c881098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.039+0000 7f2c90200700 1 -- 172.21.15.1:0/2995040569 >> 172.21.15.1:0/2995040569 conn(0x7f2c88100770 msgr2=0x7f2c88102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:16.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.039+0000 7f2c90200700 1 -- 172.21.15.1:0/2995040569 shutdown_connections 2024-04-26T01:54:16.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.039+0000 7f2c90200700 1 -- 172.21.15.1:0/2995040569 wait complete. 2024-04-26T01:54:16.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.040+0000 7f2c90200700 1 Processor -- start 2024-04-26T01:54:16.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.040+0000 7f2c90200700 1 -- start start 2024-04-26T01:54:16.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.041+0000 7f2c90200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88104ee0 0x7f2c8819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.041+0000 7f2c90200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c88105890 0x7f2c8819b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.041+0000 7f2c90200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c88109fc0 0x7f2c8819faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.041+0000 7f2c90200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2c88114e60 con 0x7f2c88109fc0 2024-04-26T01:54:16.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.041+0000 7f2c90200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2c88114ce0 con 0x7f2c88104ee0 2024-04-26T01:54:16.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.041+0000 7f2c90200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2c88114fe0 con 0x7f2c88105890 2024-04-26T01:54:16.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.041+0000 7f2c8d800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88104ee0 0x7f2c8819b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:16.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.041+0000 7f2c8d800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88104ee0 0x7f2c8819b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:46016/0 (socket says 172.21.15.1:46016) 2024-04-26T01:54:16.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.041+0000 7f2c8d800700 1 -- 172.21.15.1:0/1701581890 learned_addr learned my addr 172.21.15.1:0/1701581890 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:16.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.041+0000 7f2c8ce00700 1 --2- 172.21.15.1:0/1701581890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c88105890 0x7f2c8819b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:16.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.042+0000 7f2c8e200700 1 --2- 172.21.15.1:0/1701581890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c88109fc0 0x7f2c8819faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:16.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.042+0000 7f2c8e200700 1 -- 172.21.15.1:0/1701581890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c88105890 msgr2=0x7f2c8819b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:16.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.042+0000 7f2c8e200700 1 --2- 172.21.15.1:0/1701581890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c88105890 0x7f2c8819b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.042+0000 7f2c8e200700 1 -- 172.21.15.1:0/1701581890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88104ee0 msgr2=0x7f2c8819b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:16.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.042+0000 7f2c8e200700 1 --2- 172.21.15.1:0/1701581890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88104ee0 0x7f2c8819b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.042+0000 7f2c8e200700 1 -- 172.21.15.1:0/1701581890 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c881a0230 con 0x7f2c88109fc0 2024-04-26T01:54:16.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.043+0000 7f2c8ce00700 1 --2- 172.21.15.1:0/1701581890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c88105890 0x7f2c8819b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:54:16.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.043+0000 7f2c8e200700 1 --2- 172.21.15.1:0/1701581890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c88109fc0 0x7f2c8819faf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000b0a0 tx=0x7f2c80004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:16.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.043+0000 7f2c86000700 1 -- 172.21.15.1:0/1701581890 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80021070 con 0x7f2c88109fc0 2024-04-26T01:54:16.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.044+0000 7f2c86000700 1 -- 172.21.15.1:0/1701581890 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c80007b80 con 0x7f2c88109fc0 2024-04-26T01:54:16.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.044+0000 7f2c86000700 1 -- 172.21.15.1:0/1701581890 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80005040 con 0x7f2c88109fc0 2024-04-26T01:54:16.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.044+0000 7f2c90200700 1 -- 172.21.15.1:0/1701581890 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c881a04c0 con 0x7f2c88109fc0 2024-04-26T01:54:16.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.044+0000 7f2c90200700 1 -- 172.21.15.1:0/1701581890 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c8806c460 con 0x7f2c88109fc0 2024-04-26T01:54:16.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.050+0000 7f2c9aad6b80 1 Processor -- start 2024-04-26T01:54:16.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.051+0000 7f2c86000700 1 -- 172.21.15.1:0/1701581890 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f2c80024070 con 0x7f2c88109fc0 2024-04-26T01:54:16.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.051+0000 7f2c9aad6b80 1 -- start start 2024-04-26T01:54:16.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.052+0000 7f2c9aad6b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 0x55d2855c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.052+0000 7f2c9aad6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855c3f00 0x55d2855ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.052+0000 7f2c9aad6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855ced50 0x55d2855d1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.052+0000 7f2c9aad6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d28550e020 con 0x55d2855ced50 2024-04-26T01:54:16.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.052+0000 7f2c9aad6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d2854b48c0 con 0x55d2855c3f00 2024-04-26T01:54:16.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.052+0000 7f2c9aad6b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d285509fe0 con 0x55d2855c35e0 2024-04-26T01:54:16.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.053+0000 7f2c8ce00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 0x55d2855c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:16.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.053+0000 7f2c8ce00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 0x55d2855c39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:48610/0 (socket says 172.21.15.1:48610) 2024-04-26T01:54:16.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.053+0000 7f2c8ce00700 1 -- 172.21.15.1:0/772883050 learned_addr learned my addr 172.21.15.1:0/772883050 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:16.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.053+0000 7f2c8ce00700 1 --2- 172.21.15.1:0/772883050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855ced50 0x55d2855d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:16.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.053+0000 7f2c8d800700 1 --2- 172.21.15.1:0/772883050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855c3f00 0x55d2855ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:16.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.054+0000 7f2c86000700 1 --2- 172.21.15.1:0/1701581890 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2c700a3ed0 0x7f2c700a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.054+0000 7f2c8ce00700 1 -- 172.21.15.1:0/772883050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855c3f00 msgr2=0x55d2855ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:16.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.054+0000 7f2c8ce00700 1 --2- 172.21.15.1:0/772883050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855c3f00 0x55d2855ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.054+0000 7f2c8ce00700 1 -- 172.21.15.1:0/772883050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855ced50 msgr2=0x55d2855d1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:16.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.054+0000 7f2c8ce00700 1 --2- 172.21.15.1:0/772883050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855ced50 0x55d2855d1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.054+0000 7f2c8ce00700 1 -- 172.21.15.1:0/772883050 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2854aa810 con 0x55d2855c35e0 2024-04-26T01:54:16.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.054+0000 7f2c8e200700 1 --2- 172.21.15.1:0/1701581890 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2c700a3ed0 0x7f2c700a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:16.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.054+0000 7f2c86000700 1 -- 172.21.15.1:0/1701581890 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f2c800d5060 con 0x7f2c88109fc0 2024-04-26T01:54:16.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.055+0000 7f2c8ce00700 1 --2- 172.21.15.1:0/772883050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 0x55d2855c39c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2c740127f0 tx=0x7f2c74009710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daf19fc4ee9d6f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:16.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.055+0000 7f2c7aa00700 1 -- 172.21.15.1:0/772883050 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c74012e10 con 0x55d2855c35e0 2024-04-26T01:54:16.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.055+0000 7f2c7aa00700 1 -- 172.21.15.1:0/772883050 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c7400f900 con 0x55d2855c35e0 2024-04-26T01:54:16.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.055+0000 7f2c7aa00700 1 -- 172.21.15.1:0/772883050 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7401e890 con 0x55d2855c35e0 2024-04-26T01:54:16.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.056+0000 7f2c8e200700 1 --2- 172.21.15.1:0/1701581890 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2c700a3ed0 0x7f2c700a6390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2c8806c630 tx=0x7f2c80009040 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:16.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.056+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/772883050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 msgr2=0x55d2855c39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:16.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.056+0000 7f2c9aad6b80 1 --2- 172.21.15.1:0/772883050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 0x55d2855c39c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2c740127f0 tx=0x7f2c74009710 comp rx=0 tx=0).stop 2024-04-26T01:54:16.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.057+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/772883050 shutdown_connections 2024-04-26T01:54:16.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.057+0000 7f2c9aad6b80 1 --2- 172.21.15.1:0/772883050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855ced50 0x55d2855d1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.057+0000 7f2c9aad6b80 1 --2- 172.21.15.1:0/772883050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 0x55d2855c39c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.057+0000 7f2c9aad6b80 1 --2- 172.21.15.1:0/772883050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855c3f00 0x55d2855ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.057+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/772883050 >> 172.21.15.1:0/772883050 conn(0x55d2854ea0e0 msgr2=0x55d2855b71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:16.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.058+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/772883050 shutdown_connections 2024-04-26T01:54:16.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.058+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/772883050 wait complete. 2024-04-26T01:54:16.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.059+0000 7f2c9aad6b80 1 -- start start 2024-04-26T01:54:16.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.059+0000 7f2c9aad6b80 1 Processor -- start 2024-04-26T01:54:16.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.060+0000 7f2c9aad6b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 0x55d2855ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.060+0000 7f2c9aad6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855c3f00 0x55d2855cb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.060+0000 7f2c9aad6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855ced50 0x55d2855cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.060+0000 7f2c9aad6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d2855d1840 con 0x55d2855c3f00 2024-04-26T01:54:16.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.060+0000 7f2c9aad6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d2854c79f0 con 0x55d2855ced50 2024-04-26T01:54:16.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.060+0000 7f2c9aad6b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d2855d19c0 con 0x55d2855c35e0 2024-04-26T01:54:16.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.060+0000 7f2c9aad6b80 10 client.0 _ll_register_callbacks cb 0x55d2854a68b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:16.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.061+0000 7f2c8d800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855c3f00 0x55d2855cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:16.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.061+0000 7f2c8d800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855c3f00 0x55d2855cb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56248/0 (socket says 172.21.15.1:56248) 2024-04-26T01:54:16.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.061+0000 7f2c8d800700 1 -- 172.21.15.1:0/2606791921 learned_addr learned my addr 172.21.15.1:0/2606791921 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:16.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.061+0000 7f2c8ce00700 1 --2- 172.21.15.1:0/2606791921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855ced50 0x55d2855cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:16.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.062+0000 7f2c8ce00700 1 --2- 172.21.15.1:0/2606791921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 0x55d2855ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:16.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.062+0000 7f2c8d800700 1 -- 172.21.15.1:0/2606791921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 msgr2=0x55d2855ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:16.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.062+0000 7f2c8d800700 1 --2- 172.21.15.1:0/2606791921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 0x55d2855ce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.062+0000 7f2c8d800700 1 -- 172.21.15.1:0/2606791921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855ced50 msgr2=0x55d2855cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:16.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.062+0000 7f2c8d800700 1 --2- 172.21.15.1:0/2606791921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855ced50 0x55d2855cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.062+0000 7f2c8d800700 1 -- 172.21.15.1:0/2606791921 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d28545ba60 con 0x55d2855c3f00 2024-04-26T01:54:16.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.062+0000 7f2c8d800700 1 --2- 172.21.15.1:0/2606791921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855c3f00 0x55d2855cb770 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c0128a0 tx=0x7f2c7c012bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815229abed151e44 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:16.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.062+0000 7f2c6f400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:54:16.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.063+0000 7f2c6f400700 1 -- 172.21.15.1:0/2606791921 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c004050 con 0x55d2855c3f00 2024-04-26T01:54:16.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.063+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/2606791921 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c7400f900 con 0x55d2855c3f00 2024-04-26T01:54:16.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.064+0000 7f2c6f400700 1 -- 172.21.15.1:0/2606791921 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f2c7c0041f0 con 0x55d2855c3f00 2024-04-26T01:54:16.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.064+0000 7f2c6f400700 1 -- 172.21.15.1:0/2606791921 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2c7c0043f0 con 0x55d2855c3f00 2024-04-26T01:54:16.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.064+0000 7f2c9aad6b80 10 client.15387 fetch_fsmap learned FSMap version 30 2024-04-26T01:54:16.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.064+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/2606791921 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d2854a4c80 con 0x55d2855c3f00 2024-04-26T01:54:16.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.065+0000 7f2c6f400700 1 -- 172.21.15.1:0/2606791921 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c7400f900 con 0x55d2855c3f00 2024-04-26T01:54:16.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.065+0000 7f2c6f400700 1 -- 172.21.15.1:0/2606791921 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f2c7c00f8f0 con 0x55d2855c3f00 2024-04-26T01:54:16.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.065+0000 7f2c9aad6b80 10 client.15387 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:54:16.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.065+0000 7f2c9aad6b80 10 client.15387 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:54:16.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.066+0000 7f2c9aad6b80 20 client.15387 populate_metadata read hostname 'smithi001' 2024-04-26T01:54:16.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.066+0000 7f2c9aad6b80 1 --2- 172.21.15.1:0/2606791921 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d2855de590 0x55d2855e0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.066+0000 7f2c9aad6b80 4 client.15387 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:54:16.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.066+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/2606791921 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d28551b810 con 0x55d2855de590 2024-04-26T01:54:16.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.066+0000 7f2c8ce00700 1 --2- 172.21.15.1:0/2606791921 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d2855de590 0x55d2855e0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:16.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.067+0000 7f2c8ce00700 1 --2- 172.21.15.1:0/2606791921 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d2855de590 0x55d2855e0980 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=870b36c34645f36e server_cookie=8ebadda66172a394 in_seq=0 out_seq=0 2024-04-26T01:54:16.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.067+0000 7f2c6f400700 10 client.15387 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:16.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.069+0000 7f2c6f400700 1 -- 172.21.15.1:0/2606791921 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d28551b810 con 0x55d2855de590 2024-04-26T01:54:16.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.069+0000 7f2c6f400700 10 client.15387 handle_command_reply: tid=0 2024-04-26T01:54:16.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.070+0000 7f2c9aad6b80 1 client.15387 shutdown 2024-04-26T01:54:16.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.070+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/2606791921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855c3f00 msgr2=0x55d2855cb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:16.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.070+0000 7f2c9aad6b80 1 --2- 172.21.15.1:0/2606791921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855c3f00 0x55d2855cb770 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c0128a0 tx=0x7f2c7c012bb0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.070+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/2606791921 shutdown_connections 2024-04-26T01:54:16.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.070+0000 7f2c9aad6b80 1 --2- 172.21.15.1:0/2606791921 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d2855de590 0x55d2855e0980 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.070+0000 7f2c9aad6b80 1 --2- 172.21.15.1:0/2606791921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855ced50 0x55d2855cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.070+0000 7f2c9aad6b80 1 --2- 172.21.15.1:0/2606791921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 0x55d2855ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.070+0000 7f2c9aad6b80 1 --2- 172.21.15.1:0/2606791921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855c3f00 0x55d2855cb770 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.070+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/2606791921 >> 172.21.15.1:0/2606791921 conn(0x55d2854ea0e0 msgr2=0x55d2855b6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:16.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.071+0000 7f2c6f400700 0 client.15387 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:16.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.071+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/2606791921 shutdown_connections 2024-04-26T01:54:16.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.071+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/2606791921 wait complete. 2024-04-26T01:54:16.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.071+0000 7f2c9aad6b80 20 client.15387 trim_cache size 0 max 16384 2024-04-26T01:54:16.097 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] itrash 2024-04-26T01:54:16.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.101+0000 7f2c9aad6b80 1 Processor -- start 2024-04-26T01:54:16.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.101+0000 7f2c9aad6b80 1 -- start start 2024-04-26T01:54:16.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.101+0000 7f2c9aad6b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 0x55d2855c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.101+0000 7f2c9aad6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855c3f00 0x55d2855da7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.101+0000 7f2c9aad6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855ced50 0x55d2855e61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.101+0000 7f2c9aad6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2c88072040 con 0x55d2855ced50 2024-04-26T01:54:16.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.101+0000 7f2c9aad6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8806f870 con 0x55d2855c3f00 2024-04-26T01:54:16.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.101+0000 7f2c9aad6b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8807b0e0 con 0x55d2855c35e0 2024-04-26T01:54:16.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.102+0000 7f2c8d800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855c3f00 0x55d2855da7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:16.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.102+0000 7f2c8d800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855c3f00 0x55d2855da7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:46050/0 (socket says 172.21.15.1:46050) 2024-04-26T01:54:16.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.102+0000 7f2c8d800700 1 -- 172.21.15.1:0/3831411676 learned_addr learned my addr 172.21.15.1:0/3831411676 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:16.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.102+0000 7f2c8ce00700 1 --2- 172.21.15.1:0/3831411676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 0x55d2855c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:16.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.102+0000 7f2c8d800700 1 -- 172.21.15.1:0/3831411676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 msgr2=0x55d2855c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:16.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.102+0000 7f2c8d800700 1 --2- 172.21.15.1:0/3831411676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 0x55d2855c6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.102+0000 7f2c8d800700 1 -- 172.21.15.1:0/3831411676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855ced50 msgr2=0x55d2855e61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:16.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.102+0000 7f2c8d800700 1 --2- 172.21.15.1:0/3831411676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855ced50 0x55d2855e61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.102+0000 7f2c8d800700 1 -- 172.21.15.1:0/3831411676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2855b8760 con 0x55d2855c3f00 2024-04-26T01:54:16.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.102+0000 7f2c8ce00700 1 --2- 172.21.15.1:0/3831411676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 0x55d2855c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:54:16.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.103+0000 7f2c8d800700 1 --2- 172.21.15.1:0/3831411676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855c3f00 0x55d2855da7c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00fe60 tx=0x7f2c7c005a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e521db648f36832 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:16.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.103+0000 7f2c7a000700 1 -- 172.21.15.1:0/3831411676 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c02b070 con 0x55d2855c3f00 2024-04-26T01:54:16.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.103+0000 7f2c7a000700 1 -- 172.21.15.1:0/3831411676 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c7c00fa70 con 0x55d2855c3f00 2024-04-26T01:54:16.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.103+0000 7f2c7a000700 1 -- 172.21.15.1:0/3831411676 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c005070 con 0x55d2855c3f00 2024-04-26T01:54:16.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.104+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/3831411676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855c3f00 msgr2=0x55d2855da7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:16.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.104+0000 7f2c9aad6b80 1 --2- 172.21.15.1:0/3831411676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855c3f00 0x55d2855da7c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00fe60 tx=0x7f2c7c005a40 comp rx=0 tx=0).stop 2024-04-26T01:54:16.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.104+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/3831411676 shutdown_connections 2024-04-26T01:54:16.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.104+0000 7f2c9aad6b80 1 --2- 172.21.15.1:0/3831411676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855ced50 0x55d2855e61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.104+0000 7f2c9aad6b80 1 --2- 172.21.15.1:0/3831411676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 0x55d2855c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.104+0000 7f2c9aad6b80 1 --2- 172.21.15.1:0/3831411676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855c3f00 0x55d2855da7c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.104+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/3831411676 >> 172.21.15.1:0/3831411676 conn(0x55d2854ea0e0 msgr2=0x55d2855c6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:16.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.105+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/3831411676 shutdown_connections 2024-04-26T01:54:16.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.105+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/3831411676 wait complete. 2024-04-26T01:54:16.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.106+0000 7f2c9aad6b80 1 -- start start 2024-04-26T01:54:16.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.106+0000 7f2c9aad6b80 1 Processor -- start 2024-04-26T01:54:16.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.107+0000 7f2c9aad6b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 0x55d2855d4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.107+0000 7f2c9aad6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855c3f00 0x55d2855d4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.107+0000 7f2c9aad6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855ced50 0x55d28563eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.107+0000 7f2c9aad6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8806dd30 con 0x55d2855ced50 2024-04-26T01:54:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.107+0000 7f2c9aad6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8806f1a0 con 0x55d2855c3f00 2024-04-26T01:54:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.107+0000 7f2c9aad6b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8806e400 con 0x55d2855c35e0 2024-04-26T01:54:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.107+0000 7f2c9aad6b80 10 client.0 _ll_register_callbacks cb 0x55d2854a68b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.107+0000 7f2c8ce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855ced50 0x55d28563eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.107+0000 7f2c8d800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855c3f00 0x55d2855d4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.108+0000 7f2c8d800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855c3f00 0x55d2855d4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:46062/0 (socket says 172.21.15.1:46062) 2024-04-26T01:54:16.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.108+0000 7f2c8d800700 1 -- 172.21.15.1:0/4044508357 learned_addr learned my addr 172.21.15.1:0/4044508357 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.108+0000 7f2c8ce00700 1 --2- 172.21.15.1:0/4044508357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 0x55d2855d4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.108+0000 7f2c8ce00700 1 -- 172.21.15.1:0/4044508357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 msgr2=0x55d2855d4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.108+0000 7f2c8ce00700 1 --2- 172.21.15.1:0/4044508357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 0x55d2855d4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.108+0000 7f2c8ce00700 1 -- 172.21.15.1:0/4044508357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855c3f00 msgr2=0x55d2855d4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.108+0000 7f2c8ce00700 1 --2- 172.21.15.1:0/4044508357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855c3f00 0x55d2855d4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.108+0000 7f2c8ce00700 1 -- 172.21.15.1:0/4044508357 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d28563f0f0 con 0x55d2855ced50 2024-04-26T01:54:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.109+0000 7f2c8ce00700 1 --2- 172.21.15.1:0/4044508357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855ced50 0x55d28563eab0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2c74005d90 tx=0x7f2c74009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a083652ba45ec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.109+0000 7f2c6f400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:54:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.109+0000 7f2c6f400700 1 -- 172.21.15.1:0/4044508357 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c740051f0 con 0x55d2855ced50 2024-04-26T01:54:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.110+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/4044508357 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c7c00fa70 con 0x55d2855ced50 2024-04-26T01:54:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.110+0000 7f2c6f400700 1 -- 172.21.15.1:0/4044508357 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f2c7402eae0 con 0x55d2855ced50 2024-04-26T01:54:16.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.110+0000 7f2c6f400700 1 -- 172.21.15.1:0/4044508357 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2c74005390 con 0x55d2855ced50 2024-04-26T01:54:16.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.111+0000 7f2c9aad6b80 10 client.15390 fetch_fsmap learned FSMap version 30 2024-04-26T01:54:16.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.111+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/4044508357 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d285646410 con 0x55d2855ced50 2024-04-26T01:54:16.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.111+0000 7f2c6f400700 1 -- 172.21.15.1:0/4044508357 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c7c00fa70 con 0x55d2855ced50 2024-04-26T01:54:16.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.111+0000 7f2c6f400700 1 -- 172.21.15.1:0/4044508357 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f2c7402ece0 con 0x55d2855ced50 2024-04-26T01:54:16.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.111+0000 7f2c9aad6b80 10 client.15390 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:54:16.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.111+0000 7f2c9aad6b80 10 client.15390 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:54:16.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.112+0000 7f2c9aad6b80 20 client.15390 populate_metadata read hostname 'smithi001' 2024-04-26T01:54:16.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.112+0000 7f2c9aad6b80 1 --2- 172.21.15.1:0/4044508357 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d285646980 0x55d285648da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.112+0000 7f2c9aad6b80 4 client.15390 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-04-26T01:54:16.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.112+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/4044508357 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x55d2855158b0 con 0x55d285646980 2024-04-26T01:54:16.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.112+0000 7f2c8d800700 1 --2- 172.21.15.1:0/4044508357 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d285646980 0x55d285648da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:16.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.113+0000 7f2c6f400700 10 client.15390 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:16.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:16.114+0000 7f2c8d800700 1 --2- 172.21.15.1:0/4044508357 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d285646980 0x55d285648da0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e62ca87d6369e9 server_cookie=e9b31339309d7b54 in_seq=0 out_seq=0 2024-04-26T01:54:16.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:16 smithi173 ceph-mon[32553]: pgmap v82: 97 pgs: 97 active+clean; 33 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.3 MiB/s wr, 225 op/s 2024-04-26T01:54:16.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:16 smithi001 ceph-mon[27822]: pgmap v82: 97 pgs: 97 active+clean; 33 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.3 MiB/s wr, 225 op/s 2024-04-26T01:54:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:16 smithi097 ceph-mon[32853]: pgmap v82: 97 pgs: 97 active+clean; 33 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.3 MiB/s wr, 225 op/s 2024-04-26T01:54:18.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:18 smithi173 ceph-mon[32553]: pgmap v83: 97 pgs: 97 active+clean; 29 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 1.3 MiB/s wr, 209 op/s 2024-04-26T01:54:18.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.348+0000 7f2c6f400700 1 -- 172.21.15.1:0/4044508357 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55d2855158b0 con 0x55d285646980 2024-04-26T01:54:18.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.348+0000 7f2c6f400700 10 client.15390 handle_command_reply: tid=0 2024-04-26T01:54:18.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.348+0000 7f2c9aad6b80 1 client.15390 shutdown 2024-04-26T01:54:18.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.349+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/4044508357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855ced50 msgr2=0x55d28563eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:18.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.349+0000 7f2c9aad6b80 1 --2- 172.21.15.1:0/4044508357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855ced50 0x55d28563eab0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2c74005d90 tx=0x7f2c74009710 comp rx=0 tx=0).stop 2024-04-26T01:54:18.348 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:18 smithi001 ceph-mon[27822]: pgmap v83: 97 pgs: 97 active+clean; 29 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 1.3 MiB/s wr, 209 op/s 2024-04-26T01:54:18.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.349+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/4044508357 shutdown_connections 2024-04-26T01:54:18.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.349+0000 7f2c9aad6b80 1 --2- 172.21.15.1:0/4044508357 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d285646980 0x55d285648da0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:18.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.349+0000 7f2c9aad6b80 1 --2- 172.21.15.1:0/4044508357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2855ced50 0x55d28563eab0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:18.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.349+0000 7f2c9aad6b80 1 --2- 172.21.15.1:0/4044508357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d2855c35e0 0x55d2855d4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:18.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.349+0000 7f2c9aad6b80 1 --2- 172.21.15.1:0/4044508357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2855c3f00 0x55d2855d4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:18.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.349+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/4044508357 >> 172.21.15.1:0/4044508357 conn(0x55d2854ea0e0 msgr2=0x55d2855cd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:18.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.349+0000 7f2c6f400700 0 client.15390 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:18.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.350+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/4044508357 shutdown_connections 2024-04-26T01:54:18.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.350+0000 7f2c9aad6b80 1 -- 172.21.15.1:0/4044508357 wait complete. 2024-04-26T01:54:18.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.350+0000 7f2c9aad6b80 20 client.15390 trim_cache size 0 max 16384 2024-04-26T01:54:18.350 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T01:54:18.350 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 30, 2024-04-26T01:54:18.350 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24518, 2024-04-26T01:54:18.350 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 6, 2024-04-26T01:54:18.350 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-04-26T01:54:18.350 INFO:teuthology.orchestra.run.smithi001.stdout: "79040149": { 2024-04-26T01:54:18.350 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 6, 2024-04-26T01:54:18.350 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-04-26T01:54:18.350 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:54:18.350 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-04-26T01:54:18.351 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:54:18.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T01:54:18.351 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-04-26T01:54:18.351 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 90.0, 2024-04-26T01:54:18.351 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-04-26T01:54:18.351 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-04-26T01:54:18.351 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-04-26T01:54:18.351 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:54:18.351 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-04-26T01:54:18.351 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:54:18.351 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:54:18.351 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:54:18.351 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:54:18.351 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:54:18.351 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:54:18.351 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T01:54:18.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.351+0000 7f2c90200700 1 -- 172.21.15.1:0/1701581890 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2c700a3ed0 msgr2=0x7f2c700a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:18.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.351+0000 7f2c90200700 1 --2- 172.21.15.1:0/1701581890 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2c700a3ed0 0x7f2c700a6390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2c8806c630 tx=0x7f2c80009040 comp rx=0 tx=0).stop 2024-04-26T01:54:18.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.351+0000 7f2c90200700 1 -- 172.21.15.1:0/1701581890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c88109fc0 msgr2=0x7f2c8819faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:18.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.351+0000 7f2c90200700 1 --2- 172.21.15.1:0/1701581890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c88109fc0 0x7f2c8819faf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000b0a0 tx=0x7f2c80004d00 comp rx=0 tx=0).stop 2024-04-26T01:54:18.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.352+0000 7f2c90200700 1 -- 172.21.15.1:0/1701581890 shutdown_connections 2024-04-26T01:54:18.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.352+0000 7f2c90200700 1 --2- 172.21.15.1:0/1701581890 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2c700a3ed0 0x7f2c700a6390 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:18.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.352+0000 7f2c90200700 1 --2- 172.21.15.1:0/1701581890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c88109fc0 0x7f2c8819faf0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:18.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.352+0000 7f2c90200700 1 --2- 172.21.15.1:0/1701581890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c88104ee0 0x7f2c8819b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:18.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.352+0000 7f2c90200700 1 --2- 172.21.15.1:0/1701581890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c88105890 0x7f2c8819b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:18.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.352+0000 7f2c90200700 1 -- 172.21.15.1:0/1701581890 >> 172.21.15.1:0/1701581890 conn(0x7f2c88100770 msgr2=0x7f2c88101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:18.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.352+0000 7f2c90200700 1 -- 172.21.15.1:0/1701581890 shutdown_connections 2024-04-26T01:54:18.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:18.352+0000 7f2c90200700 1 -- 172.21.15.1:0/1701581890 wait complete. 2024-04-26T01:54:18.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:18 smithi097 ceph-mon[32853]: pgmap v83: 97 pgs: 97 active+clean; 29 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 1.3 MiB/s wr, 209 op/s 2024-04-26T01:54:19.263 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '79040149', quiesce took 10.5 seconds. Will release after: 30.0 2024-04-26T01:54:20.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:20 smithi173 ceph-mon[32553]: pgmap v84: 97 pgs: 97 active+clean; 29 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 1.2 MiB/s wr, 202 op/s 2024-04-26T01:54:20.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:20 smithi001 ceph-mon[27822]: pgmap v84: 97 pgs: 97 active+clean; 29 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 1.2 MiB/s wr, 202 op/s 2024-04-26T01:54:20.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:20 smithi097 ceph-mon[32853]: pgmap v84: 97 pgs: 97 active+clean; 29 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 1.2 MiB/s wr, 202 op/s 2024-04-26T01:54:22.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:22 smithi001 ceph-mon[27822]: pgmap v85: 97 pgs: 97 active+clean; 32 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1.4 MiB/s wr, 231 op/s 2024-04-26T01:54:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:22 smithi097 ceph-mon[32853]: pgmap v85: 97 pgs: 97 active+clean; 32 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1.4 MiB/s wr, 231 op/s 2024-04-26T01:54:22.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:22 smithi173 ceph-mon[32553]: pgmap v85: 97 pgs: 97 active+clean; 32 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1.4 MiB/s wr, 231 op/s 2024-04-26T01:54:24.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:24 smithi001 ceph-mon[27822]: pgmap v86: 97 pgs: 97 active+clean; 32 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 311 KiB/s wr, 73 op/s 2024-04-26T01:54:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:24 smithi097 ceph-mon[32853]: pgmap v86: 97 pgs: 97 active+clean; 32 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 311 KiB/s wr, 73 op/s 2024-04-26T01:54:24.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:24 smithi173 ceph-mon[32553]: pgmap v86: 97 pgs: 97 active+clean; 32 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 311 KiB/s wr, 73 op/s 2024-04-26T01:54:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.208+0000 7f0e60c00640 1 -- 172.21.15.1:0/2229522487 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e5c1021a0 msgr2=0x7f0e5c102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.208+0000 7f0e60c00640 1 --2- 172.21.15.1:0/2229522487 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e5c1021a0 0x7f0e5c102580 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c009900 tx=0x7f0e4c02a3f0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.208+0000 7f0e60c00640 1 -- 172.21.15.1:0/2229522487 shutdown_connections 2024-04-26T01:54:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.208+0000 7f0e60c00640 1 --2- 172.21.15.1:0/2229522487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e5c107270 0x7f0e5c10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.208+0000 7f0e60c00640 1 --2- 172.21.15.1:0/2229522487 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e5c102b50 0x7f0e5c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.208+0000 7f0e60c00640 1 --2- 172.21.15.1:0/2229522487 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e5c1021a0 0x7f0e5c102580 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.208+0000 7f0e60c00640 1 -- 172.21.15.1:0/2229522487 >> 172.21.15.1:0/2229522487 conn(0x7f0e5c0fd9e0 msgr2=0x7f0e5c0ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.208+0000 7f0e60c00640 1 -- 172.21.15.1:0/2229522487 shutdown_connections 2024-04-26T01:54:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.208+0000 7f0e60c00640 1 -- 172.21.15.1:0/2229522487 wait complete. 2024-04-26T01:54:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.209+0000 7f0e60c00640 1 Processor -- start 2024-04-26T01:54:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.210+0000 7f0e60c00640 1 -- start start 2024-04-26T01:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.210+0000 7f0e60c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e5c1021a0 0x7f0e5c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.210+0000 7f0e60c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e5c107270 0x7f0e5c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.210+0000 7f0e60c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e5c19a820 0x7f0e5c19acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.210+0000 7f0e60c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c112220 con 0x7f0e5c19a820 2024-04-26T01:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.210+0000 7f0e60c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1120a0 con 0x7f0e5c1021a0 2024-04-26T01:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.210+0000 7f0e60c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1123a0 con 0x7f0e5c107270 2024-04-26T01:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.210+0000 7f0e59e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e5c1021a0 0x7f0e5c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.210+0000 7f0e59400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e5c107270 0x7f0e5c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.210+0000 7f0e5a800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e5c19a820 0x7f0e5c19acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.210+0000 7f0e59e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e5c1021a0 0x7f0e5c1a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:60118/0 (socket says 172.21.15.1:60118) 2024-04-26T01:54:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.210+0000 7f0e59e00640 1 -- 172.21.15.1:0/3429692130 learned_addr learned my addr 172.21.15.1:0/3429692130 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.211+0000 7f0e59e00640 1 -- 172.21.15.1:0/3429692130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e5c107270 msgr2=0x7f0e5c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.211+0000 7f0e59e00640 1 --2- 172.21.15.1:0/3429692130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e5c107270 0x7f0e5c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.211+0000 7f0e59e00640 1 -- 172.21.15.1:0/3429692130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e5c19a820 msgr2=0x7f0e5c19acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.211+0000 7f0e59e00640 1 --2- 172.21.15.1:0/3429692130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e5c19a820 0x7f0e5c19acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.211+0000 7f0e59e00640 1 -- 172.21.15.1:0/3429692130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5c19b3b0 con 0x7f0e5c1021a0 2024-04-26T01:54:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.211+0000 7f0e59e00640 1 --2- 172.21.15.1:0/3429692130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e5c1021a0 0x7f0e5c1a06b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c031440 tx=0x7f0e4c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.211+0000 7f0e52a00640 1 -- 172.21.15.1:0/3429692130 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c03f070 con 0x7f0e5c1021a0 2024-04-26T01:54:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.212+0000 7f0e52a00640 1 -- 172.21.15.1:0/3429692130 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e4c031b50 con 0x7f0e5c1021a0 2024-04-26T01:54:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.212+0000 7f0e52a00640 1 -- 172.21.15.1:0/3429692130 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c035030 con 0x7f0e5c1021a0 2024-04-26T01:54:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.212+0000 7f0e60c00640 1 -- 172.21.15.1:0/3429692130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e5c19b640 con 0x7f0e5c1021a0 2024-04-26T01:54:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.212+0000 7f0e60c00640 1 -- 172.21.15.1:0/3429692130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e5c1a77a0 con 0x7f0e5c1021a0 2024-04-26T01:54:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.215+0000 7f0e63bd0740 1 Processor -- start 2024-04-26T01:54:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.215+0000 7f0e63bd0740 1 -- start start 2024-04-26T01:54:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.216+0000 7f0e63bd0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0ac2820 0x560ab0ac2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.216+0000 7f0e63bd0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0ac3140 0x560ab0acd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.216+0000 7f0e63bd0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0acda00 0x560ab0acfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.216+0000 7f0e63bd0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560ab0a28900 con 0x560ab0ac3140 2024-04-26T01:54:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.216+0000 7f0e63bd0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560ab09ceef0 con 0x560ab0acda00 2024-04-26T01:54:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.216+0000 7f0e63bd0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560ab0a248c0 con 0x560ab0ac2820 2024-04-26T01:54:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.216+0000 7f0e52a00640 1 -- 172.21.15.1:0/3429692130 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f0e4c042070 con 0x7f0e5c1021a0 2024-04-26T01:54:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.216+0000 7f0e5a800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0ac2820 0x560ab0ac2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.216+0000 7f0e5a800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0ac2820 0x560ab0ac2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:54434/0 (socket says 172.21.15.1:54434) 2024-04-26T01:54:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.216+0000 7f0e5a800640 1 -- 172.21.15.1:0/2595815719 learned_addr learned my addr 172.21.15.1:0/2595815719 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.216+0000 7f0e59400640 1 --2- 172.21.15.1:0/2595815719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0acda00 0x560ab0acfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.216+0000 7f0e5a800640 1 -- 172.21.15.1:0/2595815719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0acda00 msgr2=0x560ab0acfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.216+0000 7f0e5a800640 1 --2- 172.21.15.1:0/2595815719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0acda00 0x560ab0acfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.216+0000 7f0e5a800640 1 -- 172.21.15.1:0/2595815719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0ac3140 msgr2=0x560ab0acd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.217+0000 7f0e59e00640 1 --2- 172.21.15.1:0/2595815719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0ac3140 0x560ab0acd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.217+0000 7f0e5a800640 1 --2- 172.21.15.1:0/2595815719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0ac3140 0x560ab0acd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.217+0000 7f0e5a800640 1 -- 172.21.15.1:0/2595815719 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ab073b210 con 0x560ab0ac2820 2024-04-26T01:54:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.217+0000 7f0e59e00640 1 --2- 172.21.15.1:0/2595815719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0ac3140 0x560ab0acd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:54:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.217+0000 7f0e5a800640 1 --2- 172.21.15.1:0/2595815719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0ac2820 0x560ab0ac2c00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009f60 tx=0x7f0e54009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f1c8478160a6316 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.217+0000 7f0e47000640 1 -- 172.21.15.1:0/2595815719 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5401e070 con 0x560ab0ac2820 2024-04-26T01:54:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.217+0000 7f0e47000640 1 -- 172.21.15.1:0/2595815719 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e54003c30 con 0x560ab0ac2820 2024-04-26T01:54:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.217+0000 7f0e47000640 1 -- 172.21.15.1:0/2595815719 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5400f880 con 0x560ab0ac2820 2024-04-26T01:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.218+0000 7f0e63bd0740 1 -- 172.21.15.1:0/2595815719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0ac2820 msgr2=0x560ab0ac2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.218+0000 7f0e63bd0740 1 --2- 172.21.15.1:0/2595815719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0ac2820 0x560ab0ac2c00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009f60 tx=0x7f0e54009f90 comp rx=0 tx=0).stop 2024-04-26T01:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.218+0000 7f0e63bd0740 1 -- 172.21.15.1:0/2595815719 shutdown_connections 2024-04-26T01:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.218+0000 7f0e63bd0740 1 --2- 172.21.15.1:0/2595815719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0acda00 0x560ab0acfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.218+0000 7f0e63bd0740 1 --2- 172.21.15.1:0/2595815719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0ac3140 0x560ab0acd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.218+0000 7f0e63bd0740 1 --2- 172.21.15.1:0/2595815719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0ac2820 0x560ab0ac2c00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.218+0000 7f0e63bd0740 1 -- 172.21.15.1:0/2595815719 >> 172.21.15.1:0/2595815719 conn(0x560ab0abed30 msgr2=0x560ab0abf110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.219+0000 7f0e63bd0740 1 -- 172.21.15.1:0/2595815719 shutdown_connections 2024-04-26T01:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.219+0000 7f0e63bd0740 1 -- 172.21.15.1:0/2595815719 wait complete. 2024-04-26T01:54:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.219+0000 7f0e63bd0740 1 -- start start 2024-04-26T01:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.219+0000 7f0e63bd0740 1 Processor -- start 2024-04-26T01:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.220+0000 7f0e52a00640 1 --2- 172.21.15.1:0/3429692130 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0e400a3cf0 0x7f0e400a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.220+0000 7f0e63bd0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0ac2820 0x560ab0aca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.220+0000 7f0e63bd0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0ac3140 0x560ab0acaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.220+0000 7f0e63bd0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0acda00 0x560ab0acb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.220+0000 7f0e63bd0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560ab0ad0630 con 0x560ab0ac2820 2024-04-26T01:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.220+0000 7f0e63bd0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560ab09e2310 con 0x560ab0ac3140 2024-04-26T01:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.220+0000 7f0e63bd0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560ab0ad07b0 con 0x560ab0acda00 2024-04-26T01:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.220+0000 7f0e63bd0740 10 client.0 _ll_register_callbacks cb 0x560ab09b0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.220+0000 7f0e5a800640 1 --2- 172.21.15.1:0/3429692130 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0e400a3cf0 0x7f0e400a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.220+0000 7f0e52a00640 1 -- 172.21.15.1:0/3429692130 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f0e4c0f0e30 con 0x7f0e5c1021a0 2024-04-26T01:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.220+0000 7f0e5a800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0acda00 0x560ab0acb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.220+0000 7f0e59e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0ac2820 0x560ab0aca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.220+0000 7f0e5a800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0acda00 0x560ab0acb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:54448/0 (socket says 172.21.15.1:54448) 2024-04-26T01:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.220+0000 7f0e5a800640 1 -- 172.21.15.1:0/815715227 learned_addr learned my addr 172.21.15.1:0/815715227 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.221+0000 7f0e59400640 1 --2- 172.21.15.1:0/815715227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0ac3140 0x560ab0acaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.221+0000 7f0e5a800640 1 -- 172.21.15.1:0/815715227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0ac3140 msgr2=0x560ab0acaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.221+0000 7f0e5a800640 1 --2- 172.21.15.1:0/815715227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0ac3140 0x560ab0acaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.221+0000 7f0e5a800640 1 -- 172.21.15.1:0/815715227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0ac2820 msgr2=0x560ab0aca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.221+0000 7f0e5a800640 1 --2- 172.21.15.1:0/815715227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0ac2820 0x560ab0aca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.221+0000 7f0e5a800640 1 -- 172.21.15.1:0/815715227 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ab0807620 con 0x560ab0acda00 2024-04-26T01:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.221+0000 7f0e59e00640 1 --2- 172.21.15.1:0/815715227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0ac2820 0x560ab0aca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.221+0000 7f0e59400640 1 --2- 172.21.15.1:0/815715227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0ac3140 0x560ab0acaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.221+0000 7f0e5a800640 1 --2- 172.21.15.1:0/3429692130 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0e400a3cf0 0x7f0e400a61b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0e54004880 tx=0x7f0e54000f40 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.222+0000 7f0e5a800640 1 --2- 172.21.15.1:0/815715227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0acda00 0x560ab0acb3f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0e54004c20 tx=0x7f0e54005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64d7db239b8a3272 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.222+0000 7f0e3b400640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:54:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.222+0000 7f0e3b400640 1 -- 172.21.15.1:0/815715227 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54037060 con 0x560ab0acda00 2024-04-26T01:54:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.222+0000 7f0e3b400640 1 -- 172.21.15.1:0/815715227 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f0e54005240 con 0x560ab0acda00 2024-04-26T01:54:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.222+0000 7f0e63bd0740 1 -- 172.21.15.1:0/815715227 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e54003c30 con 0x560ab0acda00 2024-04-26T01:54:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.222+0000 7f0e3b400640 1 -- 172.21.15.1:0/815715227 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e5401e050 con 0x560ab0acda00 2024-04-26T01:54:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.222+0000 7f0e63bd0740 10 client.25549 fetch_fsmap learned FSMap version 30 2024-04-26T01:54:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.222+0000 7f0e63bd0740 1 -- 172.21.15.1:0/815715227 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560ab0adc850 con 0x560ab0acda00 2024-04-26T01:54:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.222+0000 7f0e3b400640 1 -- 172.21.15.1:0/815715227 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e54003c30 con 0x560ab0acda00 2024-04-26T01:54:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.223+0000 7f0e3b400640 1 -- 172.21.15.1:0/815715227 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f0e5401a540 con 0x560ab0acda00 2024-04-26T01:54:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.223+0000 7f0e63bd0740 10 client.25549 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:54:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.223+0000 7f0e63bd0740 10 client.25549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:54:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.223+0000 7f0e63bd0740 20 client.25549 populate_metadata read hostname 'smithi001' 2024-04-26T01:54:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.223+0000 7f0e63bd0740 1 --2- 172.21.15.1:0/815715227 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x560ab0addc40 0x560ab0ae0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.223+0000 7f0e63bd0740 4 client.25549 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:54:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.223+0000 7f0e63bd0740 1 -- 172.21.15.1:0/815715227 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ab0a3c8b0 con 0x560ab0addc40 2024-04-26T01:54:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.223+0000 7f0e59e00640 1 --2- 172.21.15.1:0/815715227 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x560ab0addc40 0x560ab0ae0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.224+0000 7f0e3b400640 10 client.25549 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.224+0000 7f0e59e00640 1 --2- 172.21.15.1:0/815715227 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x560ab0addc40 0x560ab0ae0050 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=653e6aff34cd1ace server_cookie=cbfc75fea75eb8cb in_seq=0 out_seq=0 2024-04-26T01:54:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.225+0000 7f0e3b400640 1 -- 172.21.15.1:0/815715227 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560ab0a3c8b0 con 0x560ab0addc40 2024-04-26T01:54:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.225+0000 7f0e3b400640 10 client.25549 handle_command_reply: tid=0 2024-04-26T01:54:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.225+0000 7f0e63bd0740 1 client.25549 shutdown 2024-04-26T01:54:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.225+0000 7f0e63bd0740 1 -- 172.21.15.1:0/815715227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0acda00 msgr2=0x560ab0acb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.225+0000 7f0e63bd0740 1 --2- 172.21.15.1:0/815715227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0acda00 0x560ab0acb3f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0e54004c20 tx=0x7f0e54005fd0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.225+0000 7f0e63bd0740 1 -- 172.21.15.1:0/815715227 shutdown_connections 2024-04-26T01:54:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.225+0000 7f0e63bd0740 1 --2- 172.21.15.1:0/815715227 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x560ab0addc40 0x560ab0ae0050 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.225+0000 7f0e63bd0740 1 --2- 172.21.15.1:0/815715227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0acda00 0x560ab0acb3f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.225+0000 7f0e63bd0740 1 --2- 172.21.15.1:0/815715227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0ac3140 0x560ab0acaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.225+0000 7f0e3b400640 0 client.25549 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.225+0000 7f0e63bd0740 1 --2- 172.21.15.1:0/815715227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0ac2820 0x560ab0aca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.225+0000 7f0e63bd0740 1 -- 172.21.15.1:0/815715227 >> 172.21.15.1:0/815715227 conn(0x560ab0abed30 msgr2=0x560ab0ac4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.225+0000 7f0e63bd0740 1 -- 172.21.15.1:0/815715227 shutdown_connections 2024-04-26T01:54:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.225+0000 7f0e63bd0740 1 -- 172.21.15.1:0/815715227 wait complete. 2024-04-26T01:54:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.225+0000 7f0e63bd0740 20 client.25549 trim_cache size 0 max 16384 2024-04-26T01:54:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.243+0000 7f0e63bd0740 1 Processor -- start 2024-04-26T01:54:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.243+0000 7f0e63bd0740 1 -- start start 2024-04-26T01:54:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.243+0000 7f0e63bd0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0ac2820 0x560ab0abe610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.243+0000 7f0e63bd0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0ac3140 0x560ab0ad9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.243+0000 7f0e63bd0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0acda00 0x560ab0ac51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.243+0000 7f0e63bd0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1b2820 con 0x560ab0acda00 2024-04-26T01:54:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.243+0000 7f0e63bd0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1b0090 con 0x560ab0ac2820 2024-04-26T01:54:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.243+0000 7f0e63bd0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1adee0 con 0x560ab0ac3140 2024-04-26T01:54:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.243+0000 7f0e59e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0ac2820 0x560ab0abe610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.243+0000 7f0e5a800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0acda00 0x560ab0ac51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.244+0000 7f0e59e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0ac2820 0x560ab0abe610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:60136/0 (socket says 172.21.15.1:60136) 2024-04-26T01:54:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.244+0000 7f0e59e00640 1 -- 172.21.15.1:0/4162459050 learned_addr learned my addr 172.21.15.1:0/4162459050 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.244+0000 7f0e59400640 1 --2- 172.21.15.1:0/4162459050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0ac3140 0x560ab0ad9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.244+0000 7f0e59e00640 1 -- 172.21.15.1:0/4162459050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0ac3140 msgr2=0x560ab0ad9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.244+0000 7f0e59e00640 1 --2- 172.21.15.1:0/4162459050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0ac3140 0x560ab0ad9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.244+0000 7f0e59e00640 1 -- 172.21.15.1:0/4162459050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0acda00 msgr2=0x560ab0ac51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.244+0000 7f0e59e00640 1 --2- 172.21.15.1:0/4162459050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0acda00 0x560ab0ac51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.244+0000 7f0e59e00640 1 -- 172.21.15.1:0/4162459050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ab0ae06d0 con 0x560ab0ac2820 2024-04-26T01:54:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.244+0000 7f0e59e00640 1 --2- 172.21.15.1:0/4162459050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0ac2820 0x560ab0abe610 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c0bcd00 tx=0x7f0e4c0f3280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8216f389d944133a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.244+0000 7f0e46600640 1 -- 172.21.15.1:0/4162459050 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c049040 con 0x560ab0ac2820 2024-04-26T01:54:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.244+0000 7f0e46600640 1 -- 172.21.15.1:0/4162459050 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e4c0c1120 con 0x560ab0ac2820 2024-04-26T01:54:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.245+0000 7f0e46600640 1 -- 172.21.15.1:0/4162459050 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c0ee7a0 con 0x560ab0ac2820 2024-04-26T01:54:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.245+0000 7f0e63bd0740 1 -- 172.21.15.1:0/4162459050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0ac2820 msgr2=0x560ab0abe610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.245+0000 7f0e63bd0740 1 --2- 172.21.15.1:0/4162459050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0ac2820 0x560ab0abe610 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c0bcd00 tx=0x7f0e4c0f3280 comp rx=0 tx=0).stop 2024-04-26T01:54:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.245+0000 7f0e63bd0740 1 -- 172.21.15.1:0/4162459050 shutdown_connections 2024-04-26T01:54:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.245+0000 7f0e63bd0740 1 --2- 172.21.15.1:0/4162459050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0acda00 0x560ab0ac51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.245+0000 7f0e63bd0740 1 --2- 172.21.15.1:0/4162459050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0ac3140 0x560ab0ad9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.245+0000 7f0e63bd0740 1 --2- 172.21.15.1:0/4162459050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0ac2820 0x560ab0abe610 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.245+0000 7f0e63bd0740 1 -- 172.21.15.1:0/4162459050 >> 172.21.15.1:0/4162459050 conn(0x560ab0abed30 msgr2=0x560ab0ac99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.245+0000 7f0e63bd0740 1 -- 172.21.15.1:0/4162459050 shutdown_connections 2024-04-26T01:54:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.245+0000 7f0e63bd0740 1 -- 172.21.15.1:0/4162459050 wait complete. 2024-04-26T01:54:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.246+0000 7f0e63bd0740 1 -- start start 2024-04-26T01:54:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.246+0000 7f0e63bd0740 1 Processor -- start 2024-04-26T01:54:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.246+0000 7f0e63bd0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0ac2820 0x560ab0ad2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.247+0000 7f0e63bd0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0ac3140 0x560ab0ad2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.247+0000 7f0e63bd0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0acda00 0x560ab0ad3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.247+0000 7f0e63bd0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1ae550 con 0x560ab0ac3140 2024-04-26T01:54:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.247+0000 7f0e63bd0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1af9c0 con 0x560ab0ac2820 2024-04-26T01:54:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.247+0000 7f0e63bd0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1aec20 con 0x560ab0acda00 2024-04-26T01:54:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.247+0000 7f0e63bd0740 10 client.0 _ll_register_callbacks cb 0x7f0e54009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.247+0000 7f0e59400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0ac3140 0x560ab0ad2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.247+0000 7f0e59e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0ac2820 0x560ab0ad2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.247+0000 7f0e59e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0ac2820 0x560ab0ad2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:60150/0 (socket says 172.21.15.1:60150) 2024-04-26T01:54:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.247+0000 7f0e59e00640 1 -- 172.21.15.1:0/568658165 learned_addr learned my addr 172.21.15.1:0/568658165 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.247+0000 7f0e5a800640 1 --2- 172.21.15.1:0/568658165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0acda00 0x560ab0ad3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.247+0000 7f0e59e00640 1 -- 172.21.15.1:0/568658165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0acda00 msgr2=0x560ab0ad3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.247+0000 7f0e59e00640 1 --2- 172.21.15.1:0/568658165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0acda00 0x560ab0ad3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.247+0000 7f0e59e00640 1 -- 172.21.15.1:0/568658165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0ac3140 msgr2=0x560ab0ad2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.247+0000 7f0e59e00640 1 --2- 172.21.15.1:0/568658165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0ac3140 0x560ab0ad2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.247+0000 7f0e59e00640 1 -- 172.21.15.1:0/568658165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ab0b42210 con 0x560ab0ac2820 2024-04-26T01:54:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.248+0000 7f0e59e00640 1 --2- 172.21.15.1:0/568658165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0ac2820 0x560ab0ad2a00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c0f0010 tx=0x7f0e4c02a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b1e54d97c31897e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.248+0000 7f0e3b400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:54:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.248+0000 7f0e3b400640 1 -- 172.21.15.1:0/568658165 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c0db060 con 0x560ab0ac2820 2024-04-26T01:54:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.248+0000 7f0e3b400640 1 -- 172.21.15.1:0/568658165 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f0e4c0ee7a0 con 0x560ab0ac2820 2024-04-26T01:54:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.248+0000 7f0e63bd0740 1 -- 172.21.15.1:0/568658165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e4c0c1120 con 0x560ab0ac2820 2024-04-26T01:54:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.248+0000 7f0e3b400640 1 -- 172.21.15.1:0/568658165 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e4c049040 con 0x560ab0ac2820 2024-04-26T01:54:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.248+0000 7f0e63bd0740 10 client.25538 fetch_fsmap learned FSMap version 30 2024-04-26T01:54:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.248+0000 7f0e63bd0740 1 -- 172.21.15.1:0/568658165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560ab0b43500 con 0x560ab0ac2820 2024-04-26T01:54:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.248+0000 7f0e3b400640 1 -- 172.21.15.1:0/568658165 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e4c0c1120 con 0x560ab0ac2820 2024-04-26T01:54:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.248+0000 7f0e3b400640 1 -- 172.21.15.1:0/568658165 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f0e4c0c5550 con 0x560ab0ac2820 2024-04-26T01:54:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.249+0000 7f0e63bd0740 10 client.25538 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:54:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.249+0000 7f0e63bd0740 10 client.25538 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:54:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.249+0000 7f0e63bd0740 20 client.25538 populate_metadata read hostname 'smithi001' 2024-04-26T01:54:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.249+0000 7f0e63bd0740 1 --2- 172.21.15.1:0/568658165 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x560ab0b43a40 0x560ab0b45e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.249+0000 7f0e63bd0740 4 client.25538 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:54:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.249+0000 7f0e63bd0740 1 -- 172.21.15.1:0/568658165 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560ab0a36320 con 0x560ab0b43a40 2024-04-26T01:54:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.249+0000 7f0e59400640 1 --2- 172.21.15.1:0/568658165 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x560ab0b43a40 0x560ab0b45e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.249+0000 7f0e59400640 1 --2- 172.21.15.1:0/568658165 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x560ab0b43a40 0x560ab0b45e60 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4023a5f4483b1ceb server_cookie=bfe9480b594f87de in_seq=0 out_seq=0 2024-04-26T01:54:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.249+0000 7f0e3b400640 10 client.25538 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.296+0000 7f0e3b400640 1 -- 172.21.15.1:0/568658165 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560ab0a36320 con 0x560ab0b43a40 2024-04-26T01:54:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.296+0000 7f0e3b400640 10 client.25538 handle_command_reply: tid=0 2024-04-26T01:54:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.296+0000 7f0e63bd0740 1 client.25538 shutdown 2024-04-26T01:54:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.296+0000 7f0e63bd0740 1 -- 172.21.15.1:0/568658165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0ac2820 msgr2=0x560ab0ad2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.296+0000 7f0e63bd0740 1 --2- 172.21.15.1:0/568658165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0ac2820 0x560ab0ad2a00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c0f0010 tx=0x7f0e4c02a3f0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.296+0000 7f0e63bd0740 1 -- 172.21.15.1:0/568658165 shutdown_connections 2024-04-26T01:54:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.296+0000 7f0e63bd0740 1 --2- 172.21.15.1:0/568658165 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x560ab0b43a40 0x560ab0b45e60 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.296+0000 7f0e63bd0740 1 --2- 172.21.15.1:0/568658165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560ab0acda00 0x560ab0ad3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.296+0000 7f0e63bd0740 1 --2- 172.21.15.1:0/568658165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab0ac3140 0x560ab0ad2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.296+0000 7f0e63bd0740 1 --2- 172.21.15.1:0/568658165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560ab0ac2820 0x560ab0ad2a00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.296+0000 7f0e3b400640 0 client.25538 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.296+0000 7f0e63bd0740 1 -- 172.21.15.1:0/568658165 >> 172.21.15.1:0/568658165 conn(0x560ab0abed30 msgr2=0x560ab0aca930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.296+0000 7f0e63bd0740 1 -- 172.21.15.1:0/568658165 shutdown_connections 2024-04-26T01:54:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.296+0000 7f0e63bd0740 1 -- 172.21.15.1:0/568658165 wait complete. 2024-04-26T01:54:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.296+0000 7f0e63bd0740 20 client.25538 trim_cache size 0 max 16384 2024-04-26T01:54:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:54:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:54:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:54:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:54:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.297+0000 7f0e60c00640 1 -- 172.21.15.1:0/3429692130 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0e400a3cf0 msgr2=0x7f0e400a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.297+0000 7f0e60c00640 1 --2- 172.21.15.1:0/3429692130 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0e400a3cf0 0x7f0e400a61b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0e54004880 tx=0x7f0e54000f40 comp rx=0 tx=0).stop 2024-04-26T01:54:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.297+0000 7f0e60c00640 1 -- 172.21.15.1:0/3429692130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e5c1021a0 msgr2=0x7f0e5c1a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.297+0000 7f0e60c00640 1 --2- 172.21.15.1:0/3429692130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e5c1021a0 0x7f0e5c1a06b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c031440 tx=0x7f0e4c003b90 comp rx=0 tx=0).stop 2024-04-26T01:54:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.297+0000 7f0e60c00640 1 -- 172.21.15.1:0/3429692130 shutdown_connections 2024-04-26T01:54:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.297+0000 7f0e60c00640 1 --2- 172.21.15.1:0/3429692130 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0e400a3cf0 0x7f0e400a61b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.297+0000 7f0e60c00640 1 --2- 172.21.15.1:0/3429692130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e5c19a820 0x7f0e5c19acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.297+0000 7f0e60c00640 1 --2- 172.21.15.1:0/3429692130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e5c107270 0x7f0e5c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.297+0000 7f0e60c00640 1 --2- 172.21.15.1:0/3429692130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e5c1021a0 0x7f0e5c1a06b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.297+0000 7f0e60c00640 1 -- 172.21.15.1:0/3429692130 >> 172.21.15.1:0/3429692130 conn(0x7f0e5c0fd9e0 msgr2=0x7f0e5c0ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.297+0000 7f0e60c00640 1 -- 172.21.15.1:0/3429692130 shutdown_connections 2024-04-26T01:54:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:26.297+0000 7f0e60c00640 1 -- 172.21.15.1:0/3429692130 wait complete. 2024-04-26T01:54:26.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:26 smithi001 ceph-mon[27822]: pgmap v87: 97 pgs: 97 active+clean; 32 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 315 KiB/s wr, 76 op/s 2024-04-26T01:54:26.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:26 smithi097 ceph-mon[32853]: pgmap v87: 97 pgs: 97 active+clean; 32 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 315 KiB/s wr, 76 op/s 2024-04-26T01:54:26.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:26 smithi173 ceph-mon[32553]: pgmap v87: 97 pgs: 97 active+clean; 32 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 315 KiB/s wr, 76 op/s 2024-04-26T01:54:28.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:28 smithi001 ceph-mon[27822]: pgmap v88: 97 pgs: 97 active+clean; 32 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 215 KiB/s wr, 36 op/s 2024-04-26T01:54:28.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:28 smithi097 ceph-mon[32853]: pgmap v88: 97 pgs: 97 active+clean; 32 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 215 KiB/s wr, 36 op/s 2024-04-26T01:54:28.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:28 smithi173 ceph-mon[32553]: pgmap v88: 97 pgs: 97 active+clean; 32 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 215 KiB/s wr, 36 op/s 2024-04-26T01:54:30.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:30 smithi001 ceph-mon[27822]: pgmap v89: 97 pgs: 97 active+clean; 32 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 211 KiB/s wr, 32 op/s 2024-04-26T01:54:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:30 smithi097 ceph-mon[32853]: pgmap v89: 97 pgs: 97 active+clean; 32 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 211 KiB/s wr, 32 op/s 2024-04-26T01:54:30.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:30 smithi173 ceph-mon[32553]: pgmap v89: 97 pgs: 97 active+clean; 32 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 211 KiB/s wr, 32 op/s 2024-04-26T01:54:32.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:32 smithi001 ceph-mon[27822]: pgmap v90: 97 pgs: 97 active+clean; 29 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 336 KiB/s wr, 70 op/s 2024-04-26T01:54:32.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:32 smithi097 ceph-mon[32853]: pgmap v90: 97 pgs: 97 active+clean; 29 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 336 KiB/s wr, 70 op/s 2024-04-26T01:54:32.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:32 smithi173 ceph-mon[32553]: pgmap v90: 97 pgs: 97 active+clean; 29 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 336 KiB/s wr, 70 op/s 2024-04-26T01:54:34.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:34 smithi001 ceph-mon[27822]: pgmap v91: 97 pgs: 97 active+clean; 29 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 129 KiB/s wr, 41 op/s 2024-04-26T01:54:34.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:34 smithi097 ceph-mon[32853]: pgmap v91: 97 pgs: 97 active+clean; 29 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 129 KiB/s wr, 41 op/s 2024-04-26T01:54:34.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:34 smithi173 ceph-mon[32553]: pgmap v91: 97 pgs: 97 active+clean; 29 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 129 KiB/s wr, 41 op/s 2024-04-26T01:54:36.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:36 smithi001 ceph-mon[27822]: pgmap v92: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 132 KiB/s wr, 45 op/s 2024-04-26T01:54:36.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:36 smithi097 ceph-mon[32853]: pgmap v92: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 132 KiB/s wr, 45 op/s 2024-04-26T01:54:36.569 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:36 smithi173 ceph-mon[32553]: pgmap v92: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 132 KiB/s wr, 45 op/s 2024-04-26T01:54:38.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:38 smithi001 ceph-mon[27822]: pgmap v93: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 128 KiB/s wr, 42 op/s 2024-04-26T01:54:38.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:38 smithi097 ceph-mon[32853]: pgmap v93: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 128 KiB/s wr, 42 op/s 2024-04-26T01:54:38.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:38 smithi173 ceph-mon[32553]: pgmap v93: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 128 KiB/s wr, 42 op/s 2024-04-26T01:54:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.444+0000 7f7c67400640 1 -- 172.21.15.1:0/1941759297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c60101680 msgr2=0x7f7c6010e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.444+0000 7f7c67400640 1 --2- 172.21.15.1:0/1941759297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c60101680 0x7f7c6010e3d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009860 tx=0x7f7c4802a340 comp rx=0 tx=0).stop 2024-04-26T01:54:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.444+0000 7f7c67400640 1 -- 172.21.15.1:0/1941759297 shutdown_connections 2024-04-26T01:54:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.444+0000 7f7c67400640 1 --2- 172.21.15.1:0/1941759297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c6010e910 0x7f7c60110d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.444+0000 7f7c67400640 1 --2- 172.21.15.1:0/1941759297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c60101680 0x7f7c6010e3d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.444+0000 7f7c67400640 1 --2- 172.21.15.1:0/1941759297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c60100d60 0x7f7c60101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.444+0000 7f7c67400640 1 -- 172.21.15.1:0/1941759297 >> 172.21.15.1:0/1941759297 conn(0x7f7c60077f80 msgr2=0x7f7c60078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.444+0000 7f7c67400640 1 -- 172.21.15.1:0/1941759297 shutdown_connections 2024-04-26T01:54:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.445+0000 7f7c67400640 1 -- 172.21.15.1:0/1941759297 wait complete. 2024-04-26T01:54:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.445+0000 7f7c67400640 1 Processor -- start 2024-04-26T01:54:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.446+0000 7f7c67400640 1 -- start start 2024-04-26T01:54:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.446+0000 7f7c67400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c60100d60 0x7f7c601a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.446+0000 7f7c67400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c60101680 0x7f7c601a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.446+0000 7f7c67400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c6010e910 0x7f7c6019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.446+0000 7f7c67400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7c60114410 con 0x7f7c60101680 2024-04-26T01:54:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.446+0000 7f7c67400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7c60114290 con 0x7f7c6010e910 2024-04-26T01:54:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.446+0000 7f7c67400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7c60114590 con 0x7f7c60100d60 2024-04-26T01:54:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.446+0000 7f7c64a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c60100d60 0x7f7c601a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.446+0000 7f7c57e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c60101680 0x7f7c601a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.446+0000 7f7c64a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c60100d60 0x7f7c601a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:58754/0 (socket says 172.21.15.1:58754) 2024-04-26T01:54:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.446+0000 7f7c64a00640 1 -- 172.21.15.1:0/2582577026 learned_addr learned my addr 172.21.15.1:0/2582577026 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.446+0000 7f7c65400640 1 --2- 172.21.15.1:0/2582577026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c6010e910 0x7f7c6019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.447+0000 7f7c64a00640 1 -- 172.21.15.1:0/2582577026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c6010e910 msgr2=0x7f7c6019a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.447+0000 7f7c64a00640 1 --2- 172.21.15.1:0/2582577026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c6010e910 0x7f7c6019a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.447+0000 7f7c64a00640 1 -- 172.21.15.1:0/2582577026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c60101680 msgr2=0x7f7c601a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.447+0000 7f7c64a00640 1 --2- 172.21.15.1:0/2582577026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c60101680 0x7f7c601a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.447+0000 7f7c64a00640 1 -- 172.21.15.1:0/2582577026 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6019b000 con 0x7f7c60100d60 2024-04-26T01:54:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.447+0000 7f7c64a00640 1 --2- 172.21.15.1:0/2582577026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c60100d60 0x7f7c601a06c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000e9f0 tx=0x7f7c5000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.447+0000 7f7c5e000640 1 -- 172.21.15.1:0/2582577026 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5000cd80 con 0x7f7c60100d60 2024-04-26T01:54:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.448+0000 7f7c67400640 1 -- 172.21.15.1:0/2582577026 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c6019b2f0 con 0x7f7c60100d60 2024-04-26T01:54:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.448+0000 7f7c5e000640 1 -- 172.21.15.1:0/2582577026 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c500108e0 con 0x7f7c60100d60 2024-04-26T01:54:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.448+0000 7f7c5e000640 1 -- 172.21.15.1:0/2582577026 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50018980 con 0x7f7c60100d60 2024-04-26T01:54:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.448+0000 7f7c67400640 1 -- 172.21.15.1:0/2582577026 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c601a7510 con 0x7f7c60100d60 2024-04-26T01:54:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.451+0000 7f7c6a3a9740 1 Processor -- start 2024-04-26T01:54:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.451+0000 7f7c6a3a9740 1 -- start start 2024-04-26T01:54:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.451+0000 7f7c6a3a9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3f9820 0x55e5cb3f9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.451+0000 7f7c5e000640 1 -- 172.21.15.1:0/2582577026 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f7c50010430 con 0x7f7c60100d60 2024-04-26T01:54:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.451+0000 7f7c6a3a9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb3fa140 0x55e5cb4044c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.451+0000 7f7c6a3a9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e5cb404a00 0x55e5cb406df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.451+0000 7f7c6a3a9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e5cb35f900 con 0x55e5cb3fa140 2024-04-26T01:54:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.451+0000 7f7c6a3a9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e5cb305ef0 con 0x55e5cb404a00 2024-04-26T01:54:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.451+0000 7f7c6a3a9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e5cb35b8c0 con 0x55e5cb3f9820 2024-04-26T01:54:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.452+0000 7f7c65400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3f9820 0x55e5cb3f9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.452+0000 7f7c64a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb3fa140 0x55e5cb4044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.452+0000 7f7c65400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3f9820 0x55e5cb3f9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:58756/0 (socket says 172.21.15.1:58756) 2024-04-26T01:54:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.452+0000 7f7c65400640 1 -- 172.21.15.1:0/1491361427 learned_addr learned my addr 172.21.15.1:0/1491361427 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.452+0000 7f7c57e00640 1 --2- 172.21.15.1:0/1491361427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e5cb404a00 0x55e5cb406df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.452+0000 7f7c65400640 1 -- 172.21.15.1:0/1491361427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e5cb404a00 msgr2=0x55e5cb406df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.452+0000 7f7c65400640 1 --2- 172.21.15.1:0/1491361427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e5cb404a00 0x55e5cb406df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.452+0000 7f7c65400640 1 -- 172.21.15.1:0/1491361427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb3fa140 msgr2=0x55e5cb4044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.452+0000 7f7c65400640 1 --2- 172.21.15.1:0/1491361427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb3fa140 0x55e5cb4044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.452+0000 7f7c65400640 1 -- 172.21.15.1:0/1491361427 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5cb072210 con 0x55e5cb3f9820 2024-04-26T01:54:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.452+0000 7f7c65400640 1 --2- 172.21.15.1:0/1491361427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3f9820 0x55e5cb3f9c00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009fc0 tx=0x7f7c580095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5159fa948f3dc678 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.453+0000 7f7c55c00640 1 -- 172.21.15.1:0/1491361427 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58003970 con 0x55e5cb3f9820 2024-04-26T01:54:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.453+0000 7f7c55c00640 1 -- 172.21.15.1:0/1491361427 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c5800d770 con 0x55e5cb3f9820 2024-04-26T01:54:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.453+0000 7f7c55c00640 1 -- 172.21.15.1:0/1491361427 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58004360 con 0x55e5cb3f9820 2024-04-26T01:54:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.453+0000 7f7c5e000640 1 --2- 172.21.15.1:0/2582577026 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7c440a3d10 0x7f7c440a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.454+0000 7f7c5e000640 1 -- 172.21.15.1:0/2582577026 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f7c500d0b40 con 0x7f7c60100d60 2024-04-26T01:54:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.454+0000 7f7c64a00640 1 --2- 172.21.15.1:0/2582577026 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7c440a3d10 0x7f7c440a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.454+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/1491361427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3f9820 msgr2=0x55e5cb3f9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.454+0000 7f7c6a3a9740 1 --2- 172.21.15.1:0/1491361427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3f9820 0x55e5cb3f9c00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009fc0 tx=0x7f7c580095c0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.454+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/1491361427 shutdown_connections 2024-04-26T01:54:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.454+0000 7f7c6a3a9740 1 --2- 172.21.15.1:0/1491361427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e5cb404a00 0x55e5cb406df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.454+0000 7f7c6a3a9740 1 --2- 172.21.15.1:0/1491361427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb3fa140 0x55e5cb4044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.454+0000 7f7c6a3a9740 1 --2- 172.21.15.1:0/1491361427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3f9820 0x55e5cb3f9c00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.454+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/1491361427 >> 172.21.15.1:0/1491361427 conn(0x55e5cb3f5d30 msgr2=0x55e5cb3f6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.454+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/1491361427 shutdown_connections 2024-04-26T01:54:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.454+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/1491361427 wait complete. 2024-04-26T01:54:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.454+0000 7f7c64a00640 1 --2- 172.21.15.1:0/2582577026 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7c440a3d10 0x7f7c440a61d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55e5cb3fadc0 tx=0x7f7c500d45f0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.454+0000 7f7c6a3a9740 1 -- start start 2024-04-26T01:54:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.455+0000 7f7c6a3a9740 1 Processor -- start 2024-04-26T01:54:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.455+0000 7f7c6a3a9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e5cb3f9820 0x55e5cb4019e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.455+0000 7f7c6a3a9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3fa140 0x55e5cb401f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.455+0000 7f7c6a3a9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb404a00 0x55e5cb402460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.455+0000 7f7c6a3a9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e5cb407630 con 0x55e5cb404a00 2024-04-26T01:54:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.455+0000 7f7c6a3a9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e5cb319310 con 0x55e5cb3f9820 2024-04-26T01:54:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.455+0000 7f7c6a3a9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e5cb4077b0 con 0x55e5cb3fa140 2024-04-26T01:54:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.455+0000 7f7c6a3a9740 10 client.0 _ll_register_callbacks cb 0x55e5cb2e7cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.455+0000 7f7c57e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3fa140 0x55e5cb401f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.455+0000 7f7c57e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3fa140 0x55e5cb401f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:58758/0 (socket says 172.21.15.1:58758) 2024-04-26T01:54:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.455+0000 7f7c57e00640 1 -- 172.21.15.1:0/3625028999 learned_addr learned my addr 172.21.15.1:0/3625028999 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.456+0000 7f7c65400640 1 --2- 172.21.15.1:0/3625028999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb404a00 0x55e5cb402460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.456+0000 7f7c65400640 1 --2- 172.21.15.1:0/3625028999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e5cb3f9820 0x55e5cb4019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.456+0000 7f7c57e00640 1 -- 172.21.15.1:0/3625028999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e5cb3f9820 msgr2=0x55e5cb4019e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.456+0000 7f7c57e00640 1 --2- 172.21.15.1:0/3625028999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e5cb3f9820 0x55e5cb4019e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.456+0000 7f7c57e00640 1 -- 172.21.15.1:0/3625028999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb404a00 msgr2=0x55e5cb402460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.456+0000 7f7c57e00640 1 --2- 172.21.15.1:0/3625028999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb404a00 0x55e5cb402460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.456+0000 7f7c57e00640 1 -- 172.21.15.1:0/3625028999 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5cb13e620 con 0x55e5cb3fa140 2024-04-26T01:54:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.456+0000 7f7c65400640 1 --2- 172.21.15.1:0/3625028999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e5cb3f9820 0x55e5cb4019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:54:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.456+0000 7f7c57e00640 1 --2- 172.21.15.1:0/3625028999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3fa140 0x55e5cb401f20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009780 tx=0x7f7c48030d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12bb589b487f8451 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.456+0000 7f7c4e000640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:54:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.457+0000 7f7c4e000640 1 -- 172.21.15.1:0/3625028999 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48037040 con 0x55e5cb3fa140 2024-04-26T01:54:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.457+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/3625028999 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c5800d770 con 0x55e5cb3fa140 2024-04-26T01:54:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.457+0000 7f7c4e000640 1 -- 172.21.15.1:0/3625028999 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f7c48004250 con 0x55e5cb3fa140 2024-04-26T01:54:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.457+0000 7f7c4e000640 1 -- 172.21.15.1:0/3625028999 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7c4804d050 con 0x55e5cb3fa140 2024-04-26T01:54:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.457+0000 7f7c6a3a9740 10 client.25567 fetch_fsmap learned FSMap version 30 2024-04-26T01:54:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.457+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/3625028999 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e5cb4138e0 con 0x55e5cb3fa140 2024-04-26T01:54:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.457+0000 7f7c4e000640 1 -- 172.21.15.1:0/3625028999 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c5800d770 con 0x55e5cb3fa140 2024-04-26T01:54:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.457+0000 7f7c4e000640 1 -- 172.21.15.1:0/3625028999 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7c48003b10 con 0x55e5cb3fa140 2024-04-26T01:54:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.458+0000 7f7c6a3a9740 10 client.25567 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:54:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.458+0000 7f7c6a3a9740 10 client.25567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:54:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.458+0000 7f7c6a3a9740 20 client.25567 populate_metadata read hostname 'smithi001' 2024-04-26T01:54:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.458+0000 7f7c6a3a9740 1 --2- 172.21.15.1:0/3625028999 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e5cb414e60 0x55e5cb417290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.458+0000 7f7c6a3a9740 4 client.25567 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:54:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.458+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/3625028999 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5cb3738b0 con 0x55e5cb414e60 2024-04-26T01:54:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.458+0000 7f7c65400640 1 --2- 172.21.15.1:0/3625028999 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e5cb414e60 0x55e5cb417290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.459+0000 7f7c65400640 1 --2- 172.21.15.1:0/3625028999 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e5cb414e60 0x55e5cb417290 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8b03ffd93b40269 server_cookie=4d3f57d9be450734 in_seq=0 out_seq=0 2024-04-26T01:54:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.459+0000 7f7c4e000640 10 client.25567 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.460+0000 7f7c4e000640 1 -- 172.21.15.1:0/3625028999 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e5cb3738b0 con 0x55e5cb414e60 2024-04-26T01:54:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.460+0000 7f7c4e000640 10 client.25567 handle_command_reply: tid=0 2024-04-26T01:54:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.460+0000 7f7c6a3a9740 1 client.25567 shutdown 2024-04-26T01:54:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.460+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/3625028999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3fa140 msgr2=0x55e5cb401f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.460+0000 7f7c6a3a9740 1 --2- 172.21.15.1:0/3625028999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3fa140 0x55e5cb401f20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009780 tx=0x7f7c48030d80 comp rx=0 tx=0).stop 2024-04-26T01:54:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.460+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/3625028999 shutdown_connections 2024-04-26T01:54:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.460+0000 7f7c6a3a9740 1 --2- 172.21.15.1:0/3625028999 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e5cb414e60 0x55e5cb417290 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.460+0000 7f7c6a3a9740 1 --2- 172.21.15.1:0/3625028999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb404a00 0x55e5cb402460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.460+0000 7f7c6a3a9740 1 --2- 172.21.15.1:0/3625028999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3fa140 0x55e5cb401f20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.460+0000 7f7c6a3a9740 1 --2- 172.21.15.1:0/3625028999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e5cb3f9820 0x55e5cb4019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.460+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/3625028999 >> 172.21.15.1:0/3625028999 conn(0x55e5cb3f5d30 msgr2=0x55e5cb3fbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.460+0000 7f7c4e000640 0 client.25567 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.460+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/3625028999 shutdown_connections 2024-04-26T01:54:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.460+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/3625028999 wait complete. 2024-04-26T01:54:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.460+0000 7f7c6a3a9740 20 client.25567 trim_cache size 0 max 16384 2024-04-26T01:54:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.477+0000 7f7c6a3a9740 1 Processor -- start 2024-04-26T01:54:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.477+0000 7f7c6a3a9740 1 -- start start 2024-04-26T01:54:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.477+0000 7f7c6a3a9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3f9820 0x55e5cb4104f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.477+0000 7f7c6a3a9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb3fa140 0x55e5cb3fc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.477+0000 7f7c6a3a9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e5cb404a00 0x55e5cb41cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.477+0000 7f7c6a3a9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7c601b27b0 con 0x55e5cb3fa140 2024-04-26T01:54:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.477+0000 7f7c6a3a9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7c601b0020 con 0x55e5cb404a00 2024-04-26T01:54:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.477+0000 7f7c6a3a9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7c601ade70 con 0x55e5cb3f9820 2024-04-26T01:54:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.478+0000 7f7c57e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb3fa140 0x55e5cb3fc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.478+0000 7f7c57e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb3fa140 0x55e5cb3fc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58586/0 (socket says 172.21.15.1:58586) 2024-04-26T01:54:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.478+0000 7f7c57e00640 1 -- 172.21.15.1:0/3197426676 learned_addr learned my addr 172.21.15.1:0/3197426676 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.478+0000 7f7c65400640 1 --2- 172.21.15.1:0/3197426676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3f9820 0x55e5cb4104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.478+0000 7f7c65400640 1 --2- 172.21.15.1:0/3197426676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e5cb404a00 0x55e5cb41cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.478+0000 7f7c57e00640 1 -- 172.21.15.1:0/3197426676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3f9820 msgr2=0x55e5cb4104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.478+0000 7f7c57e00640 1 --2- 172.21.15.1:0/3197426676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3f9820 0x55e5cb4104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.478+0000 7f7c57e00640 1 -- 172.21.15.1:0/3197426676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e5cb404a00 msgr2=0x55e5cb41cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.478+0000 7f7c57e00640 1 --2- 172.21.15.1:0/3197426676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e5cb404a00 0x55e5cb41cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.478+0000 7f7c57e00640 1 -- 172.21.15.1:0/3197426676 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5cb3fa8d0 con 0x55e5cb3fa140 2024-04-26T01:54:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.478+0000 7f7c65400640 1 --2- 172.21.15.1:0/3197426676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e5cb404a00 0x55e5cb41cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:54:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.479+0000 7f7c57e00640 1 --2- 172.21.15.1:0/3197426676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb3fa140 0x55e5cb3fc1e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7c4802a860 tx=0x7f7c4802a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93940e09ebfd8fd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.479+0000 7f7c55200640 1 -- 172.21.15.1:0/3197426676 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48056070 con 0x55e5cb3fa140 2024-04-26T01:54:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.479+0000 7f7c55200640 1 -- 172.21.15.1:0/3197426676 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c48009450 con 0x55e5cb3fa140 2024-04-26T01:54:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.479+0000 7f7c55200640 1 -- 172.21.15.1:0/3197426676 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48037070 con 0x55e5cb3fa140 2024-04-26T01:54:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.480+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/3197426676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb3fa140 msgr2=0x55e5cb3fc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.480+0000 7f7c6a3a9740 1 --2- 172.21.15.1:0/3197426676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb3fa140 0x55e5cb3fc1e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7c4802a860 tx=0x7f7c4802a340 comp rx=0 tx=0).stop 2024-04-26T01:54:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.480+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/3197426676 shutdown_connections 2024-04-26T01:54:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.480+0000 7f7c6a3a9740 1 --2- 172.21.15.1:0/3197426676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e5cb404a00 0x55e5cb41cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.480+0000 7f7c6a3a9740 1 --2- 172.21.15.1:0/3197426676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb3fa140 0x55e5cb3fc1e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.480+0000 7f7c6a3a9740 1 --2- 172.21.15.1:0/3197426676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3f9820 0x55e5cb4104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.480+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/3197426676 >> 172.21.15.1:0/3197426676 conn(0x55e5cb3f5d30 msgr2=0x55e5cb3f6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.480+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/3197426676 shutdown_connections 2024-04-26T01:54:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.480+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/3197426676 wait complete. 2024-04-26T01:54:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.480+0000 7f7c6a3a9740 1 -- start start 2024-04-26T01:54:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.480+0000 7f7c6a3a9740 1 Processor -- start 2024-04-26T01:54:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.481+0000 7f7c6a3a9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb3f9820 0x55e5cb40cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.481+0000 7f7c6a3a9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3fa140 0x55e5cb409ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.481+0000 7f7c6a3a9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e5cb404a00 0x55e5cb40a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.481+0000 7f7c6a3a9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7c601ae4e0 con 0x55e5cb3f9820 2024-04-26T01:54:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.481+0000 7f7c6a3a9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7c601af950 con 0x55e5cb404a00 2024-04-26T01:54:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.481+0000 7f7c6a3a9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7c601aebb0 con 0x55e5cb3fa140 2024-04-26T01:54:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.481+0000 7f7c6a3a9740 10 client.0 _ll_register_callbacks cb 0x7f7c48005d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.481+0000 7f7c65400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb3f9820 0x55e5cb40cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.481+0000 7f7c65400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb3f9820 0x55e5cb40cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58598/0 (socket says 172.21.15.1:58598) 2024-04-26T01:54:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.481+0000 7f7c65400640 1 -- 172.21.15.1:0/2166782040 learned_addr learned my addr 172.21.15.1:0/2166782040 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.481+0000 7f7c57e00640 1 --2- 172.21.15.1:0/2166782040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3fa140 0x55e5cb409ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.482+0000 7f7c65400640 1 -- 172.21.15.1:0/2166782040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3fa140 msgr2=0x55e5cb409ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.482+0000 7f7c65400640 1 --2- 172.21.15.1:0/2166782040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3fa140 0x55e5cb409ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.482+0000 7f7c65400640 1 -- 172.21.15.1:0/2166782040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e5cb404a00 msgr2=0x55e5cb40a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.482+0000 7f7c65400640 1 --2- 172.21.15.1:0/2166782040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e5cb404a00 0x55e5cb40a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.482+0000 7f7c65400640 1 -- 172.21.15.1:0/2166782040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5cb40aa70 con 0x55e5cb3f9820 2024-04-26T01:54:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.482+0000 7f7c57e00640 1 --2- 172.21.15.1:0/2166782040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3fa140 0x55e5cb409ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.482+0000 7f7c65400640 1 --2- 172.21.15.1:0/2166782040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb3f9820 0x55e5cb40cac0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009e20 tx=0x7f7c58009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee19c9f585d0c7d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.482+0000 7f7c4e000640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:54:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.482+0000 7f7c4e000640 1 -- 172.21.15.1:0/2166782040 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5800d770 con 0x55e5cb3f9820 2024-04-26T01:54:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.482+0000 7f7c4e000640 1 -- 172.21.15.1:0/2166782040 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f7c5801c070 con 0x55e5cb3f9820 2024-04-26T01:54:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.482+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/2166782040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c48009450 con 0x55e5cb3f9820 2024-04-26T01:54:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.482+0000 7f7c4e000640 1 -- 172.21.15.1:0/2166782040 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7c5800d910 con 0x55e5cb3f9820 2024-04-26T01:54:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.482+0000 7f7c6a3a9740 10 client.15423 fetch_fsmap learned FSMap version 30 2024-04-26T01:54:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.482+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/2166782040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e5cb47b0d0 con 0x55e5cb3f9820 2024-04-26T01:54:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.483+0000 7f7c4e000640 1 -- 172.21.15.1:0/2166782040 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c48009450 con 0x55e5cb3f9820 2024-04-26T01:54:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.483+0000 7f7c4e000640 1 -- 172.21.15.1:0/2166782040 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7c5800b040 con 0x55e5cb3f9820 2024-04-26T01:54:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.483+0000 7f7c6a3a9740 10 client.15423 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:54:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.483+0000 7f7c6a3a9740 10 client.15423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:54:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.483+0000 7f7c6a3a9740 20 client.15423 populate_metadata read hostname 'smithi001' 2024-04-26T01:54:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.483+0000 7f7c6a3a9740 1 --2- 172.21.15.1:0/2166782040 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e5cb47b700 0x55e5cb47db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.483+0000 7f7c6a3a9740 4 client.15423 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:54:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.483+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/2166782040 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e5cb36d320 con 0x55e5cb47b700 2024-04-26T01:54:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.483+0000 7f7c57e00640 1 --2- 172.21.15.1:0/2166782040 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e5cb47b700 0x55e5cb47db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.484+0000 7f7c57e00640 1 --2- 172.21.15.1:0/2166782040 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e5cb47b700 0x55e5cb47db20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e43c3c9ccc953dd server_cookie=21ffcdd33a451e2a in_seq=0 out_seq=0 2024-04-26T01:54:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.484+0000 7f7c4e000640 10 client.15423 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.504+0000 7f7c4e000640 1 -- 172.21.15.1:0/2166782040 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e5cb36d320 con 0x55e5cb47b700 2024-04-26T01:54:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.504+0000 7f7c4e000640 10 client.15423 handle_command_reply: tid=0 2024-04-26T01:54:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.504+0000 7f7c6a3a9740 1 client.15423 shutdown 2024-04-26T01:54:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.504+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/2166782040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb3f9820 msgr2=0x55e5cb40cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.504+0000 7f7c6a3a9740 1 --2- 172.21.15.1:0/2166782040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb3f9820 0x55e5cb40cac0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009e20 tx=0x7f7c58009ab0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.504+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/2166782040 shutdown_connections 2024-04-26T01:54:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.504+0000 7f7c6a3a9740 1 --2- 172.21.15.1:0/2166782040 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e5cb47b700 0x55e5cb47db20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.504+0000 7f7c6a3a9740 1 --2- 172.21.15.1:0/2166782040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e5cb404a00 0x55e5cb40a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.504+0000 7f7c6a3a9740 1 --2- 172.21.15.1:0/2166782040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e5cb3fa140 0x55e5cb409ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.504+0000 7f7c6a3a9740 1 --2- 172.21.15.1:0/2166782040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5cb3f9820 0x55e5cb40cac0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.504+0000 7f7c4e000640 0 client.15423 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.504+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/2166782040 >> 172.21.15.1:0/2166782040 conn(0x55e5cb3f5d30 msgr2=0x55e5cb3f4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.505+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/2166782040 shutdown_connections 2024-04-26T01:54:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.505+0000 7f7c6a3a9740 1 -- 172.21.15.1:0/2166782040 wait complete. 2024-04-26T01:54:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.505+0000 7f7c6a3a9740 20 client.15423 trim_cache size 0 max 16384 2024-04-26T01:54:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:54:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:54:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:54:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:54:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.505+0000 7f7c67400640 1 -- 172.21.15.1:0/2582577026 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7c440a3d10 msgr2=0x7f7c440a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.505+0000 7f7c67400640 1 --2- 172.21.15.1:0/2582577026 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7c440a3d10 0x7f7c440a61d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55e5cb3fadc0 tx=0x7f7c500d45f0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.506+0000 7f7c67400640 1 -- 172.21.15.1:0/2582577026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c60100d60 msgr2=0x7f7c601a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.506+0000 7f7c67400640 1 --2- 172.21.15.1:0/2582577026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c60100d60 0x7f7c601a06c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000e9f0 tx=0x7f7c5000eeb0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.506+0000 7f7c67400640 1 -- 172.21.15.1:0/2582577026 shutdown_connections 2024-04-26T01:54:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.506+0000 7f7c67400640 1 --2- 172.21.15.1:0/2582577026 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7c440a3d10 0x7f7c440a61d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.506+0000 7f7c67400640 1 --2- 172.21.15.1:0/2582577026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c6010e910 0x7f7c6019a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.506+0000 7f7c67400640 1 --2- 172.21.15.1:0/2582577026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c60101680 0x7f7c601a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.506+0000 7f7c67400640 1 --2- 172.21.15.1:0/2582577026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c60100d60 0x7f7c601a06c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.506+0000 7f7c67400640 1 -- 172.21.15.1:0/2582577026 >> 172.21.15.1:0/2582577026 conn(0x7f7c60077f80 msgr2=0x7f7c6010ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.506+0000 7f7c67400640 1 -- 172.21.15.1:0/2582577026 shutdown_connections 2024-04-26T01:54:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:39.506+0000 7f7c67400640 1 -- 172.21.15.1:0/2582577026 wait complete. 2024-04-26T01:54:40.369 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:40 smithi173 ceph-mon[32553]: pgmap v94: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 128 KiB/s wr, 42 op/s 2024-04-26T01:54:40.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:40 smithi001 ceph-mon[27822]: pgmap v94: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 128 KiB/s wr, 42 op/s 2024-04-26T01:54:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:40 smithi097 ceph-mon[32853]: pgmap v94: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 128 KiB/s wr, 42 op/s 2024-04-26T01:54:42.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:42 smithi001 ceph-mon[27822]: pgmap v95: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 137 KiB/s wr, 51 op/s 2024-04-26T01:54:42.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:42 smithi097 ceph-mon[32853]: pgmap v95: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 137 KiB/s wr, 51 op/s 2024-04-26T01:54:42.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:42 smithi173 ceph-mon[32553]: pgmap v95: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 137 KiB/s wr, 51 op/s 2024-04-26T01:54:44.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:44 smithi001 ceph-mon[27822]: pgmap v96: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-04-26T01:54:44.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:44 smithi097 ceph-mon[32853]: pgmap v96: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-04-26T01:54:44.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:44 smithi173 ceph-mon[32553]: pgmap v96: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-04-26T01:54:46.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:46 smithi097 ceph-mon[32853]: pgmap v97: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-26T01:54:46.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:46 smithi173 ceph-mon[32553]: pgmap v97: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-26T01:54:46.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:46 smithi001 ceph-mon[27822]: pgmap v97: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-26T01:54:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:48 smithi097 ceph-mon[32853]: pgmap v98: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-26T01:54:48.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:48 smithi173 ceph-mon[32553]: pgmap v98: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-26T01:54:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:48 smithi001 ceph-mon[27822]: pgmap v98: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-26T01:54:49.265 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:54:49.767 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:50.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:50 smithi001 ceph-mon[27822]: pgmap v99: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-26T01:54:50.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:50 smithi097 ceph-mon[32853]: pgmap v99: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-26T01:54:50.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:50 smithi173 ceph-mon[32553]: pgmap v99: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-26T01:54:51.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:51 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:54:51.542 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:51 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:54:51.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:51 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:54:52.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.083+0000 7f790f600700 1 -- 172.21.15.1:0/2591575439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f790810f440 msgr2=0x7f7908111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:52.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.083+0000 7f790f600700 1 --2- 172.21.15.1:0/2591575439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f790810f440 0x7f7908111830 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f790000b0a0 tx=0x7f790000b3b0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.083+0000 7f790f600700 1 -- 172.21.15.1:0/2591575439 shutdown_connections 2024-04-26T01:54:52.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.083+0000 7f790f600700 1 --2- 172.21.15.1:0/2591575439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f790810f440 0x7f7908111830 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.083+0000 7f790f600700 1 --2- 172.21.15.1:0/2591575439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7908074c70 0x7f7908075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.083+0000 7f790f600700 1 --2- 172.21.15.1:0/2591575439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7908075590 0x7f790810ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.083+0000 7f790f600700 1 -- 172.21.15.1:0/2591575439 >> 172.21.15.1:0/2591575439 conn(0x7f7908079ba0 msgr2=0x7f7908079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:52.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.083+0000 7f790f600700 1 -- 172.21.15.1:0/2591575439 shutdown_connections 2024-04-26T01:54:52.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.083+0000 7f790f600700 1 -- 172.21.15.1:0/2591575439 wait complete. 2024-04-26T01:54:52.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.084+0000 7f790f600700 1 Processor -- start 2024-04-26T01:54:52.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.085+0000 7f790f600700 1 -- start start 2024-04-26T01:54:52.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.085+0000 7f790f600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7908074c70 0x7f7908102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.085+0000 7f790f600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7908075590 0x7f79081030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.085+0000 7f790f600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f790810f440 0x7f79081035e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.085+0000 7f790f600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7908114df0 con 0x7f790810f440 2024-04-26T01:54:52.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.085+0000 7f790f600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7908114c70 con 0x7f7908075590 2024-04-26T01:54:52.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.085+0000 7f790f600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7908114f70 con 0x7f7908074c70 2024-04-26T01:54:52.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.085+0000 7f7907e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7908075590 0x7f79081030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.085+0000 7f7907e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7908075590 0x7f79081030a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:50956/0 (socket says 172.21.15.1:50956) 2024-04-26T01:54:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.085+0000 7f7907e00700 1 -- 172.21.15.1:0/144137040 learned_addr learned my addr 172.21.15.1:0/144137040 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.086+0000 7f790d600700 1 --2- 172.21.15.1:0/144137040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f790810f440 0x7f79081035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.086+0000 7f790cc00700 1 --2- 172.21.15.1:0/144137040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7908074c70 0x7f7908102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.086+0000 7f7907e00700 1 -- 172.21.15.1:0/144137040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7908074c70 msgr2=0x7f7908102b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.086+0000 7f7907e00700 1 --2- 172.21.15.1:0/144137040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7908074c70 0x7f7908102b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.086+0000 7f7907e00700 1 -- 172.21.15.1:0/144137040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f790810f440 msgr2=0x7f79081035e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.086+0000 7f7907e00700 1 --2- 172.21.15.1:0/144137040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f790810f440 0x7f79081035e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.086+0000 7f7907e00700 1 -- 172.21.15.1:0/144137040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790819b0e0 con 0x7f7908075590 2024-04-26T01:54:52.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.087+0000 7f7907e00700 1 --2- 172.21.15.1:0/144137040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7908075590 0x7f79081030a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f78f400e950 tx=0x7f78f400ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:52.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.087+0000 7f7905600700 1 -- 172.21.15.1:0/144137040 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f4004230 con 0x7f7908075590 2024-04-26T01:54:52.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.087+0000 7f7905600700 1 -- 172.21.15.1:0/144137040 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78f40043d0 con 0x7f7908075590 2024-04-26T01:54:52.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.087+0000 7f7905600700 1 -- 172.21.15.1:0/144137040 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f4003d60 con 0x7f7908075590 2024-04-26T01:54:52.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.087+0000 7f790f600700 1 -- 172.21.15.1:0/144137040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790819b3d0 con 0x7f7908075590 2024-04-26T01:54:52.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.087+0000 7f790f600700 1 -- 172.21.15.1:0/144137040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f790819b910 con 0x7f7908075590 2024-04-26T01:54:52.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.089+0000 7f78faa00700 1 -- 172.21.15.1:0/144137040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7908076360 con 0x7f7908075590 2024-04-26T01:54:52.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.092+0000 7f7905600700 1 -- 172.21.15.1:0/144137040 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f78f400c7e0 con 0x7f7908075590 2024-04-26T01:54:52.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.094+0000 7f7905600700 1 --2- 172.21.15.1:0/144137040 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f78f00a3e60 0x7f78f00a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.094+0000 7f7905600700 1 -- 172.21.15.1:0/144137040 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f78f40d0b80 con 0x7f7908075590 2024-04-26T01:54:52.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.095+0000 7f790cc00700 1 --2- 172.21.15.1:0/144137040 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f78f00a3e60 0x7f78f00a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.095+0000 7f7905600700 1 -- 172.21.15.1:0/144137040 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78f409d1c0 con 0x7f7908075590 2024-04-26T01:54:52.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.096+0000 7f790cc00700 1 --2- 172.21.15.1:0/144137040 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f78f00a3e60 0x7f78f00a6320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009c60 tx=0x7f78fc009260 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:52.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:52 smithi001 ceph-mon[27822]: pgmap v100: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-26T01:54:52.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:52 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:54:52.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:52 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:54:52.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:52 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:54:52.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.388+0000 7f78faa00700 1 -- 172.21.15.1:0/144137040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7908063650 con 0x7f7908075590 2024-04-26T01:54:52.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.389+0000 7f7905600700 1 -- 172.21.15.1:0/144137040 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11841 (secure 0 0 0) 0x7f78f409cac0 con 0x7f7908075590 2024-04-26T01:54:52.388 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:54:52.388 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:53:29.647029+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25414":{"gid":25414,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/453427322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":453427322},{"type":"v1","addr":"172.21.15.173:6835","nonce":453427322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24518,24499,24514]},"id":1}]} 2024-04-26T01:54:52.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.393+0000 7f78faa00700 1 -- 172.21.15.1:0/144137040 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f78f00a3e60 msgr2=0x7f78f00a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:52.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.393+0000 7f78faa00700 1 --2- 172.21.15.1:0/144137040 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f78f00a3e60 0x7f78f00a6320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009c60 tx=0x7f78fc009260 comp rx=0 tx=0).stop 2024-04-26T01:54:52.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.393+0000 7f78faa00700 1 -- 172.21.15.1:0/144137040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7908075590 msgr2=0x7f79081030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:52.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.393+0000 7f78faa00700 1 --2- 172.21.15.1:0/144137040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7908075590 0x7f79081030a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f78f400e950 tx=0x7f78f400ec60 comp rx=0 tx=0).stop 2024-04-26T01:54:52.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.394+0000 7f78faa00700 1 -- 172.21.15.1:0/144137040 shutdown_connections 2024-04-26T01:54:52.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.394+0000 7f78faa00700 1 --2- 172.21.15.1:0/144137040 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f78f00a3e60 0x7f78f00a6320 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.394+0000 7f78faa00700 1 --2- 172.21.15.1:0/144137040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f790810f440 0x7f79081035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.394+0000 7f78faa00700 1 --2- 172.21.15.1:0/144137040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7908074c70 0x7f7908102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.394+0000 7f78faa00700 1 --2- 172.21.15.1:0/144137040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7908075590 0x7f79081030a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.394+0000 7f78faa00700 1 -- 172.21.15.1:0/144137040 >> 172.21.15.1:0/144137040 conn(0x7f7908079ba0 msgr2=0x7f79080ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:52.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.394+0000 7f78faa00700 1 -- 172.21.15.1:0/144137040 shutdown_connections 2024-04-26T01:54:52.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:52.394+0000 7f78faa00700 1 -- 172.21.15.1:0/144137040 wait complete. 2024-04-26T01:54:52.394 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 30 2024-04-26T01:54:52.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:52 smithi097 ceph-mon[32853]: pgmap v100: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-26T01:54:52.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:52 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:54:52.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:52 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:54:52.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:52 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:54:52.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:52 smithi173 ceph-mon[32553]: pgmap v100: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-26T01:54:52.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:52 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:54:52.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:52 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:54:52.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:52 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:54:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.651+0000 7f4f13400640 1 -- 172.21.15.1:0/1806759518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f0c1010a0 msgr2=0x7f4f0c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.651+0000 7f4f13400640 1 --2- 172.21.15.1:0/1806759518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f0c1010a0 0x7f4f0c101480 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009900 tx=0x7f4f0002a3f0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.651+0000 7f4f13400640 1 -- 172.21.15.1:0/1806759518 shutdown_connections 2024-04-26T01:54:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.651+0000 7f4f13400640 1 --2- 172.21.15.1:0/1806759518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f0c106170 0x7f4f0c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.651+0000 7f4f13400640 1 --2- 172.21.15.1:0/1806759518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f0c101a50 0x7f4f0c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.651+0000 7f4f13400640 1 --2- 172.21.15.1:0/1806759518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f0c1010a0 0x7f4f0c101480 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.651+0000 7f4f13400640 1 -- 172.21.15.1:0/1806759518 >> 172.21.15.1:0/1806759518 conn(0x7f4f0c077f70 msgr2=0x7f4f0c078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.652+0000 7f4f13400640 1 -- 172.21.15.1:0/1806759518 shutdown_connections 2024-04-26T01:54:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.652+0000 7f4f13400640 1 -- 172.21.15.1:0/1806759518 wait complete. 2024-04-26T01:54:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.653+0000 7f4f13400640 1 Processor -- start 2024-04-26T01:54:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.654+0000 7f4f13400640 1 -- start start 2024-04-26T01:54:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.654+0000 7f4f13400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f0c1010a0 0x7f4f0c107d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.654+0000 7f4f13400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f0c106170 0x7f4f0c1082a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.654+0000 7f4f13400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f0c1087e0 0x7f4f0c1aaf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.654+0000 7f4f13400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c1121f0 con 0x7f4f0c1010a0 2024-04-26T01:54:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.654+0000 7f4f13400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c112070 con 0x7f4f0c106170 2024-04-26T01:54:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.654+0000 7f4f13400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c112370 con 0x7f4f0c1087e0 2024-04-26T01:54:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.655+0000 7f4f0be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f0c106170 0x7f4f0c1082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.655+0000 7f4f0be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f0c106170 0x7f4f0c1082a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:48754/0 (socket says 172.21.15.1:48754) 2024-04-26T01:54:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.655+0000 7f4f0be00640 1 -- 172.21.15.1:0/175006749 learned_addr learned my addr 172.21.15.1:0/175006749 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.655+0000 7f4f10a00640 1 --2- 172.21.15.1:0/175006749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f0c1010a0 0x7f4f0c107d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.655+0000 7f4f11400640 1 --2- 172.21.15.1:0/175006749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f0c1087e0 0x7f4f0c1aaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.656+0000 7f4f11400640 1 -- 172.21.15.1:0/175006749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f0c106170 msgr2=0x7f4f0c1082a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.656+0000 7f4f11400640 1 --2- 172.21.15.1:0/175006749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f0c106170 0x7f4f0c1082a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.656+0000 7f4f11400640 1 -- 172.21.15.1:0/175006749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f0c1010a0 msgr2=0x7f4f0c107d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.656+0000 7f4f11400640 1 --2- 172.21.15.1:0/175006749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f0c1010a0 0x7f4f0c107d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.656+0000 7f4f11400640 1 -- 172.21.15.1:0/175006749 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0c19a720 con 0x7f4f0c1087e0 2024-04-26T01:54:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.656+0000 7f4f10a00640 1 --2- 172.21.15.1:0/175006749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f0c1010a0 0x7f4f0c107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.657+0000 7f4f11400640 1 --2- 172.21.15.1:0/175006749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f0c1087e0 0x7f4f0c1aaf80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400ec80 tx=0x7f4f0400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.657+0000 7f4f09600640 1 -- 172.21.15.1:0/175006749 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f04003b70 con 0x7f4f0c1087e0 2024-04-26T01:54:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.657+0000 7f4f09600640 1 -- 172.21.15.1:0/175006749 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f0400ee90 con 0x7f4f0c1087e0 2024-04-26T01:54:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.657+0000 7f4f09600640 1 -- 172.21.15.1:0/175006749 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f040107a0 con 0x7f4f0c1087e0 2024-04-26T01:54:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.657+0000 7f4f13400640 1 -- 172.21.15.1:0/175006749 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f0c19a9b0 con 0x7f4f0c1087e0 2024-04-26T01:54:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.658+0000 7f4f13400640 1 -- 172.21.15.1:0/175006749 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f0c19aeb0 con 0x7f4f0c1087e0 2024-04-26T01:54:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.662+0000 7f4f09600640 1 -- 172.21.15.1:0/175006749 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f4f04004020 con 0x7f4f0c1087e0 2024-04-26T01:54:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.662+0000 7f4f16379740 1 Processor -- start 2024-04-26T01:54:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.663+0000 7f4f16379740 1 -- start start 2024-04-26T01:54:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.663+0000 7f4f16379740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad47f820 0x5611ad47fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.663+0000 7f4f16379740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad480140 0x5611ad48a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.663+0000 7f4f16379740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad48aa00 0x5611ad48cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.663+0000 7f4f16379740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5611ad3e5900 con 0x5611ad480140 2024-04-26T01:54:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.663+0000 7f4f16379740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5611ad38bef0 con 0x5611ad48aa00 2024-04-26T01:54:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.663+0000 7f4f16379740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5611ad3e18c0 con 0x5611ad47f820 2024-04-26T01:54:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.664+0000 7f4f0be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad47f820 0x5611ad47fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.664+0000 7f4f0be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad47f820 0x5611ad47fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:42218/0 (socket says 172.21.15.1:42218) 2024-04-26T01:54:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.664+0000 7f4f0be00640 1 -- 172.21.15.1:0/1177436159 learned_addr learned my addr 172.21.15.1:0/1177436159 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.664+0000 7f4f0be00640 1 --2- 172.21.15.1:0/1177436159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad48aa00 0x5611ad48cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.664+0000 7f4f10a00640 1 --2- 172.21.15.1:0/1177436159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad480140 0x5611ad48a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.664+0000 7f4f0be00640 1 -- 172.21.15.1:0/1177436159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad47f820 msgr2=0x5611ad47fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.664+0000 7f4f0be00640 1 --2- 172.21.15.1:0/1177436159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad47f820 0x5611ad47fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.664+0000 7f4f0be00640 1 -- 172.21.15.1:0/1177436159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad480140 msgr2=0x5611ad48a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.664+0000 7f4f0be00640 1 --2- 172.21.15.1:0/1177436159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad480140 0x5611ad48a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.664+0000 7f4f0be00640 1 -- 172.21.15.1:0/1177436159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611ad0f8210 con 0x5611ad48aa00 2024-04-26T01:54:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.664+0000 7f4f09600640 1 --2- 172.21.15.1:0/175006749 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f4ef40a3d90 0x7f4ef40a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.665+0000 7f4f0be00640 1 --2- 172.21.15.1:0/1177436159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad48aa00 0x5611ad48cdf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8013880 tx=0x7f4ef8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19a0a6581d8fbb06 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.665+0000 7f4f09600640 1 -- 172.21.15.1:0/175006749 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f4f040d0ca0 con 0x7f4f0c1087e0 2024-04-26T01:54:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.665+0000 7f4f10a00640 1 --2- 172.21.15.1:0/175006749 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f4ef40a3d90 0x7f4ef40a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.665+0000 7f4efdc00640 1 -- 172.21.15.1:0/1177436159 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8013170 con 0x5611ad48aa00 2024-04-26T01:54:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.665+0000 7f4efdc00640 1 -- 172.21.15.1:0/1177436159 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef800fc70 con 0x5611ad48aa00 2024-04-26T01:54:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.665+0000 7f4efdc00640 1 -- 172.21.15.1:0/1177436159 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8010990 con 0x5611ad48aa00 2024-04-26T01:54:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.666+0000 7f4f10a00640 1 --2- 172.21.15.1:0/175006749 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f4ef40a3d90 0x7f4ef40a6250 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4f00031440 tx=0x7f4f00005b50 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.666+0000 7f4f16379740 1 -- 172.21.15.1:0/1177436159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad48aa00 msgr2=0x5611ad48cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.666+0000 7f4f16379740 1 --2- 172.21.15.1:0/1177436159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad48aa00 0x5611ad48cdf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8013880 tx=0x7f4ef8004730 comp rx=0 tx=0).stop 2024-04-26T01:54:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.666+0000 7f4f16379740 1 -- 172.21.15.1:0/1177436159 shutdown_connections 2024-04-26T01:54:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.666+0000 7f4f16379740 1 --2- 172.21.15.1:0/1177436159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad48aa00 0x5611ad48cdf0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.666+0000 7f4f16379740 1 --2- 172.21.15.1:0/1177436159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad480140 0x5611ad48a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.666+0000 7f4f16379740 1 --2- 172.21.15.1:0/1177436159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad47f820 0x5611ad47fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.666+0000 7f4f16379740 1 -- 172.21.15.1:0/1177436159 >> 172.21.15.1:0/1177436159 conn(0x5611ad47bd30 msgr2=0x5611ad47c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.666+0000 7f4f16379740 1 -- 172.21.15.1:0/1177436159 shutdown_connections 2024-04-26T01:54:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.666+0000 7f4f16379740 1 -- 172.21.15.1:0/1177436159 wait complete. 2024-04-26T01:54:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.667+0000 7f4f16379740 1 -- start start 2024-04-26T01:54:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.667+0000 7f4f16379740 1 Processor -- start 2024-04-26T01:54:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.668+0000 7f4f16379740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad47f820 0x5611ad487a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.668+0000 7f4f16379740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad480140 0x5611ad487f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.668+0000 7f4f16379740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad48aa00 0x5611ad4884b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.668+0000 7f4f16379740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5611ad48d630 con 0x5611ad47f820 2024-04-26T01:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.668+0000 7f4f16379740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5611ad39f310 con 0x5611ad48aa00 2024-04-26T01:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.668+0000 7f4f16379740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5611ad48d7b0 con 0x5611ad480140 2024-04-26T01:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.668+0000 7f4f16379740 10 client.0 _ll_register_callbacks cb 0x5611ad36dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.668+0000 7f4f10a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad47f820 0x5611ad487a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.668+0000 7f4f0be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad480140 0x5611ad487f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.668+0000 7f4f10a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad47f820 0x5611ad487a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40012/0 (socket says 172.21.15.1:40012) 2024-04-26T01:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.668+0000 7f4f10a00640 1 -- 172.21.15.1:0/1565996940 learned_addr learned my addr 172.21.15.1:0/1565996940 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.669+0000 7f4f11400640 1 --2- 172.21.15.1:0/1565996940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad48aa00 0x5611ad4884b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.669+0000 7f4f0be00640 1 -- 172.21.15.1:0/1565996940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad48aa00 msgr2=0x5611ad4884b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.669+0000 7f4f0be00640 1 --2- 172.21.15.1:0/1565996940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad48aa00 0x5611ad4884b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.669+0000 7f4f0be00640 1 -- 172.21.15.1:0/1565996940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad47f820 msgr2=0x5611ad487a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.669+0000 7f4f0be00640 1 --2- 172.21.15.1:0/1565996940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad47f820 0x5611ad487a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.669+0000 7f4f0be00640 1 -- 172.21.15.1:0/1565996940 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611ad1c4620 con 0x5611ad480140 2024-04-26T01:54:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.669+0000 7f4f0be00640 1 --2- 172.21.15.1:0/1565996940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad480140 0x5611ad487f70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009570 tx=0x7f4ef8009e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=badc433b3a15ae90 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.669+0000 7f4ef2000640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:54:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.670+0000 7f4ef2000640 1 -- 172.21.15.1:0/1565996940 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800f9d0 con 0x5611ad480140 2024-04-26T01:54:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.670+0000 7f4f16379740 1 -- 172.21.15.1:0/1565996940 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef800fc70 con 0x5611ad480140 2024-04-26T01:54:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.670+0000 7f4ef2000640 1 -- 172.21.15.1:0/1565996940 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f4ef801b070 con 0x5611ad480140 2024-04-26T01:54:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.670+0000 7f4ef2000640 1 -- 172.21.15.1:0/1565996940 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4ef8005020 con 0x5611ad480140 2024-04-26T01:54:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.670+0000 7f4f16379740 10 client.25591 fetch_fsmap learned FSMap version 30 2024-04-26T01:54:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.670+0000 7f4f16379740 1 -- 172.21.15.1:0/1565996940 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5611ad499920 con 0x5611ad480140 2024-04-26T01:54:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.670+0000 7f4ef2000640 1 -- 172.21.15.1:0/1565996940 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef800fc70 con 0x5611ad480140 2024-04-26T01:54:52.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.671+0000 7f4ef2000640 1 -- 172.21.15.1:0/1565996940 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4ef8004bf0 con 0x5611ad480140 2024-04-26T01:54:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.671+0000 7f4f16379740 10 client.25591 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:54:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.671+0000 7f4f16379740 10 client.25591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:54:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.671+0000 7f4f16379740 20 client.25591 populate_metadata read hostname 'smithi001' 2024-04-26T01:54:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.671+0000 7f4f16379740 1 --2- 172.21.15.1:0/1565996940 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5611ad49acd0 0x5611ad49d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.671+0000 7f4f16379740 4 client.25591 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:54:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.671+0000 7f4f16379740 1 -- 172.21.15.1:0/1565996940 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611ad3f98b0 con 0x5611ad49acd0 2024-04-26T01:54:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.671+0000 7f4f10a00640 1 --2- 172.21.15.1:0/1565996940 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5611ad49acd0 0x5611ad49d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.672+0000 7f4f10a00640 1 --2- 172.21.15.1:0/1565996940 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5611ad49acd0 0x5611ad49d0c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a76c8046eb07b99 server_cookie=2c5c8cb3aec13730 in_seq=0 out_seq=0 2024-04-26T01:54:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.672+0000 7f4ef2000640 10 client.25591 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.673+0000 7f4ef2000640 1 -- 172.21.15.1:0/1565996940 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5611ad3f98b0 con 0x5611ad49acd0 2024-04-26T01:54:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.673+0000 7f4ef2000640 10 client.25591 handle_command_reply: tid=0 2024-04-26T01:54:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.673+0000 7f4f16379740 1 client.25591 shutdown 2024-04-26T01:54:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.673+0000 7f4f16379740 1 -- 172.21.15.1:0/1565996940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad480140 msgr2=0x5611ad487f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.673+0000 7f4f16379740 1 --2- 172.21.15.1:0/1565996940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad480140 0x5611ad487f70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009570 tx=0x7f4ef8009e80 comp rx=0 tx=0).stop 2024-04-26T01:54:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.673+0000 7f4f16379740 1 -- 172.21.15.1:0/1565996940 shutdown_connections 2024-04-26T01:54:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.673+0000 7f4f16379740 1 --2- 172.21.15.1:0/1565996940 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5611ad49acd0 0x5611ad49d0c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.673+0000 7f4f16379740 1 --2- 172.21.15.1:0/1565996940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad48aa00 0x5611ad4884b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.673+0000 7f4f16379740 1 --2- 172.21.15.1:0/1565996940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad480140 0x5611ad487f70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.673+0000 7f4f16379740 1 --2- 172.21.15.1:0/1565996940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad47f820 0x5611ad487a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.673+0000 7f4f16379740 1 -- 172.21.15.1:0/1565996940 >> 172.21.15.1:0/1565996940 conn(0x5611ad47bd30 msgr2=0x5611ad481ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.673+0000 7f4ef2000640 0 client.25591 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.673+0000 7f4f16379740 1 -- 172.21.15.1:0/1565996940 shutdown_connections 2024-04-26T01:54:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.673+0000 7f4f16379740 1 -- 172.21.15.1:0/1565996940 wait complete. 2024-04-26T01:54:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.673+0000 7f4f16379740 20 client.25591 trim_cache size 0 max 16384 2024-04-26T01:54:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.691+0000 7f4f16379740 1 Processor -- start 2024-04-26T01:54:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.691+0000 7f4f16379740 1 -- start start 2024-04-26T01:54:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.691+0000 7f4f16379740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad47f820 0x5611ad4964f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.691+0000 7f4f16379740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad480140 0x5611ad4821e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.691+0000 7f4f16379740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad48aa00 0x5611ad47ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.691+0000 7f4f16379740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c1bf400 con 0x5611ad480140 2024-04-26T01:54:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.691+0000 7f4f16379740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c1bcc70 con 0x5611ad47f820 2024-04-26T01:54:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.691+0000 7f4f16379740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c050530 con 0x5611ad48aa00 2024-04-26T01:54:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.692+0000 7f4f0be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad480140 0x5611ad4821e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.692+0000 7f4f11400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad48aa00 0x5611ad47ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.692+0000 7f4f0be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad480140 0x5611ad4821e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40022/0 (socket says 172.21.15.1:40022) 2024-04-26T01:54:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.692+0000 7f4f10a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad47f820 0x5611ad4964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.692+0000 7f4f0be00640 1 -- 172.21.15.1:0/297040263 learned_addr learned my addr 172.21.15.1:0/297040263 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.692+0000 7f4f10a00640 1 -- 172.21.15.1:0/297040263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad48aa00 msgr2=0x5611ad47ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.692+0000 7f4f10a00640 1 --2- 172.21.15.1:0/297040263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad48aa00 0x5611ad47ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.692+0000 7f4f10a00640 1 -- 172.21.15.1:0/297040263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad480140 msgr2=0x5611ad4821e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.692+0000 7f4f10a00640 1 --2- 172.21.15.1:0/297040263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad480140 0x5611ad4821e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.692+0000 7f4f10a00640 1 -- 172.21.15.1:0/297040263 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611ad48d020 con 0x5611ad47f820 2024-04-26T01:54:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.693+0000 7f4f10a00640 1 --2- 172.21.15.1:0/297040263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad47f820 0x5611ad4964f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x5611ad49d6c0 tx=0x7f4f0003e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1509ec75a8ded1b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.693+0000 7f4efd200640 1 -- 172.21.15.1:0/297040263 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00035040 con 0x5611ad47f820 2024-04-26T01:54:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.693+0000 7f4efd200640 1 -- 172.21.15.1:0/297040263 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f000528a0 con 0x5611ad47f820 2024-04-26T01:54:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.693+0000 7f4efd200640 1 -- 172.21.15.1:0/297040263 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00004770 con 0x5611ad47f820 2024-04-26T01:54:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.694+0000 7f4f16379740 1 -- 172.21.15.1:0/297040263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad47f820 msgr2=0x5611ad4964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.694+0000 7f4f16379740 1 --2- 172.21.15.1:0/297040263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad47f820 0x5611ad4964f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x5611ad49d6c0 tx=0x7f4f0003e000 comp rx=0 tx=0).stop 2024-04-26T01:54:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.694+0000 7f4f16379740 1 -- 172.21.15.1:0/297040263 shutdown_connections 2024-04-26T01:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.694+0000 7f4f16379740 1 --2- 172.21.15.1:0/297040263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad48aa00 0x5611ad47ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.694+0000 7f4f16379740 1 --2- 172.21.15.1:0/297040263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad480140 0x5611ad4821e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.694+0000 7f4f16379740 1 --2- 172.21.15.1:0/297040263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad47f820 0x5611ad4964f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.694+0000 7f4f16379740 1 -- 172.21.15.1:0/297040263 >> 172.21.15.1:0/297040263 conn(0x5611ad47bd30 msgr2=0x5611ad47a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.694+0000 7f4f16379740 1 -- 172.21.15.1:0/297040263 shutdown_connections 2024-04-26T01:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.694+0000 7f4f16379740 1 -- 172.21.15.1:0/297040263 wait complete. 2024-04-26T01:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.695+0000 7f4f16379740 1 -- start start 2024-04-26T01:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.695+0000 7f4f16379740 1 Processor -- start 2024-04-26T01:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.695+0000 7f4f16379740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad47f820 0x5611ad490440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.695+0000 7f4f16379740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad480140 0x5611ad490980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.695+0000 7f4f16379740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad48aa00 0x5611ad490ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.695+0000 7f4f16379740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c1a1bf0 con 0x5611ad47f820 2024-04-26T01:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.695+0000 7f4f16379740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c1bc5c0 con 0x5611ad480140 2024-04-26T01:54:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.695+0000 7f4f16379740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c1a22c0 con 0x5611ad48aa00 2024-04-26T01:54:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.695+0000 7f4f16379740 10 client.0 _ll_register_callbacks cb 0x7f4ef8013780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.696+0000 7f4f10a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad47f820 0x5611ad490440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.696+0000 7f4f10a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad47f820 0x5611ad490440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40030/0 (socket says 172.21.15.1:40030) 2024-04-26T01:54:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.696+0000 7f4f10a00640 1 -- 172.21.15.1:0/252977369 learned_addr learned my addr 172.21.15.1:0/252977369 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.696+0000 7f4f0be00640 1 --2- 172.21.15.1:0/252977369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad480140 0x5611ad490980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.696+0000 7f4f11400640 1 --2- 172.21.15.1:0/252977369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad48aa00 0x5611ad490ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.696+0000 7f4f10a00640 1 -- 172.21.15.1:0/252977369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad48aa00 msgr2=0x5611ad490ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.696+0000 7f4f10a00640 1 --2- 172.21.15.1:0/252977369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad48aa00 0x5611ad490ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.696+0000 7f4f10a00640 1 -- 172.21.15.1:0/252977369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad480140 msgr2=0x5611ad490980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.696+0000 7f4f10a00640 1 --2- 172.21.15.1:0/252977369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad480140 0x5611ad490980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.696+0000 7f4f10a00640 1 -- 172.21.15.1:0/252977369 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611ad4fffd0 con 0x5611ad47f820 2024-04-26T01:54:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.696+0000 7f4f10a00640 1 --2- 172.21.15.1:0/252977369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad47f820 0x5611ad490440 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4f00055040 tx=0x7f4f0002aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7c844fdfb69fba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.696+0000 7f4ef2000640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:54:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.697+0000 7f4ef2000640 1 -- 172.21.15.1:0/252977369 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0005f020 con 0x5611ad47f820 2024-04-26T01:54:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.697+0000 7f4ef2000640 1 -- 172.21.15.1:0/252977369 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f4f00005040 con 0x5611ad47f820 2024-04-26T01:54:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.697+0000 7f4f16379740 1 -- 172.21.15.1:0/252977369 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f000528a0 con 0x5611ad47f820 2024-04-26T01:54:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.697+0000 7f4ef2000640 1 -- 172.21.15.1:0/252977369 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f000053a0 con 0x5611ad47f820 2024-04-26T01:54:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.697+0000 7f4f16379740 10 client.15447 fetch_fsmap learned FSMap version 30 2024-04-26T01:54:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.697+0000 7f4f16379740 1 -- 172.21.15.1:0/252977369 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5611ad501280 con 0x5611ad47f820 2024-04-26T01:54:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.697+0000 7f4ef2000640 1 -- 172.21.15.1:0/252977369 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f000528a0 con 0x5611ad47f820 2024-04-26T01:54:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.697+0000 7f4ef2000640 1 -- 172.21.15.1:0/252977369 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4f00004ab0 con 0x5611ad47f820 2024-04-26T01:54:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.698+0000 7f4f16379740 10 client.15447 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:54:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.698+0000 7f4f16379740 10 client.15447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:54:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.698+0000 7f4f16379740 20 client.15447 populate_metadata read hostname 'smithi001' 2024-04-26T01:54:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.698+0000 7f4f16379740 1 --2- 172.21.15.1:0/252977369 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5611ad5019d0 0x5611ad503df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.698+0000 7f4f16379740 4 client.15447 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:54:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.698+0000 7f4f16379740 1 -- 172.21.15.1:0/252977369 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5611ad3f3320 con 0x5611ad5019d0 2024-04-26T01:54:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.698+0000 7f4f0be00640 1 --2- 172.21.15.1:0/252977369 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5611ad5019d0 0x5611ad503df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.699+0000 7f4f0be00640 1 --2- 172.21.15.1:0/252977369 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5611ad5019d0 0x5611ad503df0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f153f00ab61c8c36 server_cookie=ef891cb9675901f6 in_seq=0 out_seq=0 2024-04-26T01:54:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.699+0000 7f4ef2000640 10 client.15447 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.713+0000 7f4ef2000640 1 -- 172.21.15.1:0/252977369 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5611ad3f3320 con 0x5611ad5019d0 2024-04-26T01:54:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.713+0000 7f4ef2000640 10 client.15447 handle_command_reply: tid=0 2024-04-26T01:54:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.713+0000 7f4f16379740 1 client.15447 shutdown 2024-04-26T01:54:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.714+0000 7f4f16379740 1 -- 172.21.15.1:0/252977369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad47f820 msgr2=0x5611ad490440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.714+0000 7f4f16379740 1 --2- 172.21.15.1:0/252977369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad47f820 0x5611ad490440 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4f00055040 tx=0x7f4f0002aaf0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.714+0000 7f4f16379740 1 -- 172.21.15.1:0/252977369 shutdown_connections 2024-04-26T01:54:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.714+0000 7f4f16379740 1 --2- 172.21.15.1:0/252977369 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5611ad5019d0 0x5611ad503df0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.714+0000 7f4f16379740 1 --2- 172.21.15.1:0/252977369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611ad48aa00 0x5611ad490ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.714+0000 7f4f16379740 1 --2- 172.21.15.1:0/252977369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5611ad480140 0x5611ad490980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.714+0000 7f4ef2000640 0 client.15447 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.714+0000 7f4f16379740 1 --2- 172.21.15.1:0/252977369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611ad47f820 0x5611ad490440 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.714+0000 7f4f16379740 1 -- 172.21.15.1:0/252977369 >> 172.21.15.1:0/252977369 conn(0x5611ad47bd30 msgr2=0x5611ad47ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.714+0000 7f4f16379740 1 -- 172.21.15.1:0/252977369 shutdown_connections 2024-04-26T01:54:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.714+0000 7f4f16379740 1 -- 172.21.15.1:0/252977369 wait complete. 2024-04-26T01:54:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.714+0000 7f4f16379740 20 client.15447 trim_cache size 0 max 16384 2024-04-26T01:54:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:54:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:54:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:54:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:54:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.715+0000 7f4f13400640 1 -- 172.21.15.1:0/175006749 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f4ef40a3d90 msgr2=0x7f4ef40a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.715+0000 7f4f13400640 1 --2- 172.21.15.1:0/175006749 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f4ef40a3d90 0x7f4ef40a6250 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4f00031440 tx=0x7f4f00005b50 comp rx=0 tx=0).stop 2024-04-26T01:54:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.715+0000 7f4f13400640 1 -- 172.21.15.1:0/175006749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f0c1087e0 msgr2=0x7f4f0c1aaf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.715+0000 7f4f13400640 1 --2- 172.21.15.1:0/175006749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f0c1087e0 0x7f4f0c1aaf80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400ec80 tx=0x7f4f0400c6a0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.715+0000 7f4f13400640 1 -- 172.21.15.1:0/175006749 shutdown_connections 2024-04-26T01:54:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.715+0000 7f4f13400640 1 --2- 172.21.15.1:0/175006749 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f4ef40a3d90 0x7f4ef40a6250 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.715+0000 7f4f13400640 1 --2- 172.21.15.1:0/175006749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f0c1087e0 0x7f4f0c1aaf80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.715+0000 7f4f13400640 1 --2- 172.21.15.1:0/175006749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f0c106170 0x7f4f0c1082a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.715+0000 7f4f13400640 1 --2- 172.21.15.1:0/175006749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f0c1010a0 0x7f4f0c107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.715+0000 7f4f13400640 1 -- 172.21.15.1:0/175006749 >> 172.21.15.1:0/175006749 conn(0x7f4f0c077f70 msgr2=0x7f4f0c0ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.715+0000 7f4f13400640 1 -- 172.21.15.1:0/175006749 shutdown_connections 2024-04-26T01:54:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:54:52.715+0000 7f4f13400640 1 -- 172.21.15.1:0/175006749 wait complete. 2024-04-26T01:54:53.305 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24518 quiesce db --set-id 79040149 --release --await' 2024-04-26T01:54:53.306 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24518 quiesce db --set-id 79040149 --release --await 2024-04-26T01:54:53.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:53 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/144137040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:54:53.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:53 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/144137040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:54:53.608 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:53 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/144137040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:54:53.749 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:54 smithi097 ceph-mon[32853]: pgmap v101: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-04-26T01:54:54.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:54 smithi173 ceph-mon[32553]: pgmap v101: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-04-26T01:54:54.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:54 smithi001 ceph-mon[27822]: pgmap v101: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-04-26T01:54:56.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.029+0000 7f130f200700 1 -- 172.21.15.1:0/3610648739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1308101940 msgr2=0x7f1308101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:56.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.029+0000 7f130f200700 1 --2- 172.21.15.1:0/3610648739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1308101940 0x7f1308101dc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009990 tx=0x7f12f0009ca0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.029+0000 7f130f200700 1 -- 172.21.15.1:0/3610648739 shutdown_connections 2024-04-26T01:54:56.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.029+0000 7f130f200700 1 --2- 172.21.15.1:0/3610648739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1308102300 0x7f130810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.029+0000 7f130f200700 1 --2- 172.21.15.1:0/3610648739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1308107990 0x7f1308107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.029+0000 7f130f200700 1 --2- 172.21.15.1:0/3610648739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1308101940 0x7f1308101dc0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.029+0000 7f130f200700 1 -- 172.21.15.1:0/3610648739 >> 172.21.15.1:0/3610648739 conn(0x7f13080788f0 msgr2=0x7f1308078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:56.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.029+0000 7f130f200700 1 -- 172.21.15.1:0/3610648739 shutdown_connections 2024-04-26T01:54:56.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.029+0000 7f130f200700 1 -- 172.21.15.1:0/3610648739 wait complete. 2024-04-26T01:54:56.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.030+0000 7f130f200700 1 Processor -- start 2024-04-26T01:54:56.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.030+0000 7f130f200700 1 -- start start 2024-04-26T01:54:56.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.031+0000 7f130f200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1308101940 0x7f1308199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:56.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.031+0000 7f130f200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1308102300 0x7f1308199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:56.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.031+0000 7f130f200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1308107990 0x7f130819d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:56.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.031+0000 7f130f200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1308111ae0 con 0x7f1308101940 2024-04-26T01:54:56.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.031+0000 7f130f200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1308111960 con 0x7f1308107990 2024-04-26T01:54:56.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.031+0000 7f130f200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1308111c60 con 0x7f1308102300 2024-04-26T01:54:56.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.031+0000 7f1307e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1308101940 0x7f1308199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:56.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.031+0000 7f1307e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1308101940 0x7f1308199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40058/0 (socket says 172.21.15.1:40058) 2024-04-26T01:54:56.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.031+0000 7f1307e00700 1 -- 172.21.15.1:0/3217723943 learned_addr learned my addr 172.21.15.1:0/3217723943 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:56.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.031+0000 7f12ff400700 1 --2- 172.21.15.1:0/3217723943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1308102300 0x7f1308199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:56.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.031+0000 7f130d200700 1 --2- 172.21.15.1:0/3217723943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1308107990 0x7f130819d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:56.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.032+0000 7f1307e00700 1 -- 172.21.15.1:0/3217723943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1308102300 msgr2=0x7f1308199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:56.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.032+0000 7f1307e00700 1 --2- 172.21.15.1:0/3217723943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1308102300 0x7f1308199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.032+0000 7f1307e00700 1 -- 172.21.15.1:0/3217723943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1308107990 msgr2=0x7f130819d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:56.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.032+0000 7f1307e00700 1 --2- 172.21.15.1:0/3217723943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1308107990 0x7f130819d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.032+0000 7f1307e00700 1 -- 172.21.15.1:0/3217723943 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130819e120 con 0x7f1308101940 2024-04-26T01:54:56.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.032+0000 7f1307e00700 1 --2- 172.21.15.1:0/3217723943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1308101940 0x7f1308199110 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f12f800e9a0 tx=0x7f12f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:56.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.032+0000 7f1305600700 1 -- 172.21.15.1:0/3217723943 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f800ca10 con 0x7f1308101940 2024-04-26T01:54:56.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.033+0000 7f1305600700 1 -- 172.21.15.1:0/3217723943 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12f800cbb0 con 0x7f1308101940 2024-04-26T01:54:56.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.033+0000 7f130f200700 1 -- 172.21.15.1:0/3217723943 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f130819e410 con 0x7f1308101940 2024-04-26T01:54:56.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.033+0000 7f1305600700 1 -- 172.21.15.1:0/3217723943 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8018680 con 0x7f1308101940 2024-04-26T01:54:56.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.033+0000 7f130f200700 1 -- 172.21.15.1:0/3217723943 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13081a5c90 con 0x7f1308101940 2024-04-26T01:54:56.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.036+0000 7f1305600700 1 -- 172.21.15.1:0/3217723943 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f12f8010430 con 0x7f1308101940 2024-04-26T01:54:56.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.037+0000 7f1305600700 1 --2- 172.21.15.1:0/3217723943 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f12ec0a3db0 0x7f12ec0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:56.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.038+0000 7f1305600700 1 -- 172.21.15.1:0/3217723943 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f12f80d0950 con 0x7f1308101940 2024-04-26T01:54:56.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.038+0000 7f12ff400700 1 --2- 172.21.15.1:0/3217723943 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f12ec0a3db0 0x7f12ec0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:56.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.038+0000 7f1319a91b80 1 Processor -- start 2024-04-26T01:54:56.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.038+0000 7f1319a91b80 1 -- start start 2024-04-26T01:54:56.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.039+0000 7f1319a91b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514bd920 0x55e4514bdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:56.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.039+0000 7f1319a91b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 0x55e4514c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:56.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.039+0000 7f1319a91b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514c90f0 0x55e4514cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:56.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.039+0000 7f12ff400700 1 --2- 172.21.15.1:0/3217723943 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f12ec0a3db0 0x7f12ec0a6270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f12f00098e0 tx=0x7f12f0005960 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:56.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.039+0000 7f1319a91b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e451410f90 con 0x55e4514be240 2024-04-26T01:54:56.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.039+0000 7f1319a91b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e4513b7830 con 0x55e4514c90f0 2024-04-26T01:54:56.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.039+0000 7f1319a91b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e45140cf50 con 0x55e4514bd920 2024-04-26T01:54:56.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.039+0000 7f1307e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514bd920 0x55e4514bdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:56.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.039+0000 7f12ff400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 0x55e4514c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.039+0000 7f1307e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514bd920 0x55e4514bdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:42264/0 (socket says 172.21.15.1:42264) 2024-04-26T01:54:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.039+0000 7f1307e00700 1 -- 172.21.15.1:0/617943327 learned_addr learned my addr 172.21.15.1:0/617943327 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.039+0000 7f130d200700 1 --2- 172.21.15.1:0/617943327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514c90f0 0x55e4514cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.039+0000 7f1307e00700 1 -- 172.21.15.1:0/617943327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514c90f0 msgr2=0x55e4514cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.039+0000 7f1307e00700 1 --2- 172.21.15.1:0/617943327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514c90f0 0x55e4514cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.039+0000 7f1307e00700 1 -- 172.21.15.1:0/617943327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 msgr2=0x55e4514c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.039+0000 7f1307e00700 1 --2- 172.21.15.1:0/617943327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 0x55e4514c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.039+0000 7f1307e00700 1 -- 172.21.15.1:0/617943327 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4513ad780 con 0x55e4514bd920 2024-04-26T01:54:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.039+0000 7f12ff400700 1 --2- 172.21.15.1:0/617943327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 0x55e4514c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:54:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.040+0000 7f130d200700 1 --2- 172.21.15.1:0/617943327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514c90f0 0x55e4514cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:54:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.040+0000 7f1307e00700 1 --2- 172.21.15.1:0/617943327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514bd920 0x55e4514bdd00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f12f8099260 tx=0x7f12f809d080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5c9d1b312a8a286 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.040+0000 7f12fd600700 1 -- 172.21.15.1:0/617943327 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f809ab90 con 0x55e4514bd920 2024-04-26T01:54:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.040+0000 7f12fd600700 1 -- 172.21.15.1:0/617943327 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12f80870f0 con 0x55e4514bd920 2024-04-26T01:54:56.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.040+0000 7f12fd600700 1 -- 172.21.15.1:0/617943327 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f80a52d0 con 0x55e4514bd920 2024-04-26T01:54:56.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.041+0000 7f1319a91b80 1 -- 172.21.15.1:0/617943327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514bd920 msgr2=0x55e4514bdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:56.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.041+0000 7f1319a91b80 1 --2- 172.21.15.1:0/617943327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514bd920 0x55e4514bdd00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f12f8099260 tx=0x7f12f809d080 comp rx=0 tx=0).stop 2024-04-26T01:54:56.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.041+0000 7f1319a91b80 1 -- 172.21.15.1:0/617943327 shutdown_connections 2024-04-26T01:54:56.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.041+0000 7f1319a91b80 1 --2- 172.21.15.1:0/617943327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514c90f0 0x55e4514cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.041+0000 7f1319a91b80 1 --2- 172.21.15.1:0/617943327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514bd920 0x55e4514bdd00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.041+0000 7f1319a91b80 1 --2- 172.21.15.1:0/617943327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 0x55e4514c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.041+0000 7f1319a91b80 1 -- 172.21.15.1:0/617943327 >> 172.21.15.1:0/617943327 conn(0x55e4513ec6a0 msgr2=0x55e4514b9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.041+0000 7f1319a91b80 1 -- 172.21.15.1:0/617943327 shutdown_connections 2024-04-26T01:54:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.041+0000 7f1319a91b80 1 -- 172.21.15.1:0/617943327 wait complete. 2024-04-26T01:54:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.042+0000 7f1319a91b80 1 -- start start 2024-04-26T01:54:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.042+0000 7f1319a91b80 1 Processor -- start 2024-04-26T01:54:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.042+0000 7f1319a91b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514bd920 0x55e4514c8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.043+0000 7f1319a91b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 0x55e4514c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.043+0000 7f1319a91b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514c90f0 0x55e4514c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.043+0000 7f1319a91b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e4514cbbb0 con 0x55e4514be240 2024-04-26T01:54:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.043+0000 7f1319a91b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e4513ca960 con 0x55e4514bd920 2024-04-26T01:54:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.043+0000 7f1319a91b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e4514cbd30 con 0x55e4514c90f0 2024-04-26T01:54:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.043+0000 7f1319a91b80 10 client.0 _ll_register_callbacks cb 0x55e4513a9820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.043+0000 7f1307e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514bd920 0x55e4514c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.043+0000 7f12ff400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 0x55e4514c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.043+0000 7f130d200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514c90f0 0x55e4514c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:56.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.043+0000 7f1307e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514bd920 0x55e4514c8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:48838/0 (socket says 172.21.15.1:48838) 2024-04-26T01:54:56.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.043+0000 7f12ff400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 0x55e4514c5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40070/0 (socket says 172.21.15.1:40070) 2024-04-26T01:54:56.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.043+0000 7f1307e00700 1 -- 172.21.15.1:0/348909924 learned_addr learned my addr 172.21.15.1:0/348909924 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:56.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.043+0000 7f130d200700 1 -- 172.21.15.1:0/348909924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514bd920 msgr2=0x55e4514c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:56.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.043+0000 7f130d200700 1 --2- 172.21.15.1:0/348909924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514bd920 0x55e4514c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.043+0000 7f130d200700 1 -- 172.21.15.1:0/348909924 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 msgr2=0x55e4514c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:56.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.043+0000 7f130d200700 1 --2- 172.21.15.1:0/348909924 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 0x55e4514c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.043+0000 7f130d200700 1 -- 172.21.15.1:0/348909924 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e45135e9d0 con 0x55e4514c90f0 2024-04-26T01:54:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.043+0000 7f1307e00700 1 --2- 172.21.15.1:0/348909924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514bd920 0x55e4514c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.044+0000 7f130d200700 1 --2- 172.21.15.1:0/348909924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514c90f0 0x55e4514c6160 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1300007b20 tx=0x7f1300007b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aca386a1954e68c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.044+0000 7f12f6000700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:54:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.044+0000 7f12f6000700 1 -- 172.21.15.1:0/348909924 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130001a070 con 0x55e4514c90f0 2024-04-26T01:54:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.044+0000 7f1319a91b80 1 -- 172.21.15.1:0/348909924 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12f80870f0 con 0x55e4514c90f0 2024-04-26T01:54:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.044+0000 7f12f6000700 1 -- 172.21.15.1:0/348909924 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f1300004db0 con 0x55e4514c90f0 2024-04-26T01:54:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.044+0000 7f12f6000700 1 -- 172.21.15.1:0/348909924 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1300025050 con 0x55e4514c90f0 2024-04-26T01:54:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.044+0000 7f1319a91b80 10 client.25609 fetch_fsmap learned FSMap version 30 2024-04-26T01:54:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.045+0000 7f1319a91b80 1 -- 172.21.15.1:0/348909924 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e4513a7bf0 con 0x55e4514c90f0 2024-04-26T01:54:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.045+0000 7f12f6000700 1 -- 172.21.15.1:0/348909924 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12f80870f0 con 0x55e4514c90f0 2024-04-26T01:54:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.045+0000 7f12f6000700 1 -- 172.21.15.1:0/348909924 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f1300004030 con 0x55e4514c90f0 2024-04-26T01:54:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.045+0000 7f1319a91b80 10 client.25609 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:54:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.045+0000 7f1319a91b80 10 client.25609 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:54:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.045+0000 7f1319a91b80 20 client.25609 populate_metadata read hostname 'smithi001' 2024-04-26T01:54:56.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.045+0000 7f1319a91b80 1 --2- 172.21.15.1:0/348909924 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e4514d89f0 0x55e4514dade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:56.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.045+0000 7f1319a91b80 4 client.25609 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:54:56.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.045+0000 7f1319a91b80 1 -- 172.21.15.1:0/348909924 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e45141e780 con 0x55e4514d89f0 2024-04-26T01:54:56.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.046+0000 7f1307e00700 1 --2- 172.21.15.1:0/348909924 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e4514d89f0 0x55e4514dade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:56.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.046+0000 7f1307e00700 1 --2- 172.21.15.1:0/348909924 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e4514d89f0 0x55e4514dade0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b62207f20ff0119 server_cookie=cc1140ee513a3241 in_seq=0 out_seq=0 2024-04-26T01:54:56.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.046+0000 7f12f6000700 10 client.25609 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:56.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.047+0000 7f12f6000700 1 -- 172.21.15.1:0/348909924 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e45141e780 con 0x55e4514d89f0 2024-04-26T01:54:56.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.047+0000 7f12f6000700 10 client.25609 handle_command_reply: tid=0 2024-04-26T01:54:56.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.047+0000 7f1319a91b80 1 client.25609 shutdown 2024-04-26T01:54:56.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.047+0000 7f1319a91b80 1 -- 172.21.15.1:0/348909924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514c90f0 msgr2=0x55e4514c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:56.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.047+0000 7f1319a91b80 1 --2- 172.21.15.1:0/348909924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514c90f0 0x55e4514c6160 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1300007b20 tx=0x7f1300007b50 comp rx=0 tx=0).stop 2024-04-26T01:54:56.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.047+0000 7f1319a91b80 1 -- 172.21.15.1:0/348909924 shutdown_connections 2024-04-26T01:54:56.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.047+0000 7f1319a91b80 1 --2- 172.21.15.1:0/348909924 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e4514d89f0 0x55e4514dade0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.047+0000 7f1319a91b80 1 --2- 172.21.15.1:0/348909924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514c90f0 0x55e4514c6160 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.047+0000 7f1319a91b80 1 --2- 172.21.15.1:0/348909924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514bd920 0x55e4514c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.047+0000 7f1319a91b80 1 --2- 172.21.15.1:0/348909924 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 0x55e4514c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.047+0000 7f12f6000700 0 client.25609 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:56.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.047+0000 7f1319a91b80 1 -- 172.21.15.1:0/348909924 >> 172.21.15.1:0/348909924 conn(0x55e4513ec6a0 msgr2=0x55e4514b99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:56.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.048+0000 7f1319a91b80 1 -- 172.21.15.1:0/348909924 shutdown_connections 2024-04-26T01:54:56.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.048+0000 7f1319a91b80 1 -- 172.21.15.1:0/348909924 wait complete. 2024-04-26T01:54:56.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.048+0000 7f1319a91b80 20 client.25609 trim_cache size 0 max 16384 2024-04-26T01:54:56.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.069+0000 7f1319a91b80 1 Processor -- start 2024-04-26T01:54:56.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.069+0000 7f1319a91b80 1 -- start start 2024-04-26T01:54:56.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.070+0000 7f1319a91b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514bd920 0x55e4514b9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:56.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.070+0000 7f1319a91b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 0x55e4514d4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:56.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.070+0000 7f1319a91b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514c90f0 0x55e4514d0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:56.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.070+0000 7f1319a91b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f13081afd40 con 0x55e4514be240 2024-04-26T01:54:56.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.070+0000 7f1319a91b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f13081ad570 con 0x55e4514bd920 2024-04-26T01:54:56.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.070+0000 7f1319a91b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f13081ab3c0 con 0x55e4514c90f0 2024-04-26T01:54:56.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.070+0000 7f12ff400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 0x55e4514d4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:56.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.070+0000 7f1307e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514bd920 0x55e4514b9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:56.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.070+0000 7f12ff400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 0x55e4514d4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40084/0 (socket says 172.21.15.1:40084) 2024-04-26T01:54:56.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.070+0000 7f12ff400700 1 -- 172.21.15.1:0/3786095168 learned_addr learned my addr 172.21.15.1:0/3786095168 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:56.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.070+0000 7f130d200700 1 --2- 172.21.15.1:0/3786095168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514c90f0 0x55e4514d0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.071+0000 7f12ff400700 1 -- 172.21.15.1:0/3786095168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514c90f0 msgr2=0x55e4514d0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.071+0000 7f12ff400700 1 --2- 172.21.15.1:0/3786095168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514c90f0 0x55e4514d0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.071+0000 7f12ff400700 1 -- 172.21.15.1:0/3786095168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514bd920 msgr2=0x55e4514b9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.071+0000 7f12ff400700 1 --2- 172.21.15.1:0/3786095168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514bd920 0x55e4514b9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.071+0000 7f12ff400700 1 -- 172.21.15.1:0/3786095168 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4514c8f20 con 0x55e4514be240 2024-04-26T01:54:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.071+0000 7f1307e00700 1 --2- 172.21.15.1:0/3786095168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514bd920 0x55e4514b9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.071+0000 7f130d200700 1 --2- 172.21.15.1:0/3786095168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514c90f0 0x55e4514d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.071+0000 7f12ff400700 1 --2- 172.21.15.1:0/3786095168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 0x55e4514d4ab0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009990 tx=0x7f12f0022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40021eee1672030d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.071+0000 7f12fcc00700 1 -- 172.21.15.1:0/3786095168 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f0036070 con 0x55e4514be240 2024-04-26T01:54:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.072+0000 7f12fcc00700 1 -- 172.21.15.1:0/3786095168 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12f000be40 con 0x55e4514be240 2024-04-26T01:54:56.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.072+0000 7f12fcc00700 1 -- 172.21.15.1:0/3786095168 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f000f990 con 0x55e4514be240 2024-04-26T01:54:56.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.072+0000 7f1319a91b80 1 -- 172.21.15.1:0/3786095168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 msgr2=0x55e4514d4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:56.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.072+0000 7f1319a91b80 1 --2- 172.21.15.1:0/3786095168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 0x55e4514d4ab0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009990 tx=0x7f12f0022000 comp rx=0 tx=0).stop 2024-04-26T01:54:56.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.072+0000 7f1319a91b80 1 -- 172.21.15.1:0/3786095168 shutdown_connections 2024-04-26T01:54:56.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.072+0000 7f1319a91b80 1 --2- 172.21.15.1:0/3786095168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514c90f0 0x55e4514d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.072+0000 7f1319a91b80 1 --2- 172.21.15.1:0/3786095168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514bd920 0x55e4514b9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.072+0000 7f1319a91b80 1 --2- 172.21.15.1:0/3786095168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 0x55e4514d4ab0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.072+0000 7f1319a91b80 1 -- 172.21.15.1:0/3786095168 >> 172.21.15.1:0/3786095168 conn(0x55e4513ec6a0 msgr2=0x55e4514b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:56.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.072+0000 7f1319a91b80 1 -- 172.21.15.1:0/3786095168 shutdown_connections 2024-04-26T01:54:56.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.073+0000 7f1319a91b80 1 -- 172.21.15.1:0/3786095168 wait complete. 2024-04-26T01:54:56.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.073+0000 7f1319a91b80 1 -- start start 2024-04-26T01:54:56.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.073+0000 7f1319a91b80 1 Processor -- start 2024-04-26T01:54:56.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.074+0000 7f1319a91b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514bd920 0x55e4514cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:56.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.074+0000 7f1319a91b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 0x55e4514d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:56.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.074+0000 7f1319a91b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514c90f0 0x55e4514d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:56.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.074+0000 7f1319a91b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f13081aba30 con 0x55e4514be240 2024-04-26T01:54:56.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.074+0000 7f1319a91b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f13081acea0 con 0x55e4514c90f0 2024-04-26T01:54:56.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.074+0000 7f1319a91b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f13081ac100 con 0x55e4514bd920 2024-04-26T01:54:56.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.074+0000 7f1319a91b80 10 client.0 _ll_register_callbacks cb 0x55e4513a9820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:54:56.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.074+0000 7f12ff400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 0x55e4514d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:56.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.074+0000 7f130d200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514c90f0 0x55e4514d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:56.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.074+0000 7f130d200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514c90f0 0x55e4514d05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:48850/0 (socket says 172.21.15.1:48850) 2024-04-26T01:54:56.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.074+0000 7f130d200700 1 -- 172.21.15.1:0/4222038570 learned_addr learned my addr 172.21.15.1:0/4222038570 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:54:56.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.074+0000 7f1307e00700 1 --2- 172.21.15.1:0/4222038570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514bd920 0x55e4514cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:56.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.074+0000 7f130d200700 1 -- 172.21.15.1:0/4222038570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514bd920 msgr2=0x55e4514cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:56.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.075+0000 7f130d200700 1 --2- 172.21.15.1:0/4222038570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514bd920 0x55e4514cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.075+0000 7f130d200700 1 -- 172.21.15.1:0/4222038570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 msgr2=0x55e4514d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:54:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.075+0000 7f130d200700 1 --2- 172.21.15.1:0/4222038570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 0x55e4514d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.075+0000 7f130d200700 1 -- 172.21.15.1:0/4222038570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4514bdf50 con 0x55e4514c90f0 2024-04-26T01:54:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.075+0000 7f1307e00700 1 --2- 172.21.15.1:0/4222038570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514bd920 0x55e4514cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.075+0000 7f12ff400700 1 --2- 172.21.15.1:0/4222038570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 0x55e4514d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.075+0000 7f130d200700 1 --2- 172.21.15.1:0/4222038570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514c90f0 0x55e4514d05d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f13000045c0 tx=0x7f1300005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68aa70a378e9ed5e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.075+0000 7f12f6000700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:54:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.075+0000 7f12f6000700 1 -- 172.21.15.1:0/4222038570 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130001a070 con 0x55e4514c90f0 2024-04-26T01:54:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.075+0000 7f12f6000700 1 -- 172.21.15.1:0/4222038570 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f1300002030 con 0x55e4514c90f0 2024-04-26T01:54:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.075+0000 7f1319a91b80 1 -- 172.21.15.1:0/4222038570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12f000be40 con 0x55e4514c90f0 2024-04-26T01:54:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.076+0000 7f12f6000700 1 -- 172.21.15.1:0/4222038570 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1300038050 con 0x55e4514c90f0 2024-04-26T01:54:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.076+0000 7f1319a91b80 10 client.25589 fetch_fsmap learned FSMap version 30 2024-04-26T01:54:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.076+0000 7f1319a91b80 1 -- 172.21.15.1:0/4222038570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e4514bd750 con 0x55e4514c90f0 2024-04-26T01:54:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.076+0000 7f12f6000700 1 -- 172.21.15.1:0/4222038570 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12f000be40 con 0x55e4514c90f0 2024-04-26T01:54:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.076+0000 7f12f6000700 1 -- 172.21.15.1:0/4222038570 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f1300003f20 con 0x55e4514c90f0 2024-04-26T01:54:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.076+0000 7f1319a91b80 10 client.25589 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:54:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.076+0000 7f1319a91b80 10 client.25589 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:54:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.076+0000 7f1319a91b80 20 client.25589 populate_metadata read hostname 'smithi001' 2024-04-26T01:54:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.076+0000 7f1319a91b80 1 --2- 172.21.15.1:0/4222038570 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e451540770 0x55e451542b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.076+0000 7f1319a91b80 4 client.25589 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "79040149", "release": true, "await": true}] 2024-04-26T01:54:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.077+0000 7f1319a91b80 1 -- 172.21.15.1:0/4222038570 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "quiesce db", "set_id": "79040149", "release": true, "await": true}) -- 0x55e451418820 con 0x55e451540770 2024-04-26T01:54:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.077+0000 7f1307e00700 1 --2- 172.21.15.1:0/4222038570 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e451540770 0x55e451542b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.078+0000 7f1307e00700 1 --2- 172.21.15.1:0/4222038570 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e451540770 0x55e451542b90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1abff5bb763c57ca server_cookie=aa85ed56532c58f6 in_seq=0 out_seq=0 2024-04-26T01:54:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.078+0000 7f12f6000700 10 client.25589 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:56.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.264+0000 7f12f6000700 1 -- 172.21.15.1:0/4222038570 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55e451418820 con 0x55e451540770 2024-04-26T01:54:56.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.264+0000 7f12f6000700 10 client.25589 handle_command_reply: tid=0 2024-04-26T01:54:56.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.264+0000 7f1319a91b80 1 client.25589 shutdown 2024-04-26T01:54:56.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.264+0000 7f1319a91b80 1 -- 172.21.15.1:0/4222038570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514c90f0 msgr2=0x55e4514d05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:56.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.264+0000 7f1319a91b80 1 --2- 172.21.15.1:0/4222038570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514c90f0 0x55e4514d05d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f13000045c0 tx=0x7f1300005f00 comp rx=0 tx=0).stop 2024-04-26T01:54:56.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.265+0000 7f1319a91b80 1 -- 172.21.15.1:0/4222038570 shutdown_connections 2024-04-26T01:54:56.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.265+0000 7f1319a91b80 1 --2- 172.21.15.1:0/4222038570 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e451540770 0x55e451542b90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.265+0000 7f1319a91b80 1 --2- 172.21.15.1:0/4222038570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e4514c90f0 0x55e4514d05d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.265+0000 7f1319a91b80 1 --2- 172.21.15.1:0/4222038570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4514bd920 0x55e4514cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.265+0000 7f1319a91b80 1 --2- 172.21.15.1:0/4222038570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4514be240 0x55e4514d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.265+0000 7f1319a91b80 1 -- 172.21.15.1:0/4222038570 >> 172.21.15.1:0/4222038570 conn(0x55e4513ec6a0 msgr2=0x55e4514df670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:56.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.265+0000 7f12f6000700 0 client.25589 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:54:56.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.265+0000 7f1319a91b80 1 -- 172.21.15.1:0/4222038570 shutdown_connections 2024-04-26T01:54:56.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.265+0000 7f1319a91b80 1 -- 172.21.15.1:0/4222038570 wait complete. 2024-04-26T01:54:56.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.266+0000 7f1319a91b80 20 client.25589 trim_cache size 0 max 16384 2024-04-26T01:54:56.265 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T01:54:56.265 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 30, 2024-04-26T01:54:56.265 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24518, 2024-04-26T01:54:56.265 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 8, 2024-04-26T01:54:56.265 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-04-26T01:54:56.265 INFO:teuthology.orchestra.run.smithi001.stdout: "79040149": { 2024-04-26T01:54:56.266 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 8, 2024-04-26T01:54:56.266 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-04-26T01:54:56.266 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:54:56.266 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-04-26T01:54:56.266 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:54:56.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T01:54:56.266 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-04-26T01:54:56.266 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 90.0, 2024-04-26T01:54:56.266 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-04-26T01:54:56.266 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-04-26T01:54:56.266 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-04-26T01:54:56.266 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:54:56.266 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-04-26T01:54:56.266 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:54:56.267 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:54:56.267 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:54:56.267 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:54:56.267 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:54:56.267 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:54:56.267 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T01:54:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.267+0000 7f130f200700 1 -- 172.21.15.1:0/3217723943 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f12ec0a3db0 msgr2=0x7f12ec0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.267+0000 7f130f200700 1 --2- 172.21.15.1:0/3217723943 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f12ec0a3db0 0x7f12ec0a6270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f12f00098e0 tx=0x7f12f0005960 comp rx=0 tx=0).stop 2024-04-26T01:54:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.267+0000 7f130f200700 1 -- 172.21.15.1:0/3217723943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1308101940 msgr2=0x7f1308199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.267+0000 7f130f200700 1 --2- 172.21.15.1:0/3217723943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1308101940 0x7f1308199110 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f12f800e9a0 tx=0x7f12f800ecb0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.268+0000 7f130f200700 1 -- 172.21.15.1:0/3217723943 shutdown_connections 2024-04-26T01:54:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.268+0000 7f130f200700 1 --2- 172.21.15.1:0/3217723943 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f12ec0a3db0 0x7f12ec0a6270 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.268+0000 7f130f200700 1 --2- 172.21.15.1:0/3217723943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1308107990 0x7f130819d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.268+0000 7f130f200700 1 --2- 172.21.15.1:0/3217723943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1308101940 0x7f1308199110 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.268+0000 7f130f200700 1 --2- 172.21.15.1:0/3217723943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1308102300 0x7f1308199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.268+0000 7f130f200700 1 -- 172.21.15.1:0/3217723943 >> 172.21.15.1:0/3217723943 conn(0x7f13080788f0 msgr2=0x7f130806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.268+0000 7f130f200700 1 -- 172.21.15.1:0/3217723943 shutdown_connections 2024-04-26T01:54:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:54:56.268+0000 7f130f200700 1 -- 172.21.15.1:0/3217723943 wait complete. 2024-04-26T01:54:56.283 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] rename 2024-04-26T01:54:56.325 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] multi_open_unlink 2024-04-26T01:54:56.328 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] unwritten_sync 2024-04-26T01:54:56.402 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] genhashnames 2024-04-26T01:54:56.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:56 smithi097 ceph-mon[32853]: pgmap v102: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2024-04-26T01:54:56.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:56 smithi173 ceph-mon[32553]: pgmap v102: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2024-04-26T01:54:56.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:56 smithi001 ceph-mon[27822]: pgmap v102: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2024-04-26T01:54:56.648 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_holes 2024-04-26T01:54:56.656 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_writev 2024-04-26T01:54:56.709 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_truncate_cmtime 2024-04-26T01:54:56.760 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] dirhash_collide 2024-04-26T01:54:56.964 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_rename_overwrite 2024-04-26T01:54:56.965 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] holetest 2024-04-26T01:54:57.034 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_truncate_self 2024-04-26T01:54:57.101 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] af_unix 2024-04-26T01:54:57.244 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_stale_pmd 2024-04-26T01:54:57.320 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '79040149', seconds elapsed: 48.6 2024-04-26T01:54:57.321 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-26T01:54:57.321 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:54:57.332 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_cow_race 2024-04-26T01:54:57.401 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_fallocate 2024-04-26T01:54:57.418 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fsync-err 2024-04-26T01:54:57.561 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_write_ro 2024-04-26T01:54:57.685 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_ext4_dax_journal_corruption 2024-04-26T01:54:57.751 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_ext4_dax_inline_corruption 2024-04-26T01:54:57.793 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_ofd_locks 2024-04-26T01:54:57.848 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:57.904 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_collision 2024-04-26T01:54:58.014 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] mmap-write-concurrent 2024-04-26T01:54:58.076 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_get_file_time 2024-04-26T01:54:58.173 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_create_short_dirs 2024-04-26T01:54:58.322 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_create_long_dirs 2024-04-26T01:54:58.344 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_enospc 2024-04-26T01:54:58.371 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_writev_overlap 2024-04-26T01:54:58.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:54:58 smithi001 ceph-mon[27822]: pgmap v103: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-04-26T01:54:58.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:54:58 smithi097 ceph-mon[32853]: pgmap v103: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-04-26T01:54:58.506 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] checkpoint_journal 2024-04-26T01:54:58.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:54:58 smithi173 ceph-mon[32553]: pgmap v103: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-04-26T01:54:58.668 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] mmap-rw-fault 2024-04-26T01:54:58.682 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] allocstale 2024-04-26T01:54:58.730 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_cow_memory_failure 2024-04-26T01:54:58.819 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fake-dump-rootino 2024-04-26T01:54:59.006 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] dio-buf-fault 2024-04-26T01:54:59.013 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] rewinddir-test 2024-04-26T01:54:59.067 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] readdir-while-renames 2024-04-26T01:54:59.119 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] xfsctl 2024-04-26T01:54:59.311 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] bstat 2024-04-26T01:54:59.345 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mtab 2024-04-26T01:54:59.406 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] getdevicesize 2024-04-26T01:54:59.471 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] preallo_rw_pattern_reader 2024-04-26T01:54:59.696 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] preallo_rw_pattern_writer 2024-04-26T01:54:59.703 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] ftrunc 2024-04-26T01:54:59.737 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] trunc 2024-04-26T01:54:59.831 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fs_perms 2024-04-26T01:54:59.991 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] testx 2024-04-26T01:55:00.036 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] looptest 2024-04-26T01:55:00.050 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] locktest 2024-04-26T01:55:00.180 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] unwritten_mmap 2024-04-26T01:55:00.289 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] bulkstat_unlink_test 2024-04-26T01:55:00.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.384+0000 7f10ff600700 1 -- 172.21.15.1:0/3725000135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10f8108a30 msgr2=0x7f10f8108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:00.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.384+0000 7f10ff600700 1 --2- 172.21.15.1:0/3725000135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10f8108a30 0x7f10f8108e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f10ec009990 tx=0x7f10ec009ca0 comp rx=0 tx=0).stop 2024-04-26T01:55:00.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.384+0000 7f10ff600700 1 -- 172.21.15.1:0/3725000135 shutdown_connections 2024-04-26T01:55:00.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.384+0000 7f10ff600700 1 --2- 172.21.15.1:0/3725000135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10f806b0f0 0x7f10f810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:00.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.384+0000 7f10ff600700 1 --2- 172.21.15.1:0/3725000135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10f8108a30 0x7f10f8108e10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:00.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.384+0000 7f10ff600700 1 --2- 172.21.15.1:0/3725000135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10f806a750 0x7f10f806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:00.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.384+0000 7f10ff600700 1 -- 172.21.15.1:0/3725000135 >> 172.21.15.1:0/3725000135 conn(0x7f10f80788f0 msgr2=0x7f10f8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:00.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.385+0000 7f10ff600700 1 -- 172.21.15.1:0/3725000135 shutdown_connections 2024-04-26T01:55:00.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.385+0000 7f10ff600700 1 -- 172.21.15.1:0/3725000135 wait complete. 2024-04-26T01:55:00.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.387+0000 7f10ff600700 1 Processor -- start 2024-04-26T01:55:00.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.387+0000 7f10ff600700 1 -- start start 2024-04-26T01:55:00.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.387+0000 7f10ff600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10f806a750 0x7f10f8199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:00.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.388+0000 7f10ff600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10f806b0f0 0x7f10f8199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:00.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.388+0000 7f10ff600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10f8108a30 0x7f10f819da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:00.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.388+0000 7f10ff600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f10f8112c20 con 0x7f10f8108a30 2024-04-26T01:55:00.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.388+0000 7f10ff600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f10f8112aa0 con 0x7f10f806a750 2024-04-26T01:55:00.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.388+0000 7f10ff600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f10f8112da0 con 0x7f10f806b0f0 2024-04-26T01:55:00.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.388+0000 7f10fcc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10f806a750 0x7f10f8199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:00.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.388+0000 7f10fcc00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10f806a750 0x7f10f8199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:48866/0 (socket says 172.21.15.1:48866) 2024-04-26T01:55:00.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.388+0000 7f10fcc00700 1 -- 172.21.15.1:0/1967956810 learned_addr learned my addr 172.21.15.1:0/1967956810 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:00.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.388+0000 7f10fd600700 1 --2- 172.21.15.1:0/1967956810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10f8108a30 0x7f10f819da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:00.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.388+0000 7f10fcc00700 1 -- 172.21.15.1:0/1967956810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10f806b0f0 msgr2=0x7f10f8199690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T01:55:00.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.388+0000 7f10fcc00700 1 --2- 172.21.15.1:0/1967956810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10f806b0f0 0x7f10f8199690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:00.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.388+0000 7f10fcc00700 1 -- 172.21.15.1:0/1967956810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10f8108a30 msgr2=0x7f10f819da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:00.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.388+0000 7f10fcc00700 1 --2- 172.21.15.1:0/1967956810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10f8108a30 0x7f10f819da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:00.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.388+0000 7f10fcc00700 1 -- 172.21.15.1:0/1967956810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f819e160 con 0x7f10f806a750 2024-04-26T01:55:00.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.389+0000 7f10fcc00700 1 --2- 172.21.15.1:0/1967956810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10f806a750 0x7f10f8199150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f10ec000c00 tx=0x7f10ec005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:00.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.389+0000 7f10f5600700 1 -- 172.21.15.1:0/1967956810 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec02b070 con 0x7f10f806a750 2024-04-26T01:55:00.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.389+0000 7f10f5600700 1 -- 172.21.15.1:0/1967956810 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ec01ed80 con 0x7f10f806a750 2024-04-26T01:55:00.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.389+0000 7f10f5600700 1 -- 172.21.15.1:0/1967956810 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec026770 con 0x7f10f806a750 2024-04-26T01:55:00.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.389+0000 7f10ff600700 1 -- 172.21.15.1:0/1967956810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f819e3f0 con 0x7f10f806a750 2024-04-26T01:55:00.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.390+0000 7f10ff600700 1 -- 172.21.15.1:0/1967956810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f8101940 con 0x7f10f806a750 2024-04-26T01:55:00.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.391+0000 7f10ff600700 1 -- 172.21.15.1:0/1967956810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10f8104170 con 0x7f10f806a750 2024-04-26T01:55:00.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.395+0000 7f10f5600700 1 -- 172.21.15.1:0/1967956810 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f10ec009d10 con 0x7f10f806a750 2024-04-26T01:55:00.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.397+0000 7f10f5600700 1 --2- 172.21.15.1:0/1967956810 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f10e00a3e30 0x7f10e00a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:00.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.398+0000 7f10f5600700 1 -- 172.21.15.1:0/1967956810 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f10ec0d5400 con 0x7f10f806a750 2024-04-26T01:55:00.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.398+0000 7f10f5600700 1 -- 172.21.15.1:0/1967956810 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10ec023070 con 0x7f10f806a750 2024-04-26T01:55:00.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.400+0000 7f10f7e00700 1 --2- 172.21.15.1:0/1967956810 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f10e00a3e30 0x7f10e00a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:00.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.401+0000 7f10f7e00700 1 --2- 172.21.15.1:0/1967956810 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f10e00a3e30 0x7f10e00a62f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f10f8078fd0 tx=0x7f10e4009260 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:00.427 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] deduperace 2024-04-26T01:55:00.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:00 smithi097 ceph-mon[32853]: pgmap v104: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-04-26T01:55:00.546 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] bulkstat_unlink_test_modified 2024-04-26T01:55:00.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:00 smithi173 ceph-mon[32553]: pgmap v104: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-04-26T01:55:00.613 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_dir_offset 2024-04-26T01:55:00.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:00 smithi001 ceph-mon[27822]: pgmap v104: 97 pgs: 97 active+clean; 29 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-04-26T01:55:00.730 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_futimens 2024-04-26T01:55:00.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.791+0000 7f10ff600700 1 -- 172.21.15.1:0/1967956810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10f8069730 con 0x7f10f806a750 2024-04-26T01:55:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.793+0000 7f10f5600700 1 -- 172.21.15.1:0/1967956810 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11841 (secure 0 0 0) 0x7f10ec013070 con 0x7f10f806a750 2024-04-26T01:55:00.791 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:55:00.792 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:53:29.647029+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25414":{"gid":25414,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/453427322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":453427322},{"type":"v1","addr":"172.21.15.173:6835","nonce":453427322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24518,24499,24514]},"id":1}]} 2024-04-26T01:55:00.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.797+0000 7f10ff600700 1 -- 172.21.15.1:0/1967956810 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f10e00a3e30 msgr2=0x7f10e00a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:00.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.797+0000 7f10ff600700 1 --2- 172.21.15.1:0/1967956810 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f10e00a3e30 0x7f10e00a62f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f10f8078fd0 tx=0x7f10e4009260 comp rx=0 tx=0).stop 2024-04-26T01:55:00.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.798+0000 7f10ff600700 1 -- 172.21.15.1:0/1967956810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10f806a750 msgr2=0x7f10f8199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:00.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.798+0000 7f10ff600700 1 --2- 172.21.15.1:0/1967956810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10f806a750 0x7f10f8199150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f10ec000c00 tx=0x7f10ec005e80 comp rx=0 tx=0).stop 2024-04-26T01:55:00.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.798+0000 7f10ff600700 1 -- 172.21.15.1:0/1967956810 shutdown_connections 2024-04-26T01:55:00.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.798+0000 7f10ff600700 1 --2- 172.21.15.1:0/1967956810 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f10e00a3e30 0x7f10e00a62f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:00.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.798+0000 7f10ff600700 1 --2- 172.21.15.1:0/1967956810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10f8108a30 0x7f10f819da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:00.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.798+0000 7f10ff600700 1 --2- 172.21.15.1:0/1967956810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10f806a750 0x7f10f8199150 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:00.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.798+0000 7f10ff600700 1 --2- 172.21.15.1:0/1967956810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10f806b0f0 0x7f10f8199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:00.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.798+0000 7f10ff600700 1 -- 172.21.15.1:0/1967956810 >> 172.21.15.1:0/1967956810 conn(0x7f10f80788f0 msgr2=0x7f10f80ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:00.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.798+0000 7f10ff600700 1 -- 172.21.15.1:0/1967956810 shutdown_connections 2024-04-26T01:55:00.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:00.798+0000 7f10ff600700 1 -- 172.21.15.1:0/1967956810 wait complete. 2024-04-26T01:55:00.799 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 30 2024-04-26T01:55:00.874 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_immutable 2024-04-26T01:55:00.932 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] stale_handle 2024-04-26T01:55:00.947 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] pwrite_mmap_blocked 2024-04-26T01:55:01.042 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_dir_offset2 2024-04-26T01:55:01.243 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] seek_sanity_test 2024-04-26T01:55:01.252 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] seek_copy_test 2024-04-26T01:55:01.385 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_readdir_1 2024-04-26T01:55:01.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:01 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1967956810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:55:01.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:01 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1967956810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:55:01.586 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_readdir_2 2024-04-26T01:55:01.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:01 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1967956810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:55:01.694 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fsync-tester 2024-04-26T01:55:01.757 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] nsexec 2024-04-26T01:55:01.859 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24518 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-04-26T01:55:01.859 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24518 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-04-26T01:55:01.925 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] cloner 2024-04-26T01:55:01.942 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] renameat2 2024-04-26T01:55:02.121 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_getcwd 2024-04-26T01:55:02.145 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] e4compact 2024-04-26T01:55:02.308 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] test-nextquota 2024-04-26T01:55:02.317 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] punch-alternating 2024-04-26T01:55:02.392 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:02.475 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] attr-list-by-handle-cursor-test 2024-04-26T01:55:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:02 smithi097 ceph-mon[32853]: pgmap v105: 97 pgs: 97 active+clean; 29 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-26T01:55:02.554 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] listxattr 2024-04-26T01:55:02.556 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:02 smithi001 ceph-mon[27822]: pgmap v105: 97 pgs: 97 active+clean; 29 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-26T01:55:02.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:02 smithi173 ceph-mon[32553]: pgmap v105: 97 pgs: 97 active+clean; 29 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-26T01:55:02.701 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] dio-interleaved 2024-04-26T01:55:02.705 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_dir_type 2024-04-26T01:55:02.817 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] dio-invalidate-cache 2024-04-26T01:55:02.844 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] stat_test 2024-04-26T01:55:03.022 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_encrypted_d_revalidate 2024-04-26T01:55:03.039 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] attr_replace_test 2024-04-26T01:55:03.210 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] swapon 2024-04-26T01:55:03.361 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] mkswap 2024-04-26T01:55:03.362 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_attr_corruption 2024-04-26T01:55:03.367 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_open_tmpfiles 2024-04-26T01:55:03.536 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fscrypt-crypt-util 2024-04-26T01:55:03.655 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] bulkstat_null_ocount 2024-04-26T01:55:03.670 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] splice-test 2024-04-26T01:55:03.701 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] chprojid_fail 2024-04-26T01:55:04.017 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] detached_mounts_propagation 2024-04-26T01:55:04.020 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] ext4_resize 2024-04-26T01:55:04.043 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_readdir_3 2024-04-26T01:55:04.326 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] splice2pipe 2024-04-26T01:55:04.377 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] uuid_ioctl 2024-04-26T01:55:04.410 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:04 smithi001 ceph-mon[27822]: pgmap v106: 97 pgs: 97 active+clean; 29 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-26T01:55:04.424 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_snapshot_deleted_subvolume 2024-04-26T01:55:04.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:04 smithi097 ceph-mon[32853]: pgmap v106: 97 pgs: 97 active+clean; 29 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-26T01:55:04.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:04 smithi173 ceph-mon[32553]: pgmap v106: 97 pgs: 97 active+clean; 29 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-26T01:55:04.623 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fiemap-fault 2024-04-26T01:55:04.682 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fiemap-tester 2024-04-26T01:55:04.686 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_stripealign 2024-04-26T01:55:04.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.746+0000 7f9f96c00700 1 -- 172.21.15.1:0/3152134482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f90108a60 msgr2=0x7f9f90108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:04.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.746+0000 7f9f96c00700 1 --2- 172.21.15.1:0/3152134482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f90108a60 0x7f9f90108e40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009990 tx=0x7f9f80009ca0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.747+0000 7f9f96c00700 1 -- 172.21.15.1:0/3152134482 shutdown_connections 2024-04-26T01:55:04.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.747+0000 7f9f96c00700 1 --2- 172.21.15.1:0/3152134482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f9006b100 0x7f9f9010f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.747+0000 7f9f96c00700 1 --2- 172.21.15.1:0/3152134482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f90108a60 0x7f9f90108e40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.747+0000 7f9f96c00700 1 --2- 172.21.15.1:0/3152134482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f9006a740 0x7f9f9006abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.747+0000 7f9f96c00700 1 -- 172.21.15.1:0/3152134482 >> 172.21.15.1:0/3152134482 conn(0x7f9f900788e0 msgr2=0x7f9f90078cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:04.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.747+0000 7f9f96c00700 1 -- 172.21.15.1:0/3152134482 shutdown_connections 2024-04-26T01:55:04.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.748+0000 7f9f96c00700 1 -- 172.21.15.1:0/3152134482 wait complete. 2024-04-26T01:55:04.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.750+0000 7f9f96c00700 1 Processor -- start 2024-04-26T01:55:04.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.750+0000 7f9f96c00700 1 -- start start 2024-04-26T01:55:04.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.750+0000 7f9f96c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f9006a740 0x7f9f90101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:04.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.750+0000 7f9f96c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f9006b100 0x7f9f90101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:04.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.750+0000 7f9f96c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f90108a60 0x7f9f90102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:04.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.750+0000 7f9f96c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9f90112cc0 con 0x7f9f9006b100 2024-04-26T01:55:04.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.750+0000 7f9f96c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9f90112b40 con 0x7f9f90108a60 2024-04-26T01:55:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.750+0000 7f9f96c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9f90112e40 con 0x7f9f9006a740 2024-04-26T01:55:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.751+0000 7f9f94c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f90108a60 0x7f9f90102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.751+0000 7f9f94c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f90108a60 0x7f9f90102490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:49716/0 (socket says 172.21.15.1:49716) 2024-04-26T01:55:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.751+0000 7f9f94c00700 1 -- 172.21.15.1:0/3306275132 learned_addr learned my addr 172.21.15.1:0/3306275132 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.751+0000 7f9f8fe00700 1 --2- 172.21.15.1:0/3306275132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f9006a740 0x7f9f90101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.751+0000 7f9f94c00700 1 -- 172.21.15.1:0/3306275132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f9006a740 msgr2=0x7f9f90101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.751+0000 7f9f94c00700 1 --2- 172.21.15.1:0/3306275132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f9006a740 0x7f9f90101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.751+0000 7f9f94c00700 1 -- 172.21.15.1:0/3306275132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f9006b100 msgr2=0x7f9f90101f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:55:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.752+0000 7f9f94c00700 1 --2- 172.21.15.1:0/3306275132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f9006b100 0x7f9f90101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.752+0000 7f9f94c00700 1 -- 172.21.15.1:0/3306275132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f901abac0 con 0x7f9f90108a60 2024-04-26T01:55:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.752+0000 7f9f94c00700 1 --2- 172.21.15.1:0/3306275132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f90108a60 0x7f9f90102490 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800ef30 tx=0x7f9f8800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.752+0000 7f9f8cc00700 1 -- 172.21.15.1:0/3306275132 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f880092e0 con 0x7f9f90108a60 2024-04-26T01:55:04.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.752+0000 7f9f8cc00700 1 -- 172.21.15.1:0/3306275132 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f88004bb0 con 0x7f9f90108a60 2024-04-26T01:55:04.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.752+0000 7f9f8cc00700 1 -- 172.21.15.1:0/3306275132 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88004280 con 0x7f9f90108a60 2024-04-26T01:55:04.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.753+0000 7f9f96c00700 1 -- 172.21.15.1:0/3306275132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f901abd50 con 0x7f9f90108a60 2024-04-26T01:55:04.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.753+0000 7f9f96c00700 1 -- 172.21.15.1:0/3306275132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f901ac1e0 con 0x7f9f90108a60 2024-04-26T01:55:04.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.758+0000 7f9f8cc00700 1 -- 172.21.15.1:0/3306275132 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f9f8801b030 con 0x7f9f90108a60 2024-04-26T01:55:04.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.758+0000 7f9fa14b6b80 1 Processor -- start 2024-04-26T01:55:04.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.759+0000 7f9fa14b6b80 1 -- start start 2024-04-26T01:55:04.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.759+0000 7f9fa14b6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22add4dd0 0x55c22add51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:04.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.759+0000 7f9fa14b6b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22add56f0 0x55c22ade0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:04.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.759+0000 7f9fa14b6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c22ade0560 0x55c22ade2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:04.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.759+0000 7f9fa14b6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c22ad28020 con 0x55c22ade0560 2024-04-26T01:55:04.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.759+0000 7f9fa14b6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c22acce8c0 con 0x55c22add4dd0 2024-04-26T01:55:04.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.759+0000 7f9fa14b6b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c22ad23fe0 con 0x55c22add56f0 2024-04-26T01:55:04.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.760+0000 7f9f8f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22add4dd0 0x55c22add51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:04.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.760+0000 7f9f8fe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22add56f0 0x55c22ade0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:04.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.760+0000 7f9f8f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22add4dd0 0x55c22add51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:49728/0 (socket says 172.21.15.1:49728) 2024-04-26T01:55:04.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.760+0000 7f9f8f400700 1 -- 172.21.15.1:0/4080265711 learned_addr learned my addr 172.21.15.1:0/4080265711 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:04.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.760+0000 7f9f8f400700 1 --2- 172.21.15.1:0/4080265711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c22ade0560 0x55c22ade2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:04.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.760+0000 7f9f8fe00700 1 -- 172.21.15.1:0/4080265711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22add4dd0 msgr2=0x55c22add51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:04.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.760+0000 7f9f8fe00700 1 --2- 172.21.15.1:0/4080265711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22add4dd0 0x55c22add51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.760+0000 7f9f8fe00700 1 -- 172.21.15.1:0/4080265711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c22ade0560 msgr2=0x55c22ade2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:04.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.760+0000 7f9f8fe00700 1 --2- 172.21.15.1:0/4080265711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c22ade0560 0x55c22ade2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.760+0000 7f9f8fe00700 1 -- 172.21.15.1:0/4080265711 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c22acc4810 con 0x55c22add56f0 2024-04-26T01:55:04.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.761+0000 7f9f8f400700 1 --2- 172.21.15.1:0/4080265711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22add4dd0 0x55c22add51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:55:04.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.761+0000 7f9f8fe00700 1 --2- 172.21.15.1:0/4080265711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22add56f0 0x55c22ade0020 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9f80000c00 tx=0x7f9f80005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=372d6204ac47a058 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:04.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.761+0000 7f9f8cc00700 1 --2- 172.21.15.1:0/3306275132 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f9f740a3ed0 0x7f9f740a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:04.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.761+0000 7f9f85600700 1 -- 172.21.15.1:0/4080265711 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80027070 con 0x55c22add56f0 2024-04-26T01:55:04.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.762+0000 7f9f8cc00700 1 -- 172.21.15.1:0/3306275132 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f9f880d02d0 con 0x7f9f90108a60 2024-04-26T01:55:04.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.762+0000 7f9f85600700 1 -- 172.21.15.1:0/4080265711 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f800056a0 con 0x55c22add56f0 2024-04-26T01:55:04.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.762+0000 7f9f85600700 1 -- 172.21.15.1:0/4080265711 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80003cc0 con 0x55c22add56f0 2024-04-26T01:55:04.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.762+0000 7f9f8f400700 1 --2- 172.21.15.1:0/3306275132 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f9f740a3ed0 0x7f9f740a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:04.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.763+0000 7f9f8f400700 1 --2- 172.21.15.1:0/3306275132 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f9f740a3ed0 0x7f9f740a6390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55c22add67a0 tx=0x7f9f7c001040 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:04.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.763+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/4080265711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22add56f0 msgr2=0x55c22ade0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:04.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.763+0000 7f9fa14b6b80 1 --2- 172.21.15.1:0/4080265711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22add56f0 0x55c22ade0020 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9f80000c00 tx=0x7f9f80005be0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.763+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/4080265711 shutdown_connections 2024-04-26T01:55:04.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.763+0000 7f9fa14b6b80 1 --2- 172.21.15.1:0/4080265711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c22ade0560 0x55c22ade2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.763+0000 7f9fa14b6b80 1 --2- 172.21.15.1:0/4080265711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22add4dd0 0x55c22add51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.763+0000 7f9fa14b6b80 1 --2- 172.21.15.1:0/4080265711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22add56f0 0x55c22ade0020 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.763+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/4080265711 >> 172.21.15.1:0/4080265711 conn(0x55c22ad040e0 msgr2=0x55c22add12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:04.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.763+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/4080265711 shutdown_connections 2024-04-26T01:55:04.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.763+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/4080265711 wait complete. 2024-04-26T01:55:04.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.764+0000 7f9fa14b6b80 1 -- start start 2024-04-26T01:55:04.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.764+0000 7f9fa14b6b80 1 Processor -- start 2024-04-26T01:55:04.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.765+0000 7f9fa14b6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22add4dd0 0x55c22addfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:04.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.765+0000 7f9fa14b6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c22add56f0 0x55c22addcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:04.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.765+0000 7f9fa14b6b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22ade0560 0x55c22addd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:04.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.765+0000 7f9fa14b6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c22adf4000 con 0x55c22add56f0 2024-04-26T01:55:04.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.765+0000 7f9fa14b6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c22ace19f0 con 0x55c22add4dd0 2024-04-26T01:55:04.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.765+0000 7f9fa14b6b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c22adf4180 con 0x55c22ade0560 2024-04-26T01:55:04.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.765+0000 7f9fa14b6b80 10 client.0 _ll_register_callbacks cb 0x55c22acc08b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:04.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.765+0000 7f9f8fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22add4dd0 0x55c22addfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:04.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.765+0000 7f9f8fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22add4dd0 0x55c22addfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:49742/0 (socket says 172.21.15.1:49742) 2024-04-26T01:55:04.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.765+0000 7f9f8fe00700 1 -- 172.21.15.1:0/4106555844 learned_addr learned my addr 172.21.15.1:0/4106555844 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:04.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.766+0000 7f9f94c00700 1 --2- 172.21.15.1:0/4106555844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22ade0560 0x55c22addd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:04.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.766+0000 7f9f8f400700 1 --2- 172.21.15.1:0/4106555844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c22add56f0 0x55c22addcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:04.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.766+0000 7f9f94c00700 1 -- 172.21.15.1:0/4106555844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22add4dd0 msgr2=0x55c22addfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:04.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.766+0000 7f9f94c00700 1 --2- 172.21.15.1:0/4106555844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22add4dd0 0x55c22addfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.766+0000 7f9f94c00700 1 -- 172.21.15.1:0/4106555844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c22add56f0 msgr2=0x55c22addcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:04.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.766+0000 7f9f94c00700 1 --2- 172.21.15.1:0/4106555844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c22add56f0 0x55c22addcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.766+0000 7f9f94c00700 1 -- 172.21.15.1:0/4106555844 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c22ac75a60 con 0x55c22ade0560 2024-04-26T01:55:04.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.767+0000 7f9f8fe00700 1 --2- 172.21.15.1:0/4106555844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22add4dd0 0x55c22addfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:04.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.767+0000 7f9f8f400700 1 --2- 172.21.15.1:0/4106555844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c22add56f0 0x55c22addcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:04.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.767+0000 7f9f94c00700 1 --2- 172.21.15.1:0/4106555844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22ade0560 0x55c22addd480 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9f880cda20 tx=0x7f9f8800cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=263e3fede8b5ef53 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:04.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.767+0000 7f9f7a000700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:55:04.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.767+0000 7f9f7a000700 1 -- 172.21.15.1:0/4106555844 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f880cf040 con 0x55c22ade0560 2024-04-26T01:55:04.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.767+0000 7f9f7a000700 1 -- 172.21.15.1:0/4106555844 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f9f880a4a50 con 0x55c22ade0560 2024-04-26T01:55:04.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.767+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/4106555844 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f800056a0 con 0x55c22ade0560 2024-04-26T01:55:04.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.768+0000 7f9f7a000700 1 -- 172.21.15.1:0/4106555844 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9f8809c300 con 0x55c22ade0560 2024-04-26T01:55:04.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.768+0000 7f9fa14b6b80 10 client.25627 fetch_fsmap learned FSMap version 30 2024-04-26T01:55:04.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.768+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/4106555844 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c22acbec80 con 0x55c22ade0560 2024-04-26T01:55:04.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.768+0000 7f9f7a000700 1 -- 172.21.15.1:0/4106555844 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f800056a0 con 0x55c22ade0560 2024-04-26T01:55:04.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.768+0000 7f9f7a000700 1 -- 172.21.15.1:0/4106555844 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9f880a0160 con 0x55c22ade0560 2024-04-26T01:55:04.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.769+0000 7f9fa14b6b80 10 client.25627 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:55:04.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.769+0000 7f9fa14b6b80 10 client.25627 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:55:04.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.769+0000 7f9fa14b6b80 20 client.25627 populate_metadata read hostname 'smithi001' 2024-04-26T01:55:04.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.769+0000 7f9fa14b6b80 1 --2- 172.21.15.1:0/4106555844 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c22ae00c60 0x55c22ae03070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:04.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.769+0000 7f9fa14b6b80 4 client.25627 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:55:04.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.769+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/4106555844 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c22ad35810 con 0x55c22ae00c60 2024-04-26T01:55:04.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.770+0000 7f9f8fe00700 1 --2- 172.21.15.1:0/4106555844 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c22ae00c60 0x55c22ae03070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:04.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.770+0000 7f9f8fe00700 1 --2- 172.21.15.1:0/4106555844 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c22ae00c60 0x55c22ae03070 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c126928b1226d3cb server_cookie=5c13f898d6001cd in_seq=0 out_seq=0 2024-04-26T01:55:04.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.770+0000 7f9f7a000700 10 client.25627 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:04.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.773+0000 7f9f7a000700 1 -- 172.21.15.1:0/4106555844 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c22ad35810 con 0x55c22ae00c60 2024-04-26T01:55:04.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.773+0000 7f9f7a000700 10 client.25627 handle_command_reply: tid=0 2024-04-26T01:55:04.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.773+0000 7f9fa14b6b80 1 client.25627 shutdown 2024-04-26T01:55:04.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.774+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/4106555844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22ade0560 msgr2=0x55c22addd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:04.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.774+0000 7f9fa14b6b80 1 --2- 172.21.15.1:0/4106555844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22ade0560 0x55c22addd480 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9f880cda20 tx=0x7f9f8800cf20 comp rx=0 tx=0).stop 2024-04-26T01:55:04.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.774+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/4106555844 shutdown_connections 2024-04-26T01:55:04.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.774+0000 7f9fa14b6b80 1 --2- 172.21.15.1:0/4106555844 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c22ae00c60 0x55c22ae03070 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.774+0000 7f9f7a000700 0 client.25627 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:04.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.774+0000 7f9fa14b6b80 1 --2- 172.21.15.1:0/4106555844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22ade0560 0x55c22addd480 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.774+0000 7f9fa14b6b80 1 --2- 172.21.15.1:0/4106555844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22add4dd0 0x55c22addfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.774+0000 7f9fa14b6b80 1 --2- 172.21.15.1:0/4106555844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c22add56f0 0x55c22addcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.774+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/4106555844 >> 172.21.15.1:0/4106555844 conn(0x55c22ad040e0 msgr2=0x55c22add0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:04.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.774+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/4106555844 shutdown_connections 2024-04-26T01:55:04.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.774+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/4106555844 wait complete. 2024-04-26T01:55:04.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.775+0000 7f9fa14b6b80 20 client.25627 trim_cache size 0 max 16384 2024-04-26T01:55:04.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.801+0000 7f9fa14b6b80 1 Processor -- start 2024-04-26T01:55:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.801+0000 7f9fa14b6b80 1 -- start start 2024-04-26T01:55:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.801+0000 7f9fa14b6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c22add4dd0 0x55c22addede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.801+0000 7f9fa14b6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22add56f0 0x55c22adfcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.801+0000 7f9fa14b6b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22ade0560 0x55c22adda220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.801+0000 7f9fa14b6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9f901ad8b0 con 0x55c22add4dd0 2024-04-26T01:55:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.801+0000 7f9fa14b6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9f901a4760 con 0x55c22add56f0 2024-04-26T01:55:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.801+0000 7f9fa14b6b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9f901a25b0 con 0x55c22ade0560 2024-04-26T01:55:04.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.802+0000 7f9f8f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22add56f0 0x55c22adfcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.802+0000 7f9f8f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22add56f0 0x55c22adfcf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:49748/0 (socket says 172.21.15.1:49748) 2024-04-26T01:55:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.802+0000 7f9f8f400700 1 -- 172.21.15.1:0/1181658776 learned_addr learned my addr 172.21.15.1:0/1181658776 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.802+0000 7f9f94c00700 1 --2- 172.21.15.1:0/1181658776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22ade0560 0x55c22adda220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.802+0000 7f9f8f400700 1 -- 172.21.15.1:0/1181658776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22ade0560 msgr2=0x55c22adda220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.802+0000 7f9f8f400700 1 --2- 172.21.15.1:0/1181658776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22ade0560 0x55c22adda220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.802+0000 7f9f8f400700 1 -- 172.21.15.1:0/1181658776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c22add4dd0 msgr2=0x55c22addede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:04.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.802+0000 7f9f8f400700 1 --2- 172.21.15.1:0/1181658776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c22add4dd0 0x55c22addede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.802+0000 7f9f8f400700 1 -- 172.21.15.1:0/1181658776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c22add5d70 con 0x55c22add56f0 2024-04-26T01:55:04.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.803+0000 7f9f8f400700 1 --2- 172.21.15.1:0/1181658776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22add56f0 0x55c22adfcf30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009fd0 tx=0x7f9f7c007040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be1d93d7973714a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:04.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.803+0000 7f9f84c00700 1 -- 172.21.15.1:0/1181658776 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c012040 con 0x55c22add56f0 2024-04-26T01:55:04.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.803+0000 7f9f84c00700 1 -- 172.21.15.1:0/1181658776 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f7c027070 con 0x55c22add56f0 2024-04-26T01:55:04.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.803+0000 7f9f84c00700 1 -- 172.21.15.1:0/1181658776 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c016c50 con 0x55c22add56f0 2024-04-26T01:55:04.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.804+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/1181658776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22add56f0 msgr2=0x55c22adfcf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:04.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.804+0000 7f9fa14b6b80 1 --2- 172.21.15.1:0/1181658776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22add56f0 0x55c22adfcf30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009fd0 tx=0x7f9f7c007040 comp rx=0 tx=0).stop 2024-04-26T01:55:04.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.804+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/1181658776 shutdown_connections 2024-04-26T01:55:04.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.804+0000 7f9fa14b6b80 1 --2- 172.21.15.1:0/1181658776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22ade0560 0x55c22adda220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.804+0000 7f9fa14b6b80 1 --2- 172.21.15.1:0/1181658776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c22add4dd0 0x55c22addede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.804+0000 7f9fa14b6b80 1 --2- 172.21.15.1:0/1181658776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22add56f0 0x55c22adfcf30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.804+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/1181658776 >> 172.21.15.1:0/1181658776 conn(0x55c22ad040e0 msgr2=0x55c22add0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:04.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.804+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/1181658776 shutdown_connections 2024-04-26T01:55:04.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.805+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/1181658776 wait complete. 2024-04-26T01:55:04.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.805+0000 7f9fa14b6b80 1 -- start start 2024-04-26T01:55:04.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.805+0000 7f9fa14b6b80 1 Processor -- start 2024-04-26T01:55:04.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.806+0000 7f9fa14b6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c22add4dd0 0x55c22adf65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:04.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.806+0000 7f9fa14b6b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22add56f0 0x55c22adf6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:04.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.806+0000 7f9fa14b6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22ade0560 0x55c22ae671c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:04.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.806+0000 7f9fa14b6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9f901a2c20 con 0x55c22add4dd0 2024-04-26T01:55:04.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.806+0000 7f9fa14b6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9f901a4090 con 0x55c22ade0560 2024-04-26T01:55:04.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.806+0000 7f9fa14b6b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9f901a32f0 con 0x55c22add56f0 2024-04-26T01:55:04.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.806+0000 7f9fa14b6b80 10 client.0 _ll_register_callbacks cb 0x55c22acc08b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:04.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.807+0000 7f9f94c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22ade0560 0x55c22ae671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:04.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.807+0000 7f9f8f400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22add56f0 0x55c22adf6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:04.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.807+0000 7f9f8f400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22add56f0 0x55c22adf6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:43704/0 (socket says 172.21.15.1:43704) 2024-04-26T01:55:04.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.807+0000 7f9f94c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22ade0560 0x55c22ae671c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:49764/0 (socket says 172.21.15.1:49764) 2024-04-26T01:55:04.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.807+0000 7f9f8f400700 1 -- 172.21.15.1:0/1257553736 learned_addr learned my addr 172.21.15.1:0/1257553736 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:04.811 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] open_by_handle 2024-04-26T01:55:04.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.807+0000 7f9f8fe00700 1 --2- 172.21.15.1:0/1257553736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c22add4dd0 0x55c22adf65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:04.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.807+0000 7f9f94c00700 1 -- 172.21.15.1:0/1257553736 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22add56f0 msgr2=0x55c22adf6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:04.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.807+0000 7f9f94c00700 1 --2- 172.21.15.1:0/1257553736 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22add56f0 0x55c22adf6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.807+0000 7f9f94c00700 1 -- 172.21.15.1:0/1257553736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c22add4dd0 msgr2=0x55c22adf65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.807+0000 7f9f94c00700 1 --2- 172.21.15.1:0/1257553736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c22add4dd0 0x55c22adf65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.807+0000 7f9f94c00700 1 -- 172.21.15.1:0/1257553736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c22add28c0 con 0x55c22ade0560 2024-04-26T01:55:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.808+0000 7f9f94c00700 1 --2- 172.21.15.1:0/1257553736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22ade0560 0x55c22ae671c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9f8809a5b0 tx=0x7f9f88004f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f72072a48a0f20ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.808+0000 7f9f7a000700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:55:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.808+0000 7f9f7a000700 1 -- 172.21.15.1:0/1257553736 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f880cf070 con 0x55c22ade0560 2024-04-26T01:55:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.808+0000 7f9f7a000700 1 -- 172.21.15.1:0/1257553736 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f9f880a4c60 con 0x55c22ade0560 2024-04-26T01:55:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.808+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/1257553736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f7c027070 con 0x55c22ade0560 2024-04-26T01:55:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.808+0000 7f9f7a000700 1 -- 172.21.15.1:0/1257553736 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9f8809c0d0 con 0x55c22ade0560 2024-04-26T01:55:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.808+0000 7f9fa14b6b80 10 client.25613 fetch_fsmap learned FSMap version 30 2024-04-26T01:55:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.808+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/1257553736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c22ae68750 con 0x55c22ade0560 2024-04-26T01:55:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.809+0000 7f9f7a000700 1 -- 172.21.15.1:0/1257553736 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f7c027070 con 0x55c22ade0560 2024-04-26T01:55:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.809+0000 7f9f7a000700 1 -- 172.21.15.1:0/1257553736 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9f8809b2e0 con 0x55c22ade0560 2024-04-26T01:55:04.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.809+0000 7f9fa14b6b80 10 client.25613 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:55:04.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.809+0000 7f9fa14b6b80 10 client.25613 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:55:04.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.809+0000 7f9fa14b6b80 20 client.25613 populate_metadata read hostname 'smithi001' 2024-04-26T01:55:04.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.809+0000 7f9fa14b6b80 1 --2- 172.21.15.1:0/1257553736 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c22ae00c60 0x55c22ae6ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:04.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.809+0000 7f9fa14b6b80 4 client.25613 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-04-26T01:55:04.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.809+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/1257553736 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x55c22ad2f8b0 con 0x55c22ae00c60 2024-04-26T01:55:04.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.809+0000 7f9f8fe00700 1 --2- 172.21.15.1:0/1257553736 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c22ae00c60 0x55c22ae6ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:04.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.810+0000 7f9f8fe00700 1 --2- 172.21.15.1:0/1257553736 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c22ae00c60 0x55c22ae6ae40 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb0d13ee42fea218 server_cookie=e77ec5a8c9ce1bcc in_seq=0 out_seq=0 2024-04-26T01:55:04.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:04.810+0000 7f9f7a000700 10 client.25613 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.855+0000 7fdaa6400640 1 -- 172.21.15.1:0/4105600723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaa0103190 msgr2=0x7fdaa0103570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.855+0000 7fdaa6400640 1 --2- 172.21.15.1:0/4105600723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaa0103190 0x7fdaa0103570 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fda90009900 tx=0x7fda9002a480 comp rx=0 tx=0).stop 2024-04-26T01:55:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.855+0000 7fdaa6400640 1 -- 172.21.15.1:0/4105600723 shutdown_connections 2024-04-26T01:55:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.855+0000 7fdaa6400640 1 --2- 172.21.15.1:0/4105600723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaa010c5b0 0x7fdaa010ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.855+0000 7fdaa6400640 1 --2- 172.21.15.1:0/4105600723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaa0103ab0 0x7fdaa010bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.855+0000 7fdaa6400640 1 --2- 172.21.15.1:0/4105600723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaa0103190 0x7fdaa0103570 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.855+0000 7fdaa6400640 1 -- 172.21.15.1:0/4105600723 >> 172.21.15.1:0/4105600723 conn(0x7fdaa00fc970 msgr2=0x7fdaa00fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.856+0000 7fdaa6400640 1 -- 172.21.15.1:0/4105600723 shutdown_connections 2024-04-26T01:55:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.856+0000 7fdaa6400640 1 -- 172.21.15.1:0/4105600723 wait complete. 2024-04-26T01:55:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.857+0000 7fdaa6400640 1 Processor -- start 2024-04-26T01:55:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.857+0000 7fdaa6400640 1 -- start start 2024-04-26T01:55:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.858+0000 7fdaa6400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaa0103190 0x7fdaa0100de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.858+0000 7fdaa6400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaa0103ab0 0x7fdaa0101320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.858+0000 7fdaa6400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaa010c5b0 0x7fdaa0101860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.858+0000 7fdaa6400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa0112290 con 0x7fdaa0103ab0 2024-04-26T01:55:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.858+0000 7fdaa6400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa0112110 con 0x7fdaa0103190 2024-04-26T01:55:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.858+0000 7fdaa6400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa0112410 con 0x7fdaa010c5b0 2024-04-26T01:55:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.858+0000 7fda9f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaa0103190 0x7fdaa0100de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.858+0000 7fda9f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaa0103190 0x7fdaa0100de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:49786/0 (socket says 172.21.15.1:49786) 2024-04-26T01:55:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.858+0000 7fda9f400640 1 -- 172.21.15.1:0/3902706028 learned_addr learned my addr 172.21.15.1:0/3902706028 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.858+0000 7fda9fe00640 1 --2- 172.21.15.1:0/3902706028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaa010c5b0 0x7fdaa0101860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.858+0000 7fda9ea00640 1 --2- 172.21.15.1:0/3902706028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaa0103ab0 0x7fdaa0101320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.858+0000 7fda9f400640 1 -- 172.21.15.1:0/3902706028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaa010c5b0 msgr2=0x7fdaa0101860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.858+0000 7fda9f400640 1 --2- 172.21.15.1:0/3902706028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaa010c5b0 0x7fdaa0101860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.858+0000 7fda9f400640 1 -- 172.21.15.1:0/3902706028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaa0103ab0 msgr2=0x7fdaa0101320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.858+0000 7fda9f400640 1 --2- 172.21.15.1:0/3902706028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaa0103ab0 0x7fdaa0101320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.858+0000 7fda9f400640 1 -- 172.21.15.1:0/3902706028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa01a6d10 con 0x7fdaa0103190 2024-04-26T01:55:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.859+0000 7fda9f400640 1 --2- 172.21.15.1:0/3902706028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaa0103190 0x7fdaa0100de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fda90005ab0 tx=0x7fda90003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.859+0000 7fda97e00640 1 -- 172.21.15.1:0/3902706028 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9003f070 con 0x7fdaa0103190 2024-04-26T01:55:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.859+0000 7fda97e00640 1 -- 172.21.15.1:0/3902706028 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda90031d60 con 0x7fdaa0103190 2024-04-26T01:55:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.859+0000 7fda97e00640 1 -- 172.21.15.1:0/3902706028 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda90035030 con 0x7fdaa0103190 2024-04-26T01:55:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.859+0000 7fdaa6400640 1 -- 172.21.15.1:0/3902706028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa01a6fa0 con 0x7fdaa0103190 2024-04-26T01:55:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.859+0000 7fdaa6400640 1 -- 172.21.15.1:0/3902706028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa01a7410 con 0x7fdaa0103190 2024-04-26T01:55:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.863+0000 7fda97e00640 1 -- 172.21.15.1:0/3902706028 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fda90042070 con 0x7fdaa0103190 2024-04-26T01:55:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.864+0000 7fdaa9190740 1 Processor -- start 2024-04-26T01:55:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.864+0000 7fdaa9190740 1 -- start start 2024-04-26T01:55:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.865+0000 7fdaa9190740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233b9820 0x562b233b9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.865+0000 7fdaa9190740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233ba140 0x562b233c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.865+0000 7fdaa9190740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233c4a00 0x562b233c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.865+0000 7fdaa9190740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562b2331f900 con 0x562b233c4a00 2024-04-26T01:55:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.865+0000 7fdaa9190740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562b232c5ef0 con 0x562b233ba140 2024-04-26T01:55:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.865+0000 7fdaa9190740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562b2331b8c0 con 0x562b233b9820 2024-04-26T01:55:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.865+0000 7fda9ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233c4a00 0x562b233c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.865+0000 7fda9f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233ba140 0x562b233c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.865+0000 7fda9fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233b9820 0x562b233b9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.865+0000 7fda9ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233c4a00 0x562b233c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34362/0 (socket says 172.21.15.1:34362) 2024-04-26T01:55:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.865+0000 7fda9fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233b9820 0x562b233b9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:43746/0 (socket says 172.21.15.1:43746) 2024-04-26T01:55:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.865+0000 7fda9ea00640 1 -- 172.21.15.1:0/2947305979 learned_addr learned my addr 172.21.15.1:0/2947305979 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.866+0000 7fda97e00640 1 --2- 172.21.15.1:0/3902706028 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fda840a3d90 0x7fda840a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.866+0000 7fda9fe00640 1 -- 172.21.15.1:0/2947305979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233ba140 msgr2=0x562b233c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.866+0000 7fda9fe00640 1 --2- 172.21.15.1:0/2947305979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233ba140 0x562b233c44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.866+0000 7fda9fe00640 1 -- 172.21.15.1:0/2947305979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233c4a00 msgr2=0x562b233c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.866+0000 7fda9fe00640 1 --2- 172.21.15.1:0/2947305979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233c4a00 0x562b233c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.866+0000 7fda9fe00640 1 -- 172.21.15.1:0/2947305979 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b23032210 con 0x562b233b9820 2024-04-26T01:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.866+0000 7fda9ea00640 1 --2- 172.21.15.1:0/2947305979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233c4a00 0x562b233c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.866+0000 7fda97e00640 1 -- 172.21.15.1:0/3902706028 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fda900f0fa0 con 0x7fdaa0103190 2024-04-26T01:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.866+0000 7fda9fe00640 1 --2- 172.21.15.1:0/3902706028 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fda840a3d90 0x7fda840a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.866+0000 7fda9fe00640 1 --2- 172.21.15.1:0/2947305979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233b9820 0x562b233b9c00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fda98005f10 tx=0x7fda98010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4efc20a359967161 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.867+0000 7fda94c00640 1 -- 172.21.15.1:0/2947305979 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda980130d0 con 0x562b233b9820 2024-04-26T01:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.867+0000 7fda9fe00640 1 --2- 172.21.15.1:0/3902706028 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fda840a3d90 0x7fda840a6250 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fda98021000 tx=0x7fda9801e000 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.867+0000 7fda94c00640 1 -- 172.21.15.1:0/2947305979 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda9801ddc0 con 0x562b233b9820 2024-04-26T01:55:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.867+0000 7fda94c00640 1 -- 172.21.15.1:0/2947305979 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98011c30 con 0x562b233b9820 2024-04-26T01:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.868+0000 7fdaa9190740 1 -- 172.21.15.1:0/2947305979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233b9820 msgr2=0x562b233b9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.868+0000 7fdaa9190740 1 --2- 172.21.15.1:0/2947305979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233b9820 0x562b233b9c00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fda98005f10 tx=0x7fda98010860 comp rx=0 tx=0).stop 2024-04-26T01:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.868+0000 7fdaa9190740 1 -- 172.21.15.1:0/2947305979 shutdown_connections 2024-04-26T01:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.868+0000 7fdaa9190740 1 --2- 172.21.15.1:0/2947305979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233c4a00 0x562b233c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.868+0000 7fdaa9190740 1 --2- 172.21.15.1:0/2947305979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233ba140 0x562b233c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.868+0000 7fdaa9190740 1 --2- 172.21.15.1:0/2947305979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233b9820 0x562b233b9c00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.868+0000 7fdaa9190740 1 -- 172.21.15.1:0/2947305979 >> 172.21.15.1:0/2947305979 conn(0x562b233b5d30 msgr2=0x562b233b6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.868+0000 7fdaa9190740 1 -- 172.21.15.1:0/2947305979 shutdown_connections 2024-04-26T01:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.868+0000 7fdaa9190740 1 -- 172.21.15.1:0/2947305979 wait complete. 2024-04-26T01:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.868+0000 7fdaa9190740 1 -- start start 2024-04-26T01:55:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.869+0000 7fdaa9190740 1 Processor -- start 2024-04-26T01:55:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.869+0000 7fdaa9190740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233b9820 0x562b233c1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.869+0000 7fdaa9190740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233ba140 0x562b233c1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.869+0000 7fdaa9190740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233c4a00 0x562b233c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.869+0000 7fdaa9190740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562b233c7630 con 0x562b233c4a00 2024-04-26T01:55:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.869+0000 7fdaa9190740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562b232d9310 con 0x562b233b9820 2024-04-26T01:55:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.869+0000 7fdaa9190740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562b233c77b0 con 0x562b233ba140 2024-04-26T01:55:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.869+0000 7fdaa9190740 10 client.0 _ll_register_callbacks cb 0x562b232a7cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.869+0000 7fda9fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233c4a00 0x562b233c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.869+0000 7fda9fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233c4a00 0x562b233c24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34372/0 (socket says 172.21.15.1:34372) 2024-04-26T01:55:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.869+0000 7fda9fe00640 1 -- 172.21.15.1:0/1198262148 learned_addr learned my addr 172.21.15.1:0/1198262148 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.869+0000 7fda9f400640 1 --2- 172.21.15.1:0/1198262148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233b9820 0x562b233c1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:05.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.869+0000 7fda9ea00640 1 --2- 172.21.15.1:0/1198262148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233ba140 0x562b233c1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.870+0000 7fda9f400640 1 -- 172.21.15.1:0/1198262148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233ba140 msgr2=0x562b233c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.870+0000 7fda9f400640 1 --2- 172.21.15.1:0/1198262148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233ba140 0x562b233c1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.870+0000 7fda9f400640 1 -- 172.21.15.1:0/1198262148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233c4a00 msgr2=0x562b233c24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.870+0000 7fda9f400640 1 --2- 172.21.15.1:0/1198262148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233c4a00 0x562b233c24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.870+0000 7fda9f400640 1 -- 172.21.15.1:0/1198262148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b230fe620 con 0x562b233b9820 2024-04-26T01:55:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.870+0000 7fda9fe00640 1 --2- 172.21.15.1:0/1198262148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233c4a00 0x562b233c24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.870+0000 7fda9f400640 1 --2- 172.21.15.1:0/1198262148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233b9820 0x562b233c1a30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda900ee690 tx=0x7fda90009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=556f1db10e518665 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.870+0000 7fda89600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:55:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.870+0000 7fda89600640 1 -- 172.21.15.1:0/1198262148 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda900eeb10 con 0x562b233b9820 2024-04-26T01:55:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.870+0000 7fda89600640 1 -- 172.21.15.1:0/1198262148 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fda900bd140 con 0x562b233b9820 2024-04-26T01:55:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.870+0000 7fdaa9190740 1 -- 172.21.15.1:0/1198262148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda9801ddc0 con 0x562b233b9820 2024-04-26T01:55:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.870+0000 7fda89600640 1 -- 172.21.15.1:0/1198262148 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fda900bd5e0 con 0x562b233b9820 2024-04-26T01:55:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.871+0000 7fdaa9190740 10 client.25625 fetch_fsmap learned FSMap version 30 2024-04-26T01:55:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.871+0000 7fdaa9190740 1 -- 172.21.15.1:0/1198262148 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562b233d3920 con 0x562b233b9820 2024-04-26T01:55:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.871+0000 7fda89600640 1 -- 172.21.15.1:0/1198262148 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda9801ddc0 con 0x562b233b9820 2024-04-26T01:55:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.871+0000 7fda89600640 1 -- 172.21.15.1:0/1198262148 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fda900d6270 con 0x562b233b9820 2024-04-26T01:55:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.871+0000 7fdaa9190740 10 client.25625 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:55:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.871+0000 7fdaa9190740 10 client.25625 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:55:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.871+0000 7fdaa9190740 20 client.25625 populate_metadata read hostname 'smithi001' 2024-04-26T01:55:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.871+0000 7fdaa9190740 1 --2- 172.21.15.1:0/1198262148 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562b233d4e70 0x562b233d72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.871+0000 7fdaa9190740 4 client.25625 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:55:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.871+0000 7fdaa9190740 1 -- 172.21.15.1:0/1198262148 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b233338b0 con 0x562b233d4e70 2024-04-26T01:55:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.871+0000 7fda9ea00640 1 --2- 172.21.15.1:0/1198262148 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562b233d4e70 0x562b233d72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.872+0000 7fda9ea00640 1 --2- 172.21.15.1:0/1198262148 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562b233d4e70 0x562b233d72b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26f27b57d736190a server_cookie=b051edee6472947e in_seq=0 out_seq=0 2024-04-26T01:55:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.872+0000 7fda89600640 10 client.25625 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.873+0000 7fda89600640 1 -- 172.21.15.1:0/1198262148 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b233338b0 con 0x562b233d4e70 2024-04-26T01:55:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.873+0000 7fda89600640 10 client.25625 handle_command_reply: tid=0 2024-04-26T01:55:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.873+0000 7fdaa9190740 1 client.25625 shutdown 2024-04-26T01:55:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.873+0000 7fdaa9190740 1 -- 172.21.15.1:0/1198262148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233b9820 msgr2=0x562b233c1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.873+0000 7fdaa9190740 1 --2- 172.21.15.1:0/1198262148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233b9820 0x562b233c1a30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda900ee690 tx=0x7fda90009620 comp rx=0 tx=0).stop 2024-04-26T01:55:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.873+0000 7fdaa9190740 1 -- 172.21.15.1:0/1198262148 shutdown_connections 2024-04-26T01:55:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.873+0000 7fdaa9190740 1 --2- 172.21.15.1:0/1198262148 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562b233d4e70 0x562b233d72b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.873+0000 7fdaa9190740 1 --2- 172.21.15.1:0/1198262148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233c4a00 0x562b233c24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.873+0000 7fdaa9190740 1 --2- 172.21.15.1:0/1198262148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233ba140 0x562b233c1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.873+0000 7fdaa9190740 1 --2- 172.21.15.1:0/1198262148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233b9820 0x562b233c1a30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.873+0000 7fda89600640 0 client.25625 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.873+0000 7fdaa9190740 1 -- 172.21.15.1:0/1198262148 >> 172.21.15.1:0/1198262148 conn(0x562b233b5d30 msgr2=0x562b233bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.873+0000 7fdaa9190740 1 -- 172.21.15.1:0/1198262148 shutdown_connections 2024-04-26T01:55:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.873+0000 7fdaa9190740 1 -- 172.21.15.1:0/1198262148 wait complete. 2024-04-26T01:55:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.873+0000 7fdaa9190740 20 client.25625 trim_cache size 0 max 16384 2024-04-26T01:55:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.889+0000 7fdaa9190740 1 Processor -- start 2024-04-26T01:55:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.889+0000 7fdaa9190740 1 -- start start 2024-04-26T01:55:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.889+0000 7fdaa9190740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233b9820 0x562b233d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.889+0000 7fdaa9190740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233ba140 0x562b233c22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.889+0000 7fdaa9190740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233c4a00 0x562b233dcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.889+0000 7fdaa9190740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa01b2760 con 0x562b233b9820 2024-04-26T01:55:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.889+0000 7fdaa9190740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa01affd0 con 0x562b233c4a00 2024-04-26T01:55:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.889+0000 7fdaa9190740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa01ade20 con 0x562b233ba140 2024-04-26T01:55:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.890+0000 7fda9f400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233b9820 0x562b233d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.890+0000 7fda9f400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233b9820 0x562b233d04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34376/0 (socket says 172.21.15.1:34376) 2024-04-26T01:55:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.890+0000 7fda9f400640 1 -- 172.21.15.1:0/3464330706 learned_addr learned my addr 172.21.15.1:0/3464330706 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.890+0000 7fda9fe00640 1 --2- 172.21.15.1:0/3464330706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233c4a00 0x562b233dcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.890+0000 7fda9ea00640 1 --2- 172.21.15.1:0/3464330706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233ba140 0x562b233c22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.890+0000 7fda9fe00640 1 -- 172.21.15.1:0/3464330706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233ba140 msgr2=0x562b233c22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.890+0000 7fda9fe00640 1 --2- 172.21.15.1:0/3464330706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233ba140 0x562b233c22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.890+0000 7fda9fe00640 1 -- 172.21.15.1:0/3464330706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233b9820 msgr2=0x562b233d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.890+0000 7fda9fe00640 1 --2- 172.21.15.1:0/3464330706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233b9820 0x562b233d04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.890+0000 7fda9fe00640 1 -- 172.21.15.1:0/3464330706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b233d7930 con 0x562b233c4a00 2024-04-26T01:55:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.890+0000 7fda9ea00640 1 --2- 172.21.15.1:0/3464330706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233ba140 0x562b233c22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.890+0000 7fda9f400640 1 --2- 172.21.15.1:0/3464330706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233b9820 0x562b233d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.890+0000 7fda9fe00640 1 --2- 172.21.15.1:0/3464330706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233c4a00 0x562b233dcbc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fda980117e0 tx=0x7fda98015fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73191c3f1eb82584 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.891+0000 7fda8be00640 1 -- 172.21.15.1:0/3464330706 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98013050 con 0x562b233c4a00 2024-04-26T01:55:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.891+0000 7fda8be00640 1 -- 172.21.15.1:0/3464330706 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda98004e30 con 0x562b233c4a00 2024-04-26T01:55:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.891+0000 7fda8be00640 1 -- 172.21.15.1:0/3464330706 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98011c80 con 0x562b233c4a00 2024-04-26T01:55:05.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.891+0000 7fdaa9190740 1 -- 172.21.15.1:0/3464330706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233c4a00 msgr2=0x562b233dcbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.891+0000 7fdaa9190740 1 --2- 172.21.15.1:0/3464330706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233c4a00 0x562b233dcbc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fda980117e0 tx=0x7fda98015fd0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.891+0000 7fdaa9190740 1 -- 172.21.15.1:0/3464330706 shutdown_connections 2024-04-26T01:55:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.891+0000 7fdaa9190740 1 --2- 172.21.15.1:0/3464330706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233c4a00 0x562b233dcbc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.891+0000 7fdaa9190740 1 --2- 172.21.15.1:0/3464330706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233ba140 0x562b233c22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.891+0000 7fdaa9190740 1 --2- 172.21.15.1:0/3464330706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233b9820 0x562b233d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.891+0000 7fdaa9190740 1 -- 172.21.15.1:0/3464330706 >> 172.21.15.1:0/3464330706 conn(0x562b233b5d30 msgr2=0x562b233b4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.892+0000 7fdaa9190740 1 -- 172.21.15.1:0/3464330706 shutdown_connections 2024-04-26T01:55:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.892+0000 7fdaa9190740 1 -- 172.21.15.1:0/3464330706 wait complete. 2024-04-26T01:55:05.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.892+0000 7fdaa9190740 1 -- start start 2024-04-26T01:55:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.892+0000 7fdaa9190740 1 Processor -- start 2024-04-26T01:55:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.893+0000 7fdaa9190740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233b9820 0x562b233ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.893+0000 7fdaa9190740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233ba140 0x562b233cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.893+0000 7fdaa9190740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233c4a00 0x562b2343a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.893+0000 7fdaa9190740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa01ae490 con 0x562b233c4a00 2024-04-26T01:55:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.893+0000 7fdaa9190740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa01af900 con 0x562b233b9820 2024-04-26T01:55:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.893+0000 7fdaa9190740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa01aeb60 con 0x562b233ba140 2024-04-26T01:55:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.893+0000 7fdaa9190740 10 client.0 _ll_register_callbacks cb 0x7fda900bcd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.893+0000 7fda9f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233b9820 0x562b233ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.893+0000 7fda9fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233c4a00 0x562b2343a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.893+0000 7fda9f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233b9820 0x562b233ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:49828/0 (socket says 172.21.15.1:49828) 2024-04-26T01:55:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.893+0000 7fda9f400640 1 -- 172.21.15.1:0/3707117807 learned_addr learned my addr 172.21.15.1:0/3707117807 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.893+0000 7fda9ea00640 1 --2- 172.21.15.1:0/3707117807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233ba140 0x562b233cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.893+0000 7fda9f400640 1 -- 172.21.15.1:0/3707117807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233ba140 msgr2=0x562b233cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.893+0000 7fda9f400640 1 --2- 172.21.15.1:0/3707117807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233ba140 0x562b233cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.893+0000 7fda9f400640 1 -- 172.21.15.1:0/3707117807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233c4a00 msgr2=0x562b2343a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.893+0000 7fda9f400640 1 --2- 172.21.15.1:0/3707117807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233c4a00 0x562b2343a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.893+0000 7fda9f400640 1 -- 172.21.15.1:0/3707117807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b233cb5f0 con 0x562b233b9820 2024-04-26T01:55:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.894+0000 7fda9f400640 1 --2- 172.21.15.1:0/3707117807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233b9820 0x562b233ca7b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fda900b98b0 tx=0x7fda900eeb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7dff793c2711695 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.894+0000 7fda89600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:55:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.894+0000 7fda89600640 1 -- 172.21.15.1:0/3707117807 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda900bcee0 con 0x562b233b9820 2024-04-26T01:55:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.894+0000 7fda89600640 1 -- 172.21.15.1:0/3707117807 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fda900bd350 con 0x562b233b9820 2024-04-26T01:55:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.894+0000 7fdaa9190740 1 -- 172.21.15.1:0/3707117807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda98004e30 con 0x562b233b9820 2024-04-26T01:55:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.895+0000 7fda89600640 1 -- 172.21.15.1:0/3707117807 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fda900c1290 con 0x562b233b9820 2024-04-26T01:55:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.895+0000 7fdaa9190740 10 client.25631 fetch_fsmap learned FSMap version 30 2024-04-26T01:55:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.895+0000 7fdaa9190740 1 -- 172.21.15.1:0/3707117807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562b2343bac0 con 0x562b233b9820 2024-04-26T01:55:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.895+0000 7fda89600640 1 -- 172.21.15.1:0/3707117807 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda98004e30 con 0x562b233b9820 2024-04-26T01:55:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.895+0000 7fda89600640 1 -- 172.21.15.1:0/3707117807 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fda900b8db0 con 0x562b233b9820 2024-04-26T01:55:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.895+0000 7fdaa9190740 10 client.25631 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:55:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.895+0000 7fdaa9190740 10 client.25631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:55:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.895+0000 7fdaa9190740 20 client.25631 populate_metadata read hostname 'smithi001' 2024-04-26T01:55:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.895+0000 7fdaa9190740 1 --2- 172.21.15.1:0/3707117807 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562b2343c0b0 0x562b2343e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.895+0000 7fdaa9190740 4 client.25631 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:55:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.895+0000 7fdaa9190740 1 -- 172.21.15.1:0/3707117807 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b2332d320 con 0x562b2343c0b0 2024-04-26T01:55:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.896+0000 7fda9ea00640 1 --2- 172.21.15.1:0/3707117807 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562b2343c0b0 0x562b2343e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.896+0000 7fda9ea00640 1 --2- 172.21.15.1:0/3707117807 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562b2343c0b0 0x562b2343e4d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6091f13d98a3df94 server_cookie=2e0db13251a2c2a3 in_seq=0 out_seq=0 2024-04-26T01:55:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:05.896+0000 7fda89600640 10 client.25631 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.005+0000 7fda89600640 1 -- 172.21.15.1:0/3707117807 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562b2332d320 con 0x562b2343c0b0 2024-04-26T01:55:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.005+0000 7fda89600640 10 client.25631 handle_command_reply: tid=0 2024-04-26T01:55:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.005+0000 7fdaa9190740 1 client.25631 shutdown 2024-04-26T01:55:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.005+0000 7fdaa9190740 1 -- 172.21.15.1:0/3707117807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233b9820 msgr2=0x562b233ca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.005+0000 7fdaa9190740 1 --2- 172.21.15.1:0/3707117807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233b9820 0x562b233ca7b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fda900b98b0 tx=0x7fda900eeb40 comp rx=0 tx=0).stop 2024-04-26T01:55:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.005+0000 7fdaa9190740 1 -- 172.21.15.1:0/3707117807 shutdown_connections 2024-04-26T01:55:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.005+0000 7fdaa9190740 1 --2- 172.21.15.1:0/3707117807 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562b2343c0b0 0x562b2343e4d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.005+0000 7fdaa9190740 1 --2- 172.21.15.1:0/3707117807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b233c4a00 0x562b2343a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.005+0000 7fdaa9190740 1 --2- 172.21.15.1:0/3707117807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b233ba140 0x562b233cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.005+0000 7fdaa9190740 1 --2- 172.21.15.1:0/3707117807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b233b9820 0x562b233ca7b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.005+0000 7fda89600640 0 client.25631 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.005+0000 7fdaa9190740 1 -- 172.21.15.1:0/3707117807 >> 172.21.15.1:0/3707117807 conn(0x562b233b5d30 msgr2=0x562b233bb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.005+0000 7fdaa9190740 1 -- 172.21.15.1:0/3707117807 shutdown_connections 2024-04-26T01:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.006+0000 7fdaa9190740 1 -- 172.21.15.1:0/3707117807 wait complete. 2024-04-26T01:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.006+0000 7fdaa9190740 20 client.25631 trim_cache size 0 max 16384 2024-04-26T01:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:55:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:55:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.006+0000 7fdaa6400640 1 -- 172.21.15.1:0/3902706028 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fda840a3d90 msgr2=0x7fda840a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.006+0000 7fdaa6400640 1 --2- 172.21.15.1:0/3902706028 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fda840a3d90 0x7fda840a6250 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fda98021000 tx=0x7fda9801e000 comp rx=0 tx=0).stop 2024-04-26T01:55:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.006+0000 7fdaa6400640 1 -- 172.21.15.1:0/3902706028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaa0103190 msgr2=0x7fdaa0100de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.006+0000 7fdaa6400640 1 --2- 172.21.15.1:0/3902706028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaa0103190 0x7fdaa0100de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fda90005ab0 tx=0x7fda90003c80 comp rx=0 tx=0).stop 2024-04-26T01:55:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.007+0000 7fdaa6400640 1 -- 172.21.15.1:0/3902706028 shutdown_connections 2024-04-26T01:55:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.007+0000 7fdaa6400640 1 --2- 172.21.15.1:0/3902706028 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fda840a3d90 0x7fda840a6250 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.007+0000 7fdaa6400640 1 --2- 172.21.15.1:0/3902706028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaa010c5b0 0x7fdaa0101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.007+0000 7fdaa6400640 1 --2- 172.21.15.1:0/3902706028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaa0103ab0 0x7fdaa0101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.007+0000 7fdaa6400640 1 --2- 172.21.15.1:0/3902706028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaa0103190 0x7fdaa0100de0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.007+0000 7fdaa6400640 1 -- 172.21.15.1:0/3902706028 >> 172.21.15.1:0/3902706028 conn(0x7fdaa00fc970 msgr2=0x7fdaa00fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.007+0000 7fdaa6400640 1 -- 172.21.15.1:0/3902706028 shutdown_connections 2024-04-26T01:55:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:06.007+0000 7fdaa6400640 1 -- 172.21.15.1:0/3902706028 wait complete. 2024-04-26T01:55:06.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.189+0000 7f9f7a000700 1 -- 172.21.15.1:0/1257553736 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55c22ad2f8b0 con 0x55c22ae00c60 2024-04-26T01:55:06.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.189+0000 7f9f7a000700 10 client.25613 handle_command_reply: tid=0 2024-04-26T01:55:06.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.189+0000 7f9fa14b6b80 1 client.25613 shutdown 2024-04-26T01:55:06.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.189+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/1257553736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22ade0560 msgr2=0x55c22ae671c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:06.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.189+0000 7f9fa14b6b80 1 --2- 172.21.15.1:0/1257553736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22ade0560 0x55c22ae671c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9f8809a5b0 tx=0x7f9f88004f90 comp rx=0 tx=0).stop 2024-04-26T01:55:06.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.189+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/1257553736 shutdown_connections 2024-04-26T01:55:06.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.189+0000 7f9fa14b6b80 1 --2- 172.21.15.1:0/1257553736 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c22ae00c60 0x55c22ae6ae40 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.189+0000 7f9fa14b6b80 1 --2- 172.21.15.1:0/1257553736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c22ade0560 0x55c22ae671c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.189+0000 7f9fa14b6b80 1 --2- 172.21.15.1:0/1257553736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c22add4dd0 0x55c22adf65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.189+0000 7f9fa14b6b80 1 --2- 172.21.15.1:0/1257553736 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c22add56f0 0x55c22adf6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.189+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/1257553736 >> 172.21.15.1:0/1257553736 conn(0x55c22ad040e0 msgr2=0x55c22adfbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.189+0000 7f9f7a000700 0 client.25613 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.190+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/1257553736 shutdown_connections 2024-04-26T01:55:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.190+0000 7f9fa14b6b80 1 -- 172.21.15.1:0/1257553736 wait complete. 2024-04-26T01:55:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.190+0000 7f9fa14b6b80 20 client.25613 trim_cache size 0 max 16384 2024-04-26T01:55:06.189 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T01:55:06.190 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 30, 2024-04-26T01:55:06.190 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24518, 2024-04-26T01:55:06.190 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 10, 2024-04-26T01:55:06.190 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-04-26T01:55:06.190 INFO:teuthology.orchestra.run.smithi001.stdout: "9b233c3e": { 2024-04-26T01:55:06.190 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 10, 2024-04-26T01:55:06.190 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-04-26T01:55:06.190 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:55:06.190 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-04-26T01:55:06.190 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:55:06.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T01:55:06.190 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-04-26T01:55:06.190 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 90.0, 2024-04-26T01:55:06.191 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-04-26T01:55:06.191 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-04-26T01:55:06.191 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-04-26T01:55:06.191 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:55:06.191 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-04-26T01:55:06.191 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:55:06.191 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:55:06.191 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:55:06.191 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:55:06.191 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:55:06.191 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:55:06.191 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T01:55:06.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.191+0000 7f9f96c00700 1 -- 172.21.15.1:0/3306275132 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f9f740a3ed0 msgr2=0x7f9f740a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:06.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.191+0000 7f9f96c00700 1 --2- 172.21.15.1:0/3306275132 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f9f740a3ed0 0x7f9f740a6390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55c22add67a0 tx=0x7f9f7c001040 comp rx=0 tx=0).stop 2024-04-26T01:55:06.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.191+0000 7f9f96c00700 1 -- 172.21.15.1:0/3306275132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f90108a60 msgr2=0x7f9f90102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:06.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.191+0000 7f9f96c00700 1 --2- 172.21.15.1:0/3306275132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f90108a60 0x7f9f90102490 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800ef30 tx=0x7f9f8800c3a0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.191+0000 7f9f96c00700 1 -- 172.21.15.1:0/3306275132 shutdown_connections 2024-04-26T01:55:06.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.191+0000 7f9f96c00700 1 --2- 172.21.15.1:0/3306275132 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f9f740a3ed0 0x7f9f740a6390 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.191+0000 7f9f96c00700 1 --2- 172.21.15.1:0/3306275132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f90108a60 0x7f9f90102490 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.191+0000 7f9f96c00700 1 --2- 172.21.15.1:0/3306275132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f9006a740 0x7f9f90101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.191+0000 7f9f96c00700 1 --2- 172.21.15.1:0/3306275132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f9006b100 0x7f9f90101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.191+0000 7f9f96c00700 1 -- 172.21.15.1:0/3306275132 >> 172.21.15.1:0/3306275132 conn(0x7f9f900788e0 msgr2=0x7f9f901004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:06.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.192+0000 7f9f96c00700 1 -- 172.21.15.1:0/3306275132 shutdown_connections 2024-04-26T01:55:06.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:06.192+0000 7f9f96c00700 1 -- 172.21.15.1:0/3306275132 wait complete. 2024-04-26T01:55:06.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:06 smithi097 ceph-mon[32853]: pgmap v107: 97 pgs: 97 active+clean; 29 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 64 KiB/s wr, 35 op/s 2024-04-26T01:55:06.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:06 smithi173 ceph-mon[32553]: pgmap v107: 97 pgs: 97 active+clean; 29 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 64 KiB/s wr, 35 op/s 2024-04-26T01:55:06.583 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:06 smithi001 ceph-mon[27822]: pgmap v107: 97 pgs: 97 active+clean; 29 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 64 KiB/s wr, 35 op/s 2024-04-26T01:55:07.075 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9b233c3e', quiesce took 9.8 seconds. Will release after: 30.0 2024-04-26T01:55:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:08 smithi097 ceph-mon[32853]: pgmap v108: 97 pgs: 97 active+clean; 29 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 58 KiB/s wr, 30 op/s 2024-04-26T01:55:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:08 smithi173 ceph-mon[32553]: pgmap v108: 97 pgs: 97 active+clean; 29 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 58 KiB/s wr, 30 op/s 2024-04-26T01:55:08.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:08 smithi001 ceph-mon[27822]: pgmap v108: 97 pgs: 97 active+clean; 29 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 58 KiB/s wr, 30 op/s 2024-04-26T01:55:09.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:09 smithi173 ceph-mon[32553]: pgmap v109: 97 pgs: 97 active+clean; 29 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 58 KiB/s wr, 29 op/s 2024-04-26T01:55:09.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:09 smithi001 ceph-mon[27822]: pgmap v109: 97 pgs: 97 active+clean; 29 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 58 KiB/s wr, 29 op/s 2024-04-26T01:55:09.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:09 smithi097 ceph-mon[32853]: pgmap v109: 97 pgs: 97 active+clean; 29 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 58 KiB/s wr, 29 op/s 2024-04-26T01:55:12.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:11 smithi001 ceph-mon[27822]: pgmap v110: 97 pgs: 97 active+clean; 31 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 256 KiB/s wr, 65 op/s 2024-04-26T01:55:12.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:11 smithi097 ceph-mon[32853]: pgmap v110: 97 pgs: 97 active+clean; 31 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 256 KiB/s wr, 65 op/s 2024-04-26T01:55:12.269 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:11 smithi173 ceph-mon[32553]: pgmap v110: 97 pgs: 97 active+clean; 31 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 256 KiB/s wr, 65 op/s 2024-04-26T01:55:14.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:13 smithi001 ceph-mon[27822]: pgmap v111: 97 pgs: 97 active+clean; 31 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 240 KiB/s wr, 58 op/s 2024-04-26T01:55:14.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:13 smithi097 ceph-mon[32853]: pgmap v111: 97 pgs: 97 active+clean; 31 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 240 KiB/s wr, 58 op/s 2024-04-26T01:55:14.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:13 smithi173 ceph-mon[32553]: pgmap v111: 97 pgs: 97 active+clean; 31 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 240 KiB/s wr, 58 op/s 2024-04-26T01:55:16.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:15 smithi001 ceph-mon[27822]: pgmap v112: 97 pgs: 97 active+clean; 33 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 365 KiB/s wr, 64 op/s 2024-04-26T01:55:16.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:15 smithi097 ceph-mon[32853]: pgmap v112: 97 pgs: 97 active+clean; 33 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 365 KiB/s wr, 64 op/s 2024-04-26T01:55:16.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:15 smithi173 ceph-mon[32553]: pgmap v112: 97 pgs: 97 active+clean; 33 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 365 KiB/s wr, 64 op/s 2024-04-26T01:55:18.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:17 smithi001 ceph-mon[27822]: pgmap v113: 97 pgs: 97 active+clean; 33 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 322 KiB/s wr, 42 op/s 2024-04-26T01:55:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:17 smithi097 ceph-mon[32853]: pgmap v113: 97 pgs: 97 active+clean; 33 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 322 KiB/s wr, 42 op/s 2024-04-26T01:55:18.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:17 smithi173 ceph-mon[32553]: pgmap v113: 97 pgs: 97 active+clean; 33 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 322 KiB/s wr, 42 op/s 2024-04-26T01:55:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.151+0000 7fda1a600640 1 -- 172.21.15.1:0/105560479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda14107090 msgr2=0x7fda1410ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.151+0000 7fda1a600640 1 --2- 172.21.15.1:0/105560479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda14107090 0x7fda1410ec60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009be0 tx=0x7fda0c02a6c0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.150 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.152+0000 7fda1a600640 1 -- 172.21.15.1:0/105560479 shutdown_connections 2024-04-26T01:55:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.152+0000 7fda1a600640 1 --2- 172.21.15.1:0/105560479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda14107090 0x7fda1410ec60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.152+0000 7fda1a600640 1 --2- 172.21.15.1:0/105560479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda14102970 0x7fda141069b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.152+0000 7fda1a600640 1 --2- 172.21.15.1:0/105560479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda14101fc0 0x7fda141023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.152+0000 7fda1a600640 1 -- 172.21.15.1:0/105560479 >> 172.21.15.1:0/105560479 conn(0x7fda140fd880 msgr2=0x7fda140ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.152+0000 7fda1a600640 1 -- 172.21.15.1:0/105560479 shutdown_connections 2024-04-26T01:55:19.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.152+0000 7fda1a600640 1 -- 172.21.15.1:0/105560479 wait complete. 2024-04-26T01:55:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.153+0000 7fda1a600640 1 Processor -- start 2024-04-26T01:55:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.153+0000 7fda1a600640 1 -- start start 2024-04-26T01:55:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.153+0000 7fda1a600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda14101fc0 0x7fda14074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.153+0000 7fda1a600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda14107090 0x7fda14074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.153+0000 7fda1a600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda14074cd0 0x7fda141a6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.153+0000 7fda1a600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fda14111fe0 con 0x7fda14107090 2024-04-26T01:55:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.153+0000 7fda1a600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fda14111e60 con 0x7fda14101fc0 2024-04-26T01:55:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.153+0000 7fda1a600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fda14112160 con 0x7fda14074cd0 2024-04-26T01:55:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.154+0000 7fda19200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda14101fc0 0x7fda14074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.154+0000 7fda19200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda14101fc0 0x7fda14074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:43044/0 (socket says 172.21.15.1:43044) 2024-04-26T01:55:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.154+0000 7fda19200640 1 -- 172.21.15.1:0/2506934899 learned_addr learned my addr 172.21.15.1:0/2506934899 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.154+0000 7fda13e00640 1 --2- 172.21.15.1:0/2506934899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda14107090 0x7fda14074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.154+0000 7fda19c00640 1 --2- 172.21.15.1:0/2506934899 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda14074cd0 0x7fda141a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.154+0000 7fda19200640 1 -- 172.21.15.1:0/2506934899 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda14074cd0 msgr2=0x7fda141a6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.154+0000 7fda19200640 1 --2- 172.21.15.1:0/2506934899 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda14074cd0 0x7fda141a6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.154+0000 7fda19200640 1 -- 172.21.15.1:0/2506934899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda14107090 msgr2=0x7fda14074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.154+0000 7fda19200640 1 --2- 172.21.15.1:0/2506934899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda14107090 0x7fda14074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.154+0000 7fda19200640 1 -- 172.21.15.1:0/2506934899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda141a6e10 con 0x7fda14101fc0 2024-04-26T01:55:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.155+0000 7fda19200640 1 --2- 172.21.15.1:0/2506934899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda14101fc0 0x7fda14074250 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fda0800e9c0 tx=0x7fda0800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.155+0000 7fda11600640 1 -- 172.21.15.1:0/2506934899 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0800cde0 con 0x7fda14101fc0 2024-04-26T01:55:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.155+0000 7fda11600640 1 -- 172.21.15.1:0/2506934899 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda080035f0 con 0x7fda14101fc0 2024-04-26T01:55:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.155+0000 7fda1a600640 1 -- 172.21.15.1:0/2506934899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda141a7100 con 0x7fda14101fc0 2024-04-26T01:55:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.155+0000 7fda11600640 1 -- 172.21.15.1:0/2506934899 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda08010690 con 0x7fda14101fc0 2024-04-26T01:55:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.155+0000 7fda1a600640 1 -- 172.21.15.1:0/2506934899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda141a7650 con 0x7fda14101fc0 2024-04-26T01:55:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.158+0000 7fda1e96e740 1 Processor -- start 2024-04-26T01:55:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.159+0000 7fda1e96e740 1 -- start start 2024-04-26T01:55:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.159+0000 7fda1e96e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b419b0 0x564ae6b41d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.159+0000 7fda1e96e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b422d0 0x564ae6b4c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.159+0000 7fda1e96e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b4cb90 0x564ae6b4ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.159+0000 7fda1e96e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564ae6aa7900 con 0x564ae6b4cb90 2024-04-26T01:55:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.159+0000 7fda1e96e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564ae6a4def0 con 0x564ae6b419b0 2024-04-26T01:55:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.159+0000 7fda1e96e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564ae6aa38c0 con 0x564ae6b422d0 2024-04-26T01:55:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.159+0000 7fda11600640 1 -- 172.21.15.1:0/2506934899 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fda08004000 con 0x7fda14101fc0 2024-04-26T01:55:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.159+0000 7fda13e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b4cb90 0x564ae6b4ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.159+0000 7fda13e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b4cb90 0x564ae6b4ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60648/0 (socket says 172.21.15.1:60648) 2024-04-26T01:55:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.159+0000 7fda13e00640 1 -- 172.21.15.1:0/2647423283 learned_addr learned my addr 172.21.15.1:0/2647423283 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.159+0000 7fda19c00640 1 --2- 172.21.15.1:0/2647423283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b419b0 0x564ae6b41d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.159+0000 7fda19200640 1 --2- 172.21.15.1:0/2647423283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b422d0 0x564ae6b4c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.160+0000 7fda19c00640 1 -- 172.21.15.1:0/2647423283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b422d0 msgr2=0x564ae6b4c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.160+0000 7fda19c00640 1 --2- 172.21.15.1:0/2647423283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b422d0 0x564ae6b4c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.160+0000 7fda19c00640 1 -- 172.21.15.1:0/2647423283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b4cb90 msgr2=0x564ae6b4ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.160+0000 7fda19c00640 1 --2- 172.21.15.1:0/2647423283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b4cb90 0x564ae6b4ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.160+0000 7fda19c00640 1 -- 172.21.15.1:0/2647423283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ae67ba210 con 0x564ae6b419b0 2024-04-26T01:55:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.160+0000 7fda13e00640 1 --2- 172.21.15.1:0/2647423283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b4cb90 0x564ae6b4ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.160+0000 7fda19200640 1 --2- 172.21.15.1:0/2647423283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b422d0 0x564ae6b4c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.160+0000 7fda19c00640 1 --2- 172.21.15.1:0/2647423283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b419b0 0x564ae6b41d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009b30 tx=0x7fda0c030430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37a72bc721ba7a8e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.161+0000 7fda06000640 1 -- 172.21.15.1:0/2647423283 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c041030 con 0x564ae6b419b0 2024-04-26T01:55:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.161+0000 7fda06000640 1 -- 172.21.15.1:0/2647423283 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda0c003e20 con 0x564ae6b419b0 2024-04-26T01:55:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.161+0000 7fda06000640 1 -- 172.21.15.1:0/2647423283 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c036040 con 0x564ae6b419b0 2024-04-26T01:55:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.161+0000 7fda11600640 1 --2- 172.21.15.1:0/2506934899 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd9fc0a3d30 0x7fd9fc0a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.161+0000 7fda11600640 1 -- 172.21.15.1:0/2506934899 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fda080d0b50 con 0x7fda14101fc0 2024-04-26T01:55:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.161+0000 7fda19200640 1 --2- 172.21.15.1:0/2506934899 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd9fc0a3d30 0x7fd9fc0a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.161+0000 7fda1e96e740 1 -- 172.21.15.1:0/2647423283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b419b0 msgr2=0x564ae6b41d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.161+0000 7fda1e96e740 1 --2- 172.21.15.1:0/2647423283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b419b0 0x564ae6b41d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009b30 tx=0x7fda0c030430 comp rx=0 tx=0).stop 2024-04-26T01:55:19.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.162+0000 7fda1e96e740 1 -- 172.21.15.1:0/2647423283 shutdown_connections 2024-04-26T01:55:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.162+0000 7fda1e96e740 1 --2- 172.21.15.1:0/2647423283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b4cb90 0x564ae6b4ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.162+0000 7fda1e96e740 1 --2- 172.21.15.1:0/2647423283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b422d0 0x564ae6b4c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.162+0000 7fda1e96e740 1 --2- 172.21.15.1:0/2647423283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b419b0 0x564ae6b41d90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.162+0000 7fda1e96e740 1 -- 172.21.15.1:0/2647423283 >> 172.21.15.1:0/2647423283 conn(0x564ae6b3dec0 msgr2=0x564ae6b3e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.162+0000 7fda1e96e740 1 -- 172.21.15.1:0/2647423283 shutdown_connections 2024-04-26T01:55:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.162+0000 7fda1e96e740 1 -- 172.21.15.1:0/2647423283 wait complete. 2024-04-26T01:55:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.162+0000 7fda19200640 1 --2- 172.21.15.1:0/2506934899 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd9fc0a3d30 0x7fd9fc0a61f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x564ae6b42f50 tx=0x7fda080d4600 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.162+0000 7fda1e96e740 1 -- start start 2024-04-26T01:55:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.163+0000 7fda1e96e740 1 Processor -- start 2024-04-26T01:55:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.163+0000 7fda1e96e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b419b0 0x564ae6b49b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.163+0000 7fda1e96e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b422d0 0x564ae6b4a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.165 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.163+0000 7fda1e96e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b4cb90 0x564ae6b4a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.163+0000 7fda1e96e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564ae6b4f760 con 0x564ae6b422d0 2024-04-26T01:55:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.163+0000 7fda1e96e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564ae6a61310 con 0x564ae6b4cb90 2024-04-26T01:55:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.163+0000 7fda1e96e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564ae6b4f8e0 con 0x564ae6b419b0 2024-04-26T01:55:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.163+0000 7fda1e96e740 10 client.0 _ll_register_callbacks cb 0x564ae6a2fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.163+0000 7fda13e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b422d0 0x564ae6b4a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.163+0000 7fda13e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b422d0 0x564ae6b4a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60660/0 (socket says 172.21.15.1:60660) 2024-04-26T01:55:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.163+0000 7fda13e00640 1 -- 172.21.15.1:0/2586033529 learned_addr learned my addr 172.21.15.1:0/2586033529 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.163+0000 7fda19c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b419b0 0x564ae6b49b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.164+0000 7fda19c00640 1 --2- 172.21.15.1:0/2586033529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b4cb90 0x564ae6b4a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.164+0000 7fda13e00640 1 -- 172.21.15.1:0/2586033529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b419b0 msgr2=0x564ae6b49b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.164+0000 7fda13e00640 1 --2- 172.21.15.1:0/2586033529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b419b0 0x564ae6b49b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.164+0000 7fda13e00640 1 -- 172.21.15.1:0/2586033529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b4cb90 msgr2=0x564ae6b4a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:19.166 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.164+0000 7fda13e00640 1 --2- 172.21.15.1:0/2586033529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b4cb90 0x564ae6b4a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.164+0000 7fda13e00640 1 -- 172.21.15.1:0/2586033529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ae6886620 con 0x564ae6b422d0 2024-04-26T01:55:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.164+0000 7fda13e00640 1 --2- 172.21.15.1:0/2586033529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b422d0 0x564ae6b4a0b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fda00012700 tx=0x7fda00012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb557c57e55acd61 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.164+0000 7fd9faa00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:55:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.165+0000 7fd9faa00640 1 -- 172.21.15.1:0/2586033529 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00004030 con 0x564ae6b422d0 2024-04-26T01:55:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.165+0000 7fd9faa00640 1 -- 172.21.15.1:0/2586033529 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fda000041d0 con 0x564ae6b422d0 2024-04-26T01:55:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.165+0000 7fda1e96e740 1 -- 172.21.15.1:0/2586033529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda0c003e20 con 0x564ae6b422d0 2024-04-26T01:55:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.165+0000 7fd9faa00640 1 -- 172.21.15.1:0/2586033529 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fda000043d0 con 0x564ae6b422d0 2024-04-26T01:55:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.165+0000 7fda1e96e740 10 client.15510 fetch_fsmap learned FSMap version 30 2024-04-26T01:55:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.165+0000 7fda1e96e740 1 -- 172.21.15.1:0/2586033529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564ae6b5ba70 con 0x564ae6b422d0 2024-04-26T01:55:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.165+0000 7fd9faa00640 1 -- 172.21.15.1:0/2586033529 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda0c003e20 con 0x564ae6b422d0 2024-04-26T01:55:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.165+0000 7fd9faa00640 1 -- 172.21.15.1:0/2586033529 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fda00003b90 con 0x564ae6b422d0 2024-04-26T01:55:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.166+0000 7fda1e96e740 10 client.15510 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:55:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.166+0000 7fda1e96e740 10 client.15510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:55:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.166+0000 7fda1e96e740 20 client.15510 populate_metadata read hostname 'smithi001' 2024-04-26T01:55:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.166+0000 7fda1e96e740 1 --2- 172.21.15.1:0/2586033529 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ae6b5cf80 0x564ae6b5f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.166+0000 7fda1e96e740 4 client.15510 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:55:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.166+0000 7fda1e96e740 1 -- 172.21.15.1:0/2586033529 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ae6abb8b0 con 0x564ae6b5cf80 2024-04-26T01:55:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.166+0000 7fda19c00640 1 --2- 172.21.15.1:0/2586033529 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ae6b5cf80 0x564ae6b5f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.166+0000 7fda19c00640 1 --2- 172.21.15.1:0/2586033529 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ae6b5cf80 0x564ae6b5f370 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11e9d24d5434c6ee server_cookie=a593bc95f637fc64 in_seq=0 out_seq=0 2024-04-26T01:55:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.166+0000 7fd9faa00640 10 client.15510 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.167+0000 7fd9faa00640 1 -- 172.21.15.1:0/2586033529 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ae6abb8b0 con 0x564ae6b5cf80 2024-04-26T01:55:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.167+0000 7fd9faa00640 10 client.15510 handle_command_reply: tid=0 2024-04-26T01:55:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.167+0000 7fda1e96e740 1 client.15510 shutdown 2024-04-26T01:55:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.167+0000 7fda1e96e740 1 -- 172.21.15.1:0/2586033529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b422d0 msgr2=0x564ae6b4a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.167+0000 7fda1e96e740 1 --2- 172.21.15.1:0/2586033529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b422d0 0x564ae6b4a0b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fda00012700 tx=0x7fda00012bc0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.168+0000 7fda1e96e740 1 -- 172.21.15.1:0/2586033529 shutdown_connections 2024-04-26T01:55:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.168+0000 7fda1e96e740 1 --2- 172.21.15.1:0/2586033529 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ae6b5cf80 0x564ae6b5f370 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.168+0000 7fda1e96e740 1 --2- 172.21.15.1:0/2586033529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b4cb90 0x564ae6b4a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.168+0000 7fd9faa00640 0 client.15510 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.168+0000 7fda1e96e740 1 --2- 172.21.15.1:0/2586033529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b422d0 0x564ae6b4a0b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.168+0000 7fda1e96e740 1 --2- 172.21.15.1:0/2586033529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b419b0 0x564ae6b49b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.168+0000 7fda1e96e740 1 -- 172.21.15.1:0/2586033529 >> 172.21.15.1:0/2586033529 conn(0x564ae6b3dec0 msgr2=0x564ae6b43bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.168+0000 7fda1e96e740 1 -- 172.21.15.1:0/2586033529 shutdown_connections 2024-04-26T01:55:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.168+0000 7fda1e96e740 1 -- 172.21.15.1:0/2586033529 wait complete. 2024-04-26T01:55:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.168+0000 7fda1e96e740 20 client.15510 trim_cache size 0 max 16384 2024-04-26T01:55:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.184+0000 7fda1e96e740 1 Processor -- start 2024-04-26T01:55:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.184+0000 7fda1e96e740 1 -- start start 2024-04-26T01:55:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.185+0000 7fda1e96e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b419b0 0x564ae6b58620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.185+0000 7fda1e96e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b422d0 0x564ae6b4a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.185+0000 7fda1e96e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b4cb90 0x564ae6b3cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.185+0000 7fda1e96e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fda141b2500 con 0x564ae6b419b0 2024-04-26T01:55:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.185+0000 7fda1e96e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fda141afd70 con 0x564ae6b422d0 2024-04-26T01:55:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.185+0000 7fda1e96e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fda141adbc0 con 0x564ae6b4cb90 2024-04-26T01:55:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.185+0000 7fda19c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b419b0 0x564ae6b58620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.185+0000 7fda13e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b422d0 0x564ae6b4a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.185+0000 7fda19c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b419b0 0x564ae6b58620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60670/0 (socket says 172.21.15.1:60670) 2024-04-26T01:55:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.185+0000 7fda19c00640 1 -- 172.21.15.1:0/797150521 learned_addr learned my addr 172.21.15.1:0/797150521 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.185+0000 7fda19c00640 1 --2- 172.21.15.1:0/797150521 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b4cb90 0x564ae6b3cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.185+0000 7fda19c00640 1 -- 172.21.15.1:0/797150521 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b4cb90 msgr2=0x564ae6b3cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.185+0000 7fda19c00640 1 --2- 172.21.15.1:0/797150521 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b4cb90 0x564ae6b3cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.185+0000 7fda19c00640 1 -- 172.21.15.1:0/797150521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b422d0 msgr2=0x564ae6b4a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.185+0000 7fda19c00640 1 --2- 172.21.15.1:0/797150521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b422d0 0x564ae6b4a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.185+0000 7fda19c00640 1 -- 172.21.15.1:0/797150521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ae6b39820 con 0x564ae6b419b0 2024-04-26T01:55:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.185+0000 7fda13e00640 1 --2- 172.21.15.1:0/797150521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b422d0 0x564ae6b4a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.186+0000 7fda19c00640 1 --2- 172.21.15.1:0/797150521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b419b0 0x564ae6b58620 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fda0c030dd0 tx=0x7fda0c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1582d4842a919385 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.186+0000 7fda05600640 1 -- 172.21.15.1:0/797150521 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c036040 con 0x564ae6b419b0 2024-04-26T01:55:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.186+0000 7fda05600640 1 -- 172.21.15.1:0/797150521 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda0c003e20 con 0x564ae6b419b0 2024-04-26T01:55:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.186+0000 7fda05600640 1 -- 172.21.15.1:0/797150521 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c041040 con 0x564ae6b419b0 2024-04-26T01:55:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.187+0000 7fda1e96e740 1 -- 172.21.15.1:0/797150521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b419b0 msgr2=0x564ae6b58620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.187+0000 7fda1e96e740 1 --2- 172.21.15.1:0/797150521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b419b0 0x564ae6b58620 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fda0c030dd0 tx=0x7fda0c02a6c0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.187+0000 7fda1e96e740 1 -- 172.21.15.1:0/797150521 shutdown_connections 2024-04-26T01:55:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.187+0000 7fda1e96e740 1 --2- 172.21.15.1:0/797150521 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b4cb90 0x564ae6b3cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.187+0000 7fda1e96e740 1 --2- 172.21.15.1:0/797150521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b422d0 0x564ae6b4a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.187+0000 7fda1e96e740 1 --2- 172.21.15.1:0/797150521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b419b0 0x564ae6b58620 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.187+0000 7fda1e96e740 1 -- 172.21.15.1:0/797150521 >> 172.21.15.1:0/797150521 conn(0x564ae6b3dec0 msgr2=0x564ae6b3c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.187+0000 7fda1e96e740 1 -- 172.21.15.1:0/797150521 shutdown_connections 2024-04-26T01:55:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.187+0000 7fda1e96e740 1 -- 172.21.15.1:0/797150521 wait complete. 2024-04-26T01:55:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.187+0000 7fda1e96e740 1 -- start start 2024-04-26T01:55:19.189 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.187+0000 7fda1e96e740 1 Processor -- start 2024-04-26T01:55:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.188+0000 7fda1e96e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b419b0 0x564ae6b52510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.188+0000 7fda1e96e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b422d0 0x564ae6b52a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.188+0000 7fda1e96e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b4cb90 0x564ae6b52f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.188+0000 7fda1e96e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fda141ae230 con 0x564ae6b419b0 2024-04-26T01:55:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.188+0000 7fda1e96e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fda141af6a0 con 0x564ae6b4cb90 2024-04-26T01:55:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.188+0000 7fda1e96e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fda141ae900 con 0x564ae6b422d0 2024-04-26T01:55:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.188+0000 7fda1e96e740 10 client.0 _ll_register_callbacks cb 0x7fda00009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.188+0000 7fda19c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b419b0 0x564ae6b52510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.188+0000 7fda13e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b422d0 0x564ae6b52a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.188+0000 7fda19c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b419b0 0x564ae6b52510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60674/0 (socket says 172.21.15.1:60674) 2024-04-26T01:55:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.188+0000 7fda19c00640 1 -- 172.21.15.1:0/3703360619 learned_addr learned my addr 172.21.15.1:0/3703360619 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.188+0000 7fda19c00640 1 --2- 172.21.15.1:0/3703360619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b4cb90 0x564ae6b52f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.189+0000 7fda19c00640 1 -- 172.21.15.1:0/3703360619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b422d0 msgr2=0x564ae6b52a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.189+0000 7fda19c00640 1 --2- 172.21.15.1:0/3703360619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b422d0 0x564ae6b52a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.189+0000 7fda19c00640 1 -- 172.21.15.1:0/3703360619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b4cb90 msgr2=0x564ae6b52f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.189+0000 7fda19c00640 1 --2- 172.21.15.1:0/3703360619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b4cb90 0x564ae6b52f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.189+0000 7fda19c00640 1 -- 172.21.15.1:0/3703360619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ae6bc2500 con 0x564ae6b419b0 2024-04-26T01:55:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.189+0000 7fda13e00640 1 --2- 172.21.15.1:0/3703360619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b422d0 0x564ae6b52a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.189+0000 7fda19c00640 1 --2- 172.21.15.1:0/3703360619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b419b0 0x564ae6b52510 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fda0c054350 tx=0x7fda0c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffb31e9a929493a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.189+0000 7fd9faa00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:55:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.189+0000 7fd9faa00640 1 -- 172.21.15.1:0/3703360619 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c041070 con 0x564ae6b419b0 2024-04-26T01:55:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.189+0000 7fd9faa00640 1 -- 172.21.15.1:0/3703360619 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fda0c0305a0 con 0x564ae6b419b0 2024-04-26T01:55:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.189+0000 7fda1e96e740 1 -- 172.21.15.1:0/3703360619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda0c003e20 con 0x564ae6b419b0 2024-04-26T01:55:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.189+0000 7fd9faa00640 1 -- 172.21.15.1:0/3703360619 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fda0c036040 con 0x564ae6b419b0 2024-04-26T01:55:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.189+0000 7fda1e96e740 10 client.15516 fetch_fsmap learned FSMap version 30 2024-04-26T01:55:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.189+0000 7fda1e96e740 1 -- 172.21.15.1:0/3703360619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564ae6bc37b0 con 0x564ae6b419b0 2024-04-26T01:55:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.190+0000 7fd9faa00640 1 -- 172.21.15.1:0/3703360619 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda0c003e20 con 0x564ae6b419b0 2024-04-26T01:55:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.190+0000 7fd9faa00640 1 -- 172.21.15.1:0/3703360619 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fda0c054c60 con 0x564ae6b419b0 2024-04-26T01:55:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.190+0000 7fda1e96e740 10 client.15516 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T01:55:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.190+0000 7fda1e96e740 10 client.15516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:55:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.190+0000 7fda1e96e740 20 client.15516 populate_metadata read hostname 'smithi001' 2024-04-26T01:55:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.190+0000 7fda1e96e740 1 --2- 172.21.15.1:0/3703360619 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ae6bc3980 0x564ae6bc5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.190+0000 7fda1e96e740 4 client.15516 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:55:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.190+0000 7fda1e96e740 1 -- 172.21.15.1:0/3703360619 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564ae6ab5320 con 0x564ae6bc3980 2024-04-26T01:55:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.190+0000 7fda13e00640 1 --2- 172.21.15.1:0/3703360619 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ae6bc3980 0x564ae6bc5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.191+0000 7fda13e00640 1 --2- 172.21.15.1:0/3703360619 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ae6bc3980 0x564ae6bc5d70 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b4fa1a544bddf03 server_cookie=8a88a85179a4b5b4 in_seq=0 out_seq=0 2024-04-26T01:55:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.191+0000 7fd9faa00640 10 client.15516 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.213+0000 7fd9faa00640 1 -- 172.21.15.1:0/3703360619 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564ae6ab5320 con 0x564ae6bc3980 2024-04-26T01:55:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.213+0000 7fd9faa00640 10 client.15516 handle_command_reply: tid=0 2024-04-26T01:55:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.213+0000 7fda1e96e740 1 client.15516 shutdown 2024-04-26T01:55:19.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.213+0000 7fda1e96e740 1 -- 172.21.15.1:0/3703360619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b419b0 msgr2=0x564ae6b52510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:19.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.213+0000 7fda1e96e740 1 --2- 172.21.15.1:0/3703360619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b419b0 0x564ae6b52510 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fda0c054350 tx=0x7fda0c005fd0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.213+0000 7fda1e96e740 1 -- 172.21.15.1:0/3703360619 shutdown_connections 2024-04-26T01:55:19.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.213+0000 7fda1e96e740 1 --2- 172.21.15.1:0/3703360619 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564ae6bc3980 0x564ae6bc5d70 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.213+0000 7fda1e96e740 1 --2- 172.21.15.1:0/3703360619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ae6b4cb90 0x564ae6b52f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.213+0000 7fda1e96e740 1 --2- 172.21.15.1:0/3703360619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564ae6b422d0 0x564ae6b52a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.213+0000 7fda1e96e740 1 --2- 172.21.15.1:0/3703360619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ae6b419b0 0x564ae6b52510 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.213+0000 7fd9faa00640 0 client.15516 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:19.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.213+0000 7fda1e96e740 1 -- 172.21.15.1:0/3703360619 >> 172.21.15.1:0/3703360619 conn(0x564ae6b3dec0 msgr2=0x564ae6b43bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:19.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.213+0000 7fda1e96e740 1 -- 172.21.15.1:0/3703360619 shutdown_connections 2024-04-26T01:55:19.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.213+0000 7fda1e96e740 1 -- 172.21.15.1:0/3703360619 wait complete. 2024-04-26T01:55:19.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.213+0000 7fda1e96e740 20 client.15516 trim_cache size 0 max 16384 2024-04-26T01:55:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:55:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:55:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:55:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:55:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.214+0000 7fda1a600640 1 -- 172.21.15.1:0/2506934899 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd9fc0a3d30 msgr2=0x7fd9fc0a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.214+0000 7fda1a600640 1 --2- 172.21.15.1:0/2506934899 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd9fc0a3d30 0x7fd9fc0a61f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x564ae6b42f50 tx=0x7fda080d4600 comp rx=0 tx=0).stop 2024-04-26T01:55:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.214+0000 7fda1a600640 1 -- 172.21.15.1:0/2506934899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda14101fc0 msgr2=0x7fda14074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.214+0000 7fda1a600640 1 --2- 172.21.15.1:0/2506934899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda14101fc0 0x7fda14074250 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fda0800e9c0 tx=0x7fda0800ee80 comp rx=0 tx=0).stop 2024-04-26T01:55:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.214+0000 7fda1a600640 1 -- 172.21.15.1:0/2506934899 shutdown_connections 2024-04-26T01:55:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.214+0000 7fda1a600640 1 --2- 172.21.15.1:0/2506934899 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd9fc0a3d30 0x7fd9fc0a61f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.214+0000 7fda1a600640 1 --2- 172.21.15.1:0/2506934899 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda14074cd0 0x7fda141a6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.214+0000 7fda1a600640 1 --2- 172.21.15.1:0/2506934899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda14107090 0x7fda14074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.214+0000 7fda1a600640 1 --2- 172.21.15.1:0/2506934899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda14101fc0 0x7fda14074250 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.214+0000 7fda1a600640 1 -- 172.21.15.1:0/2506934899 >> 172.21.15.1:0/2506934899 conn(0x7fda140fd880 msgr2=0x7fda140febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.214+0000 7fda1a600640 1 -- 172.21.15.1:0/2506934899 shutdown_connections 2024-04-26T01:55:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:19.215+0000 7fda1a600640 1 -- 172.21.15.1:0/2506934899 wait complete. 2024-04-26T01:55:20.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:19 smithi001 ceph-mon[27822]: pgmap v114: 97 pgs: 97 active+clean; 33 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 322 KiB/s wr, 42 op/s 2024-04-26T01:55:20.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:19 smithi097 ceph-mon[32853]: pgmap v114: 97 pgs: 97 active+clean; 33 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 322 KiB/s wr, 42 op/s 2024-04-26T01:55:20.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:19 smithi173 ceph-mon[32553]: pgmap v114: 97 pgs: 97 active+clean; 33 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 322 KiB/s wr, 42 op/s 2024-04-26T01:55:22.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:21 smithi001 ceph-mon[27822]: pgmap v115: 97 pgs: 97 active+clean; 33 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 337 KiB/s wr, 49 op/s 2024-04-26T01:55:22.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:21 smithi097 ceph-mon[32853]: pgmap v115: 97 pgs: 97 active+clean; 33 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 337 KiB/s wr, 49 op/s 2024-04-26T01:55:22.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:21 smithi173 ceph-mon[32553]: pgmap v115: 97 pgs: 97 active+clean; 33 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 337 KiB/s wr, 49 op/s 2024-04-26T01:55:24.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:23 smithi001 ceph-mon[27822]: pgmap v116: 97 pgs: 97 active+clean; 33 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 139 KiB/s wr, 14 op/s 2024-04-26T01:55:24.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:23 smithi097 ceph-mon[32853]: pgmap v116: 97 pgs: 97 active+clean; 33 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 139 KiB/s wr, 14 op/s 2024-04-26T01:55:24.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:23 smithi173 ceph-mon[32553]: pgmap v116: 97 pgs: 97 active+clean; 33 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 139 KiB/s wr, 14 op/s 2024-04-26T01:55:26.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:25 smithi001 ceph-mon[27822]: pgmap v117: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 140 KiB/s wr, 17 op/s 2024-04-26T01:55:26.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:25 smithi097 ceph-mon[32853]: pgmap v117: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 140 KiB/s wr, 17 op/s 2024-04-26T01:55:26.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:25 smithi173 ceph-mon[32553]: pgmap v117: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 140 KiB/s wr, 17 op/s 2024-04-26T01:55:27.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:27 smithi001 ceph-mon[27822]: pgmap v118: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-04-26T01:55:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:27 smithi097 ceph-mon[32853]: pgmap v118: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-04-26T01:55:28.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:27 smithi173 ceph-mon[32553]: pgmap v118: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-04-26T01:55:30.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:29 smithi001 ceph-mon[27822]: pgmap v119: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-04-26T01:55:30.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:29 smithi097 ceph-mon[32853]: pgmap v119: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-04-26T01:55:30.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:29 smithi173 ceph-mon[32553]: pgmap v119: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-04-26T01:55:31.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:30 smithi097 ceph-mon[32853]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:55:31.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:30 smithi173 ceph-mon[32553]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:55:31.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:30 smithi001 ceph-mon[27822]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:55:32.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:31 smithi097 ceph-mon[32853]: pgmap v120: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-04-26T01:55:32.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:31 smithi173 ceph-mon[32553]: pgmap v120: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-04-26T01:55:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.365+0000 7fc43de00640 1 -- 172.21.15.1:0/2236281238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4381092d0 msgr2=0x7fc438110ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.365+0000 7fc43de00640 1 --2- 172.21.15.1:0/2236281238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4381092d0 0x7fc438110ea0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc430009ca0 tx=0x7fc43002a8c0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.365+0000 7fc43de00640 1 -- 172.21.15.1:0/2236281238 shutdown_connections 2024-04-26T01:55:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.365+0000 7fc43de00640 1 --2- 172.21.15.1:0/2236281238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4381092d0 0x7fc438110ea0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.365+0000 7fc43de00640 1 --2- 172.21.15.1:0/2236281238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc438104bb0 0x7fc438108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.365+0000 7fc43de00640 1 --2- 172.21.15.1:0/2236281238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc438104200 0x7fc4381045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.365+0000 7fc43de00640 1 -- 172.21.15.1:0/2236281238 >> 172.21.15.1:0/2236281238 conn(0x7fc4380ffa60 msgr2=0x7fc438101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.365+0000 7fc43de00640 1 -- 172.21.15.1:0/2236281238 shutdown_connections 2024-04-26T01:55:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.365+0000 7fc43de00640 1 -- 172.21.15.1:0/2236281238 wait complete. 2024-04-26T01:55:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.366+0000 7fc43de00640 1 Processor -- start 2024-04-26T01:55:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.367+0000 7fc43de00640 1 -- start start 2024-04-26T01:55:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.367+0000 7fc43de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc438104200 0x7fc4381a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.367+0000 7fc43de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4381092d0 0x7fc4381a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.367+0000 7fc43de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc43819a860 0x7fc43819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.367+0000 7fc43de00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc438114240 con 0x7fc4381092d0 2024-04-26T01:55:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.367+0000 7fc43de00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc4381140c0 con 0x7fc438104200 2024-04-26T01:55:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.367+0000 7fc43de00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc4381143c0 con 0x7fc43819a860 2024-04-26T01:55:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.367+0000 7fc42fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4381092d0 0x7fc4381a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.367+0000 7fc43ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc438104200 0x7fc4381a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.367+0000 7fc43ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc438104200 0x7fc4381a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:53696/0 (socket says 172.21.15.1:53696) 2024-04-26T01:55:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.367+0000 7fc43ca00640 1 -- 172.21.15.1:0/2294501516 learned_addr learned my addr 172.21.15.1:0/2294501516 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.367+0000 7fc43d400640 1 --2- 172.21.15.1:0/2294501516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc43819a860 0x7fc43819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.368+0000 7fc43ca00640 1 -- 172.21.15.1:0/2294501516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc43819a860 msgr2=0x7fc43819ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.368+0000 7fc43ca00640 1 --2- 172.21.15.1:0/2294501516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc43819a860 0x7fc43819ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.368+0000 7fc43ca00640 1 -- 172.21.15.1:0/2294501516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4381092d0 msgr2=0x7fc4381a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.368+0000 7fc43ca00640 1 --2- 172.21.15.1:0/2294501516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4381092d0 0x7fc4381a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.368+0000 7fc43ca00640 1 -- 172.21.15.1:0/2294501516 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43819b380 con 0x7fc438104200 2024-04-26T01:55:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.368+0000 7fc43d400640 1 --2- 172.21.15.1:0/2294501516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc43819a860 0x7fc43819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.368+0000 7fc43ca00640 1 --2- 172.21.15.1:0/2294501516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc438104200 0x7fc4381a06b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc42800e9f0 tx=0x7fc42800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.369+0000 7fc436000640 1 -- 172.21.15.1:0/2294501516 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42800cd80 con 0x7fc438104200 2024-04-26T01:55:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.369+0000 7fc436000640 1 -- 172.21.15.1:0/2294501516 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4280108e0 con 0x7fc438104200 2024-04-26T01:55:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.369+0000 7fc43de00640 1 -- 172.21.15.1:0/2294501516 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc43819b670 con 0x7fc438104200 2024-04-26T01:55:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.369+0000 7fc436000640 1 -- 172.21.15.1:0/2294501516 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc428018980 con 0x7fc438104200 2024-04-26T01:55:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.369+0000 7fc43de00640 1 -- 172.21.15.1:0/2294501516 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4381a7910 con 0x7fc438104200 2024-04-26T01:55:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.372+0000 7fc442230740 1 Processor -- start 2024-04-26T01:55:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.372+0000 7fc442230740 1 -- start start 2024-04-26T01:55:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.372+0000 7fc436000640 1 -- 172.21.15.1:0/2294501516 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fc428010430 con 0x7fc438104200 2024-04-26T01:55:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.372+0000 7fc442230740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f29b0 0x555ff21f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.373+0000 7fc442230740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff21f32d0 0x555ff21fd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.373+0000 7fc442230740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21fdb90 0x555ff21fff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.373+0000 7fc442230740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555ff2158900 con 0x555ff21f29b0 2024-04-26T01:55:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.373+0000 7fc442230740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555ff20feef0 con 0x555ff21f32d0 2024-04-26T01:55:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.373+0000 7fc442230740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555ff21548c0 con 0x555ff21fdb90 2024-04-26T01:55:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.373+0000 7fc43d400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f29b0 0x555ff21f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.373+0000 7fc43d400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f29b0 0x555ff21f2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37424/0 (socket says 172.21.15.1:37424) 2024-04-26T01:55:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.373+0000 7fc43d400640 1 -- 172.21.15.1:0/1336530674 learned_addr learned my addr 172.21.15.1:0/1336530674 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.373+0000 7fc43ca00640 1 --2- 172.21.15.1:0/1336530674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff21f32d0 0x555ff21fd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.373+0000 7fc42fe00640 1 --2- 172.21.15.1:0/1336530674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21fdb90 0x555ff21fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.373+0000 7fc43d400640 1 -- 172.21.15.1:0/1336530674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21fdb90 msgr2=0x555ff21fff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.373+0000 7fc43d400640 1 --2- 172.21.15.1:0/1336530674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21fdb90 0x555ff21fff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.373+0000 7fc43d400640 1 -- 172.21.15.1:0/1336530674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff21f32d0 msgr2=0x555ff21fd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.373+0000 7fc43d400640 1 --2- 172.21.15.1:0/1336530674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff21f32d0 0x555ff21fd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.373+0000 7fc43d400640 1 -- 172.21.15.1:0/1336530674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ff1e6b210 con 0x555ff21f29b0 2024-04-26T01:55:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.373+0000 7fc42fe00640 1 --2- 172.21.15.1:0/1336530674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21fdb90 0x555ff21fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:55:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.374+0000 7fc43d400640 1 --2- 172.21.15.1:0/1336530674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f29b0 0x555ff21f2d90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc430033040 tx=0x7fc43002f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a4b9ed34c4b04c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.374+0000 7fc42dc00640 1 -- 172.21.15.1:0/1336530674 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc430031450 con 0x555ff21f29b0 2024-04-26T01:55:32.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.374+0000 7fc42dc00640 1 -- 172.21.15.1:0/1336530674 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc430036040 con 0x555ff21f29b0 2024-04-26T01:55:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.374+0000 7fc42dc00640 1 -- 172.21.15.1:0/1336530674 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4300043c0 con 0x555ff21f29b0 2024-04-26T01:55:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.374+0000 7fc436000640 1 --2- 172.21.15.1:0/2294501516 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc41c0a3d80 0x7fc41c0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.375+0000 7fc436000640 1 -- 172.21.15.1:0/2294501516 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fc4280d0b30 con 0x7fc438104200 2024-04-26T01:55:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.375+0000 7fc43ca00640 1 --2- 172.21.15.1:0/2294501516 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc41c0a3d80 0x7fc41c0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.375+0000 7fc442230740 1 -- 172.21.15.1:0/1336530674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f29b0 msgr2=0x555ff21f2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.375+0000 7fc442230740 1 --2- 172.21.15.1:0/1336530674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f29b0 0x555ff21f2d90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc430033040 tx=0x7fc43002f310 comp rx=0 tx=0).stop 2024-04-26T01:55:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.375+0000 7fc442230740 1 -- 172.21.15.1:0/1336530674 shutdown_connections 2024-04-26T01:55:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.375+0000 7fc442230740 1 --2- 172.21.15.1:0/1336530674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21fdb90 0x555ff21fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.375+0000 7fc442230740 1 --2- 172.21.15.1:0/1336530674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff21f32d0 0x555ff21fd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.375+0000 7fc442230740 1 --2- 172.21.15.1:0/1336530674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f29b0 0x555ff21f2d90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.375+0000 7fc442230740 1 -- 172.21.15.1:0/1336530674 >> 172.21.15.1:0/1336530674 conn(0x555ff21eeec0 msgr2=0x555ff21ef2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.375+0000 7fc442230740 1 -- 172.21.15.1:0/1336530674 shutdown_connections 2024-04-26T01:55:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.375+0000 7fc442230740 1 -- 172.21.15.1:0/1336530674 wait complete. 2024-04-26T01:55:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.375+0000 7fc43ca00640 1 --2- 172.21.15.1:0/2294501516 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc41c0a3d80 0x7fc41c0a6240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x555ff21f3f50 tx=0x7fc4280d45e0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:32.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.376+0000 7fc442230740 1 -- start start 2024-04-26T01:55:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.376+0000 7fc442230740 1 Processor -- start 2024-04-26T01:55:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.376+0000 7fc442230740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21f29b0 0x555ff21fab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.376+0000 7fc442230740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f32d0 0x555ff21fb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.376+0000 7fc442230740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff21fdb90 0x555ff21fb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.376+0000 7fc442230740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555ff22005e0 con 0x555ff21f32d0 2024-04-26T01:55:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.376+0000 7fc442230740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555ff2112310 con 0x555ff21fdb90 2024-04-26T01:55:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.376+0000 7fc442230740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555ff2200760 con 0x555ff21f29b0 2024-04-26T01:55:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.376+0000 7fc442230740 10 client.0 _ll_register_callbacks cb 0x555ff20e0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.377+0000 7fc43d400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21f29b0 0x555ff21fab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.377+0000 7fc42fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f32d0 0x555ff21fb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.377+0000 7fc43d400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21f29b0 0x555ff21fab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:34852/0 (socket says 172.21.15.1:34852) 2024-04-26T01:55:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.377+0000 7fc43d400640 1 -- 172.21.15.1:0/2233606935 learned_addr learned my addr 172.21.15.1:0/2233606935 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.377+0000 7fc43d400640 1 --2- 172.21.15.1:0/2233606935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff21fdb90 0x555ff21fb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.377+0000 7fc42fe00640 1 -- 172.21.15.1:0/2233606935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21f29b0 msgr2=0x555ff21fab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.377+0000 7fc42fe00640 1 --2- 172.21.15.1:0/2233606935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21f29b0 0x555ff21fab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.377+0000 7fc42fe00640 1 -- 172.21.15.1:0/2233606935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff21fdb90 msgr2=0x555ff21fb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.377+0000 7fc42fe00640 1 --2- 172.21.15.1:0/2233606935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff21fdb90 0x555ff21fb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.377+0000 7fc42fe00640 1 -- 172.21.15.1:0/2233606935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ff1f37620 con 0x555ff21f32d0 2024-04-26T01:55:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.378+0000 7fc42fe00640 1 --2- 172.21.15.1:0/2233606935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f32d0 0x555ff21fb040 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x555ff21f0050 tx=0x7fc4200128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3736c75f7ddc3c0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.378+0000 7fc426000640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:55:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.378+0000 7fc426000640 1 -- 172.21.15.1:0/2233606935 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42001c470 con 0x555ff21f32d0 2024-04-26T01:55:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.378+0000 7fc426000640 1 -- 172.21.15.1:0/2233606935 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fc42001c610 con 0x555ff21f32d0 2024-04-26T01:55:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.378+0000 7fc442230740 1 -- 172.21.15.1:0/2233606935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc430036040 con 0x555ff21f32d0 2024-04-26T01:55:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.378+0000 7fc426000640 1 -- 172.21.15.1:0/2233606935 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc42001c830 con 0x555ff21f32d0 2024-04-26T01:55:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.378+0000 7fc442230740 10 client.15528 fetch_fsmap learned FSMap version 31 2024-04-26T01:55:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.378+0000 7fc442230740 1 -- 172.21.15.1:0/2233606935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555ff220c8c0 con 0x555ff21f32d0 2024-04-26T01:55:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.378+0000 7fc426000640 1 -- 172.21.15.1:0/2233606935 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc430036040 con 0x555ff21f32d0 2024-04-26T01:55:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.379+0000 7fc426000640 1 -- 172.21.15.1:0/2233606935 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7fc4200109a0 con 0x555ff21f32d0 2024-04-26T01:55:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.379+0000 7fc442230740 10 client.15528 fetch_fsmap finished waiting for FSMap version 31 2024-04-26T01:55:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.379+0000 7fc442230740 10 client.15528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:55:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.379+0000 7fc442230740 20 client.15528 populate_metadata read hostname 'smithi001' 2024-04-26T01:55:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.379+0000 7fc442230740 1 --2- 172.21.15.1:0/2233606935 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x555ff220ddd0 0x555ff22101c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.379+0000 7fc442230740 4 client.15528 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:55:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.379+0000 7fc442230740 1 -- 172.21.15.1:0/2233606935 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ff216c8b0 con 0x555ff220ddd0 2024-04-26T01:55:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.379+0000 7fc43d400640 1 --2- 172.21.15.1:0/2233606935 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x555ff220ddd0 0x555ff22101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.380+0000 7fc43d400640 1 --2- 172.21.15.1:0/2233606935 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x555ff220ddd0 0x555ff22101c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b161c3b54d6b361b server_cookie=dfcda67d6da7ced5 in_seq=0 out_seq=0 2024-04-26T01:55:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.380+0000 7fc426000640 10 client.15528 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.381+0000 7fc426000640 1 -- 172.21.15.1:0/2233606935 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ff216c8b0 con 0x555ff220ddd0 2024-04-26T01:55:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.381+0000 7fc426000640 10 client.15528 handle_command_reply: tid=0 2024-04-26T01:55:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.381+0000 7fc442230740 1 client.15528 shutdown 2024-04-26T01:55:32.383 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:31 smithi001 ceph-mon[27822]: pgmap v120: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-04-26T01:55:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.381+0000 7fc442230740 1 -- 172.21.15.1:0/2233606935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f32d0 msgr2=0x555ff21fb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.381+0000 7fc442230740 1 --2- 172.21.15.1:0/2233606935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f32d0 0x555ff21fb040 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x555ff21f0050 tx=0x7fc4200128e0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.381+0000 7fc442230740 1 -- 172.21.15.1:0/2233606935 shutdown_connections 2024-04-26T01:55:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.381+0000 7fc442230740 1 --2- 172.21.15.1:0/2233606935 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x555ff220ddd0 0x555ff22101c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.381+0000 7fc442230740 1 --2- 172.21.15.1:0/2233606935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff21fdb90 0x555ff21fb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.381+0000 7fc442230740 1 --2- 172.21.15.1:0/2233606935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f32d0 0x555ff21fb040 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.381+0000 7fc442230740 1 --2- 172.21.15.1:0/2233606935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21f29b0 0x555ff21fab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.381+0000 7fc426000640 0 client.15528 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.381+0000 7fc442230740 1 -- 172.21.15.1:0/2233606935 >> 172.21.15.1:0/2233606935 conn(0x555ff21eeec0 msgr2=0x555ff21f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.381+0000 7fc442230740 1 -- 172.21.15.1:0/2233606935 shutdown_connections 2024-04-26T01:55:32.385 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.381+0000 7fc442230740 1 -- 172.21.15.1:0/2233606935 wait complete. 2024-04-26T01:55:32.385 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.381+0000 7fc442230740 20 client.15528 trim_cache size 0 max 16384 2024-04-26T01:55:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.398+0000 7fc442230740 1 Processor -- start 2024-04-26T01:55:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.398+0000 7fc442230740 1 -- start start 2024-04-26T01:55:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.398+0000 7fc442230740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f29b0 0x555ff22094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.398+0000 7fc442230740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21f32d0 0x555ff21fb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.398+0000 7fc442230740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff21fdb90 0x555ff2215a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.398+0000 7fc442230740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc4381b2580 con 0x555ff21f29b0 2024-04-26T01:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.398+0000 7fc442230740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc4381afe20 con 0x555ff21fdb90 2024-04-26T01:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.398+0000 7fc442230740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc4381adc70 con 0x555ff21f32d0 2024-04-26T01:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.398+0000 7fc43d400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f29b0 0x555ff22094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.398+0000 7fc43d400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f29b0 0x555ff22094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37444/0 (socket says 172.21.15.1:37444) 2024-04-26T01:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.398+0000 7fc43d400640 1 -- 172.21.15.1:0/3100227247 learned_addr learned my addr 172.21.15.1:0/3100227247 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.398+0000 7fc42fe00640 1 --2- 172.21.15.1:0/3100227247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21f32d0 0x555ff21fb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.399+0000 7fc43d400640 1 --2- 172.21.15.1:0/3100227247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff21fdb90 0x555ff2215a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.399+0000 7fc42fe00640 1 -- 172.21.15.1:0/3100227247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff21fdb90 msgr2=0x555ff2215a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.399+0000 7fc42fe00640 1 --2- 172.21.15.1:0/3100227247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff21fdb90 0x555ff2215a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.399+0000 7fc42fe00640 1 -- 172.21.15.1:0/3100227247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f29b0 msgr2=0x555ff22094a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.399+0000 7fc42fe00640 1 --2- 172.21.15.1:0/3100227247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f29b0 0x555ff22094a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.399+0000 7fc42fe00640 1 -- 172.21.15.1:0/3100227247 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ff21ea820 con 0x555ff21f32d0 2024-04-26T01:55:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.399+0000 7fc43d400640 1 --2- 172.21.15.1:0/3100227247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f29b0 0x555ff22094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.399+0000 7fc42fe00640 1 --2- 172.21.15.1:0/3100227247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21f32d0 0x555ff21fb3a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc420010dc0 tx=0x7fc420009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eede0ed8adeeb4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.400+0000 7fc42d200640 1 -- 172.21.15.1:0/3100227247 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc420029070 con 0x555ff21f32d0 2024-04-26T01:55:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.400+0000 7fc42d200640 1 -- 172.21.15.1:0/3100227247 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc42000f480 con 0x555ff21f32d0 2024-04-26T01:55:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.400+0000 7fc42d200640 1 -- 172.21.15.1:0/3100227247 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc420005370 con 0x555ff21f32d0 2024-04-26T01:55:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.400+0000 7fc442230740 1 -- 172.21.15.1:0/3100227247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21f32d0 msgr2=0x555ff21fb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.400+0000 7fc442230740 1 --2- 172.21.15.1:0/3100227247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21f32d0 0x555ff21fb3a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc420010dc0 tx=0x7fc420009620 comp rx=0 tx=0).stop 2024-04-26T01:55:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.400+0000 7fc442230740 1 -- 172.21.15.1:0/3100227247 shutdown_connections 2024-04-26T01:55:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.400+0000 7fc442230740 1 --2- 172.21.15.1:0/3100227247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff21fdb90 0x555ff2215a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.400+0000 7fc442230740 1 --2- 172.21.15.1:0/3100227247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21f32d0 0x555ff21fb3a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.400+0000 7fc442230740 1 --2- 172.21.15.1:0/3100227247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f29b0 0x555ff22094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.400+0000 7fc442230740 1 -- 172.21.15.1:0/3100227247 >> 172.21.15.1:0/3100227247 conn(0x555ff21eeec0 msgr2=0x555ff21f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.400+0000 7fc442230740 1 -- 172.21.15.1:0/3100227247 shutdown_connections 2024-04-26T01:55:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.400+0000 7fc442230740 1 -- 172.21.15.1:0/3100227247 wait complete. 2024-04-26T01:55:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.401+0000 7fc442230740 1 -- start start 2024-04-26T01:55:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.401+0000 7fc442230740 1 Processor -- start 2024-04-26T01:55:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.401+0000 7fc442230740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f29b0 0x555ff2205d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.401+0000 7fc442230740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21f32d0 0x555ff2203100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.401+0000 7fc442230740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff21fdb90 0x555ff2203640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.401+0000 7fc442230740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc4381ae2e0 con 0x555ff21f29b0 2024-04-26T01:55:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.401+0000 7fc442230740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc4381af750 con 0x555ff21fdb90 2024-04-26T01:55:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.401+0000 7fc442230740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc4381ae9b0 con 0x555ff21f32d0 2024-04-26T01:55:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.401+0000 7fc442230740 10 client.0 _ll_register_callbacks cb 0x7fc420009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.402+0000 7fc43d400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f29b0 0x555ff2205d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.402+0000 7fc43d400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f29b0 0x555ff2205d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37448/0 (socket says 172.21.15.1:37448) 2024-04-26T01:55:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.402+0000 7fc43d400640 1 -- 172.21.15.1:0/3822503704 learned_addr learned my addr 172.21.15.1:0/3822503704 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.402+0000 7fc42fe00640 1 --2- 172.21.15.1:0/3822503704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21f32d0 0x555ff2203100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.402+0000 7fc43d400640 1 --2- 172.21.15.1:0/3822503704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff21fdb90 0x555ff2203640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.402+0000 7fc42fe00640 1 -- 172.21.15.1:0/3822503704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff21fdb90 msgr2=0x555ff2203640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.402+0000 7fc42fe00640 1 --2- 172.21.15.1:0/3822503704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff21fdb90 0x555ff2203640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.402+0000 7fc42fe00640 1 -- 172.21.15.1:0/3822503704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f29b0 msgr2=0x555ff2205d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.402+0000 7fc42fe00640 1 --2- 172.21.15.1:0/3822503704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f29b0 0x555ff2205d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.402+0000 7fc42fe00640 1 -- 172.21.15.1:0/3822503704 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ff2203e20 con 0x555ff21f32d0 2024-04-26T01:55:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.402+0000 7fc43d400640 1 --2- 172.21.15.1:0/3822503704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f29b0 0x555ff2205d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.403+0000 7fc42fe00640 1 --2- 172.21.15.1:0/3822503704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21f32d0 0x555ff2203100 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x555ff21f32a0 tx=0x7fc42001cc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb8a3bf7f2a80b3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.403+0000 7fc426000640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:55:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.403+0000 7fc426000640 1 -- 172.21.15.1:0/3822503704 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc420029070 con 0x555ff21f32d0 2024-04-26T01:55:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.403+0000 7fc442230740 1 -- 172.21.15.1:0/3822503704 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc42000f480 con 0x555ff21f32d0 2024-04-26T01:55:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.403+0000 7fc426000640 1 -- 172.21.15.1:0/3822503704 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fc4200052c0 con 0x555ff21f32d0 2024-04-26T01:55:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.403+0000 7fc426000640 1 -- 172.21.15.1:0/3822503704 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc42003c050 con 0x555ff21f32d0 2024-04-26T01:55:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.403+0000 7fc442230740 10 client.25687 fetch_fsmap learned FSMap version 31 2024-04-26T01:55:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.403+0000 7fc442230740 1 -- 172.21.15.1:0/3822503704 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555ff2274310 con 0x555ff21f32d0 2024-04-26T01:55:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.404+0000 7fc426000640 1 -- 172.21.15.1:0/3822503704 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc42000f480 con 0x555ff21f32d0 2024-04-26T01:55:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.404+0000 7fc426000640 1 -- 172.21.15.1:0/3822503704 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7fc4200035f0 con 0x555ff21f32d0 2024-04-26T01:55:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.404+0000 7fc442230740 10 client.25687 fetch_fsmap finished waiting for FSMap version 31 2024-04-26T01:55:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.404+0000 7fc442230740 10 client.25687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:55:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.404+0000 7fc442230740 20 client.25687 populate_metadata read hostname 'smithi001' 2024-04-26T01:55:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.404+0000 7fc442230740 1 --2- 172.21.15.1:0/3822503704 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x555ff2274a00 0x555ff2276e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.404+0000 7fc442230740 4 client.25687 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:55:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.404+0000 7fc442230740 1 -- 172.21.15.1:0/3822503704 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555ff2166320 con 0x555ff2274a00 2024-04-26T01:55:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.404+0000 7fc43d400640 1 --2- 172.21.15.1:0/3822503704 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x555ff2274a00 0x555ff2276e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.405+0000 7fc43d400640 1 --2- 172.21.15.1:0/3822503704 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x555ff2274a00 0x555ff2276e20 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b922638e9ce5bf6e server_cookie=68ac85cd0a5847df in_seq=0 out_seq=0 2024-04-26T01:55:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.405+0000 7fc426000640 10 client.25687 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.418+0000 7fc426000640 1 -- 172.21.15.1:0/3822503704 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555ff2166320 con 0x555ff2274a00 2024-04-26T01:55:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.418+0000 7fc426000640 10 client.25687 handle_command_reply: tid=0 2024-04-26T01:55:32.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.419+0000 7fc442230740 1 client.25687 shutdown 2024-04-26T01:55:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.419+0000 7fc442230740 1 -- 172.21.15.1:0/3822503704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21f32d0 msgr2=0x555ff2203100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.419+0000 7fc442230740 1 --2- 172.21.15.1:0/3822503704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21f32d0 0x555ff2203100 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x555ff21f32a0 tx=0x7fc42001cc00 comp rx=0 tx=0).stop 2024-04-26T01:55:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.419+0000 7fc442230740 1 -- 172.21.15.1:0/3822503704 shutdown_connections 2024-04-26T01:55:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.419+0000 7fc442230740 1 --2- 172.21.15.1:0/3822503704 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x555ff2274a00 0x555ff2276e20 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.419+0000 7fc442230740 1 --2- 172.21.15.1:0/3822503704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff21fdb90 0x555ff2203640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.419+0000 7fc442230740 1 --2- 172.21.15.1:0/3822503704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ff21f32d0 0x555ff2203100 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.419+0000 7fc442230740 1 --2- 172.21.15.1:0/3822503704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ff21f29b0 0x555ff2205d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.419+0000 7fc426000640 0 client.25687 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.419+0000 7fc442230740 1 -- 172.21.15.1:0/3822503704 >> 172.21.15.1:0/3822503704 conn(0x555ff21eeec0 msgr2=0x555ff21f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.419+0000 7fc442230740 1 -- 172.21.15.1:0/3822503704 shutdown_connections 2024-04-26T01:55:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.419+0000 7fc442230740 1 -- 172.21.15.1:0/3822503704 wait complete. 2024-04-26T01:55:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.419+0000 7fc442230740 20 client.25687 trim_cache size 0 max 16384 2024-04-26T01:55:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:55:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:55:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:55:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:55:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.420+0000 7fc43de00640 1 -- 172.21.15.1:0/2294501516 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc41c0a3d80 msgr2=0x7fc41c0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.420+0000 7fc43de00640 1 --2- 172.21.15.1:0/2294501516 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc41c0a3d80 0x7fc41c0a6240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x555ff21f3f50 tx=0x7fc4280d45e0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.420+0000 7fc43de00640 1 -- 172.21.15.1:0/2294501516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc438104200 msgr2=0x7fc4381a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.420+0000 7fc43de00640 1 --2- 172.21.15.1:0/2294501516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc438104200 0x7fc4381a06b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc42800e9f0 tx=0x7fc42800eeb0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.420+0000 7fc43de00640 1 -- 172.21.15.1:0/2294501516 shutdown_connections 2024-04-26T01:55:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.420+0000 7fc43de00640 1 --2- 172.21.15.1:0/2294501516 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc41c0a3d80 0x7fc41c0a6240 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.420+0000 7fc43de00640 1 --2- 172.21.15.1:0/2294501516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc43819a860 0x7fc43819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.420+0000 7fc43de00640 1 --2- 172.21.15.1:0/2294501516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4381092d0 0x7fc4381a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.420+0000 7fc43de00640 1 --2- 172.21.15.1:0/2294501516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc438104200 0x7fc4381a06b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.420+0000 7fc43de00640 1 -- 172.21.15.1:0/2294501516 >> 172.21.15.1:0/2294501516 conn(0x7fc4380ffa60 msgr2=0x7fc438101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.420+0000 7fc43de00640 1 -- 172.21.15.1:0/2294501516 shutdown_connections 2024-04-26T01:55:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:32.421+0000 7fc43de00640 1 -- 172.21.15.1:0/2294501516 wait complete. 2024-04-26T01:55:34.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:33 smithi097 ceph-mon[32853]: pgmap v121: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-26T01:55:34.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:33 smithi173 ceph-mon[32553]: pgmap v121: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-26T01:55:34.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:33 smithi001 ceph-mon[27822]: pgmap v121: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-26T01:55:36.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:35 smithi097 ceph-mon[32853]: pgmap v122: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 KiB/s wr, 10 op/s 2024-04-26T01:55:36.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:35 smithi173 ceph-mon[32553]: pgmap v122: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 KiB/s wr, 10 op/s 2024-04-26T01:55:36.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:35 smithi001 ceph-mon[27822]: pgmap v122: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 KiB/s wr, 10 op/s 2024-04-26T01:55:37.076 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:55:37.544 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:37.956 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:37 smithi001 ceph-mon[27822]: pgmap v123: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2024-04-26T01:55:38.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:37 smithi097 ceph-mon[32853]: pgmap v123: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2024-04-26T01:55:38.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:37 smithi173 ceph-mon[32553]: pgmap v123: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2024-04-26T01:55:39.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.891+0000 7f3809400700 1 -- 172.21.15.1:0/2329199179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f380406b0f0 msgr2=0x7f380410f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:39.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.891+0000 7f3809400700 1 --2- 172.21.15.1:0/2329199179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f380406b0f0 0x7f380410f900 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00b0a0 tx=0x7f37fc00b3b0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.892+0000 7f3809400700 1 -- 172.21.15.1:0/2329199179 shutdown_connections 2024-04-26T01:55:39.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.892+0000 7f3809400700 1 --2- 172.21.15.1:0/2329199179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f380406b0f0 0x7f380410f900 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.892+0000 7f3809400700 1 --2- 172.21.15.1:0/2329199179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3804108a30 0x7f3804108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.892+0000 7f3809400700 1 --2- 172.21.15.1:0/2329199179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f380406a750 0x7f380406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.892+0000 7f3809400700 1 -- 172.21.15.1:0/2329199179 >> 172.21.15.1:0/2329199179 conn(0x7f38040788f0 msgr2=0x7f3804078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:39.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.892+0000 7f3809400700 1 -- 172.21.15.1:0/2329199179 shutdown_connections 2024-04-26T01:55:39.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.892+0000 7f3809400700 1 -- 172.21.15.1:0/2329199179 wait complete. 2024-04-26T01:55:39.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.893+0000 7f3809400700 1 Processor -- start 2024-04-26T01:55:39.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.893+0000 7f3809400700 1 -- start start 2024-04-26T01:55:39.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.894+0000 7f3809400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f380406a750 0x7f380419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.894+0000 7f3809400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f380406b0f0 0x7f380419b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.894+0000 7f3809400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3804108a30 0x7f380419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.894+0000 7f3809400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3804112c20 con 0x7f3804108a30 2024-04-26T01:55:39.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.894+0000 7f3809400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3804112aa0 con 0x7f380406b0f0 2024-04-26T01:55:39.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.894+0000 7f3809400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3804112da0 con 0x7f380406a750 2024-04-26T01:55:39.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.894+0000 7f3803200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3804108a30 0x7f380419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.894+0000 7f3802800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f380406a750 0x7f380419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.894+0000 7f3803200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3804108a30 0x7f380419fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60790/0 (socket says 172.21.15.1:60790) 2024-04-26T01:55:39.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.894+0000 7f3803200700 1 -- 172.21.15.1:0/1751196936 learned_addr learned my addr 172.21.15.1:0/1751196936 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:39.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.895+0000 7f37f9e00700 1 --2- 172.21.15.1:0/1751196936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f380406b0f0 0x7f380419b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.895+0000 7f3803200700 1 -- 172.21.15.1:0/1751196936 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f380406a750 msgr2=0x7f380419b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:39.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.895+0000 7f3803200700 1 --2- 172.21.15.1:0/1751196936 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f380406a750 0x7f380419b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.895+0000 7f3803200700 1 -- 172.21.15.1:0/1751196936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f380406b0f0 msgr2=0x7f380419b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:39.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.895+0000 7f3803200700 1 --2- 172.21.15.1:0/1751196936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f380406b0f0 0x7f380419b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:39.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.895+0000 7f3803200700 1 -- 172.21.15.1:0/1751196936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38041a0300 con 0x7f3804108a30 2024-04-26T01:55:39.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.895+0000 7f37f9e00700 1 --2- 172.21.15.1:0/1751196936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f380406b0f0 0x7f380419b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:55:39.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.895+0000 7f3802800700 1 --2- 172.21.15.1:0/1751196936 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f380406a750 0x7f380419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:39.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.895+0000 7f3803200700 1 --2- 172.21.15.1:0/1751196936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3804108a30 0x7f380419fbc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00b070 tx=0x7f37fc009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:39.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.896+0000 7f37fbe00700 1 -- 172.21.15.1:0/1751196936 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc021070 con 0x7f3804108a30 2024-04-26T01:55:39.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.896+0000 7f3809400700 1 -- 172.21.15.1:0/1751196936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38041a0590 con 0x7f3804108a30 2024-04-26T01:55:39.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.896+0000 7f37fbe00700 1 -- 172.21.15.1:0/1751196936 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37fc007e30 con 0x7f3804108a30 2024-04-26T01:55:39.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.896+0000 7f37fbe00700 1 -- 172.21.15.1:0/1751196936 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc0046a0 con 0x7f3804108a30 2024-04-26T01:55:39.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.896+0000 7f3809400700 1 -- 172.21.15.1:0/1751196936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38041a7e70 con 0x7f3804108a30 2024-04-26T01:55:39.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.897+0000 7f3809400700 1 -- 172.21.15.1:0/1751196936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3804069730 con 0x7f3804108a30 2024-04-26T01:55:39.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.899+0000 7f37fbe00700 1 -- 172.21.15.1:0/1751196936 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f37fc007980 con 0x7f3804108a30 2024-04-26T01:55:39.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.900+0000 7f37fbe00700 1 --2- 172.21.15.1:0/1751196936 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f37e80a3e30 0x7f37e80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:39.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.901+0000 7f37fbe00700 1 -- 172.21.15.1:0/1751196936 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f37fc00f020 con 0x7f3804108a30 2024-04-26T01:55:39.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.901+0000 7f3802800700 1 --2- 172.21.15.1:0/1751196936 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f37e80a3e30 0x7f37e80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:39.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.902+0000 7f3802800700 1 --2- 172.21.15.1:0/1751196936 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f37e80a3e30 0x7f37e80a62f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f37f4009fd0 tx=0x7f37f4009260 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:39.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:39.906+0000 7f37fbe00700 1 -- 172.21.15.1:0/1751196936 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f37fc02a020 con 0x7f3804108a30 2024-04-26T01:55:40.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:39 smithi001 ceph-mon[27822]: pgmap v124: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2024-04-26T01:55:40.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:40.187+0000 7f3809400700 1 -- 172.21.15.1:0/1751196936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38041a8040 con 0x7f3804108a30 2024-04-26T01:55:40.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:40.189+0000 7f37fbe00700 1 -- 172.21.15.1:0/1751196936 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11839 (secure 0 0 0) 0x7f37fc09e250 con 0x7f3804108a30 2024-04-26T01:55:40.187 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:55:40.187 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:55:29.645677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25414":{"gid":25414,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/453427322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":453427322},{"type":"v1","addr":"172.21.15.173:6835","nonce":453427322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24514,24518,24499]},"id":1}]} 2024-04-26T01:55:40.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:40.191+0000 7f3809400700 1 -- 172.21.15.1:0/1751196936 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f37e80a3e30 msgr2=0x7f37e80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:40.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:40.191+0000 7f3809400700 1 --2- 172.21.15.1:0/1751196936 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f37e80a3e30 0x7f37e80a62f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f37f4009fd0 tx=0x7f37f4009260 comp rx=0 tx=0).stop 2024-04-26T01:55:40.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:40.191+0000 7f3809400700 1 -- 172.21.15.1:0/1751196936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3804108a30 msgr2=0x7f380419fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:40.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:40.191+0000 7f3809400700 1 --2- 172.21.15.1:0/1751196936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3804108a30 0x7f380419fbc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00b070 tx=0x7f37fc009c90 comp rx=0 tx=0).stop 2024-04-26T01:55:40.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:40.191+0000 7f3809400700 1 -- 172.21.15.1:0/1751196936 shutdown_connections 2024-04-26T01:55:40.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:40.191+0000 7f3809400700 1 --2- 172.21.15.1:0/1751196936 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f37e80a3e30 0x7f37e80a62f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:40.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:40.192+0000 7f3809400700 1 --2- 172.21.15.1:0/1751196936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3804108a30 0x7f380419fbc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:40.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:40.192+0000 7f3809400700 1 --2- 172.21.15.1:0/1751196936 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f380406a750 0x7f380419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:40.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:40.192+0000 7f3809400700 1 --2- 172.21.15.1:0/1751196936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f380406b0f0 0x7f380419b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:40.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:40.192+0000 7f3809400700 1 -- 172.21.15.1:0/1751196936 >> 172.21.15.1:0/1751196936 conn(0x7f38040788f0 msgr2=0x7f38040ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:40.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:40.192+0000 7f3809400700 1 -- 172.21.15.1:0/1751196936 shutdown_connections 2024-04-26T01:55:40.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:40.192+0000 7f3809400700 1 -- 172.21.15.1:0/1751196936 wait complete. 2024-04-26T01:55:40.192 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 31 2024-04-26T01:55:40.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:39 smithi097 ceph-mon[32853]: pgmap v124: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2024-04-26T01:55:40.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:39 smithi173 ceph-mon[32553]: pgmap v124: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2024-04-26T01:55:40.996 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24518 quiesce db --set-id 9b233c3e --release --await' 2024-04-26T01:55:40.997 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24518 quiesce db --set-id 9b233c3e --release --await 2024-04-26T01:55:41.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:40 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1751196936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:55:41.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:40 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1751196936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:55:41.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:40 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1751196936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:55:41.483 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:42.218 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:41 smithi173 ceph-mon[32553]: pgmap v125: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-04-26T01:55:42.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:41 smithi097 ceph-mon[32853]: pgmap v125: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-04-26T01:55:42.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:41 smithi001 ceph-mon[27822]: pgmap v125: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-04-26T01:55:43.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.728+0000 7fe126a00700 1 -- 172.21.15.1:0/2127634462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe120108f80 msgr2=0x7fe120109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:43.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.728+0000 7fe126a00700 1 --2- 172.21.15.1:0/2127634462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe120108f80 0x7fe120109400 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0098f0 tx=0x7fe10c009c00 comp rx=0 tx=0).stop 2024-04-26T01:55:43.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.728+0000 7fe126a00700 1 -- 172.21.15.1:0/2127634462 shutdown_connections 2024-04-26T01:55:43.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.728+0000 7fe126a00700 1 --2- 172.21.15.1:0/2127634462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe120109940 0x7fe120115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.728+0000 7fe126a00700 1 --2- 172.21.15.1:0/2127634462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe12010f030 0x7fe12010f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.728+0000 7fe126a00700 1 --2- 172.21.15.1:0/2127634462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe120108f80 0x7fe120109400 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.728+0000 7fe126a00700 1 -- 172.21.15.1:0/2127634462 >> 172.21.15.1:0/2127634462 conn(0x7fe1200787d0 msgr2=0x7fe120107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:43.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.728+0000 7fe126a00700 1 -- 172.21.15.1:0/2127634462 shutdown_connections 2024-04-26T01:55:43.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.728+0000 7fe126a00700 1 -- 172.21.15.1:0/2127634462 wait complete. 2024-04-26T01:55:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.729+0000 7fe126a00700 1 Processor -- start 2024-04-26T01:55:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.729+0000 7fe126a00700 1 -- start start 2024-04-26T01:55:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.730+0000 7fe126a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe120108f80 0x7fe12019f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.730+0000 7fe126a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe120109940 0x7fe12019fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.730+0000 7fe126a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe12010f030 0x7fe1201a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.730+0000 7fe126a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe120119140 con 0x7fe12010f030 2024-04-26T01:55:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.730+0000 7fe126a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe120118fc0 con 0x7fe120108f80 2024-04-26T01:55:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.730+0000 7fe126a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe1201192c0 con 0x7fe120109940 2024-04-26T01:55:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.730+0000 7fe126000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe12010f030 0x7fe1201a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.730+0000 7fe126000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe12010f030 0x7fe1201a3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54904/0 (socket says 172.21.15.1:54904) 2024-04-26T01:55:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.730+0000 7fe126000700 1 -- 172.21.15.1:0/847696782 learned_addr learned my addr 172.21.15.1:0/847696782 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.730+0000 7fe124c00700 1 --2- 172.21.15.1:0/847696782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe120109940 0x7fe12019fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:43.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.730+0000 7fe125600700 1 --2- 172.21.15.1:0/847696782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe120108f80 0x7fe12019f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:43.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.731+0000 7fe124c00700 1 -- 172.21.15.1:0/847696782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe120108f80 msgr2=0x7fe12019f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:43.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.731+0000 7fe124c00700 1 --2- 172.21.15.1:0/847696782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe120108f80 0x7fe12019f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.731+0000 7fe124c00700 1 -- 172.21.15.1:0/847696782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe12010f030 msgr2=0x7fe1201a3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.731+0000 7fe124c00700 1 --2- 172.21.15.1:0/847696782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe12010f030 0x7fe1201a3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.731+0000 7fe124c00700 1 -- 172.21.15.1:0/847696782 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1201a4620 con 0x7fe120109940 2024-04-26T01:55:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.731+0000 7fe125600700 1 --2- 172.21.15.1:0/847696782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe120108f80 0x7fe12019f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.731+0000 7fe124c00700 1 --2- 172.21.15.1:0/847696782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe120109940 0x7fe12019fb50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00c010 tx=0x7fe10c00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.732+0000 7fe11e000700 1 -- 172.21.15.1:0/847696782 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c0051c0 con 0x7fe120109940 2024-04-26T01:55:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.732+0000 7fe11e000700 1 -- 172.21.15.1:0/847696782 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe10c00b560 con 0x7fe120109940 2024-04-26T01:55:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.732+0000 7fe126a00700 1 -- 172.21.15.1:0/847696782 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1201a48b0 con 0x7fe120109940 2024-04-26T01:55:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.732+0000 7fe11e000700 1 -- 172.21.15.1:0/847696782 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c003970 con 0x7fe120109940 2024-04-26T01:55:43.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.732+0000 7fe126a00700 1 -- 172.21.15.1:0/847696782 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1201ac190 con 0x7fe120109940 2024-04-26T01:55:43.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.735+0000 7fe11e000700 1 -- 172.21.15.1:0/847696782 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fe10c00b930 con 0x7fe120109940 2024-04-26T01:55:43.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.736+0000 7fe1328c1b80 1 Processor -- start 2024-04-26T01:55:43.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.737+0000 7fe1328c1b80 1 -- start start 2024-04-26T01:55:43.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.737+0000 7fe1328c1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adeaf0 0x559ee1adeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:43.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.737+0000 7fe1328c1b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1adf410 0x559ee1ae9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:43.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.737+0000 7fe1328c1b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1aea2c0 0x559ee1aec6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:43.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.737+0000 7fe1328c1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559ee1a31f90 con 0x559ee1adeaf0 2024-04-26T01:55:43.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.737+0000 7fe1328c1b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559ee19d8830 con 0x559ee1adf410 2024-04-26T01:55:43.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.737+0000 7fe1328c1b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559ee1a2df50 con 0x559ee1aea2c0 2024-04-26T01:55:43.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.737+0000 7fe126000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adeaf0 0x559ee1adeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:43.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.737+0000 7fe126000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adeaf0 0x559ee1adeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54918/0 (socket says 172.21.15.1:54918) 2024-04-26T01:55:43.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.737+0000 7fe126000700 1 -- 172.21.15.1:0/1875162085 learned_addr learned my addr 172.21.15.1:0/1875162085 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:43.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.737+0000 7fe11e000700 1 --2- 172.21.15.1:0/847696782 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe1080a3db0 0x7fe1080a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:43.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.738+0000 7fe125600700 1 --2- 172.21.15.1:0/1875162085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1adf410 0x559ee1ae9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:43.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.738+0000 7fe124c00700 1 --2- 172.21.15.1:0/1875162085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1aea2c0 0x559ee1aec6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:43.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.738+0000 7fe125600700 1 -- 172.21.15.1:0/1875162085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1aea2c0 msgr2=0x559ee1aec6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:43.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.738+0000 7fe125600700 1 --2- 172.21.15.1:0/1875162085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1aea2c0 0x559ee1aec6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.738+0000 7fe11e000700 1 -- 172.21.15.1:0/847696782 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fe10c0d15c0 con 0x7fe120109940 2024-04-26T01:55:43.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.738+0000 7fe125600700 1 -- 172.21.15.1:0/1875162085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adeaf0 msgr2=0x559ee1adeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:43.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.738+0000 7fe125600700 1 --2- 172.21.15.1:0/1875162085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adeaf0 0x559ee1adeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.738+0000 7fe125600700 1 -- 172.21.15.1:0/1875162085 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ee19ce780 con 0x559ee1adf410 2024-04-26T01:55:43.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.738+0000 7fe126000700 1 --2- 172.21.15.1:0/1875162085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adeaf0 0x559ee1adeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:43.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.738+0000 7fe126000700 1 --2- 172.21.15.1:0/847696782 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe1080a3db0 0x7fe1080a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:43.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.738+0000 7fe125600700 1 --2- 172.21.15.1:0/1875162085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1adf410 0x559ee1ae9d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe1140114c0 tx=0x7fe1140117d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adc876785094a7c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:43.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.739+0000 7fe112a00700 1 -- 172.21.15.1:0/1875162085 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114004230 con 0x559ee1adf410 2024-04-26T01:55:43.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.739+0000 7fe112a00700 1 -- 172.21.15.1:0/1875162085 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe114011e50 con 0x559ee1adf410 2024-04-26T01:55:43.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.739+0000 7fe112a00700 1 -- 172.21.15.1:0/1875162085 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11400e6b0 con 0x559ee1adf410 2024-04-26T01:55:43.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.740+0000 7fe1328c1b80 1 -- 172.21.15.1:0/1875162085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1adf410 msgr2=0x559ee1ae9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:43.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.740+0000 7fe126000700 1 --2- 172.21.15.1:0/847696782 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe1080a3db0 0x7fe1080a6270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x559ee1adbe30 tx=0x7fe118004ad0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:43.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.740+0000 7fe1328c1b80 1 --2- 172.21.15.1:0/1875162085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1adf410 0x559ee1ae9d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe1140114c0 tx=0x7fe1140117d0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.740+0000 7fe1328c1b80 1 -- 172.21.15.1:0/1875162085 shutdown_connections 2024-04-26T01:55:43.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.740+0000 7fe1328c1b80 1 --2- 172.21.15.1:0/1875162085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1aea2c0 0x559ee1aec6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.740+0000 7fe1328c1b80 1 --2- 172.21.15.1:0/1875162085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adeaf0 0x559ee1adeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.740+0000 7fe1328c1b80 1 --2- 172.21.15.1:0/1875162085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1adf410 0x559ee1ae9d80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.740+0000 7fe1328c1b80 1 -- 172.21.15.1:0/1875162085 >> 172.21.15.1:0/1875162085 conn(0x559ee1a0d6a0 msgr2=0x559ee1adaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:43.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.740+0000 7fe1328c1b80 1 -- 172.21.15.1:0/1875162085 shutdown_connections 2024-04-26T01:55:43.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.740+0000 7fe1328c1b80 1 -- 172.21.15.1:0/1875162085 wait complete. 2024-04-26T01:55:43.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.741+0000 7fe1328c1b80 1 -- start start 2024-04-26T01:55:43.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.741+0000 7fe1328c1b80 1 Processor -- start 2024-04-26T01:55:43.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.741+0000 7fe1328c1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adeaf0 0x559ee1ae99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:43.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.741+0000 7fe1328c1b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1adf410 0x559ee1ae6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:43.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.741+0000 7fe1328c1b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1aea2c0 0x559ee1ae7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:43.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.741+0000 7fe1328c1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559ee1aece40 con 0x559ee1adeaf0 2024-04-26T01:55:43.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.741+0000 7fe1328c1b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559ee19eb960 con 0x559ee1aea2c0 2024-04-26T01:55:43.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.741+0000 7fe1328c1b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559ee1aecfc0 con 0x559ee1adf410 2024-04-26T01:55:43.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.741+0000 7fe1328c1b80 10 client.0 _ll_register_callbacks cb 0x559ee19ca820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:43.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.742+0000 7fe124c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1adf410 0x559ee1ae6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:43.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.742+0000 7fe125600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adeaf0 0x559ee1ae99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:43.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.742+0000 7fe124c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1adf410 0x559ee1ae6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:39636/0 (socket says 172.21.15.1:39636) 2024-04-26T01:55:43.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.742+0000 7fe124c00700 1 -- 172.21.15.1:0/247171138 learned_addr learned my addr 172.21.15.1:0/247171138 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:43.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.742+0000 7fe126000700 1 --2- 172.21.15.1:0/247171138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1aea2c0 0x559ee1ae7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:43.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.742+0000 7fe124c00700 1 -- 172.21.15.1:0/247171138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1aea2c0 msgr2=0x559ee1ae7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:43.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.742+0000 7fe124c00700 1 --2- 172.21.15.1:0/247171138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1aea2c0 0x559ee1ae7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.742+0000 7fe124c00700 1 -- 172.21.15.1:0/247171138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adeaf0 msgr2=0x559ee1ae99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:43.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.742+0000 7fe124c00700 1 --2- 172.21.15.1:0/247171138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adeaf0 0x559ee1ae99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.742+0000 7fe124c00700 1 -- 172.21.15.1:0/247171138 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ee197f9d0 con 0x559ee1adf410 2024-04-26T01:55:43.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.742+0000 7fe125600700 1 --2- 172.21.15.1:0/247171138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adeaf0 0x559ee1ae99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:43.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.743+0000 7fe124c00700 1 --2- 172.21.15.1:0/247171138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1adf410 0x559ee1ae6cf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0999f0 tx=0x7fe10c00bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62ccb514cfd78303 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:43.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.743+0000 7fe107400700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:55:43.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.743+0000 7fe107400700 1 -- 172.21.15.1:0/247171138 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c09cef0 con 0x559ee1adf410 2024-04-26T01:55:43.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.743+0000 7fe107400700 1 -- 172.21.15.1:0/247171138 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fe10c021380 con 0x559ee1adf410 2024-04-26T01:55:43.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.743+0000 7fe1328c1b80 1 -- 172.21.15.1:0/247171138 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe114011e50 con 0x559ee1adf410 2024-04-26T01:55:43.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.743+0000 7fe107400700 1 -- 172.21.15.1:0/247171138 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe10c0cf130 con 0x559ee1adf410 2024-04-26T01:55:43.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.743+0000 7fe1328c1b80 10 client.25705 fetch_fsmap learned FSMap version 31 2024-04-26T01:55:43.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.743+0000 7fe1328c1b80 1 -- 172.21.15.1:0/247171138 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559ee19c8bf0 con 0x559ee1adf410 2024-04-26T01:55:43.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.744+0000 7fe107400700 1 -- 172.21.15.1:0/247171138 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe114011e50 con 0x559ee1adf410 2024-04-26T01:55:43.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.744+0000 7fe107400700 1 -- 172.21.15.1:0/247171138 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7fe10c09d7f0 con 0x559ee1adf410 2024-04-26T01:55:43.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.744+0000 7fe1328c1b80 10 client.25705 fetch_fsmap finished waiting for FSMap version 31 2024-04-26T01:55:43.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.744+0000 7fe1328c1b80 10 client.25705 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:55:43.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.744+0000 7fe1328c1b80 20 client.25705 populate_metadata read hostname 'smithi001' 2024-04-26T01:55:43.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.744+0000 7fe1328c1b80 1 --2- 172.21.15.1:0/247171138 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559ee1af99b0 0x559ee1afbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:43.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.744+0000 7fe1328c1b80 4 client.25705 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:55:43.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.744+0000 7fe1328c1b80 1 -- 172.21.15.1:0/247171138 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ee1a3f780 con 0x559ee1af99b0 2024-04-26T01:55:43.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.745+0000 7fe125600700 1 --2- 172.21.15.1:0/247171138 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559ee1af99b0 0x559ee1afbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:43.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.745+0000 7fe125600700 1 --2- 172.21.15.1:0/247171138 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559ee1af99b0 0x559ee1afbe30 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c47abe979b05f852 server_cookie=2a90ef3ef853142c in_seq=0 out_seq=0 2024-04-26T01:55:43.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.745+0000 7fe107400700 10 client.25705 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:43.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.746+0000 7fe107400700 1 -- 172.21.15.1:0/247171138 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ee1a3f780 con 0x559ee1af99b0 2024-04-26T01:55:43.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.746+0000 7fe107400700 10 client.25705 handle_command_reply: tid=0 2024-04-26T01:55:43.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.746+0000 7fe1328c1b80 1 client.25705 shutdown 2024-04-26T01:55:43.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.747+0000 7fe1328c1b80 1 -- 172.21.15.1:0/247171138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1adf410 msgr2=0x559ee1ae6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:43.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.747+0000 7fe1328c1b80 1 --2- 172.21.15.1:0/247171138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1adf410 0x559ee1ae6cf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0999f0 tx=0x7fe10c00bcd0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.747+0000 7fe1328c1b80 1 -- 172.21.15.1:0/247171138 shutdown_connections 2024-04-26T01:55:43.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.747+0000 7fe1328c1b80 1 --2- 172.21.15.1:0/247171138 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559ee1af99b0 0x559ee1afbe30 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.747+0000 7fe1328c1b80 1 --2- 172.21.15.1:0/247171138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1aea2c0 0x559ee1ae7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.747+0000 7fe1328c1b80 1 --2- 172.21.15.1:0/247171138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adeaf0 0x559ee1ae99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.747+0000 7fe1328c1b80 1 --2- 172.21.15.1:0/247171138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1adf410 0x559ee1ae6cf0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.747+0000 7fe107400700 0 client.25705 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:43.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.747+0000 7fe1328c1b80 1 -- 172.21.15.1:0/247171138 >> 172.21.15.1:0/247171138 conn(0x559ee1a0d6a0 msgr2=0x559ee1ae0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:43.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.747+0000 7fe1328c1b80 1 -- 172.21.15.1:0/247171138 shutdown_connections 2024-04-26T01:55:43.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.747+0000 7fe1328c1b80 1 -- 172.21.15.1:0/247171138 wait complete. 2024-04-26T01:55:43.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.747+0000 7fe1328c1b80 20 client.25705 trim_cache size 0 max 16384 2024-04-26T01:55:43.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.769+0000 7fe1328c1b80 1 Processor -- start 2024-04-26T01:55:43.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.769+0000 7fe1328c1b80 1 -- start start 2024-04-26T01:55:43.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.769+0000 7fe1328c1b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1adeaf0 0x559ee1ad4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:43.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.769+0000 7fe1328c1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adf410 0x559ee1b01740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:43.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.769+0000 7fe1328c1b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1aea2c0 0x559ee1b01c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:43.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.769+0000 7fe1328c1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe1201b60a0 con 0x559ee1adf410 2024-04-26T01:55:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.769+0000 7fe1328c1b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe1201b3900 con 0x559ee1adeaf0 2024-04-26T01:55:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.769+0000 7fe1328c1b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe1201b1750 con 0x559ee1aea2c0 2024-04-26T01:55:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.770+0000 7fe124c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adf410 0x559ee1b01740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.770+0000 7fe124c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adf410 0x559ee1b01740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54936/0 (socket says 172.21.15.1:54936) 2024-04-26T01:55:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.770+0000 7fe124c00700 1 -- 172.21.15.1:0/1787073804 learned_addr learned my addr 172.21.15.1:0/1787073804 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.770+0000 7fe126000700 1 --2- 172.21.15.1:0/1787073804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1aea2c0 0x559ee1b01c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.770+0000 7fe125600700 1 --2- 172.21.15.1:0/1787073804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1adeaf0 0x559ee1ad4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.770+0000 7fe124c00700 1 -- 172.21.15.1:0/1787073804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1aea2c0 msgr2=0x559ee1b01c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.770+0000 7fe124c00700 1 --2- 172.21.15.1:0/1787073804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1aea2c0 0x559ee1b01c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.770+0000 7fe124c00700 1 -- 172.21.15.1:0/1787073804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1adeaf0 msgr2=0x559ee1ad4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.770+0000 7fe124c00700 1 --2- 172.21.15.1:0/1787073804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1adeaf0 0x559ee1ad4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.770+0000 7fe124c00700 1 -- 172.21.15.1:0/1787073804 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ee1adfb10 con 0x559ee1adf410 2024-04-26T01:55:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.770+0000 7fe126000700 1 --2- 172.21.15.1:0/1787073804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1aea2c0 0x559ee1b01c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:43.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.770+0000 7fe124c00700 1 --2- 172.21.15.1:0/1787073804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adf410 0x559ee1b01740 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe10c09b840 tx=0x7fe10c09d190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341ecf6ed50e7bd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.771+0000 7fe112000700 1 -- 172.21.15.1:0/1787073804 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c0cf130 con 0x559ee1adf410 2024-04-26T01:55:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.771+0000 7fe112000700 1 -- 172.21.15.1:0/1787073804 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe10c09dc90 con 0x559ee1adf410 2024-04-26T01:55:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.771+0000 7fe112000700 1 -- 172.21.15.1:0/1787073804 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c021380 con 0x559ee1adf410 2024-04-26T01:55:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.771+0000 7fe1328c1b80 1 -- 172.21.15.1:0/1787073804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adf410 msgr2=0x559ee1b01740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.771+0000 7fe1328c1b80 1 --2- 172.21.15.1:0/1787073804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adf410 0x559ee1b01740 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe10c09b840 tx=0x7fe10c09d190 comp rx=0 tx=0).stop 2024-04-26T01:55:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.771+0000 7fe1328c1b80 1 -- 172.21.15.1:0/1787073804 shutdown_connections 2024-04-26T01:55:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.771+0000 7fe1328c1b80 1 --2- 172.21.15.1:0/1787073804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1aea2c0 0x559ee1b01c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.771+0000 7fe1328c1b80 1 --2- 172.21.15.1:0/1787073804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1adeaf0 0x559ee1ad4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.771+0000 7fe1328c1b80 1 --2- 172.21.15.1:0/1787073804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adf410 0x559ee1b01740 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.772+0000 7fe1328c1b80 1 -- 172.21.15.1:0/1787073804 >> 172.21.15.1:0/1787073804 conn(0x559ee1a0d6a0 msgr2=0x559ee1ae1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.772+0000 7fe1328c1b80 1 -- 172.21.15.1:0/1787073804 shutdown_connections 2024-04-26T01:55:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.772+0000 7fe1328c1b80 1 -- 172.21.15.1:0/1787073804 wait complete. 2024-04-26T01:55:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.772+0000 7fe1328c1b80 1 -- start start 2024-04-26T01:55:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.772+0000 7fe1328c1b80 1 Processor -- start 2024-04-26T01:55:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.773+0000 7fe1328c1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adeaf0 0x559ee1aeef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.773+0000 7fe1328c1b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1adf410 0x559ee1aef4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.773+0000 7fe1328c1b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1aea2c0 0x559ee1b5ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.773+0000 7fe1328c1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe1201b1dc0 con 0x559ee1adeaf0 2024-04-26T01:55:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.773+0000 7fe1328c1b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe1201b3230 con 0x559ee1adf410 2024-04-26T01:55:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.773+0000 7fe1328c1b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe1201b2490 con 0x559ee1aea2c0 2024-04-26T01:55:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.773+0000 7fe1328c1b80 10 client.0 _ll_register_callbacks cb 0x559ee19ca820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.773+0000 7fe125600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adeaf0 0x559ee1aeef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.773+0000 7fe125600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adeaf0 0x559ee1aeef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54944/0 (socket says 172.21.15.1:54944) 2024-04-26T01:55:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.773+0000 7fe125600700 1 -- 172.21.15.1:0/2032621231 learned_addr learned my addr 172.21.15.1:0/2032621231 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.773+0000 7fe124c00700 1 --2- 172.21.15.1:0/2032621231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1adf410 0x559ee1aef4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.773+0000 7fe126000700 1 --2- 172.21.15.1:0/2032621231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1aea2c0 0x559ee1b5ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.774+0000 7fe125600700 1 -- 172.21.15.1:0/2032621231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1aea2c0 msgr2=0x559ee1b5ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:43.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.774+0000 7fe125600700 1 --2- 172.21.15.1:0/2032621231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1aea2c0 0x559ee1b5ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.774+0000 7fe125600700 1 -- 172.21.15.1:0/2032621231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1adf410 msgr2=0x559ee1aef4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:43.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.774+0000 7fe125600700 1 --2- 172.21.15.1:0/2032621231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1adf410 0x559ee1aef4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.774+0000 7fe125600700 1 -- 172.21.15.1:0/2032621231 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ee1adb470 con 0x559ee1adeaf0 2024-04-26T01:55:43.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.774+0000 7fe124c00700 1 --2- 172.21.15.1:0/2032621231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1adf410 0x559ee1aef4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:43.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.774+0000 7fe125600700 1 --2- 172.21.15.1:0/2032621231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adeaf0 0x559ee1aeef80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe11401b040 tx=0x7fe114009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5ac534fb28d6af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:43.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.774+0000 7fe107400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:55:43.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.774+0000 7fe107400700 1 -- 172.21.15.1:0/2032621231 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114012e60 con 0x559ee1adeaf0 2024-04-26T01:55:43.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.774+0000 7fe1328c1b80 1 -- 172.21.15.1:0/2032621231 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe10c09dc90 con 0x559ee1adeaf0 2024-04-26T01:55:43.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.775+0000 7fe107400700 1 -- 172.21.15.1:0/2032621231 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fe1140051c0 con 0x559ee1adeaf0 2024-04-26T01:55:43.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.775+0000 7fe107400700 1 -- 172.21.15.1:0/2032621231 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe114005da0 con 0x559ee1adeaf0 2024-04-26T01:55:43.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.775+0000 7fe1328c1b80 10 client.15558 fetch_fsmap learned FSMap version 31 2024-04-26T01:55:43.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.775+0000 7fe1328c1b80 1 -- 172.21.15.1:0/2032621231 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559ee1b61450 con 0x559ee1adeaf0 2024-04-26T01:55:43.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.775+0000 7fe107400700 1 -- 172.21.15.1:0/2032621231 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe10c09dc90 con 0x559ee1adeaf0 2024-04-26T01:55:43.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.775+0000 7fe107400700 1 -- 172.21.15.1:0/2032621231 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7fe11403d480 con 0x559ee1adeaf0 2024-04-26T01:55:43.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.775+0000 7fe1328c1b80 10 client.15558 fetch_fsmap finished waiting for FSMap version 31 2024-04-26T01:55:43.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.776+0000 7fe1328c1b80 10 client.15558 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:55:43.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.776+0000 7fe1328c1b80 20 client.15558 populate_metadata read hostname 'smithi001' 2024-04-26T01:55:43.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.776+0000 7fe1328c1b80 1 --2- 172.21.15.1:0/2032621231 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559ee1b61970 0x559ee1b63d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:43.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.776+0000 7fe1328c1b80 4 client.15558 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9b233c3e", "release": true, "await": true}] 2024-04-26T01:55:43.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.776+0000 7fe1328c1b80 1 -- 172.21.15.1:0/2032621231 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9b233c3e", "release": true, "await": true}) -- 0x559ee1a39820 con 0x559ee1b61970 2024-04-26T01:55:43.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.776+0000 7fe124c00700 1 --2- 172.21.15.1:0/2032621231 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559ee1b61970 0x559ee1b63d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:43.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.776+0000 7fe124c00700 1 --2- 172.21.15.1:0/2032621231 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559ee1b61970 0x559ee1b63d90 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e6eb7af74004819 server_cookie=dc83955a1379ee58 in_seq=0 out_seq=0 2024-04-26T01:55:43.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.776+0000 7fe107400700 10 client.15558 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:43.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.959+0000 7fe107400700 1 -- 172.21.15.1:0/2032621231 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x559ee1a39820 con 0x559ee1b61970 2024-04-26T01:55:43.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.959+0000 7fe107400700 10 client.15558 handle_command_reply: tid=0 2024-04-26T01:55:43.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.959+0000 7fe1328c1b80 1 client.15558 shutdown 2024-04-26T01:55:43.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.959+0000 7fe1328c1b80 1 -- 172.21.15.1:0/2032621231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adeaf0 msgr2=0x559ee1aeef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:43.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.959+0000 7fe1328c1b80 1 --2- 172.21.15.1:0/2032621231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adeaf0 0x559ee1aeef80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe11401b040 tx=0x7fe114009b40 comp rx=0 tx=0).stop 2024-04-26T01:55:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.959+0000 7fe1328c1b80 1 -- 172.21.15.1:0/2032621231 shutdown_connections 2024-04-26T01:55:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.959+0000 7fe1328c1b80 1 --2- 172.21.15.1:0/2032621231 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559ee1b61970 0x559ee1b63d90 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.959+0000 7fe1328c1b80 1 --2- 172.21.15.1:0/2032621231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559ee1aea2c0 0x559ee1b5ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.959+0000 7fe1328c1b80 1 --2- 172.21.15.1:0/2032621231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ee1adeaf0 0x559ee1aeef80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.959+0000 7fe1328c1b80 1 --2- 172.21.15.1:0/2032621231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ee1adf410 0x559ee1aef4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.959+0000 7fe1328c1b80 1 -- 172.21.15.1:0/2032621231 >> 172.21.15.1:0/2032621231 conn(0x559ee1a0d6a0 msgr2=0x559ee1ad42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.959+0000 7fe107400700 0 client.15558 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.960+0000 7fe1328c1b80 1 -- 172.21.15.1:0/2032621231 shutdown_connections 2024-04-26T01:55:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.960+0000 7fe1328c1b80 1 -- 172.21.15.1:0/2032621231 wait complete. 2024-04-26T01:55:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.960+0000 7fe1328c1b80 20 client.15558 trim_cache size 0 max 16384 2024-04-26T01:55:43.960 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T01:55:43.960 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 31, 2024-04-26T01:55:43.960 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24518, 2024-04-26T01:55:43.960 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 12, 2024-04-26T01:55:43.960 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-04-26T01:55:43.960 INFO:teuthology.orchestra.run.smithi001.stdout: "9b233c3e": { 2024-04-26T01:55:43.960 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 12, 2024-04-26T01:55:43.961 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-04-26T01:55:43.961 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:55:43.961 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-04-26T01:55:43.961 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:55:43.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T01:55:43.961 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-04-26T01:55:43.961 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 90.0, 2024-04-26T01:55:43.961 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-04-26T01:55:43.961 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-04-26T01:55:43.961 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-04-26T01:55:43.961 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:55:43.962 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-04-26T01:55:43.962 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:55:43.962 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:55:43.962 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:55:43.962 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:55:43.962 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:55:43.962 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:55:43.962 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T01:55:43.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.962+0000 7fe126a00700 1 -- 172.21.15.1:0/847696782 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe1080a3db0 msgr2=0x7fe1080a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:43.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.962+0000 7fe126a00700 1 --2- 172.21.15.1:0/847696782 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe1080a3db0 0x7fe1080a6270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x559ee1adbe30 tx=0x7fe118004ad0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.962+0000 7fe126a00700 1 -- 172.21.15.1:0/847696782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe120109940 msgr2=0x7fe12019fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:43.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.962+0000 7fe126a00700 1 --2- 172.21.15.1:0/847696782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe120109940 0x7fe12019fb50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00c010 tx=0x7fe10c00fa00 comp rx=0 tx=0).stop 2024-04-26T01:55:43.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.962+0000 7fe126a00700 1 -- 172.21.15.1:0/847696782 shutdown_connections 2024-04-26T01:55:43.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.962+0000 7fe126a00700 1 --2- 172.21.15.1:0/847696782 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe1080a3db0 0x7fe1080a6270 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.962+0000 7fe126a00700 1 --2- 172.21.15.1:0/847696782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe12010f030 0x7fe1201a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.962+0000 7fe126a00700 1 --2- 172.21.15.1:0/847696782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe120108f80 0x7fe12019f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.962+0000 7fe126a00700 1 --2- 172.21.15.1:0/847696782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe120109940 0x7fe12019fb50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:43.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.962+0000 7fe126a00700 1 -- 172.21.15.1:0/847696782 >> 172.21.15.1:0/847696782 conn(0x7fe1200787d0 msgr2=0x7fe120079b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:43.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.962+0000 7fe126a00700 1 -- 172.21.15.1:0/847696782 shutdown_connections 2024-04-26T01:55:43.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:55:43.962+0000 7fe126a00700 1 -- 172.21.15.1:0/847696782 wait complete. 2024-04-26T01:55:44.016 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_dio 2024-04-26T01:55:44.017 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] xfsfind 2024-04-26T01:55:44.019 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_reflink_read_race 2024-04-26T01:55:44.026 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fssum 2024-04-26T01:55:44.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:43 smithi097 ceph-mon[32853]: pgmap v126: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.7 KiB/s wr, 7 op/s 2024-04-26T01:55:44.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:43 smithi173 ceph-mon[32553]: pgmap v126: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.7 KiB/s wr, 7 op/s 2024-04-26T01:55:44.369 INFO:tasks.workunit.client.0.smithi001.stdout:Building log-writes 2024-04-26T01:55:44.377 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:43 smithi001 ceph-mon[27822]: pgmap v126: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.7 KiB/s wr, 7 op/s 2024-04-26T01:55:44.441 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] replay-log 2024-04-26T01:55:44.443 INFO:tasks.workunit.client.0.smithi001.stdout:Building perf 2024-04-26T01:55:44.453 INFO:tasks.workunit.client.0.smithi001.stdout:Building aio-dio-regress 2024-04-26T01:55:44.461 INFO:tasks.workunit.client.0.smithi001.stdout:Building vfs 2024-04-26T01:55:44.475 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aiocp 2024-04-26T01:55:44.479 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] vfstest 2024-04-26T01:55:44.779 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-append-write-fallocate-race 2024-04-26T01:55:44.958 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-append-write-read-race 2024-04-26T01:55:45.012 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9b233c3e', seconds elapsed: 47.7 2024-04-26T01:55:45.035 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] mount-idmapped 2024-04-26T01:55:45.137 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-cow-race 2024-04-26T01:55:45.298 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-cycle-write 2024-04-26T01:55:45.448 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-eof-race 2024-04-26T01:55:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.620+0000 7f5ae0200640 1 -- 172.21.15.1:0/1033026253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ad810c590 msgr2=0x7f5ad810e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.620+0000 7f5ae0200640 1 --2- 172.21.15.1:0/1033026253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ad810c590 0x7f5ad810e9c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009be0 tx=0x7f5ad002a750 comp rx=0 tx=0).stop 2024-04-26T01:55:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.620+0000 7f5ae0200640 1 -- 172.21.15.1:0/1033026253 shutdown_connections 2024-04-26T01:55:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.620+0000 7f5ae0200640 1 --2- 172.21.15.1:0/1033026253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ad810c590 0x7f5ad810e9c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.620+0000 7f5ae0200640 1 --2- 172.21.15.1:0/1033026253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad8103ae0 0x7f5ad810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.620+0000 7f5ae0200640 1 --2- 172.21.15.1:0/1033026253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ad81031c0 0x7f5ad81035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.620+0000 7f5ae0200640 1 -- 172.21.15.1:0/1033026253 >> 172.21.15.1:0/1033026253 conn(0x7f5ad8077f70 msgr2=0x7f5ad8078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.620+0000 7f5ae0200640 1 -- 172.21.15.1:0/1033026253 shutdown_connections 2024-04-26T01:55:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.620+0000 7f5ae0200640 1 -- 172.21.15.1:0/1033026253 wait complete. 2024-04-26T01:55:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.621+0000 7f5ae0200640 1 Processor -- start 2024-04-26T01:55:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.622+0000 7f5ae0200640 1 -- start start 2024-04-26T01:55:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.622+0000 7f5ae0200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad81031c0 0x7f5ad81964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.622+0000 7f5ae0200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ad8103ae0 0x7f5ad8196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.622+0000 7f5ae0200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ad810c590 0x7f5ad819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.622+0000 7f5ae0200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad810ffe0 con 0x7f5ad81031c0 2024-04-26T01:55:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.622+0000 7f5ae0200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad810fe60 con 0x7f5ad810c590 2024-04-26T01:55:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.622+0000 7f5ae0200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad8110160 con 0x7f5ad8103ae0 2024-04-26T01:55:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.623+0000 7f5ade200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ad810c590 0x7f5ad819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.623+0000 7f5ade200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ad810c590 0x7f5ad819ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:47278/0 (socket says 172.21.15.1:47278) 2024-04-26T01:55:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.623+0000 7f5ade200640 1 -- 172.21.15.1:0/3102589145 learned_addr learned my addr 172.21.15.1:0/3102589145 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:45.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.623+0000 7f5add800640 1 --2- 172.21.15.1:0/3102589145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad81031c0 0x7f5ad81964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.623+0000 7f5adce00640 1 --2- 172.21.15.1:0/3102589145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ad8103ae0 0x7f5ad8196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.623+0000 7f5ade200640 1 -- 172.21.15.1:0/3102589145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ad8103ae0 msgr2=0x7f5ad8196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.623+0000 7f5ade200640 1 --2- 172.21.15.1:0/3102589145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ad8103ae0 0x7f5ad8196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.623+0000 7f5ade200640 1 -- 172.21.15.1:0/3102589145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad81031c0 msgr2=0x7f5ad81964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.623+0000 7f5ade200640 1 --2- 172.21.15.1:0/3102589145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad81031c0 0x7f5ad81964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.623+0000 7f5ade200640 1 -- 172.21.15.1:0/3102589145 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad819b4d0 con 0x7f5ad810c590 2024-04-26T01:55:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.623+0000 7f5add800640 1 --2- 172.21.15.1:0/3102589145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad81031c0 0x7f5ad81964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:55:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.623+0000 7f5adce00640 1 --2- 172.21.15.1:0/3102589145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ad8103ae0 0x7f5ad8196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:55:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.623+0000 7f5ade200640 1 --2- 172.21.15.1:0/3102589145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ad810c590 0x7f5ad819ada0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009b30 tx=0x7f5ad0003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.624+0000 7f5ad6000640 1 -- 172.21.15.1:0/3102589145 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0041020 con 0x7f5ad810c590 2024-04-26T01:55:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.624+0000 7f5ae0200640 1 -- 172.21.15.1:0/3102589145 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad819b760 con 0x7f5ad810c590 2024-04-26T01:55:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.624+0000 7f5ad6000640 1 -- 172.21.15.1:0/3102589145 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ad00318f0 con 0x7f5ad810c590 2024-04-26T01:55:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.624+0000 7f5ad6000640 1 -- 172.21.15.1:0/3102589145 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0036040 con 0x7f5ad810c590 2024-04-26T01:55:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.624+0000 7f5ae0200640 1 -- 172.21.15.1:0/3102589145 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad81a3050 con 0x7f5ad810c590 2024-04-26T01:55:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.628+0000 7f5ad6000640 1 -- 172.21.15.1:0/3102589145 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f5ad0044020 con 0x7f5ad810c590 2024-04-26T01:55:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.629+0000 7f5ae3012740 1 Processor -- start 2024-04-26T01:55:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.629+0000 7f5ae3012740 1 -- start start 2024-04-26T01:55:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.630+0000 7f5ad6000640 1 --2- 172.21.15.1:0/3102589145 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f5ac00a3d90 0x7f5ac00a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.630+0000 7f5ae3012740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b7a740 0x560d83b7ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.630+0000 7f5ae3012740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d83b7b060 0x560d83b8dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.630+0000 7f5ae3012740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b8e210 0x560d83b90600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.630+0000 7f5ae3012740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560d83ae0900 con 0x560d83b7b060 2024-04-26T01:55:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.630+0000 7f5ae3012740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560d83a86ef0 con 0x560d83b7a740 2024-04-26T01:55:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.630+0000 7f5ae3012740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560d83adc8c0 con 0x560d83b8e210 2024-04-26T01:55:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.630+0000 7f5ad6000640 1 -- 172.21.15.1:0/3102589145 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f5ad00f13c0 con 0x7f5ad810c590 2024-04-26T01:55:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.631+0000 7f5ade200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b8e210 0x560d83b90600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.631+0000 7f5add800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b7a740 0x560d83b7ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.631+0000 7f5add800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b7a740 0x560d83b7ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:47280/0 (socket says 172.21.15.1:47280) 2024-04-26T01:55:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.631+0000 7f5add800640 1 -- 172.21.15.1:0/1557718776 learned_addr learned my addr 172.21.15.1:0/1557718776 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.631+0000 7f5adce00640 1 --2- 172.21.15.1:0/3102589145 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f5ac00a3d90 0x7f5ac00a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.631+0000 7f5add800640 1 -- 172.21.15.1:0/1557718776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b8e210 msgr2=0x560d83b90600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.631+0000 7f5add800640 1 --2- 172.21.15.1:0/1557718776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b8e210 0x560d83b90600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.631+0000 7f5add800640 1 -- 172.21.15.1:0/1557718776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d83b7b060 msgr2=0x560d83b8dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.631+0000 7f5add800640 1 --2- 172.21.15.1:0/1557718776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d83b7b060 0x560d83b8dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.631+0000 7f5add800640 1 -- 172.21.15.1:0/1557718776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d837f3210 con 0x560d83b7a740 2024-04-26T01:55:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.632+0000 7f5add800640 1 --2- 172.21.15.1:0/1557718776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b7a740 0x560d83b7ab20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00ddf0 tx=0x7f5acc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cad1b98bf295a700 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.632+0000 7f5acaa00640 1 -- 172.21.15.1:0/1557718776 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc0165f0 con 0x560d83b7a740 2024-04-26T01:55:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.632+0000 7f5acaa00640 1 -- 172.21.15.1:0/1557718776 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5acc01b070 con 0x560d83b7a740 2024-04-26T01:55:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.632+0000 7f5acaa00640 1 -- 172.21.15.1:0/1557718776 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc0159c0 con 0x560d83b7a740 2024-04-26T01:55:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.633+0000 7f5ae3012740 1 -- 172.21.15.1:0/1557718776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b7a740 msgr2=0x560d83b7ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.633+0000 7f5adce00640 1 --2- 172.21.15.1:0/3102589145 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f5ac00a3d90 0x7f5ac00a6250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x560d83b7d340 tx=0x7f5ac4011810 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.633+0000 7f5ae3012740 1 --2- 172.21.15.1:0/1557718776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b7a740 0x560d83b7ab20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00ddf0 tx=0x7f5acc005f90 comp rx=0 tx=0).stop 2024-04-26T01:55:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.633+0000 7f5ae3012740 1 -- 172.21.15.1:0/1557718776 shutdown_connections 2024-04-26T01:55:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.633+0000 7f5ae3012740 1 --2- 172.21.15.1:0/1557718776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b8e210 0x560d83b90600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.633+0000 7f5ae3012740 1 --2- 172.21.15.1:0/1557718776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d83b7b060 0x560d83b8dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.633+0000 7f5ae3012740 1 --2- 172.21.15.1:0/1557718776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b7a740 0x560d83b7ab20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.633+0000 7f5ae3012740 1 -- 172.21.15.1:0/1557718776 >> 172.21.15.1:0/1557718776 conn(0x560d83b76d30 msgr2=0x560d83b77110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.634+0000 7f5ae3012740 1 -- 172.21.15.1:0/1557718776 shutdown_connections 2024-04-26T01:55:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.634+0000 7f5ae3012740 1 -- 172.21.15.1:0/1557718776 wait complete. 2024-04-26T01:55:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.634+0000 7f5ae3012740 1 -- start start 2024-04-26T01:55:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.635+0000 7f5ae3012740 1 Processor -- start 2024-04-26T01:55:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.635+0000 7f5ae3012740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b7a740 0x560d83b8b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.635+0000 7f5ae3012740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b7b060 0x560d83b8b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.635+0000 7f5ae3012740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d83b8e210 0x560d83b8bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.635+0000 7f5ae3012740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560d83b90e40 con 0x560d83b8e210 2024-04-26T01:55:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.635+0000 7f5ae3012740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560d83a9a310 con 0x560d83b7a740 2024-04-26T01:55:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.635+0000 7f5ae3012740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560d83b90fc0 con 0x560d83b7b060 2024-04-26T01:55:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.636+0000 7f5ae3012740 10 client.0 _ll_register_callbacks cb 0x560d83a68cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.636+0000 7f5adce00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b7b060 0x560d83b8b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.636+0000 7f5adce00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b7b060 0x560d83b8b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:39698/0 (socket says 172.21.15.1:39698) 2024-04-26T01:55:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.636+0000 7f5adce00640 1 -- 172.21.15.1:0/729061122 learned_addr learned my addr 172.21.15.1:0/729061122 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.636+0000 7f5add800640 1 --2- 172.21.15.1:0/729061122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b7a740 0x560d83b8b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.636+0000 7f5adce00640 1 -- 172.21.15.1:0/729061122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b7a740 msgr2=0x560d83b8b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.636+0000 7f5adce00640 1 --2- 172.21.15.1:0/729061122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b7a740 0x560d83b8b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.636+0000 7f5adce00640 1 -- 172.21.15.1:0/729061122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d83b8e210 msgr2=0x560d83b8bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.636+0000 7f5adce00640 1 --2- 172.21.15.1:0/729061122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d83b8e210 0x560d83b8bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.636+0000 7f5adce00640 1 -- 172.21.15.1:0/729061122 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d838bf620 con 0x560d83b7b060 2024-04-26T01:55:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.636+0000 7f5add800640 1 --2- 172.21.15.1:0/729061122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b7a740 0x560d83b8b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:55:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.637+0000 7f5adce00640 1 --2- 172.21.15.1:0/729061122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b7b060 0x560d83b8b710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009df0 tx=0x7f5ac4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1a5f268250703a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.637+0000 7f5abf400640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:55:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.637+0000 7f5abf400640 1 -- 172.21.15.1:0/729061122 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4010930 con 0x560d83b7b060 2024-04-26T01:55:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.637+0000 7f5abf400640 1 -- 172.21.15.1:0/729061122 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f5ac4010ad0 con 0x560d83b7b060 2024-04-26T01:55:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.639+0000 7f5ae3012740 1 -- 172.21.15.1:0/729061122 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5acc01b070 con 0x560d83b7b060 2024-04-26T01:55:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.639+0000 7f5abf400640 1 -- 172.21.15.1:0/729061122 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5ac4010cd0 con 0x560d83b7b060 2024-04-26T01:55:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.639+0000 7f5ae3012740 10 client.25723 fetch_fsmap learned FSMap version 31 2024-04-26T01:55:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.639+0000 7f5ae3012740 1 -- 172.21.15.1:0/729061122 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560d83b80320 con 0x560d83b7b060 2024-04-26T01:55:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.640+0000 7f5abf400640 1 -- 172.21.15.1:0/729061122 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5acc01b070 con 0x560d83b7b060 2024-04-26T01:55:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.640+0000 7f5abf400640 1 -- 172.21.15.1:0/729061122 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7f5ac4018030 con 0x560d83b7b060 2024-04-26T01:55:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.642+0000 7f5ae3012740 10 client.25723 fetch_fsmap finished waiting for FSMap version 31 2024-04-26T01:55:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.642+0000 7f5ae3012740 10 client.25723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:55:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.642+0000 7f5ae3012740 20 client.25723 populate_metadata read hostname 'smithi001' 2024-04-26T01:55:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.642+0000 7f5ae3012740 1 --2- 172.21.15.1:0/729061122 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x560d83b817e0 0x560d83b83ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.642+0000 7f5ae3012740 4 client.25723 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:55:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.642+0000 7f5ae3012740 1 -- 172.21.15.1:0/729061122 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d83af48b0 con 0x560d83b817e0 2024-04-26T01:55:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.642+0000 7f5add800640 1 --2- 172.21.15.1:0/729061122 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x560d83b817e0 0x560d83b83ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.643+0000 7f5add800640 1 --2- 172.21.15.1:0/729061122 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x560d83b817e0 0x560d83b83ca0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ef61445df820fa5 server_cookie=91c965de335a6d41 in_seq=0 out_seq=0 2024-04-26T01:55:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.643+0000 7f5abf400640 10 client.25723 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.644+0000 7f5abf400640 1 -- 172.21.15.1:0/729061122 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d83af48b0 con 0x560d83b817e0 2024-04-26T01:55:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.644+0000 7f5abf400640 10 client.25723 handle_command_reply: tid=0 2024-04-26T01:55:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.644+0000 7f5ae3012740 1 client.25723 shutdown 2024-04-26T01:55:45.643 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-extend-stat 2024-04-26T01:55:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.644+0000 7f5ae3012740 1 -- 172.21.15.1:0/729061122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b7b060 msgr2=0x560d83b8b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.644+0000 7f5ae3012740 1 --2- 172.21.15.1:0/729061122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b7b060 0x560d83b8b710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009df0 tx=0x7f5ac4009260 comp rx=0 tx=0).stop 2024-04-26T01:55:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.644+0000 7f5ae3012740 1 -- 172.21.15.1:0/729061122 shutdown_connections 2024-04-26T01:55:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.644+0000 7f5ae3012740 1 --2- 172.21.15.1:0/729061122 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x560d83b817e0 0x560d83b83ca0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.644+0000 7f5ae3012740 1 --2- 172.21.15.1:0/729061122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d83b8e210 0x560d83b8bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.644+0000 7f5ae3012740 1 --2- 172.21.15.1:0/729061122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b7b060 0x560d83b8b710 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.644+0000 7f5ae3012740 1 --2- 172.21.15.1:0/729061122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b7a740 0x560d83b8b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.644+0000 7f5ae3012740 1 -- 172.21.15.1:0/729061122 >> 172.21.15.1:0/729061122 conn(0x560d83b76d30 msgr2=0x560d83b7cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.644+0000 7f5abf400640 0 client.25723 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.645+0000 7f5ae3012740 1 -- 172.21.15.1:0/729061122 shutdown_connections 2024-04-26T01:55:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.645+0000 7f5ae3012740 1 -- 172.21.15.1:0/729061122 wait complete. 2024-04-26T01:55:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.645+0000 7f5ae3012740 20 client.25723 trim_cache size 0 max 16384 2024-04-26T01:55:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.669+0000 7f5ae3012740 1 Processor -- start 2024-04-26T01:55:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.670+0000 7f5ae3012740 1 -- start start 2024-04-26T01:55:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.670+0000 7f5ae3012740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b7a740 0x560d83b99d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.670+0000 7f5ae3012740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d83b7b060 0x560d83b8a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.670+0000 7f5ae3012740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b8e210 0x560d83b75d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.670+0000 7f5ae3012740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad81ae2d0 con 0x560d83b7b060 2024-04-26T01:55:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.670+0000 7f5ae3012740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad81abb70 con 0x560d83b8e210 2024-04-26T01:55:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.670+0000 7f5ae3012740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad81a99c0 con 0x560d83b7a740 2024-04-26T01:55:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.670+0000 7f5adce00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d83b7b060 0x560d83b8a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.670+0000 7f5add800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b7a740 0x560d83b99d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.670+0000 7f5adce00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d83b7b060 0x560d83b8a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54978/0 (socket says 172.21.15.1:54978) 2024-04-26T01:55:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.670+0000 7f5adce00640 1 -- 172.21.15.1:0/166754968 learned_addr learned my addr 172.21.15.1:0/166754968 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.671+0000 7f5ade200640 1 --2- 172.21.15.1:0/166754968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b8e210 0x560d83b75d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.671+0000 7f5add800640 1 -- 172.21.15.1:0/166754968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b8e210 msgr2=0x560d83b75d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.671+0000 7f5add800640 1 --2- 172.21.15.1:0/166754968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b8e210 0x560d83b75d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.671+0000 7f5add800640 1 -- 172.21.15.1:0/166754968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d83b7b060 msgr2=0x560d83b8a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.671+0000 7f5add800640 1 --2- 172.21.15.1:0/166754968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d83b7b060 0x560d83b8a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.671+0000 7f5add800640 1 -- 172.21.15.1:0/166754968 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d83b89720 con 0x560d83b7a740 2024-04-26T01:55:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.671+0000 7f5add800640 1 --2- 172.21.15.1:0/166754968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b7a740 0x560d83b99d00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5acc000c00 tx=0x7f5acc015560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73f25cf489a1bb5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.672+0000 7f5aca000640 1 -- 172.21.15.1:0/166754968 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc005240 con 0x560d83b7a740 2024-04-26T01:55:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.672+0000 7f5aca000640 1 -- 172.21.15.1:0/166754968 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5acc01b040 con 0x560d83b7a740 2024-04-26T01:55:45.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.672+0000 7f5aca000640 1 -- 172.21.15.1:0/166754968 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc003850 con 0x560d83b7a740 2024-04-26T01:55:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.673+0000 7f5ae3012740 1 -- 172.21.15.1:0/166754968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b7a740 msgr2=0x560d83b99d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.673+0000 7f5ae3012740 1 --2- 172.21.15.1:0/166754968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b7a740 0x560d83b99d00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5acc000c00 tx=0x7f5acc015560 comp rx=0 tx=0).stop 2024-04-26T01:55:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.673+0000 7f5ae3012740 1 -- 172.21.15.1:0/166754968 shutdown_connections 2024-04-26T01:55:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.673+0000 7f5ae3012740 1 --2- 172.21.15.1:0/166754968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b8e210 0x560d83b75d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.673+0000 7f5ae3012740 1 --2- 172.21.15.1:0/166754968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d83b7b060 0x560d83b8a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.673+0000 7f5ae3012740 1 --2- 172.21.15.1:0/166754968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b7a740 0x560d83b99d00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.673+0000 7f5ae3012740 1 -- 172.21.15.1:0/166754968 >> 172.21.15.1:0/166754968 conn(0x560d83b76d30 msgr2=0x560d83b75600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.673+0000 7f5ae3012740 1 -- 172.21.15.1:0/166754968 shutdown_connections 2024-04-26T01:55:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.673+0000 7f5ae3012740 1 -- 172.21.15.1:0/166754968 wait complete. 2024-04-26T01:55:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.673+0000 7f5ae3012740 1 -- start start 2024-04-26T01:55:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.674+0000 7f5ae3012740 1 Processor -- start 2024-04-26T01:55:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.674+0000 7f5ae3012740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b7a740 0x560d83b84c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.674+0000 7f5ae3012740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d83b7b060 0x560d83b851a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.674+0000 7f5ae3012740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b8e210 0x560d83b856e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.674+0000 7f5ae3012740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad81aa030 con 0x560d83b7b060 2024-04-26T01:55:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.674+0000 7f5ae3012740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad81ab4a0 con 0x560d83b8e210 2024-04-26T01:55:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.674+0000 7f5ae3012740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad81aa700 con 0x560d83b7a740 2024-04-26T01:55:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.674+0000 7f5ae3012740 10 client.0 _ll_register_callbacks cb 0x7f5ac4005490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.675+0000 7f5add800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b7a740 0x560d83b84c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.675+0000 7f5add800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b7a740 0x560d83b84c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:39708/0 (socket says 172.21.15.1:39708) 2024-04-26T01:55:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.675+0000 7f5ade200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b8e210 0x560d83b856e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.675+0000 7f5add800640 1 -- 172.21.15.1:0/121217914 learned_addr learned my addr 172.21.15.1:0/121217914 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.675+0000 7f5adce00640 1 --2- 172.21.15.1:0/121217914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d83b7b060 0x560d83b851a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.675+0000 7f5add800640 1 -- 172.21.15.1:0/121217914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b8e210 msgr2=0x560d83b856e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.675+0000 7f5add800640 1 --2- 172.21.15.1:0/121217914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b8e210 0x560d83b856e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.675+0000 7f5add800640 1 -- 172.21.15.1:0/121217914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d83b7b060 msgr2=0x560d83b851a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.675+0000 7f5add800640 1 --2- 172.21.15.1:0/121217914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d83b7b060 0x560d83b851a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.675+0000 7f5add800640 1 -- 172.21.15.1:0/121217914 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d83c0bc10 con 0x560d83b7a740 2024-04-26T01:55:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.675+0000 7f5adce00640 1 --2- 172.21.15.1:0/121217914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d83b7b060 0x560d83b851a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:55:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.676+0000 7f5add800640 1 --2- 172.21.15.1:0/121217914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b7a740 0x560d83b84c60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00b010 tx=0x7f5acc00d450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b030d839ecebdf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.676+0000 7f5abf400640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:55:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.676+0000 7f5abf400640 1 -- 172.21.15.1:0/121217914 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc005610 con 0x560d83b7a740 2024-04-26T01:55:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.676+0000 7f5ae3012740 1 -- 172.21.15.1:0/121217914 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5acc01b040 con 0x560d83b7a740 2024-04-26T01:55:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.676+0000 7f5abf400640 1 -- 172.21.15.1:0/121217914 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f5acc02a330 con 0x560d83b7a740 2024-04-26T01:55:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.677+0000 7f5abf400640 1 -- 172.21.15.1:0/121217914 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5acc02a530 con 0x560d83b7a740 2024-04-26T01:55:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.677+0000 7f5ae3012740 10 client.25729 fetch_fsmap learned FSMap version 31 2024-04-26T01:55:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.677+0000 7f5ae3012740 1 -- 172.21.15.1:0/121217914 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560d83c0cf00 con 0x560d83b7a740 2024-04-26T01:55:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.677+0000 7f5abf400640 1 -- 172.21.15.1:0/121217914 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5acc01b040 con 0x560d83b7a740 2024-04-26T01:55:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.678+0000 7f5abf400640 1 -- 172.21.15.1:0/121217914 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7f5acc0155d0 con 0x560d83b7a740 2024-04-26T01:55:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.678+0000 7f5ae3012740 10 client.25729 fetch_fsmap finished waiting for FSMap version 31 2024-04-26T01:55:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.678+0000 7f5ae3012740 10 client.25729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:55:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.678+0000 7f5ae3012740 20 client.25729 populate_metadata read hostname 'smithi001' 2024-04-26T01:55:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.678+0000 7f5ae3012740 1 --2- 172.21.15.1:0/121217914 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x560d83c0d4d0 0x560d83c0f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.678+0000 7f5ae3012740 4 client.25729 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:55:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.678+0000 7f5ae3012740 1 -- 172.21.15.1:0/121217914 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d83aee320 con 0x560d83c0d4d0 2024-04-26T01:55:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.679+0000 7f5adce00640 1 --2- 172.21.15.1:0/121217914 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x560d83c0d4d0 0x560d83c0f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.679+0000 7f5adce00640 1 --2- 172.21.15.1:0/121217914 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x560d83c0d4d0 0x560d83c0f8f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4100703403c9dff server_cookie=45b450976b5e2064 in_seq=0 out_seq=0 2024-04-26T01:55:45.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.679+0000 7f5abf400640 10 client.25729 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.702+0000 7f5abf400640 1 -- 172.21.15.1:0/121217914 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560d83aee320 con 0x560d83c0d4d0 2024-04-26T01:55:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.702+0000 7f5abf400640 10 client.25729 handle_command_reply: tid=0 2024-04-26T01:55:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.702+0000 7f5ae3012740 1 client.25729 shutdown 2024-04-26T01:55:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.702+0000 7f5ae3012740 1 -- 172.21.15.1:0/121217914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b7a740 msgr2=0x560d83b84c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.702+0000 7f5ae3012740 1 --2- 172.21.15.1:0/121217914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b7a740 0x560d83b84c60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00b010 tx=0x7f5acc00d450 comp rx=0 tx=0).stop 2024-04-26T01:55:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.703+0000 7f5ae3012740 1 -- 172.21.15.1:0/121217914 shutdown_connections 2024-04-26T01:55:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.703+0000 7f5ae3012740 1 --2- 172.21.15.1:0/121217914 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x560d83c0d4d0 0x560d83c0f8f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.703+0000 7f5ae3012740 1 --2- 172.21.15.1:0/121217914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d83b8e210 0x560d83b856e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.703+0000 7f5ae3012740 1 --2- 172.21.15.1:0/121217914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d83b7b060 0x560d83b851a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.703+0000 7f5ae3012740 1 --2- 172.21.15.1:0/121217914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560d83b7a740 0x560d83b84c60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.703+0000 7f5abf400640 0 client.25729 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.703+0000 7f5ae3012740 1 -- 172.21.15.1:0/121217914 >> 172.21.15.1:0/121217914 conn(0x560d83b76d30 msgr2=0x560d83b75e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.703+0000 7f5ae3012740 1 -- 172.21.15.1:0/121217914 shutdown_connections 2024-04-26T01:55:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.703+0000 7f5ae3012740 1 -- 172.21.15.1:0/121217914 wait complete. 2024-04-26T01:55:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.703+0000 7f5ae3012740 20 client.25729 trim_cache size 0 max 16384 2024-04-26T01:55:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:55:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:55:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:55:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:55:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.704+0000 7f5ae0200640 1 -- 172.21.15.1:0/3102589145 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f5ac00a3d90 msgr2=0x7f5ac00a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.704+0000 7f5ae0200640 1 --2- 172.21.15.1:0/3102589145 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f5ac00a3d90 0x7f5ac00a6250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x560d83b7d340 tx=0x7f5ac4011810 comp rx=0 tx=0).stop 2024-04-26T01:55:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.704+0000 7f5ae0200640 1 -- 172.21.15.1:0/3102589145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ad810c590 msgr2=0x7f5ad819ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.704+0000 7f5ae0200640 1 --2- 172.21.15.1:0/3102589145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ad810c590 0x7f5ad819ada0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009b30 tx=0x7f5ad0003ba0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.704+0000 7f5ae0200640 1 -- 172.21.15.1:0/3102589145 shutdown_connections 2024-04-26T01:55:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.704+0000 7f5ae0200640 1 --2- 172.21.15.1:0/3102589145 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f5ac00a3d90 0x7f5ac00a6250 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.704+0000 7f5ae0200640 1 --2- 172.21.15.1:0/3102589145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ad810c590 0x7f5ad819ada0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.704+0000 7f5ae0200640 1 --2- 172.21.15.1:0/3102589145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ad8103ae0 0x7f5ad8196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.704+0000 7f5ae0200640 1 --2- 172.21.15.1:0/3102589145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad81031c0 0x7f5ad81964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.704+0000 7f5ae0200640 1 -- 172.21.15.1:0/3102589145 >> 172.21.15.1:0/3102589145 conn(0x7f5ad8077f70 msgr2=0x7f5ad80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.704+0000 7f5ae0200640 1 -- 172.21.15.1:0/3102589145 shutdown_connections 2024-04-26T01:55:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:45.704+0000 7f5ae0200640 1 -- 172.21.15.1:0/3102589145 wait complete. 2024-04-26T01:55:45.751 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-fcntl-race 2024-04-26T01:55:45.806 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-hole-filling-race 2024-04-26T01:55:45.993 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-invalidate-failure 2024-04-26T01:55:46.048 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-invalidate-readahead 2024-04-26T01:55:46.108 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aiodio_sparse2 2024-04-26T01:55:46.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:45 smithi097 ceph-mon[32853]: pgmap v127: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-26T01:55:46.299 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-subblock-eof-read 2024-04-26T01:55:46.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:45 smithi173 ceph-mon[32553]: pgmap v127: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-26T01:55:46.363 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-write-verify 2024-04-26T01:55:46.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:45 smithi001 ceph-mon[27822]: pgmap v127: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-26T01:55:46.468 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-04-26T01:55:46.591 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-04-26T01:55:46.723 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-last-ref-held-by-io 2024-04-26T01:55:47.307 INFO:tasks.workunit.client.0.smithi001.stderr:+ cd .. 2024-04-26T01:55:47.307 INFO:tasks.workunit.client.0.smithi001.stderr:+ cp xfstests-dev/ltp/fsx . 2024-04-26T01:55:47.322 INFO:tasks.workunit.client.0.smithi001.stderr:+ OPTIONS=-z 2024-04-26T01:55:47.322 INFO:tasks.workunit.client.0.smithi001.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2024-04-26T01:55:47.325 INFO:tasks.workunit.client.0.smithi001.stdout:Seed set to 1 2024-04-26T01:55:47.369 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-26T01:55:47.369 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-26T01:55:47.372 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-26T01:55:47.372 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-26T01:55:47.372 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support clone range, disabling! 2024-04-26T01:55:47.373 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-26T01:55:47.376 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support exchange range, disabling! 2024-04-26T01:55:47.377 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero size read 2024-04-26T01:55:47.377 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero size read 2024-04-26T01:55:47.377 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x81d54 2024-04-26T01:55:47.383 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xb86d7 2024-04-26T01:55:47.395 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xe627e 2024-04-26T01:55:47.461 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0xf3194 2024-04-26T01:55:47.500 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xf3ceb 2024-04-26T01:55:47.514 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0xf47e4 2024-04-26T01:55:47.679 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xfaccc 2024-04-26T01:55:48.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:47 smithi097 ceph-mon[32853]: pgmap v128: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-04-26T01:55:48.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:47 smithi173 ceph-mon[32553]: pgmap v128: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-04-26T01:55:48.357 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xfe423 2024-04-26T01:55:48.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:47 smithi001 ceph-mon[27822]: pgmap v128: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-04-26T01:55:50.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:49 smithi097 ceph-mon[32853]: pgmap v129: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-04-26T01:55:50.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:49 smithi173 ceph-mon[32553]: pgmap v129: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-04-26T01:55:50.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:49 smithi001 ceph-mon[27822]: pgmap v129: 97 pgs: 97 active+clean; 33 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-04-26T01:55:51.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:50 smithi097 ceph-mon[32853]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:55:51.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:50 smithi173 ceph-mon[32553]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:55:51.335 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:50 smithi001 ceph-mon[27822]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T01:55:52.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:51 smithi097 ceph-mon[32853]: pgmap v130: 97 pgs: 97 active+clean; 35 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 652 KiB/s wr, 117 op/s 2024-04-26T01:55:52.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:51 smithi173 ceph-mon[32553]: pgmap v130: 97 pgs: 97 active+clean; 35 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 652 KiB/s wr, 117 op/s 2024-04-26T01:55:52.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:51 smithi001 ceph-mon[27822]: pgmap v130: 97 pgs: 97 active+clean; 35 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 652 KiB/s wr, 117 op/s 2024-04-26T01:55:52.401 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xfe492 2024-04-26T01:55:53.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:52 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:55:53.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:52 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:55:53.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:52 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:55:54.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:53 smithi097 ceph-mon[32853]: pgmap v131: 97 pgs: 97 active+clean; 35 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 652 KiB/s wr, 115 op/s 2024-04-26T01:55:54.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:53 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:55:54.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:53 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:55:54.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:53 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:55:54.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:53 smithi173 ceph-mon[32553]: pgmap v131: 97 pgs: 97 active+clean; 35 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 652 KiB/s wr, 115 op/s 2024-04-26T01:55:54.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:53 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:55:54.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:53 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:55:54.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:53 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:55:54.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:53 smithi001 ceph-mon[27822]: pgmap v131: 97 pgs: 97 active+clean; 35 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 652 KiB/s wr, 115 op/s 2024-04-26T01:55:54.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:53 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:55:54.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:53 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:55:54.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:53 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:55:55.584 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xff038 2024-04-26T01:55:56.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:55 smithi173 ceph-mon[32553]: pgmap v132: 97 pgs: 97 active+clean; 38 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 316 op/s 2024-04-26T01:55:56.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:55 smithi001 ceph-mon[27822]: pgmap v132: 97 pgs: 97 active+clean; 38 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 316 op/s 2024-04-26T01:55:56.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:55 smithi097 ceph-mon[32853]: pgmap v132: 97 pgs: 97 active+clean; 38 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 316 op/s 2024-04-26T01:55:57.021 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0xffcac 2024-04-26T01:55:57.222 INFO:tasks.workunit.client.0.smithi001.stdout:10000 copy from 0x2d237 to 0x30552, (0x331b bytes) at 0x82aed 2024-04-26T01:55:58.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:57 smithi173 ceph-mon[32553]: pgmap v133: 97 pgs: 97 active+clean; 38 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 311 op/s 2024-04-26T01:55:58.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:57 smithi001 ceph-mon[27822]: pgmap v133: 97 pgs: 97 active+clean; 38 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 311 op/s 2024-04-26T01:55:58.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:57 smithi097 ceph-mon[32853]: pgmap v133: 97 pgs: 97 active+clean; 38 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 311 op/s 2024-04-26T01:55:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.870+0000 7f7da9000640 1 -- 172.21.15.1:0/485757352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7da4100ed0 msgr2=0x7f7da41012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.870+0000 7f7da9000640 1 --2- 172.21.15.1:0/485757352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7da4100ed0 0x7f7da41012b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009900 tx=0x7f7d9402a410 comp rx=0 tx=0).stop 2024-04-26T01:55:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.871+0000 7f7da9000640 1 -- 172.21.15.1:0/485757352 shutdown_connections 2024-04-26T01:55:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.871+0000 7f7da9000640 1 --2- 172.21.15.1:0/485757352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7da4105fa0 0x7f7da410db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.871+0000 7f7da9000640 1 --2- 172.21.15.1:0/485757352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7da4101880 0x7f7da41058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.871+0000 7f7da9000640 1 --2- 172.21.15.1:0/485757352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7da4100ed0 0x7f7da41012b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.871+0000 7f7da9000640 1 -- 172.21.15.1:0/485757352 >> 172.21.15.1:0/485757352 conn(0x7f7da40fc730 msgr2=0x7f7da40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.871+0000 7f7da9000640 1 -- 172.21.15.1:0/485757352 shutdown_connections 2024-04-26T01:55:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.871+0000 7f7da9000640 1 -- 172.21.15.1:0/485757352 wait complete. 2024-04-26T01:55:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.872+0000 7f7da9000640 1 Processor -- start 2024-04-26T01:55:58.871 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.873+0000 7f7da9000640 1 -- start start 2024-04-26T01:55:58.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.873+0000 7f7da9000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7da4100ed0 0x7f7da41a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.873+0000 7f7da9000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7da4105fa0 0x7f7da41a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.873+0000 7f7da9000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7da419a5d0 0x7f7da419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.873+0000 7f7da9000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7da4112050 con 0x7f7da4100ed0 2024-04-26T01:55:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.873+0000 7f7da9000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7da4111ed0 con 0x7f7da4105fa0 2024-04-26T01:55:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.873+0000 7f7da9000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7da41121d0 con 0x7f7da419a5d0 2024-04-26T01:55:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.873+0000 7f7d9be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7da4105fa0 0x7f7da41a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.873+0000 7f7d9be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7da4105fa0 0x7f7da41a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:46648/0 (socket says 172.21.15.1:46648) 2024-04-26T01:55:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.873+0000 7f7d9be00640 1 -- 172.21.15.1:0/1382328748 learned_addr learned my addr 172.21.15.1:0/1382328748 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.873+0000 7f7da3400640 1 --2- 172.21.15.1:0/1382328748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7da4100ed0 0x7f7da41a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.873+0000 7f7da3e00640 1 --2- 172.21.15.1:0/1382328748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7da419a5d0 0x7f7da419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.874+0000 7f7d9be00640 1 -- 172.21.15.1:0/1382328748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7da419a5d0 msgr2=0x7f7da419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.874+0000 7f7d9be00640 1 --2- 172.21.15.1:0/1382328748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7da419a5d0 0x7f7da419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.874+0000 7f7d9be00640 1 -- 172.21.15.1:0/1382328748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7da4100ed0 msgr2=0x7f7da41a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.874+0000 7f7d9be00640 1 --2- 172.21.15.1:0/1382328748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7da4100ed0 0x7f7da41a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.874+0000 7f7d9be00640 1 -- 172.21.15.1:0/1382328748 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da419b0f0 con 0x7f7da4105fa0 2024-04-26T01:55:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.874+0000 7f7d9be00640 1 --2- 172.21.15.1:0/1382328748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7da4105fa0 0x7f7da41a0960 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00e9c0 tx=0x7f7d8c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.874+0000 7f7da0c00640 1 -- 172.21.15.1:0/1382328748 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c00cd80 con 0x7f7da4105fa0 2024-04-26T01:55:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.874+0000 7f7da0c00640 1 -- 172.21.15.1:0/1382328748 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d8c0054d0 con 0x7f7da4105fa0 2024-04-26T01:55:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.874+0000 7f7da9000640 1 -- 172.21.15.1:0/1382328748 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da419b3e0 con 0x7f7da4105fa0 2024-04-26T01:55:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.874+0000 7f7da0c00640 1 -- 172.21.15.1:0/1382328748 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c010430 con 0x7f7da4105fa0 2024-04-26T01:55:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.874+0000 7f7da9000640 1 -- 172.21.15.1:0/1382328748 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da41a7680 con 0x7f7da4105fa0 2024-04-26T01:55:58.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.878+0000 7f7dad457740 1 Processor -- start 2024-04-26T01:55:58.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.878+0000 7f7da0c00640 1 -- 172.21.15.1:0/1382328748 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f7d8c004fd0 con 0x7f7da4105fa0 2024-04-26T01:55:58.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.878+0000 7f7dad457740 1 -- start start 2024-04-26T01:55:58.877 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.879+0000 7f7dad457740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e0269b0 0x55c61e026d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.877 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.879+0000 7f7dad457740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e0272d0 0x55c61e031650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.879+0000 7f7dad457740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e031b90 0x55c61e033f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.879+0000 7f7dad457740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c61df8c900 con 0x55c61e0272d0 2024-04-26T01:55:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.879+0000 7f7dad457740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c61df32ef0 con 0x55c61e031b90 2024-04-26T01:55:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.879+0000 7f7dad457740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c61df888c0 con 0x55c61e0269b0 2024-04-26T01:55:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.879+0000 7f7da3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e0269b0 0x55c61e026d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.879+0000 7f7da3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e0269b0 0x55c61e026d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:51852/0 (socket says 172.21.15.1:51852) 2024-04-26T01:55:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.879+0000 7f7da3e00640 1 -- 172.21.15.1:0/3947728379 learned_addr learned my addr 172.21.15.1:0/3947728379 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.879+0000 7f7d9be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e031b90 0x55c61e033f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.879+0000 7f7da3400640 1 --2- 172.21.15.1:0/3947728379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e0272d0 0x55c61e031650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.879+0000 7f7da3e00640 1 -- 172.21.15.1:0/3947728379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e031b90 msgr2=0x55c61e033f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.880+0000 7f7da3e00640 1 --2- 172.21.15.1:0/3947728379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e031b90 0x55c61e033f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.880+0000 7f7da3e00640 1 -- 172.21.15.1:0/3947728379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e0272d0 msgr2=0x55c61e031650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.880+0000 7f7da3e00640 1 --2- 172.21.15.1:0/3947728379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e0272d0 0x55c61e031650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.880+0000 7f7da3e00640 1 -- 172.21.15.1:0/3947728379 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c61dc9f210 con 0x55c61e0269b0 2024-04-26T01:55:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.880+0000 7f7d9be00640 1 --2- 172.21.15.1:0/3947728379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e031b90 0x55c61e033f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.880+0000 7f7da3e00640 1 --2- 172.21.15.1:0/3947728379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e0269b0 0x55c61e026d90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009f40 tx=0x7f7d9c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e7085dc810840c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.880+0000 7f7da0c00640 1 --2- 172.21.15.1:0/1382328748 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7d880a3d80 0x7f7d880a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.880+0000 7f7d98c00640 1 -- 172.21.15.1:0/3947728379 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c003b70 con 0x55c61e0269b0 2024-04-26T01:55:58.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.880+0000 7f7d98c00640 1 -- 172.21.15.1:0/3947728379 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d9c003d10 con 0x55c61e0269b0 2024-04-26T01:55:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.880+0000 7f7d98c00640 1 -- 172.21.15.1:0/3947728379 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c011a00 con 0x55c61e0269b0 2024-04-26T01:55:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.881+0000 7f7da0c00640 1 -- 172.21.15.1:0/1382328748 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f7d8c0d0cc0 con 0x7f7da4105fa0 2024-04-26T01:55:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.881+0000 7f7da3400640 1 --2- 172.21.15.1:0/1382328748 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7d880a3d80 0x7f7d880a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.881+0000 7f7dad457740 1 -- 172.21.15.1:0/3947728379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e0269b0 msgr2=0x55c61e026d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.881+0000 7f7dad457740 1 --2- 172.21.15.1:0/3947728379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e0269b0 0x55c61e026d90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009f40 tx=0x7f7d9c010860 comp rx=0 tx=0).stop 2024-04-26T01:55:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.882+0000 7f7dad457740 1 -- 172.21.15.1:0/3947728379 shutdown_connections 2024-04-26T01:55:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.882+0000 7f7dad457740 1 --2- 172.21.15.1:0/3947728379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e031b90 0x55c61e033f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.882+0000 7f7dad457740 1 --2- 172.21.15.1:0/3947728379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e0272d0 0x55c61e031650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.882+0000 7f7dad457740 1 --2- 172.21.15.1:0/3947728379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e0269b0 0x55c61e026d90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.882+0000 7f7dad457740 1 -- 172.21.15.1:0/3947728379 >> 172.21.15.1:0/3947728379 conn(0x55c61e022ec0 msgr2=0x55c61e0232a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.882+0000 7f7dad457740 1 -- 172.21.15.1:0/3947728379 shutdown_connections 2024-04-26T01:55:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.882+0000 7f7dad457740 1 -- 172.21.15.1:0/3947728379 wait complete. 2024-04-26T01:55:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.883+0000 7f7da3400640 1 --2- 172.21.15.1:0/1382328748 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7d880a3d80 0x7f7d880a6240 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7d94000c00 tx=0x7f7d94005b50 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.883+0000 7f7dad457740 1 -- start start 2024-04-26T01:55:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.883+0000 7f7dad457740 1 Processor -- start 2024-04-26T01:55:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.884+0000 7f7dad457740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e0269b0 0x55c61e02eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.884+0000 7f7dad457740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e0272d0 0x55c61e02f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.884+0000 7f7dad457740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e031b90 0x55c61e02f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.884+0000 7f7dad457740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c61e0347c0 con 0x55c61e0269b0 2024-04-26T01:55:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.884+0000 7f7dad457740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c61df46310 con 0x55c61e031b90 2024-04-26T01:55:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.884+0000 7f7dad457740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c61e034940 con 0x55c61e0272d0 2024-04-26T01:55:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.884+0000 7f7dad457740 10 client.0 _ll_register_callbacks cb 0x55c61df14cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.884+0000 7f7da3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e031b90 0x55c61e02f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.884+0000 7f7da3400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e0269b0 0x55c61e02eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.884+0000 7f7da3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e031b90 0x55c61e02f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:46670/0 (socket says 172.21.15.1:46670) 2024-04-26T01:55:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.884+0000 7f7da3e00640 1 -- 172.21.15.1:0/2167187150 learned_addr learned my addr 172.21.15.1:0/2167187150 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.884+0000 7f7d9be00640 1 --2- 172.21.15.1:0/2167187150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e0272d0 0x55c61e02f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.884+0000 7f7da3e00640 1 -- 172.21.15.1:0/2167187150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e0272d0 msgr2=0x55c61e02f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.885+0000 7f7da3e00640 1 --2- 172.21.15.1:0/2167187150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e0272d0 0x55c61e02f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.885+0000 7f7da3e00640 1 -- 172.21.15.1:0/2167187150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e0269b0 msgr2=0x55c61e02eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.885+0000 7f7da3e00640 1 --2- 172.21.15.1:0/2167187150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e0269b0 0x55c61e02eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.885+0000 7f7da3e00640 1 -- 172.21.15.1:0/2167187150 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c61dd6b620 con 0x55c61e031b90 2024-04-26T01:55:58.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.885+0000 7f7da3e00640 1 --2- 172.21.15.1:0/2167187150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e031b90 0x55c61e02f5c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c006010 tx=0x7f7d9c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9510386063b6203 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.885+0000 7f7d91600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:55:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.885+0000 7f7d91600640 1 -- 172.21.15.1:0/2167187150 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c011740 con 0x55c61e031b90 2024-04-26T01:55:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.885+0000 7f7d91600640 1 -- 172.21.15.1:0/2167187150 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f7d9c0118e0 con 0x55c61e031b90 2024-04-26T01:55:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.885+0000 7f7dad457740 1 -- 172.21.15.1:0/2167187150 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d9c003d10 con 0x55c61e031b90 2024-04-26T01:55:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.886+0000 7f7d91600640 1 -- 172.21.15.1:0/2167187150 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7d9c011ae0 con 0x55c61e031b90 2024-04-26T01:55:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.886+0000 7f7dad457740 10 client.25718 fetch_fsmap learned FSMap version 32 2024-04-26T01:55:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.886+0000 7f7dad457740 1 -- 172.21.15.1:0/2167187150 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c61e040a40 con 0x55c61e031b90 2024-04-26T01:55:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.886+0000 7f7d91600640 1 -- 172.21.15.1:0/2167187150 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d9c003d10 con 0x55c61e031b90 2024-04-26T01:55:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.886+0000 7f7d91600640 1 -- 172.21.15.1:0/2167187150 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f7d9c005040 con 0x55c61e031b90 2024-04-26T01:55:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.886+0000 7f7dad457740 10 client.25718 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:55:58.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.886+0000 7f7dad457740 10 client.25718 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:55:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.886+0000 7f7dad457740 20 client.25718 populate_metadata read hostname 'smithi001' 2024-04-26T01:55:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.886+0000 7f7dad457740 1 --2- 172.21.15.1:0/2167187150 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c61e041e40 0x55c61e044230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.887+0000 7f7dad457740 4 client.25718 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:55:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.887+0000 7f7dad457740 1 -- 172.21.15.1:0/2167187150 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c61dfa08b0 con 0x55c61e041e40 2024-04-26T01:55:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.887+0000 7f7da3400640 1 --2- 172.21.15.1:0/2167187150 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c61e041e40 0x55c61e044230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.887+0000 7f7da3400640 1 --2- 172.21.15.1:0/2167187150 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c61e041e40 0x55c61e044230 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf7256e17985e6f6 server_cookie=23b306396def9f64 in_seq=0 out_seq=0 2024-04-26T01:55:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.887+0000 7f7d91600640 10 client.25718 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.888+0000 7f7d91600640 1 -- 172.21.15.1:0/2167187150 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c61dfa08b0 con 0x55c61e041e40 2024-04-26T01:55:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.888+0000 7f7d91600640 10 client.25718 handle_command_reply: tid=0 2024-04-26T01:55:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.888+0000 7f7dad457740 1 client.25718 shutdown 2024-04-26T01:55:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.889+0000 7f7dad457740 1 -- 172.21.15.1:0/2167187150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e031b90 msgr2=0x55c61e02f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.889+0000 7f7dad457740 1 --2- 172.21.15.1:0/2167187150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e031b90 0x55c61e02f5c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c006010 tx=0x7f7d9c004a40 comp rx=0 tx=0).stop 2024-04-26T01:55:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.889+0000 7f7dad457740 1 -- 172.21.15.1:0/2167187150 shutdown_connections 2024-04-26T01:55:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.889+0000 7f7dad457740 1 --2- 172.21.15.1:0/2167187150 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c61e041e40 0x55c61e044230 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.889+0000 7f7dad457740 1 --2- 172.21.15.1:0/2167187150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e031b90 0x55c61e02f5c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.889+0000 7f7dad457740 1 --2- 172.21.15.1:0/2167187150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e0272d0 0x55c61e02f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.889+0000 7f7dad457740 1 --2- 172.21.15.1:0/2167187150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e0269b0 0x55c61e02eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.889+0000 7f7d91600640 0 client.25718 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.889+0000 7f7dad457740 1 -- 172.21.15.1:0/2167187150 >> 172.21.15.1:0/2167187150 conn(0x55c61e022ec0 msgr2=0x55c61e028c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.889+0000 7f7dad457740 1 -- 172.21.15.1:0/2167187150 shutdown_connections 2024-04-26T01:55:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.889+0000 7f7dad457740 1 -- 172.21.15.1:0/2167187150 wait complete. 2024-04-26T01:55:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.889+0000 7f7dad457740 20 client.25718 trim_cache size 0 max 16384 2024-04-26T01:55:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.909+0000 7f7dad457740 1 Processor -- start 2024-04-26T01:55:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.909+0000 7f7dad457740 1 -- start start 2024-04-26T01:55:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.909+0000 7f7dad457740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e0269b0 0x55c61e021ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.909+0000 7f7dad457740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e0272d0 0x55c61e02eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.909+0000 7f7dad457740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e031b90 0x55c61e02f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.909+0000 7f7dad457740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7da41b2710 con 0x55c61e0269b0 2024-04-26T01:55:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.909+0000 7f7dad457740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7da41affe0 con 0x55c61e0272d0 2024-04-26T01:55:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.909+0000 7f7dad457740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7da41ade30 con 0x55c61e031b90 2024-04-26T01:55:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.910+0000 7f7da3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e031b90 0x55c61e02f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.910+0000 7f7d9be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e0272d0 0x55c61e02eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.910+0000 7f7d9be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e0272d0 0x55c61e02eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:46682/0 (socket says 172.21.15.1:46682) 2024-04-26T01:55:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.910+0000 7f7d9be00640 1 -- 172.21.15.1:0/2147940592 learned_addr learned my addr 172.21.15.1:0/2147940592 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.910+0000 7f7da3e00640 1 --2- 172.21.15.1:0/2147940592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e031b90 0x55c61e02f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:51864/0 (socket says 172.21.15.1:51864) 2024-04-26T01:55:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.910+0000 7f7da3400640 1 --2- 172.21.15.1:0/2147940592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e0269b0 0x55c61e021ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.910+0000 7f7d9be00640 1 -- 172.21.15.1:0/2147940592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e031b90 msgr2=0x55c61e02f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.910+0000 7f7d9be00640 1 --2- 172.21.15.1:0/2147940592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e031b90 0x55c61e02f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.910+0000 7f7d9be00640 1 -- 172.21.15.1:0/2147940592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e0269b0 msgr2=0x55c61e021ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.910+0000 7f7d9be00640 1 --2- 172.21.15.1:0/2147940592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e0269b0 0x55c61e021ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.910+0000 7f7d9be00640 1 -- 172.21.15.1:0/2147940592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c61e028070 con 0x55c61e0272d0 2024-04-26T01:55:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.910+0000 7f7d9be00640 1 --2- 172.21.15.1:0/2147940592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e0272d0 0x55c61e02eb00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c09c960 tx=0x7f7d8c0d4800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b787f93ea1dbd103 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.910+0000 7f7d93e00640 1 -- 172.21.15.1:0/2147940592 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c09ca30 con 0x55c61e0272d0 2024-04-26T01:55:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.911+0000 7f7d93e00640 1 -- 172.21.15.1:0/2147940592 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d8c0046c0 con 0x55c61e0272d0 2024-04-26T01:55:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.911+0000 7f7d93e00640 1 -- 172.21.15.1:0/2147940592 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c09c5c0 con 0x55c61e0272d0 2024-04-26T01:55:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.911+0000 7f7dad457740 1 -- 172.21.15.1:0/2147940592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e0272d0 msgr2=0x55c61e02eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.911+0000 7f7dad457740 1 --2- 172.21.15.1:0/2147940592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e0272d0 0x55c61e02eb00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c09c960 tx=0x7f7d8c0d4800 comp rx=0 tx=0).stop 2024-04-26T01:55:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.911+0000 7f7dad457740 1 -- 172.21.15.1:0/2147940592 shutdown_connections 2024-04-26T01:55:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.912+0000 7f7dad457740 1 --2- 172.21.15.1:0/2147940592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e031b90 0x55c61e02f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.912+0000 7f7dad457740 1 --2- 172.21.15.1:0/2147940592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e0272d0 0x55c61e02eb00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.912+0000 7f7dad457740 1 --2- 172.21.15.1:0/2147940592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e0269b0 0x55c61e021ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.912+0000 7f7dad457740 1 -- 172.21.15.1:0/2147940592 >> 172.21.15.1:0/2147940592 conn(0x55c61e022ec0 msgr2=0x55c61e021790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.912+0000 7f7dad457740 1 -- 172.21.15.1:0/2147940592 shutdown_connections 2024-04-26T01:55:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.912+0000 7f7dad457740 1 -- 172.21.15.1:0/2147940592 wait complete. 2024-04-26T01:55:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.912+0000 7f7dad457740 1 -- start start 2024-04-26T01:55:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.912+0000 7f7dad457740 1 Processor -- start 2024-04-26T01:55:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.913+0000 7f7dad457740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e0269b0 0x55c61e0a8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.913+0000 7f7dad457740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e0272d0 0x55c61e0a9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.913+0000 7f7dad457740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e031b90 0x55c61e0468b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.913+0000 7f7dad457740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7da41ae4a0 con 0x55c61e031b90 2024-04-26T01:55:58.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.913+0000 7f7dad457740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7da41af910 con 0x55c61e0272d0 2024-04-26T01:55:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.913+0000 7f7dad457740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7da41aeb70 con 0x55c61e0269b0 2024-04-26T01:55:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.913+0000 7f7dad457740 10 client.0 _ll_register_callbacks cb 0x7f7d9c009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:55:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.913+0000 7f7d9be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e0272d0 0x55c61e0a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.913+0000 7f7da3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e031b90 0x55c61e0468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.913+0000 7f7da3400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e0269b0 0x55c61e0a8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.913+0000 7f7d9be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e0272d0 0x55c61e0a9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:46694/0 (socket says 172.21.15.1:46694) 2024-04-26T01:55:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.913+0000 7f7d9be00640 1 -- 172.21.15.1:0/4289871180 learned_addr learned my addr 172.21.15.1:0/4289871180 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:55:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.913+0000 7f7da3400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e0269b0 0x55c61e0a8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:51872/0 (socket says 172.21.15.1:51872) 2024-04-26T01:55:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.913+0000 7f7da3e00640 1 --2- 172.21.15.1:0/4289871180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e031b90 0x55c61e0468b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45824/0 (socket says 172.21.15.1:45824) 2024-04-26T01:55:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.914+0000 7f7d9be00640 1 -- 172.21.15.1:0/4289871180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e0269b0 msgr2=0x55c61e0a8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.914+0000 7f7d9be00640 1 --2- 172.21.15.1:0/4289871180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e0269b0 0x55c61e0a8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.914+0000 7f7d9be00640 1 -- 172.21.15.1:0/4289871180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e031b90 msgr2=0x55c61e0468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:55:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.914+0000 7f7d9be00640 1 --2- 172.21.15.1:0/4289871180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e031b90 0x55c61e0468b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.914+0000 7f7d9be00640 1 -- 172.21.15.1:0/4289871180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c61e046fa0 con 0x55c61e0272d0 2024-04-26T01:55:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.914+0000 7f7da3400640 1 --2- 172.21.15.1:0/4289871180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e0269b0 0x55c61e0a8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.914+0000 7f7da3e00640 1 --2- 172.21.15.1:0/4289871180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e031b90 0x55c61e0468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.914+0000 7f7d9be00640 1 --2- 172.21.15.1:0/4289871180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e0272d0 0x55c61e0a9460 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0ce4c0 tx=0x7f7d8c0987e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7709918e4e779b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.914+0000 7f7d91600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:55:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.914+0000 7f7d91600640 1 -- 172.21.15.1:0/4289871180 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c09c7a0 con 0x55c61e0272d0 2024-04-26T01:55:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.914+0000 7f7d91600640 1 -- 172.21.15.1:0/4289871180 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f7d8c0ce690 con 0x55c61e0272d0 2024-04-26T01:55:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.914+0000 7f7dad457740 1 -- 172.21.15.1:0/4289871180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c61df9a320 con 0x55c61e0272d0 2024-04-26T01:55:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.914+0000 7f7d91600640 1 -- 172.21.15.1:0/4289871180 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7d8c09ca30 con 0x55c61e0272d0 2024-04-26T01:55:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.914+0000 7f7dad457740 10 client.25724 fetch_fsmap learned FSMap version 32 2024-04-26T01:55:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.914+0000 7f7dad457740 1 -- 172.21.15.1:0/4289871180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c61e0b06f0 con 0x55c61e0272d0 2024-04-26T01:55:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.915+0000 7f7d91600640 1 -- 172.21.15.1:0/4289871180 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c61df9a320 con 0x55c61e0272d0 2024-04-26T01:55:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.915+0000 7f7d91600640 1 -- 172.21.15.1:0/4289871180 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f7d8c0bd0b0 con 0x55c61e0272d0 2024-04-26T01:55:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.915+0000 7f7dad457740 10 client.25724 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:55:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.915+0000 7f7dad457740 10 client.25724 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:55:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.915+0000 7f7dad457740 20 client.25724 populate_metadata read hostname 'smithi001' 2024-04-26T01:55:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.915+0000 7f7dad457740 1 --2- 172.21.15.1:0/4289871180 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c61e0b0ba0 0x55c61e0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.915+0000 7f7dad457740 4 client.25724 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:55:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.915+0000 7f7dad457740 1 -- 172.21.15.1:0/4289871180 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c61df94190 con 0x55c61e0b0ba0 2024-04-26T01:55:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.915+0000 7f7da3400640 1 --2- 172.21.15.1:0/4289871180 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c61e0b0ba0 0x55c61e0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.916+0000 7f7da3400640 1 --2- 172.21.15.1:0/4289871180 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c61e0b0ba0 0x55c61e0b2fc0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c274c00bf42e702 server_cookie=195ebdfcd09bbdcf in_seq=0 out_seq=0 2024-04-26T01:55:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.916+0000 7f7d91600640 10 client.25724 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.934+0000 7f7d91600640 1 -- 172.21.15.1:0/4289871180 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c61df94190 con 0x55c61e0b0ba0 2024-04-26T01:55:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.934+0000 7f7d91600640 10 client.25724 handle_command_reply: tid=0 2024-04-26T01:55:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.934+0000 7f7dad457740 1 client.25724 shutdown 2024-04-26T01:55:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.934+0000 7f7dad457740 1 -- 172.21.15.1:0/4289871180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e0272d0 msgr2=0x55c61e0a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.934+0000 7f7dad457740 1 --2- 172.21.15.1:0/4289871180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e0272d0 0x55c61e0a9460 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0ce4c0 tx=0x7f7d8c0987e0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.934+0000 7f7dad457740 1 -- 172.21.15.1:0/4289871180 shutdown_connections 2024-04-26T01:55:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.934+0000 7f7dad457740 1 --2- 172.21.15.1:0/4289871180 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c61e0b0ba0 0x55c61e0b2fc0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.934+0000 7f7dad457740 1 --2- 172.21.15.1:0/4289871180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c61e031b90 0x55c61e0468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.934+0000 7f7dad457740 1 --2- 172.21.15.1:0/4289871180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c61e0272d0 0x55c61e0a9460 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.934+0000 7f7dad457740 1 --2- 172.21.15.1:0/4289871180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c61e0269b0 0x55c61e0a8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.934+0000 7f7d91600640 0 client.25724 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:55:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.934+0000 7f7dad457740 1 -- 172.21.15.1:0/4289871180 >> 172.21.15.1:0/4289871180 conn(0x55c61e022ec0 msgr2=0x55c61e03d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.934+0000 7f7dad457740 1 -- 172.21.15.1:0/4289871180 shutdown_connections 2024-04-26T01:55:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.934+0000 7f7dad457740 1 -- 172.21.15.1:0/4289871180 wait complete. 2024-04-26T01:55:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.934+0000 7f7dad457740 20 client.25724 trim_cache size 0 max 16384 2024-04-26T01:55:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:55:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:55:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:55:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:55:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.935+0000 7f7da9000640 1 -- 172.21.15.1:0/1382328748 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7d880a3d80 msgr2=0x7f7d880a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.935+0000 7f7da9000640 1 --2- 172.21.15.1:0/1382328748 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7d880a3d80 0x7f7d880a6240 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7d94000c00 tx=0x7f7d94005b50 comp rx=0 tx=0).stop 2024-04-26T01:55:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.935+0000 7f7da9000640 1 -- 172.21.15.1:0/1382328748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7da4105fa0 msgr2=0x7f7da41a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.935+0000 7f7da9000640 1 --2- 172.21.15.1:0/1382328748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7da4105fa0 0x7f7da41a0960 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00e9c0 tx=0x7f7d8c00ee80 comp rx=0 tx=0).stop 2024-04-26T01:55:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.935+0000 7f7da9000640 1 -- 172.21.15.1:0/1382328748 shutdown_connections 2024-04-26T01:55:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.935+0000 7f7da9000640 1 --2- 172.21.15.1:0/1382328748 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f7d880a3d80 0x7f7d880a6240 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.935+0000 7f7da9000640 1 --2- 172.21.15.1:0/1382328748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7da419a5d0 0x7f7da419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.935+0000 7f7da9000640 1 --2- 172.21.15.1:0/1382328748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7da4105fa0 0x7f7da41a0960 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.935+0000 7f7da9000640 1 --2- 172.21.15.1:0/1382328748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7da4100ed0 0x7f7da41a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.935+0000 7f7da9000640 1 -- 172.21.15.1:0/1382328748 >> 172.21.15.1:0/1382328748 conn(0x7f7da40fc730 msgr2=0x7f7da40fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.935+0000 7f7da9000640 1 -- 172.21.15.1:0/1382328748 shutdown_connections 2024-04-26T01:55:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:55:58.936+0000 7f7da9000640 1 -- 172.21.15.1:0/1382328748 wait complete. 2024-04-26T01:56:00.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:55:59 smithi173 ceph-mon[32553]: pgmap v134: 97 pgs: 97 active+clean; 38 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 311 op/s 2024-04-26T01:56:00.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:55:59 smithi001 ceph-mon[27822]: pgmap v134: 97 pgs: 97 active+clean; 38 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 311 op/s 2024-04-26T01:56:00.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:55:59 smithi097 ceph-mon[32853]: pgmap v134: 97 pgs: 97 active+clean; 38 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 311 op/s 2024-04-26T01:56:02.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:02 smithi173 ceph-mon[32553]: pgmap v135: 97 pgs: 97 active+clean; 31 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 500 op/s 2024-04-26T01:56:02.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:02 smithi001 ceph-mon[27822]: pgmap v135: 97 pgs: 97 active+clean; 31 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 500 op/s 2024-04-26T01:56:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:02 smithi097 ceph-mon[32853]: pgmap v135: 97 pgs: 97 active+clean; 31 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 500 op/s 2024-04-26T01:56:04.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:04 smithi173 ceph-mon[32553]: pgmap v136: 97 pgs: 97 active+clean; 31 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 390 op/s 2024-04-26T01:56:04.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:04 smithi001 ceph-mon[27822]: pgmap v136: 97 pgs: 97 active+clean; 31 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 390 op/s 2024-04-26T01:56:04.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:04 smithi097 ceph-mon[32853]: pgmap v136: 97 pgs: 97 active+clean; 31 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 390 op/s 2024-04-26T01:56:06.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:06 smithi173 ceph-mon[32553]: pgmap v137: 97 pgs: 97 active+clean; 34 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 538 op/s 2024-04-26T01:56:06.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:06 smithi001 ceph-mon[27822]: pgmap v137: 97 pgs: 97 active+clean; 34 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 538 op/s 2024-04-26T01:56:06.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:06 smithi097 ceph-mon[32853]: pgmap v137: 97 pgs: 97 active+clean; 34 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 538 op/s 2024-04-26T01:56:08.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:08 smithi173 ceph-mon[32553]: pgmap v138: 97 pgs: 97 active+clean; 34 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-04-26T01:56:08.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:08 smithi001 ceph-mon[27822]: pgmap v138: 97 pgs: 97 active+clean; 34 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-04-26T01:56:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:08 smithi097 ceph-mon[32853]: pgmap v138: 97 pgs: 97 active+clean; 34 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-04-26T01:56:10.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:10 smithi173 ceph-mon[32553]: pgmap v139: 97 pgs: 97 active+clean; 34 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-04-26T01:56:10.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:10 smithi001 ceph-mon[27822]: pgmap v139: 97 pgs: 97 active+clean; 34 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-04-26T01:56:10.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:10 smithi097 ceph-mon[32853]: pgmap v139: 97 pgs: 97 active+clean; 34 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-04-26T01:56:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.091+0000 7fa497e00640 1 -- 172.21.15.1:0/379279638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa49810c350 msgr2=0x7fa49810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.091+0000 7fa497e00640 1 --2- 172.21.15.1:0/379279638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa49810c350 0x7fa49810e7f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa490009900 tx=0x7fa49002a480 comp rx=0 tx=0).stop 2024-04-26T01:56:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.091+0000 7fa497e00640 1 -- 172.21.15.1:0/379279638 shutdown_connections 2024-04-26T01:56:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.091+0000 7fa497e00640 1 --2- 172.21.15.1:0/379279638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa49810c350 0x7fa49810e7f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.091+0000 7fa497e00640 1 --2- 172.21.15.1:0/379279638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa498103850 0x7fa49810bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.091+0000 7fa497e00640 1 --2- 172.21.15.1:0/379279638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa498102f30 0x7fa498103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.091+0000 7fa497e00640 1 -- 172.21.15.1:0/379279638 >> 172.21.15.1:0/379279638 conn(0x7fa4980fc730 msgr2=0x7fa4980feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.091+0000 7fa497e00640 1 -- 172.21.15.1:0/379279638 shutdown_connections 2024-04-26T01:56:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.091+0000 7fa497e00640 1 -- 172.21.15.1:0/379279638 wait complete. 2024-04-26T01:56:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.092+0000 7fa497e00640 1 Processor -- start 2024-04-26T01:56:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.092+0000 7fa497e00640 1 -- start start 2024-04-26T01:56:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.092+0000 7fa497e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa498102f30 0x7fa49819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.093+0000 7fa497e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa498103850 0x7fa49819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.093+0000 7fa497e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa49810c350 0x7fa49819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.093+0000 7fa497e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa498112030 con 0x7fa49810c350 2024-04-26T01:56:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.093+0000 7fa497e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa498111eb0 con 0x7fa498103850 2024-04-26T01:56:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.093+0000 7fa497e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981121b0 con 0x7fa498102f30 2024-04-26T01:56:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.093+0000 7fa496000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa498103850 0x7fa49819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.093+0000 7fa496a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa498102f30 0x7fa49819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.093+0000 7fa496a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa498102f30 0x7fa49819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:35198/0 (socket says 172.21.15.1:35198) 2024-04-26T01:56:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.093+0000 7fa496000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa498103850 0x7fa49819ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:40416/0 (socket says 172.21.15.1:40416) 2024-04-26T01:56:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.093+0000 7fa496a00640 1 -- 172.21.15.1:0/3301182885 learned_addr learned my addr 172.21.15.1:0/3301182885 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:56:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.093+0000 7fa497400640 1 --2- 172.21.15.1:0/3301182885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa49810c350 0x7fa49819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.093+0000 7fa496000640 1 -- 172.21.15.1:0/3301182885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa498102f30 msgr2=0x7fa49819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.093+0000 7fa496000640 1 --2- 172.21.15.1:0/3301182885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa498102f30 0x7fa49819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.093+0000 7fa496000640 1 -- 172.21.15.1:0/3301182885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa49810c350 msgr2=0x7fa49819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.093+0000 7fa496000640 1 --2- 172.21.15.1:0/3301182885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa49810c350 0x7fa49819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.093+0000 7fa496000640 1 -- 172.21.15.1:0/3301182885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49819f740 con 0x7fa498103850 2024-04-26T01:56:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.093+0000 7fa496a00640 1 --2- 172.21.15.1:0/3301182885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa498102f30 0x7fa49819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.094+0000 7fa496000640 1 --2- 172.21.15.1:0/3301182885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa498103850 0x7fa49819ac90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa48000e7c0 tx=0x7fa48000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.094+0000 7fa48f400640 1 -- 172.21.15.1:0/3301182885 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480009530 con 0x7fa498103850 2024-04-26T01:56:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.094+0000 7fa48f400640 1 -- 172.21.15.1:0/3301182885 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa48000ccc0 con 0x7fa498103850 2024-04-26T01:56:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.094+0000 7fa497e00640 1 -- 172.21.15.1:0/3301182885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa49819fa30 con 0x7fa498103850 2024-04-26T01:56:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.094+0000 7fa48f400640 1 -- 172.21.15.1:0/3301182885 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480004a00 con 0x7fa498103850 2024-04-26T01:56:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.094+0000 7fa497e00640 1 -- 172.21.15.1:0/3301182885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4981a72c0 con 0x7fa498103850 2024-04-26T01:56:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.097+0000 7fa48f400640 1 -- 172.21.15.1:0/3301182885 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fa480004ba0 con 0x7fa498103850 2024-04-26T01:56:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.098+0000 7fa4a0637740 1 Processor -- start 2024-04-26T01:56:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.098+0000 7fa4a0637740 1 -- start start 2024-04-26T01:56:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.099+0000 7fa4a0637740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b889b0 0x562f37b88d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.099+0000 7fa4a0637740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b892d0 0x562f37b93650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.099+0000 7fa4a0637740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b93b90 0x562f37b95f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.099+0000 7fa4a0637740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562f37aee900 con 0x562f37b93b90 2024-04-26T01:56:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.099+0000 7fa4a0637740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562f37a94ef0 con 0x562f37b889b0 2024-04-26T01:56:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.099+0000 7fa4a0637740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562f37aea8c0 con 0x562f37b892d0 2024-04-26T01:56:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.099+0000 7fa497400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b889b0 0x562f37b88d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.099+0000 7fa497400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b889b0 0x562f37b88d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:40420/0 (socket says 172.21.15.1:40420) 2024-04-26T01:56:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.099+0000 7fa497400640 1 -- 172.21.15.1:0/3916881604 learned_addr learned my addr 172.21.15.1:0/3916881604 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:56:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.099+0000 7fa496000640 1 --2- 172.21.15.1:0/3916881604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b93b90 0x562f37b95f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.099+0000 7fa496a00640 1 --2- 172.21.15.1:0/3916881604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b892d0 0x562f37b93650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.100+0000 7fa497400640 1 -- 172.21.15.1:0/3916881604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b892d0 msgr2=0x562f37b93650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.100+0000 7fa497400640 1 --2- 172.21.15.1:0/3916881604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b892d0 0x562f37b93650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.100+0000 7fa497400640 1 -- 172.21.15.1:0/3916881604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b93b90 msgr2=0x562f37b95f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.100+0000 7fa497400640 1 --2- 172.21.15.1:0/3916881604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b93b90 0x562f37b95f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.100+0000 7fa497400640 1 -- 172.21.15.1:0/3916881604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f37801210 con 0x562f37b889b0 2024-04-26T01:56:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.100+0000 7fa496000640 1 --2- 172.21.15.1:0/3916881604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b93b90 0x562f37b95f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:56:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.100+0000 7fa48f400640 1 --2- 172.21.15.1:0/3301182885 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fa47c0a3d80 0x7fa47c0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.100+0000 7fa497400640 1 --2- 172.21.15.1:0/3916881604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b889b0 0x562f37b88d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa49002aa70 tx=0x7fa490005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5152404318aa3661 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.100+0000 7fa48f400640 1 -- 172.21.15.1:0/3301182885 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fa4800d05d0 con 0x7fa498103850 2024-04-26T01:56:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.100+0000 7fa487e00640 1 -- 172.21.15.1:0/3916881604 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490040070 con 0x562f37b889b0 2024-04-26T01:56:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.100+0000 7fa487e00640 1 -- 172.21.15.1:0/3916881604 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4900044c0 con 0x562f37b889b0 2024-04-26T01:56:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.100+0000 7fa487e00640 1 -- 172.21.15.1:0/3916881604 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490035040 con 0x562f37b889b0 2024-04-26T01:56:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.101+0000 7fa496a00640 1 --2- 172.21.15.1:0/3301182885 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fa47c0a3d80 0x7fa47c0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.101+0000 7fa4a0637740 1 -- 172.21.15.1:0/3916881604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b889b0 msgr2=0x562f37b88d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.101+0000 7fa4a0637740 1 --2- 172.21.15.1:0/3916881604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b889b0 0x562f37b88d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa49002aa70 tx=0x7fa490005be0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.101+0000 7fa4a0637740 1 -- 172.21.15.1:0/3916881604 shutdown_connections 2024-04-26T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.101+0000 7fa4a0637740 1 --2- 172.21.15.1:0/3916881604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b93b90 0x562f37b95f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.101+0000 7fa4a0637740 1 --2- 172.21.15.1:0/3916881604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b892d0 0x562f37b93650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.101+0000 7fa4a0637740 1 --2- 172.21.15.1:0/3916881604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b889b0 0x562f37b88d90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.101+0000 7fa4a0637740 1 -- 172.21.15.1:0/3916881604 >> 172.21.15.1:0/3916881604 conn(0x562f37b84ec0 msgr2=0x562f37b852a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.101+0000 7fa4a0637740 1 -- 172.21.15.1:0/3916881604 shutdown_connections 2024-04-26T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.102+0000 7fa496a00640 1 --2- 172.21.15.1:0/3301182885 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fa47c0a3d80 0x7fa47c0a6240 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x562f37b89f50 tx=0x7fa488009500 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.102+0000 7fa4a0637740 1 -- 172.21.15.1:0/3916881604 wait complete. 2024-04-26T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.102+0000 7fa4a0637740 1 -- start start 2024-04-26T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.102+0000 7fa4a0637740 1 Processor -- start 2024-04-26T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.103+0000 7fa4a0637740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b889b0 0x562f37b90be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.103+0000 7fa4a0637740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b892d0 0x562f37b91120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.103+0000 7fa4a0637740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b93b90 0x562f37b91660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.103+0000 7fa4a0637740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562f37b96760 con 0x562f37b889b0 2024-04-26T01:56:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.103+0000 7fa4a0637740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562f37aa8310 con 0x562f37b93b90 2024-04-26T01:56:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.103+0000 7fa4a0637740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562f37b968e0 con 0x562f37b892d0 2024-04-26T01:56:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.103+0000 7fa4a0637740 10 client.0 _ll_register_callbacks cb 0x562f37a76cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.103+0000 7fa496000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b892d0 0x562f37b91120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.103+0000 7fa496000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b892d0 0x562f37b91120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:35218/0 (socket says 172.21.15.1:35218) 2024-04-26T01:56:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.103+0000 7fa496a00640 1 --2- 172.21.15.1:0/2364339335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b889b0 0x562f37b90be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.103+0000 7fa496000640 1 -- 172.21.15.1:0/2364339335 learned_addr learned my addr 172.21.15.1:0/2364339335 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:56:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.103+0000 7fa497400640 1 --2- 172.21.15.1:0/2364339335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b93b90 0x562f37b91660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.104+0000 7fa496000640 1 -- 172.21.15.1:0/2364339335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b93b90 msgr2=0x562f37b91660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.104+0000 7fa496000640 1 --2- 172.21.15.1:0/2364339335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b93b90 0x562f37b91660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.104+0000 7fa496000640 1 -- 172.21.15.1:0/2364339335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b889b0 msgr2=0x562f37b90be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.104+0000 7fa496000640 1 --2- 172.21.15.1:0/2364339335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b889b0 0x562f37b90be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.104+0000 7fa496000640 1 -- 172.21.15.1:0/2364339335 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f378cd620 con 0x562f37b892d0 2024-04-26T01:56:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.104+0000 7fa497400640 1 --2- 172.21.15.1:0/2364339335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b93b90 0x562f37b91660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.104+0000 7fa496000640 1 --2- 172.21.15.1:0/2364339335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b892d0 0x562f37b91120 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa48000e790 tx=0x7fa480004e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2434c72c792e00cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.104+0000 7fa484c00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:56:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.104+0000 7fa484c00640 1 -- 172.21.15.1:0/2364339335 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48001c070 con 0x562f37b892d0 2024-04-26T01:56:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.104+0000 7fa484c00640 1 -- 172.21.15.1:0/2364339335 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fa4800a4cf0 con 0x562f37b892d0 2024-04-26T01:56:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.104+0000 7fa4a0637740 1 -- 172.21.15.1:0/2364339335 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562f37b028b0 con 0x562f37b892d0 2024-04-26T01:56:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.104+0000 7fa484c00640 1 -- 172.21.15.1:0/2364339335 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa4800b5070 con 0x562f37b892d0 2024-04-26T01:56:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.105+0000 7fa4a0637740 10 client.25759 fetch_fsmap learned FSMap version 32 2024-04-26T01:56:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.105+0000 7fa4a0637740 1 -- 172.21.15.1:0/2364339335 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562f37ba2ae0 con 0x562f37b892d0 2024-04-26T01:56:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.105+0000 7fa484c00640 1 -- 172.21.15.1:0/2364339335 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562f37b028b0 con 0x562f37b892d0 2024-04-26T01:56:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.105+0000 7fa484c00640 1 -- 172.21.15.1:0/2364339335 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fa4800bb250 con 0x562f37b892d0 2024-04-26T01:56:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.105+0000 7fa4a0637740 10 client.25759 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:56:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.105+0000 7fa4a0637740 10 client.25759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:56:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.105+0000 7fa4a0637740 20 client.25759 populate_metadata read hostname 'smithi001' 2024-04-26T01:56:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.105+0000 7fa4a0637740 1 --2- 172.21.15.1:0/2364339335 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562f37ba3f50 0x562f37ba6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.105+0000 7fa4a0637740 4 client.25759 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:56:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.105+0000 7fa4a0637740 1 -- 172.21.15.1:0/2364339335 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f37afc320 con 0x562f37ba3f50 2024-04-26T01:56:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.105+0000 7fa496a00640 1 --2- 172.21.15.1:0/2364339335 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562f37ba3f50 0x562f37ba6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.106+0000 7fa496a00640 1 --2- 172.21.15.1:0/2364339335 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562f37ba3f50 0x562f37ba6340 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fc6e35fb5d89e78 server_cookie=1c575414e1456bcb in_seq=0 out_seq=0 2024-04-26T01:56:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.106+0000 7fa484c00640 10 client.25759 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:56:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.107+0000 7fa484c00640 1 -- 172.21.15.1:0/2364339335 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562f37afc320 con 0x562f37ba3f50 2024-04-26T01:56:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.107+0000 7fa484c00640 10 client.25759 handle_command_reply: tid=0 2024-04-26T01:56:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.107+0000 7fa4a0637740 1 client.25759 shutdown 2024-04-26T01:56:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.107+0000 7fa4a0637740 1 -- 172.21.15.1:0/2364339335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b892d0 msgr2=0x562f37b91120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.107+0000 7fa4a0637740 1 --2- 172.21.15.1:0/2364339335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b892d0 0x562f37b91120 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa48000e790 tx=0x7fa480004e90 comp rx=0 tx=0).stop 2024-04-26T01:56:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.107+0000 7fa4a0637740 1 -- 172.21.15.1:0/2364339335 shutdown_connections 2024-04-26T01:56:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.107+0000 7fa4a0637740 1 --2- 172.21.15.1:0/2364339335 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562f37ba3f50 0x562f37ba6340 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.107+0000 7fa4a0637740 1 --2- 172.21.15.1:0/2364339335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b93b90 0x562f37b91660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.107+0000 7fa4a0637740 1 --2- 172.21.15.1:0/2364339335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b892d0 0x562f37b91120 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.107+0000 7fa4a0637740 1 --2- 172.21.15.1:0/2364339335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b889b0 0x562f37b90be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.107+0000 7fa484c00640 0 client.25759 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:56:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.107+0000 7fa4a0637740 1 -- 172.21.15.1:0/2364339335 >> 172.21.15.1:0/2364339335 conn(0x562f37b84ec0 msgr2=0x562f37b8abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.107+0000 7fa4a0637740 1 -- 172.21.15.1:0/2364339335 shutdown_connections 2024-04-26T01:56:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.107+0000 7fa4a0637740 1 -- 172.21.15.1:0/2364339335 wait complete. 2024-04-26T01:56:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.108+0000 7fa4a0637740 20 client.25759 trim_cache size 0 max 16384 2024-04-26T01:56:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.126+0000 7fa4a0637740 1 Processor -- start 2024-04-26T01:56:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.126+0000 7fa4a0637740 1 -- start start 2024-04-26T01:56:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.126+0000 7fa4a0637740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b889b0 0x562f37b9f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.126+0000 7fa4a0637740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b892d0 0x562f37b83ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.126+0000 7fa4a0637740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b93b90 0x562f37babbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.126+0000 7fa4a0637740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981b2550 con 0x562f37b892d0 2024-04-26T01:56:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.126+0000 7fa4a0637740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981afe20 con 0x562f37b889b0 2024-04-26T01:56:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.126+0000 7fa4a0637740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981adc70 con 0x562f37b93b90 2024-04-26T01:56:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.127+0000 7fa496a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b889b0 0x562f37b9f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.127+0000 7fa496a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b889b0 0x562f37b9f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:40442/0 (socket says 172.21.15.1:40442) 2024-04-26T01:56:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.127+0000 7fa496a00640 1 -- 172.21.15.1:0/3054991118 learned_addr learned my addr 172.21.15.1:0/3054991118 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:56:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.127+0000 7fa496000640 1 --2- 172.21.15.1:0/3054991118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b892d0 0x562f37b83ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.127+0000 7fa497400640 1 --2- 172.21.15.1:0/3054991118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b93b90 0x562f37babbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.127+0000 7fa496a00640 1 -- 172.21.15.1:0/3054991118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b93b90 msgr2=0x562f37babbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.127+0000 7fa496a00640 1 --2- 172.21.15.1:0/3054991118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b93b90 0x562f37babbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.127+0000 7fa496a00640 1 -- 172.21.15.1:0/3054991118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b892d0 msgr2=0x562f37b83ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.127+0000 7fa496a00640 1 --2- 172.21.15.1:0/3054991118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b892d0 0x562f37b83ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.127+0000 7fa496a00640 1 -- 172.21.15.1:0/3054991118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f37ba6920 con 0x562f37b889b0 2024-04-26T01:56:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.127+0000 7fa496a00640 1 --2- 172.21.15.1:0/3054991118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b889b0 0x562f37b9f620 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa4880117b0 tx=0x7fa488019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac64c2cb64d12f59 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.128+0000 7fa487400640 1 -- 172.21.15.1:0/3054991118 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48802f070 con 0x562f37b889b0 2024-04-26T01:56:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.128+0000 7fa487400640 1 -- 172.21.15.1:0/3054991118 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa488003e40 con 0x562f37b889b0 2024-04-26T01:56:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.128+0000 7fa487400640 1 -- 172.21.15.1:0/3054991118 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa488004540 con 0x562f37b889b0 2024-04-26T01:56:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.128+0000 7fa4a0637740 1 -- 172.21.15.1:0/3054991118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b889b0 msgr2=0x562f37b9f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.129+0000 7fa4a0637740 1 --2- 172.21.15.1:0/3054991118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b889b0 0x562f37b9f620 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa4880117b0 tx=0x7fa488019000 comp rx=0 tx=0).stop 2024-04-26T01:56:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.129+0000 7fa4a0637740 1 -- 172.21.15.1:0/3054991118 shutdown_connections 2024-04-26T01:56:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.129+0000 7fa4a0637740 1 --2- 172.21.15.1:0/3054991118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b93b90 0x562f37babbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.129+0000 7fa4a0637740 1 --2- 172.21.15.1:0/3054991118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b892d0 0x562f37b83ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.129+0000 7fa4a0637740 1 --2- 172.21.15.1:0/3054991118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b889b0 0x562f37b9f620 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.129+0000 7fa4a0637740 1 -- 172.21.15.1:0/3054991118 >> 172.21.15.1:0/3054991118 conn(0x562f37b84ec0 msgr2=0x562f37b83790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.129+0000 7fa4a0637740 1 -- 172.21.15.1:0/3054991118 shutdown_connections 2024-04-26T01:56:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.129+0000 7fa4a0637740 1 -- 172.21.15.1:0/3054991118 wait complete. 2024-04-26T01:56:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.129+0000 7fa4a0637740 1 -- start start 2024-04-26T01:56:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.129+0000 7fa4a0637740 1 Processor -- start 2024-04-26T01:56:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.130+0000 7fa4a0637740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b889b0 0x562f37b99d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.130+0000 7fa4a0637740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b892d0 0x562f37b9a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.130+0000 7fa4a0637740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b93b90 0x562f37c09430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.130+0000 7fa4a0637740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981ae2e0 con 0x562f37b892d0 2024-04-26T01:56:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.130+0000 7fa4a0637740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981af750 con 0x562f37b93b90 2024-04-26T01:56:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.130+0000 7fa4a0637740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981ae9b0 con 0x562f37b889b0 2024-04-26T01:56:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.130+0000 7fa4a0637740 10 client.0 _ll_register_callbacks cb 0x7fa48000ce70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.130+0000 7fa496000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b892d0 0x562f37b9a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.130+0000 7fa496a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b889b0 0x562f37b99d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.130+0000 7fa496000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b892d0 0x562f37b9a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59258/0 (socket says 172.21.15.1:59258) 2024-04-26T01:56:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.130+0000 7fa496000640 1 -- 172.21.15.1:0/3717935176 learned_addr learned my addr 172.21.15.1:0/3717935176 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:56:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.130+0000 7fa497400640 1 --2- 172.21.15.1:0/3717935176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b93b90 0x562f37c09430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.131+0000 7fa496a00640 1 -- 172.21.15.1:0/3717935176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b93b90 msgr2=0x562f37c09430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.131+0000 7fa496a00640 1 --2- 172.21.15.1:0/3717935176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b93b90 0x562f37c09430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.131+0000 7fa496a00640 1 -- 172.21.15.1:0/3717935176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b892d0 msgr2=0x562f37b9a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.131+0000 7fa496a00640 1 --2- 172.21.15.1:0/3717935176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b892d0 0x562f37b9a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.131+0000 7fa496a00640 1 -- 172.21.15.1:0/3717935176 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f37c09a20 con 0x562f37b889b0 2024-04-26T01:56:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.131+0000 7fa496a00640 1 --2- 172.21.15.1:0/3717935176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b889b0 0x562f37b99d50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa48802d040 tx=0x7fa488004000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65392b188395aea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.131+0000 7fa484c00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:56:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.131+0000 7fa484c00640 1 -- 172.21.15.1:0/3717935176 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa488039060 con 0x562f37b889b0 2024-04-26T01:56:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.132+0000 7fa484c00640 1 -- 172.21.15.1:0/3717935176 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fa488004c40 con 0x562f37b889b0 2024-04-26T01:56:12.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.132+0000 7fa4a0637740 1 -- 172.21.15.1:0/3717935176 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562f37af6190 con 0x562f37b889b0 2024-04-26T01:56:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.132+0000 7fa484c00640 1 -- 172.21.15.1:0/3717935176 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa48802f040 con 0x562f37b889b0 2024-04-26T01:56:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.132+0000 7fa4a0637740 10 client.25765 fetch_fsmap learned FSMap version 32 2024-04-26T01:56:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.132+0000 7fa4a0637740 1 -- 172.21.15.1:0/3717935176 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562f37c0ad10 con 0x562f37b889b0 2024-04-26T01:56:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.132+0000 7fa484c00640 1 -- 172.21.15.1:0/3717935176 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562f37af6190 con 0x562f37b889b0 2024-04-26T01:56:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.132+0000 7fa484c00640 1 -- 172.21.15.1:0/3717935176 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fa488004660 con 0x562f37b889b0 2024-04-26T01:56:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.132+0000 7fa4a0637740 10 client.25765 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:56:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.132+0000 7fa4a0637740 10 client.25765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:56:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.132+0000 7fa4a0637740 20 client.25765 populate_metadata read hostname 'smithi001' 2024-04-26T01:56:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.132+0000 7fa4a0637740 1 --2- 172.21.15.1:0/3717935176 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562f37c0b370 0x562f37c0d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.132+0000 7fa4a0637740 4 client.25765 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:56:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.132+0000 7fa4a0637740 1 -- 172.21.15.1:0/3717935176 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f37a724b0 con 0x562f37c0b370 2024-04-26T01:56:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.133+0000 7fa496000640 1 --2- 172.21.15.1:0/3717935176 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562f37c0b370 0x562f37c0d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.133+0000 7fa496000640 1 --2- 172.21.15.1:0/3717935176 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562f37c0b370 0x562f37c0d790 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c2a3ac8c04b3e3d server_cookie=b34d6263edba6204 in_seq=0 out_seq=0 2024-04-26T01:56:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.133+0000 7fa484c00640 10 client.25765 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:56:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.146+0000 7fa484c00640 1 -- 172.21.15.1:0/3717935176 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562f37a724b0 con 0x562f37c0b370 2024-04-26T01:56:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.146+0000 7fa484c00640 10 client.25765 handle_command_reply: tid=0 2024-04-26T01:56:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.146+0000 7fa4a0637740 1 client.25765 shutdown 2024-04-26T01:56:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.146+0000 7fa4a0637740 1 -- 172.21.15.1:0/3717935176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b889b0 msgr2=0x562f37b99d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.146+0000 7fa4a0637740 1 --2- 172.21.15.1:0/3717935176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b889b0 0x562f37b99d50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa48802d040 tx=0x7fa488004000 comp rx=0 tx=0).stop 2024-04-26T01:56:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.146+0000 7fa4a0637740 1 -- 172.21.15.1:0/3717935176 shutdown_connections 2024-04-26T01:56:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.146+0000 7fa4a0637740 1 --2- 172.21.15.1:0/3717935176 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562f37c0b370 0x562f37c0d790 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.146+0000 7fa4a0637740 1 --2- 172.21.15.1:0/3717935176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562f37b93b90 0x562f37c09430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.146+0000 7fa4a0637740 1 --2- 172.21.15.1:0/3717935176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562f37b892d0 0x562f37b9a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.146+0000 7fa4a0637740 1 --2- 172.21.15.1:0/3717935176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562f37b889b0 0x562f37b99d50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.146+0000 7fa484c00640 0 client.25765 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:56:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.146+0000 7fa4a0637740 1 -- 172.21.15.1:0/3717935176 >> 172.21.15.1:0/3717935176 conn(0x562f37b84ec0 msgr2=0x562f37b8abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.146+0000 7fa4a0637740 1 -- 172.21.15.1:0/3717935176 shutdown_connections 2024-04-26T01:56:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.147+0000 7fa4a0637740 1 -- 172.21.15.1:0/3717935176 wait complete. 2024-04-26T01:56:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.147+0000 7fa4a0637740 20 client.25765 trim_cache size 0 max 16384 2024-04-26T01:56:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:56:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:56:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:56:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:56:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.147+0000 7fa497e00640 1 -- 172.21.15.1:0/3301182885 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fa47c0a3d80 msgr2=0x7fa47c0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.147+0000 7fa497e00640 1 --2- 172.21.15.1:0/3301182885 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fa47c0a3d80 0x7fa47c0a6240 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x562f37b89f50 tx=0x7fa488009500 comp rx=0 tx=0).stop 2024-04-26T01:56:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.147+0000 7fa497e00640 1 -- 172.21.15.1:0/3301182885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa498103850 msgr2=0x7fa49819ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.147+0000 7fa497e00640 1 --2- 172.21.15.1:0/3301182885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa498103850 0x7fa49819ac90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa48000e7c0 tx=0x7fa48000ec80 comp rx=0 tx=0).stop 2024-04-26T01:56:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.148+0000 7fa497e00640 1 -- 172.21.15.1:0/3301182885 shutdown_connections 2024-04-26T01:56:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.148+0000 7fa497e00640 1 --2- 172.21.15.1:0/3301182885 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fa47c0a3d80 0x7fa47c0a6240 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.148+0000 7fa497e00640 1 --2- 172.21.15.1:0/3301182885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa49810c350 0x7fa49819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.148+0000 7fa497e00640 1 --2- 172.21.15.1:0/3301182885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa498103850 0x7fa49819ac90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.148+0000 7fa497e00640 1 --2- 172.21.15.1:0/3301182885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa498102f30 0x7fa49819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.148+0000 7fa497e00640 1 -- 172.21.15.1:0/3301182885 >> 172.21.15.1:0/3301182885 conn(0x7fa4980fc730 msgr2=0x7fa4980fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.148+0000 7fa497e00640 1 -- 172.21.15.1:0/3301182885 shutdown_connections 2024-04-26T01:56:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:12.148+0000 7fa497e00640 1 -- 172.21.15.1:0/3301182885 wait complete. 2024-04-26T01:56:12.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:12 smithi173 ceph-mon[32553]: pgmap v140: 97 pgs: 97 active+clean; 37 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 466 op/s 2024-04-26T01:56:12.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:12 smithi001 ceph-mon[27822]: pgmap v140: 97 pgs: 97 active+clean; 37 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 466 op/s 2024-04-26T01:56:12.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:12 smithi097 ceph-mon[32853]: pgmap v140: 97 pgs: 97 active+clean; 37 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 466 op/s 2024-04-26T01:56:14.285 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:14 smithi097 ceph-mon[32853]: pgmap v141: 97 pgs: 97 active+clean; 37 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 277 op/s 2024-04-26T01:56:14.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:14 smithi173 ceph-mon[32553]: pgmap v141: 97 pgs: 97 active+clean; 37 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 277 op/s 2024-04-26T01:56:14.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:14 smithi001 ceph-mon[27822]: pgmap v141: 97 pgs: 97 active+clean; 37 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 277 op/s 2024-04-26T01:56:16.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:16 smithi173 ceph-mon[32553]: pgmap v142: 97 pgs: 97 active+clean; 35 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 410 op/s 2024-04-26T01:56:16.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:16 smithi001 ceph-mon[27822]: pgmap v142: 97 pgs: 97 active+clean; 35 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 410 op/s 2024-04-26T01:56:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:16 smithi097 ceph-mon[32853]: pgmap v142: 97 pgs: 97 active+clean; 35 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 410 op/s 2024-04-26T01:56:18.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:18 smithi173 ceph-mon[32553]: pgmap v143: 97 pgs: 97 active+clean; 35 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 262 op/s 2024-04-26T01:56:18.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:18 smithi001 ceph-mon[27822]: pgmap v143: 97 pgs: 97 active+clean; 35 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 262 op/s 2024-04-26T01:56:18.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:18 smithi097 ceph-mon[32853]: pgmap v143: 97 pgs: 97 active+clean; 35 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 262 op/s 2024-04-26T01:56:20.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:20 smithi173 ceph-mon[32553]: pgmap v144: 97 pgs: 97 active+clean; 35 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 262 op/s 2024-04-26T01:56:20.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:20 smithi001 ceph-mon[27822]: pgmap v144: 97 pgs: 97 active+clean; 35 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 262 op/s 2024-04-26T01:56:20.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:20 smithi097 ceph-mon[32853]: pgmap v144: 97 pgs: 97 active+clean; 35 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 262 op/s 2024-04-26T01:56:22.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:22 smithi173 ceph-mon[32553]: pgmap v145: 97 pgs: 97 active+clean; 36 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 387 op/s 2024-04-26T01:56:22.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:22 smithi001 ceph-mon[27822]: pgmap v145: 97 pgs: 97 active+clean; 36 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 387 op/s 2024-04-26T01:56:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:22 smithi097 ceph-mon[32853]: pgmap v145: 97 pgs: 97 active+clean; 36 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 387 op/s 2024-04-26T01:56:24.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:24 smithi173 ceph-mon[32553]: pgmap v146: 97 pgs: 97 active+clean; 36 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 257 op/s 2024-04-26T01:56:24.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:24 smithi001 ceph-mon[27822]: pgmap v146: 97 pgs: 97 active+clean; 36 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 257 op/s 2024-04-26T01:56:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:24 smithi097 ceph-mon[32853]: pgmap v146: 97 pgs: 97 active+clean; 36 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 257 op/s 2024-04-26T01:56:25.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.309+0000 7f7885a00640 1 -- 172.21.15.1:0/1567951376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7880109490 msgr2=0x7f7880111060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.309+0000 7f7885a00640 1 --2- 172.21.15.1:0/1567951376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7880109490 0x7f7880111060 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7878009be0 tx=0x7f787802a6c0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.310+0000 7f7885a00640 1 -- 172.21.15.1:0/1567951376 shutdown_connections 2024-04-26T01:56:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.310+0000 7f7885a00640 1 --2- 172.21.15.1:0/1567951376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7880109490 0x7f7880111060 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.310+0000 7f7885a00640 1 --2- 172.21.15.1:0/1567951376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7880104d70 0x7f7880108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.310+0000 7f7885a00640 1 --2- 172.21.15.1:0/1567951376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78801043c0 0x7f78801047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.310+0000 7f7885a00640 1 -- 172.21.15.1:0/1567951376 >> 172.21.15.1:0/1567951376 conn(0x7f78800ffc40 msgr2=0x7f7880102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.310+0000 7f7885a00640 1 -- 172.21.15.1:0/1567951376 shutdown_connections 2024-04-26T01:56:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.310+0000 7f7885a00640 1 -- 172.21.15.1:0/1567951376 wait complete. 2024-04-26T01:56:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.311+0000 7f7885a00640 1 Processor -- start 2024-04-26T01:56:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.311+0000 7f7885a00640 1 -- start start 2024-04-26T01:56:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.312+0000 7f7885a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78801043c0 0x7f78801a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.312+0000 7f7885a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7880109490 0x7f78801a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.312+0000 7f7885a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f788019a740 0x7f788019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.312+0000 7f7885a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f78801143e0 con 0x7f788019a740 2024-04-26T01:56:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.312+0000 7f7885a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7880114260 con 0x7f78801043c0 2024-04-26T01:56:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.312+0000 7f7885a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7880114560 con 0x7f7880109490 2024-04-26T01:56:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.312+0000 7f787ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78801043c0 0x7f78801a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.312+0000 7f787ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78801043c0 0x7f78801a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:38644/0 (socket says 172.21.15.1:38644) 2024-04-26T01:56:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.312+0000 7f787ea00640 1 -- 172.21.15.1:0/1055858343 learned_addr learned my addr 172.21.15.1:0/1055858343 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:56:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.312+0000 7f787e000640 1 --2- 172.21.15.1:0/1055858343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7880109490 0x7f78801a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.312+0000 7f787f400640 1 --2- 172.21.15.1:0/1055858343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f788019a740 0x7f788019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.312+0000 7f787ea00640 1 -- 172.21.15.1:0/1055858343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7880109490 msgr2=0x7f78801a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.312+0000 7f787ea00640 1 --2- 172.21.15.1:0/1055858343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7880109490 0x7f78801a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.312+0000 7f787ea00640 1 -- 172.21.15.1:0/1055858343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f788019a740 msgr2=0x7f788019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.312+0000 7f787ea00640 1 --2- 172.21.15.1:0/1055858343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f788019a740 0x7f788019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.312+0000 7f787ea00640 1 -- 172.21.15.1:0/1055858343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788019b340 con 0x7f78801043c0 2024-04-26T01:56:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.313+0000 7f787ea00640 1 --2- 172.21.15.1:0/1055858343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78801043c0 0x7f78801a0680 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f787000e9c0 tx=0x7f787000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.313+0000 7f7877400640 1 -- 172.21.15.1:0/1055858343 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787000cde0 con 0x7f78801043c0 2024-04-26T01:56:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.313+0000 7f7877400640 1 -- 172.21.15.1:0/1055858343 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78700035f0 con 0x7f78801043c0 2024-04-26T01:56:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.313+0000 7f7877400640 1 -- 172.21.15.1:0/1055858343 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870010690 con 0x7f78801043c0 2024-04-26T01:56:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.313+0000 7f7885a00640 1 -- 172.21.15.1:0/1055858343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78801a7420 con 0x7f78801043c0 2024-04-26T01:56:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.313+0000 7f7885a00640 1 -- 172.21.15.1:0/1055858343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78801a77d0 con 0x7f78801043c0 2024-04-26T01:56:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.317+0000 7f7877400640 1 -- 172.21.15.1:0/1055858343 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f7870004000 con 0x7f78801043c0 2024-04-26T01:56:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.317+0000 7f78887f0740 1 Processor -- start 2024-04-26T01:56:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.317+0000 7f78887f0740 1 -- start start 2024-04-26T01:56:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.317+0000 7f78887f0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c7b820 0x55d510c7bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.317+0000 7f78887f0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c7c140 0x55d510c864c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.317+0000 7f78887f0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d510c86a00 0x55d510c88df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.317+0000 7f78887f0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d510be1900 con 0x55d510c86a00 2024-04-26T01:56:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.317+0000 7f78887f0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d510b87ef0 con 0x55d510c7b820 2024-04-26T01:56:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.317+0000 7f78887f0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d510bdd8c0 con 0x55d510c7c140 2024-04-26T01:56:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.318+0000 7f787f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c7b820 0x55d510c7bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.318+0000 7f787ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c7c140 0x55d510c864c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.318+0000 7f787f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c7b820 0x55d510c7bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:38658/0 (socket says 172.21.15.1:38658) 2024-04-26T01:56:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.318+0000 7f787f400640 1 -- 172.21.15.1:0/2128359960 learned_addr learned my addr 172.21.15.1:0/2128359960 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:56:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.318+0000 7f787e000640 1 --2- 172.21.15.1:0/2128359960 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d510c86a00 0x55d510c88df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.318+0000 7f787f400640 1 -- 172.21.15.1:0/2128359960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c7c140 msgr2=0x55d510c864c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.318+0000 7f787f400640 1 --2- 172.21.15.1:0/2128359960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c7c140 0x55d510c864c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.318+0000 7f787f400640 1 -- 172.21.15.1:0/2128359960 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d510c86a00 msgr2=0x55d510c88df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.318+0000 7f787f400640 1 --2- 172.21.15.1:0/2128359960 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d510c86a00 0x55d510c88df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.318+0000 7f787f400640 1 -- 172.21.15.1:0/2128359960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5108f4210 con 0x55d510c7b820 2024-04-26T01:56:25.318 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.318+0000 7f787f400640 1 --2- 172.21.15.1:0/2128359960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c7b820 0x55d510c7bc00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7878009be0 tx=0x7f7878030430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3287ef4aaabf022a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.319+0000 7f7877400640 1 --2- 172.21.15.1:0/1055858343 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f78640a3d90 0x7f78640a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.319+0000 7f786be00640 1 -- 172.21.15.1:0/2128359960 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878041030 con 0x55d510c7b820 2024-04-26T01:56:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.319+0000 7f786be00640 1 -- 172.21.15.1:0/2128359960 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7878003e20 con 0x55d510c7b820 2024-04-26T01:56:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.319+0000 7f786be00640 1 -- 172.21.15.1:0/2128359960 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878036040 con 0x55d510c7b820 2024-04-26T01:56:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.319+0000 7f7877400640 1 -- 172.21.15.1:0/1055858343 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f78700d0b00 con 0x7f78801043c0 2024-04-26T01:56:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.319+0000 7f787ea00640 1 --2- 172.21.15.1:0/1055858343 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f78640a3d90 0x7f78640a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.320+0000 7f78887f0740 1 -- 172.21.15.1:0/2128359960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c7b820 msgr2=0x55d510c7bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.320+0000 7f78887f0740 1 --2- 172.21.15.1:0/2128359960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c7b820 0x55d510c7bc00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7878009be0 tx=0x7f7878030430 comp rx=0 tx=0).stop 2024-04-26T01:56:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.320+0000 7f787ea00640 1 --2- 172.21.15.1:0/1055858343 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f78640a3d90 0x7f78640a6250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55d510c7cdc0 tx=0x7f78700d45b0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.320+0000 7f78887f0740 1 -- 172.21.15.1:0/2128359960 shutdown_connections 2024-04-26T01:56:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.320+0000 7f78887f0740 1 --2- 172.21.15.1:0/2128359960 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d510c86a00 0x55d510c88df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.320+0000 7f78887f0740 1 --2- 172.21.15.1:0/2128359960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c7c140 0x55d510c864c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.320+0000 7f78887f0740 1 --2- 172.21.15.1:0/2128359960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c7b820 0x55d510c7bc00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.320+0000 7f78887f0740 1 -- 172.21.15.1:0/2128359960 >> 172.21.15.1:0/2128359960 conn(0x55d510c77d30 msgr2=0x55d510c78110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.320+0000 7f78887f0740 1 -- 172.21.15.1:0/2128359960 shutdown_connections 2024-04-26T01:56:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.320+0000 7f78887f0740 1 -- 172.21.15.1:0/2128359960 wait complete. 2024-04-26T01:56:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.320+0000 7f78887f0740 1 -- start start 2024-04-26T01:56:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.321+0000 7f78887f0740 1 Processor -- start 2024-04-26T01:56:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.321+0000 7f78887f0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c7b820 0x55d510c839c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.321+0000 7f78887f0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d510c7c140 0x55d510c83f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.321+0000 7f78887f0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c86a00 0x55d510c84440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.321+0000 7f78887f0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d510c895d0 con 0x55d510c7c140 2024-04-26T01:56:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.321+0000 7f78887f0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d510b9b310 con 0x55d510c86a00 2024-04-26T01:56:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.321+0000 7f78887f0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d510c89750 con 0x55d510c7b820 2024-04-26T01:56:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.321+0000 7f78887f0740 10 client.0 _ll_register_callbacks cb 0x55d510b69cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.321+0000 7f787f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c7b820 0x55d510c839c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.321+0000 7f787f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c7b820 0x55d510c839c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:35832/0 (socket says 172.21.15.1:35832) 2024-04-26T01:56:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.321+0000 7f787f400640 1 -- 172.21.15.1:0/2678847083 learned_addr learned my addr 172.21.15.1:0/2678847083 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:56:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.321+0000 7f787f400640 1 --2- 172.21.15.1:0/2678847083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c86a00 0x55d510c84440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.322+0000 7f787e000640 1 --2- 172.21.15.1:0/2678847083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d510c7c140 0x55d510c83f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.322+0000 7f787f400640 1 -- 172.21.15.1:0/2678847083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c86a00 msgr2=0x55d510c84440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.322+0000 7f787f400640 1 --2- 172.21.15.1:0/2678847083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c86a00 0x55d510c84440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.322+0000 7f787f400640 1 -- 172.21.15.1:0/2678847083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d510c7c140 msgr2=0x55d510c83f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.322+0000 7f787f400640 1 --2- 172.21.15.1:0/2678847083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d510c7c140 0x55d510c83f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.322+0000 7f787f400640 1 -- 172.21.15.1:0/2678847083 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5109c0620 con 0x55d510c7b820 2024-04-26T01:56:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.322+0000 7f787f400640 1 --2- 172.21.15.1:0/2678847083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c7b820 0x55d510c839c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f78780099a0 tx=0x7f7878030d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce28de03016f19d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.322+0000 7f7868c00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:56:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.322+0000 7f7868c00640 1 -- 172.21.15.1:0/2678847083 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878036050 con 0x55d510c7b820 2024-04-26T01:56:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.322+0000 7f7868c00640 1 -- 172.21.15.1:0/2678847083 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f7878030a00 con 0x55d510c7b820 2024-04-26T01:56:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.322+0000 7f78887f0740 1 -- 172.21.15.1:0/2678847083 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7878003e20 con 0x55d510c7b820 2024-04-26T01:56:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.323+0000 7f7868c00640 1 -- 172.21.15.1:0/2678847083 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7878059050 con 0x55d510c7b820 2024-04-26T01:56:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.323+0000 7f78887f0740 10 client.25777 fetch_fsmap learned FSMap version 32 2024-04-26T01:56:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.323+0000 7f78887f0740 1 -- 172.21.15.1:0/2678847083 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d510c895d0 con 0x55d510c7b820 2024-04-26T01:56:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.323+0000 7f7868c00640 1 -- 172.21.15.1:0/2678847083 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7878003e20 con 0x55d510c7b820 2024-04-26T01:56:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.323+0000 7f7868c00640 1 -- 172.21.15.1:0/2678847083 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f787803d460 con 0x55d510c7b820 2024-04-26T01:56:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.323+0000 7f78887f0740 10 client.25777 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:56:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.323+0000 7f78887f0740 10 client.25777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:56:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.323+0000 7f78887f0740 20 client.25777 populate_metadata read hostname 'smithi001' 2024-04-26T01:56:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.324+0000 7f78887f0740 1 --2- 172.21.15.1:0/2678847083 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d510c96a70 0x55d510c98e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.324+0000 7f78887f0740 4 client.25777 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:56:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.324+0000 7f78887f0740 1 -- 172.21.15.1:0/2678847083 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d510bf58b0 con 0x55d510c96a70 2024-04-26T01:56:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.324+0000 7f787e000640 1 --2- 172.21.15.1:0/2678847083 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d510c96a70 0x55d510c98e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.324+0000 7f787e000640 1 --2- 172.21.15.1:0/2678847083 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d510c96a70 0x55d510c98e60 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29215a9e7503face server_cookie=da899eb29800e966 in_seq=0 out_seq=0 2024-04-26T01:56:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.324+0000 7f7868c00640 10 client.25777 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:56:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.325+0000 7f7868c00640 1 -- 172.21.15.1:0/2678847083 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d510bf58b0 con 0x55d510c96a70 2024-04-26T01:56:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.325+0000 7f7868c00640 10 client.25777 handle_command_reply: tid=0 2024-04-26T01:56:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.325+0000 7f78887f0740 1 client.25777 shutdown 2024-04-26T01:56:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.325+0000 7f78887f0740 1 -- 172.21.15.1:0/2678847083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c7b820 msgr2=0x55d510c839c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.325+0000 7f78887f0740 1 --2- 172.21.15.1:0/2678847083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c7b820 0x55d510c839c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f78780099a0 tx=0x7f7878030d00 comp rx=0 tx=0).stop 2024-04-26T01:56:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.326+0000 7f78887f0740 1 -- 172.21.15.1:0/2678847083 shutdown_connections 2024-04-26T01:56:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.326+0000 7f78887f0740 1 --2- 172.21.15.1:0/2678847083 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d510c96a70 0x55d510c98e60 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.326+0000 7f78887f0740 1 --2- 172.21.15.1:0/2678847083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c86a00 0x55d510c84440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.326+0000 7f78887f0740 1 --2- 172.21.15.1:0/2678847083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d510c7c140 0x55d510c83f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.326+0000 7f78887f0740 1 --2- 172.21.15.1:0/2678847083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c7b820 0x55d510c839c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.326+0000 7f78887f0740 1 -- 172.21.15.1:0/2678847083 >> 172.21.15.1:0/2678847083 conn(0x55d510c77d30 msgr2=0x55d510c7da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.326+0000 7f7868c00640 0 client.25777 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:56:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.326+0000 7f78887f0740 1 -- 172.21.15.1:0/2678847083 shutdown_connections 2024-04-26T01:56:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.326+0000 7f78887f0740 1 -- 172.21.15.1:0/2678847083 wait complete. 2024-04-26T01:56:25.330 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.326+0000 7f78887f0740 20 client.25777 trim_cache size 0 max 16384 2024-04-26T01:56:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.343+0000 7f78887f0740 1 Processor -- start 2024-04-26T01:56:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.344+0000 7f78887f0740 1 -- start start 2024-04-26T01:56:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.344+0000 7f78887f0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c7b820 0x55d510c92490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.344+0000 7f78887f0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c7c140 0x55d510c82b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.344+0000 7f78887f0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d510c86a00 0x55d510c84260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.344+0000 7f78887f0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f78801ae300 con 0x55d510c86a00 2024-04-26T01:56:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.344+0000 7f78887f0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f78801abbd0 con 0x55d510c7b820 2024-04-26T01:56:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.344+0000 7f78887f0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f78801a9a20 con 0x55d510c7c140 2024-04-26T01:56:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.344+0000 7f787e000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c7c140 0x55d510c82b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.344+0000 7f787e000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c7c140 0x55d510c82b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:35846/0 (socket says 172.21.15.1:35846) 2024-04-26T01:56:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.344+0000 7f787e000640 1 -- 172.21.15.1:0/368840558 learned_addr learned my addr 172.21.15.1:0/368840558 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:56:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.345+0000 7f787f400640 1 --2- 172.21.15.1:0/368840558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c7b820 0x55d510c92490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.345+0000 7f787f400640 1 --2- 172.21.15.1:0/368840558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d510c86a00 0x55d510c84260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.345+0000 7f787e000640 1 -- 172.21.15.1:0/368840558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c7b820 msgr2=0x55d510c92490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.345+0000 7f787e000640 1 --2- 172.21.15.1:0/368840558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c7b820 0x55d510c92490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.345+0000 7f787e000640 1 -- 172.21.15.1:0/368840558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d510c86a00 msgr2=0x55d510c84260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.345+0000 7f787e000640 1 --2- 172.21.15.1:0/368840558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d510c86a00 0x55d510c84260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.345+0000 7f787e000640 1 -- 172.21.15.1:0/368840558 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d510c73690 con 0x55d510c7c140 2024-04-26T01:56:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.345+0000 7f787f400640 1 --2- 172.21.15.1:0/368840558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d510c86a00 0x55d510c84260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:56:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.345+0000 7f787e000640 1 --2- 172.21.15.1:0/368840558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c7c140 0x55d510c82b20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f786c000c00 tx=0x7f786c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23f91f4b610714e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.346+0000 7f786b400640 1 -- 172.21.15.1:0/368840558 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c024030 con 0x55d510c7c140 2024-04-26T01:56:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.346+0000 7f786b400640 1 -- 172.21.15.1:0/368840558 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f786c0054d0 con 0x55d510c7c140 2024-04-26T01:56:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.346+0000 7f786b400640 1 -- 172.21.15.1:0/368840558 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c0200f0 con 0x55d510c7c140 2024-04-26T01:56:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.346+0000 7f78887f0740 1 -- 172.21.15.1:0/368840558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c7c140 msgr2=0x55d510c82b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.346+0000 7f78887f0740 1 --2- 172.21.15.1:0/368840558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c7c140 0x55d510c82b20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f786c000c00 tx=0x7f786c005f90 comp rx=0 tx=0).stop 2024-04-26T01:56:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.347+0000 7f78887f0740 1 -- 172.21.15.1:0/368840558 shutdown_connections 2024-04-26T01:56:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.347+0000 7f78887f0740 1 --2- 172.21.15.1:0/368840558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d510c86a00 0x55d510c84260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.347+0000 7f78887f0740 1 --2- 172.21.15.1:0/368840558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c7c140 0x55d510c82b20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.347+0000 7f78887f0740 1 --2- 172.21.15.1:0/368840558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c7b820 0x55d510c92490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.347+0000 7f78887f0740 1 -- 172.21.15.1:0/368840558 >> 172.21.15.1:0/368840558 conn(0x55d510c77d30 msgr2=0x55d510c7d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.347+0000 7f78887f0740 1 -- 172.21.15.1:0/368840558 shutdown_connections 2024-04-26T01:56:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.347+0000 7f78887f0740 1 -- 172.21.15.1:0/368840558 wait complete. 2024-04-26T01:56:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.347+0000 7f78887f0740 1 -- start start 2024-04-26T01:56:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.347+0000 7f78887f0740 1 Processor -- start 2024-04-26T01:56:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.348+0000 7f78887f0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c7b820 0x55d510c8bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.348+0000 7f78887f0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d510c7c140 0x55d510c8c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.348+0000 7f78887f0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c86a00 0x55d510cfae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.348+0000 7f78887f0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f78801aa090 con 0x55d510c7c140 2024-04-26T01:56:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.348+0000 7f78887f0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f78801ab500 con 0x55d510c7b820 2024-04-26T01:56:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.348+0000 7f78887f0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f78801aa760 con 0x55d510c86a00 2024-04-26T01:56:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.348+0000 7f78887f0740 10 client.0 _ll_register_callbacks cb 0x7f78780041b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.348+0000 7f787f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c7b820 0x55d510c8bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.348+0000 7f787f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c7b820 0x55d510c8bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:38666/0 (socket says 172.21.15.1:38666) 2024-04-26T01:56:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.348+0000 7f787f400640 1 -- 172.21.15.1:0/3591627003 learned_addr learned my addr 172.21.15.1:0/3591627003 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:56:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.348+0000 7f787e000640 1 --2- 172.21.15.1:0/3591627003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d510c7c140 0x55d510c8c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.348+0000 7f787f400640 1 --2- 172.21.15.1:0/3591627003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c86a00 0x55d510cfae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.349+0000 7f787f400640 1 -- 172.21.15.1:0/3591627003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c7b820 msgr2=0x55d510c8bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.349+0000 7f787f400640 1 --2- 172.21.15.1:0/3591627003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c7b820 0x55d510c8bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.349+0000 7f787f400640 1 -- 172.21.15.1:0/3591627003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d510c7c140 msgr2=0x55d510c8c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.349+0000 7f787f400640 1 --2- 172.21.15.1:0/3591627003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d510c7c140 0x55d510c8c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.349+0000 7f787f400640 1 -- 172.21.15.1:0/3591627003 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d510cfb3f0 con 0x55d510c86a00 2024-04-26T01:56:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.349+0000 7f787f400640 1 --2- 172.21.15.1:0/3591627003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c86a00 0x55d510cfae60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7878006010 tx=0x7f7878030d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97d89e863dbf0dae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.349+0000 7f7868c00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:56:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.349+0000 7f7868c00640 1 -- 172.21.15.1:0/3591627003 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878036070 con 0x55d510c86a00 2024-04-26T01:56:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.349+0000 7f78887f0740 1 -- 172.21.15.1:0/3591627003 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f786c0054d0 con 0x55d510c86a00 2024-04-26T01:56:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.349+0000 7f7868c00640 1 -- 172.21.15.1:0/3591627003 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f787803d460 con 0x55d510c86a00 2024-04-26T01:56:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.350+0000 7f7868c00640 1 -- 172.21.15.1:0/3591627003 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f787806c050 con 0x55d510c86a00 2024-04-26T01:56:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.350+0000 7f78887f0740 10 client.25783 fetch_fsmap learned FSMap version 32 2024-04-26T01:56:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.350+0000 7f78887f0740 1 -- 172.21.15.1:0/3591627003 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d510cfc670 con 0x55d510c86a00 2024-04-26T01:56:25.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.350+0000 7f7868c00640 1 -- 172.21.15.1:0/3591627003 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f786c0054d0 con 0x55d510c86a00 2024-04-26T01:56:25.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.350+0000 7f7868c00640 1 -- 172.21.15.1:0/3591627003 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f787803daa0 con 0x55d510c86a00 2024-04-26T01:56:25.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.350+0000 7f78887f0740 10 client.25783 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:56:25.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.350+0000 7f78887f0740 10 client.25783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:56:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.350+0000 7f78887f0740 20 client.25783 populate_metadata read hostname 'smithi001' 2024-04-26T01:56:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.350+0000 7f78887f0740 1 --2- 172.21.15.1:0/3591627003 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d510cfcd30 0x55d510cff150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.350+0000 7f78887f0740 4 client.25783 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:56:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.350+0000 7f78887f0740 1 -- 172.21.15.1:0/3591627003 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d510bef320 con 0x55d510cfcd30 2024-04-26T01:56:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.350+0000 7f787e000640 1 --2- 172.21.15.1:0/3591627003 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d510cfcd30 0x55d510cff150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.351+0000 7f787e000640 1 --2- 172.21.15.1:0/3591627003 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d510cfcd30 0x55d510cff150 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=688c92a727af8cc2 server_cookie=8ffe1fbba9b47df1 in_seq=0 out_seq=0 2024-04-26T01:56:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.351+0000 7f7868c00640 10 client.25783 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:56:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.372+0000 7f7868c00640 1 -- 172.21.15.1:0/3591627003 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d510bef320 con 0x55d510cfcd30 2024-04-26T01:56:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.372+0000 7f7868c00640 10 client.25783 handle_command_reply: tid=0 2024-04-26T01:56:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.372+0000 7f78887f0740 1 client.25783 shutdown 2024-04-26T01:56:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.372+0000 7f78887f0740 1 -- 172.21.15.1:0/3591627003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c86a00 msgr2=0x55d510cfae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.372+0000 7f78887f0740 1 --2- 172.21.15.1:0/3591627003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c86a00 0x55d510cfae60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7878006010 tx=0x7f7878030d00 comp rx=0 tx=0).stop 2024-04-26T01:56:25.371 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.373+0000 7f78887f0740 1 -- 172.21.15.1:0/3591627003 shutdown_connections 2024-04-26T01:56:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.373+0000 7f78887f0740 1 --2- 172.21.15.1:0/3591627003 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d510cfcd30 0x55d510cff150 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.373+0000 7f78887f0740 1 --2- 172.21.15.1:0/3591627003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d510c86a00 0x55d510cfae60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.373+0000 7f78887f0740 1 --2- 172.21.15.1:0/3591627003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d510c7c140 0x55d510c8c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.373+0000 7f7868c00640 0 client.25783 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:56:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.373+0000 7f78887f0740 1 --2- 172.21.15.1:0/3591627003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d510c7b820 0x55d510c8bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.373+0000 7f78887f0740 1 -- 172.21.15.1:0/3591627003 >> 172.21.15.1:0/3591627003 conn(0x55d510c77d30 msgr2=0x55d510c7da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.373+0000 7f78887f0740 1 -- 172.21.15.1:0/3591627003 shutdown_connections 2024-04-26T01:56:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.373+0000 7f78887f0740 1 -- 172.21.15.1:0/3591627003 wait complete. 2024-04-26T01:56:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.373+0000 7f78887f0740 20 client.25783 trim_cache size 0 max 16384 2024-04-26T01:56:25.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:56:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:56:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:56:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:56:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.374+0000 7f7885a00640 1 -- 172.21.15.1:0/1055858343 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f78640a3d90 msgr2=0x7f78640a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.374+0000 7f7885a00640 1 --2- 172.21.15.1:0/1055858343 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f78640a3d90 0x7f78640a6250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55d510c7cdc0 tx=0x7f78700d45b0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.374+0000 7f7885a00640 1 -- 172.21.15.1:0/1055858343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78801043c0 msgr2=0x7f78801a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.374+0000 7f7885a00640 1 --2- 172.21.15.1:0/1055858343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78801043c0 0x7f78801a0680 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f787000e9c0 tx=0x7f787000ee80 comp rx=0 tx=0).stop 2024-04-26T01:56:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.374+0000 7f7885a00640 1 -- 172.21.15.1:0/1055858343 shutdown_connections 2024-04-26T01:56:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.374+0000 7f7885a00640 1 --2- 172.21.15.1:0/1055858343 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f78640a3d90 0x7f78640a6250 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.374+0000 7f7885a00640 1 --2- 172.21.15.1:0/1055858343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f788019a740 0x7f788019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.374+0000 7f7885a00640 1 --2- 172.21.15.1:0/1055858343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7880109490 0x7f78801a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.374+0000 7f7885a00640 1 --2- 172.21.15.1:0/1055858343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78801043c0 0x7f78801a0680 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.374+0000 7f7885a00640 1 -- 172.21.15.1:0/1055858343 >> 172.21.15.1:0/1055858343 conn(0x7f78800ffc40 msgr2=0x7f7880101f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.374+0000 7f7885a00640 1 -- 172.21.15.1:0/1055858343 shutdown_connections 2024-04-26T01:56:25.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:25.374+0000 7f7885a00640 1 -- 172.21.15.1:0/1055858343 wait complete. 2024-04-26T01:56:26.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:26 smithi173 ceph-mon[32553]: pgmap v147: 97 pgs: 97 active+clean; 40 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 389 op/s 2024-04-26T01:56:26.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:26 smithi001 ceph-mon[27822]: pgmap v147: 97 pgs: 97 active+clean; 40 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 389 op/s 2024-04-26T01:56:26.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:26 smithi097 ceph-mon[32853]: pgmap v147: 97 pgs: 97 active+clean; 40 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 389 op/s 2024-04-26T01:56:27.643 INFO:tasks.workunit.client.0.smithi001.stdout:30000 trunc from 0x7bb2f to 0xcdf83 2024-04-26T01:56:28.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:28 smithi173 ceph-mon[32553]: pgmap v148: 97 pgs: 97 active+clean; 40 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 257 op/s 2024-04-26T01:56:28.340 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xffcdd 2024-04-26T01:56:28.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:28 smithi001 ceph-mon[27822]: pgmap v148: 97 pgs: 97 active+clean; 40 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 257 op/s 2024-04-26T01:56:28.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:28 smithi097 ceph-mon[32853]: pgmap v148: 97 pgs: 97 active+clean; 40 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 257 op/s 2024-04-26T01:56:30.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:30 smithi001 ceph-mon[27822]: pgmap v149: 97 pgs: 97 active+clean; 40 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 257 op/s 2024-04-26T01:56:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:30 smithi097 ceph-mon[32853]: pgmap v149: 97 pgs: 97 active+clean; 40 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 257 op/s 2024-04-26T01:56:30.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:30 smithi173 ceph-mon[32553]: pgmap v149: 97 pgs: 97 active+clean; 40 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 257 op/s 2024-04-26T01:56:32.285 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xffd97 2024-04-26T01:56:32.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:32 smithi001 ceph-mon[27822]: pgmap v150: 97 pgs: 97 active+clean; 38 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 384 op/s 2024-04-26T01:56:32.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:32 smithi097 ceph-mon[32853]: pgmap v150: 97 pgs: 97 active+clean; 38 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 384 op/s 2024-04-26T01:56:32.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:32 smithi173 ceph-mon[32553]: pgmap v150: 97 pgs: 97 active+clean; 38 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 384 op/s 2024-04-26T01:56:34.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:34 smithi001 ceph-mon[27822]: pgmap v151: 97 pgs: 97 active+clean; 38 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 259 op/s 2024-04-26T01:56:34.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:34 smithi097 ceph-mon[32853]: pgmap v151: 97 pgs: 97 active+clean; 38 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 259 op/s 2024-04-26T01:56:34.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:34 smithi173 ceph-mon[32553]: pgmap v151: 97 pgs: 97 active+clean; 38 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 259 op/s 2024-04-26T01:56:36.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:36 smithi001 ceph-mon[27822]: pgmap v152: 97 pgs: 97 active+clean; 40 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 363 op/s 2024-04-26T01:56:36.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:36 smithi097 ceph-mon[32853]: pgmap v152: 97 pgs: 97 active+clean; 40 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 363 op/s 2024-04-26T01:56:36.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:36 smithi173 ceph-mon[32553]: pgmap v152: 97 pgs: 97 active+clean; 40 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 363 op/s 2024-04-26T01:56:38.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:38 smithi001 ceph-mon[27822]: pgmap v153: 97 pgs: 97 active+clean; 40 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2024-04-26T01:56:38.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:38 smithi097 ceph-mon[32853]: pgmap v153: 97 pgs: 97 active+clean; 40 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2024-04-26T01:56:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.542+0000 7f0a57e00640 1 -- 172.21.15.1:0/423099958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a5810e470 msgr2=0x7f0a5810e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.542+0000 7f0a57e00640 1 --2- 172.21.15.1:0/423099958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a5810e470 0x7f0a5810e850 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009900 tx=0x7f0a4802a490 comp rx=0 tx=0).stop 2024-04-26T01:56:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.542+0000 7f0a57e00640 1 -- 172.21.15.1:0/423099958 shutdown_connections 2024-04-26T01:56:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.542+0000 7f0a57e00640 1 --2- 172.21.15.1:0/423099958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a58108d50 0x7f0a58115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.542+0000 7f0a57e00640 1 --2- 172.21.15.1:0/423099958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a581083b0 0x7f0a58108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.542+0000 7f0a57e00640 1 --2- 172.21.15.1:0/423099958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a5810e470 0x7f0a5810e850 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.542+0000 7f0a57e00640 1 -- 172.21.15.1:0/423099958 >> 172.21.15.1:0/423099958 conn(0x7f0a58077b00 msgr2=0x7f0a58079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.542+0000 7f0a57e00640 1 -- 172.21.15.1:0/423099958 shutdown_connections 2024-04-26T01:56:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.542+0000 7f0a57e00640 1 -- 172.21.15.1:0/423099958 wait complete. 2024-04-26T01:56:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.544+0000 7f0a57e00640 1 Processor -- start 2024-04-26T01:56:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.544+0000 7f0a57e00640 1 -- start start 2024-04-26T01:56:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.544+0000 7f0a57e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a581083b0 0x7f0a581a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.544+0000 7f0a57e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a58108d50 0x7f0a581a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.544+0000 7f0a57e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a5810e470 0x7f0a5819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.544+0000 7f0a57e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0a58118650 con 0x7f0a5810e470 2024-04-26T01:56:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.544+0000 7f0a57e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581184d0 con 0x7f0a58108d50 2024-04-26T01:56:38.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.544+0000 7f0a57e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581187d0 con 0x7f0a581083b0 2024-04-26T01:56:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.544+0000 7f0a56a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a581083b0 0x7f0a581a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.544+0000 7f0a56a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a581083b0 0x7f0a581a4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:60104/0 (socket says 172.21.15.1:60104) 2024-04-26T01:56:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.545+0000 7f0a56a00640 1 -- 172.21.15.1:0/1818684027 learned_addr learned my addr 172.21.15.1:0/1818684027 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:56:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.545+0000 7f0a4e000640 1 --2- 172.21.15.1:0/1818684027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a58108d50 0x7f0a581a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.545+0000 7f0a57400640 1 --2- 172.21.15.1:0/1818684027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a5810e470 0x7f0a5819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.545+0000 7f0a56a00640 1 -- 172.21.15.1:0/1818684027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a58108d50 msgr2=0x7f0a581a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.545+0000 7f0a56a00640 1 --2- 172.21.15.1:0/1818684027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a58108d50 0x7f0a581a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.545+0000 7f0a56a00640 1 -- 172.21.15.1:0/1818684027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a5810e470 msgr2=0x7f0a5819e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.545+0000 7f0a56a00640 1 --2- 172.21.15.1:0/1818684027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a5810e470 0x7f0a5819e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.545+0000 7f0a56a00640 1 -- 172.21.15.1:0/1818684027 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5819f0f0 con 0x7f0a581083b0 2024-04-26T01:56:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.545+0000 7f0a56a00640 1 --2- 172.21.15.1:0/1818684027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a581083b0 0x7f0a581a4770 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0a48005ab0 tx=0x7f0a48003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.546+0000 7f0a5ca00640 1 -- 172.21.15.1:0/1818684027 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4803f070 con 0x7f0a581083b0 2024-04-26T01:56:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.546+0000 7f0a5ca00640 1 -- 172.21.15.1:0/1818684027 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a48031d60 con 0x7f0a581083b0 2024-04-26T01:56:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.546+0000 7f0a5ca00640 1 -- 172.21.15.1:0/1818684027 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48035030 con 0x7f0a581083b0 2024-04-26T01:56:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.546+0000 7f0a57e00640 1 -- 172.21.15.1:0/1818684027 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a5819f380 con 0x7f0a581083b0 2024-04-26T01:56:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.546+0000 7f0a57e00640 1 -- 172.21.15.1:0/1818684027 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a581ab630 con 0x7f0a581083b0 2024-04-26T01:56:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.549+0000 7f0a5ca00640 1 -- 172.21.15.1:0/1818684027 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f0a48042070 con 0x7f0a581083b0 2024-04-26T01:56:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.550+0000 7f0a60c2b740 1 Processor -- start 2024-04-26T01:56:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.550+0000 7f0a60c2b740 1 -- start start 2024-04-26T01:56:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.550+0000 7f0a60c2b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf336e49b0 0x55bf336e4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.550+0000 7f0a60c2b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336e52d0 0x55bf336ef650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.550+0000 7f0a60c2b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336efb90 0x55bf336f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.550+0000 7f0a60c2b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bf3364a900 con 0x55bf336e49b0 2024-04-26T01:56:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.550+0000 7f0a60c2b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bf335f0ef0 con 0x55bf336efb90 2024-04-26T01:56:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.550+0000 7f0a60c2b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bf336468c0 con 0x55bf336e52d0 2024-04-26T01:56:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.551+0000 7f0a4e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336efb90 0x55bf336f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.551+0000 7f0a56a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336e52d0 0x55bf336ef650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.551+0000 7f0a4e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336efb90 0x55bf336f1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:58088/0 (socket says 172.21.15.1:58088) 2024-04-26T01:56:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.551+0000 7f0a4e000640 1 -- 172.21.15.1:0/272586983 learned_addr learned my addr 172.21.15.1:0/272586983 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:56:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.551+0000 7f0a4e000640 1 -- 172.21.15.1:0/272586983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336e52d0 msgr2=0x55bf336ef650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.551+0000 7f0a4e000640 1 --2- 172.21.15.1:0/272586983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336e52d0 0x55bf336ef650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.551+0000 7f0a4e000640 1 -- 172.21.15.1:0/272586983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf336e49b0 msgr2=0x55bf336e4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.551+0000 7f0a57400640 1 --2- 172.21.15.1:0/272586983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf336e49b0 0x55bf336e4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.551+0000 7f0a4e000640 1 --2- 172.21.15.1:0/272586983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf336e49b0 0x55bf336e4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.551+0000 7f0a4e000640 1 -- 172.21.15.1:0/272586983 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf3335d210 con 0x55bf336efb90 2024-04-26T01:56:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.551+0000 7f0a57400640 1 --2- 172.21.15.1:0/272586983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf336e49b0 0x55bf336e4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:56:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.551+0000 7f0a5ca00640 1 --2- 172.21.15.1:0/1818684027 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0a3c0a3d80 0x7f0a3c0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.552+0000 7f0a4e000640 1 --2- 172.21.15.1:0/272586983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336efb90 0x55bf336f1f80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000bcd0 tx=0x7f0a400107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f47bf2c01910386a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.552+0000 7f0a5ca00640 1 -- 172.21.15.1:0/1818684027 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f0a480f0fe0 con 0x7f0a581083b0 2024-04-26T01:56:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.552+0000 7f0a57400640 1 --2- 172.21.15.1:0/1818684027 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0a3c0a3d80 0x7f0a3c0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.552+0000 7f0a47e00640 1 -- 172.21.15.1:0/272586983 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4001c680 con 0x55bf336efb90 2024-04-26T01:56:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.552+0000 7f0a47e00640 1 -- 172.21.15.1:0/272586983 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a4001ccd0 con 0x55bf336efb90 2024-04-26T01:56:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.552+0000 7f0a47e00640 1 -- 172.21.15.1:0/272586983 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40011c70 con 0x55bf336efb90 2024-04-26T01:56:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.553+0000 7f0a60c2b740 1 -- 172.21.15.1:0/272586983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336efb90 msgr2=0x55bf336f1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.553+0000 7f0a60c2b740 1 --2- 172.21.15.1:0/272586983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336efb90 0x55bf336f1f80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000bcd0 tx=0x7f0a400107a0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.553+0000 7f0a57400640 1 --2- 172.21.15.1:0/1818684027 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0a3c0a3d80 0x7f0a3c0a6240 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0a50005fd0 tx=0x7f0a50009600 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.553+0000 7f0a60c2b740 1 -- 172.21.15.1:0/272586983 shutdown_connections 2024-04-26T01:56:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.553+0000 7f0a60c2b740 1 --2- 172.21.15.1:0/272586983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336efb90 0x55bf336f1f80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.553+0000 7f0a60c2b740 1 --2- 172.21.15.1:0/272586983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336e52d0 0x55bf336ef650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.553+0000 7f0a60c2b740 1 --2- 172.21.15.1:0/272586983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf336e49b0 0x55bf336e4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.553+0000 7f0a60c2b740 1 -- 172.21.15.1:0/272586983 >> 172.21.15.1:0/272586983 conn(0x55bf336e0ec0 msgr2=0x55bf336e12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.553+0000 7f0a60c2b740 1 -- 172.21.15.1:0/272586983 shutdown_connections 2024-04-26T01:56:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.553+0000 7f0a60c2b740 1 -- 172.21.15.1:0/272586983 wait complete. 2024-04-26T01:56:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.554+0000 7f0a60c2b740 1 -- start start 2024-04-26T01:56:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.554+0000 7f0a60c2b740 1 Processor -- start 2024-04-26T01:56:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.555+0000 7f0a60c2b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336e49b0 0x55bf336eca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.555+0000 7f0a60c2b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336e52d0 0x55bf336ecfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.555+0000 7f0a60c2b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf336efb90 0x55bf336ed4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.555+0000 7f0a60c2b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bf336f27c0 con 0x55bf336efb90 2024-04-26T01:56:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.555+0000 7f0a60c2b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55bf33604310 con 0x55bf336e52d0 2024-04-26T01:56:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.555+0000 7f0a60c2b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55bf336f2940 con 0x55bf336e49b0 2024-04-26T01:56:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.555+0000 7f0a60c2b740 10 client.0 _ll_register_callbacks cb 0x55bf335d2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.555+0000 7f0a4e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336e52d0 0x55bf336ecfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.555+0000 7f0a4e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336e52d0 0x55bf336ecfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:58094/0 (socket says 172.21.15.1:58094) 2024-04-26T01:56:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.555+0000 7f0a4e000640 1 -- 172.21.15.1:0/725549766 learned_addr learned my addr 172.21.15.1:0/725549766 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:56:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.555+0000 7f0a56a00640 1 --2- 172.21.15.1:0/725549766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336e49b0 0x55bf336eca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.555+0000 7f0a57400640 1 --2- 172.21.15.1:0/725549766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf336efb90 0x55bf336ed4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.555+0000 7f0a4e000640 1 -- 172.21.15.1:0/725549766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336e49b0 msgr2=0x55bf336eca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.555+0000 7f0a4e000640 1 --2- 172.21.15.1:0/725549766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336e49b0 0x55bf336eca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.555+0000 7f0a4e000640 1 -- 172.21.15.1:0/725549766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf336efb90 msgr2=0x55bf336ed4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.555+0000 7f0a4e000640 1 --2- 172.21.15.1:0/725549766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf336efb90 0x55bf336ed4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.555+0000 7f0a4e000640 1 -- 172.21.15.1:0/725549766 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf33429620 con 0x55bf336e52d0 2024-04-26T01:56:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.556+0000 7f0a4e000640 1 --2- 172.21.15.1:0/725549766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336e52d0 0x55bf336ecfb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0a40011770 tx=0x7f0a40011670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdbb0e11b0486913 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.556+0000 7f0a44c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:56:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.556+0000 7f0a44c00640 1 -- 172.21.15.1:0/725549766 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40011d80 con 0x55bf336e52d0 2024-04-26T01:56:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.556+0000 7f0a44c00640 1 -- 172.21.15.1:0/725549766 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f0a4001c480 con 0x55bf336e52d0 2024-04-26T01:56:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.556+0000 7f0a60c2b740 1 -- 172.21.15.1:0/725549766 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a4001ccd0 con 0x55bf336e52d0 2024-04-26T01:56:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.557+0000 7f0a44c00640 1 -- 172.21.15.1:0/725549766 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0a4000b8c0 con 0x55bf336e52d0 2024-04-26T01:56:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.557+0000 7f0a60c2b740 10 client.25772 fetch_fsmap learned FSMap version 32 2024-04-26T01:56:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.557+0000 7f0a60c2b740 1 -- 172.21.15.1:0/725549766 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bf336fe910 con 0x55bf336e52d0 2024-04-26T01:56:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.557+0000 7f0a44c00640 1 -- 172.21.15.1:0/725549766 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a4001ccd0 con 0x55bf336e52d0 2024-04-26T01:56:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.557+0000 7f0a44c00640 1 -- 172.21.15.1:0/725549766 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f0a4002d030 con 0x55bf336e52d0 2024-04-26T01:56:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.557+0000 7f0a60c2b740 10 client.25772 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:56:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.557+0000 7f0a60c2b740 10 client.25772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:56:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.557+0000 7f0a60c2b740 20 client.25772 populate_metadata read hostname 'smithi001' 2024-04-26T01:56:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.557+0000 7f0a60c2b740 1 --2- 172.21.15.1:0/725549766 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55bf336ffcc0 0x55bf337020b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.557+0000 7f0a60c2b740 4 client.25772 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:56:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.557+0000 7f0a60c2b740 1 -- 172.21.15.1:0/725549766 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf3365e8b0 con 0x55bf336ffcc0 2024-04-26T01:56:38.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.558+0000 7f0a56a00640 1 --2- 172.21.15.1:0/725549766 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55bf336ffcc0 0x55bf337020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.558+0000 7f0a56a00640 1 --2- 172.21.15.1:0/725549766 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55bf336ffcc0 0x55bf337020b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34ea8dff527783ef server_cookie=ca1374735e3a23a8 in_seq=0 out_seq=0 2024-04-26T01:56:38.559 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.558+0000 7f0a44c00640 10 client.25772 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:56:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.559+0000 7f0a44c00640 1 -- 172.21.15.1:0/725549766 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf3365e8b0 con 0x55bf336ffcc0 2024-04-26T01:56:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.559+0000 7f0a44c00640 10 client.25772 handle_command_reply: tid=0 2024-04-26T01:56:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.559+0000 7f0a60c2b740 1 client.25772 shutdown 2024-04-26T01:56:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.559+0000 7f0a60c2b740 1 -- 172.21.15.1:0/725549766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336e52d0 msgr2=0x55bf336ecfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.559+0000 7f0a60c2b740 1 --2- 172.21.15.1:0/725549766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336e52d0 0x55bf336ecfb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0a40011770 tx=0x7f0a40011670 comp rx=0 tx=0).stop 2024-04-26T01:56:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.559+0000 7f0a60c2b740 1 -- 172.21.15.1:0/725549766 shutdown_connections 2024-04-26T01:56:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.559+0000 7f0a60c2b740 1 --2- 172.21.15.1:0/725549766 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55bf336ffcc0 0x55bf337020b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.559+0000 7f0a60c2b740 1 --2- 172.21.15.1:0/725549766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf336efb90 0x55bf336ed4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.559+0000 7f0a60c2b740 1 --2- 172.21.15.1:0/725549766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336e52d0 0x55bf336ecfb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.559+0000 7f0a60c2b740 1 --2- 172.21.15.1:0/725549766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336e49b0 0x55bf336eca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.559+0000 7f0a44c00640 0 client.25772 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:56:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.559+0000 7f0a60c2b740 1 -- 172.21.15.1:0/725549766 >> 172.21.15.1:0/725549766 conn(0x55bf336e0ec0 msgr2=0x55bf336e6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.559+0000 7f0a60c2b740 1 -- 172.21.15.1:0/725549766 shutdown_connections 2024-04-26T01:56:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.559+0000 7f0a60c2b740 1 -- 172.21.15.1:0/725549766 wait complete. 2024-04-26T01:56:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.560+0000 7f0a60c2b740 20 client.25772 trim_cache size 0 max 16384 2024-04-26T01:56:38.566 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:38 smithi173 ceph-mon[32553]: pgmap v153: 97 pgs: 97 active+clean; 40 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2024-04-26T01:56:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.580+0000 7f0a60c2b740 1 Processor -- start 2024-04-26T01:56:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.580+0000 7f0a60c2b740 1 -- start start 2024-04-26T01:56:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.580+0000 7f0a60c2b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336e49b0 0x55bf336fb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.580+0000 7f0a60c2b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336e52d0 0x55bf336e7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.580+0000 7f0a60c2b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf336efb90 0x55bf336ed140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.580+0000 7f0a60c2b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581b6930 con 0x55bf336efb90 2024-04-26T01:56:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.580+0000 7f0a60c2b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581b4200 con 0x55bf336e49b0 2024-04-26T01:56:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.580+0000 7f0a60c2b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581b2050 con 0x55bf336e52d0 2024-04-26T01:56:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.581+0000 7f0a4e000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336e52d0 0x55bf336e7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.581+0000 7f0a56a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336e49b0 0x55bf336fb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.581+0000 7f0a4e000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336e52d0 0x55bf336e7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:60120/0 (socket says 172.21.15.1:60120) 2024-04-26T01:56:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.581+0000 7f0a57400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf336efb90 0x55bf336ed140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.581+0000 7f0a56a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336e49b0 0x55bf336fb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:58100/0 (socket says 172.21.15.1:58100) 2024-04-26T01:56:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.581+0000 7f0a4e000640 1 -- 172.21.15.1:0/1946721985 learned_addr learned my addr 172.21.15.1:0/1946721985 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:56:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.581+0000 7f0a4e000640 1 -- 172.21.15.1:0/1946721985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336e49b0 msgr2=0x55bf336fb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.581+0000 7f0a4e000640 1 --2- 172.21.15.1:0/1946721985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336e49b0 0x55bf336fb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.581+0000 7f0a4e000640 1 -- 172.21.15.1:0/1946721985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf336efb90 msgr2=0x55bf336ed140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.581+0000 7f0a4e000640 1 --2- 172.21.15.1:0/1946721985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf336efb90 0x55bf336ed140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.581+0000 7f0a4e000640 1 -- 172.21.15.1:0/1946721985 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf336e78b0 con 0x55bf336e52d0 2024-04-26T01:56:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.581+0000 7f0a4e000640 1 --2- 172.21.15.1:0/1946721985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336e52d0 0x55bf336e7370 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0a40001eb0 tx=0x7f0a400107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bb51a106eb0bad0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.582+0000 7f0a47400640 1 -- 172.21.15.1:0/1946721985 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4003b020 con 0x55bf336e52d0 2024-04-26T01:56:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.582+0000 7f0a47400640 1 -- 172.21.15.1:0/1946721985 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a40011b80 con 0x55bf336e52d0 2024-04-26T01:56:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.582+0000 7f0a47400640 1 -- 172.21.15.1:0/1946721985 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40005040 con 0x55bf336e52d0 2024-04-26T01:56:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.582+0000 7f0a60c2b740 1 -- 172.21.15.1:0/1946721985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336e52d0 msgr2=0x55bf336e7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.582+0000 7f0a60c2b740 1 --2- 172.21.15.1:0/1946721985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336e52d0 0x55bf336e7370 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0a40001eb0 tx=0x7f0a400107a0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.582+0000 7f0a60c2b740 1 -- 172.21.15.1:0/1946721985 shutdown_connections 2024-04-26T01:56:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.582+0000 7f0a60c2b740 1 --2- 172.21.15.1:0/1946721985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf336efb90 0x55bf336ed140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.583+0000 7f0a60c2b740 1 --2- 172.21.15.1:0/1946721985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336e52d0 0x55bf336e7370 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.583+0000 7f0a60c2b740 1 --2- 172.21.15.1:0/1946721985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336e49b0 0x55bf336fb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.583+0000 7f0a60c2b740 1 -- 172.21.15.1:0/1946721985 >> 172.21.15.1:0/1946721985 conn(0x55bf336e0ec0 msgr2=0x55bf336eca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.583+0000 7f0a60c2b740 1 -- 172.21.15.1:0/1946721985 shutdown_connections 2024-04-26T01:56:38.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.583+0000 7f0a60c2b740 1 -- 172.21.15.1:0/1946721985 wait complete. 2024-04-26T01:56:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.583+0000 7f0a60c2b740 1 -- start start 2024-04-26T01:56:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.583+0000 7f0a60c2b740 1 Processor -- start 2024-04-26T01:56:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.584+0000 7f0a60c2b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336e49b0 0x55bf336f7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.584+0000 7f0a60c2b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf336e52d0 0x55bf336f5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.584+0000 7f0a60c2b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336efb90 0x55bf336f5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.584+0000 7f0a60c2b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581b26c0 con 0x55bf336e52d0 2024-04-26T01:56:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.584+0000 7f0a60c2b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581b3b30 con 0x55bf336e49b0 2024-04-26T01:56:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.584+0000 7f0a60c2b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581b2d90 con 0x55bf336efb90 2024-04-26T01:56:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.584+0000 7f0a60c2b740 10 client.0 _ll_register_callbacks cb 0x7f0a400056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.584+0000 7f0a56a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336e49b0 0x55bf336f7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.584+0000 7f0a56a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336e49b0 0x55bf336f7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:58108/0 (socket says 172.21.15.1:58108) 2024-04-26T01:56:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.584+0000 7f0a56a00640 1 -- 172.21.15.1:0/2919896195 learned_addr learned my addr 172.21.15.1:0/2919896195 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:56:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.584+0000 7f0a4e000640 1 --2- 172.21.15.1:0/2919896195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf336e52d0 0x55bf336f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.584+0000 7f0a57400640 1 --2- 172.21.15.1:0/2919896195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336efb90 0x55bf336f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.584+0000 7f0a56a00640 1 -- 172.21.15.1:0/2919896195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336efb90 msgr2=0x55bf336f5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.584+0000 7f0a56a00640 1 --2- 172.21.15.1:0/2919896195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336efb90 0x55bf336f5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.584+0000 7f0a56a00640 1 -- 172.21.15.1:0/2919896195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf336e52d0 msgr2=0x55bf336f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.584+0000 7f0a56a00640 1 --2- 172.21.15.1:0/2919896195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf336e52d0 0x55bf336f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.584+0000 7f0a56a00640 1 -- 172.21.15.1:0/2919896195 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf336f5ea0 con 0x55bf336e49b0 2024-04-26T01:56:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.585+0000 7f0a56a00640 1 --2- 172.21.15.1:0/2919896195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336e49b0 0x55bf336f7e90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0a480bcfa0 tx=0x7f0a48009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6568e8f17f889a93 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.585+0000 7f0a44c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:56:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.585+0000 7f0a44c00640 1 -- 172.21.15.1:0/2919896195 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a480eee60 con 0x55bf336e49b0 2024-04-26T01:56:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.585+0000 7f0a44c00640 1 -- 172.21.15.1:0/2919896195 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f0a4803b580 con 0x55bf336e49b0 2024-04-26T01:56:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.585+0000 7f0a60c2b740 1 -- 172.21.15.1:0/2919896195 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a40011b80 con 0x55bf336e49b0 2024-04-26T01:56:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.585+0000 7f0a44c00640 1 -- 172.21.15.1:0/2919896195 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0a48049040 con 0x55bf336e49b0 2024-04-26T01:56:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.585+0000 7f0a60c2b740 10 client.25778 fetch_fsmap learned FSMap version 32 2024-04-26T01:56:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.585+0000 7f0a60c2b740 1 -- 172.21.15.1:0/2919896195 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bf33766040 con 0x55bf336e49b0 2024-04-26T01:56:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.586+0000 7f0a44c00640 1 -- 172.21.15.1:0/2919896195 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a40011b80 con 0x55bf336e49b0 2024-04-26T01:56:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.586+0000 7f0a44c00640 1 -- 172.21.15.1:0/2919896195 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f0a480f4c60 con 0x55bf336e49b0 2024-04-26T01:56:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.586+0000 7f0a60c2b740 10 client.25778 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:56:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.586+0000 7f0a60c2b740 10 client.25778 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:56:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.586+0000 7f0a60c2b740 20 client.25778 populate_metadata read hostname 'smithi001' 2024-04-26T01:56:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.586+0000 7f0a60c2b740 1 --2- 172.21.15.1:0/2919896195 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55bf33766760 0x55bf33768b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.586+0000 7f0a60c2b740 4 client.25778 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:56:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.586+0000 7f0a60c2b740 1 -- 172.21.15.1:0/2919896195 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf33658320 con 0x55bf33766760 2024-04-26T01:56:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.586+0000 7f0a4e000640 1 --2- 172.21.15.1:0/2919896195 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55bf33766760 0x55bf33768b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.587+0000 7f0a4e000640 1 --2- 172.21.15.1:0/2919896195 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55bf33766760 0x55bf33768b80 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=312b14e942283dbb server_cookie=a792c02c08c4c9cc in_seq=0 out_seq=0 2024-04-26T01:56:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.587+0000 7f0a44c00640 10 client.25778 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:56:38.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.604+0000 7f0a44c00640 1 -- 172.21.15.1:0/2919896195 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf33658320 con 0x55bf33766760 2024-04-26T01:56:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.604+0000 7f0a44c00640 10 client.25778 handle_command_reply: tid=0 2024-04-26T01:56:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.605+0000 7f0a60c2b740 1 client.25778 shutdown 2024-04-26T01:56:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.605+0000 7f0a60c2b740 1 -- 172.21.15.1:0/2919896195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336e49b0 msgr2=0x55bf336f7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.605+0000 7f0a60c2b740 1 --2- 172.21.15.1:0/2919896195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336e49b0 0x55bf336f7e90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0a480bcfa0 tx=0x7f0a48009680 comp rx=0 tx=0).stop 2024-04-26T01:56:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.605+0000 7f0a60c2b740 1 -- 172.21.15.1:0/2919896195 shutdown_connections 2024-04-26T01:56:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.605+0000 7f0a60c2b740 1 --2- 172.21.15.1:0/2919896195 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55bf33766760 0x55bf33768b80 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.605+0000 7f0a60c2b740 1 --2- 172.21.15.1:0/2919896195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55bf336efb90 0x55bf336f5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.605+0000 7f0a60c2b740 1 --2- 172.21.15.1:0/2919896195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf336e52d0 0x55bf336f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.605+0000 7f0a60c2b740 1 --2- 172.21.15.1:0/2919896195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55bf336e49b0 0x55bf336f7e90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.605+0000 7f0a44c00640 0 client.25778 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:56:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.605+0000 7f0a60c2b740 1 -- 172.21.15.1:0/2919896195 >> 172.21.15.1:0/2919896195 conn(0x55bf336e0ec0 msgr2=0x55bf336ed240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.605+0000 7f0a60c2b740 1 -- 172.21.15.1:0/2919896195 shutdown_connections 2024-04-26T01:56:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.605+0000 7f0a60c2b740 1 -- 172.21.15.1:0/2919896195 wait complete. 2024-04-26T01:56:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.605+0000 7f0a60c2b740 20 client.25778 trim_cache size 0 max 16384 2024-04-26T01:56:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:56:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:56:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:56:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:56:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.606+0000 7f0a57e00640 1 -- 172.21.15.1:0/1818684027 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0a3c0a3d80 msgr2=0x7f0a3c0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.606+0000 7f0a57e00640 1 --2- 172.21.15.1:0/1818684027 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0a3c0a3d80 0x7f0a3c0a6240 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0a50005fd0 tx=0x7f0a50009600 comp rx=0 tx=0).stop 2024-04-26T01:56:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.606+0000 7f0a57e00640 1 -- 172.21.15.1:0/1818684027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a581083b0 msgr2=0x7f0a581a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.606+0000 7f0a57e00640 1 --2- 172.21.15.1:0/1818684027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a581083b0 0x7f0a581a4770 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0a48005ab0 tx=0x7f0a48003cd0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.606+0000 7f0a57e00640 1 -- 172.21.15.1:0/1818684027 shutdown_connections 2024-04-26T01:56:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.606+0000 7f0a57e00640 1 --2- 172.21.15.1:0/1818684027 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0a3c0a3d80 0x7f0a3c0a6240 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.606+0000 7f0a57e00640 1 --2- 172.21.15.1:0/1818684027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a5810e470 0x7f0a5819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.606+0000 7f0a57e00640 1 --2- 172.21.15.1:0/1818684027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a58108d50 0x7f0a581a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.606+0000 7f0a57e00640 1 --2- 172.21.15.1:0/1818684027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a581083b0 0x7f0a581a4770 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.606+0000 7f0a57e00640 1 -- 172.21.15.1:0/1818684027 >> 172.21.15.1:0/1818684027 conn(0x7f0a58077b00 msgr2=0x7f0a580794b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.606+0000 7f0a57e00640 1 -- 172.21.15.1:0/1818684027 shutdown_connections 2024-04-26T01:56:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:38.606+0000 7f0a57e00640 1 -- 172.21.15.1:0/1818684027 wait complete. 2024-04-26T01:56:40.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:40 smithi001 ceph-mon[27822]: pgmap v154: 97 pgs: 97 active+clean; 40 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2024-04-26T01:56:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:40 smithi097 ceph-mon[32853]: pgmap v154: 97 pgs: 97 active+clean; 40 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2024-04-26T01:56:40.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:40 smithi173 ceph-mon[32553]: pgmap v154: 97 pgs: 97 active+clean; 40 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 231 op/s 2024-04-26T01:56:42.369 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:42 smithi173 ceph-mon[32553]: pgmap v155: 97 pgs: 97 active+clean; 37 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 333 op/s 2024-04-26T01:56:42.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:42 smithi001 ceph-mon[27822]: pgmap v155: 97 pgs: 97 active+clean; 37 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 333 op/s 2024-04-26T01:56:42.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:42 smithi097 ceph-mon[32853]: pgmap v155: 97 pgs: 97 active+clean; 37 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 333 op/s 2024-04-26T01:56:44.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:44 smithi001 ceph-mon[27822]: pgmap v156: 97 pgs: 97 active+clean; 37 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 206 op/s 2024-04-26T01:56:44.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:44 smithi097 ceph-mon[32853]: pgmap v156: 97 pgs: 97 active+clean; 37 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 206 op/s 2024-04-26T01:56:44.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:44 smithi173 ceph-mon[32553]: pgmap v156: 97 pgs: 97 active+clean; 37 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 206 op/s 2024-04-26T01:56:46.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:46 smithi001 ceph-mon[27822]: pgmap v157: 97 pgs: 97 active+clean; 39 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 318 op/s 2024-04-26T01:56:46.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:46 smithi097 ceph-mon[32853]: pgmap v157: 97 pgs: 97 active+clean; 39 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 318 op/s 2024-04-26T01:56:46.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:46 smithi173 ceph-mon[32553]: pgmap v157: 97 pgs: 97 active+clean; 39 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 318 op/s 2024-04-26T01:56:48.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:48 smithi001 ceph-mon[27822]: pgmap v158: 97 pgs: 97 active+clean; 39 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2024-04-26T01:56:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:48 smithi097 ceph-mon[32853]: pgmap v158: 97 pgs: 97 active+clean; 39 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2024-04-26T01:56:48.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:48 smithi173 ceph-mon[32553]: pgmap v158: 97 pgs: 97 active+clean; 39 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2024-04-26T01:56:49.721 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xffdb8 2024-04-26T01:56:50.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:50 smithi001 ceph-mon[27822]: pgmap v159: 97 pgs: 97 active+clean; 39 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2024-04-26T01:56:50.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:50 smithi097 ceph-mon[32853]: pgmap v159: 97 pgs: 97 active+clean; 39 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2024-04-26T01:56:50.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:50 smithi173 ceph-mon[32553]: pgmap v159: 97 pgs: 97 active+clean; 39 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 214 op/s 2024-04-26T01:56:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.772+0000 7f182d200640 1 -- 172.21.15.1:0/759509133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18281031c0 msgr2=0x7f18281035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.772+0000 7f182d200640 1 --2- 172.21.15.1:0/759509133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18281031c0 0x7f18281035a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1818009900 tx=0x7f181802a480 comp rx=0 tx=0).stop 2024-04-26T01:56:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.772+0000 7f182d200640 1 -- 172.21.15.1:0/759509133 shutdown_connections 2024-04-26T01:56:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.772+0000 7f182d200640 1 --2- 172.21.15.1:0/759509133 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f182810c590 0x7f182810e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.772+0000 7f182d200640 1 --2- 172.21.15.1:0/759509133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1828103ae0 0x7f182810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.772+0000 7f182d200640 1 --2- 172.21.15.1:0/759509133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18281031c0 0x7f18281035a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.772+0000 7f182d200640 1 -- 172.21.15.1:0/759509133 >> 172.21.15.1:0/759509133 conn(0x7f1828077fa0 msgr2=0x7f18280783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.772+0000 7f182d200640 1 -- 172.21.15.1:0/759509133 shutdown_connections 2024-04-26T01:56:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.772+0000 7f182d200640 1 -- 172.21.15.1:0/759509133 wait complete. 2024-04-26T01:56:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.774+0000 7f182d200640 1 Processor -- start 2024-04-26T01:56:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.774+0000 7f182d200640 1 -- start start 2024-04-26T01:56:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.774+0000 7f182d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18281031c0 0x7f18281a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.774+0000 7f182d200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1828103ae0 0x7f18281a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.774+0000 7f182d200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f182810c590 0x7f182819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.774+0000 7f182d200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f18281121d0 con 0x7f182810c590 2024-04-26T01:56:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.774+0000 7f182d200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1828112050 con 0x7f18281031c0 2024-04-26T01:56:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.774+0000 7f182d200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1828112350 con 0x7f1828103ae0 2024-04-26T01:56:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.775+0000 7f1826800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f182810c590 0x7f182819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.775+0000 7f1825e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18281031c0 0x7f18281a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.775+0000 7f1825400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1828103ae0 0x7f18281a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.775+0000 7f1826800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f182810c590 0x7f182819a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53952/0 (socket says 172.21.15.1:53952) 2024-04-26T01:56:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.775+0000 7f1825e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18281031c0 0x7f18281a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:51650/0 (socket says 172.21.15.1:51650) 2024-04-26T01:56:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.775+0000 7f1826800640 1 -- 172.21.15.1:0/3900808520 learned_addr learned my addr 172.21.15.1:0/3900808520 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:56:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.775+0000 7f1825400640 1 --2- 172.21.15.1:0/3900808520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1828103ae0 0x7f18281a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:36622/0 (socket says 172.21.15.1:36622) 2024-04-26T01:56:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.775+0000 7f1825400640 1 -- 172.21.15.1:0/3900808520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18281031c0 msgr2=0x7f18281a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.775+0000 7f1825400640 1 --2- 172.21.15.1:0/3900808520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18281031c0 0x7f18281a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.775+0000 7f1825400640 1 -- 172.21.15.1:0/3900808520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f182810c590 msgr2=0x7f182819a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.775+0000 7f1825400640 1 --2- 172.21.15.1:0/3900808520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f182810c590 0x7f182819a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.775+0000 7f1825400640 1 -- 172.21.15.1:0/3900808520 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f182819afa0 con 0x7f1828103ae0 2024-04-26T01:56:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.776+0000 7f1825400640 1 --2- 172.21.15.1:0/3900808520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1828103ae0 0x7f18281a0b50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f181400e9f0 tx=0x7f181400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.776+0000 7f181ea00640 1 -- 172.21.15.1:0/3900808520 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181400cd80 con 0x7f1828103ae0 2024-04-26T01:56:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.776+0000 7f181ea00640 1 -- 172.21.15.1:0/3900808520 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18140035f0 con 0x7f1828103ae0 2024-04-26T01:56:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.776+0000 7f182d200640 1 -- 172.21.15.1:0/3900808520 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f182819b290 con 0x7f1828103ae0 2024-04-26T01:56:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.776+0000 7f181ea00640 1 -- 172.21.15.1:0/3900808520 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1814010670 con 0x7f1828103ae0 2024-04-26T01:56:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.776+0000 7f182d200640 1 -- 172.21.15.1:0/3900808520 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18281a7460 con 0x7f1828103ae0 2024-04-26T01:56:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.779+0000 7f181ea00640 1 -- 172.21.15.1:0/3900808520 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f18140037a0 con 0x7f1828103ae0 2024-04-26T01:56:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.780+0000 7f1830166740 1 Processor -- start 2024-04-26T01:56:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.780+0000 7f1830166740 1 -- start start 2024-04-26T01:56:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.780+0000 7f1830166740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f030 0x56047196f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.781+0000 7f1830166740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047196f950 0x560471979cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.781+0000 7f1830166740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047197a210 0x56047197c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.781+0000 7f1830166740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5604718cc900 con 0x56047196f030 2024-04-26T01:56:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.781+0000 7f1830166740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560471872ef0 con 0x56047196f950 2024-04-26T01:56:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.781+0000 7f1830166740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5604718c88c0 con 0x56047197a210 2024-04-26T01:56:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.781+0000 7f1826800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f030 0x56047196f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.781+0000 7f1825e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047196f950 0x560471979cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.781+0000 7f1825e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047196f950 0x560471979cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:51666/0 (socket says 172.21.15.1:51666) 2024-04-26T01:56:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.781+0000 7f1825e00640 1 -- 172.21.15.1:0/1330322272 learned_addr learned my addr 172.21.15.1:0/1330322272 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:56:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.781+0000 7f1825400640 1 --2- 172.21.15.1:0/1330322272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047197a210 0x56047197c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.781+0000 7f1825e00640 1 -- 172.21.15.1:0/1330322272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047197a210 msgr2=0x56047197c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.781+0000 7f1825e00640 1 --2- 172.21.15.1:0/1330322272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047197a210 0x56047197c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.781+0000 7f1825e00640 1 -- 172.21.15.1:0/1330322272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f030 msgr2=0x56047196f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.781+0000 7f1825e00640 1 --2- 172.21.15.1:0/1330322272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f030 0x56047196f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.781+0000 7f1825e00640 1 -- 172.21.15.1:0/1330322272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604715df210 con 0x56047196f950 2024-04-26T01:56:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.781+0000 7f181ea00640 1 --2- 172.21.15.1:0/3900808520 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f180c0a3d10 0x7f180c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.782+0000 7f181ea00640 1 -- 172.21.15.1:0/3900808520 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f18140d0ae0 con 0x7f1828103ae0 2024-04-26T01:56:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.782+0000 7f1825e00640 1 --2- 172.21.15.1:0/1330322272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047196f950 0x560471979cd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1818009900 tx=0x7f1818032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17e9cfa30ed6c84c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.782+0000 7f1826800640 1 --2- 172.21.15.1:0/3900808520 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f180c0a3d10 0x7f180c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.782+0000 7f1813400640 1 -- 172.21.15.1:0/1330322272 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818040070 con 0x56047196f950 2024-04-26T01:56:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.782+0000 7f1813400640 1 -- 172.21.15.1:0/1330322272 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1818004030 con 0x56047196f950 2024-04-26T01:56:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.782+0000 7f1813400640 1 -- 172.21.15.1:0/1330322272 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818035040 con 0x56047196f950 2024-04-26T01:56:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.783+0000 7f1830166740 1 -- 172.21.15.1:0/1330322272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047196f950 msgr2=0x560471979cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.783+0000 7f1826800640 1 --2- 172.21.15.1:0/3900808520 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f180c0a3d10 0x7f180c0a61d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f18200049d0 tx=0x7f1820009500 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.783+0000 7f1830166740 1 --2- 172.21.15.1:0/1330322272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047196f950 0x560471979cd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1818009900 tx=0x7f1818032b20 comp rx=0 tx=0).stop 2024-04-26T01:56:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.783+0000 7f1830166740 1 -- 172.21.15.1:0/1330322272 shutdown_connections 2024-04-26T01:56:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.783+0000 7f1830166740 1 --2- 172.21.15.1:0/1330322272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047197a210 0x56047197c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.783+0000 7f1830166740 1 --2- 172.21.15.1:0/1330322272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047196f950 0x560471979cd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.783+0000 7f1830166740 1 --2- 172.21.15.1:0/1330322272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f030 0x56047196f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.783+0000 7f1830166740 1 -- 172.21.15.1:0/1330322272 >> 172.21.15.1:0/1330322272 conn(0x56047196b540 msgr2=0x56047196b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.783+0000 7f1830166740 1 -- 172.21.15.1:0/1330322272 shutdown_connections 2024-04-26T01:56:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.783+0000 7f1830166740 1 -- 172.21.15.1:0/1330322272 wait complete. 2024-04-26T01:56:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.784+0000 7f1830166740 1 -- start start 2024-04-26T01:56:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.784+0000 7f1830166740 1 Processor -- start 2024-04-26T01:56:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.785+0000 7f1830166740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047196f030 0x560471977170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.785+0000 7f1830166740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f950 0x5604719776b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.785+0000 7f1830166740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047197a210 0x560471977bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.785+0000 7f1830166740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56047197cde0 con 0x56047196f950 2024-04-26T01:56:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.785+0000 7f1830166740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560471886310 con 0x56047197a210 2024-04-26T01:56:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.785+0000 7f1830166740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56047197cf60 con 0x56047196f030 2024-04-26T01:56:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.785+0000 7f1830166740 10 client.0 _ll_register_callbacks cb 0x560471854cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.785+0000 7f1825e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047196f030 0x560471977170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.785+0000 7f1826800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047197a210 0x560471977bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.785+0000 7f1825e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047196f030 0x560471977170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:36638/0 (socket says 172.21.15.1:36638) 2024-04-26T01:56:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.785+0000 7f1825e00640 1 -- 172.21.15.1:0/3631336157 learned_addr learned my addr 172.21.15.1:0/3631336157 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:56:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.785+0000 7f1825400640 1 --2- 172.21.15.1:0/3631336157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f950 0x5604719776b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.785+0000 7f1825e00640 1 -- 172.21.15.1:0/3631336157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047197a210 msgr2=0x560471977bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.785+0000 7f1825e00640 1 --2- 172.21.15.1:0/3631336157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047197a210 0x560471977bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.785+0000 7f1825e00640 1 -- 172.21.15.1:0/3631336157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f950 msgr2=0x5604719776b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.785+0000 7f1825e00640 1 --2- 172.21.15.1:0/3631336157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f950 0x5604719776b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.785+0000 7f1825e00640 1 -- 172.21.15.1:0/3631336157 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604716ab620 con 0x56047196f030 2024-04-26T01:56:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.786+0000 7f1825e00640 1 --2- 172.21.15.1:0/3631336157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047196f030 0x560471977170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1818032ec0 tx=0x7f1818032ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8cdffc4b574511 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.786+0000 7f1807e00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:56:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.786+0000 7f1807e00640 1 -- 172.21.15.1:0/3631336157 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818040070 con 0x56047196f030 2024-04-26T01:56:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.786+0000 7f1830166740 1 -- 172.21.15.1:0/3631336157 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1818004030 con 0x56047196f030 2024-04-26T01:56:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.786+0000 7f1807e00640 1 -- 172.21.15.1:0/3631336157 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f18180316a0 con 0x56047196f030 2024-04-26T01:56:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.787+0000 7f1807e00640 1 -- 172.21.15.1:0/3631336157 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1818056050 con 0x56047196f030 2024-04-26T01:56:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.787+0000 7f1830166740 10 client.25813 fetch_fsmap learned FSMap version 32 2024-04-26T01:56:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.787+0000 7f1830166740 1 -- 172.21.15.1:0/3631336157 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56047197cde0 con 0x56047196f030 2024-04-26T01:56:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.787+0000 7f1807e00640 1 -- 172.21.15.1:0/3631336157 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1818004030 con 0x56047196f030 2024-04-26T01:56:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.787+0000 7f1807e00640 1 -- 172.21.15.1:0/3631336157 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f1818004b20 con 0x56047196f030 2024-04-26T01:56:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.787+0000 7f1830166740 10 client.25813 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:56:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.787+0000 7f1830166740 10 client.25813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:56:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.787+0000 7f1830166740 20 client.25813 populate_metadata read hostname 'smithi001' 2024-04-26T01:56:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.787+0000 7f1830166740 1 --2- 172.21.15.1:0/3631336157 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56047198a180 0x56047198c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.787+0000 7f1830166740 4 client.25813 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:56:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.787+0000 7f1830166740 1 -- 172.21.15.1:0/3631336157 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604718e08b0 con 0x56047198a180 2024-04-26T01:56:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.788+0000 7f1825400640 1 --2- 172.21.15.1:0/3631336157 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56047198a180 0x56047198c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.788+0000 7f1825400640 1 --2- 172.21.15.1:0/3631336157 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56047198a180 0x56047198c570 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91b207b8378ee7 server_cookie=e25ef4de86c45e27 in_seq=0 out_seq=0 2024-04-26T01:56:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.788+0000 7f1807e00640 10 client.25813 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:56:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.789+0000 7f1807e00640 1 -- 172.21.15.1:0/3631336157 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5604718e08b0 con 0x56047198a180 2024-04-26T01:56:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.789+0000 7f1807e00640 10 client.25813 handle_command_reply: tid=0 2024-04-26T01:56:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.789+0000 7f1830166740 1 client.25813 shutdown 2024-04-26T01:56:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.789+0000 7f1830166740 1 -- 172.21.15.1:0/3631336157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047196f030 msgr2=0x560471977170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.789+0000 7f1830166740 1 --2- 172.21.15.1:0/3631336157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047196f030 0x560471977170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1818032ec0 tx=0x7f1818032ef0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.789+0000 7f1830166740 1 -- 172.21.15.1:0/3631336157 shutdown_connections 2024-04-26T01:56:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.789+0000 7f1830166740 1 --2- 172.21.15.1:0/3631336157 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56047198a180 0x56047198c570 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.789+0000 7f1830166740 1 --2- 172.21.15.1:0/3631336157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047197a210 0x560471977bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.789+0000 7f1807e00640 0 client.25813 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:56:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.789+0000 7f1830166740 1 --2- 172.21.15.1:0/3631336157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f950 0x5604719776b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.789+0000 7f1830166740 1 --2- 172.21.15.1:0/3631336157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047196f030 0x560471977170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.789+0000 7f1830166740 1 -- 172.21.15.1:0/3631336157 >> 172.21.15.1:0/3631336157 conn(0x56047196b540 msgr2=0x5604719712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.790+0000 7f1830166740 1 -- 172.21.15.1:0/3631336157 shutdown_connections 2024-04-26T01:56:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.790+0000 7f1830166740 1 -- 172.21.15.1:0/3631336157 wait complete. 2024-04-26T01:56:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.790+0000 7f1830166740 20 client.25813 trim_cache size 0 max 16384 2024-04-26T01:56:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.808+0000 7f1830166740 1 Processor -- start 2024-04-26T01:56:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.809+0000 7f1830166740 1 -- start start 2024-04-26T01:56:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.809+0000 7f1830166740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f030 0x560471985ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.809+0000 7f1830166740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047196f950 0x560471961d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.809+0000 7f1830166740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047197a210 0x560471991c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.809+0000 7f1830166740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f18281ae2e0 con 0x56047196f030 2024-04-26T01:56:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.809+0000 7f1830166740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f18281abbb0 con 0x56047197a210 2024-04-26T01:56:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.809+0000 7f1830166740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f18281a9a20 con 0x56047196f950 2024-04-26T01:56:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.809+0000 7f1825e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f030 0x560471985ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.809+0000 7f1826800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047197a210 0x560471991c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.809+0000 7f1825e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f030 0x560471985ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53992/0 (socket says 172.21.15.1:53992) 2024-04-26T01:56:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.809+0000 7f1825e00640 1 -- 172.21.15.1:0/1233469370 learned_addr learned my addr 172.21.15.1:0/1233469370 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:56:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.809+0000 7f1825400640 1 --2- 172.21.15.1:0/1233469370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047196f950 0x560471961d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.810+0000 7f1825e00640 1 -- 172.21.15.1:0/1233469370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047196f950 msgr2=0x560471961d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.810+0000 7f1825e00640 1 --2- 172.21.15.1:0/1233469370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047196f950 0x560471961d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.810+0000 7f1825e00640 1 -- 172.21.15.1:0/1233469370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047197a210 msgr2=0x560471991c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.810+0000 7f1825e00640 1 --2- 172.21.15.1:0/1233469370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047197a210 0x560471991c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.810+0000 7f1825e00640 1 -- 172.21.15.1:0/1233469370 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604719756c0 con 0x56047196f030 2024-04-26T01:56:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.810+0000 7f1825400640 1 --2- 172.21.15.1:0/1233469370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047196f950 0x560471961d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:56:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.810+0000 7f1826800640 1 --2- 172.21.15.1:0/1233469370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047197a210 0x560471991c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.810+0000 7f1825e00640 1 --2- 172.21.15.1:0/1233469370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f030 0x560471985ca0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f181802a9a0 tx=0x7f1818031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ea081a63cefef6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.810+0000 7f1812a00640 1 -- 172.21.15.1:0/1233469370 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181805f070 con 0x56047196f030 2024-04-26T01:56:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.811+0000 7f1812a00640 1 -- 172.21.15.1:0/1233469370 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f181804d070 con 0x56047196f030 2024-04-26T01:56:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.811+0000 7f1812a00640 1 -- 172.21.15.1:0/1233469370 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818040070 con 0x56047196f030 2024-04-26T01:56:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.811+0000 7f1830166740 1 -- 172.21.15.1:0/1233469370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f030 msgr2=0x560471985ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.811+0000 7f1830166740 1 --2- 172.21.15.1:0/1233469370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f030 0x560471985ca0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f181802a9a0 tx=0x7f1818031400 comp rx=0 tx=0).stop 2024-04-26T01:56:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.811+0000 7f1830166740 1 -- 172.21.15.1:0/1233469370 shutdown_connections 2024-04-26T01:56:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.811+0000 7f1830166740 1 --2- 172.21.15.1:0/1233469370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047197a210 0x560471991c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.811+0000 7f1830166740 1 --2- 172.21.15.1:0/1233469370 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047196f950 0x560471961d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.811+0000 7f1830166740 1 --2- 172.21.15.1:0/1233469370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f030 0x560471985ca0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.811+0000 7f1830166740 1 -- 172.21.15.1:0/1233469370 >> 172.21.15.1:0/1233469370 conn(0x56047196b540 msgr2=0x560471961600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.812+0000 7f1830166740 1 -- 172.21.15.1:0/1233469370 shutdown_connections 2024-04-26T01:56:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.812+0000 7f1830166740 1 -- 172.21.15.1:0/1233469370 wait complete. 2024-04-26T01:56:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.812+0000 7f1830166740 1 -- start start 2024-04-26T01:56:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.812+0000 7f1830166740 1 Processor -- start 2024-04-26T01:56:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.813+0000 7f1830166740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f030 0x560471981e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.813+0000 7f1830166740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047196f950 0x5604719823a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.813+0000 7f1830166740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047197a210 0x56047197f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.813+0000 7f1830166740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f18281aa070 con 0x56047196f030 2024-04-26T01:56:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.813+0000 7f1830166740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f18281ab4e0 con 0x56047197a210 2024-04-26T01:56:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.813+0000 7f1830166740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f18281aa740 con 0x56047196f950 2024-04-26T01:56:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.813+0000 7f1830166740 10 client.0 _ll_register_callbacks cb 0x7f18180095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:56:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.813+0000 7f1825e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f030 0x560471981e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.813+0000 7f1825e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f030 0x560471981e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53998/0 (socket says 172.21.15.1:53998) 2024-04-26T01:56:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.813+0000 7f1825e00640 1 -- 172.21.15.1:0/4157757596 learned_addr learned my addr 172.21.15.1:0/4157757596 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:56:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.813+0000 7f1825400640 1 --2- 172.21.15.1:0/4157757596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047196f950 0x5604719823a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.813+0000 7f1826800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047197a210 0x56047197f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.813+0000 7f1825e00640 1 -- 172.21.15.1:0/4157757596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047196f950 msgr2=0x5604719823a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.813+0000 7f1825e00640 1 --2- 172.21.15.1:0/4157757596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047196f950 0x5604719823a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.813+0000 7f1825e00640 1 -- 172.21.15.1:0/4157757596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047197a210 msgr2=0x56047197f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:56:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.813+0000 7f1825e00640 1 --2- 172.21.15.1:0/4157757596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047197a210 0x56047197f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.813+0000 7f1825e00640 1 -- 172.21.15.1:0/4157757596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56047197fc50 con 0x56047196f030 2024-04-26T01:56:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.813+0000 7f1825400640 1 --2- 172.21.15.1:0/4157757596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047196f950 0x5604719823a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.814+0000 7f1825e00640 1 --2- 172.21.15.1:0/4157757596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f030 0x560471981e60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f181805d040 tx=0x7f1818004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37f33507a0bd95 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.814+0000 7f1807e00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:56:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.814+0000 7f1807e00640 1 -- 172.21.15.1:0/4157757596 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181805f070 con 0x56047196f030 2024-04-26T01:56:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.814+0000 7f1807e00640 1 -- 172.21.15.1:0/4157757596 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f1818005400 con 0x56047196f030 2024-04-26T01:56:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.814+0000 7f1830166740 1 -- 172.21.15.1:0/4157757596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f181804d070 con 0x56047196f030 2024-04-26T01:56:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.814+0000 7f1807e00640 1 -- 172.21.15.1:0/4157757596 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1818040040 con 0x56047196f030 2024-04-26T01:56:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.814+0000 7f1830166740 10 client.15654 fetch_fsmap learned FSMap version 32 2024-04-26T01:56:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.814+0000 7f1830166740 1 -- 172.21.15.1:0/4157757596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5604719eff70 con 0x56047196f030 2024-04-26T01:56:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.815+0000 7f1807e00640 1 -- 172.21.15.1:0/4157757596 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f181804d070 con 0x56047196f030 2024-04-26T01:56:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.815+0000 7f1807e00640 1 -- 172.21.15.1:0/4157757596 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f1818032040 con 0x56047196f030 2024-04-26T01:56:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.815+0000 7f1830166740 10 client.15654 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:56:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.815+0000 7f1830166740 10 client.15654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:56:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.815+0000 7f1830166740 20 client.15654 populate_metadata read hostname 'smithi001' 2024-04-26T01:56:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.815+0000 7f1830166740 1 --2- 172.21.15.1:0/4157757596 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5604719f0600 0x5604719f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.815+0000 7f1830166740 4 client.15654 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:56:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.815+0000 7f1830166740 1 -- 172.21.15.1:0/4157757596 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5604718da320 con 0x5604719f0600 2024-04-26T01:56:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.815+0000 7f1825400640 1 --2- 172.21.15.1:0/4157757596 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5604719f0600 0x5604719f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.816+0000 7f1825400640 1 --2- 172.21.15.1:0/4157757596 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5604719f0600 0x5604719f2a20 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fbdccfaacc3cee5 server_cookie=3d85b6c3b52de310 in_seq=0 out_seq=0 2024-04-26T01:56:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.816+0000 7f1807e00640 10 client.15654 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:56:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.829+0000 7f1807e00640 1 -- 172.21.15.1:0/4157757596 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5604718da320 con 0x5604719f0600 2024-04-26T01:56:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.829+0000 7f1807e00640 10 client.15654 handle_command_reply: tid=0 2024-04-26T01:56:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.829+0000 7f1830166740 1 client.15654 shutdown 2024-04-26T01:56:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.830+0000 7f1830166740 1 -- 172.21.15.1:0/4157757596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f030 msgr2=0x560471981e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.830+0000 7f1830166740 1 --2- 172.21.15.1:0/4157757596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f030 0x560471981e60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f181805d040 tx=0x7f1818004970 comp rx=0 tx=0).stop 2024-04-26T01:56:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.830+0000 7f1830166740 1 -- 172.21.15.1:0/4157757596 shutdown_connections 2024-04-26T01:56:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.830+0000 7f1830166740 1 --2- 172.21.15.1:0/4157757596 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5604719f0600 0x5604719f2a20 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.830+0000 7f1830166740 1 --2- 172.21.15.1:0/4157757596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56047197a210 0x56047197f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.830+0000 7f1830166740 1 --2- 172.21.15.1:0/4157757596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047196f950 0x5604719823a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.830+0000 7f1830166740 1 --2- 172.21.15.1:0/4157757596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56047196f030 0x560471981e60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.830+0000 7f1830166740 1 -- 172.21.15.1:0/4157757596 >> 172.21.15.1:0/4157757596 conn(0x56047196b540 msgr2=0x5604719779a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.830+0000 7f1807e00640 0 client.15654 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:56:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.830+0000 7f1830166740 1 -- 172.21.15.1:0/4157757596 shutdown_connections 2024-04-26T01:56:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.830+0000 7f1830166740 1 -- 172.21.15.1:0/4157757596 wait complete. 2024-04-26T01:56:51.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.830+0000 7f1830166740 20 client.15654 trim_cache size 0 max 16384 2024-04-26T01:56:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:56:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:56:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:56:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:56:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.831+0000 7f182d200640 1 -- 172.21.15.1:0/3900808520 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f180c0a3d10 msgr2=0x7f180c0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.831+0000 7f182d200640 1 --2- 172.21.15.1:0/3900808520 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f180c0a3d10 0x7f180c0a61d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f18200049d0 tx=0x7f1820009500 comp rx=0 tx=0).stop 2024-04-26T01:56:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.831+0000 7f182d200640 1 -- 172.21.15.1:0/3900808520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1828103ae0 msgr2=0x7f18281a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.831+0000 7f182d200640 1 --2- 172.21.15.1:0/3900808520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1828103ae0 0x7f18281a0b50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f181400e9f0 tx=0x7f181400eeb0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.831+0000 7f182d200640 1 -- 172.21.15.1:0/3900808520 shutdown_connections 2024-04-26T01:56:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.831+0000 7f182d200640 1 --2- 172.21.15.1:0/3900808520 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f180c0a3d10 0x7f180c0a61d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.831+0000 7f182d200640 1 --2- 172.21.15.1:0/3900808520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f182810c590 0x7f182819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.831+0000 7f182d200640 1 --2- 172.21.15.1:0/3900808520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1828103ae0 0x7f18281a0b50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.832 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.831+0000 7f182d200640 1 --2- 172.21.15.1:0/3900808520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18281031c0 0x7f18281a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:51.832 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.831+0000 7f182d200640 1 -- 172.21.15.1:0/3900808520 >> 172.21.15.1:0/3900808520 conn(0x7f1828077fa0 msgr2=0x7f18281084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:51.832 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.831+0000 7f182d200640 1 -- 172.21.15.1:0/3900808520 shutdown_connections 2024-04-26T01:56:51.832 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:56:51.831+0000 7f182d200640 1 -- 172.21.15.1:0/3900808520 wait complete. 2024-04-26T01:56:52.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:52 smithi097 ceph-mon[32853]: pgmap v160: 97 pgs: 97 active+clean; 40 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 325 op/s 2024-04-26T01:56:52.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:52 smithi173 ceph-mon[32553]: pgmap v160: 97 pgs: 97 active+clean; 40 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 325 op/s 2024-04-26T01:56:52.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:52 smithi001 ceph-mon[27822]: pgmap v160: 97 pgs: 97 active+clean; 40 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 325 op/s 2024-04-26T01:56:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:54 smithi097 ceph-mon[32853]: pgmap v161: 97 pgs: 97 active+clean; 40 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 222 op/s 2024-04-26T01:56:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:54 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:56:54.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:54 smithi173 ceph-mon[32553]: pgmap v161: 97 pgs: 97 active+clean; 40 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 222 op/s 2024-04-26T01:56:54.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:54 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:56:54.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:54 smithi001 ceph-mon[27822]: pgmap v161: 97 pgs: 97 active+clean; 40 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 222 op/s 2024-04-26T01:56:54.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:54 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:56:56.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:56 smithi001 ceph-mon[27822]: pgmap v162: 97 pgs: 97 active+clean; 38 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-04-26T01:56:56.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:56 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:56:56.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:56 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:56:56.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:56 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:56:56.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:56 smithi097 ceph-mon[32853]: pgmap v162: 97 pgs: 97 active+clean; 38 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-04-26T01:56:56.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:56 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:56:56.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:56 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:56:56.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:56 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:56:56.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:56 smithi173 ceph-mon[32553]: pgmap v162: 97 pgs: 97 active+clean; 38 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-04-26T01:56:56.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:56 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:56:56.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:56 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:56:56.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:56 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:56:58.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:56:58 smithi097 ceph-mon[32853]: pgmap v163: 97 pgs: 97 active+clean; 38 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1.4 MiB/s wr, 225 op/s 2024-04-26T01:56:58.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:56:58 smithi173 ceph-mon[32553]: pgmap v163: 97 pgs: 97 active+clean; 38 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1.4 MiB/s wr, 225 op/s 2024-04-26T01:56:58.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:56:58 smithi001 ceph-mon[27822]: pgmap v163: 97 pgs: 97 active+clean; 38 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1.4 MiB/s wr, 225 op/s 2024-04-26T01:57:00.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:00 smithi097 ceph-mon[32853]: pgmap v164: 97 pgs: 97 active+clean; 38 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1.4 MiB/s wr, 225 op/s 2024-04-26T01:57:00.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:00 smithi173 ceph-mon[32553]: pgmap v164: 97 pgs: 97 active+clean; 38 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1.4 MiB/s wr, 225 op/s 2024-04-26T01:57:00.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:00 smithi001 ceph-mon[27822]: pgmap v164: 97 pgs: 97 active+clean; 38 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1.4 MiB/s wr, 225 op/s 2024-04-26T01:57:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:02 smithi097 ceph-mon[32853]: pgmap v165: 97 pgs: 97 active+clean; 39 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 324 op/s 2024-04-26T01:57:02.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:02 smithi173 ceph-mon[32553]: pgmap v165: 97 pgs: 97 active+clean; 39 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 324 op/s 2024-04-26T01:57:02.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:02 smithi001 ceph-mon[27822]: pgmap v165: 97 pgs: 97 active+clean; 39 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 324 op/s 2024-04-26T01:57:04.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:04 smithi097 ceph-mon[32853]: pgmap v166: 97 pgs: 97 active+clean; 39 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.2 MiB/s wr, 213 op/s 2024-04-26T01:57:04.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:04 smithi173 ceph-mon[32553]: pgmap v166: 97 pgs: 97 active+clean; 39 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.2 MiB/s wr, 213 op/s 2024-04-26T01:57:04.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:04 smithi001 ceph-mon[27822]: pgmap v166: 97 pgs: 97 active+clean; 39 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.2 MiB/s wr, 213 op/s 2024-04-26T01:57:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.990+0000 7f1442000640 1 -- 172.21.15.1:0/2187282610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f143c101880 msgr2=0x7f143c1058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.990+0000 7f1442000640 1 --2- 172.21.15.1:0/2187282610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f143c101880 0x7f143c1058c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1428009860 tx=0x7f142802a340 comp rx=0 tx=0).stop 2024-04-26T01:57:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.990+0000 7f1442000640 1 -- 172.21.15.1:0/2187282610 shutdown_connections 2024-04-26T01:57:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.990+0000 7f1442000640 1 --2- 172.21.15.1:0/2187282610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f143c105fa0 0x7f143c10db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.990+0000 7f1442000640 1 --2- 172.21.15.1:0/2187282610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f143c101880 0x7f143c1058c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.990+0000 7f1442000640 1 --2- 172.21.15.1:0/2187282610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f143c100ed0 0x7f143c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.990+0000 7f1442000640 1 -- 172.21.15.1:0/2187282610 >> 172.21.15.1:0/2187282610 conn(0x7f143c0fc730 msgr2=0x7f143c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.990+0000 7f1442000640 1 -- 172.21.15.1:0/2187282610 shutdown_connections 2024-04-26T01:57:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.990+0000 7f1442000640 1 -- 172.21.15.1:0/2187282610 wait complete. 2024-04-26T01:57:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.992+0000 7f1442000640 1 Processor -- start 2024-04-26T01:57:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.992+0000 7f1442000640 1 -- start start 2024-04-26T01:57:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.992+0000 7f1442000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f143c100ed0 0x7f143c19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.992+0000 7f1442000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f143c105fa0 0x7f143c19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.992+0000 7f1442000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f143c19f020 0x7f143c19f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.992+0000 7f1442000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f143c111fe0 con 0x7f143c105fa0 2024-04-26T01:57:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.992+0000 7f1442000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f143c111e60 con 0x7f143c19f020 2024-04-26T01:57:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.992+0000 7f1442000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f143c112160 con 0x7f143c100ed0 2024-04-26T01:57:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.993+0000 7f1440c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f143c100ed0 0x7f143c19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.993+0000 7f143be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f143c105fa0 0x7f143c19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.993+0000 7f1440c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f143c100ed0 0x7f143c19a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:34872/0 (socket says 172.21.15.1:34872) 2024-04-26T01:57:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.993+0000 7f1440c00640 1 -- 172.21.15.1:0/3708728482 learned_addr learned my addr 172.21.15.1:0/3708728482 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.993+0000 7f1441600640 1 --2- 172.21.15.1:0/3708728482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f143c19f020 0x7f143c19f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.993+0000 7f1440c00640 1 -- 172.21.15.1:0/3708728482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f143c19f020 msgr2=0x7f143c19f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.993+0000 7f1440c00640 1 --2- 172.21.15.1:0/3708728482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f143c19f020 0x7f143c19f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.993+0000 7f1440c00640 1 -- 172.21.15.1:0/3708728482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f143c105fa0 msgr2=0x7f143c19aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.993+0000 7f1440c00640 1 --2- 172.21.15.1:0/3708728482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f143c105fa0 0x7f143c19aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.993+0000 7f1440c00640 1 -- 172.21.15.1:0/3708728482 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143c19fb10 con 0x7f143c100ed0 2024-04-26T01:57:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.994+0000 7f1440c00640 1 --2- 172.21.15.1:0/3708728482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f143c100ed0 0x7f143c19a760 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f143000e9c0 tx=0x7f143000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.994+0000 7f1439600640 1 -- 172.21.15.1:0/3708728482 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143000cde0 con 0x7f143c100ed0 2024-04-26T01:57:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.994+0000 7f1439600640 1 -- 172.21.15.1:0/3708728482 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14300035f0 con 0x7f143c100ed0 2024-04-26T01:57:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.994+0000 7f1439600640 1 -- 172.21.15.1:0/3708728482 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1430010690 con 0x7f143c100ed0 2024-04-26T01:57:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.994+0000 7f1442000640 1 -- 172.21.15.1:0/3708728482 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f143c19fe00 con 0x7f143c100ed0 2024-04-26T01:57:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.994+0000 7f1442000640 1 -- 172.21.15.1:0/3708728482 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f143c1a7610 con 0x7f143c100ed0 2024-04-26T01:57:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.999+0000 7f1446375740 1 Processor -- start 2024-04-26T01:57:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.999+0000 7f1446375740 1 -- start start 2024-04-26T01:57:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.999+0000 7f1439600640 1 -- 172.21.15.1:0/3708728482 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f1430004000 con 0x7f143c100ed0 2024-04-26T01:57:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.999+0000 7f1446375740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad1059b0 0x5572ad105d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.999+0000 7f1446375740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad1062d0 0x5572ad110650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:04.999+0000 7f1446375740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad110b90 0x5572ad112f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.000+0000 7f1446375740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5572ad06b900 con 0x5572ad110b90 2024-04-26T01:57:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.000+0000 7f1446375740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5572ad011ef0 con 0x5572ad1059b0 2024-04-26T01:57:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.000+0000 7f1446375740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5572ad0678c0 con 0x5572ad1062d0 2024-04-26T01:57:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.000+0000 7f1440c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad1062d0 0x5572ad110650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.000+0000 7f1441600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad1059b0 0x5572ad105d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.000+0000 7f1440c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad1062d0 0x5572ad110650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:34878/0 (socket says 172.21.15.1:34878) 2024-04-26T01:57:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.000+0000 7f1440c00640 1 -- 172.21.15.1:0/2959727101 learned_addr learned my addr 172.21.15.1:0/2959727101 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.000+0000 7f143be00640 1 --2- 172.21.15.1:0/2959727101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad110b90 0x5572ad112f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.000+0000 7f1441600640 1 -- 172.21.15.1:0/2959727101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad1062d0 msgr2=0x5572ad110650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.000+0000 7f1441600640 1 --2- 172.21.15.1:0/2959727101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad1062d0 0x5572ad110650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.000+0000 7f1441600640 1 -- 172.21.15.1:0/2959727101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad110b90 msgr2=0x5572ad112f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.000+0000 7f1441600640 1 --2- 172.21.15.1:0/2959727101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad110b90 0x5572ad112f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.000+0000 7f1441600640 1 -- 172.21.15.1:0/2959727101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572acd7e210 con 0x5572ad1059b0 2024-04-26T01:57:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.001+0000 7f1441600640 1 --2- 172.21.15.1:0/2959727101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad1059b0 0x5572ad105d90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1434009e50 tx=0x7f14340107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95561f59bcf282be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.001+0000 7f142dc00640 1 -- 172.21.15.1:0/2959727101 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1434003b70 con 0x5572ad1059b0 2024-04-26T01:57:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.001+0000 7f142dc00640 1 -- 172.21.15.1:0/2959727101 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1434015070 con 0x5572ad1059b0 2024-04-26T01:57:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.001+0000 7f142dc00640 1 -- 172.21.15.1:0/2959727101 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14340116e0 con 0x5572ad1059b0 2024-04-26T01:57:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.001+0000 7f1439600640 1 --2- 172.21.15.1:0/3708728482 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f14240a3d80 0x7f14240a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.001+0000 7f1439600640 1 -- 172.21.15.1:0/3708728482 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f14300d0b50 con 0x7f143c100ed0 2024-04-26T01:57:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.001+0000 7f1440c00640 1 --2- 172.21.15.1:0/3708728482 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f14240a3d80 0x7f14240a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.002+0000 7f1446375740 1 -- 172.21.15.1:0/2959727101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad1059b0 msgr2=0x5572ad105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.002+0000 7f1446375740 1 --2- 172.21.15.1:0/2959727101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad1059b0 0x5572ad105d90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1434009e50 tx=0x7f14340107a0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.002+0000 7f1446375740 1 -- 172.21.15.1:0/2959727101 shutdown_connections 2024-04-26T01:57:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.002+0000 7f1446375740 1 --2- 172.21.15.1:0/2959727101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad110b90 0x5572ad112f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.002+0000 7f1446375740 1 --2- 172.21.15.1:0/2959727101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad1062d0 0x5572ad110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.002+0000 7f1446375740 1 --2- 172.21.15.1:0/2959727101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad1059b0 0x5572ad105d90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.002+0000 7f1446375740 1 -- 172.21.15.1:0/2959727101 >> 172.21.15.1:0/2959727101 conn(0x5572ad101ec0 msgr2=0x5572ad1022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.002+0000 7f1446375740 1 -- 172.21.15.1:0/2959727101 shutdown_connections 2024-04-26T01:57:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.002+0000 7f1446375740 1 -- 172.21.15.1:0/2959727101 wait complete. 2024-04-26T01:57:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.002+0000 7f1440c00640 1 --2- 172.21.15.1:0/3708728482 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f14240a3d80 0x7f14240a6240 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f14300ce300 tx=0x7f14300d46a0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.003+0000 7f1446375740 1 -- start start 2024-04-26T01:57:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.003+0000 7f1446375740 1 Processor -- start 2024-04-26T01:57:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.004+0000 7f1446375740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad1059b0 0x5572ad10db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.004+0000 7f1446375740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad1062d0 0x5572ad10e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.004+0000 7f1446375740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad110b90 0x5572ad10e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.004+0000 7f1446375740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5572ad1137c0 con 0x5572ad1062d0 2024-04-26T01:57:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.004+0000 7f1446375740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5572ad025310 con 0x5572ad110b90 2024-04-26T01:57:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.004+0000 7f1446375740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5572ad113940 con 0x5572ad1059b0 2024-04-26T01:57:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.004+0000 7f1446375740 10 client.0 _ll_register_callbacks cb 0x5572acff3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.004+0000 7f1441600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad1059b0 0x5572ad10db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.004+0000 7f1441600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad1059b0 0x5572ad10db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:34882/0 (socket says 172.21.15.1:34882) 2024-04-26T01:57:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.004+0000 7f1441600640 1 -- 172.21.15.1:0/2648402987 learned_addr learned my addr 172.21.15.1:0/2648402987 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.004+0000 7f143be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad1062d0 0x5572ad10e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.004+0000 7f1441600640 1 --2- 172.21.15.1:0/2648402987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad110b90 0x5572ad10e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.004+0000 7f143be00640 1 -- 172.21.15.1:0/2648402987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad1059b0 msgr2=0x5572ad10db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.004+0000 7f143be00640 1 --2- 172.21.15.1:0/2648402987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad1059b0 0x5572ad10db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.004+0000 7f143be00640 1 -- 172.21.15.1:0/2648402987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad110b90 msgr2=0x5572ad10e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.005+0000 7f143be00640 1 --2- 172.21.15.1:0/2648402987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad110b90 0x5572ad10e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.005+0000 7f143be00640 1 -- 172.21.15.1:0/2648402987 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572ace4a620 con 0x5572ad1062d0 2024-04-26T01:57:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.005+0000 7f1441600640 1 --2- 172.21.15.1:0/2648402987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad110b90 0x5572ad10e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:57:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.005+0000 7f143be00640 1 --2- 172.21.15.1:0/2648402987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad1062d0 0x5572ad10e060 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1428009780 tx=0x7f1428004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2727741e01d39622 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.005+0000 7f1422000640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:57:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.005+0000 7f1422000640 1 -- 172.21.15.1:0/2648402987 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1428041070 con 0x5572ad1062d0 2024-04-26T01:57:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.005+0000 7f1422000640 1 -- 172.21.15.1:0/2648402987 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f1428030450 con 0x5572ad1062d0 2024-04-26T01:57:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.005+0000 7f1446375740 1 -- 172.21.15.1:0/2648402987 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1434015070 con 0x5572ad1062d0 2024-04-26T01:57:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.006+0000 7f1422000640 1 -- 172.21.15.1:0/2648402987 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f142804d050 con 0x5572ad1062d0 2024-04-26T01:57:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.006+0000 7f1446375740 10 client.15666 fetch_fsmap learned FSMap version 32 2024-04-26T01:57:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.006+0000 7f1446375740 1 -- 172.21.15.1:0/2648402987 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5572ad11fa20 con 0x5572ad1062d0 2024-04-26T01:57:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.006+0000 7f1422000640 1 -- 172.21.15.1:0/2648402987 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1434015070 con 0x5572ad1062d0 2024-04-26T01:57:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.006+0000 7f1422000640 1 -- 172.21.15.1:0/2648402987 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f1428003cd0 con 0x5572ad1062d0 2024-04-26T01:57:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.006+0000 7f1446375740 10 client.15666 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:57:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.006+0000 7f1446375740 10 client.15666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:57:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.006+0000 7f1446375740 20 client.15666 populate_metadata read hostname 'smithi001' 2024-04-26T01:57:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.006+0000 7f1446375740 1 --2- 172.21.15.1:0/2648402987 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5572ad120ee0 0x5572ad1232d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.006+0000 7f1446375740 4 client.15666 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:57:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.006+0000 7f1446375740 1 -- 172.21.15.1:0/2648402987 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572ad07f8b0 con 0x5572ad120ee0 2024-04-26T01:57:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.007+0000 7f1441600640 1 --2- 172.21.15.1:0/2648402987 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5572ad120ee0 0x5572ad1232d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.007+0000 7f1441600640 1 --2- 172.21.15.1:0/2648402987 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5572ad120ee0 0x5572ad1232d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f91fd10b4b4f714 server_cookie=7273da4cde343ee7 in_seq=0 out_seq=0 2024-04-26T01:57:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.007+0000 7f1422000640 10 client.15666 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.008+0000 7f1422000640 1 -- 172.21.15.1:0/2648402987 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5572ad07f8b0 con 0x5572ad120ee0 2024-04-26T01:57:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.008+0000 7f1422000640 10 client.15666 handle_command_reply: tid=0 2024-04-26T01:57:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.008+0000 7f1446375740 1 client.15666 shutdown 2024-04-26T01:57:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.008+0000 7f1446375740 1 -- 172.21.15.1:0/2648402987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad1062d0 msgr2=0x5572ad10e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.009+0000 7f1446375740 1 --2- 172.21.15.1:0/2648402987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad1062d0 0x5572ad10e060 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1428009780 tx=0x7f1428004270 comp rx=0 tx=0).stop 2024-04-26T01:57:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.009+0000 7f1446375740 1 -- 172.21.15.1:0/2648402987 shutdown_connections 2024-04-26T01:57:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.009+0000 7f1446375740 1 --2- 172.21.15.1:0/2648402987 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5572ad120ee0 0x5572ad1232d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.009+0000 7f1446375740 1 --2- 172.21.15.1:0/2648402987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad110b90 0x5572ad10e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.009+0000 7f1446375740 1 --2- 172.21.15.1:0/2648402987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad1062d0 0x5572ad10e060 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.009+0000 7f1422000640 0 client.15666 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.009+0000 7f1446375740 1 --2- 172.21.15.1:0/2648402987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad1059b0 0x5572ad10db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.009+0000 7f1446375740 1 -- 172.21.15.1:0/2648402987 >> 172.21.15.1:0/2648402987 conn(0x5572ad101ec0 msgr2=0x5572ad107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.009+0000 7f1446375740 1 -- 172.21.15.1:0/2648402987 shutdown_connections 2024-04-26T01:57:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.009+0000 7f1446375740 1 -- 172.21.15.1:0/2648402987 wait complete. 2024-04-26T01:57:05.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.009+0000 7f1446375740 20 client.15666 trim_cache size 0 max 16384 2024-04-26T01:57:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.029+0000 7f1446375740 1 Processor -- start 2024-04-26T01:57:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.029+0000 7f1446375740 1 -- start start 2024-04-26T01:57:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.030+0000 7f1446375740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad1059b0 0x5572ad11c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.030+0000 7f1446375740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad1062d0 0x5572ad108370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.030+0000 7f1446375740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad110b90 0x5572ad128be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.030+0000 7f1446375740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f143c1b2590 con 0x5572ad110b90 2024-04-26T01:57:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.030+0000 7f1446375740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f143c1afe60 con 0x5572ad1059b0 2024-04-26T01:57:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.030+0000 7f1446375740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f143c1adcb0 con 0x5572ad1062d0 2024-04-26T01:57:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.030+0000 7f143be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad1062d0 0x5572ad108370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.030+0000 7f143be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad1062d0 0x5572ad108370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:34890/0 (socket says 172.21.15.1:34890) 2024-04-26T01:57:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.030+0000 7f143be00640 1 -- 172.21.15.1:0/2062322103 learned_addr learned my addr 172.21.15.1:0/2062322103 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.030+0000 7f1441600640 1 --2- 172.21.15.1:0/2062322103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad1059b0 0x5572ad11c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.030+0000 7f1441600640 1 --2- 172.21.15.1:0/2062322103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad110b90 0x5572ad128be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.030+0000 7f143be00640 1 -- 172.21.15.1:0/2062322103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad1059b0 msgr2=0x5572ad11c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.030+0000 7f143be00640 1 --2- 172.21.15.1:0/2062322103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad1059b0 0x5572ad11c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.030+0000 7f143be00640 1 -- 172.21.15.1:0/2062322103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad110b90 msgr2=0x5572ad128be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.030+0000 7f143be00640 1 --2- 172.21.15.1:0/2062322103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad110b90 0x5572ad128be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.030+0000 7f143be00640 1 -- 172.21.15.1:0/2062322103 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572ad123640 con 0x5572ad1062d0 2024-04-26T01:57:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.030+0000 7f1441600640 1 --2- 172.21.15.1:0/2062322103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad1059b0 0x5572ad11c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:57:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.031+0000 7f143be00640 1 --2- 172.21.15.1:0/2062322103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad1062d0 0x5572ad108370 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f14280303f0 tx=0x7f142802a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9501d86004441023 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.031+0000 7f142d200640 1 -- 172.21.15.1:0/2062322103 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1428056070 con 0x5572ad1062d0 2024-04-26T01:57:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.031+0000 7f142d200640 1 -- 172.21.15.1:0/2062322103 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1428009450 con 0x5572ad1062d0 2024-04-26T01:57:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.031+0000 7f142d200640 1 -- 172.21.15.1:0/2062322103 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1428041040 con 0x5572ad1062d0 2024-04-26T01:57:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.032+0000 7f1446375740 1 -- 172.21.15.1:0/2062322103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad1062d0 msgr2=0x5572ad108370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.032+0000 7f1446375740 1 --2- 172.21.15.1:0/2062322103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad1062d0 0x5572ad108370 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f14280303f0 tx=0x7f142802a340 comp rx=0 tx=0).stop 2024-04-26T01:57:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.032+0000 7f1446375740 1 -- 172.21.15.1:0/2062322103 shutdown_connections 2024-04-26T01:57:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.032+0000 7f1446375740 1 --2- 172.21.15.1:0/2062322103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad110b90 0x5572ad128be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.032+0000 7f1446375740 1 --2- 172.21.15.1:0/2062322103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad1062d0 0x5572ad108370 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.032+0000 7f1446375740 1 --2- 172.21.15.1:0/2062322103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad1059b0 0x5572ad11c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.032+0000 7f1446375740 1 -- 172.21.15.1:0/2062322103 >> 172.21.15.1:0/2062322103 conn(0x5572ad101ec0 msgr2=0x5572ad10e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.032+0000 7f1446375740 1 -- 172.21.15.1:0/2062322103 shutdown_connections 2024-04-26T01:57:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.032+0000 7f1446375740 1 -- 172.21.15.1:0/2062322103 wait complete. 2024-04-26T01:57:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.033+0000 7f1446375740 1 -- start start 2024-04-26T01:57:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.033+0000 7f1446375740 1 Processor -- start 2024-04-26T01:57:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.033+0000 7f1446375740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad1059b0 0x5572ad1165c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.033+0000 7f1446375740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad1062d0 0x5572ad116b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.033+0000 7f1446375740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad110b90 0x5572ad117040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.033+0000 7f1446375740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f143c1ae320 con 0x5572ad1059b0 2024-04-26T01:57:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.033+0000 7f1446375740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f143c1af790 con 0x5572ad1062d0 2024-04-26T01:57:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.033+0000 7f1446375740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f143c1ae9f0 con 0x5572ad110b90 2024-04-26T01:57:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.033+0000 7f1446375740 10 client.0 _ll_register_callbacks cb 0x7f14280096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.034+0000 7f1441600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad1059b0 0x5572ad1165c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.034+0000 7f1441600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad1059b0 0x5572ad1165c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45514/0 (socket says 172.21.15.1:45514) 2024-04-26T01:57:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.034+0000 7f1441600640 1 -- 172.21.15.1:0/1756676525 learned_addr learned my addr 172.21.15.1:0/1756676525 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.034+0000 7f143be00640 1 --2- 172.21.15.1:0/1756676525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad1062d0 0x5572ad116b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.034+0000 7f1441600640 1 --2- 172.21.15.1:0/1756676525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad110b90 0x5572ad117040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.034+0000 7f143be00640 1 -- 172.21.15.1:0/1756676525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad110b90 msgr2=0x5572ad117040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.034+0000 7f143be00640 1 --2- 172.21.15.1:0/1756676525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad110b90 0x5572ad117040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.034+0000 7f143be00640 1 -- 172.21.15.1:0/1756676525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad1059b0 msgr2=0x5572ad1165c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.034+0000 7f143be00640 1 --2- 172.21.15.1:0/1756676525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad1059b0 0x5572ad1165c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.034+0000 7f143be00640 1 -- 172.21.15.1:0/1756676525 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572ad186460 con 0x5572ad1062d0 2024-04-26T01:57:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.034+0000 7f143be00640 1 --2- 172.21.15.1:0/1756676525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad1062d0 0x5572ad116b00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x5572ad10ebe0 tx=0x7f142802aac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90ab2ad36866fb93 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.034+0000 7f1422000640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:57:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.035+0000 7f1422000640 1 -- 172.21.15.1:0/1756676525 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1428056070 con 0x5572ad1062d0 2024-04-26T01:57:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.035+0000 7f1422000640 1 -- 172.21.15.1:0/1756676525 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f14280304f0 con 0x5572ad1062d0 2024-04-26T01:57:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.035+0000 7f1446375740 1 -- 172.21.15.1:0/1756676525 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1428009450 con 0x5572ad1062d0 2024-04-26T01:57:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.035+0000 7f1422000640 1 -- 172.21.15.1:0/1756676525 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1428041070 con 0x5572ad1062d0 2024-04-26T01:57:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.035+0000 7f1446375740 10 client.25814 fetch_fsmap learned FSMap version 32 2024-04-26T01:57:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.035+0000 7f1446375740 1 -- 172.21.15.1:0/1756676525 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5572ad187710 con 0x5572ad1062d0 2024-04-26T01:57:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.035+0000 7f1422000640 1 -- 172.21.15.1:0/1756676525 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1428009450 con 0x5572ad1062d0 2024-04-26T01:57:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.036+0000 7f1422000640 1 -- 172.21.15.1:0/1756676525 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f1428003c60 con 0x5572ad1062d0 2024-04-26T01:57:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.036+0000 7f1446375740 10 client.25814 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:57:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.036+0000 7f1446375740 10 client.25814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:57:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.036+0000 7f1446375740 20 client.25814 populate_metadata read hostname 'smithi001' 2024-04-26T01:57:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.036+0000 7f1446375740 1 --2- 172.21.15.1:0/1756676525 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5572ad187d70 0x5572ad18a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.036+0000 7f1446375740 4 client.25814 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:57:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.036+0000 7f1446375740 1 -- 172.21.15.1:0/1756676525 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5572ad079320 con 0x5572ad187d70 2024-04-26T01:57:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.036+0000 7f1441600640 1 --2- 172.21.15.1:0/1756676525 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5572ad187d70 0x5572ad18a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.036+0000 7f1441600640 1 --2- 172.21.15.1:0/1756676525 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5572ad187d70 0x5572ad18a190 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=742ec26c030336a3 server_cookie=1031174ecd1e7c19 in_seq=0 out_seq=0 2024-04-26T01:57:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.036+0000 7f1422000640 10 client.25814 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.052+0000 7f1422000640 1 -- 172.21.15.1:0/1756676525 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5572ad079320 con 0x5572ad187d70 2024-04-26T01:57:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.052+0000 7f1422000640 10 client.25814 handle_command_reply: tid=0 2024-04-26T01:57:05.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.052+0000 7f1446375740 1 client.25814 shutdown 2024-04-26T01:57:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.052+0000 7f1446375740 1 -- 172.21.15.1:0/1756676525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad1062d0 msgr2=0x5572ad116b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.052+0000 7f1446375740 1 --2- 172.21.15.1:0/1756676525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad1062d0 0x5572ad116b00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x5572ad10ebe0 tx=0x7f142802aac0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.052+0000 7f1446375740 1 -- 172.21.15.1:0/1756676525 shutdown_connections 2024-04-26T01:57:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.052+0000 7f1446375740 1 --2- 172.21.15.1:0/1756676525 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5572ad187d70 0x5572ad18a190 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.052+0000 7f1446375740 1 --2- 172.21.15.1:0/1756676525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5572ad110b90 0x5572ad117040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.052+0000 7f1446375740 1 --2- 172.21.15.1:0/1756676525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572ad1062d0 0x5572ad116b00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.052+0000 7f1446375740 1 --2- 172.21.15.1:0/1756676525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572ad1059b0 0x5572ad1165c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.052+0000 7f1446375740 1 -- 172.21.15.1:0/1756676525 >> 172.21.15.1:0/1756676525 conn(0x5572ad101ec0 msgr2=0x5572ad100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.052+0000 7f1422000640 0 client.25814 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.053+0000 7f1446375740 1 -- 172.21.15.1:0/1756676525 shutdown_connections 2024-04-26T01:57:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.053+0000 7f1446375740 1 -- 172.21.15.1:0/1756676525 wait complete. 2024-04-26T01:57:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.053+0000 7f1446375740 20 client.25814 trim_cache size 0 max 16384 2024-04-26T01:57:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:57:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:57:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:57:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:57:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.054+0000 7f1442000640 1 -- 172.21.15.1:0/3708728482 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f14240a3d80 msgr2=0x7f14240a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.054+0000 7f1442000640 1 --2- 172.21.15.1:0/3708728482 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f14240a3d80 0x7f14240a6240 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f14300ce300 tx=0x7f14300d46a0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.054+0000 7f1442000640 1 -- 172.21.15.1:0/3708728482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f143c100ed0 msgr2=0x7f143c19a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.054+0000 7f1442000640 1 --2- 172.21.15.1:0/3708728482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f143c100ed0 0x7f143c19a760 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f143000e9c0 tx=0x7f143000ee80 comp rx=0 tx=0).stop 2024-04-26T01:57:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.054+0000 7f1442000640 1 -- 172.21.15.1:0/3708728482 shutdown_connections 2024-04-26T01:57:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.054+0000 7f1442000640 1 --2- 172.21.15.1:0/3708728482 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f14240a3d80 0x7f14240a6240 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.054+0000 7f1442000640 1 --2- 172.21.15.1:0/3708728482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f143c19f020 0x7f143c19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.054+0000 7f1442000640 1 --2- 172.21.15.1:0/3708728482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f143c105fa0 0x7f143c19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.054+0000 7f1442000640 1 --2- 172.21.15.1:0/3708728482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f143c100ed0 0x7f143c19a760 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.054+0000 7f1442000640 1 -- 172.21.15.1:0/3708728482 >> 172.21.15.1:0/3708728482 conn(0x7f143c0fc730 msgr2=0x7f143c0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.054+0000 7f1442000640 1 -- 172.21.15.1:0/3708728482 shutdown_connections 2024-04-26T01:57:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:05.054+0000 7f1442000640 1 -- 172.21.15.1:0/3708728482 wait complete. 2024-04-26T01:57:06.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:06 smithi097 ceph-mon[32853]: pgmap v167: 97 pgs: 97 active+clean; 42 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 318 op/s 2024-04-26T01:57:06.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:06 smithi173 ceph-mon[32553]: pgmap v167: 97 pgs: 97 active+clean; 42 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 318 op/s 2024-04-26T01:57:06.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:06 smithi001 ceph-mon[27822]: pgmap v167: 97 pgs: 97 active+clean; 42 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 318 op/s 2024-04-26T01:57:07.039 INFO:tasks.workunit.client.0.smithi001.stdout:50000 copy from 0x898b8 to 0x92015, (0x875d bytes) at 0xb142c 2024-04-26T01:57:07.039 INFO:tasks.workunit.client.0.smithi001.stdout:All 50000 operations completed A-OK! 2024-04-26T01:57:07.045 INFO:tasks.workunit.client.0.smithi001.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2024-04-26T01:57:07.048 INFO:tasks.workunit.client.0.smithi001.stdout:Seed set to 1 2024-04-26T01:57:07.215 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-26T01:57:07.215 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-26T01:57:07.218 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-26T01:57:07.219 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-26T01:57:07.219 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support clone range, disabling! 2024-04-26T01:57:07.224 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-26T01:57:07.228 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero length punch hole 2024-04-26T01:57:07.228 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero length punch hole 2024-04-26T01:57:07.228 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero length punch hole 2024-04-26T01:57:07.228 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero size read 2024-04-26T01:57:07.228 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support exchange range, disabling! 2024-04-26T01:57:07.229 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero length punch hole 2024-04-26T01:57:07.229 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero size read 2024-04-26T01:57:07.229 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero length punch hole 2024-04-26T01:57:07.229 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x4310e9 2024-04-26T01:57:07.241 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0x954a45 2024-04-26T01:57:07.303 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x9bc582 2024-04-26T01:57:07.355 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x9eecbc 2024-04-26T01:57:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:08 smithi097 ceph-mon[32853]: pgmap v168: 97 pgs: 97 active+clean; 42 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 204 op/s 2024-04-26T01:57:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:08 smithi173 ceph-mon[32553]: pgmap v168: 97 pgs: 97 active+clean; 42 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 204 op/s 2024-04-26T01:57:08.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:08 smithi001 ceph-mon[27822]: pgmap v168: 97 pgs: 97 active+clean; 42 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 204 op/s 2024-04-26T01:57:10.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:10 smithi097 ceph-mon[32853]: pgmap v169: 97 pgs: 97 active+clean; 42 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 204 op/s 2024-04-26T01:57:10.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:10 smithi173 ceph-mon[32553]: pgmap v169: 97 pgs: 97 active+clean; 42 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 204 op/s 2024-04-26T01:57:10.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:10 smithi001 ceph-mon[27822]: pgmap v169: 97 pgs: 97 active+clean; 42 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 204 op/s 2024-04-26T01:57:12.087 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0x9fdd84 2024-04-26T01:57:12.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:12 smithi097 ceph-mon[32853]: pgmap v170: 97 pgs: 97 active+clean; 44 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 325 op/s 2024-04-26T01:57:12.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:12 smithi173 ceph-mon[32553]: pgmap v170: 97 pgs: 97 active+clean; 44 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 325 op/s 2024-04-26T01:57:12.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:12 smithi001 ceph-mon[27822]: pgmap v170: 97 pgs: 97 active+clean; 44 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 325 op/s 2024-04-26T01:57:14.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:14 smithi097 ceph-mon[32853]: pgmap v171: 97 pgs: 97 active+clean; 44 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 226 op/s 2024-04-26T01:57:14.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:14 smithi173 ceph-mon[32553]: pgmap v171: 97 pgs: 97 active+clean; 44 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 226 op/s 2024-04-26T01:57:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:14 smithi001 ceph-mon[27822]: pgmap v171: 97 pgs: 97 active+clean; 44 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 226 op/s 2024-04-26T01:57:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:16 smithi097 ceph-mon[32853]: pgmap v172: 97 pgs: 97 active+clean; 49 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 372 op/s 2024-04-26T01:57:16.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:16 smithi173 ceph-mon[32553]: pgmap v172: 97 pgs: 97 active+clean; 49 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 372 op/s 2024-04-26T01:57:16.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:16 smithi001 ceph-mon[27822]: pgmap v172: 97 pgs: 97 active+clean; 49 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 372 op/s 2024-04-26T01:57:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.218+0000 7f53e0a00640 1 -- 172.21.15.1:0/1291380922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53dc1031c0 msgr2=0x7f53dc1035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.218+0000 7f53e0a00640 1 --2- 172.21.15.1:0/1291380922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53dc1031c0 0x7f53dc1035a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009ca0 tx=0x7f53d402a950 comp rx=0 tx=0).stop 2024-04-26T01:57:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.218+0000 7f53e0a00640 1 -- 172.21.15.1:0/1291380922 shutdown_connections 2024-04-26T01:57:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.218+0000 7f53e0a00640 1 --2- 172.21.15.1:0/1291380922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53dc10c590 0x7f53dc10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.218+0000 7f53e0a00640 1 --2- 172.21.15.1:0/1291380922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53dc103ae0 0x7f53dc10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.219+0000 7f53e0a00640 1 --2- 172.21.15.1:0/1291380922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53dc1031c0 0x7f53dc1035a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.219+0000 7f53e0a00640 1 -- 172.21.15.1:0/1291380922 >> 172.21.15.1:0/1291380922 conn(0x7f53dc077f70 msgr2=0x7f53dc078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.219+0000 7f53e0a00640 1 -- 172.21.15.1:0/1291380922 shutdown_connections 2024-04-26T01:57:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.219+0000 7f53e0a00640 1 -- 172.21.15.1:0/1291380922 wait complete. 2024-04-26T01:57:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.220+0000 7f53e0a00640 1 Processor -- start 2024-04-26T01:57:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.220+0000 7f53e0a00640 1 -- start start 2024-04-26T01:57:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.221+0000 7f53e0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53dc1031c0 0x7f53dc1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.221+0000 7f53e0a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53dc103ae0 0x7f53dc1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.221+0000 7f53e0a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53dc10c590 0x7f53dc19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.221+0000 7f53e0a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc114440 con 0x7f53dc103ae0 2024-04-26T01:57:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.221+0000 7f53e0a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1142c0 con 0x7f53dc1031c0 2024-04-26T01:57:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.221+0000 7f53e0a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1145c0 con 0x7f53dc10c590 2024-04-26T01:57:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.221+0000 7f53d9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53dc1031c0 0x7f53dc1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.221+0000 7f53d9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53dc1031c0 0x7f53dc1a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:43730/0 (socket says 172.21.15.1:43730) 2024-04-26T01:57:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.221+0000 7f53d9e00640 1 -- 172.21.15.1:0/2385203954 learned_addr learned my addr 172.21.15.1:0/2385203954 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.221+0000 7f53da800640 1 --2- 172.21.15.1:0/2385203954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53dc10c590 0x7f53dc19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.221+0000 7f53d9e00640 1 -- 172.21.15.1:0/2385203954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53dc10c590 msgr2=0x7f53dc19a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.221+0000 7f53d9e00640 1 --2- 172.21.15.1:0/2385203954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53dc10c590 0x7f53dc19a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.221+0000 7f53d9e00640 1 -- 172.21.15.1:0/2385203954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53dc103ae0 msgr2=0x7f53dc1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.221+0000 7f53d9400640 1 --2- 172.21.15.1:0/2385203954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53dc103ae0 0x7f53dc1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.221+0000 7f53d9e00640 1 --2- 172.21.15.1:0/2385203954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53dc103ae0 0x7f53dc1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.221+0000 7f53d9e00640 1 -- 172.21.15.1:0/2385203954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53dc19afb0 con 0x7f53dc1031c0 2024-04-26T01:57:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.222+0000 7f53d9400640 1 --2- 172.21.15.1:0/2385203954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53dc103ae0 0x7f53dc1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:57:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.222+0000 7f53d9e00640 1 --2- 172.21.15.1:0/2385203954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53dc1031c0 0x7f53dc1a0670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f53d4005e30 tx=0x7f53d4003d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.222+0000 7f53d2a00640 1 -- 172.21.15.1:0/2385203954 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d4034040 con 0x7f53dc1031c0 2024-04-26T01:57:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.222+0000 7f53d2a00640 1 -- 172.21.15.1:0/2385203954 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53d4030850 con 0x7f53dc1031c0 2024-04-26T01:57:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.222+0000 7f53d2a00640 1 -- 172.21.15.1:0/2385203954 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d40046e0 con 0x7f53dc1031c0 2024-04-26T01:57:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.222+0000 7f53e0a00640 1 -- 172.21.15.1:0/2385203954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53dc19b240 con 0x7f53dc1031c0 2024-04-26T01:57:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.222+0000 7f53e0a00640 1 -- 172.21.15.1:0/2385203954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53dc1a74c0 con 0x7f53dc1031c0 2024-04-26T01:57:18.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.226+0000 7f53d2a00640 1 -- 172.21.15.1:0/2385203954 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f53d4037020 con 0x7f53dc1031c0 2024-04-26T01:57:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.226+0000 7f53e3860740 1 Processor -- start 2024-04-26T01:57:18.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.226+0000 7f53e3860740 1 -- start start 2024-04-26T01:57:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.227+0000 7f53e3860740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6f4820 0x55760e6f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.227+0000 7f53e3860740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f5140 0x55760e6ff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.227+0000 7f53e3860740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55760e6ffa00 0x55760e701df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.227+0000 7f53e3860740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55760e65a900 con 0x55760e6ffa00 2024-04-26T01:57:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.227+0000 7f53e3860740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55760e600ef0 con 0x55760e6f4820 2024-04-26T01:57:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.227+0000 7f53e3860740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55760e6568c0 con 0x55760e6f5140 2024-04-26T01:57:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.227+0000 7f53da800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6f4820 0x55760e6f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.227+0000 7f53da800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6f4820 0x55760e6f4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:43734/0 (socket says 172.21.15.1:43734) 2024-04-26T01:57:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.227+0000 7f53da800640 1 -- 172.21.15.1:0/3071561950 learned_addr learned my addr 172.21.15.1:0/3071561950 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:18.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.227+0000 7f53d9400640 1 --2- 172.21.15.1:0/3071561950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55760e6ffa00 0x55760e701df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.227+0000 7f53d9e00640 1 --2- 172.21.15.1:0/3071561950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f5140 0x55760e6ff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.227+0000 7f53da800640 1 -- 172.21.15.1:0/3071561950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f5140 msgr2=0x55760e6ff4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.227+0000 7f53da800640 1 --2- 172.21.15.1:0/3071561950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f5140 0x55760e6ff4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.227+0000 7f53da800640 1 -- 172.21.15.1:0/3071561950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55760e6ffa00 msgr2=0x55760e701df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.228+0000 7f53da800640 1 --2- 172.21.15.1:0/3071561950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55760e6ffa00 0x55760e701df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.228+0000 7f53da800640 1 -- 172.21.15.1:0/3071561950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55760e36d210 con 0x55760e6f4820 2024-04-26T01:57:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.228+0000 7f53da800640 1 --2- 172.21.15.1:0/3071561950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6f4820 0x55760e6f4c00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009ba0 tx=0x7f53cc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf4b3bf56b715a96 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.228+0000 7f53d2a00640 1 --2- 172.21.15.1:0/2385203954 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f53c00a3d90 0x7f53c00a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.228+0000 7f53cb000640 1 -- 172.21.15.1:0/3071561950 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc004230 con 0x55760e6f4820 2024-04-26T01:57:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.228+0000 7f53cb000640 1 -- 172.21.15.1:0/3071561950 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53cc009e10 con 0x55760e6f4820 2024-04-26T01:57:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.228+0000 7f53cb000640 1 -- 172.21.15.1:0/3071561950 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc004d00 con 0x55760e6f4820 2024-04-26T01:57:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.229+0000 7f53d2a00640 1 -- 172.21.15.1:0/2385203954 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f53d40f15b0 con 0x7f53dc1031c0 2024-04-26T01:57:18.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.229+0000 7f53d9e00640 1 --2- 172.21.15.1:0/2385203954 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f53c00a3d90 0x7f53c00a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:18.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.229+0000 7f53e3860740 1 -- 172.21.15.1:0/3071561950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6f4820 msgr2=0x55760e6f4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:18.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.229+0000 7f53e3860740 1 --2- 172.21.15.1:0/3071561950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6f4820 0x55760e6f4c00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009ba0 tx=0x7f53cc011810 comp rx=0 tx=0).stop 2024-04-26T01:57:18.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.229+0000 7f53e3860740 1 -- 172.21.15.1:0/3071561950 shutdown_connections 2024-04-26T01:57:18.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.229+0000 7f53e3860740 1 --2- 172.21.15.1:0/3071561950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55760e6ffa00 0x55760e701df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.229+0000 7f53e3860740 1 --2- 172.21.15.1:0/3071561950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f5140 0x55760e6ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.229+0000 7f53e3860740 1 --2- 172.21.15.1:0/3071561950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6f4820 0x55760e6f4c00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.229+0000 7f53e3860740 1 -- 172.21.15.1:0/3071561950 >> 172.21.15.1:0/3071561950 conn(0x55760e6f0d30 msgr2=0x55760e6f1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.229+0000 7f53e3860740 1 -- 172.21.15.1:0/3071561950 shutdown_connections 2024-04-26T01:57:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.230+0000 7f53d9e00640 1 --2- 172.21.15.1:0/2385203954 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f53c00a3d90 0x7f53c00a6250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55760e6f5dc0 tx=0x7f53d402a950 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.230+0000 7f53e3860740 1 -- 172.21.15.1:0/3071561950 wait complete. 2024-04-26T01:57:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.230+0000 7f53e3860740 1 -- start start 2024-04-26T01:57:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.230+0000 7f53e3860740 1 Processor -- start 2024-04-26T01:57:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.231+0000 7f53e3860740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f4820 0x55760e6fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.231+0000 7f53e3860740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6f5140 0x55760e6fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.231+0000 7f53e3860740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55760e6ffa00 0x55760e6fd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.231+0000 7f53e3860740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55760e702630 con 0x55760e6ffa00 2024-04-26T01:57:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.231+0000 7f53e3860740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55760e614310 con 0x55760e6f5140 2024-04-26T01:57:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.231+0000 7f53e3860740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55760e7027b0 con 0x55760e6f4820 2024-04-26T01:57:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.231+0000 7f53e3860740 10 client.0 _ll_register_callbacks cb 0x55760e5e2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.231+0000 7f53d9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6f5140 0x55760e6fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.231+0000 7f53d9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6f5140 0x55760e6fcef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:43736/0 (socket says 172.21.15.1:43736) 2024-04-26T01:57:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.231+0000 7f53d9400640 1 -- 172.21.15.1:0/2540675285 learned_addr learned my addr 172.21.15.1:0/2540675285 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.231+0000 7f53da800640 1 --2- 172.21.15.1:0/2540675285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f4820 0x55760e6fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.231+0000 7f53da800640 1 --2- 172.21.15.1:0/2540675285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55760e6ffa00 0x55760e6fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.232+0000 7f53d9400640 1 -- 172.21.15.1:0/2540675285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f4820 msgr2=0x55760e6fc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.232+0000 7f53d9400640 1 --2- 172.21.15.1:0/2540675285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f4820 0x55760e6fc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.232+0000 7f53d9400640 1 -- 172.21.15.1:0/2540675285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55760e6ffa00 msgr2=0x55760e6fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.232+0000 7f53d9400640 1 --2- 172.21.15.1:0/2540675285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55760e6ffa00 0x55760e6fd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.232+0000 7f53d9400640 1 -- 172.21.15.1:0/2540675285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55760e439620 con 0x55760e6f5140 2024-04-26T01:57:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.232+0000 7f53da800640 1 --2- 172.21.15.1:0/2540675285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55760e6ffa00 0x55760e6fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:57:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.232+0000 7f53d9400640 1 --2- 172.21.15.1:0/2540675285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6f5140 0x55760e6fcef0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f53c4012470 tx=0x7f53c4012930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f961ffba3c342d5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.232+0000 7f53bf400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:57:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.232+0000 7f53bf400640 1 -- 172.21.15.1:0/2540675285 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c4004440 con 0x55760e6f5140 2024-04-26T01:57:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.232+0000 7f53bf400640 1 -- 172.21.15.1:0/2540675285 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f53c40035f0 con 0x55760e6f5140 2024-04-26T01:57:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.233+0000 7f53e3860740 1 -- 172.21.15.1:0/2540675285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53cc009e10 con 0x55760e6f5140 2024-04-26T01:57:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.233+0000 7f53bf400640 1 -- 172.21.15.1:0/2540675285 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f53c40037f0 con 0x55760e6f5140 2024-04-26T01:57:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.233+0000 7f53e3860740 10 client.25826 fetch_fsmap learned FSMap version 32 2024-04-26T01:57:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.233+0000 7f53e3860740 1 -- 172.21.15.1:0/2540675285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55760e70e980 con 0x55760e6f5140 2024-04-26T01:57:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.233+0000 7f53bf400640 1 -- 172.21.15.1:0/2540675285 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53cc009e10 con 0x55760e6f5140 2024-04-26T01:57:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.233+0000 7f53bf400640 1 -- 172.21.15.1:0/2540675285 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f53c400f040 con 0x55760e6f5140 2024-04-26T01:57:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.234+0000 7f53e3860740 10 client.25826 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:57:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.234+0000 7f53e3860740 10 client.25826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:57:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.234+0000 7f53e3860740 20 client.25826 populate_metadata read hostname 'smithi001' 2024-04-26T01:57:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.234+0000 7f53e3860740 1 --2- 172.21.15.1:0/2540675285 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55760e70fe90 0x55760e712280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.234+0000 7f53e3860740 4 client.25826 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:57:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.234+0000 7f53e3860740 1 -- 172.21.15.1:0/2540675285 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55760e66e8b0 con 0x55760e70fe90 2024-04-26T01:57:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.234+0000 7f53da800640 1 --2- 172.21.15.1:0/2540675285 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55760e70fe90 0x55760e712280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.235+0000 7f53da800640 1 --2- 172.21.15.1:0/2540675285 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55760e70fe90 0x55760e712280 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c40b9dab1d0d6e03 server_cookie=7b56b9e888357d77 in_seq=0 out_seq=0 2024-04-26T01:57:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.235+0000 7f53bf400640 10 client.25826 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.235+0000 7f53bf400640 1 -- 172.21.15.1:0/2540675285 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55760e66e8b0 con 0x55760e70fe90 2024-04-26T01:57:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.235+0000 7f53bf400640 10 client.25826 handle_command_reply: tid=0 2024-04-26T01:57:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.235+0000 7f53e3860740 1 client.25826 shutdown 2024-04-26T01:57:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.236+0000 7f53e3860740 1 -- 172.21.15.1:0/2540675285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6f5140 msgr2=0x55760e6fcef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.236+0000 7f53e3860740 1 --2- 172.21.15.1:0/2540675285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6f5140 0x55760e6fcef0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f53c4012470 tx=0x7f53c4012930 comp rx=0 tx=0).stop 2024-04-26T01:57:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.236+0000 7f53e3860740 1 -- 172.21.15.1:0/2540675285 shutdown_connections 2024-04-26T01:57:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.236+0000 7f53e3860740 1 --2- 172.21.15.1:0/2540675285 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55760e70fe90 0x55760e712280 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.236+0000 7f53e3860740 1 --2- 172.21.15.1:0/2540675285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55760e6ffa00 0x55760e6fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.236+0000 7f53e3860740 1 --2- 172.21.15.1:0/2540675285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6f5140 0x55760e6fcef0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.236+0000 7f53bf400640 0 client.25826 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.236+0000 7f53e3860740 1 --2- 172.21.15.1:0/2540675285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f4820 0x55760e6fc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.236+0000 7f53e3860740 1 -- 172.21.15.1:0/2540675285 >> 172.21.15.1:0/2540675285 conn(0x55760e6f0d30 msgr2=0x55760e6f6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.236+0000 7f53e3860740 1 -- 172.21.15.1:0/2540675285 shutdown_connections 2024-04-26T01:57:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.236+0000 7f53e3860740 1 -- 172.21.15.1:0/2540675285 wait complete. 2024-04-26T01:57:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.236+0000 7f53e3860740 20 client.25826 trim_cache size 0 max 16384 2024-04-26T01:57:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.255+0000 7f53e3860740 1 Processor -- start 2024-04-26T01:57:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.255+0000 7f53e3860740 1 -- start start 2024-04-26T01:57:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.255+0000 7f53e3860740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f4820 0x55760e70b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.255+0000 7f53e3860740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6f5140 0x55760e6efd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.255+0000 7f53e3860740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55760e6ffa00 0x55760e717b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.255+0000 7f53e3860740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1b2660 con 0x55760e6ffa00 2024-04-26T01:57:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.255+0000 7f53e3860740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1aff30 con 0x55760e6f5140 2024-04-26T01:57:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.255+0000 7f53e3860740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1add80 con 0x55760e6f4820 2024-04-26T01:57:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.256+0000 7f53d9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6f5140 0x55760e6efd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.256+0000 7f53d9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6f5140 0x55760e6efd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:43744/0 (socket says 172.21.15.1:43744) 2024-04-26T01:57:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.256+0000 7f53d9400640 1 -- 172.21.15.1:0/56147282 learned_addr learned my addr 172.21.15.1:0/56147282 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.256+0000 7f53da800640 1 --2- 172.21.15.1:0/56147282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f4820 0x55760e70b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.256+0000 7f53da800640 1 --2- 172.21.15.1:0/56147282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55760e6ffa00 0x55760e717b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.256+0000 7f53d9400640 1 -- 172.21.15.1:0/56147282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f4820 msgr2=0x55760e70b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.256+0000 7f53d9400640 1 --2- 172.21.15.1:0/56147282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f4820 0x55760e70b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.256+0000 7f53d9400640 1 -- 172.21.15.1:0/56147282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55760e6ffa00 msgr2=0x55760e717b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.256+0000 7f53d9400640 1 --2- 172.21.15.1:0/56147282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55760e6ffa00 0x55760e717b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.256+0000 7f53d9400640 1 -- 172.21.15.1:0/56147282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55760e7125f0 con 0x55760e6f5140 2024-04-26T01:57:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.256+0000 7f53d9400640 1 --2- 172.21.15.1:0/56147282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6f5140 0x55760e6efd10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f53c4010d20 tx=0x7f53c4003670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd7418c32c3f0921 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.257+0000 7f53ca600640 1 -- 172.21.15.1:0/56147282 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c402a070 con 0x55760e6f5140 2024-04-26T01:57:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.257+0000 7f53ca600640 1 -- 172.21.15.1:0/56147282 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53c40101a0 con 0x55760e6f5140 2024-04-26T01:57:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.257+0000 7f53ca600640 1 -- 172.21.15.1:0/56147282 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c4003e90 con 0x55760e6f5140 2024-04-26T01:57:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.258+0000 7f53e3860740 1 -- 172.21.15.1:0/56147282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6f5140 msgr2=0x55760e6efd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.258+0000 7f53e3860740 1 --2- 172.21.15.1:0/56147282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6f5140 0x55760e6efd10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f53c4010d20 tx=0x7f53c4003670 comp rx=0 tx=0).stop 2024-04-26T01:57:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.258+0000 7f53e3860740 1 -- 172.21.15.1:0/56147282 shutdown_connections 2024-04-26T01:57:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.258+0000 7f53e3860740 1 --2- 172.21.15.1:0/56147282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55760e6ffa00 0x55760e717b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.258+0000 7f53e3860740 1 --2- 172.21.15.1:0/56147282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6f5140 0x55760e6efd10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.258+0000 7f53e3860740 1 --2- 172.21.15.1:0/56147282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f4820 0x55760e70b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.258+0000 7f53e3860740 1 -- 172.21.15.1:0/56147282 >> 172.21.15.1:0/56147282 conn(0x55760e6f0d30 msgr2=0x55760e6ef600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.258+0000 7f53e3860740 1 -- 172.21.15.1:0/56147282 shutdown_connections 2024-04-26T01:57:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.258+0000 7f53e3860740 1 -- 172.21.15.1:0/56147282 wait complete. 2024-04-26T01:57:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.258+0000 7f53e3860740 1 -- start start 2024-04-26T01:57:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.259+0000 7f53e3860740 1 Processor -- start 2024-04-26T01:57:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.259+0000 7f53e3860740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f4820 0x55760e707fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.259+0000 7f53e3860740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55760e6f5140 0x55760e705340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.259+0000 7f53e3860740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6ffa00 0x55760e705a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.259+0000 7f53e3860740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1ae3f0 con 0x55760e6f5140 2024-04-26T01:57:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.259+0000 7f53e3860740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1af860 con 0x55760e6ffa00 2024-04-26T01:57:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.259+0000 7f53e3860740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc1aeac0 con 0x55760e6f4820 2024-04-26T01:57:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.259+0000 7f53e3860740 10 client.0 _ll_register_callbacks cb 0x7f53c4009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.260+0000 7f53da800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f4820 0x55760e707fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.260+0000 7f53da800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f4820 0x55760e707fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:52956/0 (socket says 172.21.15.1:52956) 2024-04-26T01:57:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.260+0000 7f53da800640 1 -- 172.21.15.1:0/3424752693 learned_addr learned my addr 172.21.15.1:0/3424752693 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.260+0000 7f53da800640 1 --2- 172.21.15.1:0/3424752693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6ffa00 0x55760e705a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.260+0000 7f53da800640 1 -- 172.21.15.1:0/3424752693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6ffa00 msgr2=0x55760e705a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.260+0000 7f53da800640 1 --2- 172.21.15.1:0/3424752693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6ffa00 0x55760e705a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.260+0000 7f53da800640 1 -- 172.21.15.1:0/3424752693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55760e6f5140 msgr2=0x55760e705340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.260+0000 7f53da800640 1 --2- 172.21.15.1:0/3424752693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55760e6f5140 0x55760e705340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.260+0000 7f53da800640 1 -- 172.21.15.1:0/3424752693 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55760e77dbe0 con 0x55760e6f4820 2024-04-26T01:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.261+0000 7f53da800640 1 --2- 172.21.15.1:0/3424752693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f4820 0x55760e707fb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0058c0 tx=0x7f53cc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8439d96f52fefa42 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.261+0000 7f53bf400640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.261+0000 7f53bf400640 1 -- 172.21.15.1:0/3424752693 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc004380 con 0x55760e6f4820 2024-04-26T01:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.261+0000 7f53bf400640 1 -- 172.21.15.1:0/3424752693 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f53cc01e040 con 0x55760e6f4820 2024-04-26T01:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.261+0000 7f53e3860740 1 -- 172.21.15.1:0/3424752693 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53c40101a0 con 0x55760e6f4820 2024-04-26T01:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.261+0000 7f53bf400640 1 -- 172.21.15.1:0/3424752693 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f53cc03b050 con 0x55760e6f4820 2024-04-26T01:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.261+0000 7f53e3860740 10 client.25855 fetch_fsmap learned FSMap version 32 2024-04-26T01:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.261+0000 7f53e3860740 1 -- 172.21.15.1:0/3424752693 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55760e77ee10 con 0x55760e6f4820 2024-04-26T01:57:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.262+0000 7f53bf400640 1 -- 172.21.15.1:0/3424752693 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53c40101a0 con 0x55760e6f4820 2024-04-26T01:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.262+0000 7f53bf400640 1 -- 172.21.15.1:0/3424752693 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f53cc024040 con 0x55760e6f4820 2024-04-26T01:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.262+0000 7f53e3860740 10 client.25855 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.262+0000 7f53e3860740 10 client.25855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.262+0000 7f53e3860740 20 client.25855 populate_metadata read hostname 'smithi001' 2024-04-26T01:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.262+0000 7f53e3860740 1 --2- 172.21.15.1:0/3424752693 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55760e77f480 0x55760e7818c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.262+0000 7f53e3860740 4 client.25855 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.262+0000 7f53e3860740 1 -- 172.21.15.1:0/3424752693 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55760e668320 con 0x55760e77f480 2024-04-26T01:57:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.262+0000 7f53d9400640 1 --2- 172.21.15.1:0/3424752693 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55760e77f480 0x55760e7818c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.263+0000 7f53d9400640 1 --2- 172.21.15.1:0/3424752693 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55760e77f480 0x55760e7818c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f41d332d81be01c server_cookie=f59c842816e279a2 in_seq=0 out_seq=0 2024-04-26T01:57:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.263+0000 7f53bf400640 10 client.25855 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.279+0000 7f53bf400640 1 -- 172.21.15.1:0/3424752693 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55760e668320 con 0x55760e77f480 2024-04-26T01:57:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.279+0000 7f53bf400640 10 client.25855 handle_command_reply: tid=0 2024-04-26T01:57:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.279+0000 7f53e3860740 1 client.25855 shutdown 2024-04-26T01:57:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.279+0000 7f53e3860740 1 -- 172.21.15.1:0/3424752693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f4820 msgr2=0x55760e707fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.279+0000 7f53e3860740 1 --2- 172.21.15.1:0/3424752693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f4820 0x55760e707fb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0058c0 tx=0x7f53cc011810 comp rx=0 tx=0).stop 2024-04-26T01:57:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.279+0000 7f53e3860740 1 -- 172.21.15.1:0/3424752693 shutdown_connections 2024-04-26T01:57:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.279+0000 7f53e3860740 1 --2- 172.21.15.1:0/3424752693 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55760e77f480 0x55760e7818c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.280+0000 7f53e3860740 1 --2- 172.21.15.1:0/3424752693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55760e6ffa00 0x55760e705a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.280+0000 7f53e3860740 1 --2- 172.21.15.1:0/3424752693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55760e6f5140 0x55760e705340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.280+0000 7f53e3860740 1 --2- 172.21.15.1:0/3424752693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55760e6f4820 0x55760e707fb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.280+0000 7f53e3860740 1 -- 172.21.15.1:0/3424752693 >> 172.21.15.1:0/3424752693 conn(0x55760e6f0d30 msgr2=0x55760e6efe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.280+0000 7f53bf400640 0 client.25855 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.280+0000 7f53e3860740 1 -- 172.21.15.1:0/3424752693 shutdown_connections 2024-04-26T01:57:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.280+0000 7f53e3860740 1 -- 172.21.15.1:0/3424752693 wait complete. 2024-04-26T01:57:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.280+0000 7f53e3860740 20 client.25855 trim_cache size 0 max 16384 2024-04-26T01:57:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:57:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:57:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:57:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:57:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.281+0000 7f53e0a00640 1 -- 172.21.15.1:0/2385203954 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f53c00a3d90 msgr2=0x7f53c00a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.281+0000 7f53e0a00640 1 --2- 172.21.15.1:0/2385203954 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f53c00a3d90 0x7f53c00a6250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55760e6f5dc0 tx=0x7f53d402a950 comp rx=0 tx=0).stop 2024-04-26T01:57:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.281+0000 7f53e0a00640 1 -- 172.21.15.1:0/2385203954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53dc1031c0 msgr2=0x7f53dc1a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.281+0000 7f53e0a00640 1 --2- 172.21.15.1:0/2385203954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53dc1031c0 0x7f53dc1a0670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f53d4005e30 tx=0x7f53d4003d60 comp rx=0 tx=0).stop 2024-04-26T01:57:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.281+0000 7f53e0a00640 1 -- 172.21.15.1:0/2385203954 shutdown_connections 2024-04-26T01:57:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.281+0000 7f53e0a00640 1 --2- 172.21.15.1:0/2385203954 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f53c00a3d90 0x7f53c00a6250 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.281+0000 7f53e0a00640 1 --2- 172.21.15.1:0/2385203954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53dc10c590 0x7f53dc19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.281+0000 7f53e0a00640 1 --2- 172.21.15.1:0/2385203954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53dc103ae0 0x7f53dc1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.281+0000 7f53e0a00640 1 --2- 172.21.15.1:0/2385203954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53dc1031c0 0x7f53dc1a0670 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.281+0000 7f53e0a00640 1 -- 172.21.15.1:0/2385203954 >> 172.21.15.1:0/2385203954 conn(0x7f53dc077f70 msgr2=0x7f53dc0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.282+0000 7f53e0a00640 1 -- 172.21.15.1:0/2385203954 shutdown_connections 2024-04-26T01:57:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:18.283+0000 7f53e0a00640 1 -- 172.21.15.1:0/2385203954 wait complete. 2024-04-26T01:57:18.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:18 smithi097 ceph-mon[32853]: pgmap v173: 97 pgs: 97 active+clean; 49 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 267 op/s 2024-04-26T01:57:18.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:18 smithi173 ceph-mon[32553]: pgmap v173: 97 pgs: 97 active+clean; 49 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 267 op/s 2024-04-26T01:57:18.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:18 smithi001 ceph-mon[27822]: pgmap v173: 97 pgs: 97 active+clean; 49 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 267 op/s 2024-04-26T01:57:20.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:20 smithi097 ceph-mon[32853]: pgmap v174: 97 pgs: 97 active+clean; 49 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 267 op/s 2024-04-26T01:57:20.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:20 smithi173 ceph-mon[32553]: pgmap v174: 97 pgs: 97 active+clean; 49 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 267 op/s 2024-04-26T01:57:20.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:20 smithi001 ceph-mon[27822]: pgmap v174: 97 pgs: 97 active+clean; 49 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 267 op/s 2024-04-26T01:57:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:22 smithi097 ceph-mon[32853]: pgmap v175: 97 pgs: 97 active+clean; 40 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.3 MiB/s wr, 412 op/s 2024-04-26T01:57:22.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:22 smithi173 ceph-mon[32553]: pgmap v175: 97 pgs: 97 active+clean; 40 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.3 MiB/s wr, 412 op/s 2024-04-26T01:57:22.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:22 smithi001 ceph-mon[27822]: pgmap v175: 97 pgs: 97 active+clean; 40 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.3 MiB/s wr, 412 op/s 2024-04-26T01:57:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:24 smithi097 ceph-mon[32853]: pgmap v176: 97 pgs: 97 active+clean; 40 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.6 MiB/s wr, 290 op/s 2024-04-26T01:57:24.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:24 smithi173 ceph-mon[32553]: pgmap v176: 97 pgs: 97 active+clean; 40 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.6 MiB/s wr, 290 op/s 2024-04-26T01:57:24.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:24 smithi001 ceph-mon[27822]: pgmap v176: 97 pgs: 97 active+clean; 40 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.6 MiB/s wr, 290 op/s 2024-04-26T01:57:26.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:26 smithi097 ceph-mon[32853]: pgmap v177: 97 pgs: 97 active+clean; 41 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.5 MiB/s wr, 425 op/s 2024-04-26T01:57:26.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:26 smithi173 ceph-mon[32553]: pgmap v177: 97 pgs: 97 active+clean; 41 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.5 MiB/s wr, 425 op/s 2024-04-26T01:57:26.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:26 smithi001 ceph-mon[27822]: pgmap v177: 97 pgs: 97 active+clean; 41 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.5 MiB/s wr, 425 op/s 2024-04-26T01:57:28.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:28 smithi097 ceph-mon[32853]: pgmap v178: 97 pgs: 97 active+clean; 41 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.7 MiB/s wr, 280 op/s 2024-04-26T01:57:28.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:28 smithi173 ceph-mon[32553]: pgmap v178: 97 pgs: 97 active+clean; 41 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.7 MiB/s wr, 280 op/s 2024-04-26T01:57:28.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:28 smithi001 ceph-mon[27822]: pgmap v178: 97 pgs: 97 active+clean; 41 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.7 MiB/s wr, 280 op/s 2024-04-26T01:57:30.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:30 smithi097 ceph-mon[32853]: pgmap v179: 97 pgs: 97 active+clean; 41 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.7 MiB/s wr, 280 op/s 2024-04-26T01:57:30.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:30 smithi173 ceph-mon[32553]: pgmap v179: 97 pgs: 97 active+clean; 41 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.7 MiB/s wr, 280 op/s 2024-04-26T01:57:30.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:30 smithi001 ceph-mon[27822]: pgmap v179: 97 pgs: 97 active+clean; 41 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.7 MiB/s wr, 280 op/s 2024-04-26T01:57:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.439+0000 7fe436200640 1 -- 172.21.15.1:0/307976460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe43010c330 msgr2=0x7fe43010e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.439+0000 7fe436200640 1 --2- 172.21.15.1:0/307976460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe43010c330 0x7fe43010e7d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe428009ca0 tx=0x7fe42802a940 comp rx=0 tx=0).stop 2024-04-26T01:57:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.439+0000 7fe436200640 1 -- 172.21.15.1:0/307976460 shutdown_connections 2024-04-26T01:57:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.439+0000 7fe436200640 1 --2- 172.21.15.1:0/307976460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe43010c330 0x7fe43010e7d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.439+0000 7fe436200640 1 --2- 172.21.15.1:0/307976460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe430103830 0x7fe43010bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.439+0000 7fe436200640 1 --2- 172.21.15.1:0/307976460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe430102f10 0x7fe4301032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.439+0000 7fe436200640 1 -- 172.21.15.1:0/307976460 >> 172.21.15.1:0/307976460 conn(0x7fe4300fc770 msgr2=0x7fe4300feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.439+0000 7fe436200640 1 -- 172.21.15.1:0/307976460 shutdown_connections 2024-04-26T01:57:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.440+0000 7fe436200640 1 -- 172.21.15.1:0/307976460 wait complete. 2024-04-26T01:57:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.441+0000 7fe436200640 1 Processor -- start 2024-04-26T01:57:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.441+0000 7fe436200640 1 -- start start 2024-04-26T01:57:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.442+0000 7fe436200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe430102f10 0x7fe430100b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.442+0000 7fe436200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe430103830 0x7fe4301010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.442+0000 7fe436200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe43010c330 0x7fe4301015e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.442+0000 7fe436200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe430112010 con 0x7fe430102f10 2024-04-26T01:57:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.442+0000 7fe436200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe430111e90 con 0x7fe430103830 2024-04-26T01:57:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.442+0000 7fe436200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe430112190 con 0x7fe43010c330 2024-04-26T01:57:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.442+0000 7fe434e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe430102f10 0x7fe430100b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.442+0000 7fe42fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe430103830 0x7fe4301010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.442+0000 7fe434e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe430102f10 0x7fe430100b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46440/0 (socket says 172.21.15.1:46440) 2024-04-26T01:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.442+0000 7fe42fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe430103830 0x7fe4301010a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:34290/0 (socket says 172.21.15.1:34290) 2024-04-26T01:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.442+0000 7fe434e00640 1 -- 172.21.15.1:0/3143736920 learned_addr learned my addr 172.21.15.1:0/3143736920 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.442+0000 7fe435800640 1 --2- 172.21.15.1:0/3143736920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe43010c330 0x7fe4301015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.443+0000 7fe42fe00640 1 -- 172.21.15.1:0/3143736920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe43010c330 msgr2=0x7fe4301015e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.443+0000 7fe42fe00640 1 --2- 172.21.15.1:0/3143736920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe43010c330 0x7fe4301015e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.443+0000 7fe42fe00640 1 -- 172.21.15.1:0/3143736920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe430102f10 msgr2=0x7fe430100b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.443+0000 7fe42fe00640 1 --2- 172.21.15.1:0/3143736920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe430102f10 0x7fe430100b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.443+0000 7fe42fe00640 1 -- 172.21.15.1:0/3143736920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4301a6a10 con 0x7fe430103830 2024-04-26T01:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.443+0000 7fe435800640 1 --2- 172.21.15.1:0/3143736920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe43010c330 0x7fe4301015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.443+0000 7fe42fe00640 1 --2- 172.21.15.1:0/3143736920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe430103830 0x7fe4301010a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00e7c0 tx=0x7fe41c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.443+0000 7fe42d600640 1 -- 172.21.15.1:0/3143736920 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c009530 con 0x7fe430103830 2024-04-26T01:57:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.443+0000 7fe42d600640 1 -- 172.21.15.1:0/3143736920 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe41c00ccc0 con 0x7fe430103830 2024-04-26T01:57:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.443+0000 7fe42d600640 1 -- 172.21.15.1:0/3143736920 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c004a00 con 0x7fe430103830 2024-04-26T01:57:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.443+0000 7fe436200640 1 -- 172.21.15.1:0/3143736920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4301a6d00 con 0x7fe430103830 2024-04-26T01:57:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.443+0000 7fe436200640 1 -- 172.21.15.1:0/3143736920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4301a71f0 con 0x7fe430103830 2024-04-26T01:57:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.447+0000 7fe42d600640 1 -- 172.21.15.1:0/3143736920 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fe41c004ba0 con 0x7fe430103830 2024-04-26T01:57:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.447+0000 7fe43a61a740 1 Processor -- start 2024-04-26T01:57:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.447+0000 7fe43a61a740 1 -- start start 2024-04-26T01:57:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.448+0000 7fe43a61a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b5bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.448+0000 7fe43a61a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b5c2d0 0x55d0a4b66650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.448+0000 7fe43a61a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b66b90 0x55d0a4b68f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.448+0000 7fe43a61a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d0a4ac1900 con 0x55d0a4b66b90 2024-04-26T01:57:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.448+0000 7fe43a61a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d0a4a67ef0 con 0x55d0a4b5b9b0 2024-04-26T01:57:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.448+0000 7fe43a61a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d0a4abd8c0 con 0x55d0a4b5c2d0 2024-04-26T01:57:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.448+0000 7fe435800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b5bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.448+0000 7fe42fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b66b90 0x55d0a4b68f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.448+0000 7fe434e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b5c2d0 0x55d0a4b66650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.448+0000 7fe435800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b5bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:34300/0 (socket says 172.21.15.1:34300) 2024-04-26T01:57:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.448+0000 7fe435800640 1 -- 172.21.15.1:0/2725524541 learned_addr learned my addr 172.21.15.1:0/2725524541 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.449+0000 7fe434e00640 1 -- 172.21.15.1:0/2725524541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5b9b0 msgr2=0x55d0a4b5bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.449+0000 7fe434e00640 1 --2- 172.21.15.1:0/2725524541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b5bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.449+0000 7fe434e00640 1 -- 172.21.15.1:0/2725524541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b66b90 msgr2=0x55d0a4b68f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.449+0000 7fe434e00640 1 --2- 172.21.15.1:0/2725524541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b66b90 0x55d0a4b68f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.449+0000 7fe434e00640 1 -- 172.21.15.1:0/2725524541 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0a47d4210 con 0x55d0a4b5c2d0 2024-04-26T01:57:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.449+0000 7fe42d600640 1 --2- 172.21.15.1:0/3143736920 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe4180a3d80 0x7fe4180a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.449+0000 7fe434e00640 1 --2- 172.21.15.1:0/2725524541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b5c2d0 0x55d0a4b66650 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe424009c80 tx=0x7fe424011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc50d1948a9f04e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.449+0000 7fe435800640 1 --2- 172.21.15.1:0/3143736920 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe4180a3d80 0x7fe4180a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.450+0000 7fe42d600640 1 -- 172.21.15.1:0/3143736920 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fe41c0d05d0 con 0x7fe430103830 2024-04-26T01:57:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.450+0000 7fe422000640 1 -- 172.21.15.1:0/2725524541 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe424004230 con 0x55d0a4b5c2d0 2024-04-26T01:57:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.450+0000 7fe422000640 1 -- 172.21.15.1:0/2725524541 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4240043d0 con 0x55d0a4b5c2d0 2024-04-26T01:57:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.450+0000 7fe422000640 1 -- 172.21.15.1:0/2725524541 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe424004d80 con 0x55d0a4b5c2d0 2024-04-26T01:57:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.451+0000 7fe43a61a740 1 -- 172.21.15.1:0/2725524541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b5c2d0 msgr2=0x55d0a4b66650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.451+0000 7fe435800640 1 --2- 172.21.15.1:0/3143736920 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe4180a3d80 0x7fe4180a6240 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe428009a60 tx=0x7fe428005d80 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.451+0000 7fe43a61a740 1 --2- 172.21.15.1:0/2725524541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b5c2d0 0x55d0a4b66650 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe424009c80 tx=0x7fe424011810 comp rx=0 tx=0).stop 2024-04-26T01:57:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.451+0000 7fe43a61a740 1 -- 172.21.15.1:0/2725524541 shutdown_connections 2024-04-26T01:57:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.451+0000 7fe43a61a740 1 --2- 172.21.15.1:0/2725524541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b66b90 0x55d0a4b68f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.451+0000 7fe43a61a740 1 --2- 172.21.15.1:0/2725524541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b5c2d0 0x55d0a4b66650 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.451+0000 7fe43a61a740 1 --2- 172.21.15.1:0/2725524541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b5bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.451+0000 7fe43a61a740 1 -- 172.21.15.1:0/2725524541 >> 172.21.15.1:0/2725524541 conn(0x55d0a4b57ec0 msgr2=0x55d0a4b582a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.451+0000 7fe43a61a740 1 -- 172.21.15.1:0/2725524541 shutdown_connections 2024-04-26T01:57:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.451+0000 7fe43a61a740 1 -- 172.21.15.1:0/2725524541 wait complete. 2024-04-26T01:57:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.451+0000 7fe43a61a740 1 -- start start 2024-04-26T01:57:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.452+0000 7fe43a61a740 1 Processor -- start 2024-04-26T01:57:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.452+0000 7fe43a61a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b63a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.452+0000 7fe43a61a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5c2d0 0x55d0a4b63f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.452+0000 7fe43a61a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b66b90 0x55d0a4b644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.452+0000 7fe43a61a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d0a4b697c0 con 0x55d0a4b66b90 2024-04-26T01:57:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.452+0000 7fe43a61a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d0a4a7b310 con 0x55d0a4b5c2d0 2024-04-26T01:57:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.452+0000 7fe43a61a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d0a4b69940 con 0x55d0a4b5b9b0 2024-04-26T01:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.452+0000 7fe43a61a740 10 client.0 _ll_register_callbacks cb 0x55d0a4a49cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.452+0000 7fe434e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b63a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.452+0000 7fe434e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b63a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:42148/0 (socket says 172.21.15.1:42148) 2024-04-26T01:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.452+0000 7fe434e00640 1 -- 172.21.15.1:0/1949770138 learned_addr learned my addr 172.21.15.1:0/1949770138 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.452+0000 7fe42fe00640 1 --2- 172.21.15.1:0/1949770138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5c2d0 0x55d0a4b63f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.453+0000 7fe435800640 1 --2- 172.21.15.1:0/1949770138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b66b90 0x55d0a4b644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.453+0000 7fe434e00640 1 -- 172.21.15.1:0/1949770138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5c2d0 msgr2=0x55d0a4b63f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.453+0000 7fe434e00640 1 --2- 172.21.15.1:0/1949770138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5c2d0 0x55d0a4b63f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.453+0000 7fe434e00640 1 -- 172.21.15.1:0/1949770138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b66b90 msgr2=0x55d0a4b644a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.453+0000 7fe434e00640 1 --2- 172.21.15.1:0/1949770138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b66b90 0x55d0a4b644a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.453+0000 7fe434e00640 1 -- 172.21.15.1:0/1949770138 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0a48a0620 con 0x55d0a4b5b9b0 2024-04-26T01:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.453+0000 7fe435800640 1 --2- 172.21.15.1:0/1949770138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b66b90 0x55d0a4b644a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:57:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.453+0000 7fe434e00640 1 --2- 172.21.15.1:0/1949770138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b63a20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe424009620 tx=0x7fe424019a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1576f12111c5ee35 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.453+0000 7fe416a00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.453+0000 7fe416a00640 1 -- 172.21.15.1:0/1949770138 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe424028020 con 0x55d0a4b5b9b0 2024-04-26T01:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.454+0000 7fe416a00640 1 -- 172.21.15.1:0/1949770138 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fe424024620 con 0x55d0a4b5b9b0 2024-04-26T01:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.454+0000 7fe43a61a740 1 -- 172.21.15.1:0/1949770138 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4240043d0 con 0x55d0a4b5b9b0 2024-04-26T01:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.454+0000 7fe416a00640 1 -- 172.21.15.1:0/1949770138 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe424033050 con 0x55d0a4b5b9b0 2024-04-26T01:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.454+0000 7fe43a61a740 10 client.25867 fetch_fsmap learned FSMap version 32 2024-04-26T01:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.454+0000 7fe43a61a740 1 -- 172.21.15.1:0/1949770138 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d0a4b75960 con 0x55d0a4b5b9b0 2024-04-26T01:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.454+0000 7fe416a00640 1 -- 172.21.15.1:0/1949770138 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4240043d0 con 0x55d0a4b5b9b0 2024-04-26T01:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.454+0000 7fe416a00640 1 -- 172.21.15.1:0/1949770138 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fe424018070 con 0x55d0a4b5b9b0 2024-04-26T01:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.455+0000 7fe43a61a740 10 client.25867 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.455+0000 7fe43a61a740 10 client.25867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.455+0000 7fe43a61a740 20 client.25867 populate_metadata read hostname 'smithi001' 2024-04-26T01:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.455+0000 7fe43a61a740 1 --2- 172.21.15.1:0/1949770138 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d0a4b76d60 0x55d0a4b79150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.455+0000 7fe43a61a740 4 client.25867 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:57:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.455+0000 7fe43a61a740 1 -- 172.21.15.1:0/1949770138 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0a4ad58b0 con 0x55d0a4b76d60 2024-04-26T01:57:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.455+0000 7fe42fe00640 1 --2- 172.21.15.1:0/1949770138 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d0a4b76d60 0x55d0a4b79150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.455+0000 7fe42fe00640 1 --2- 172.21.15.1:0/1949770138 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d0a4b76d60 0x55d0a4b79150 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f67b294f45c16e72 server_cookie=a096f11fa9078385 in_seq=0 out_seq=0 2024-04-26T01:57:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.455+0000 7fe416a00640 10 client.25867 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.457+0000 7fe416a00640 1 -- 172.21.15.1:0/1949770138 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d0a4ad58b0 con 0x55d0a4b76d60 2024-04-26T01:57:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.457+0000 7fe416a00640 10 client.25867 handle_command_reply: tid=0 2024-04-26T01:57:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.457+0000 7fe43a61a740 1 client.25867 shutdown 2024-04-26T01:57:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.457+0000 7fe43a61a740 1 -- 172.21.15.1:0/1949770138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b5b9b0 msgr2=0x55d0a4b63a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.457+0000 7fe43a61a740 1 --2- 172.21.15.1:0/1949770138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b63a20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe424009620 tx=0x7fe424019a50 comp rx=0 tx=0).stop 2024-04-26T01:57:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.457+0000 7fe43a61a740 1 -- 172.21.15.1:0/1949770138 shutdown_connections 2024-04-26T01:57:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.457+0000 7fe43a61a740 1 --2- 172.21.15.1:0/1949770138 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d0a4b76d60 0x55d0a4b79150 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.457+0000 7fe43a61a740 1 --2- 172.21.15.1:0/1949770138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b66b90 0x55d0a4b644a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.457+0000 7fe43a61a740 1 --2- 172.21.15.1:0/1949770138 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5c2d0 0x55d0a4b63f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.457+0000 7fe43a61a740 1 --2- 172.21.15.1:0/1949770138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b63a20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.457+0000 7fe43a61a740 1 -- 172.21.15.1:0/1949770138 >> 172.21.15.1:0/1949770138 conn(0x55d0a4b57ec0 msgr2=0x55d0a4b5dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.457+0000 7fe416a00640 0 client.25867 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.458+0000 7fe43a61a740 1 -- 172.21.15.1:0/1949770138 shutdown_connections 2024-04-26T01:57:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.458+0000 7fe43a61a740 1 -- 172.21.15.1:0/1949770138 wait complete. 2024-04-26T01:57:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.458+0000 7fe43a61a740 20 client.25867 trim_cache size 0 max 16384 2024-04-26T01:57:31.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.475+0000 7fe43a61a740 1 Processor -- start 2024-04-26T01:57:31.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.475+0000 7fe43a61a740 1 -- start start 2024-04-26T01:57:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.475+0000 7fe43a61a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b72680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.475+0000 7fe43a61a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5c2d0 0x55d0a4b63300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.475+0000 7fe43a61a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b66b90 0x55d0a4b5e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.475+0000 7fe43a61a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe4301b2560 con 0x55d0a4b66b90 2024-04-26T01:57:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.475+0000 7fe43a61a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe4301afe30 con 0x55d0a4b5c2d0 2024-04-26T01:57:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.475+0000 7fe43a61a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe4301adc80 con 0x55d0a4b5b9b0 2024-04-26T01:57:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.476+0000 7fe434e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b72680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.476+0000 7fe42fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5c2d0 0x55d0a4b63300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.476+0000 7fe434e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b72680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:42156/0 (socket says 172.21.15.1:42156) 2024-04-26T01:57:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.476+0000 7fe434e00640 1 -- 172.21.15.1:0/2282621427 learned_addr learned my addr 172.21.15.1:0/2282621427 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.476+0000 7fe42fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5c2d0 0x55d0a4b63300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:34314/0 (socket says 172.21.15.1:34314) 2024-04-26T01:57:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.476+0000 7fe435800640 1 --2- 172.21.15.1:0/2282621427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b66b90 0x55d0a4b5e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:31.475 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.476+0000 7fe42fe00640 1 -- 172.21.15.1:0/2282621427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b5b9b0 msgr2=0x55d0a4b72680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.476+0000 7fe42fe00640 1 --2- 172.21.15.1:0/2282621427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b72680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.476+0000 7fe42fe00640 1 -- 172.21.15.1:0/2282621427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b66b90 msgr2=0x55d0a4b5e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.476+0000 7fe42fe00640 1 --2- 172.21.15.1:0/2282621427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b66b90 0x55d0a4b5e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.476+0000 7fe42fe00640 1 -- 172.21.15.1:0/2282621427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0a4b5e8b0 con 0x55d0a4b5c2d0 2024-04-26T01:57:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.476+0000 7fe434e00640 1 --2- 172.21.15.1:0/2282621427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b72680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.476+0000 7fe42fe00640 1 --2- 172.21.15.1:0/2282621427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5c2d0 0x55d0a4b63300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0cdcc0 tx=0x7fe41c0d2a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9103caad5bd1bef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.477+0000 7fe421600640 1 -- 172.21.15.1:0/2282621427 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c01c040 con 0x55d0a4b5c2d0 2024-04-26T01:57:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.477+0000 7fe421600640 1 -- 172.21.15.1:0/2282621427 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe41c005020 con 0x55d0a4b5c2d0 2024-04-26T01:57:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.477+0000 7fe421600640 1 -- 172.21.15.1:0/2282621427 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c09c510 con 0x55d0a4b5c2d0 2024-04-26T01:57:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.477+0000 7fe43a61a740 1 -- 172.21.15.1:0/2282621427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5c2d0 msgr2=0x55d0a4b63300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.477+0000 7fe43a61a740 1 --2- 172.21.15.1:0/2282621427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5c2d0 0x55d0a4b63300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0cdcc0 tx=0x7fe41c0d2a20 comp rx=0 tx=0).stop 2024-04-26T01:57:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.477+0000 7fe43a61a740 1 -- 172.21.15.1:0/2282621427 shutdown_connections 2024-04-26T01:57:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.477+0000 7fe43a61a740 1 --2- 172.21.15.1:0/2282621427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b66b90 0x55d0a4b5e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.477+0000 7fe43a61a740 1 --2- 172.21.15.1:0/2282621427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5c2d0 0x55d0a4b63300 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.478+0000 7fe43a61a740 1 --2- 172.21.15.1:0/2282621427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b72680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.478+0000 7fe43a61a740 1 -- 172.21.15.1:0/2282621427 >> 172.21.15.1:0/2282621427 conn(0x55d0a4b57ec0 msgr2=0x55d0a4b5edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.478+0000 7fe43a61a740 1 -- 172.21.15.1:0/2282621427 shutdown_connections 2024-04-26T01:57:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.478+0000 7fe43a61a740 1 -- 172.21.15.1:0/2282621427 wait complete. 2024-04-26T01:57:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.478+0000 7fe43a61a740 1 -- start start 2024-04-26T01:57:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.478+0000 7fe43a61a740 1 Processor -- start 2024-04-26T01:57:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.479+0000 7fe43a61a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b6bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.479+0000 7fe43a61a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5c2d0 0x55d0a4b6c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.479+0000 7fe43a61a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b66b90 0x55d0a4b6c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.479+0000 7fe43a61a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe4301ae2f0 con 0x55d0a4b5b9b0 2024-04-26T01:57:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.479+0000 7fe43a61a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe4301af760 con 0x55d0a4b5c2d0 2024-04-26T01:57:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.479+0000 7fe43a61a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe4301ae9c0 con 0x55d0a4b66b90 2024-04-26T01:57:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.479+0000 7fe43a61a740 10 client.0 _ll_register_callbacks cb 0x7fe424009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.479+0000 7fe434e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b6bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.479+0000 7fe434e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b6bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46476/0 (socket says 172.21.15.1:46476) 2024-04-26T01:57:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.479+0000 7fe434e00640 1 -- 172.21.15.1:0/615062668 learned_addr learned my addr 172.21.15.1:0/615062668 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.479+0000 7fe42fe00640 1 --2- 172.21.15.1:0/615062668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5c2d0 0x55d0a4b6c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.479+0000 7fe435800640 1 --2- 172.21.15.1:0/615062668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b66b90 0x55d0a4b6c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.479+0000 7fe434e00640 1 -- 172.21.15.1:0/615062668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b66b90 msgr2=0x55d0a4b6c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.479+0000 7fe434e00640 1 --2- 172.21.15.1:0/615062668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b66b90 0x55d0a4b6c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.479+0000 7fe434e00640 1 -- 172.21.15.1:0/615062668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5c2d0 msgr2=0x55d0a4b6c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.479+0000 7fe434e00640 1 --2- 172.21.15.1:0/615062668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5c2d0 0x55d0a4b6c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.479+0000 7fe434e00640 1 -- 172.21.15.1:0/615062668 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0a4bdb300 con 0x55d0a4b5b9b0 2024-04-26T01:57:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.480+0000 7fe434e00640 1 --2- 172.21.15.1:0/615062668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b6bb60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe4240059f0 tx=0x7fe424019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8676234d5269eba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.480+0000 7fe416a00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:57:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.480+0000 7fe416a00640 1 -- 172.21.15.1:0/615062668 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe424028040 con 0x55d0a4b5b9b0 2024-04-26T01:57:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.480+0000 7fe416a00640 1 -- 172.21.15.1:0/615062668 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fe424018800 con 0x55d0a4b5b9b0 2024-04-26T01:57:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.480+0000 7fe43a61a740 1 -- 172.21.15.1:0/615062668 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d0a4acf320 con 0x55d0a4b5b9b0 2024-04-26T01:57:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.480+0000 7fe416a00640 1 -- 172.21.15.1:0/615062668 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe424005c50 con 0x55d0a4b5b9b0 2024-04-26T01:57:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.480+0000 7fe43a61a740 10 client.15702 fetch_fsmap learned FSMap version 32 2024-04-26T01:57:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.480+0000 7fe43a61a740 1 -- 172.21.15.1:0/615062668 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d0a4bdc5d0 con 0x55d0a4b5b9b0 2024-04-26T01:57:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.481+0000 7fe416a00640 1 -- 172.21.15.1:0/615062668 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d0a4acf320 con 0x55d0a4b5b9b0 2024-04-26T01:57:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.481+0000 7fe416a00640 1 -- 172.21.15.1:0/615062668 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fe424018040 con 0x55d0a4b5b9b0 2024-04-26T01:57:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.481+0000 7fe43a61a740 10 client.15702 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:57:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.481+0000 7fe43a61a740 10 client.15702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:57:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.481+0000 7fe43a61a740 20 client.15702 populate_metadata read hostname 'smithi001' 2024-04-26T01:57:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.481+0000 7fe43a61a740 1 --2- 172.21.15.1:0/615062668 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d0a4bdcc00 0x55d0a4bdf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.481+0000 7fe43a61a740 4 client.15702 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:57:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.481+0000 7fe43a61a740 1 -- 172.21.15.1:0/615062668 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0a4ac9190 con 0x55d0a4bdcc00 2024-04-26T01:57:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.481+0000 7fe42fe00640 1 --2- 172.21.15.1:0/615062668 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d0a4bdcc00 0x55d0a4bdf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.482+0000 7fe42fe00640 1 --2- 172.21.15.1:0/615062668 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d0a4bdcc00 0x55d0a4bdf020 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6caa5ce516c9c0e7 server_cookie=33ae6699aeeaed2e in_seq=0 out_seq=0 2024-04-26T01:57:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.482+0000 7fe416a00640 10 client.15702 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.496+0000 7fe416a00640 1 -- 172.21.15.1:0/615062668 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d0a4ac9190 con 0x55d0a4bdcc00 2024-04-26T01:57:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.496+0000 7fe416a00640 10 client.15702 handle_command_reply: tid=0 2024-04-26T01:57:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.496+0000 7fe43a61a740 1 client.15702 shutdown 2024-04-26T01:57:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.496+0000 7fe43a61a740 1 -- 172.21.15.1:0/615062668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b5b9b0 msgr2=0x55d0a4b6bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.496+0000 7fe43a61a740 1 --2- 172.21.15.1:0/615062668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b6bb60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe4240059f0 tx=0x7fe424019700 comp rx=0 tx=0).stop 2024-04-26T01:57:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.496+0000 7fe43a61a740 1 -- 172.21.15.1:0/615062668 shutdown_connections 2024-04-26T01:57:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.496+0000 7fe43a61a740 1 --2- 172.21.15.1:0/615062668 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d0a4bdcc00 0x55d0a4bdf020 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.496+0000 7fe43a61a740 1 --2- 172.21.15.1:0/615062668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0a4b66b90 0x55d0a4b6c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.496+0000 7fe43a61a740 1 --2- 172.21.15.1:0/615062668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a4b5c2d0 0x55d0a4b6c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.496+0000 7fe43a61a740 1 --2- 172.21.15.1:0/615062668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0a4b5b9b0 0x55d0a4b6bb60 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.496+0000 7fe416a00640 0 client.15702 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.496+0000 7fe43a61a740 1 -- 172.21.15.1:0/615062668 >> 172.21.15.1:0/615062668 conn(0x55d0a4b57ec0 msgr2=0x55d0a4b56790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.496+0000 7fe43a61a740 1 -- 172.21.15.1:0/615062668 shutdown_connections 2024-04-26T01:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.496+0000 7fe43a61a740 1 -- 172.21.15.1:0/615062668 wait complete. 2024-04-26T01:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.496+0000 7fe43a61a740 20 client.15702 trim_cache size 0 max 16384 2024-04-26T01:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:57:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:57:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:57:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.497+0000 7fe436200640 1 -- 172.21.15.1:0/3143736920 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe4180a3d80 msgr2=0x7fe4180a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.497+0000 7fe436200640 1 --2- 172.21.15.1:0/3143736920 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe4180a3d80 0x7fe4180a6240 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe428009a60 tx=0x7fe428005d80 comp rx=0 tx=0).stop 2024-04-26T01:57:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.497+0000 7fe436200640 1 -- 172.21.15.1:0/3143736920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe430103830 msgr2=0x7fe4301010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.497+0000 7fe436200640 1 --2- 172.21.15.1:0/3143736920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe430103830 0x7fe4301010a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00e7c0 tx=0x7fe41c00ec80 comp rx=0 tx=0).stop 2024-04-26T01:57:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.497+0000 7fe436200640 1 -- 172.21.15.1:0/3143736920 shutdown_connections 2024-04-26T01:57:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.497+0000 7fe436200640 1 --2- 172.21.15.1:0/3143736920 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe4180a3d80 0x7fe4180a6240 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.497+0000 7fe436200640 1 --2- 172.21.15.1:0/3143736920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe43010c330 0x7fe4301015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.497+0000 7fe436200640 1 --2- 172.21.15.1:0/3143736920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe430103830 0x7fe4301010a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.497+0000 7fe436200640 1 --2- 172.21.15.1:0/3143736920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe430102f10 0x7fe430100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.497+0000 7fe436200640 1 -- 172.21.15.1:0/3143736920 >> 172.21.15.1:0/3143736920 conn(0x7fe4300fc770 msgr2=0x7fe4300fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.498+0000 7fe436200640 1 -- 172.21.15.1:0/3143736920 shutdown_connections 2024-04-26T01:57:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:31.498+0000 7fe436200640 1 -- 172.21.15.1:0/3143736920 wait complete. 2024-04-26T01:57:32.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:32 smithi173 ceph-mon[32553]: pgmap v180: 97 pgs: 97 active+clean; 42 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.7 MiB/s wr, 436 op/s 2024-04-26T01:57:32.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:32 smithi001 ceph-mon[27822]: pgmap v180: 97 pgs: 97 active+clean; 42 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.7 MiB/s wr, 436 op/s 2024-04-26T01:57:32.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:32 smithi097 ceph-mon[32853]: pgmap v180: 97 pgs: 97 active+clean; 42 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 2.7 MiB/s wr, 436 op/s 2024-04-26T01:57:33.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:33 smithi173 ceph-mon[32553]: pgmap v181: 97 pgs: 97 active+clean; 42 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 291 op/s 2024-04-26T01:57:33.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:33 smithi001 ceph-mon[27822]: pgmap v181: 97 pgs: 97 active+clean; 42 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 291 op/s 2024-04-26T01:57:33.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:33 smithi097 ceph-mon[32853]: pgmap v181: 97 pgs: 97 active+clean; 42 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 291 op/s 2024-04-26T01:57:36.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:35 smithi097 ceph-mon[32853]: pgmap v182: 97 pgs: 97 active+clean; 46 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.7 MiB/s wr, 439 op/s 2024-04-26T01:57:36.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:35 smithi173 ceph-mon[32553]: pgmap v182: 97 pgs: 97 active+clean; 46 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.7 MiB/s wr, 439 op/s 2024-04-26T01:57:36.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:35 smithi001 ceph-mon[27822]: pgmap v182: 97 pgs: 97 active+clean; 46 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.7 MiB/s wr, 439 op/s 2024-04-26T01:57:36.442 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-04-26T01:57:36.443 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:57:36.957 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:57:37.207 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0x9ff773 2024-04-26T01:57:38.049 INFO:tasks.workunit.client.0.smithi001.stdout:20000 punch from 0x34ddcb to 0x35324f, (0x5484 bytes) 2024-04-26T01:57:38.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:37 smithi097 ceph-mon[32853]: pgmap v183: 97 pgs: 97 active+clean; 46 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.9 MiB/s wr, 304 op/s 2024-04-26T01:57:38.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:37 smithi173 ceph-mon[32553]: pgmap v183: 97 pgs: 97 active+clean; 46 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.9 MiB/s wr, 304 op/s 2024-04-26T01:57:38.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:37 smithi001 ceph-mon[27822]: pgmap v183: 97 pgs: 97 active+clean; 46 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.9 MiB/s wr, 304 op/s 2024-04-26T01:57:39.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.406+0000 7f9028c00700 1 -- 172.21.15.1:0/1681546790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9024105890 msgr2=0x7f90241098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:39.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.406+0000 7f9028c00700 1 --2- 172.21.15.1:0/1681546790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9024105890 0x7f90241098e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f90100098f0 tx=0x7f9010009c00 comp rx=0 tx=0).stop 2024-04-26T01:57:39.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.406+0000 7f9028c00700 1 -- 172.21.15.1:0/1681546790 shutdown_connections 2024-04-26T01:57:39.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.406+0000 7f9028c00700 1 --2- 172.21.15.1:0/1681546790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9024109fc0 0x7f9024111b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.406+0000 7f9028c00700 1 --2- 172.21.15.1:0/1681546790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9024104ee0 0x7f90241052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.406+0000 7f9028c00700 1 --2- 172.21.15.1:0/1681546790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9024105890 0x7f90241098e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.406+0000 7f9028c00700 1 -- 172.21.15.1:0/1681546790 >> 172.21.15.1:0/1681546790 conn(0x7f9024100770 msgr2=0x7f9024102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:39.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.406+0000 7f9028c00700 1 -- 172.21.15.1:0/1681546790 shutdown_connections 2024-04-26T01:57:39.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.406+0000 7f9028c00700 1 -- 172.21.15.1:0/1681546790 wait complete. 2024-04-26T01:57:39.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.407+0000 7f9028c00700 1 Processor -- start 2024-04-26T01:57:39.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.408+0000 7f9028c00700 1 -- start start 2024-04-26T01:57:39.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.408+0000 7f9028c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9024104ee0 0x7f902419b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.408+0000 7f9028c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9024105890 0x7f902419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.408+0000 7f9028c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9024109fc0 0x7f902419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.408+0000 7f9028c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9024114e80 con 0x7f9024105890 2024-04-26T01:57:39.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.408+0000 7f9028c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9024114d00 con 0x7f9024104ee0 2024-04-26T01:57:39.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.408+0000 7f9028c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9024115000 con 0x7f9024109fc0 2024-04-26T01:57:39.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.409+0000 7f9021e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9024104ee0 0x7f902419b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.409+0000 7f9021e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9024104ee0 0x7f902419b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:51938/0 (socket says 172.21.15.1:51938) 2024-04-26T01:57:39.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.409+0000 7f9021e00700 1 -- 172.21.15.1:0/1668933910 learned_addr learned my addr 172.21.15.1:0/1668933910 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:39.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.409+0000 7f9022800700 1 --2- 172.21.15.1:0/1668933910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9024109fc0 0x7f902419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.409+0000 7f9021400700 1 --2- 172.21.15.1:0/1668933910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9024105890 0x7f902419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.409+0000 7f9021400700 1 -- 172.21.15.1:0/1668933910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9024109fc0 msgr2=0x7f902419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:39.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.409+0000 7f9021400700 1 --2- 172.21.15.1:0/1668933910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9024109fc0 0x7f902419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.409+0000 7f9021400700 1 -- 172.21.15.1:0/1668933910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9024104ee0 msgr2=0x7f902419b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:39.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.409+0000 7f9021400700 1 --2- 172.21.15.1:0/1668933910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9024104ee0 0x7f902419b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.409+0000 7f9021400700 1 -- 172.21.15.1:0/1668933910 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90241a0460 con 0x7f9024105890 2024-04-26T01:57:39.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.410+0000 7f9021400700 1 --2- 172.21.15.1:0/1668933910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9024105890 0x7f902419b990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9010000c00 tx=0x7f901000fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:39.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.410+0000 7f901aa00700 1 -- 172.21.15.1:0/1668933910 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90100050c0 con 0x7f9024105890 2024-04-26T01:57:39.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.410+0000 7f901aa00700 1 -- 172.21.15.1:0/1668933910 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f901000b560 con 0x7f9024105890 2024-04-26T01:57:39.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.410+0000 7f901aa00700 1 -- 172.21.15.1:0/1668933910 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90100039b0 con 0x7f9024105890 2024-04-26T01:57:39.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.410+0000 7f9028c00700 1 -- 172.21.15.1:0/1668933910 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90241a06f0 con 0x7f9024105890 2024-04-26T01:57:39.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.411+0000 7f9028c00700 1 -- 172.21.15.1:0/1668933910 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90241a0b50 con 0x7f9024105890 2024-04-26T01:57:39.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.412+0000 7f9028c00700 1 -- 172.21.15.1:0/1668933910 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9024069730 con 0x7f9024105890 2024-04-26T01:57:39.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.415+0000 7f901aa00700 1 -- 172.21.15.1:0/1668933910 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f901000b9e0 con 0x7f9024105890 2024-04-26T01:57:39.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.416+0000 7f901aa00700 1 --2- 172.21.15.1:0/1668933910 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f90080a3f00 0x7f90080a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:39.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.417+0000 7f901aa00700 1 -- 172.21.15.1:0/1668933910 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f90100d1880 con 0x7f9024105890 2024-04-26T01:57:39.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.417+0000 7f9021e00700 1 --2- 172.21.15.1:0/1668933910 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f90080a3f00 0x7f90080a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:39.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.418+0000 7f9021e00700 1 --2- 172.21.15.1:0/1668933910 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f90080a3f00 0x7f90080a63c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9014009680 tx=0x7f9014009260 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:39.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.423+0000 7f901aa00700 1 -- 172.21.15.1:0/1668933910 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90241a0b50 con 0x7f9024105890 2024-04-26T01:57:39.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.770+0000 7f9028c00700 1 -- 172.21.15.1:0/1668933910 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f902402df80 con 0x7f9024105890 2024-04-26T01:57:39.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.771+0000 7f901aa00700 1 -- 172.21.15.1:0/1668933910 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11838 (secure 0 0 0) 0x7f9010013030 con 0x7f9024105890 2024-04-26T01:57:39.770 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:57:39.770 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:55:49.645549+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25414":{"gid":25414,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/453427322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":453427322},{"type":"v1","addr":"172.21.15.173:6835","nonce":453427322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24499,24514,24518]},"id":1}]} 2024-04-26T01:57:39.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.774+0000 7f9028c00700 1 -- 172.21.15.1:0/1668933910 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f90080a3f00 msgr2=0x7f90080a63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:39.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.774+0000 7f9028c00700 1 --2- 172.21.15.1:0/1668933910 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f90080a3f00 0x7f90080a63c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9014009680 tx=0x7f9014009260 comp rx=0 tx=0).stop 2024-04-26T01:57:39.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.774+0000 7f9028c00700 1 -- 172.21.15.1:0/1668933910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9024105890 msgr2=0x7f902419b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:39.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.774+0000 7f9028c00700 1 --2- 172.21.15.1:0/1668933910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9024105890 0x7f902419b990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9010000c00 tx=0x7f901000fa20 comp rx=0 tx=0).stop 2024-04-26T01:57:39.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.774+0000 7f9028c00700 1 -- 172.21.15.1:0/1668933910 shutdown_connections 2024-04-26T01:57:39.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.774+0000 7f9028c00700 1 --2- 172.21.15.1:0/1668933910 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f90080a3f00 0x7f90080a63c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.774+0000 7f9028c00700 1 --2- 172.21.15.1:0/1668933910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9024109fc0 0x7f902419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.774+0000 7f9028c00700 1 --2- 172.21.15.1:0/1668933910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9024104ee0 0x7f902419b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.774+0000 7f9028c00700 1 --2- 172.21.15.1:0/1668933910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9024105890 0x7f902419b990 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:39.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.774+0000 7f9028c00700 1 -- 172.21.15.1:0/1668933910 >> 172.21.15.1:0/1668933910 conn(0x7f9024100770 msgr2=0x7f9024102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:39.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.774+0000 7f9028c00700 1 -- 172.21.15.1:0/1668933910 shutdown_connections 2024-04-26T01:57:39.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:39.774+0000 7f9028c00700 1 -- 172.21.15.1:0/1668933910 wait complete. 2024-04-26T01:57:39.775 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 32 2024-04-26T01:57:40.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:39 smithi001 ceph-mon[27822]: pgmap v184: 97 pgs: 97 active+clean; 46 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.9 MiB/s wr, 304 op/s 2024-04-26T01:57:40.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:39 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1668933910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:57:40.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:39 smithi097 ceph-mon[32853]: pgmap v184: 97 pgs: 97 active+clean; 46 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.9 MiB/s wr, 304 op/s 2024-04-26T01:57:40.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:39 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1668933910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:57:40.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:39 smithi173 ceph-mon[32553]: pgmap v184: 97 pgs: 97 active+clean; 46 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.9 MiB/s wr, 304 op/s 2024-04-26T01:57:40.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:39 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1668933910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:57:40.761 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24518 quiesce db / --timeout 60 --expiration 70 --await-for 42.0' 2024-04-26T01:57:40.761 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24518 quiesce db / --timeout 60 --expiration 70 --await-for 42.0 2024-04-26T01:57:41.271 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:57:42.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:41 smithi001 ceph-mon[27822]: pgmap v185: 97 pgs: 97 active+clean; 47 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.7 MiB/s wr, 449 op/s 2024-04-26T01:57:42.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:41 smithi097 ceph-mon[32853]: pgmap v185: 97 pgs: 97 active+clean; 47 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.7 MiB/s wr, 449 op/s 2024-04-26T01:57:42.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:41 smithi173 ceph-mon[32553]: pgmap v185: 97 pgs: 97 active+clean; 47 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.7 MiB/s wr, 449 op/s 2024-04-26T01:57:43.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.656+0000 7fdfe2000700 1 -- 172.21.15.1:0/2318732933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfdc06b100 msgr2=0x7fdfdc10f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:43.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.656+0000 7fdfe2000700 1 --2- 172.21.15.1:0/2318732933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfdc06b100 0x7fdfdc10f8c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400b0a0 tx=0x7fdfd400b3b0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.656+0000 7fdfe2000700 1 -- 172.21.15.1:0/2318732933 shutdown_connections 2024-04-26T01:57:43.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.656+0000 7fdfe2000700 1 --2- 172.21.15.1:0/2318732933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfdc06b100 0x7fdfdc10f8c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.656+0000 7fdfe2000700 1 --2- 172.21.15.1:0/2318732933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfdc108a60 0x7fdfdc108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.656+0000 7fdfe2000700 1 --2- 172.21.15.1:0/2318732933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfdc06a740 0x7fdfdc06abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.657+0000 7fdfe2000700 1 -- 172.21.15.1:0/2318732933 >> 172.21.15.1:0/2318732933 conn(0x7fdfdc0788e0 msgr2=0x7fdfdc078cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:43.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.657+0000 7fdfe2000700 1 -- 172.21.15.1:0/2318732933 shutdown_connections 2024-04-26T01:57:43.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.657+0000 7fdfe2000700 1 -- 172.21.15.1:0/2318732933 wait complete. 2024-04-26T01:57:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.658+0000 7fdfe2000700 1 Processor -- start 2024-04-26T01:57:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.658+0000 7fdfe2000700 1 -- start start 2024-04-26T01:57:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.658+0000 7fdfe2000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfdc06a740 0x7fdfdc19b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.658+0000 7fdfe2000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfdc06b100 0x7fdfdc19b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.658+0000 7fdfe2000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfdc108a60 0x7fdfdc19fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.658+0000 7fdfe2000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc112c60 con 0x7fdfdc06a740 2024-04-26T01:57:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.658+0000 7fdfe2000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc112ae0 con 0x7fdfdc06b100 2024-04-26T01:57:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.658+0000 7fdfe2000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc112de0 con 0x7fdfdc108a60 2024-04-26T01:57:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.659+0000 7fdfdaa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfdc06b100 0x7fdfdc19b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.659+0000 7fdfdbe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfdc108a60 0x7fdfdc19fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.659+0000 7fdfdaa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfdc06b100 0x7fdfdc19b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:34638/0 (socket says 172.21.15.1:34638) 2024-04-26T01:57:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.659+0000 7fdfdaa00700 1 -- 172.21.15.1:0/2189398041 learned_addr learned my addr 172.21.15.1:0/2189398041 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:43.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.659+0000 7fdfdb400700 1 --2- 172.21.15.1:0/2189398041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfdc06a740 0x7fdfdc19b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:43.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.659+0000 7fdfdaa00700 1 -- 172.21.15.1:0/2189398041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfdc108a60 msgr2=0x7fdfdc19fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:43.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.659+0000 7fdfdaa00700 1 --2- 172.21.15.1:0/2189398041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfdc108a60 0x7fdfdc19fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.659+0000 7fdfdaa00700 1 -- 172.21.15.1:0/2189398041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfdc06a740 msgr2=0x7fdfdc19b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:43.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.659+0000 7fdfdaa00700 1 --2- 172.21.15.1:0/2189398041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfdc06a740 0x7fdfdc19b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.659+0000 7fdfdaa00700 1 -- 172.21.15.1:0/2189398041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfdc1a01f0 con 0x7fdfdc06b100 2024-04-26T01:57:43.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.660+0000 7fdfdaa00700 1 --2- 172.21.15.1:0/2189398041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfdc06b100 0x7fdfdc19b720 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdfc800e9a0 tx=0x7fdfc800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:43.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.660+0000 7fdfd3e00700 1 -- 172.21.15.1:0/2189398041 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc800ca10 con 0x7fdfdc06b100 2024-04-26T01:57:43.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.660+0000 7fdfd3e00700 1 -- 172.21.15.1:0/2189398041 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc800cbb0 con 0x7fdfdc06b100 2024-04-26T01:57:43.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.660+0000 7fdfd3e00700 1 -- 172.21.15.1:0/2189398041 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8018680 con 0x7fdfdc06b100 2024-04-26T01:57:43.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.660+0000 7fdfe2000700 1 -- 172.21.15.1:0/2189398041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfdc1a04e0 con 0x7fdfdc06b100 2024-04-26T01:57:43.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.661+0000 7fdfe2000700 1 -- 172.21.15.1:0/2189398041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfdc1a7d60 con 0x7fdfdc06b100 2024-04-26T01:57:43.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.664+0000 7fdfd3e00700 1 -- 172.21.15.1:0/2189398041 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fdfc8010430 con 0x7fdfdc06b100 2024-04-26T01:57:43.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.665+0000 7fdfd3e00700 1 --2- 172.21.15.1:0/2189398041 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fdfc00a3ed0 0x7fdfc00a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:43.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.665+0000 7fdfec8a6b80 1 Processor -- start 2024-04-26T01:57:43.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.665+0000 7fdfd3e00700 1 -- 172.21.15.1:0/2189398041 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fdfc80d09a0 con 0x7fdfdc06b100 2024-04-26T01:57:43.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.665+0000 7fdfdb400700 1 --2- 172.21.15.1:0/2189398041 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fdfc00a3ed0 0x7fdfc00a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:43.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.666+0000 7fdfec8a6b80 1 -- start start 2024-04-26T01:57:43.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.666+0000 7fdfec8a6b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c5389b0 0x56157c538d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:43.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.666+0000 7fdfec8a6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c5392d0 0x56157c543c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:43.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.666+0000 7fdfec8a6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c544180 0x56157c546570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:43.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.666+0000 7fdfec8a6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56157c48c020 con 0x56157c5392d0 2024-04-26T01:57:43.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.666+0000 7fdfec8a6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56157c4328c0 con 0x56157c544180 2024-04-26T01:57:43.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.666+0000 7fdfec8a6b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56157c487fe0 con 0x56157c5389b0 2024-04-26T01:57:43.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.666+0000 7fdfdaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c5392d0 0x56157c543c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:43.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.666+0000 7fdfdaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c5392d0 0x56157c543c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48448/0 (socket says 172.21.15.1:48448) 2024-04-26T01:57:43.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.666+0000 7fdfdaa00700 1 -- 172.21.15.1:0/1021040813 learned_addr learned my addr 172.21.15.1:0/1021040813 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:43.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.666+0000 7fdfdbe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c544180 0x56157c546570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:43.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.666+0000 7fdfdb400700 1 --2- 172.21.15.1:0/1021040813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c5389b0 0x56157c538d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:43.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.667+0000 7fdfdb400700 1 --2- 172.21.15.1:0/2189398041 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fdfc00a3ed0 0x7fdfc00a6390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc009be0 tx=0x7fdfcc009260 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:43.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.667+0000 7fdfdb400700 1 -- 172.21.15.1:0/1021040813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c544180 msgr2=0x56157c546570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:43.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.667+0000 7fdfdb400700 1 --2- 172.21.15.1:0/1021040813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c544180 0x56157c546570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.667+0000 7fdfdb400700 1 -- 172.21.15.1:0/1021040813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c5392d0 msgr2=0x56157c543c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:43.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.667+0000 7fdfdb400700 1 --2- 172.21.15.1:0/1021040813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c5392d0 0x56157c543c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.667+0000 7fdfdb400700 1 -- 172.21.15.1:0/1021040813 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56157c428810 con 0x56157c5389b0 2024-04-26T01:57:43.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.667+0000 7fdfdb400700 1 --2- 172.21.15.1:0/1021040813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c5389b0 0x56157c538d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0117d0 tx=0x7fdfcc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75c427996af8bfea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:43.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.667+0000 7fdfd0c00700 1 -- 172.21.15.1:0/1021040813 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc004030 con 0x56157c5389b0 2024-04-26T01:57:43.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.668+0000 7fdfd0c00700 1 -- 172.21.15.1:0/1021040813 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfcc00e840 con 0x56157c5389b0 2024-04-26T01:57:43.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.668+0000 7fdfd0c00700 1 -- 172.21.15.1:0/1021040813 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc003be0 con 0x56157c5389b0 2024-04-26T01:57:43.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.668+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/1021040813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c5389b0 msgr2=0x56157c538d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:43.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.668+0000 7fdfec8a6b80 1 --2- 172.21.15.1:0/1021040813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c5389b0 0x56157c538d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0117d0 tx=0x7fdfcc014000 comp rx=0 tx=0).stop 2024-04-26T01:57:43.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.668+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/1021040813 shutdown_connections 2024-04-26T01:57:43.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.668+0000 7fdfec8a6b80 1 --2- 172.21.15.1:0/1021040813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c544180 0x56157c546570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.669+0000 7fdfec8a6b80 1 --2- 172.21.15.1:0/1021040813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c5389b0 0x56157c538d90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.669+0000 7fdfec8a6b80 1 --2- 172.21.15.1:0/1021040813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c5392d0 0x56157c543c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.669+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/1021040813 >> 172.21.15.1:0/1021040813 conn(0x56157c467730 msgr2=0x56157c534e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:43.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.669+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/1021040813 shutdown_connections 2024-04-26T01:57:43.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.669+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/1021040813 wait complete. 2024-04-26T01:57:43.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.669+0000 7fdfec8a6b80 1 -- start start 2024-04-26T01:57:43.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.669+0000 7fdfec8a6b80 1 Processor -- start 2024-04-26T01:57:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.670+0000 7fdfec8a6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c5389b0 0x56157c5438d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.670+0000 7fdfec8a6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c5392d0 0x56157c540bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.670+0000 7fdfec8a6b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c544180 0x56157c541110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.670+0000 7fdfec8a6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56157c546ca0 con 0x56157c5392d0 2024-04-26T01:57:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.670+0000 7fdfec8a6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56157c4459f0 con 0x56157c5389b0 2024-04-26T01:57:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.670+0000 7fdfec8a6b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56157c546e20 con 0x56157c544180 2024-04-26T01:57:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.670+0000 7fdfec8a6b80 10 client.0 _ll_register_callbacks cb 0x56157c4248b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.670+0000 7fdfdbe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c544180 0x56157c541110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.670+0000 7fdfdb400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c5389b0 0x56157c5438d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.671+0000 7fdfdaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c5392d0 0x56157c540bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.671+0000 7fdfdb400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c5389b0 0x56157c5438d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:34662/0 (socket says 172.21.15.1:34662) 2024-04-26T01:57:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.671+0000 7fdfdbe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c544180 0x56157c541110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:58354/0 (socket says 172.21.15.1:58354) 2024-04-26T01:57:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.671+0000 7fdfdb400700 1 -- 172.21.15.1:0/4230398963 learned_addr learned my addr 172.21.15.1:0/4230398963 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.671+0000 7fdfdbe00700 1 -- 172.21.15.1:0/4230398963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c5389b0 msgr2=0x56157c5438d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.671+0000 7fdfdbe00700 1 --2- 172.21.15.1:0/4230398963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c5389b0 0x56157c5438d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.671+0000 7fdfdbe00700 1 -- 172.21.15.1:0/4230398963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c5392d0 msgr2=0x56157c540bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.671+0000 7fdfdbe00700 1 --2- 172.21.15.1:0/4230398963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c5392d0 0x56157c540bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.671+0000 7fdfdbe00700 1 -- 172.21.15.1:0/4230398963 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56157c3d9a60 con 0x56157c544180 2024-04-26T01:57:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.671+0000 7fdfdbe00700 1 --2- 172.21.15.1:0/4230398963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c544180 0x56157c541110 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40062a0 tx=0x7fdfd400be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39805583cadc0270 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.671+0000 7fdfc5600700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:57:43.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.672+0000 7fdfc5600700 1 -- 172.21.15.1:0/4230398963 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd40076e0 con 0x56157c544180 2024-04-26T01:57:43.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.672+0000 7fdfc5600700 1 -- 172.21.15.1:0/4230398963 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fdfd4007880 con 0x56157c544180 2024-04-26T01:57:43.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.672+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/4230398963 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfcc00e840 con 0x56157c544180 2024-04-26T01:57:43.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.672+0000 7fdfc5600700 1 -- 172.21.15.1:0/4230398963 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdfd4007a80 con 0x56157c544180 2024-04-26T01:57:43.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.672+0000 7fdfec8a6b80 10 client.25891 fetch_fsmap learned FSMap version 32 2024-04-26T01:57:43.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.672+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/4230398963 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56157c422c80 con 0x56157c544180 2024-04-26T01:57:43.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.672+0000 7fdfc5600700 1 -- 172.21.15.1:0/4230398963 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfcc00e840 con 0x56157c544180 2024-04-26T01:57:43.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.672+0000 7fdfc5600700 1 -- 172.21.15.1:0/4230398963 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fdfd4011070 con 0x56157c544180 2024-04-26T01:57:43.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.673+0000 7fdfec8a6b80 10 client.25891 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:57:43.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.673+0000 7fdfec8a6b80 10 client.25891 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:57:43.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.673+0000 7fdfec8a6b80 20 client.25891 populate_metadata read hostname 'smithi001' 2024-04-26T01:57:43.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.673+0000 7fdfec8a6b80 1 --2- 172.21.15.1:0/4230398963 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56157c5538e0 0x56157c555cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:43.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.673+0000 7fdfec8a6b80 4 client.25891 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:57:43.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.673+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/4230398963 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56157c499810 con 0x56157c5538e0 2024-04-26T01:57:43.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.673+0000 7fdfdb400700 1 --2- 172.21.15.1:0/4230398963 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56157c5538e0 0x56157c555cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:43.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.673+0000 7fdfdb400700 1 --2- 172.21.15.1:0/4230398963 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56157c5538e0 0x56157c555cd0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4824d80bb55de94 server_cookie=ab433d05e9ac7299 in_seq=0 out_seq=0 2024-04-26T01:57:43.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.673+0000 7fdfc5600700 10 client.25891 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:43.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.674+0000 7fdfc5600700 1 -- 172.21.15.1:0/4230398963 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56157c499810 con 0x56157c5538e0 2024-04-26T01:57:43.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.674+0000 7fdfc5600700 10 client.25891 handle_command_reply: tid=0 2024-04-26T01:57:43.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.674+0000 7fdfec8a6b80 1 client.25891 shutdown 2024-04-26T01:57:43.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.674+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/4230398963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c544180 msgr2=0x56157c541110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.675+0000 7fdfec8a6b80 1 --2- 172.21.15.1:0/4230398963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c544180 0x56157c541110 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40062a0 tx=0x7fdfd400be20 comp rx=0 tx=0).stop 2024-04-26T01:57:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.675+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/4230398963 shutdown_connections 2024-04-26T01:57:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.675+0000 7fdfec8a6b80 1 --2- 172.21.15.1:0/4230398963 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56157c5538e0 0x56157c555cd0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.675+0000 7fdfec8a6b80 1 --2- 172.21.15.1:0/4230398963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c544180 0x56157c541110 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.675+0000 7fdfc5600700 0 client.25891 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.675+0000 7fdfec8a6b80 1 --2- 172.21.15.1:0/4230398963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c5389b0 0x56157c5438d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.675+0000 7fdfec8a6b80 1 --2- 172.21.15.1:0/4230398963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c5392d0 0x56157c540bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.675+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/4230398963 >> 172.21.15.1:0/4230398963 conn(0x56157c467730 msgr2=0x56157c534a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.675+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/4230398963 shutdown_connections 2024-04-26T01:57:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.675+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/4230398963 wait complete. 2024-04-26T01:57:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.675+0000 7fdfec8a6b80 20 client.25891 trim_cache size 0 max 16384 2024-04-26T01:57:43.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.700+0000 7fdfec8a6b80 1 Processor -- start 2024-04-26T01:57:43.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.700+0000 7fdfec8a6b80 1 -- start start 2024-04-26T01:57:43.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.700+0000 7fdfec8a6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c5389b0 0x56157c54fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:43.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.700+0000 7fdfec8a6b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c5392d0 0x56157c55b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:43.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.700+0000 7fdfec8a6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c544180 0x56157c55ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:43.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.700+0000 7fdfec8a6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc1b1de0 con 0x56157c544180 2024-04-26T01:57:43.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.700+0000 7fdfec8a6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc1af670 con 0x56157c5389b0 2024-04-26T01:57:43.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.700+0000 7fdfec8a6b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc1ad4c0 con 0x56157c5392d0 2024-04-26T01:57:43.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.700+0000 7fdfdb400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c5389b0 0x56157c54fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:43.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.700+0000 7fdfdbe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c544180 0x56157c55ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:43.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.700+0000 7fdfdb400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c5389b0 0x56157c54fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:34668/0 (socket says 172.21.15.1:34668) 2024-04-26T01:57:43.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.700+0000 7fdfdb400700 1 -- 172.21.15.1:0/395853143 learned_addr learned my addr 172.21.15.1:0/395853143 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:43.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.700+0000 7fdfdaa00700 1 --2- 172.21.15.1:0/395853143 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c5392d0 0x56157c55b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:43.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.701+0000 7fdfdb400700 1 -- 172.21.15.1:0/395853143 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c5392d0 msgr2=0x56157c55b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:43.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.701+0000 7fdfdb400700 1 --2- 172.21.15.1:0/395853143 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c5392d0 0x56157c55b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.701+0000 7fdfdb400700 1 -- 172.21.15.1:0/395853143 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c544180 msgr2=0x56157c55ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:43.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.701+0000 7fdfdb400700 1 --2- 172.21.15.1:0/395853143 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c544180 0x56157c55ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.701+0000 7fdfdb400700 1 -- 172.21.15.1:0/395853143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56157c530a20 con 0x56157c5389b0 2024-04-26T01:57:43.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.701+0000 7fdfdb400700 1 --2- 172.21.15.1:0/395853143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c5389b0 0x56157c54fba0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0117d0 tx=0x7fdfcc029040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eefb4c0baca7b2e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.701+0000 7fdfc7e00700 1 -- 172.21.15.1:0/395853143 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc00ea40 con 0x56157c5389b0 2024-04-26T01:57:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.701+0000 7fdfc7e00700 1 -- 172.21.15.1:0/395853143 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfcc0055c0 con 0x56157c5389b0 2024-04-26T01:57:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.701+0000 7fdfc7e00700 1 -- 172.21.15.1:0/395853143 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc01e420 con 0x56157c5389b0 2024-04-26T01:57:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.702+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/395853143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c5389b0 msgr2=0x56157c54fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.702+0000 7fdfec8a6b80 1 --2- 172.21.15.1:0/395853143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c5389b0 0x56157c54fba0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0117d0 tx=0x7fdfcc029040 comp rx=0 tx=0).stop 2024-04-26T01:57:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.702+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/395853143 shutdown_connections 2024-04-26T01:57:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.702+0000 7fdfec8a6b80 1 --2- 172.21.15.1:0/395853143 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c544180 0x56157c55ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.702+0000 7fdfec8a6b80 1 --2- 172.21.15.1:0/395853143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c5389b0 0x56157c54fba0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.702+0000 7fdfec8a6b80 1 --2- 172.21.15.1:0/395853143 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c5392d0 0x56157c55b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.702+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/395853143 >> 172.21.15.1:0/395853143 conn(0x56157c467730 msgr2=0x56157c534990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:43.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.703+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/395853143 shutdown_connections 2024-04-26T01:57:43.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.703+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/395853143 wait complete. 2024-04-26T01:57:43.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.703+0000 7fdfec8a6b80 1 -- start start 2024-04-26T01:57:43.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.703+0000 7fdfec8a6b80 1 Processor -- start 2024-04-26T01:57:43.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.704+0000 7fdfec8a6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c5389b0 0x56157c547070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:43.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.704+0000 7fdfec8a6b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c5392d0 0x56157c5475b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:43.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.704+0000 7fdfec8a6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c544180 0x56157c547af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:43.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.704+0000 7fdfec8a6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc1adb30 con 0x56157c5389b0 2024-04-26T01:57:43.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.704+0000 7fdfec8a6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc1aefa0 con 0x56157c544180 2024-04-26T01:57:43.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.704+0000 7fdfec8a6b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc1ae200 con 0x56157c5392d0 2024-04-26T01:57:43.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.704+0000 7fdfec8a6b80 10 client.0 _ll_register_callbacks cb 0x56157c4248b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.704+0000 7fdfdaa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c5392d0 0x56157c5475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.704+0000 7fdfdaa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c5392d0 0x56157c5475b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:58376/0 (socket says 172.21.15.1:58376) 2024-04-26T01:57:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.704+0000 7fdfdaa00700 1 -- 172.21.15.1:0/268005163 learned_addr learned my addr 172.21.15.1:0/268005163 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.704+0000 7fdfdb400700 1 --2- 172.21.15.1:0/268005163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c5389b0 0x56157c547070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.704+0000 7fdfdbe00700 1 --2- 172.21.15.1:0/268005163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c544180 0x56157c547af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.705+0000 7fdfdbe00700 1 -- 172.21.15.1:0/268005163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c5392d0 msgr2=0x56157c5475b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.705+0000 7fdfdbe00700 1 --2- 172.21.15.1:0/268005163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c5392d0 0x56157c5475b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.705+0000 7fdfdbe00700 1 -- 172.21.15.1:0/268005163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c5389b0 msgr2=0x56157c547070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.705+0000 7fdfdbe00700 1 --2- 172.21.15.1:0/268005163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c5389b0 0x56157c547070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.705+0000 7fdfdbe00700 1 -- 172.21.15.1:0/268005163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56157c5c2730 con 0x56157c544180 2024-04-26T01:57:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.705+0000 7fdfdaa00700 1 --2- 172.21.15.1:0/268005163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c5392d0 0x56157c5475b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:43.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.705+0000 7fdfdb400700 1 --2- 172.21.15.1:0/268005163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c5389b0 0x56157c547070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.706+0000 7fdfdbe00700 1 --2- 172.21.15.1:0/268005163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c544180 0x56157c547af0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400b3f0 tx=0x7fdfd4005470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e98451d43648140 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.706+0000 7fdfc5600700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:57:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.706+0000 7fdfc5600700 1 -- 172.21.15.1:0/268005163 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4005e70 con 0x56157c544180 2024-04-26T01:57:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.706+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/268005163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfcc0055c0 con 0x56157c544180 2024-04-26T01:57:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.706+0000 7fdfc5600700 1 -- 172.21.15.1:0/268005163 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fdfd40074a0 con 0x56157c544180 2024-04-26T01:57:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.707+0000 7fdfc5600700 1 -- 172.21.15.1:0/268005163 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdfd40076a0 con 0x56157c544180 2024-04-26T01:57:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.707+0000 7fdfec8a6b80 10 client.25874 fetch_fsmap learned FSMap version 32 2024-04-26T01:57:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.707+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/268005163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56157c5c39d0 con 0x56157c544180 2024-04-26T01:57:43.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.707+0000 7fdfc5600700 1 -- 172.21.15.1:0/268005163 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfcc0055c0 con 0x56157c544180 2024-04-26T01:57:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.707+0000 7fdfc5600700 1 -- 172.21.15.1:0/268005163 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fdfd4011040 con 0x56157c544180 2024-04-26T01:57:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.707+0000 7fdfec8a6b80 10 client.25874 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:57:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.707+0000 7fdfec8a6b80 10 client.25874 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:57:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.707+0000 7fdfec8a6b80 20 client.25874 populate_metadata read hostname 'smithi001' 2024-04-26T01:57:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.707+0000 7fdfec8a6b80 1 --2- 172.21.15.1:0/268005163 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56157c5c40e0 0x56157c5c6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.707+0000 7fdfec8a6b80 4 client.25874 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}] 2024-04-26T01:57:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.707+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/268005163 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}) -- 0x56157c533b30 con 0x56157c5c40e0 2024-04-26T01:57:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.707+0000 7fdfdb400700 1 --2- 172.21.15.1:0/268005163 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56157c5c40e0 0x56157c5c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.708+0000 7fdfdb400700 1 --2- 172.21.15.1:0/268005163 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56157c5c40e0 0x56157c5c6500 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=740fa6b98ed8dd6a server_cookie=fc8fa4a102c7cebd in_seq=0 out_seq=0 2024-04-26T01:57:43.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:43.708+0000 7fdfc5600700 10 client.25874 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:44.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:43 smithi001 ceph-mon[27822]: pgmap v186: 97 pgs: 97 active+clean; 47 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.7 MiB/s wr, 292 op/s 2024-04-26T01:57:44.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:43 smithi097 ceph-mon[32853]: pgmap v186: 97 pgs: 97 active+clean; 47 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.7 MiB/s wr, 292 op/s 2024-04-26T01:57:44.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:43 smithi173 ceph-mon[32553]: pgmap v186: 97 pgs: 97 active+clean; 47 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.7 MiB/s wr, 292 op/s 2024-04-26T01:57:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.648+0000 7ffa36a00640 1 -- 172.21.15.1:0/2467267041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa3010e660 msgr2=0x7ffa3010ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.648+0000 7ffa36a00640 1 --2- 172.21.15.1:0/2467267041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa3010e660 0x7ffa3010ea40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffa20009930 tx=0x7ffa2002a420 comp rx=0 tx=0).stop 2024-04-26T01:57:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.649+0000 7ffa36a00640 1 -- 172.21.15.1:0/2467267041 shutdown_connections 2024-04-26T01:57:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.649+0000 7ffa36a00640 1 --2- 172.21.15.1:0/2467267041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa30108f60 0x7ffa30115420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.649+0000 7ffa36a00640 1 --2- 172.21.15.1:0/2467267041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa301085a0 0x7ffa30108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.649+0000 7ffa36a00640 1 --2- 172.21.15.1:0/2467267041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa3010e660 0x7ffa3010ea40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.649+0000 7ffa36a00640 1 -- 172.21.15.1:0/2467267041 >> 172.21.15.1:0/2467267041 conn(0x7ffa30076460 msgr2=0x7ffa30078880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.649+0000 7ffa36a00640 1 -- 172.21.15.1:0/2467267041 shutdown_connections 2024-04-26T01:57:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.649+0000 7ffa36a00640 1 -- 172.21.15.1:0/2467267041 wait complete. 2024-04-26T01:57:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.650+0000 7ffa36a00640 1 Processor -- start 2024-04-26T01:57:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.650+0000 7ffa36a00640 1 -- start start 2024-04-26T01:57:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.651+0000 7ffa36a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa301085a0 0x7ffa3019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.651+0000 7ffa36a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa30108f60 0x7ffa3019f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.651+0000 7ffa36a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa3010e660 0x7ffa301a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.651+0000 7ffa36a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffa301188a0 con 0x7ffa3010e660 2024-04-26T01:57:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.651+0000 7ffa36a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffa30118720 con 0x7ffa301085a0 2024-04-26T01:57:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.651+0000 7ffa36a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffa30118a20 con 0x7ffa30108f60 2024-04-26T01:57:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.651+0000 7ffa2fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa301085a0 0x7ffa3019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.651+0000 7ffa2fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa301085a0 0x7ffa3019ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:34708/0 (socket says 172.21.15.1:34708) 2024-04-26T01:57:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.651+0000 7ffa2fe00640 1 -- 172.21.15.1:0/406250840 learned_addr learned my addr 172.21.15.1:0/406250840 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.651+0000 7ffa34a00640 1 --2- 172.21.15.1:0/406250840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa3010e660 0x7ffa301a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.651+0000 7ffa2f400640 1 --2- 172.21.15.1:0/406250840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa30108f60 0x7ffa3019f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.651+0000 7ffa2fe00640 1 -- 172.21.15.1:0/406250840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa30108f60 msgr2=0x7ffa3019f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.651+0000 7ffa2fe00640 1 --2- 172.21.15.1:0/406250840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa30108f60 0x7ffa3019f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.651+0000 7ffa2fe00640 1 -- 172.21.15.1:0/406250840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa3010e660 msgr2=0x7ffa301a36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.651+0000 7ffa2fe00640 1 --2- 172.21.15.1:0/406250840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa3010e660 0x7ffa301a36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.651+0000 7ffa2fe00640 1 -- 172.21.15.1:0/406250840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa301a3e10 con 0x7ffa301085a0 2024-04-26T01:57:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.652+0000 7ffa2fe00640 1 --2- 172.21.15.1:0/406250840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa301085a0 0x7ffa3019ee20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ffa20005ab0 tx=0x7ffa20003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.652+0000 7ffa2cc00640 1 -- 172.21.15.1:0/406250840 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2003f070 con 0x7ffa301085a0 2024-04-26T01:57:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.652+0000 7ffa2cc00640 1 -- 172.21.15.1:0/406250840 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa20031b50 con 0x7ffa301085a0 2024-04-26T01:57:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.652+0000 7ffa2cc00640 1 -- 172.21.15.1:0/406250840 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20035030 con 0x7ffa301085a0 2024-04-26T01:57:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.652+0000 7ffa36a00640 1 -- 172.21.15.1:0/406250840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa301a40a0 con 0x7ffa301085a0 2024-04-26T01:57:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.652+0000 7ffa36a00640 1 -- 172.21.15.1:0/406250840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa301ab990 con 0x7ffa301085a0 2024-04-26T01:57:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.655+0000 7ffa397ac740 1 Processor -- start 2024-04-26T01:57:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.656+0000 7ffa397ac740 1 -- start start 2024-04-26T01:57:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.656+0000 7ffa2cc00640 1 -- 172.21.15.1:0/406250840 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7ffa20042070 con 0x7ffa301085a0 2024-04-26T01:57:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.656+0000 7ffa397ac740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef4820 0x55aa63ef4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.656+0000 7ffa397ac740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63ef5140 0x55aa63eff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.656+0000 7ffa397ac740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa63effa00 0x55aa63f01df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.656+0000 7ffa397ac740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55aa63e5a900 con 0x55aa63ef4820 2024-04-26T01:57:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.656+0000 7ffa397ac740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55aa63e00ef0 con 0x55aa63ef5140 2024-04-26T01:57:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.656+0000 7ffa397ac740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55aa63e568c0 con 0x55aa63effa00 2024-04-26T01:57:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.656+0000 7ffa2fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63ef5140 0x55aa63eff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.656+0000 7ffa34a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef4820 0x55aa63ef4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.656+0000 7ffa2f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa63effa00 0x55aa63f01df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.656+0000 7ffa2fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63ef5140 0x55aa63eff4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:34720/0 (socket says 172.21.15.1:34720) 2024-04-26T01:57:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.657+0000 7ffa2fe00640 1 -- 172.21.15.1:0/473810075 learned_addr learned my addr 172.21.15.1:0/473810075 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.657+0000 7ffa2f400640 1 -- 172.21.15.1:0/473810075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63ef5140 msgr2=0x55aa63eff4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.657+0000 7ffa2f400640 1 --2- 172.21.15.1:0/473810075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63ef5140 0x55aa63eff4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.657+0000 7ffa2f400640 1 -- 172.21.15.1:0/473810075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef4820 msgr2=0x55aa63ef4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.657+0000 7ffa2f400640 1 --2- 172.21.15.1:0/473810075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef4820 0x55aa63ef4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.657+0000 7ffa2f400640 1 -- 172.21.15.1:0/473810075 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa63b6d210 con 0x55aa63effa00 2024-04-26T01:57:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.657+0000 7ffa2fe00640 1 --2- 172.21.15.1:0/473810075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63ef5140 0x55aa63eff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.657+0000 7ffa2f400640 1 --2- 172.21.15.1:0/473810075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa63effa00 0x55aa63f01df0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009b40 tx=0x7ffa1c0113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51a0fcae79d7b045 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.658+0000 7ffa25200640 1 -- 172.21.15.1:0/473810075 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c011e10 con 0x55aa63effa00 2024-04-26T01:57:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.658+0000 7ffa25200640 1 -- 172.21.15.1:0/473810075 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa1c012900 con 0x55aa63effa00 2024-04-26T01:57:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.658+0000 7ffa2cc00640 1 --2- 172.21.15.1:0/406250840 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ffa140a3d10 0x7ffa140a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.658+0000 7ffa25200640 1 -- 172.21.15.1:0/473810075 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c00f910 con 0x55aa63effa00 2024-04-26T01:57:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.658+0000 7ffa2cc00640 1 -- 172.21.15.1:0/406250840 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7ffa200f0ec0 con 0x7ffa301085a0 2024-04-26T01:57:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.658+0000 7ffa34a00640 1 --2- 172.21.15.1:0/406250840 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ffa140a3d10 0x7ffa140a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.658+0000 7ffa397ac740 1 -- 172.21.15.1:0/473810075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa63effa00 msgr2=0x55aa63f01df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.658+0000 7ffa397ac740 1 --2- 172.21.15.1:0/473810075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa63effa00 0x55aa63f01df0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009b40 tx=0x7ffa1c0113e0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.658+0000 7ffa397ac740 1 -- 172.21.15.1:0/473810075 shutdown_connections 2024-04-26T01:57:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.658+0000 7ffa397ac740 1 --2- 172.21.15.1:0/473810075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa63effa00 0x55aa63f01df0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.658+0000 7ffa397ac740 1 --2- 172.21.15.1:0/473810075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63ef5140 0x55aa63eff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.658+0000 7ffa397ac740 1 --2- 172.21.15.1:0/473810075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef4820 0x55aa63ef4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.658+0000 7ffa397ac740 1 -- 172.21.15.1:0/473810075 >> 172.21.15.1:0/473810075 conn(0x55aa63ef0d30 msgr2=0x55aa63ef1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:44.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.659+0000 7ffa397ac740 1 -- 172.21.15.1:0/473810075 shutdown_connections 2024-04-26T01:57:44.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.659+0000 7ffa397ac740 1 -- 172.21.15.1:0/473810075 wait complete. 2024-04-26T01:57:44.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.659+0000 7ffa34a00640 1 --2- 172.21.15.1:0/406250840 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ffa140a3d10 0x7ffa140a61d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffa28005fd0 tx=0x7ffa28009500 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:44.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.659+0000 7ffa397ac740 1 -- start start 2024-04-26T01:57:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.659+0000 7ffa397ac740 1 Processor -- start 2024-04-26T01:57:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.660+0000 7ffa397ac740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef4820 0x55aa63efc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.660+0000 7ffa397ac740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa63ef5140 0x55aa63efcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.660+0000 7ffa397ac740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63effa00 0x55aa63efd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.660+0000 7ffa397ac740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55aa63f02630 con 0x55aa63ef4820 2024-04-26T01:57:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.660+0000 7ffa397ac740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55aa63e14310 con 0x55aa63effa00 2024-04-26T01:57:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.660+0000 7ffa397ac740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55aa63f027b0 con 0x55aa63ef5140 2024-04-26T01:57:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.660+0000 7ffa397ac740 10 client.0 _ll_register_callbacks cb 0x55aa63de2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.660+0000 7ffa2fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef4820 0x55aa63efc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.660+0000 7ffa34a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63effa00 0x55aa63efd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.660+0000 7ffa34a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63effa00 0x55aa63efd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:34734/0 (socket says 172.21.15.1:34734) 2024-04-26T01:57:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.660+0000 7ffa2fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef4820 0x55aa63efc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48498/0 (socket says 172.21.15.1:48498) 2024-04-26T01:57:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.660+0000 7ffa34a00640 1 -- 172.21.15.1:0/1977612470 learned_addr learned my addr 172.21.15.1:0/1977612470 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.660+0000 7ffa2f400640 1 --2- 172.21.15.1:0/1977612470 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa63ef5140 0x55aa63efcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.661+0000 7ffa34a00640 1 -- 172.21.15.1:0/1977612470 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa63ef5140 msgr2=0x55aa63efcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.661+0000 7ffa34a00640 1 --2- 172.21.15.1:0/1977612470 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa63ef5140 0x55aa63efcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.661+0000 7ffa34a00640 1 -- 172.21.15.1:0/1977612470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef4820 msgr2=0x55aa63efc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.661+0000 7ffa34a00640 1 --2- 172.21.15.1:0/1977612470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef4820 0x55aa63efc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.661+0000 7ffa34a00640 1 -- 172.21.15.1:0/1977612470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa63c39620 con 0x55aa63effa00 2024-04-26T01:57:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.661+0000 7ffa34a00640 1 --2- 172.21.15.1:0/1977612470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63effa00 0x55aa63efd3e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffa28012010 tx=0x7ffa28004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=734648cac992aa2b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.661+0000 7ffa19600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:57:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.661+0000 7ffa19600640 1 -- 172.21.15.1:0/1977612470 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2800f7d0 con 0x55aa63effa00 2024-04-26T01:57:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.661+0000 7ffa397ac740 1 -- 172.21.15.1:0/1977612470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa1c012900 con 0x55aa63effa00 2024-04-26T01:57:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.661+0000 7ffa19600640 1 -- 172.21.15.1:0/1977612470 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7ffa2800f970 con 0x55aa63effa00 2024-04-26T01:57:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.662+0000 7ffa19600640 1 -- 172.21.15.1:0/1977612470 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ffa2800fb70 con 0x55aa63effa00 2024-04-26T01:57:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.662+0000 7ffa397ac740 10 client.25886 fetch_fsmap learned FSMap version 32 2024-04-26T01:57:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.662+0000 7ffa397ac740 1 -- 172.21.15.1:0/1977612470 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55aa63f0e900 con 0x55aa63effa00 2024-04-26T01:57:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.662+0000 7ffa19600640 1 -- 172.21.15.1:0/1977612470 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa1c012900 con 0x55aa63effa00 2024-04-26T01:57:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.662+0000 7ffa19600640 1 -- 172.21.15.1:0/1977612470 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7ffa28026070 con 0x55aa63effa00 2024-04-26T01:57:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.662+0000 7ffa397ac740 10 client.25886 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:57:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.662+0000 7ffa397ac740 10 client.25886 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:57:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.662+0000 7ffa397ac740 20 client.25886 populate_metadata read hostname 'smithi001' 2024-04-26T01:57:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.662+0000 7ffa397ac740 1 --2- 172.21.15.1:0/1977612470 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aa63f0fdc0 0x55aa63f121d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.662+0000 7ffa397ac740 4 client.25886 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:57:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.662+0000 7ffa397ac740 1 -- 172.21.15.1:0/1977612470 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa63e6e8b0 con 0x55aa63f0fdc0 2024-04-26T01:57:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.663+0000 7ffa2fe00640 1 --2- 172.21.15.1:0/1977612470 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aa63f0fdc0 0x55aa63f121d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.663+0000 7ffa2fe00640 1 --2- 172.21.15.1:0/1977612470 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aa63f0fdc0 0x55aa63f121d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fec159fe14f51685 server_cookie=5b46a94f8b1203b4 in_seq=0 out_seq=0 2024-04-26T01:57:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.663+0000 7ffa19600640 10 client.25886 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.668+0000 7ffa19600640 1 -- 172.21.15.1:0/1977612470 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aa63e6e8b0 con 0x55aa63f0fdc0 2024-04-26T01:57:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.668+0000 7ffa19600640 10 client.25886 handle_command_reply: tid=0 2024-04-26T01:57:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.668+0000 7ffa397ac740 1 client.25886 shutdown 2024-04-26T01:57:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.668+0000 7ffa397ac740 1 -- 172.21.15.1:0/1977612470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63effa00 msgr2=0x55aa63efd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.668+0000 7ffa397ac740 1 --2- 172.21.15.1:0/1977612470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63effa00 0x55aa63efd3e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffa28012010 tx=0x7ffa28004ab0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.668+0000 7ffa397ac740 1 -- 172.21.15.1:0/1977612470 shutdown_connections 2024-04-26T01:57:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.668+0000 7ffa397ac740 1 --2- 172.21.15.1:0/1977612470 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aa63f0fdc0 0x55aa63f121d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.668+0000 7ffa397ac740 1 --2- 172.21.15.1:0/1977612470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63effa00 0x55aa63efd3e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.668+0000 7ffa397ac740 1 --2- 172.21.15.1:0/1977612470 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa63ef5140 0x55aa63efcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.668+0000 7ffa397ac740 1 --2- 172.21.15.1:0/1977612470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef4820 0x55aa63efc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.668+0000 7ffa19600640 0 client.25886 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.669+0000 7ffa397ac740 1 -- 172.21.15.1:0/1977612470 >> 172.21.15.1:0/1977612470 conn(0x55aa63ef0d30 msgr2=0x55aa63ef6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.669+0000 7ffa397ac740 1 -- 172.21.15.1:0/1977612470 shutdown_connections 2024-04-26T01:57:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.669+0000 7ffa397ac740 1 -- 172.21.15.1:0/1977612470 wait complete. 2024-04-26T01:57:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.669+0000 7ffa397ac740 20 client.25886 trim_cache size 0 max 16384 2024-04-26T01:57:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.685+0000 7ffa397ac740 1 Processor -- start 2024-04-26T01:57:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.685+0000 7ffa397ac740 1 -- start start 2024-04-26T01:57:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.686+0000 7ffa397ac740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa63ef4820 0x55aa63f0bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.686+0000 7ffa397ac740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef5140 0x55aa63f0b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.686+0000 7ffa397ac740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63effa00 0x55aa63ef7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.686+0000 7ffa397ac740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffa301b6c10 con 0x55aa63ef5140 2024-04-26T01:57:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.686+0000 7ffa397ac740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffa301b44e0 con 0x55aa63effa00 2024-04-26T01:57:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.686+0000 7ffa397ac740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffa301b2330 con 0x55aa63ef4820 2024-04-26T01:57:44.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.686+0000 7ffa2f400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef5140 0x55aa63f0b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.686+0000 7ffa2f400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef5140 0x55aa63f0b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48504/0 (socket says 172.21.15.1:48504) 2024-04-26T01:57:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.686+0000 7ffa2f400640 1 -- 172.21.15.1:0/2801284017 learned_addr learned my addr 172.21.15.1:0/2801284017 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.686+0000 7ffa34a00640 1 --2- 172.21.15.1:0/2801284017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63effa00 0x55aa63ef7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.686+0000 7ffa2fe00640 1 --2- 172.21.15.1:0/2801284017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa63ef4820 0x55aa63f0bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.687+0000 7ffa2f400640 1 -- 172.21.15.1:0/2801284017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa63ef4820 msgr2=0x55aa63f0bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.687+0000 7ffa2f400640 1 --2- 172.21.15.1:0/2801284017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa63ef4820 0x55aa63f0bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.687+0000 7ffa2f400640 1 -- 172.21.15.1:0/2801284017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63effa00 msgr2=0x55aa63ef7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.687+0000 7ffa2f400640 1 --2- 172.21.15.1:0/2801284017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63effa00 0x55aa63ef7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.687+0000 7ffa2f400640 1 -- 172.21.15.1:0/2801284017 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa63efb910 con 0x55aa63ef5140 2024-04-26T01:57:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.687+0000 7ffa2f400640 1 --2- 172.21.15.1:0/2801284017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef5140 0x55aa63f0b4f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c005d80 tx=0x7ffa1c009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c97de821ac3889e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.687+0000 7ffa1be00640 1 -- 172.21.15.1:0/2801284017 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c029020 con 0x55aa63ef5140 2024-04-26T01:57:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.687+0000 7ffa1be00640 1 -- 172.21.15.1:0/2801284017 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa1c009770 con 0x55aa63ef5140 2024-04-26T01:57:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.687+0000 7ffa1be00640 1 -- 172.21.15.1:0/2801284017 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c003b20 con 0x55aa63ef5140 2024-04-26T01:57:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.688+0000 7ffa397ac740 1 -- 172.21.15.1:0/2801284017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef5140 msgr2=0x55aa63f0b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.688+0000 7ffa397ac740 1 --2- 172.21.15.1:0/2801284017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef5140 0x55aa63f0b4f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c005d80 tx=0x7ffa1c009550 comp rx=0 tx=0).stop 2024-04-26T01:57:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.688+0000 7ffa397ac740 1 -- 172.21.15.1:0/2801284017 shutdown_connections 2024-04-26T01:57:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.688+0000 7ffa397ac740 1 --2- 172.21.15.1:0/2801284017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63effa00 0x55aa63ef7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.688+0000 7ffa397ac740 1 --2- 172.21.15.1:0/2801284017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef5140 0x55aa63f0b4f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.688+0000 7ffa397ac740 1 --2- 172.21.15.1:0/2801284017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa63ef4820 0x55aa63f0bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.688+0000 7ffa397ac740 1 -- 172.21.15.1:0/2801284017 >> 172.21.15.1:0/2801284017 conn(0x55aa63ef0d30 msgr2=0x55aa63ef1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.688+0000 7ffa397ac740 1 -- 172.21.15.1:0/2801284017 shutdown_connections 2024-04-26T01:57:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.688+0000 7ffa397ac740 1 -- 172.21.15.1:0/2801284017 wait complete. 2024-04-26T01:57:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.689+0000 7ffa397ac740 1 -- start start 2024-04-26T01:57:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.689+0000 7ffa397ac740 1 Processor -- start 2024-04-26T01:57:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.689+0000 7ffa397ac740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa63ef4820 0x55aa63f07f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.689+0000 7ffa397ac740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef5140 0x55aa63f052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.689+0000 7ffa397ac740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63effa00 0x55aa63f059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.689+0000 7ffa397ac740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffa301b29a0 con 0x55aa63ef5140 2024-04-26T01:57:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.689+0000 7ffa397ac740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffa301b3e10 con 0x55aa63effa00 2024-04-26T01:57:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.689+0000 7ffa397ac740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffa301b3070 con 0x55aa63ef4820 2024-04-26T01:57:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.689+0000 7ffa397ac740 10 client.0 _ll_register_callbacks cb 0x7ffa28003ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.690+0000 7ffa2f400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef5140 0x55aa63f052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.690+0000 7ffa2fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa63ef4820 0x55aa63f07f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.690+0000 7ffa2f400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef5140 0x55aa63f052c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48518/0 (socket says 172.21.15.1:48518) 2024-04-26T01:57:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.690+0000 7ffa2f400640 1 -- 172.21.15.1:0/3189470107 learned_addr learned my addr 172.21.15.1:0/3189470107 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.690+0000 7ffa34a00640 1 --2- 172.21.15.1:0/3189470107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63effa00 0x55aa63f059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.690+0000 7ffa2f400640 1 -- 172.21.15.1:0/3189470107 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa63ef4820 msgr2=0x55aa63f07f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.690+0000 7ffa2f400640 1 --2- 172.21.15.1:0/3189470107 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa63ef4820 0x55aa63f07f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.690+0000 7ffa2f400640 1 -- 172.21.15.1:0/3189470107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63effa00 msgr2=0x55aa63f059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.690+0000 7ffa2f400640 1 --2- 172.21.15.1:0/3189470107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63effa00 0x55aa63f059e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.690+0000 7ffa2f400640 1 -- 172.21.15.1:0/3189470107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa63f75360 con 0x55aa63ef5140 2024-04-26T01:57:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.690+0000 7ffa2f400640 1 --2- 172.21.15.1:0/3189470107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef5140 0x55aa63f052c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c000c00 tx=0x7ffa1c003660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef700539894ebf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.690+0000 7ffa19600640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:57:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.691+0000 7ffa19600640 1 -- 172.21.15.1:0/3189470107 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c032070 con 0x55aa63ef5140 2024-04-26T01:57:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.691+0000 7ffa19600640 1 -- 172.21.15.1:0/3189470107 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7ffa1c0039a0 con 0x55aa63ef5140 2024-04-26T01:57:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.691+0000 7ffa397ac740 1 -- 172.21.15.1:0/3189470107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aa63e68320 con 0x55aa63ef5140 2024-04-26T01:57:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.691+0000 7ffa19600640 1 -- 172.21.15.1:0/3189470107 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ffa1c03e050 con 0x55aa63ef5140 2024-04-26T01:57:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.691+0000 7ffa397ac740 10 client.15744 fetch_fsmap learned FSMap version 32 2024-04-26T01:57:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.691+0000 7ffa397ac740 1 -- 172.21.15.1:0/3189470107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55aa63f764f0 con 0x55aa63ef5140 2024-04-26T01:57:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.691+0000 7ffa19600640 1 -- 172.21.15.1:0/3189470107 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55aa63e68320 con 0x55aa63ef5140 2024-04-26T01:57:44.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.691+0000 7fdfc5600700 1 -- 172.21.15.1:0/268005163 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x56157c533b30 con 0x56157c5c40e0 2024-04-26T01:57:44.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.691+0000 7fdfc5600700 10 client.25874 handle_command_reply: tid=0 2024-04-26T01:57:44.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.691+0000 7fdfec8a6b80 1 client.25874 shutdown 2024-04-26T01:57:44.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.692+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/268005163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c544180 msgr2=0x56157c547af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:44.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.692+0000 7fdfec8a6b80 1 --2- 172.21.15.1:0/268005163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c544180 0x56157c547af0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400b3f0 tx=0x7fdfd4005470 comp rx=0 tx=0).stop 2024-04-26T01:57:44.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.692+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/268005163 shutdown_connections 2024-04-26T01:57:44.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.692+0000 7fdfec8a6b80 1 --2- 172.21.15.1:0/268005163 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56157c5c40e0 0x56157c5c6500 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.692+0000 7fdfec8a6b80 1 --2- 172.21.15.1:0/268005163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56157c544180 0x56157c547af0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.692+0000 7fdfec8a6b80 1 --2- 172.21.15.1:0/268005163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56157c5389b0 0x56157c547070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.692+0000 7fdfec8a6b80 1 --2- 172.21.15.1:0/268005163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56157c5392d0 0x56157c5475b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.692+0000 7fdfc5600700 0 client.25874 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:44.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.692+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/268005163 >> 172.21.15.1:0/268005163 conn(0x56157c467730 msgr2=0x56157c53bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:44.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.692+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/268005163 shutdown_connections 2024-04-26T01:57:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.692+0000 7ffa19600640 1 -- 172.21.15.1:0/3189470107 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7ffa1c00fc40 con 0x55aa63ef5140 2024-04-26T01:57:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.692+0000 7ffa397ac740 10 client.15744 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:57:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.692+0000 7ffa397ac740 10 client.15744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:57:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.692+0000 7ffa397ac740 20 client.15744 populate_metadata read hostname 'smithi001' 2024-04-26T01:57:44.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.692+0000 7fdfec8a6b80 1 -- 172.21.15.1:0/268005163 wait complete. 2024-04-26T01:57:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.692+0000 7ffa397ac740 1 --2- 172.21.15.1:0/3189470107 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aa63f769a0 0x55aa63f78dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.692+0000 7ffa397ac740 4 client.15744 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:57:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.692+0000 7ffa397ac740 1 -- 172.21.15.1:0/3189470107 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aa63e62190 con 0x55aa63f769a0 2024-04-26T01:57:44.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.692+0000 7fdfec8a6b80 20 client.25874 trim_cache size 0 max 16384 2024-04-26T01:57:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.692+0000 7ffa2fe00640 1 --2- 172.21.15.1:0/3189470107 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aa63f769a0 0x55aa63f78dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.696 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T01:57:44.696 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 32, 2024-04-26T01:57:44.696 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24518, 2024-04-26T01:57:44.696 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 14, 2024-04-26T01:57:44.696 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-04-26T01:57:44.696 INFO:teuthology.orchestra.run.smithi001.stdout: "5f41eab3": { 2024-04-26T01:57:44.697 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 14, 2024-04-26T01:57:44.697 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-04-26T01:57:44.697 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:57:44.697 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-04-26T01:57:44.697 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:57:44.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T01:57:44.697 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-04-26T01:57:44.697 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 70.0, 2024-04-26T01:57:44.697 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-04-26T01:57:44.697 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-04-26T01:57:44.697 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-04-26T01:57:44.697 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:57:44.697 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-04-26T01:57:44.697 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:57:44.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:57:44.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:57:44.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:57:44.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:57:44.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:57:44.698 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T01:57:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.693+0000 7ffa2fe00640 1 --2- 172.21.15.1:0/3189470107 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aa63f769a0 0x55aa63f78dc0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2fb34a4288fd7c6 server_cookie=34118a9806fe90a5 in_seq=0 out_seq=0 2024-04-26T01:57:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.693+0000 7ffa19600640 10 client.15744 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:44.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.693+0000 7fdfe2000700 1 -- 172.21.15.1:0/2189398041 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fdfc00a3ed0 msgr2=0x7fdfc00a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:44.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.693+0000 7fdfe2000700 1 --2- 172.21.15.1:0/2189398041 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fdfc00a3ed0 0x7fdfc00a6390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc009be0 tx=0x7fdfcc009260 comp rx=0 tx=0).stop 2024-04-26T01:57:44.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.693+0000 7fdfe2000700 1 -- 172.21.15.1:0/2189398041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfdc06b100 msgr2=0x7fdfdc19b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:44.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.693+0000 7fdfe2000700 1 --2- 172.21.15.1:0/2189398041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfdc06b100 0x7fdfdc19b720 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdfc800e9a0 tx=0x7fdfc800ecb0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.693+0000 7fdfe2000700 1 -- 172.21.15.1:0/2189398041 shutdown_connections 2024-04-26T01:57:44.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.693+0000 7fdfe2000700 1 --2- 172.21.15.1:0/2189398041 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fdfc00a3ed0 0x7fdfc00a6390 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.693+0000 7fdfe2000700 1 --2- 172.21.15.1:0/2189398041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfdc108a60 0x7fdfdc19fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.693+0000 7fdfe2000700 1 --2- 172.21.15.1:0/2189398041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfdc06a740 0x7fdfdc19b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.693+0000 7fdfe2000700 1 --2- 172.21.15.1:0/2189398041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfdc06b100 0x7fdfdc19b720 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.693+0000 7fdfe2000700 1 -- 172.21.15.1:0/2189398041 >> 172.21.15.1:0/2189398041 conn(0x7fdfdc0788e0 msgr2=0x7fdfdc10dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:44.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.694+0000 7fdfe2000700 1 -- 172.21.15.1:0/2189398041 shutdown_connections 2024-04-26T01:57:44.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:44.694+0000 7fdfe2000700 1 -- 172.21.15.1:0/2189398041 wait complete. 2024-04-26T01:57:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.704+0000 7ffa19600640 1 -- 172.21.15.1:0/3189470107 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aa63e62190 con 0x55aa63f769a0 2024-04-26T01:57:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.704+0000 7ffa19600640 10 client.15744 handle_command_reply: tid=0 2024-04-26T01:57:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.704+0000 7ffa397ac740 1 client.15744 shutdown 2024-04-26T01:57:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.704+0000 7ffa397ac740 1 -- 172.21.15.1:0/3189470107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef5140 msgr2=0x55aa63f052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.704+0000 7ffa397ac740 1 --2- 172.21.15.1:0/3189470107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef5140 0x55aa63f052c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c000c00 tx=0x7ffa1c003660 comp rx=0 tx=0).stop 2024-04-26T01:57:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.705+0000 7ffa397ac740 1 -- 172.21.15.1:0/3189470107 shutdown_connections 2024-04-26T01:57:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.705+0000 7ffa397ac740 1 --2- 172.21.15.1:0/3189470107 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aa63f769a0 0x55aa63f78dc0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.705+0000 7ffa397ac740 1 --2- 172.21.15.1:0/3189470107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa63effa00 0x55aa63f059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.705+0000 7ffa397ac740 1 --2- 172.21.15.1:0/3189470107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa63ef5140 0x55aa63f052c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.705+0000 7ffa397ac740 1 --2- 172.21.15.1:0/3189470107 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa63ef4820 0x55aa63f07f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.705+0000 7ffa397ac740 1 -- 172.21.15.1:0/3189470107 >> 172.21.15.1:0/3189470107 conn(0x55aa63ef0d30 msgr2=0x55aa63efd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.705+0000 7ffa19600640 0 client.15744 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.705+0000 7ffa397ac740 1 -- 172.21.15.1:0/3189470107 shutdown_connections 2024-04-26T01:57:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.705+0000 7ffa397ac740 1 -- 172.21.15.1:0/3189470107 wait complete. 2024-04-26T01:57:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.705+0000 7ffa397ac740 20 client.15744 trim_cache size 0 max 16384 2024-04-26T01:57:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:57:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:57:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:57:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:57:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.706+0000 7ffa36a00640 1 -- 172.21.15.1:0/406250840 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ffa140a3d10 msgr2=0x7ffa140a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.706+0000 7ffa36a00640 1 --2- 172.21.15.1:0/406250840 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ffa140a3d10 0x7ffa140a61d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffa28005fd0 tx=0x7ffa28009500 comp rx=0 tx=0).stop 2024-04-26T01:57:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.706+0000 7ffa36a00640 1 -- 172.21.15.1:0/406250840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa301085a0 msgr2=0x7ffa3019ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.706+0000 7ffa36a00640 1 --2- 172.21.15.1:0/406250840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa301085a0 0x7ffa3019ee20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ffa20005ab0 tx=0x7ffa20003b90 comp rx=0 tx=0).stop 2024-04-26T01:57:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.706+0000 7ffa36a00640 1 -- 172.21.15.1:0/406250840 shutdown_connections 2024-04-26T01:57:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.706+0000 7ffa36a00640 1 --2- 172.21.15.1:0/406250840 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ffa140a3d10 0x7ffa140a61d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.707+0000 7ffa36a00640 1 --2- 172.21.15.1:0/406250840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa3010e660 0x7ffa301a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.707+0000 7ffa36a00640 1 --2- 172.21.15.1:0/406250840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa30108f60 0x7ffa3019f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.707+0000 7ffa36a00640 1 --2- 172.21.15.1:0/406250840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa301085a0 0x7ffa3019ee20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.707+0000 7ffa36a00640 1 -- 172.21.15.1:0/406250840 >> 172.21.15.1:0/406250840 conn(0x7ffa30076460 msgr2=0x7ffa30077ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.707+0000 7ffa36a00640 1 -- 172.21.15.1:0/406250840 shutdown_connections 2024-04-26T01:57:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:44.707+0000 7ffa36a00640 1 -- 172.21.15.1:0/406250840 wait complete. 2024-04-26T01:57:45.566 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5f41eab3', quiesce took 9.1 seconds. Will release after: 10 2024-04-26T01:57:46.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:45 smithi097 ceph-mon[32853]: pgmap v187: 97 pgs: 97 active+clean; 42 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.6 MiB/s wr, 431 op/s 2024-04-26T01:57:46.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:45 smithi173 ceph-mon[32553]: pgmap v187: 97 pgs: 97 active+clean; 42 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.6 MiB/s wr, 431 op/s 2024-04-26T01:57:46.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:45 smithi001 ceph-mon[27822]: pgmap v187: 97 pgs: 97 active+clean; 42 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.6 MiB/s wr, 431 op/s 2024-04-26T01:57:48.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:47 smithi097 ceph-mon[32853]: pgmap v188: 97 pgs: 97 active+clean; 42 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.7 MiB/s wr, 283 op/s 2024-04-26T01:57:48.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:47 smithi173 ceph-mon[32553]: pgmap v188: 97 pgs: 97 active+clean; 42 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.7 MiB/s wr, 283 op/s 2024-04-26T01:57:48.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:47 smithi001 ceph-mon[27822]: pgmap v188: 97 pgs: 97 active+clean; 42 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.7 MiB/s wr, 283 op/s 2024-04-26T01:57:50.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:49 smithi097 ceph-mon[32853]: pgmap v189: 97 pgs: 97 active+clean; 42 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.7 MiB/s wr, 283 op/s 2024-04-26T01:57:50.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:49 smithi173 ceph-mon[32553]: pgmap v189: 97 pgs: 97 active+clean; 42 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.7 MiB/s wr, 283 op/s 2024-04-26T01:57:50.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:49 smithi001 ceph-mon[27822]: pgmap v189: 97 pgs: 97 active+clean; 42 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.7 MiB/s wr, 283 op/s 2024-04-26T01:57:52.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:51 smithi097 ceph-mon[32853]: pgmap v190: 97 pgs: 97 active+clean; 42 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.7 MiB/s wr, 286 op/s 2024-04-26T01:57:52.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:51 smithi173 ceph-mon[32553]: pgmap v190: 97 pgs: 97 active+clean; 42 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.7 MiB/s wr, 286 op/s 2024-04-26T01:57:52.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:51 smithi001 ceph-mon[27822]: pgmap v190: 97 pgs: 97 active+clean; 42 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.7 MiB/s wr, 286 op/s 2024-04-26T01:57:54.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:53 smithi097 ceph-mon[32853]: pgmap v191: 97 pgs: 97 active+clean; 42 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 880 KiB/s wr, 141 op/s 2024-04-26T01:57:54.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:53 smithi173 ceph-mon[32553]: pgmap v191: 97 pgs: 97 active+clean; 42 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 880 KiB/s wr, 141 op/s 2024-04-26T01:57:54.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:53 smithi001 ceph-mon[27822]: pgmap v191: 97 pgs: 97 active+clean; 42 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 880 KiB/s wr, 141 op/s 2024-04-26T01:57:55.567 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:57:56.032 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:57:56.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:55 smithi097 ceph-mon[32853]: pgmap v192: 97 pgs: 97 active+clean; 42 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 883 KiB/s wr, 144 op/s 2024-04-26T01:57:56.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:55 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:57:56.238 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:55 smithi001 ceph-mon[27822]: pgmap v192: 97 pgs: 97 active+clean; 42 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 883 KiB/s wr, 144 op/s 2024-04-26T01:57:56.239 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:55 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:57:56.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:55 smithi173 ceph-mon[32553]: pgmap v192: 97 pgs: 97 active+clean; 42 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 883 KiB/s wr, 144 op/s 2024-04-26T01:57:56.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:55 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:57:57.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:56 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:57:57.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:56 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:57:57.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:56 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:57:57.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:56 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:57:57.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:56 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:57:57.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:56 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:57:57.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:56 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:57:57.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:56 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:57:57.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:56 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:57:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.849+0000 7f6a9ae00640 1 -- 172.21.15.1:0/304467686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a941038f0 msgr2=0x7f6a9410bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.849+0000 7f6a9ae00640 1 --2- 172.21.15.1:0/304467686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a941038f0 0x7f6a9410bdd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009900 tx=0x7f6a8002a480 comp rx=0 tx=0).stop 2024-04-26T01:57:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.849+0000 7f6a9ae00640 1 -- 172.21.15.1:0/304467686 shutdown_connections 2024-04-26T01:57:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.849+0000 7f6a9ae00640 1 --2- 172.21.15.1:0/304467686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a9410c3a0 0x7f6a9410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.849+0000 7f6a9ae00640 1 --2- 172.21.15.1:0/304467686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a941038f0 0x7f6a9410bdd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.849+0000 7f6a9ae00640 1 --2- 172.21.15.1:0/304467686 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a94102fd0 0x7f6a941033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.849+0000 7f6a9ae00640 1 -- 172.21.15.1:0/304467686 >> 172.21.15.1:0/304467686 conn(0x7f6a940fc730 msgr2=0x7f6a940feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.849+0000 7f6a9ae00640 1 -- 172.21.15.1:0/304467686 shutdown_connections 2024-04-26T01:57:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.849+0000 7f6a9ae00640 1 -- 172.21.15.1:0/304467686 wait complete. 2024-04-26T01:57:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.850+0000 7f6a9ae00640 1 Processor -- start 2024-04-26T01:57:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.850+0000 7f6a9ae00640 1 -- start start 2024-04-26T01:57:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.851+0000 7f6a9ae00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a94102fd0 0x7f6a94100c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.851+0000 7f6a9ae00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a941038f0 0x7f6a94101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.851+0000 7f6a9ae00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a9410c3a0 0x7f6a941016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.851+0000 7f6a9ae00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94112030 con 0x7f6a941038f0 2024-04-26T01:57:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.851+0000 7f6a9ae00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94111eb0 con 0x7f6a9410c3a0 2024-04-26T01:57:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.851+0000 7f6a9ae00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941121b0 con 0x7f6a94102fd0 2024-04-26T01:57:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.851+0000 7f6a99000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a941038f0 0x7f6a94101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.851+0000 7f6a99000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a941038f0 0x7f6a94101160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36702/0 (socket says 172.21.15.1:36702) 2024-04-26T01:57:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.851+0000 7f6a99000640 1 -- 172.21.15.1:0/3005471440 learned_addr learned my addr 172.21.15.1:0/3005471440 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.851+0000 7f6a99a00640 1 --2- 172.21.15.1:0/3005471440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a94102fd0 0x7f6a94100c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.851+0000 7f6a9a400640 1 --2- 172.21.15.1:0/3005471440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a9410c3a0 0x7f6a941016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.852+0000 7f6a99a00640 1 -- 172.21.15.1:0/3005471440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a9410c3a0 msgr2=0x7f6a941016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.852+0000 7f6a99a00640 1 --2- 172.21.15.1:0/3005471440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a9410c3a0 0x7f6a941016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.852+0000 7f6a99a00640 1 -- 172.21.15.1:0/3005471440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a941038f0 msgr2=0x7f6a94101160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.852+0000 7f6a99a00640 1 --2- 172.21.15.1:0/3005471440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a941038f0 0x7f6a94101160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.852+0000 7f6a99a00640 1 -- 172.21.15.1:0/3005471440 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a941a6a80 con 0x7f6a94102fd0 2024-04-26T01:57:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.852+0000 7f6a99000640 1 --2- 172.21.15.1:0/3005471440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a941038f0 0x7f6a94101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.852+0000 7f6a99a00640 1 --2- 172.21.15.1:0/3005471440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a94102fd0 0x7f6a94100c20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400e9f0 tx=0x7f6a8400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.852+0000 7f6a92000640 1 -- 172.21.15.1:0/3005471440 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8400ccc0 con 0x7f6a94102fd0 2024-04-26T01:57:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.852+0000 7f6a9ae00640 1 -- 172.21.15.1:0/3005471440 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a941a6d70 con 0x7f6a94102fd0 2024-04-26T01:57:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.852+0000 7f6a92000640 1 -- 172.21.15.1:0/3005471440 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a84005020 con 0x7f6a94102fd0 2024-04-26T01:57:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.852+0000 7f6a92000640 1 -- 172.21.15.1:0/3005471440 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8400ce60 con 0x7f6a94102fd0 2024-04-26T01:57:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.853+0000 7f6a9ae00640 1 -- 172.21.15.1:0/3005471440 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a941a7200 con 0x7f6a94102fd0 2024-04-26T01:57:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.857+0000 7f6a9f1c5740 1 Processor -- start 2024-04-26T01:57:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.857+0000 7f6a92000640 1 -- 172.21.15.1:0/3005471440 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f6a840051d0 con 0x7f6a94102fd0 2024-04-26T01:57:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.858+0000 7f6a9f1c5740 1 -- start start 2024-04-26T01:57:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.858+0000 7f6a9f1c5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ec8d0 0x5614203eccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.858+0000 7f6a9f1c5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614203ed1f0 0x5614203ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.858+0000 7f6a9f1c5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614204003a0 0x561420402790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.858+0000 7f6a9f1c5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561420352900 con 0x5614203ed1f0 2024-04-26T01:57:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.858+0000 7f6a9f1c5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5614202f8ef0 con 0x5614204003a0 2024-04-26T01:57:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.858+0000 7f6a9f1c5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56142034e8c0 con 0x5614203ec8d0 2024-04-26T01:57:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.858+0000 7f6a9a400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ec8d0 0x5614203eccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.858+0000 7f6a9a400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ec8d0 0x5614203eccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:43992/0 (socket says 172.21.15.1:43992) 2024-04-26T01:57:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.859+0000 7f6a9a400640 1 -- 172.21.15.1:0/3076876522 learned_addr learned my addr 172.21.15.1:0/3076876522 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.859+0000 7f6a99a00640 1 --2- 172.21.15.1:0/3076876522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614203ed1f0 0x5614203ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.859+0000 7f6a99000640 1 --2- 172.21.15.1:0/3076876522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614204003a0 0x561420402790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.859+0000 7f6a9a400640 1 -- 172.21.15.1:0/3076876522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614204003a0 msgr2=0x561420402790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.859+0000 7f6a9a400640 1 --2- 172.21.15.1:0/3076876522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614204003a0 0x561420402790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.859+0000 7f6a9a400640 1 -- 172.21.15.1:0/3076876522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614203ed1f0 msgr2=0x5614203ffe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.859+0000 7f6a9a400640 1 --2- 172.21.15.1:0/3076876522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614203ed1f0 0x5614203ffe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.859+0000 7f6a9a400640 1 -- 172.21.15.1:0/3076876522 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561420065210 con 0x5614203ec8d0 2024-04-26T01:57:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.859+0000 7f6a99000640 1 --2- 172.21.15.1:0/3076876522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614204003a0 0x561420402790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:57:57.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.859+0000 7f6a9a400640 1 --2- 172.21.15.1:0/3076876522 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ec8d0 0x5614203eccb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009f60 tx=0x7f6a8c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62fc440333badfa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:57.860 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.860+0000 7f6a8aa00640 1 -- 172.21.15.1:0/3076876522 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c01e070 con 0x5614203ec8d0 2024-04-26T01:57:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.860+0000 7f6a8aa00640 1 -- 172.21.15.1:0/3076876522 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a8c004030 con 0x5614203ec8d0 2024-04-26T01:57:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.860+0000 7f6a8aa00640 1 -- 172.21.15.1:0/3076876522 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c00f6e0 con 0x5614203ec8d0 2024-04-26T01:57:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.860+0000 7f6a92000640 1 --2- 172.21.15.1:0/3005471440 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6a780a3d30 0x7f6a780a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.860+0000 7f6a92000640 1 -- 172.21.15.1:0/3005471440 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f6a840d0710 con 0x7f6a94102fd0 2024-04-26T01:57:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.861+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/3076876522 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ec8d0 msgr2=0x5614203eccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.861+0000 7f6a9f1c5740 1 --2- 172.21.15.1:0/3076876522 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ec8d0 0x5614203eccb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009f60 tx=0x7f6a8c009f90 comp rx=0 tx=0).stop 2024-04-26T01:57:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.861+0000 7f6a99a00640 1 --2- 172.21.15.1:0/3005471440 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6a780a3d30 0x7f6a780a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.861+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/3076876522 shutdown_connections 2024-04-26T01:57:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.861+0000 7f6a9f1c5740 1 --2- 172.21.15.1:0/3076876522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614204003a0 0x561420402790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.861+0000 7f6a9f1c5740 1 --2- 172.21.15.1:0/3076876522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614203ed1f0 0x5614203ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.861+0000 7f6a9f1c5740 1 --2- 172.21.15.1:0/3076876522 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ec8d0 0x5614203eccb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.861+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/3076876522 >> 172.21.15.1:0/3076876522 conn(0x5614203e8ec0 msgr2=0x5614203e92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.861+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/3076876522 shutdown_connections 2024-04-26T01:57:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.861+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/3076876522 wait complete. 2024-04-26T01:57:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.862+0000 7f6a99a00640 1 --2- 172.21.15.1:0/3005471440 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6a780a3d30 0x7f6a780a61f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6a840cdf10 tx=0x7f6a84009f00 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.862+0000 7f6a9f1c5740 1 -- start start 2024-04-26T01:57:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.862+0000 7f6a9f1c5740 1 Processor -- start 2024-04-26T01:57:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.863+0000 7f6a9f1c5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614203ec8d0 0x5614203fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.863+0000 7f6a9f1c5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ed1f0 0x5614203fd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.863+0000 7f6a9f1c5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614204003a0 0x5614203fde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.863+0000 7f6a9f1c5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561420402fd0 con 0x5614203ec8d0 2024-04-26T01:57:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.863+0000 7f6a9f1c5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56142030c310 con 0x5614204003a0 2024-04-26T01:57:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.863+0000 7f6a9f1c5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561420403150 con 0x5614203ed1f0 2024-04-26T01:57:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.863+0000 7f6a9f1c5740 10 client.0 _ll_register_callbacks cb 0x5614202dacb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.863+0000 7f6a9a400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614203ec8d0 0x5614203fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.863+0000 7f6a99000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ed1f0 0x5614203fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.863+0000 7f6a99000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ed1f0 0x5614203fd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:44002/0 (socket says 172.21.15.1:44002) 2024-04-26T01:57:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.863+0000 7f6a99000640 1 -- 172.21.15.1:0/2585763641 learned_addr learned my addr 172.21.15.1:0/2585763641 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.863+0000 7f6a9a400640 1 --2- 172.21.15.1:0/2585763641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614204003a0 0x5614203fde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.864+0000 7f6a99000640 1 -- 172.21.15.1:0/2585763641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614204003a0 msgr2=0x5614203fde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.864+0000 7f6a99000640 1 --2- 172.21.15.1:0/2585763641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614204003a0 0x5614203fde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.864+0000 7f6a99000640 1 -- 172.21.15.1:0/2585763641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614203ec8d0 msgr2=0x5614203fd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.864+0000 7f6a99000640 1 --2- 172.21.15.1:0/2585763641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614203ec8d0 0x5614203fd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.864+0000 7f6a99000640 1 -- 172.21.15.1:0/2585763641 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561420131620 con 0x5614203ed1f0 2024-04-26T01:57:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.864+0000 7f6a99000640 1 --2- 172.21.15.1:0/2585763641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ed1f0 0x5614203fd900 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6a8002aa70 tx=0x7f6a800038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e39baf3c75b4730 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.864+0000 7f6a7f400640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:57:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.864+0000 7f6a7f400640 1 -- 172.21.15.1:0/2585763641 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80035040 con 0x5614203ed1f0 2024-04-26T01:57:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.865+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/2585763641 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a8c004030 con 0x5614203ed1f0 2024-04-26T01:57:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.865+0000 7f6a7f400640 1 -- 172.21.15.1:0/2585763641 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f6a80003c90 con 0x5614203ed1f0 2024-04-26T01:57:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.865+0000 7f6a7f400640 1 -- 172.21.15.1:0/2585763641 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6a80003e90 con 0x5614203ed1f0 2024-04-26T01:57:57.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.865+0000 7f6a9f1c5740 10 client.25927 fetch_fsmap learned FSMap version 32 2024-04-26T01:57:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.865+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/2585763641 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56142040f2c0 con 0x5614203ed1f0 2024-04-26T01:57:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.865+0000 7f6a7f400640 1 -- 172.21.15.1:0/2585763641 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a8c004030 con 0x5614203ed1f0 2024-04-26T01:57:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.865+0000 7f6a7f400640 1 -- 172.21.15.1:0/2585763641 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f6a8003d570 con 0x5614203ed1f0 2024-04-26T01:57:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.866+0000 7f6a9f1c5740 10 client.25927 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:57:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.866+0000 7f6a9f1c5740 10 client.25927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:57:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.866+0000 7f6a9f1c5740 20 client.25927 populate_metadata read hostname 'smithi001' 2024-04-26T01:57:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.866+0000 7f6a9f1c5740 1 --2- 172.21.15.1:0/2585763641 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561420410780 0x561420412b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.866+0000 7f6a9f1c5740 4 client.25927 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:57:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.866+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/2585763641 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614203668b0 con 0x561420410780 2024-04-26T01:57:57.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.866+0000 7f6a9a400640 1 --2- 172.21.15.1:0/2585763641 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561420410780 0x561420412b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.867+0000 7f6a9a400640 1 --2- 172.21.15.1:0/2585763641 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561420410780 0x561420412b70 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25f8dd1953145e79 server_cookie=c6b42701a00c673e in_seq=0 out_seq=0 2024-04-26T01:57:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.867+0000 7f6a7f400640 10 client.25927 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.867+0000 7f6a7f400640 1 -- 172.21.15.1:0/2585763641 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5614203668b0 con 0x561420410780 2024-04-26T01:57:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.867+0000 7f6a7f400640 10 client.25927 handle_command_reply: tid=0 2024-04-26T01:57:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.867+0000 7f6a9f1c5740 1 client.25927 shutdown 2024-04-26T01:57:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.868+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/2585763641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ed1f0 msgr2=0x5614203fd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.868+0000 7f6a9f1c5740 1 --2- 172.21.15.1:0/2585763641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ed1f0 0x5614203fd900 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6a8002aa70 tx=0x7f6a800038a0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.868+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/2585763641 shutdown_connections 2024-04-26T01:57:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.868+0000 7f6a9f1c5740 1 --2- 172.21.15.1:0/2585763641 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561420410780 0x561420412b70 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.868+0000 7f6a7f400640 0 client.25927 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.868+0000 7f6a9f1c5740 1 --2- 172.21.15.1:0/2585763641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614204003a0 0x5614203fde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.868+0000 7f6a9f1c5740 1 --2- 172.21.15.1:0/2585763641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ed1f0 0x5614203fd900 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.868+0000 7f6a9f1c5740 1 --2- 172.21.15.1:0/2585763641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614203ec8d0 0x5614203fd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.868+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/2585763641 >> 172.21.15.1:0/2585763641 conn(0x5614203e8ec0 msgr2=0x5614203eed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.868+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/2585763641 shutdown_connections 2024-04-26T01:57:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.868+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/2585763641 wait complete. 2024-04-26T01:57:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.868+0000 7f6a9f1c5740 20 client.25927 trim_cache size 0 max 16384 2024-04-26T01:57:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.885+0000 7f6a9f1c5740 1 Processor -- start 2024-04-26T01:57:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.885+0000 7f6a9f1c5740 1 -- start start 2024-04-26T01:57:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.886+0000 7f6a9f1c5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614203ec8d0 0x56142040be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.886+0000 7f6a9f1c5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ed1f0 0x5614203ff300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.886+0000 7f6a9f1c5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614204003a0 0x561420418480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.886+0000 7f6a9f1c5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941ae1b0 con 0x5614204003a0 2024-04-26T01:57:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.886+0000 7f6a9f1c5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941aba80 con 0x5614203ec8d0 2024-04-26T01:57:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.886+0000 7f6a9f1c5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941a98f0 con 0x5614203ed1f0 2024-04-26T01:57:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.886+0000 7f6a9a400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614203ec8d0 0x56142040be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.886+0000 7f6a9a400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614203ec8d0 0x56142040be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:38906/0 (socket says 172.21.15.1:38906) 2024-04-26T01:57:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.886+0000 7f6a9a400640 1 -- 172.21.15.1:0/3220221999 learned_addr learned my addr 172.21.15.1:0/3220221999 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.886+0000 7f6a9a400640 1 --2- 172.21.15.1:0/3220221999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614204003a0 0x561420418480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.887+0000 7f6a9a400640 1 -- 172.21.15.1:0/3220221999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ed1f0 msgr2=0x5614203ff300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T01:57:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.887+0000 7f6a9a400640 1 --2- 172.21.15.1:0/3220221999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ed1f0 0x5614203ff300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.887+0000 7f6a9a400640 1 -- 172.21.15.1:0/3220221999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614204003a0 msgr2=0x561420418480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.887+0000 7f6a9a400640 1 --2- 172.21.15.1:0/3220221999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614204003a0 0x561420418480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.887+0000 7f6a9a400640 1 -- 172.21.15.1:0/3220221999 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614203ef3f0 con 0x5614203ec8d0 2024-04-26T01:57:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.887+0000 7f6a9a400640 1 --2- 172.21.15.1:0/3220221999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614203ec8d0 0x56142040be90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009b80 tx=0x7f6a8c004990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7d6b017c02e9cd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.888+0000 7f6a8a000640 1 -- 172.21.15.1:0/3220221999 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c01e040 con 0x5614203ec8d0 2024-04-26T01:57:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.888+0000 7f6a8a000640 1 -- 172.21.15.1:0/3220221999 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a8c004570 con 0x5614203ec8d0 2024-04-26T01:57:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.888+0000 7f6a8a000640 1 -- 172.21.15.1:0/3220221999 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c00fab0 con 0x5614203ec8d0 2024-04-26T01:57:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.889+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/3220221999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614203ec8d0 msgr2=0x56142040be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.889+0000 7f6a9f1c5740 1 --2- 172.21.15.1:0/3220221999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614203ec8d0 0x56142040be90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009b80 tx=0x7f6a8c004990 comp rx=0 tx=0).stop 2024-04-26T01:57:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.889+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/3220221999 shutdown_connections 2024-04-26T01:57:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.889+0000 7f6a9f1c5740 1 --2- 172.21.15.1:0/3220221999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614204003a0 0x561420418480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.889+0000 7f6a9f1c5740 1 --2- 172.21.15.1:0/3220221999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ed1f0 0x5614203ff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.889+0000 7f6a9f1c5740 1 --2- 172.21.15.1:0/3220221999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614203ec8d0 0x56142040be90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.889+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/3220221999 >> 172.21.15.1:0/3220221999 conn(0x5614203e8ec0 msgr2=0x5614203febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.889+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/3220221999 shutdown_connections 2024-04-26T01:57:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.889+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/3220221999 wait complete. 2024-04-26T01:57:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.890+0000 7f6a9f1c5740 1 -- start start 2024-04-26T01:57:57.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.890+0000 7f6a9f1c5740 1 Processor -- start 2024-04-26T01:57:57.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.890+0000 7f6a9f1c5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614203ec8d0 0x5614204083b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.890+0000 7f6a9f1c5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ed1f0 0x5614204088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.890+0000 7f6a9f1c5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614204003a0 0x5614204034a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.890+0000 7f6a9f1c5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941a9f40 con 0x5614204003a0 2024-04-26T01:57:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.890+0000 7f6a9f1c5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941ab3b0 con 0x5614203ec8d0 2024-04-26T01:57:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.890+0000 7f6a9f1c5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941aa610 con 0x5614203ed1f0 2024-04-26T01:57:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.890+0000 7f6a9f1c5740 10 client.0 _ll_register_callbacks cb 0x7f6a80004430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.891+0000 7f6a9a400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614203ec8d0 0x5614204083b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.891+0000 7f6a99000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ed1f0 0x5614204088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.891+0000 7f6a9a400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614203ec8d0 0x5614204083b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:38912/0 (socket says 172.21.15.1:38912) 2024-04-26T01:57:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.891+0000 7f6a9a400640 1 -- 172.21.15.1:0/3875545946 learned_addr learned my addr 172.21.15.1:0/3875545946 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.891+0000 7f6a9a400640 1 --2- 172.21.15.1:0/3875545946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614204003a0 0x5614204034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.891+0000 7f6a99000640 1 -- 172.21.15.1:0/3875545946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614203ec8d0 msgr2=0x5614204083b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.891+0000 7f6a99000640 1 --2- 172.21.15.1:0/3875545946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614203ec8d0 0x5614204083b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.891+0000 7f6a99000640 1 -- 172.21.15.1:0/3875545946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614204003a0 msgr2=0x5614204034a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.891+0000 7f6a99000640 1 --2- 172.21.15.1:0/3875545946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614204003a0 0x5614204034a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.891+0000 7f6a99000640 1 -- 172.21.15.1:0/3875545946 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561420403cd0 con 0x5614203ed1f0 2024-04-26T01:57:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.891+0000 7f6a99000640 1 --2- 172.21.15.1:0/3875545946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ed1f0 0x5614204088f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x56142040c3d0 tx=0x7f6a8003d680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eca78340a099b707 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.891+0000 7f6a7f400640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:57:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.892+0000 7f6a7f400640 1 -- 172.21.15.1:0/3875545946 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80052040 con 0x5614203ed1f0 2024-04-26T01:57:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.892+0000 7f6a7f400640 1 -- 172.21.15.1:0/3875545946 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f6a80003b10 con 0x5614203ed1f0 2024-04-26T01:57:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.892+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/3875545946 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a8c004570 con 0x5614203ed1f0 2024-04-26T01:57:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.892+0000 7f6a7f400640 1 -- 172.21.15.1:0/3875545946 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6a80005850 con 0x5614203ed1f0 2024-04-26T01:57:57.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.892+0000 7f6a9f1c5740 10 client.25930 fetch_fsmap learned FSMap version 32 2024-04-26T01:57:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.892+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/3875545946 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561420488930 con 0x5614203ed1f0 2024-04-26T01:57:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.892+0000 7f6a7f400640 1 -- 172.21.15.1:0/3875545946 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a8c004570 con 0x5614203ed1f0 2024-04-26T01:57:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.893+0000 7f6a7f400640 1 -- 172.21.15.1:0/3875545946 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f6a80031ae0 con 0x5614203ed1f0 2024-04-26T01:57:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.893+0000 7f6a9f1c5740 10 client.25930 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:57:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.893+0000 7f6a9f1c5740 10 client.25930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:57:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.893+0000 7f6a9f1c5740 20 client.25930 populate_metadata read hostname 'smithi001' 2024-04-26T01:57:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.893+0000 7f6a9f1c5740 1 --2- 172.21.15.1:0/3875545946 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561420410780 0x56142048b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.893+0000 7f6a9f1c5740 4 client.25930 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:57:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.893+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/3875545946 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561420360320 con 0x561420410780 2024-04-26T01:57:57.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.893+0000 7f6a9a400640 1 --2- 172.21.15.1:0/3875545946 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561420410780 0x56142048b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.894+0000 7f6a9a400640 1 --2- 172.21.15.1:0/3875545946 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561420410780 0x56142048b0f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a24e12d45e36a0f4 server_cookie=24b731aa5b5f0219 in_seq=0 out_seq=0 2024-04-26T01:57:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.894+0000 7f6a7f400640 10 client.25930 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.918+0000 7f6a7f400640 1 -- 172.21.15.1:0/3875545946 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561420360320 con 0x561420410780 2024-04-26T01:57:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.918+0000 7f6a7f400640 10 client.25930 handle_command_reply: tid=0 2024-04-26T01:57:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.918+0000 7f6a9f1c5740 1 client.25930 shutdown 2024-04-26T01:57:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.918+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/3875545946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ed1f0 msgr2=0x5614204088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:57.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.918+0000 7f6a9f1c5740 1 --2- 172.21.15.1:0/3875545946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ed1f0 0x5614204088f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x56142040c3d0 tx=0x7f6a8003d680 comp rx=0 tx=0).stop 2024-04-26T01:57:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.918+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/3875545946 shutdown_connections 2024-04-26T01:57:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.918+0000 7f6a9f1c5740 1 --2- 172.21.15.1:0/3875545946 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561420410780 0x56142048b0f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.918+0000 7f6a9f1c5740 1 --2- 172.21.15.1:0/3875545946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614204003a0 0x5614204034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.918+0000 7f6a9f1c5740 1 --2- 172.21.15.1:0/3875545946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5614203ed1f0 0x5614204088f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.918+0000 7f6a9f1c5740 1 --2- 172.21.15.1:0/3875545946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5614203ec8d0 0x5614204083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.918+0000 7f6a7f400640 0 client.25930 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:57:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.918+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/3875545946 >> 172.21.15.1:0/3875545946 conn(0x5614203e8ec0 msgr2=0x5614203ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.919+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/3875545946 shutdown_connections 2024-04-26T01:57:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.919+0000 7f6a9f1c5740 1 -- 172.21.15.1:0/3875545946 wait complete. 2024-04-26T01:57:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.919+0000 7f6a9f1c5740 20 client.25930 trim_cache size 0 max 16384 2024-04-26T01:57:57.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:57:57.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:57:57.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:57:57.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:57:57.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.919+0000 7f6a9ae00640 1 -- 172.21.15.1:0/3005471440 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6a780a3d30 msgr2=0x7f6a780a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.919+0000 7f6a9ae00640 1 --2- 172.21.15.1:0/3005471440 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6a780a3d30 0x7f6a780a61f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6a840cdf10 tx=0x7f6a84009f00 comp rx=0 tx=0).stop 2024-04-26T01:57:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.919+0000 7f6a9ae00640 1 -- 172.21.15.1:0/3005471440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a94102fd0 msgr2=0x7f6a94100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.919+0000 7f6a9ae00640 1 --2- 172.21.15.1:0/3005471440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a94102fd0 0x7f6a94100c20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400e9f0 tx=0x7f6a8400edf0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.920+0000 7f6a9ae00640 1 -- 172.21.15.1:0/3005471440 shutdown_connections 2024-04-26T01:57:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.920+0000 7f6a9ae00640 1 --2- 172.21.15.1:0/3005471440 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6a780a3d30 0x7f6a780a61f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.920+0000 7f6a9ae00640 1 --2- 172.21.15.1:0/3005471440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a9410c3a0 0x7f6a941016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.920+0000 7f6a9ae00640 1 --2- 172.21.15.1:0/3005471440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a941038f0 0x7f6a94101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.920+0000 7f6a9ae00640 1 --2- 172.21.15.1:0/3005471440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a94102fd0 0x7f6a94100c20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.920+0000 7f6a9ae00640 1 -- 172.21.15.1:0/3005471440 >> 172.21.15.1:0/3005471440 conn(0x7f6a940fc730 msgr2=0x7f6a940fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.920+0000 7f6a9ae00640 1 -- 172.21.15.1:0/3005471440 shutdown_connections 2024-04-26T01:57:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:57:57.920+0000 7f6a9ae00640 1 -- 172.21.15.1:0/3005471440 wait complete. 2024-04-26T01:57:58.030 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:57 smithi001 ceph-mon[27822]: pgmap v193: 97 pgs: 97 active+clean; 42 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2024-04-26T01:57:58.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.198+0000 7fba9b000700 1 -- 172.21.15.1:0/1787037161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba94108a50 msgr2=0x7fba94108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:58.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.198+0000 7fba9b000700 1 --2- 172.21.15.1:0/1787037161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba94108a50 0x7fba94108e30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fba84009990 tx=0x7fba84009ca0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.198+0000 7fba9b000700 1 -- 172.21.15.1:0/1787037161 shutdown_connections 2024-04-26T01:57:58.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.198+0000 7fba9b000700 1 --2- 172.21.15.1:0/1787037161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba94103450 0x7fba9410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.198+0000 7fba9b000700 1 --2- 172.21.15.1:0/1787037161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba94108a50 0x7fba94108e30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.198+0000 7fba9b000700 1 --2- 172.21.15.1:0/1787037161 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba94102a90 0x7fba94102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.198+0000 7fba9b000700 1 -- 172.21.15.1:0/1787037161 >> 172.21.15.1:0/1787037161 conn(0x7fba94079ba0 msgr2=0x7fba94079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:58.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.199+0000 7fba9b000700 1 -- 172.21.15.1:0/1787037161 shutdown_connections 2024-04-26T01:57:58.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.199+0000 7fba9b000700 1 -- 172.21.15.1:0/1787037161 wait complete. 2024-04-26T01:57:58.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.200+0000 7fba9b000700 1 Processor -- start 2024-04-26T01:57:58.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.200+0000 7fba9b000700 1 -- start start 2024-04-26T01:57:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.200+0000 7fba9b000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba94102a90 0x7fba9419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.200+0000 7fba9b000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba94103450 0x7fba9419b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.200+0000 7fba9b000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba94108a50 0x7fba9419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.201+0000 7fba9b000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fba94112c30 con 0x7fba94102a90 2024-04-26T01:57:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.201+0000 7fba9b000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fba94112ab0 con 0x7fba94103450 2024-04-26T01:57:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.201+0000 7fba9b000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fba94112db0 con 0x7fba94108a50 2024-04-26T01:57:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.201+0000 7fba93e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba94102a90 0x7fba9419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.201+0000 7fba93400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba94103450 0x7fba9419b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.201+0000 7fba93e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba94102a90 0x7fba9419b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36746/0 (socket says 172.21.15.1:36746) 2024-04-26T01:57:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.201+0000 7fba93e00700 1 -- 172.21.15.1:0/549432083 learned_addr learned my addr 172.21.15.1:0/549432083 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:57:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.201+0000 7fba99000700 1 --2- 172.21.15.1:0/549432083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba94108a50 0x7fba9419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.201+0000 7fba93400700 1 -- 172.21.15.1:0/549432083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba94108a50 msgr2=0x7fba9419fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:58.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.201+0000 7fba93400700 1 --2- 172.21.15.1:0/549432083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba94108a50 0x7fba9419fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.201+0000 7fba93400700 1 -- 172.21.15.1:0/549432083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba94102a90 msgr2=0x7fba9419b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:58.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.201+0000 7fba93400700 1 --2- 172.21.15.1:0/549432083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba94102a90 0x7fba9419b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.201+0000 7fba93400700 1 -- 172.21.15.1:0/549432083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba941a03b0 con 0x7fba94103450 2024-04-26T01:57:58.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.201+0000 7fba93e00700 1 --2- 172.21.15.1:0/549432083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba94102a90 0x7fba9419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:58.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.202+0000 7fba93400700 1 --2- 172.21.15.1:0/549432083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba94103450 0x7fba9419b8e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fba8000e9f0 tx=0x7fba8000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:58.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.202+0000 7fba90c00700 1 -- 172.21.15.1:0/549432083 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8000cb70 con 0x7fba94103450 2024-04-26T01:57:58.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.202+0000 7fba90c00700 1 -- 172.21.15.1:0/549432083 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba8000ee90 con 0x7fba94103450 2024-04-26T01:57:58.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.202+0000 7fba9b000700 1 -- 172.21.15.1:0/549432083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba941a06a0 con 0x7fba94103450 2024-04-26T01:57:58.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.202+0000 7fba90c00700 1 -- 172.21.15.1:0/549432083 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80003d40 con 0x7fba94103450 2024-04-26T01:57:58.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.202+0000 7fba9b000700 1 -- 172.21.15.1:0/549432083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba941a7f20 con 0x7fba94103450 2024-04-26T01:57:58.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.203+0000 7fba9b000700 1 -- 172.21.15.1:0/549432083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba94069730 con 0x7fba94103450 2024-04-26T01:57:58.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.205+0000 7fba90c00700 1 -- 172.21.15.1:0/549432083 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fba80003ee0 con 0x7fba94103450 2024-04-26T01:57:58.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.207+0000 7fba90c00700 1 --2- 172.21.15.1:0/549432083 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fba780a3e30 0x7fba780a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:58.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.207+0000 7fba90c00700 1 -- 172.21.15.1:0/549432083 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7fba800d0890 con 0x7fba94103450 2024-04-26T01:57:58.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.207+0000 7fba93e00700 1 --2- 172.21.15.1:0/549432083 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fba780a3e30 0x7fba780a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:58.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.208+0000 7fba93e00700 1 --2- 172.21.15.1:0/549432083 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fba780a3e30 0x7fba780a62f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fba84006010 tx=0x7fba8400b280 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:58.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.211+0000 7fba90c00700 1 -- 172.21.15.1:0/549432083 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba8009ced0 con 0x7fba94103450 2024-04-26T01:57:58.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:57 smithi097 ceph-mon[32853]: pgmap v193: 97 pgs: 97 active+clean; 42 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2024-04-26T01:57:58.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:57 smithi173 ceph-mon[32553]: pgmap v193: 97 pgs: 97 active+clean; 42 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2024-04-26T01:57:58.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.493+0000 7fba9b000700 1 -- 172.21.15.1:0/549432083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba941a8150 con 0x7fba94103450 2024-04-26T01:57:58.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.494+0000 7fba90c00700 1 -- 172.21.15.1:0/549432083 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11838 (secure 0 0 0) 0x7fba8009c7d0 con 0x7fba94103450 2024-04-26T01:57:58.493 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:57:58.493 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:55:49.645549+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25414":{"gid":25414,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/453427322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":453427322},{"type":"v1","addr":"172.21.15.173:6835","nonce":453427322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24499,24514,24518]},"id":1}]} 2024-04-26T01:57:58.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.497+0000 7fba9b000700 1 -- 172.21.15.1:0/549432083 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fba780a3e30 msgr2=0x7fba780a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:58.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.497+0000 7fba9b000700 1 --2- 172.21.15.1:0/549432083 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fba780a3e30 0x7fba780a62f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fba84006010 tx=0x7fba8400b280 comp rx=0 tx=0).stop 2024-04-26T01:57:58.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.497+0000 7fba9b000700 1 -- 172.21.15.1:0/549432083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba94103450 msgr2=0x7fba9419b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:58.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.497+0000 7fba9b000700 1 --2- 172.21.15.1:0/549432083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba94103450 0x7fba9419b8e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fba8000e9f0 tx=0x7fba8000ed00 comp rx=0 tx=0).stop 2024-04-26T01:57:58.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.497+0000 7fba9b000700 1 -- 172.21.15.1:0/549432083 shutdown_connections 2024-04-26T01:57:58.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.497+0000 7fba9b000700 1 --2- 172.21.15.1:0/549432083 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fba780a3e30 0x7fba780a62f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.497+0000 7fba9b000700 1 --2- 172.21.15.1:0/549432083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba94108a50 0x7fba9419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.497+0000 7fba9b000700 1 --2- 172.21.15.1:0/549432083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba94102a90 0x7fba9419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.497+0000 7fba9b000700 1 --2- 172.21.15.1:0/549432083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba94103450 0x7fba9419b8e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.497+0000 7fba9b000700 1 -- 172.21.15.1:0/549432083 >> 172.21.15.1:0/549432083 conn(0x7fba94079ba0 msgr2=0x7fba9410df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:58.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.497+0000 7fba9b000700 1 -- 172.21.15.1:0/549432083 shutdown_connections 2024-04-26T01:57:58.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:57:58.497+0000 7fba9b000700 1 -- 172.21.15.1:0/549432083 wait complete. 2024-04-26T01:57:58.497 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 32 2024-04-26T01:57:59.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:58 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/549432083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:57:59.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:58 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/549432083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:57:59.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:58 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/549432083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:57:59.333 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24518 quiesce db --set-id 5f41eab3 --release --await' 2024-04-26T01:57:59.333 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24518 quiesce db --set-id 5f41eab3 --release --await 2024-04-26T01:57:59.808 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:58:00.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:57:59 smithi097 ceph-mon[32853]: pgmap v194: 97 pgs: 97 active+clean; 42 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2024-04-26T01:58:00.256 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:57:59 smithi001 ceph-mon[27822]: pgmap v194: 97 pgs: 97 active+clean; 42 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2024-04-26T01:58:00.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:57:59 smithi173 ceph-mon[32553]: pgmap v194: 97 pgs: 97 active+clean; 42 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2024-04-26T01:58:02.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.005+0000 7f5cb6200700 1 -- 172.21.15.1:0/2604892069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cb0105760 msgr2=0x7f5cb01097b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:02.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.005+0000 7f5cb6200700 1 --2- 172.21.15.1:0/2604892069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cb0105760 0x7f5cb01097b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009920 tx=0x7f5c9c009c30 comp rx=0 tx=0).stop 2024-04-26T01:58:02.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.005+0000 7f5cb6200700 1 -- 172.21.15.1:0/2604892069 shutdown_connections 2024-04-26T01:58:02.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.005+0000 7f5cb6200700 1 --2- 172.21.15.1:0/2604892069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5cb0109e90 0x7f5cb0111a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.005+0000 7f5cb6200700 1 --2- 172.21.15.1:0/2604892069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cb0104db0 0x7f5cb0105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.005+0000 7f5cb6200700 1 --2- 172.21.15.1:0/2604892069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cb0105760 0x7f5cb01097b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.005+0000 7f5cb6200700 1 -- 172.21.15.1:0/2604892069 >> 172.21.15.1:0/2604892069 conn(0x7f5cb0100680 msgr2=0x7f5cb0102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:02.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.005+0000 7f5cb6200700 1 -- 172.21.15.1:0/2604892069 shutdown_connections 2024-04-26T01:58:02.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.005+0000 7f5cb6200700 1 -- 172.21.15.1:0/2604892069 wait complete. 2024-04-26T01:58:02.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.007+0000 7f5cb6200700 1 Processor -- start 2024-04-26T01:58:02.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.007+0000 7f5cb6200700 1 -- start start 2024-04-26T01:58:02.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.007+0000 7f5cb6200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cb0104db0 0x7f5cb019b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.007+0000 7f5cb6200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5cb0105760 0x7f5cb019b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.007+0000 7f5cb6200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cb0109e90 0x7f5cb019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.007+0000 7f5cb6200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb0114d60 con 0x7f5cb0105760 2024-04-26T01:58:02.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.007+0000 7f5cb6200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb0114be0 con 0x7f5cb0104db0 2024-04-26T01:58:02.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.007+0000 7f5cb6200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb0114ee0 con 0x7f5cb0109e90 2024-04-26T01:58:02.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.008+0000 7f5cb4e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cb0104db0 0x7f5cb019b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.008+0000 7f5cb5800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cb0109e90 0x7f5cb019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.008+0000 7f5cb4e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cb0104db0 0x7f5cb019b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:38968/0 (socket says 172.21.15.1:38968) 2024-04-26T01:58:02.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.008+0000 7f5cb5800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cb0109e90 0x7f5cb019fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:44038/0 (socket says 172.21.15.1:44038) 2024-04-26T01:58:02.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.008+0000 7f5cb4e00700 1 -- 172.21.15.1:0/1746367553 learned_addr learned my addr 172.21.15.1:0/1746367553 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:02.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.008+0000 7f5cafe00700 1 --2- 172.21.15.1:0/1746367553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5cb0105760 0x7f5cb019b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.008+0000 7f5cb4e00700 1 -- 172.21.15.1:0/1746367553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cb0109e90 msgr2=0x7f5cb019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:02.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.008+0000 7f5cb4e00700 1 --2- 172.21.15.1:0/1746367553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cb0109e90 0x7f5cb019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.008+0000 7f5cb4e00700 1 -- 172.21.15.1:0/1746367553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5cb0105760 msgr2=0x7f5cb019b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:02.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.008+0000 7f5cb4e00700 1 --2- 172.21.15.1:0/1746367553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5cb0105760 0x7f5cb019b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.008+0000 7f5cb4e00700 1 -- 172.21.15.1:0/1746367553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb01a03a0 con 0x7f5cb0104db0 2024-04-26T01:58:02.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.008+0000 7f5cb5800700 1 --2- 172.21.15.1:0/1746367553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cb0109e90 0x7f5cb019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:02.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.009+0000 7f5cb4e00700 1 --2- 172.21.15.1:0/1746367553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cb0104db0 0x7f5cb019b3e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400e9a0 tx=0x7f5ca400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:02.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.009+0000 7f5cad600700 1 -- 172.21.15.1:0/1746367553 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca400ca10 con 0x7f5cb0104db0 2024-04-26T01:58:02.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.009+0000 7f5cad600700 1 -- 172.21.15.1:0/1746367553 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ca400cbb0 con 0x7f5cb0104db0 2024-04-26T01:58:02.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.009+0000 7f5cb6200700 1 -- 172.21.15.1:0/1746367553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb01a0690 con 0x7f5cb0104db0 2024-04-26T01:58:02.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.009+0000 7f5cad600700 1 -- 172.21.15.1:0/1746367553 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4018680 con 0x7f5cb0104db0 2024-04-26T01:58:02.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.009+0000 7f5cb6200700 1 -- 172.21.15.1:0/1746367553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb01a7f10 con 0x7f5cb0104db0 2024-04-26T01:58:02.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.012+0000 7f5cad600700 1 -- 172.21.15.1:0/1746367553 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f5ca4010430 con 0x7f5cb0104db0 2024-04-26T01:58:02.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.014+0000 7f5cad600700 1 --2- 172.21.15.1:0/1746367553 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f5c980a3e30 0x7f5c980a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.014+0000 7f5cad600700 1 -- 172.21.15.1:0/1746367553 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f5ca40d0960 con 0x7f5cb0104db0 2024-04-26T01:58:02.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.014+0000 7f5cafe00700 1 --2- 172.21.15.1:0/1746367553 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f5c980a3e30 0x7f5c980a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.014+0000 7f5cc226fb80 1 Processor -- start 2024-04-26T01:58:02.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.014+0000 7f5cc226fb80 1 -- start start 2024-04-26T01:58:02.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.015+0000 7f5cc226fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291dbea0 0x55d0291dc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.015+0000 7f5cc226fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dc7c0 0x55d0291ef940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.015+0000 7f5cc226fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291efe80 0x55d0291f2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.015+0000 7f5cc226fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d02912ef90 con 0x55d0291efe80 2024-04-26T01:58:02.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.015+0000 7f5cc226fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d0290d5830 con 0x55d0291dbea0 2024-04-26T01:58:02.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.015+0000 7f5cc226fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d02912af50 con 0x55d0291dc7c0 2024-04-26T01:58:02.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.015+0000 7f5cafe00700 1 --2- 172.21.15.1:0/1746367553 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f5c980a3e30 0x7f5c980a62f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0098f0 tx=0x7f5c9c019040 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:02.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.015+0000 7f5cb4e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291dbea0 0x55d0291dc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.015+0000 7f5cb5800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291efe80 0x55d0291f2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.015+0000 7f5cb4e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291dbea0 0x55d0291dc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:38984/0 (socket says 172.21.15.1:38984) 2024-04-26T01:58:02.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.015+0000 7f5cb4e00700 1 -- 172.21.15.1:0/1944728115 learned_addr learned my addr 172.21.15.1:0/1944728115 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:02.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.016+0000 7f5cafe00700 1 --2- 172.21.15.1:0/1944728115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dc7c0 0x55d0291ef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.016+0000 7f5cb4e00700 1 -- 172.21.15.1:0/1944728115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dc7c0 msgr2=0x55d0291ef940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:02.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.016+0000 7f5cb4e00700 1 --2- 172.21.15.1:0/1944728115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dc7c0 0x55d0291ef940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.016+0000 7f5cb4e00700 1 -- 172.21.15.1:0/1944728115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291efe80 msgr2=0x55d0291f2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:02.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.016+0000 7f5cb4e00700 1 --2- 172.21.15.1:0/1944728115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291efe80 0x55d0291f2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.016+0000 7f5cb4e00700 1 -- 172.21.15.1:0/1944728115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0290cb780 con 0x55d0291dbea0 2024-04-26T01:58:02.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.016+0000 7f5cb4e00700 1 --2- 172.21.15.1:0/1944728115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291dbea0 0x55d0291dc280 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4099270 tx=0x7f5ca409c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1632f5f47ecf760 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:02.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.016+0000 7f5ca2000700 1 -- 172.21.15.1:0/1944728115 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40a5510 con 0x55d0291dbea0 2024-04-26T01:58:02.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.016+0000 7f5ca2000700 1 -- 172.21.15.1:0/1944728115 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ca409aba0 con 0x55d0291dbea0 2024-04-26T01:58:02.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.016+0000 7f5ca2000700 1 -- 172.21.15.1:0/1944728115 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca409de30 con 0x55d0291dbea0 2024-04-26T01:58:02.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.017+0000 7f5cc226fb80 1 -- 172.21.15.1:0/1944728115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291dbea0 msgr2=0x55d0291dc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:02.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.017+0000 7f5cc226fb80 1 --2- 172.21.15.1:0/1944728115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291dbea0 0x55d0291dc280 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4099270 tx=0x7f5ca409c8a0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.017+0000 7f5cc226fb80 1 -- 172.21.15.1:0/1944728115 shutdown_connections 2024-04-26T01:58:02.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.017+0000 7f5cc226fb80 1 --2- 172.21.15.1:0/1944728115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291efe80 0x55d0291f2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.017+0000 7f5cc226fb80 1 --2- 172.21.15.1:0/1944728115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291dbea0 0x55d0291dc280 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.017+0000 7f5cc226fb80 1 --2- 172.21.15.1:0/1944728115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dc7c0 0x55d0291ef940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.017+0000 7f5cc226fb80 1 -- 172.21.15.1:0/1944728115 >> 172.21.15.1:0/1944728115 conn(0x55d02910b050 msgr2=0x55d0291d83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:02.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.017+0000 7f5cc226fb80 1 -- 172.21.15.1:0/1944728115 shutdown_connections 2024-04-26T01:58:02.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.017+0000 7f5cc226fb80 1 -- 172.21.15.1:0/1944728115 wait complete. 2024-04-26T01:58:02.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.018+0000 7f5cc226fb80 1 -- start start 2024-04-26T01:58:02.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.018+0000 7f5cc226fb80 1 Processor -- start 2024-04-26T01:58:02.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.019+0000 7f5cc226fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291dbea0 0x55d0291ef6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.019+0000 7f5cc226fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dc7c0 0x55d0291ec9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.019+0000 7f5cc226fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291efe80 0x55d0291ecee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.019+0000 7f5cc226fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d0291f2910 con 0x55d0291dbea0 2024-04-26T01:58:02.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.019+0000 7f5cc226fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d0290e8960 con 0x55d0291efe80 2024-04-26T01:58:02.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.019+0000 7f5cc226fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d0291f2a90 con 0x55d0291dc7c0 2024-04-26T01:58:02.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.019+0000 7f5cc226fb80 10 client.0 _ll_register_callbacks cb 0x55d0290c7820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:02.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.019+0000 7f5cb4e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291dbea0 0x55d0291ef6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.019+0000 7f5cb4e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291dbea0 0x55d0291ef6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36772/0 (socket says 172.21.15.1:36772) 2024-04-26T01:58:02.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.019+0000 7f5cafe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dc7c0 0x55d0291ec9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.019+0000 7f5cb4e00700 1 -- 172.21.15.1:0/2921139141 learned_addr learned my addr 172.21.15.1:0/2921139141 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:02.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.019+0000 7f5cb5800700 1 --2- 172.21.15.1:0/2921139141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291efe80 0x55d0291ecee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.020+0000 7f5cb4e00700 1 -- 172.21.15.1:0/2921139141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dc7c0 msgr2=0x55d0291ec9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:02.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.020+0000 7f5cb4e00700 1 --2- 172.21.15.1:0/2921139141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dc7c0 0x55d0291ec9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.020+0000 7f5cb4e00700 1 -- 172.21.15.1:0/2921139141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291efe80 msgr2=0x55d0291ecee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:02.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.020+0000 7f5cb4e00700 1 --2- 172.21.15.1:0/2921139141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291efe80 0x55d0291ecee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.020+0000 7f5cb4e00700 1 -- 172.21.15.1:0/2921139141 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d02907c9d0 con 0x55d0291dbea0 2024-04-26T01:58:02.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.020+0000 7f5cb5800700 1 --2- 172.21.15.1:0/2921139141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291efe80 0x55d0291ecee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:02.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.020+0000 7f5cb4e00700 1 --2- 172.21.15.1:0/2921139141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291dbea0 0x55d0291ef6a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40a4fe0 tx=0x7f5ca40cdff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40c8465b361e17c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:02.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.020+0000 7f5c96a00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:58:02.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.020+0000 7f5c96a00700 1 -- 172.21.15.1:0/2921139141 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40ba070 con 0x55d0291dbea0 2024-04-26T01:58:02.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.020+0000 7f5c96a00700 1 -- 172.21.15.1:0/2921139141 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f5ca409dbd0 con 0x55d0291dbea0 2024-04-26T01:58:02.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.020+0000 7f5cc226fb80 1 -- 172.21.15.1:0/2921139141 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ca409aba0 con 0x55d0291dbea0 2024-04-26T01:58:02.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.021+0000 7f5c96a00700 1 -- 172.21.15.1:0/2921139141 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5ca409ddd0 con 0x55d0291dbea0 2024-04-26T01:58:02.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.021+0000 7f5cc226fb80 10 client.15780 fetch_fsmap learned FSMap version 32 2024-04-26T01:58:02.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.021+0000 7f5cc226fb80 1 -- 172.21.15.1:0/2921139141 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d0290c5bf0 con 0x55d0291dbea0 2024-04-26T01:58:02.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.021+0000 7f5c96a00700 1 -- 172.21.15.1:0/2921139141 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ca409aba0 con 0x55d0291dbea0 2024-04-26T01:58:02.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.021+0000 7f5c96a00700 1 -- 172.21.15.1:0/2921139141 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f5ca40985f0 con 0x55d0291dbea0 2024-04-26T01:58:02.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.021+0000 7f5cc226fb80 10 client.15780 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:58:02.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.021+0000 7f5cc226fb80 10 client.15780 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:58:02.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.021+0000 7f5cc226fb80 20 client.15780 populate_metadata read hostname 'smithi001' 2024-04-26T01:58:02.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.021+0000 7f5cc226fb80 1 --2- 172.21.15.1:0/2921139141 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d0291ff520 0x55d029201910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.021+0000 7f5cc226fb80 4 client.15780 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:58:02.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.021+0000 7f5cc226fb80 1 -- 172.21.15.1:0/2921139141 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d02913c780 con 0x55d0291ff520 2024-04-26T01:58:02.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.022+0000 7f5cafe00700 1 --2- 172.21.15.1:0/2921139141 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d0291ff520 0x55d029201910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.022+0000 7f5cafe00700 1 --2- 172.21.15.1:0/2921139141 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d0291ff520 0x55d029201910 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c40c124458d96d3d server_cookie=31a312af4bb3eb72 in_seq=0 out_seq=0 2024-04-26T01:58:02.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.022+0000 7f5c96a00700 10 client.15780 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:02.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.023+0000 7f5c96a00700 1 -- 172.21.15.1:0/2921139141 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d02913c780 con 0x55d0291ff520 2024-04-26T01:58:02.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.023+0000 7f5c96a00700 10 client.15780 handle_command_reply: tid=0 2024-04-26T01:58:02.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.023+0000 7f5cc226fb80 1 client.15780 shutdown 2024-04-26T01:58:02.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.023+0000 7f5cc226fb80 1 -- 172.21.15.1:0/2921139141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291dbea0 msgr2=0x55d0291ef6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:02.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.023+0000 7f5cc226fb80 1 --2- 172.21.15.1:0/2921139141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291dbea0 0x55d0291ef6a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40a4fe0 tx=0x7f5ca40cdff0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.024+0000 7f5cc226fb80 1 -- 172.21.15.1:0/2921139141 shutdown_connections 2024-04-26T01:58:02.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.024+0000 7f5cc226fb80 1 --2- 172.21.15.1:0/2921139141 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d0291ff520 0x55d029201910 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.024+0000 7f5cc226fb80 1 --2- 172.21.15.1:0/2921139141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291efe80 0x55d0291ecee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.024+0000 7f5cc226fb80 1 --2- 172.21.15.1:0/2921139141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291dbea0 0x55d0291ef6a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.024+0000 7f5cc226fb80 1 --2- 172.21.15.1:0/2921139141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dc7c0 0x55d0291ec9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.024+0000 7f5cc226fb80 1 -- 172.21.15.1:0/2921139141 >> 172.21.15.1:0/2921139141 conn(0x55d02910b050 msgr2=0x55d0291d7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:02.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.024+0000 7f5c96a00700 0 client.15780 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:02.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.024+0000 7f5cc226fb80 1 -- 172.21.15.1:0/2921139141 shutdown_connections 2024-04-26T01:58:02.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.024+0000 7f5cc226fb80 1 -- 172.21.15.1:0/2921139141 wait complete. 2024-04-26T01:58:02.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.024+0000 7f5cc226fb80 20 client.15780 trim_cache size 0 max 16384 2024-04-26T01:58:02.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.045+0000 7f5cc226fb80 1 Processor -- start 2024-04-26T01:58:02.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.046+0000 7f5cc226fb80 1 -- start start 2024-04-26T01:58:02.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.046+0000 7f5cc226fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dbea0 0x55d0291d8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.046+0000 7f5cc226fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291dc7c0 0x55d0291e7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.046+0000 7f5cc226fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291efe80 0x55d0291e9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.046+0000 7f5cc226fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb01ad3c0 con 0x55d0291dc7c0 2024-04-26T01:58:02.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.046+0000 7f5cc226fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb01ada90 con 0x55d0291efe80 2024-04-26T01:58:02.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.046+0000 7f5cc226fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb01ab320 con 0x55d0291dbea0 2024-04-26T01:58:02.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.046+0000 7f5cafe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291dc7c0 0x55d0291e7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.047+0000 7f5cafe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291dc7c0 0x55d0291e7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36782/0 (socket says 172.21.15.1:36782) 2024-04-26T01:58:02.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.047+0000 7f5cafe00700 1 -- 172.21.15.1:0/3221588518 learned_addr learned my addr 172.21.15.1:0/3221588518 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:02.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.047+0000 7f5cb4e00700 1 --2- 172.21.15.1:0/3221588518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dbea0 0x55d0291d8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.047+0000 7f5cb5800700 1 --2- 172.21.15.1:0/3221588518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291efe80 0x55d0291e9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.047+0000 7f5cb4e00700 1 -- 172.21.15.1:0/3221588518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291efe80 msgr2=0x55d0291e9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:02.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.047+0000 7f5cb4e00700 1 --2- 172.21.15.1:0/3221588518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291efe80 0x55d0291e9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.047+0000 7f5cb4e00700 1 -- 172.21.15.1:0/3221588518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291dc7c0 msgr2=0x55d0291e7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:02.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.047+0000 7f5cb4e00700 1 --2- 172.21.15.1:0/3221588518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291dc7c0 0x55d0291e7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.047+0000 7f5cb4e00700 1 -- 172.21.15.1:0/3221588518 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0291d8ec0 con 0x55d0291dbea0 2024-04-26T01:58:02.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.047+0000 7f5cb4e00700 1 --2- 172.21.15.1:0/3221588518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dbea0 0x55d0291d8570 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4099270 tx=0x7f5ca4087370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f7aa9943e3d18e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:02.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.048+0000 7f5ca1600700 1 -- 172.21.15.1:0/3221588518 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40ba070 con 0x55d0291dbea0 2024-04-26T01:58:02.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.048+0000 7f5ca1600700 1 -- 172.21.15.1:0/3221588518 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ca40a5510 con 0x55d0291dbea0 2024-04-26T01:58:02.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.048+0000 7f5ca1600700 1 -- 172.21.15.1:0/3221588518 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40985f0 con 0x55d0291dbea0 2024-04-26T01:58:02.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.048+0000 7f5cc226fb80 1 -- 172.21.15.1:0/3221588518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dbea0 msgr2=0x55d0291d8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:02.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.048+0000 7f5cc226fb80 1 --2- 172.21.15.1:0/3221588518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dbea0 0x55d0291d8570 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4099270 tx=0x7f5ca4087370 comp rx=0 tx=0).stop 2024-04-26T01:58:02.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.048+0000 7f5cc226fb80 1 -- 172.21.15.1:0/3221588518 shutdown_connections 2024-04-26T01:58:02.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.048+0000 7f5cc226fb80 1 --2- 172.21.15.1:0/3221588518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291efe80 0x55d0291e9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.048+0000 7f5cc226fb80 1 --2- 172.21.15.1:0/3221588518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dbea0 0x55d0291d8570 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.048+0000 7f5cc226fb80 1 --2- 172.21.15.1:0/3221588518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291dc7c0 0x55d0291e7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.048+0000 7f5cc226fb80 1 -- 172.21.15.1:0/3221588518 >> 172.21.15.1:0/3221588518 conn(0x55d02910b050 msgr2=0x55d0291eea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:02.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.049+0000 7f5cc226fb80 1 -- 172.21.15.1:0/3221588518 shutdown_connections 2024-04-26T01:58:02.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.049+0000 7f5cc226fb80 1 -- 172.21.15.1:0/3221588518 wait complete. 2024-04-26T01:58:02.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.049+0000 7f5cc226fb80 1 -- start start 2024-04-26T01:58:02.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.049+0000 7f5cc226fb80 1 Processor -- start 2024-04-26T01:58:02.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.050+0000 7f5cc226fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dbea0 0x55d0292048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.050+0000 7f5cc226fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291dc7c0 0x55d029202c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.050+0000 7f5cc226fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291efe80 0x55d0292031a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.050+0000 7f5cc226fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb01a97e0 con 0x55d0291dc7c0 2024-04-26T01:58:02.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.050+0000 7f5cc226fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb01aac50 con 0x55d0291efe80 2024-04-26T01:58:02.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.050+0000 7f5cc226fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb01a9eb0 con 0x55d0291dbea0 2024-04-26T01:58:02.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.050+0000 7f5cc226fb80 10 client.0 _ll_register_callbacks cb 0x55d0290c7820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:02.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.050+0000 7f5cb4e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dbea0 0x55d0292048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.050+0000 7f5cafe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291dc7c0 0x55d029202c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.050+0000 7f5cb4e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dbea0 0x55d0292048b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:44066/0 (socket says 172.21.15.1:44066) 2024-04-26T01:58:02.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.050+0000 7f5cb4e00700 1 -- 172.21.15.1:0/2267626213 learned_addr learned my addr 172.21.15.1:0/2267626213 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:02.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.051+0000 7f5cb5800700 1 --2- 172.21.15.1:0/2267626213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291efe80 0x55d0292031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.051+0000 7f5cb4e00700 1 -- 172.21.15.1:0/2267626213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291efe80 msgr2=0x55d0292031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:02.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.051+0000 7f5cb4e00700 1 --2- 172.21.15.1:0/2267626213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291efe80 0x55d0292031a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.051+0000 7f5cb4e00700 1 -- 172.21.15.1:0/2267626213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291dc7c0 msgr2=0x55d029202c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:02.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.051+0000 7f5cb4e00700 1 --2- 172.21.15.1:0/2267626213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291dc7c0 0x55d029202c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.051+0000 7f5cb4e00700 1 -- 172.21.15.1:0/2267626213 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0291dc4d0 con 0x55d0291dbea0 2024-04-26T01:58:02.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.051+0000 7f5cb4e00700 1 --2- 172.21.15.1:0/2267626213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dbea0 0x55d0292048b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400c2d0 tx=0x7f5ca409d070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f294fe065d1bc7b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:02.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.051+0000 7f5c96a00700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:58:02.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.051+0000 7f5c96a00700 1 -- 172.21.15.1:0/2267626213 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40ba040 con 0x55d0291dbea0 2024-04-26T01:58:02.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.051+0000 7f5c96a00700 1 -- 172.21.15.1:0/2267626213 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8527+0+0 (secure 0 0 0) 0x7f5ca40985f0 con 0x55d0291dbea0 2024-04-26T01:58:02.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.051+0000 7f5cc226fb80 1 -- 172.21.15.1:0/2267626213 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ca40a5510 con 0x55d0291dbea0 2024-04-26T01:58:02.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.052+0000 7f5c96a00700 1 -- 172.21.15.1:0/2267626213 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5ca40987f0 con 0x55d0291dbea0 2024-04-26T01:58:02.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.052+0000 7f5cc226fb80 10 client.25954 fetch_fsmap learned FSMap version 32 2024-04-26T01:58:02.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.052+0000 7f5cc226fb80 1 -- 172.21.15.1:0/2267626213 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d0291dc4d0 con 0x55d0291dbea0 2024-04-26T01:58:02.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.052+0000 7f5c96a00700 1 -- 172.21.15.1:0/2267626213 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ca40a5510 con 0x55d0291dbea0 2024-04-26T01:58:02.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.052+0000 7f5c96a00700 1 -- 172.21.15.1:0/2267626213 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f5ca40a1de0 con 0x55d0291dbea0 2024-04-26T01:58:02.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.052+0000 7f5cc226fb80 10 client.25954 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:58:02.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.052+0000 7f5cc226fb80 10 client.25954 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:58:02.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.052+0000 7f5cc226fb80 20 client.25954 populate_metadata read hostname 'smithi001' 2024-04-26T01:58:02.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.052+0000 7f5cc226fb80 1 --2- 172.21.15.1:0/2267626213 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d029266810 0x55d029268c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.052+0000 7f5cc226fb80 4 client.25954 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5f41eab3", "release": true, "await": true}] 2024-04-26T01:58:02.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.053+0000 7f5cc226fb80 1 -- 172.21.15.1:0/2267626213 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5f41eab3", "release": true, "await": true}) -- 0x55d029136820 con 0x55d029266810 2024-04-26T01:58:02.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.053+0000 7f5cafe00700 1 --2- 172.21.15.1:0/2267626213 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d029266810 0x55d029268c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.053+0000 7f5cafe00700 1 --2- 172.21.15.1:0/2267626213 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d029266810 0x55d029268c00 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f5f5ff1ffab45d8 server_cookie=957675c852860a50 in_seq=0 out_seq=0 2024-04-26T01:58:02.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.053+0000 7f5c96a00700 10 client.25954 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:02.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:01 smithi001 ceph-mon[27822]: pgmap v195: 97 pgs: 97 active+clean; 42 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.8 KiB/s wr, 10 op/s 2024-04-26T01:58:02.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:01 smithi097 ceph-mon[32853]: pgmap v195: 97 pgs: 97 active+clean; 42 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.8 KiB/s wr, 10 op/s 2024-04-26T01:58:02.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.261+0000 7f5c96a00700 1 -- 172.21.15.1:0/2267626213 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55d029136820 con 0x55d029266810 2024-04-26T01:58:02.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.261+0000 7f5c96a00700 10 client.25954 handle_command_reply: tid=0 2024-04-26T01:58:02.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.261+0000 7f5cc226fb80 1 client.25954 shutdown 2024-04-26T01:58:02.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.261+0000 7f5cc226fb80 1 -- 172.21.15.1:0/2267626213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dbea0 msgr2=0x55d0292048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:02.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.261+0000 7f5cc226fb80 1 --2- 172.21.15.1:0/2267626213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dbea0 0x55d0292048b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400c2d0 tx=0x7f5ca409d070 comp rx=0 tx=0).stop 2024-04-26T01:58:02.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.262+0000 7f5cc226fb80 1 -- 172.21.15.1:0/2267626213 shutdown_connections 2024-04-26T01:58:02.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.262+0000 7f5cc226fb80 1 --2- 172.21.15.1:0/2267626213 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55d029266810 0x55d029268c00 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.262+0000 7f5cc226fb80 1 --2- 172.21.15.1:0/2267626213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0291efe80 0x55d0292031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.262+0000 7f5cc226fb80 1 --2- 172.21.15.1:0/2267626213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d0291dbea0 0x55d0292048b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.262+0000 7f5cc226fb80 1 --2- 172.21.15.1:0/2267626213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d0291dc7c0 0x55d029202c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.262+0000 7f5cc226fb80 1 -- 172.21.15.1:0/2267626213 >> 172.21.15.1:0/2267626213 conn(0x55d02910b050 msgr2=0x55d0291eea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:02.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.262+0000 7f5c96a00700 0 client.25954 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:02.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.262+0000 7f5cc226fb80 1 -- 172.21.15.1:0/2267626213 shutdown_connections 2024-04-26T01:58:02.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.262+0000 7f5cc226fb80 1 -- 172.21.15.1:0/2267626213 wait complete. 2024-04-26T01:58:02.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.263+0000 7f5cc226fb80 20 client.25954 trim_cache size 0 max 16384 2024-04-26T01:58:02.262 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T01:58:02.262 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 32, 2024-04-26T01:58:02.262 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24518, 2024-04-26T01:58:02.263 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 16, 2024-04-26T01:58:02.263 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-04-26T01:58:02.263 INFO:teuthology.orchestra.run.smithi001.stdout: "5f41eab3": { 2024-04-26T01:58:02.263 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 16, 2024-04-26T01:58:02.263 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-04-26T01:58:02.263 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:58:02.263 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-04-26T01:58:02.263 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:58:02.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T01:58:02.263 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-04-26T01:58:02.264 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 70.0, 2024-04-26T01:58:02.264 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-04-26T01:58:02.264 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-04-26T01:58:02.264 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-04-26T01:58:02.264 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:58:02.264 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-04-26T01:58:02.264 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:58:02.264 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:58:02.264 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:58:02.264 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:58:02.264 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:58:02.264 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:58:02.264 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T01:58:02.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.264+0000 7f5cb6200700 1 -- 172.21.15.1:0/1746367553 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f5c980a3e30 msgr2=0x7f5c980a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:02.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.264+0000 7f5cb6200700 1 --2- 172.21.15.1:0/1746367553 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f5c980a3e30 0x7f5c980a62f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0098f0 tx=0x7f5c9c019040 comp rx=0 tx=0).stop 2024-04-26T01:58:02.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.264+0000 7f5cb6200700 1 -- 172.21.15.1:0/1746367553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cb0104db0 msgr2=0x7f5cb019b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:02.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.264+0000 7f5cb6200700 1 --2- 172.21.15.1:0/1746367553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cb0104db0 0x7f5cb019b3e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400e9a0 tx=0x7f5ca400ecb0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.264+0000 7f5cb6200700 1 -- 172.21.15.1:0/1746367553 shutdown_connections 2024-04-26T01:58:02.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.264+0000 7f5cb6200700 1 --2- 172.21.15.1:0/1746367553 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f5c980a3e30 0x7f5c980a62f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.264+0000 7f5cb6200700 1 --2- 172.21.15.1:0/1746367553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cb0109e90 0x7f5cb019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.264+0000 7f5cb6200700 1 --2- 172.21.15.1:0/1746367553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cb0104db0 0x7f5cb019b3e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.264+0000 7f5cb6200700 1 --2- 172.21.15.1:0/1746367553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5cb0105760 0x7f5cb019b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.264+0000 7f5cb6200700 1 -- 172.21.15.1:0/1746367553 >> 172.21.15.1:0/1746367553 conn(0x7f5cb0100680 msgr2=0x7f5cb0101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.265+0000 7f5cb6200700 1 -- 172.21.15.1:0/1746367553 shutdown_connections 2024-04-26T01:58:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:02.265+0000 7f5cb6200700 1 -- 172.21.15.1:0/1746367553 wait complete. 2024-04-26T01:58:02.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:01 smithi173 ceph-mon[32553]: pgmap v195: 97 pgs: 97 active+clean; 42 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.8 KiB/s wr, 10 op/s 2024-04-26T01:58:03.293 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5f41eab3', seconds elapsed: 26.9 2024-04-26T01:58:04.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:03 smithi173 ceph-mon[32553]: pgmap v196: 97 pgs: 97 active+clean; 42 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.7 KiB/s wr, 7 op/s 2024-04-26T01:58:04.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:03 smithi001 ceph-mon[27822]: pgmap v196: 97 pgs: 97 active+clean; 42 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.7 KiB/s wr, 7 op/s 2024-04-26T01:58:04.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:03 smithi097 ceph-mon[32853]: pgmap v196: 97 pgs: 97 active+clean; 42 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.7 KiB/s wr, 7 op/s 2024-04-26T01:58:06.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:06 smithi097 ceph-mon[32853]: pgmap v197: 97 pgs: 97 active+clean; 49 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 507 KiB/s wr, 101 op/s 2024-04-26T01:58:06.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:06 smithi097 ceph-mon[32853]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T01:58:06.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:06 smithi173 ceph-mon[32553]: pgmap v197: 97 pgs: 97 active+clean; 49 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 507 KiB/s wr, 101 op/s 2024-04-26T01:58:06.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:06 smithi173 ceph-mon[32553]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T01:58:06.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:06 smithi001 ceph-mon[27822]: pgmap v197: 97 pgs: 97 active+clean; 49 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 507 KiB/s wr, 101 op/s 2024-04-26T01:58:06.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:06 smithi001 ceph-mon[27822]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T01:58:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:08 smithi097 ceph-mon[32853]: pgmap v199: 97 pgs: 97 active+clean; 49 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 605 KiB/s wr, 117 op/s 2024-04-26T01:58:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:08 smithi173 ceph-mon[32553]: pgmap v199: 97 pgs: 97 active+clean; 49 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 605 KiB/s wr, 117 op/s 2024-04-26T01:58:08.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:08 smithi001 ceph-mon[27822]: pgmap v199: 97 pgs: 97 active+clean; 49 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 605 KiB/s wr, 117 op/s 2024-04-26T01:58:10.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:10 smithi097 ceph-mon[32853]: pgmap v200: 97 pgs: 97 active+clean; 49 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 605 KiB/s wr, 117 op/s 2024-04-26T01:58:10.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:10 smithi173 ceph-mon[32553]: pgmap v200: 97 pgs: 97 active+clean; 49 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 605 KiB/s wr, 117 op/s 2024-04-26T01:58:10.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:10 smithi001 ceph-mon[27822]: pgmap v200: 97 pgs: 97 active+clean; 49 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 605 KiB/s wr, 117 op/s 2024-04-26T01:58:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.075+0000 7faba9000640 1 -- 172.21.15.1:0/1353700771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba41083b0 msgr2=0x7faba4108810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.075+0000 7faba9000640 1 --2- 172.21.15.1:0/1353700771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba41083b0 0x7faba4108810 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fab94009900 tx=0x7fab9402a480 comp rx=0 tx=0).stop 2024-04-26T01:58:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.075+0000 7faba9000640 1 -- 172.21.15.1:0/1353700771 shutdown_connections 2024-04-26T01:58:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.075+0000 7faba9000640 1 --2- 172.21.15.1:0/1353700771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faba4108d50 0x7faba4115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.075+0000 7faba9000640 1 --2- 172.21.15.1:0/1353700771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba41083b0 0x7faba4108810 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.075+0000 7faba9000640 1 --2- 172.21.15.1:0/1353700771 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faba410e470 0x7faba410e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.075+0000 7faba9000640 1 -- 172.21.15.1:0/1353700771 >> 172.21.15.1:0/1353700771 conn(0x7faba4077b00 msgr2=0x7faba4079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.075+0000 7faba9000640 1 -- 172.21.15.1:0/1353700771 shutdown_connections 2024-04-26T01:58:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.075+0000 7faba9000640 1 -- 172.21.15.1:0/1353700771 wait complete. 2024-04-26T01:58:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.076+0000 7faba9000640 1 Processor -- start 2024-04-26T01:58:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.077+0000 7faba9000640 1 -- start start 2024-04-26T01:58:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.077+0000 7faba9000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faba41083b0 0x7faba41a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.077+0000 7faba9000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faba4108d50 0x7faba41a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.077+0000 7faba9000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba410e470 0x7faba419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.077+0000 7faba9000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faba4118650 con 0x7faba4108d50 2024-04-26T01:58:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.077+0000 7faba9000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faba41184d0 con 0x7faba410e470 2024-04-26T01:58:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.077+0000 7faba9000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faba41187d0 con 0x7faba41083b0 2024-04-26T01:58:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.077+0000 7faba3400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faba41083b0 0x7faba41a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.077+0000 7faba3400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faba41083b0 0x7faba41a47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:58348/0 (socket says 172.21.15.1:58348) 2024-04-26T01:58:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.077+0000 7faba3400640 1 -- 172.21.15.1:0/1489254054 learned_addr learned my addr 172.21.15.1:0/1489254054 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.077+0000 7faba2a00640 1 --2- 172.21.15.1:0/1489254054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faba4108d50 0x7faba41a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.077+0000 7faba3e00640 1 --2- 172.21.15.1:0/1489254054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba410e470 0x7faba419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.078+0000 7faba3e00640 1 -- 172.21.15.1:0/1489254054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faba41083b0 msgr2=0x7faba41a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.078+0000 7faba3e00640 1 --2- 172.21.15.1:0/1489254054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faba41083b0 0x7faba41a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.078+0000 7faba3e00640 1 -- 172.21.15.1:0/1489254054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faba4108d50 msgr2=0x7faba41a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.078+0000 7faba3e00640 1 --2- 172.21.15.1:0/1489254054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faba4108d50 0x7faba41a4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.078+0000 7faba3e00640 1 -- 172.21.15.1:0/1489254054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba419f0d0 con 0x7faba410e470 2024-04-26T01:58:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.078+0000 7faba3e00640 1 --2- 172.21.15.1:0/1489254054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba410e470 0x7faba419e950 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009fd0 tx=0x7fab9c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.078+0000 7fab9be00640 1 -- 172.21.15.1:0/1489254054 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c00ce20 con 0x7faba410e470 2024-04-26T01:58:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.079+0000 7fab9be00640 1 -- 172.21.15.1:0/1489254054 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab9c005020 con 0x7faba410e470 2024-04-26T01:58:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.079+0000 7faba9000640 1 -- 172.21.15.1:0/1489254054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba419f3c0 con 0x7faba410e470 2024-04-26T01:58:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.079+0000 7fab9be00640 1 -- 172.21.15.1:0/1489254054 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c010690 con 0x7faba410e470 2024-04-26T01:58:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.079+0000 7faba9000640 1 -- 172.21.15.1:0/1489254054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba41ab660 con 0x7faba410e470 2024-04-26T01:58:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.082+0000 7fab9be00640 1 -- 172.21.15.1:0/1489254054 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fab9c0054d0 con 0x7faba410e470 2024-04-26T01:58:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.083+0000 7fabad44f740 1 Processor -- start 2024-04-26T01:58:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.083+0000 7fabad44f740 1 -- start start 2024-04-26T01:58:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.084+0000 7fabad44f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55daf8e3b9b0 0x55daf8e3bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.084+0000 7fabad44f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 0x55daf8e46650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.084+0000 7fabad44f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e46b90 0x55daf8e48f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.084+0000 7fabad44f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55daf8da1900 con 0x55daf8e3c2d0 2024-04-26T01:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.084+0000 7fabad44f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55daf8d47ef0 con 0x55daf8e46b90 2024-04-26T01:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.084+0000 7fabad44f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55daf8d9d8c0 con 0x55daf8e3b9b0 2024-04-26T01:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.084+0000 7faba2a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55daf8e3b9b0 0x55daf8e3bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.084+0000 7faba3400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 0x55daf8e46650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.084+0000 7faba2a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55daf8e3b9b0 0x55daf8e3bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:58350/0 (socket says 172.21.15.1:58350) 2024-04-26T01:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.084+0000 7faba2a00640 1 -- 172.21.15.1:0/1413863681 learned_addr learned my addr 172.21.15.1:0/1413863681 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.084+0000 7faba2a00640 1 --2- 172.21.15.1:0/1413863681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e46b90 0x55daf8e48f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.085+0000 7fab9be00640 1 --2- 172.21.15.1:0/1489254054 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fab880a3d80 0x7fab880a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.085+0000 7faba3400640 1 -- 172.21.15.1:0/1413863681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55daf8e3b9b0 msgr2=0x55daf8e3bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.085+0000 7faba3400640 1 --2- 172.21.15.1:0/1413863681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55daf8e3b9b0 0x55daf8e3bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.085+0000 7faba3400640 1 -- 172.21.15.1:0/1413863681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e46b90 msgr2=0x55daf8e48f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.085+0000 7faba3400640 1 --2- 172.21.15.1:0/1413863681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e46b90 0x55daf8e48f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.085+0000 7faba3400640 1 -- 172.21.15.1:0/1413863681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daf8ab4210 con 0x55daf8e3c2d0 2024-04-26T01:58:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.085+0000 7fab9be00640 1 -- 172.21.15.1:0/1489254054 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7fab9c0d0f40 con 0x7faba410e470 2024-04-26T01:58:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.085+0000 7faba3e00640 1 --2- 172.21.15.1:0/1489254054 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fab880a3d80 0x7fab880a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.085+0000 7faba3400640 1 --2- 172.21.15.1:0/1413863681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 0x55daf8e46650 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fab90009cc0 tx=0x7fab90011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8c92a06fae5915 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.085+0000 7fab98c00640 1 -- 172.21.15.1:0/1413863681 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90005220 con 0x55daf8e3c2d0 2024-04-26T01:58:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.086+0000 7fab98c00640 1 -- 172.21.15.1:0/1413863681 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab90004c50 con 0x55daf8e3c2d0 2024-04-26T01:58:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.086+0000 7fab98c00640 1 -- 172.21.15.1:0/1413863681 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90019720 con 0x55daf8e3c2d0 2024-04-26T01:58:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.086+0000 7fabad44f740 1 -- 172.21.15.1:0/1413863681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 msgr2=0x55daf8e46650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.086+0000 7fabad44f740 1 --2- 172.21.15.1:0/1413863681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 0x55daf8e46650 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fab90009cc0 tx=0x7fab90011810 comp rx=0 tx=0).stop 2024-04-26T01:58:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.086+0000 7faba3e00640 1 --2- 172.21.15.1:0/1489254054 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fab880a3d80 0x7fab880a6240 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0997f0 tx=0x7fab9c0057c0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.086+0000 7fabad44f740 1 -- 172.21.15.1:0/1413863681 shutdown_connections 2024-04-26T01:58:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.086+0000 7fabad44f740 1 --2- 172.21.15.1:0/1413863681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e46b90 0x55daf8e48f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.086+0000 7fabad44f740 1 --2- 172.21.15.1:0/1413863681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 0x55daf8e46650 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.086+0000 7fabad44f740 1 --2- 172.21.15.1:0/1413863681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55daf8e3b9b0 0x55daf8e3bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.086+0000 7fabad44f740 1 -- 172.21.15.1:0/1413863681 >> 172.21.15.1:0/1413863681 conn(0x55daf8e37ec0 msgr2=0x55daf8e382a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.087+0000 7fabad44f740 1 -- 172.21.15.1:0/1413863681 shutdown_connections 2024-04-26T01:58:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.087+0000 7fabad44f740 1 -- 172.21.15.1:0/1413863681 wait complete. 2024-04-26T01:58:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.087+0000 7fabad44f740 1 -- start start 2024-04-26T01:58:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.087+0000 7fabad44f740 1 Processor -- start 2024-04-26T01:58:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.088+0000 7fabad44f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e3b9b0 0x55daf8e43ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.088+0000 7fabad44f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 0x55daf8e43ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.088+0000 7fabad44f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55daf8e46b90 0x55daf8e44530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.088+0000 7fabad44f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55daf8e497c0 con 0x55daf8e3c2d0 2024-04-26T01:58:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.088+0000 7fabad44f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55daf8d5b310 con 0x55daf8e3b9b0 2024-04-26T01:58:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.088+0000 7fabad44f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55daf8e49940 con 0x55daf8e46b90 2024-04-26T01:58:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.088+0000 7faba2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e3b9b0 0x55daf8e43ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.088+0000 7faba2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e3b9b0 0x55daf8e43ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:58168/0 (socket says 172.21.15.1:58168) 2024-04-26T01:58:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.088+0000 7faba2a00640 1 -- 172.21.15.1:0/1429810632 learned_addr learned my addr 172.21.15.1:0/1429810632 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.088+0000 7faba3400640 1 --2- 172.21.15.1:0/1429810632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 0x55daf8e43ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.088+0000 7fabad44f740 10 client.0 _ll_register_callbacks cb 0x55daf8d29cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.088+0000 7faba2a00640 1 --2- 172.21.15.1:0/1429810632 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55daf8e46b90 0x55daf8e44530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.089+0000 7faba2a00640 1 -- 172.21.15.1:0/1429810632 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55daf8e46b90 msgr2=0x55daf8e44530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.089+0000 7faba2a00640 1 --2- 172.21.15.1:0/1429810632 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55daf8e46b90 0x55daf8e44530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.089+0000 7faba2a00640 1 -- 172.21.15.1:0/1429810632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 msgr2=0x55daf8e43ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.089+0000 7faba2a00640 1 --2- 172.21.15.1:0/1429810632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 0x55daf8e43ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.089+0000 7faba2a00640 1 -- 172.21.15.1:0/1429810632 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daf8b80620 con 0x55daf8e3b9b0 2024-04-26T01:58:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.089+0000 7faba2a00640 1 --2- 172.21.15.1:0/1429810632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e3b9b0 0x55daf8e43ab0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fab94009900 tx=0x7fab94043040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1c6ecf0aec25290 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.089+0000 7fab8d600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:58:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.089+0000 7fab8d600640 1 -- 172.21.15.1:0/1429810632 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94035040 con 0x55daf8e3b9b0 2024-04-26T01:58:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.089+0000 7fab8d600640 1 -- 172.21.15.1:0/1429810632 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7fab94004a30 con 0x55daf8e3b9b0 2024-04-26T01:58:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.089+0000 7fabad44f740 1 -- 172.21.15.1:0/1429810632 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab90004c50 con 0x55daf8e3b9b0 2024-04-26T01:58:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.089+0000 7fab8d600640 1 -- 172.21.15.1:0/1429810632 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fab94004c30 con 0x55daf8e3b9b0 2024-04-26T01:58:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.089+0000 7fabad44f740 10 client.25946 fetch_fsmap learned FSMap version 32 2024-04-26T01:58:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.089+0000 7fabad44f740 1 -- 172.21.15.1:0/1429810632 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55daf8e55a10 con 0x55daf8e3b9b0 2024-04-26T01:58:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.090+0000 7fab8d600640 1 -- 172.21.15.1:0/1429810632 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab90004c50 con 0x55daf8e3b9b0 2024-04-26T01:58:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.090+0000 7fab8d600640 1 -- 172.21.15.1:0/1429810632 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fab940035f0 con 0x55daf8e3b9b0 2024-04-26T01:58:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.090+0000 7fabad44f740 10 client.25946 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:58:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.090+0000 7fabad44f740 10 client.25946 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:58:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.090+0000 7fabad44f740 20 client.25946 populate_metadata read hostname 'smithi001' 2024-04-26T01:58:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.090+0000 7fabad44f740 1 --2- 172.21.15.1:0/1429810632 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55daf8e56ec0 0x55daf8e592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.090+0000 7fabad44f740 4 client.25946 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:58:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.090+0000 7fabad44f740 1 -- 172.21.15.1:0/1429810632 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55daf8db58b0 con 0x55daf8e56ec0 2024-04-26T01:58:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.090+0000 7faba3400640 1 --2- 172.21.15.1:0/1429810632 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55daf8e56ec0 0x55daf8e592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.091+0000 7faba3400640 1 --2- 172.21.15.1:0/1429810632 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55daf8e56ec0 0x55daf8e592d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1e433fef295ec7f server_cookie=6e523b737833034b in_seq=0 out_seq=0 2024-04-26T01:58:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.091+0000 7fab8d600640 10 client.25946 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.092+0000 7fab8d600640 1 -- 172.21.15.1:0/1429810632 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55daf8db58b0 con 0x55daf8e56ec0 2024-04-26T01:58:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.092+0000 7fab8d600640 10 client.25946 handle_command_reply: tid=0 2024-04-26T01:58:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.092+0000 7fabad44f740 1 client.25946 shutdown 2024-04-26T01:58:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.092+0000 7fabad44f740 1 -- 172.21.15.1:0/1429810632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e3b9b0 msgr2=0x55daf8e43ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.092+0000 7fabad44f740 1 --2- 172.21.15.1:0/1429810632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e3b9b0 0x55daf8e43ab0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fab94009900 tx=0x7fab94043040 comp rx=0 tx=0).stop 2024-04-26T01:58:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.092+0000 7fabad44f740 1 -- 172.21.15.1:0/1429810632 shutdown_connections 2024-04-26T01:58:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.092+0000 7fabad44f740 1 --2- 172.21.15.1:0/1429810632 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55daf8e56ec0 0x55daf8e592d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.092+0000 7fabad44f740 1 --2- 172.21.15.1:0/1429810632 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55daf8e46b90 0x55daf8e44530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.092+0000 7fabad44f740 1 --2- 172.21.15.1:0/1429810632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 0x55daf8e43ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.092+0000 7fabad44f740 1 --2- 172.21.15.1:0/1429810632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e3b9b0 0x55daf8e43ab0 secure :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fab94009900 tx=0x7fab94043040 comp rx=0 tx=0).stop 2024-04-26T01:58:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.092+0000 7fabad44f740 1 -- 172.21.15.1:0/1429810632 >> 172.21.15.1:0/1429810632 conn(0x55daf8e37ec0 msgr2=0x55daf8e3dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.092+0000 7fab8d600640 0 client.25946 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.093+0000 7fabad44f740 1 -- 172.21.15.1:0/1429810632 shutdown_connections 2024-04-26T01:58:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.093+0000 7fabad44f740 1 -- 172.21.15.1:0/1429810632 wait complete. 2024-04-26T01:58:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.093+0000 7fabad44f740 20 client.25946 trim_cache size 0 max 16384 2024-04-26T01:58:11.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.112+0000 7fabad44f740 1 Processor -- start 2024-04-26T01:58:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.112+0000 7fabad44f740 1 -- start start 2024-04-26T01:58:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.112+0000 7fabad44f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 0x55daf8e526b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.112+0000 7fabad44f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55daf8e46b90 0x55daf8e3e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.112+0000 7fabad44f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e462f0 0x55daf8e43370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.112+0000 7fabad44f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faba41b6910 con 0x55daf8e3c2d0 2024-04-26T01:58:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.112+0000 7fabad44f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faba41b41e0 con 0x55daf8e462f0 2024-04-26T01:58:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.112+0000 7fabad44f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faba41b2050 con 0x55daf8e46b90 2024-04-26T01:58:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.113+0000 7faba2a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 0x55daf8e526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.113+0000 7faba2a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 0x55daf8e526b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37742/0 (socket says 172.21.15.1:37742) 2024-04-26T01:58:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.113+0000 7faba2a00640 1 -- 172.21.15.1:0/4254179784 learned_addr learned my addr 172.21.15.1:0/4254179784 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.113+0000 7faba3400640 1 --2- 172.21.15.1:0/4254179784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55daf8e46b90 0x55daf8e3e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.113+0000 7faba2a00640 1 --2- 172.21.15.1:0/4254179784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e462f0 0x55daf8e43370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.113+0000 7faba2a00640 1 -- 172.21.15.1:0/4254179784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55daf8e46b90 msgr2=0x55daf8e3e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.113+0000 7faba2a00640 1 --2- 172.21.15.1:0/4254179784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55daf8e46b90 0x55daf8e3e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.113+0000 7faba2a00640 1 -- 172.21.15.1:0/4254179784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e462f0 msgr2=0x55daf8e43370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.113+0000 7faba2a00640 1 --2- 172.21.15.1:0/4254179784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e462f0 0x55daf8e43370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.113+0000 7faba2a00640 1 -- 172.21.15.1:0/4254179784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daf8e33820 con 0x55daf8e3c2d0 2024-04-26T01:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.114+0000 7faba2a00640 1 --2- 172.21.15.1:0/4254179784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 0x55daf8e526b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fab940057c0 tx=0x7fab94005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd95d040770bd08 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.114+0000 7fab8fe00640 1 -- 172.21.15.1:0/4254179784 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94035070 con 0x55daf8e3c2d0 2024-04-26T01:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.114+0000 7fab8fe00640 1 -- 172.21.15.1:0/4254179784 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab94004090 con 0x55daf8e3c2d0 2024-04-26T01:58:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.114+0000 7fab8fe00640 1 -- 172.21.15.1:0/4254179784 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94031820 con 0x55daf8e3c2d0 2024-04-26T01:58:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.115+0000 7fabad44f740 1 -- 172.21.15.1:0/4254179784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 msgr2=0x55daf8e526b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.115+0000 7fabad44f740 1 --2- 172.21.15.1:0/4254179784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 0x55daf8e526b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fab940057c0 tx=0x7fab94005fd0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.115+0000 7fabad44f740 1 -- 172.21.15.1:0/4254179784 shutdown_connections 2024-04-26T01:58:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.115+0000 7fabad44f740 1 --2- 172.21.15.1:0/4254179784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e462f0 0x55daf8e43370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.115+0000 7fabad44f740 1 --2- 172.21.15.1:0/4254179784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55daf8e46b90 0x55daf8e3e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.115+0000 7fabad44f740 1 --2- 172.21.15.1:0/4254179784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 0x55daf8e526b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.115+0000 7fabad44f740 1 -- 172.21.15.1:0/4254179784 >> 172.21.15.1:0/4254179784 conn(0x55daf8e37ec0 msgr2=0x55daf8e382a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.115+0000 7fabad44f740 1 -- 172.21.15.1:0/4254179784 shutdown_connections 2024-04-26T01:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.115+0000 7fabad44f740 1 -- 172.21.15.1:0/4254179784 wait complete. 2024-04-26T01:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.116+0000 7fabad44f740 1 -- start start 2024-04-26T01:58:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.116+0000 7fabad44f740 1 Processor -- start 2024-04-26T01:58:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.116+0000 7fabad44f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 0x55daf8e4eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.116+0000 7fabad44f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e462f0 0x55daf8e4c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.116+0000 7fabad44f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55daf8e46b90 0x55daf8e4c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.116+0000 7fabad44f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faba41b26a0 con 0x55daf8e3c2d0 2024-04-26T01:58:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.116+0000 7fabad44f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faba41b3b10 con 0x55daf8e462f0 2024-04-26T01:58:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.116+0000 7fabad44f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faba41b2d70 con 0x55daf8e46b90 2024-04-26T01:58:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.116+0000 7fabad44f740 10 client.0 _ll_register_callbacks cb 0x7fab94005220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.117+0000 7faba3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e462f0 0x55daf8e4c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.117+0000 7faba3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e462f0 0x55daf8e4c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:58190/0 (socket says 172.21.15.1:58190) 2024-04-26T01:58:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.117+0000 7faba3400640 1 -- 172.21.15.1:0/114371368 learned_addr learned my addr 172.21.15.1:0/114371368 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.117+0000 7faba2a00640 1 --2- 172.21.15.1:0/114371368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 0x55daf8e4eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.117+0000 7faba2a00640 1 --2- 172.21.15.1:0/114371368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55daf8e46b90 0x55daf8e4c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.117+0000 7faba3400640 1 -- 172.21.15.1:0/114371368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55daf8e46b90 msgr2=0x55daf8e4c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.117+0000 7faba3400640 1 --2- 172.21.15.1:0/114371368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55daf8e46b90 0x55daf8e4c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.117+0000 7faba3400640 1 -- 172.21.15.1:0/114371368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 msgr2=0x55daf8e4eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.117+0000 7faba3400640 1 --2- 172.21.15.1:0/114371368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 0x55daf8e4eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.117+0000 7faba3400640 1 -- 172.21.15.1:0/114371368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daf8e4cee0 con 0x55daf8e462f0 2024-04-26T01:58:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.117+0000 7faba3400640 1 --2- 172.21.15.1:0/114371368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e462f0 0x55daf8e4c260 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fab90009cc0 tx=0x7fab90009880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab37abb5faf8816d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.118+0000 7fab8d600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:58:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.118+0000 7fab8d600640 1 -- 172.21.15.1:0/114371368 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90011d10 con 0x55daf8e462f0 2024-04-26T01:58:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.118+0000 7fab8d600640 1 -- 172.21.15.1:0/114371368 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7fab9002e340 con 0x55daf8e462f0 2024-04-26T01:58:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.118+0000 7fabad44f740 1 -- 172.21.15.1:0/114371368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab94004090 con 0x55daf8e462f0 2024-04-26T01:58:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.118+0000 7fab8d600640 1 -- 172.21.15.1:0/114371368 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fab900098f0 con 0x55daf8e462f0 2024-04-26T01:58:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.118+0000 7fabad44f740 10 client.25952 fetch_fsmap learned FSMap version 32 2024-04-26T01:58:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.118+0000 7fabad44f740 1 -- 172.21.15.1:0/114371368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55daf8ebd3a0 con 0x55daf8e462f0 2024-04-26T01:58:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.119+0000 7fab8d600640 1 -- 172.21.15.1:0/114371368 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab94004090 con 0x55daf8e462f0 2024-04-26T01:58:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.119+0000 7fab8d600640 1 -- 172.21.15.1:0/114371368 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fab90037bb0 con 0x55daf8e462f0 2024-04-26T01:58:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.119+0000 7fabad44f740 10 client.25952 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:58:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.119+0000 7fabad44f740 10 client.25952 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:58:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.119+0000 7fabad44f740 20 client.25952 populate_metadata read hostname 'smithi001' 2024-04-26T01:58:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.119+0000 7fabad44f740 1 --2- 172.21.15.1:0/114371368 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55daf8ebd9c0 0x55daf8ebfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.119+0000 7fabad44f740 4 client.25952 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:58:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.119+0000 7fabad44f740 1 -- 172.21.15.1:0/114371368 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55daf8daf320 con 0x55daf8ebd9c0 2024-04-26T01:58:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.119+0000 7faba2a00640 1 --2- 172.21.15.1:0/114371368 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55daf8ebd9c0 0x55daf8ebfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.120+0000 7fab8d600640 10 client.25952 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.120+0000 7faba2a00640 1 --2- 172.21.15.1:0/114371368 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55daf8ebd9c0 0x55daf8ebfde0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d87e65fb42bc9bc2 server_cookie=985daa24f9268587 in_seq=0 out_seq=0 2024-04-26T01:58:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.134+0000 7fab8d600640 1 -- 172.21.15.1:0/114371368 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55daf8daf320 con 0x55daf8ebd9c0 2024-04-26T01:58:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.134+0000 7fab8d600640 10 client.25952 handle_command_reply: tid=0 2024-04-26T01:58:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.134+0000 7fabad44f740 1 client.25952 shutdown 2024-04-26T01:58:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.134+0000 7fabad44f740 1 -- 172.21.15.1:0/114371368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e462f0 msgr2=0x55daf8e4c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.134+0000 7fabad44f740 1 --2- 172.21.15.1:0/114371368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e462f0 0x55daf8e4c260 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fab90009cc0 tx=0x7fab90009880 comp rx=0 tx=0).stop 2024-04-26T01:58:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.135+0000 7fabad44f740 1 -- 172.21.15.1:0/114371368 shutdown_connections 2024-04-26T01:58:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.135+0000 7fabad44f740 1 --2- 172.21.15.1:0/114371368 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55daf8ebd9c0 0x55daf8ebfde0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.135+0000 7fabad44f740 1 --2- 172.21.15.1:0/114371368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55daf8e46b90 0x55daf8e4c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.135+0000 7fabad44f740 1 --2- 172.21.15.1:0/114371368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55daf8e462f0 0x55daf8e4c260 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.135+0000 7fabad44f740 1 --2- 172.21.15.1:0/114371368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55daf8e3c2d0 0x55daf8e4eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.135+0000 7fab8d600640 0 client.25952 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.135+0000 7fabad44f740 1 -- 172.21.15.1:0/114371368 >> 172.21.15.1:0/114371368 conn(0x55daf8e37ec0 msgr2=0x55daf8e36790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.135+0000 7fabad44f740 1 -- 172.21.15.1:0/114371368 shutdown_connections 2024-04-26T01:58:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.135+0000 7fabad44f740 1 -- 172.21.15.1:0/114371368 wait complete. 2024-04-26T01:58:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.135+0000 7fabad44f740 20 client.25952 trim_cache size 0 max 16384 2024-04-26T01:58:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:58:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:58:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:58:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:58:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.136+0000 7faba9000640 1 -- 172.21.15.1:0/1489254054 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fab880a3d80 msgr2=0x7fab880a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.136+0000 7faba9000640 1 --2- 172.21.15.1:0/1489254054 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fab880a3d80 0x7fab880a6240 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0997f0 tx=0x7fab9c0057c0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.136+0000 7faba9000640 1 -- 172.21.15.1:0/1489254054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba410e470 msgr2=0x7faba419e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.136+0000 7faba9000640 1 --2- 172.21.15.1:0/1489254054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba410e470 0x7faba419e950 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009fd0 tx=0x7fab9c00ef10 comp rx=0 tx=0).stop 2024-04-26T01:58:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.136+0000 7faba9000640 1 -- 172.21.15.1:0/1489254054 shutdown_connections 2024-04-26T01:58:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.136+0000 7faba9000640 1 --2- 172.21.15.1:0/1489254054 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fab880a3d80 0x7fab880a6240 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.136+0000 7faba9000640 1 --2- 172.21.15.1:0/1489254054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba410e470 0x7faba419e950 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.136+0000 7faba9000640 1 --2- 172.21.15.1:0/1489254054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faba4108d50 0x7faba41a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.136+0000 7faba9000640 1 --2- 172.21.15.1:0/1489254054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faba41083b0 0x7faba41a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.136+0000 7faba9000640 1 -- 172.21.15.1:0/1489254054 >> 172.21.15.1:0/1489254054 conn(0x7faba4077b00 msgr2=0x7faba4079320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.138+0000 7faba9000640 1 -- 172.21.15.1:0/1489254054 shutdown_connections 2024-04-26T01:58:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:11.138+0000 7faba9000640 1 -- 172.21.15.1:0/1489254054 wait complete. 2024-04-26T01:58:11.659 INFO:tasks.workunit.client.0.smithi001.stdout:30000 trunc from 0x7a5444 to 0x143a5 2024-04-26T01:58:12.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:12 smithi097 ceph-mon[32853]: pgmap v201: 97 pgs: 97 active+clean; 49 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 276 op/s 2024-04-26T01:58:12.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:12 smithi173 ceph-mon[32553]: pgmap v201: 97 pgs: 97 active+clean; 49 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 276 op/s 2024-04-26T01:58:12.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:12 smithi001 ceph-mon[27822]: pgmap v201: 97 pgs: 97 active+clean; 49 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 276 op/s 2024-04-26T01:58:14.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:14 smithi097 ceph-mon[32853]: pgmap v202: 97 pgs: 97 active+clean; 49 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 276 op/s 2024-04-26T01:58:14.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:14 smithi097 ceph-mon[32853]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T01:58:14.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:14 smithi173 ceph-mon[32553]: pgmap v202: 97 pgs: 97 active+clean; 49 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 276 op/s 2024-04-26T01:58:14.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:14 smithi173 ceph-mon[32553]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T01:58:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:14 smithi001 ceph-mon[27822]: pgmap v202: 97 pgs: 97 active+clean; 49 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 276 op/s 2024-04-26T01:58:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:14 smithi001 ceph-mon[27822]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T01:58:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:16 smithi097 ceph-mon[32853]: pgmap v204: 97 pgs: 97 active+clean; 52 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.2 MiB/s wr, 350 op/s 2024-04-26T01:58:16.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:16 smithi173 ceph-mon[32553]: pgmap v204: 97 pgs: 97 active+clean; 52 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.2 MiB/s wr, 350 op/s 2024-04-26T01:58:16.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:16 smithi001 ceph-mon[27822]: pgmap v204: 97 pgs: 97 active+clean; 52 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.2 MiB/s wr, 350 op/s 2024-04-26T01:58:18.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:18 smithi097 ceph-mon[32853]: pgmap v205: 97 pgs: 97 active+clean; 52 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.2 MiB/s wr, 339 op/s 2024-04-26T01:58:18.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:18 smithi173 ceph-mon[32553]: pgmap v205: 97 pgs: 97 active+clean; 52 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.2 MiB/s wr, 339 op/s 2024-04-26T01:58:18.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:18 smithi001 ceph-mon[27822]: pgmap v205: 97 pgs: 97 active+clean; 52 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.2 MiB/s wr, 339 op/s 2024-04-26T01:58:20.137 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x9ffb01 2024-04-26T01:58:20.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:20 smithi097 ceph-mon[32853]: pgmap v206: 97 pgs: 97 active+clean; 52 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.2 MiB/s wr, 339 op/s 2024-04-26T01:58:20.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:20 smithi173 ceph-mon[32553]: pgmap v206: 97 pgs: 97 active+clean; 52 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.2 MiB/s wr, 339 op/s 2024-04-26T01:58:20.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:20 smithi001 ceph-mon[27822]: pgmap v206: 97 pgs: 97 active+clean; 52 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.2 MiB/s wr, 339 op/s 2024-04-26T01:58:21.866 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 15.9 2024-04-26T01:58:21.866 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:58:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:58:22.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:22 smithi001 ceph-mon[27822]: pgmap v207: 97 pgs: 97 active+clean; 49 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.1 MiB/s wr, 335 op/s 2024-04-26T01:58:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:22 smithi097 ceph-mon[32853]: pgmap v207: 97 pgs: 97 active+clean; 49 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.1 MiB/s wr, 335 op/s 2024-04-26T01:58:22.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:22 smithi173 ceph-mon[32553]: pgmap v207: 97 pgs: 97 active+clean; 49 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.1 MiB/s wr, 335 op/s 2024-04-26T01:58:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.297+0000 7fd572c00640 1 -- 172.21.15.1:0/3307539274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd56c1070b0 msgr2=0x7fd56c10ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.297+0000 7fd572c00640 1 --2- 172.21.15.1:0/3307539274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd56c1070b0 0x7fd56c10ec80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd564009ca0 tx=0x7fd56402a8c0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.297+0000 7fd572c00640 1 -- 172.21.15.1:0/3307539274 shutdown_connections 2024-04-26T01:58:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.297+0000 7fd572c00640 1 --2- 172.21.15.1:0/3307539274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd56c1070b0 0x7fd56c10ec80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.298+0000 7fd572c00640 1 --2- 172.21.15.1:0/3307539274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd56c102990 0x7fd56c1069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.298+0000 7fd572c00640 1 --2- 172.21.15.1:0/3307539274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd56c101fe0 0x7fd56c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.298+0000 7fd572c00640 1 -- 172.21.15.1:0/3307539274 >> 172.21.15.1:0/3307539274 conn(0x7fd56c0fd860 msgr2=0x7fd56c0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.298+0000 7fd572c00640 1 -- 172.21.15.1:0/3307539274 shutdown_connections 2024-04-26T01:58:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.298+0000 7fd572c00640 1 -- 172.21.15.1:0/3307539274 wait complete. 2024-04-26T01:58:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.299+0000 7fd572c00640 1 Processor -- start 2024-04-26T01:58:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.299+0000 7fd572c00640 1 -- start start 2024-04-26T01:58:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.299+0000 7fd572c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd56c101fe0 0x7fd56c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.299+0000 7fd572c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd56c1070b0 0x7fd56c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.299+0000 7fd572c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd56c19ce10 0x7fd56c19d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.299+0000 7fd572c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c112050 con 0x7fd56c1070b0 2024-04-26T01:58:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.299+0000 7fd572c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c111ed0 con 0x7fd56c101fe0 2024-04-26T01:58:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.299+0000 7fd572c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c1121d0 con 0x7fd56c19ce10 2024-04-26T01:58:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.300+0000 7fd571800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd56c101fe0 0x7fd56c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.300+0000 7fd572200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd56c19ce10 0x7fd56c19d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.300+0000 7fd571800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd56c101fe0 0x7fd56c198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:59228/0 (socket says 172.21.15.1:59228) 2024-04-26T01:58:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.300+0000 7fd570e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd56c1070b0 0x7fd56c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.300+0000 7fd571800640 1 -- 172.21.15.1:0/2239675371 learned_addr learned my addr 172.21.15.1:0/2239675371 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.300+0000 7fd571800640 1 -- 172.21.15.1:0/2239675371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd56c19ce10 msgr2=0x7fd56c19d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.300+0000 7fd571800640 1 --2- 172.21.15.1:0/2239675371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd56c19ce10 0x7fd56c19d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.300+0000 7fd571800640 1 -- 172.21.15.1:0/2239675371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd56c1070b0 msgr2=0x7fd56c198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.300+0000 7fd571800640 1 --2- 172.21.15.1:0/2239675371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd56c1070b0 0x7fd56c198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.300+0000 7fd571800640 1 -- 172.21.15.1:0/2239675371 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56c19d900 con 0x7fd56c101fe0 2024-04-26T01:58:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.300+0000 7fd572200640 1 --2- 172.21.15.1:0/2239675371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd56c19ce10 0x7fd56c19d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.300+0000 7fd571800640 1 --2- 172.21.15.1:0/2239675371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd56c101fe0 0x7fd56c198550 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd56000e9f0 tx=0x7fd56000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.301+0000 7fd56a000640 1 -- 172.21.15.1:0/2239675371 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56000cd80 con 0x7fd56c101fe0 2024-04-26T01:58:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.301+0000 7fd56a000640 1 -- 172.21.15.1:0/2239675371 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5600108e0 con 0x7fd56c101fe0 2024-04-26T01:58:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.301+0000 7fd56a000640 1 -- 172.21.15.1:0/2239675371 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd560018980 con 0x7fd56c101fe0 2024-04-26T01:58:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.301+0000 7fd572c00640 1 -- 172.21.15.1:0/2239675371 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd56c19dbf0 con 0x7fd56c101fe0 2024-04-26T01:58:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.301+0000 7fd572c00640 1 -- 172.21.15.1:0/2239675371 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd56c1a5430 con 0x7fd56c101fe0 2024-04-26T01:58:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.304+0000 7fd56a000640 1 -- 172.21.15.1:0/2239675371 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fd560010430 con 0x7fd56c101fe0 2024-04-26T01:58:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.304+0000 7fd5771f2740 1 Processor -- start 2024-04-26T01:58:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.304+0000 7fd5771f2740 1 -- start start 2024-04-26T01:58:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.305+0000 7fd5771f2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9bdc0 0x56252be9c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.305+0000 7fd5771f2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9c6e0 0x56252bea6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.305+0000 7fd5771f2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56252bea6fa0 0x56252bea9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.305+0000 7fd5771f2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56252bdfd900 con 0x56252bea6fa0 2024-04-26T01:58:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.305+0000 7fd5771f2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56252bda3ef0 con 0x56252be9c6e0 2024-04-26T01:58:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.305+0000 7fd5771f2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56252bdf98c0 con 0x56252be9bdc0 2024-04-26T01:58:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.305+0000 7fd571800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9c6e0 0x56252bea6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.305+0000 7fd572200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9bdc0 0x56252be9c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.305+0000 7fd570e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56252bea6fa0 0x56252bea9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.305+0000 7fd571800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9c6e0 0x56252bea6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:59232/0 (socket says 172.21.15.1:59232) 2024-04-26T01:58:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.305+0000 7fd571800640 1 -- 172.21.15.1:0/2417576765 learned_addr learned my addr 172.21.15.1:0/2417576765 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.306+0000 7fd571800640 1 -- 172.21.15.1:0/2417576765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9bdc0 msgr2=0x56252be9c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.306+0000 7fd571800640 1 --2- 172.21.15.1:0/2417576765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9bdc0 0x56252be9c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.306+0000 7fd571800640 1 -- 172.21.15.1:0/2417576765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56252bea6fa0 msgr2=0x56252bea9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.306+0000 7fd571800640 1 --2- 172.21.15.1:0/2417576765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56252bea6fa0 0x56252bea9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.306+0000 7fd571800640 1 -- 172.21.15.1:0/2417576765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56252bb10210 con 0x56252be9c6e0 2024-04-26T01:58:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.306+0000 7fd56a000640 1 --2- 172.21.15.1:0/2239675371 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd5540a3d80 0x7fd5540a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.306+0000 7fd571800640 1 --2- 172.21.15.1:0/2417576765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9c6e0 0x56252bea6a60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd5600ce160 tx=0x7fd5600d9620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ef75cc987fe9eff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.306+0000 7fd56a000640 1 -- 172.21.15.1:0/2239675371 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fd5600d0b60 con 0x7fd56c101fe0 2024-04-26T01:58:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.306+0000 7fd572200640 1 --2- 172.21.15.1:0/2239675371 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd5540a3d80 0x7fd5540a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.306+0000 7fd55ea00640 1 -- 172.21.15.1:0/2417576765 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd560098dc0 con 0x56252be9c6e0 2024-04-26T01:58:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.306+0000 7fd55ea00640 1 -- 172.21.15.1:0/2417576765 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd560010720 con 0x56252be9c6e0 2024-04-26T01:58:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.306+0000 7fd55ea00640 1 -- 172.21.15.1:0/2417576765 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56009c480 con 0x56252be9c6e0 2024-04-26T01:58:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.307+0000 7fd5771f2740 1 -- 172.21.15.1:0/2417576765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9c6e0 msgr2=0x56252bea6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.307+0000 7fd572200640 1 --2- 172.21.15.1:0/2239675371 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd5540a3d80 0x7fd5540a6240 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd564006010 tx=0x7fd564005d80 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.307+0000 7fd5771f2740 1 --2- 172.21.15.1:0/2417576765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9c6e0 0x56252bea6a60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd5600ce160 tx=0x7fd5600d9620 comp rx=0 tx=0).stop 2024-04-26T01:58:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.307+0000 7fd5771f2740 1 -- 172.21.15.1:0/2417576765 shutdown_connections 2024-04-26T01:58:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.307+0000 7fd5771f2740 1 --2- 172.21.15.1:0/2417576765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56252bea6fa0 0x56252bea9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.307+0000 7fd5771f2740 1 --2- 172.21.15.1:0/2417576765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9c6e0 0x56252bea6a60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.307+0000 7fd5771f2740 1 --2- 172.21.15.1:0/2417576765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9bdc0 0x56252be9c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.307+0000 7fd5771f2740 1 -- 172.21.15.1:0/2417576765 >> 172.21.15.1:0/2417576765 conn(0x56252be982d0 msgr2=0x56252be986b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.307+0000 7fd5771f2740 1 -- 172.21.15.1:0/2417576765 shutdown_connections 2024-04-26T01:58:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.307+0000 7fd5771f2740 1 -- 172.21.15.1:0/2417576765 wait complete. 2024-04-26T01:58:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.308+0000 7fd5771f2740 1 -- start start 2024-04-26T01:58:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.308+0000 7fd5771f2740 1 Processor -- start 2024-04-26T01:58:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.309+0000 7fd5771f2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9bdc0 0x56252bea3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.309+0000 7fd5771f2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9c6e0 0x56252bea44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.309+0000 7fd5771f2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56252bea6fa0 0x56252bea4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.309+0000 7fd5771f2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56252bea9b70 con 0x56252bea6fa0 2024-04-26T01:58:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.309+0000 7fd5771f2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56252bdb7310 con 0x56252be9c6e0 2024-04-26T01:58:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.309+0000 7fd5771f2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56252bea9cf0 con 0x56252be9bdc0 2024-04-26T01:58:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.309+0000 7fd5771f2740 10 client.0 _ll_register_callbacks cb 0x56252bd85cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.309+0000 7fd571800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9bdc0 0x56252bea3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.309+0000 7fd570e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9c6e0 0x56252bea44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.309+0000 7fd571800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9bdc0 0x56252bea3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:60078/0 (socket says 172.21.15.1:60078) 2024-04-26T01:58:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.309+0000 7fd571800640 1 -- 172.21.15.1:0/4113331413 learned_addr learned my addr 172.21.15.1:0/4113331413 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.309+0000 7fd572200640 1 --2- 172.21.15.1:0/4113331413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56252bea6fa0 0x56252bea4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.309+0000 7fd571800640 1 -- 172.21.15.1:0/4113331413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9c6e0 msgr2=0x56252bea44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.309+0000 7fd571800640 1 --2- 172.21.15.1:0/4113331413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9c6e0 0x56252bea44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.309+0000 7fd571800640 1 -- 172.21.15.1:0/4113331413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56252bea6fa0 msgr2=0x56252bea4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.309+0000 7fd571800640 1 --2- 172.21.15.1:0/4113331413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56252bea6fa0 0x56252bea4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.309+0000 7fd571800640 1 -- 172.21.15.1:0/4113331413 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56252bbdc620 con 0x56252be9bdc0 2024-04-26T01:58:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.310+0000 7fd570e00640 1 --2- 172.21.15.1:0/4113331413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9c6e0 0x56252bea44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.310+0000 7fd571800640 1 --2- 172.21.15.1:0/4113331413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9bdc0 0x56252bea3f90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd5600d0e50 tx=0x7fd560018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c95ca5a29da95b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.310+0000 7fd553400640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:58:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.310+0000 7fd553400640 1 -- 172.21.15.1:0/4113331413 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56009c660 con 0x56252be9bdc0 2024-04-26T01:58:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.310+0000 7fd553400640 1 -- 172.21.15.1:0/4113331413 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fd560099190 con 0x56252be9bdc0 2024-04-26T01:58:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.310+0000 7fd5771f2740 1 -- 172.21.15.1:0/4113331413 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56252be118b0 con 0x56252be9bdc0 2024-04-26T01:58:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.310+0000 7fd553400640 1 -- 172.21.15.1:0/4113331413 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd56009c8f0 con 0x56252be9bdc0 2024-04-26T01:58:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.310+0000 7fd5771f2740 10 client.25981 fetch_fsmap learned FSMap version 32 2024-04-26T01:58:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.310+0000 7fd5771f2740 1 -- 172.21.15.1:0/4113331413 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56252bea9b70 con 0x56252be9bdc0 2024-04-26T01:58:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.311+0000 7fd553400640 1 -- 172.21.15.1:0/4113331413 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56252be118b0 con 0x56252be9bdc0 2024-04-26T01:58:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.311+0000 7fd553400640 1 -- 172.21.15.1:0/4113331413 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fd5600ce4a0 con 0x56252be9bdc0 2024-04-26T01:58:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.311+0000 7fd5771f2740 10 client.25981 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:58:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.311+0000 7fd5771f2740 10 client.25981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:58:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.311+0000 7fd5771f2740 20 client.25981 populate_metadata read hostname 'smithi001' 2024-04-26T01:58:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.311+0000 7fd5771f2740 1 --2- 172.21.15.1:0/4113331413 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56252beb6fb0 0x56252beb93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.311+0000 7fd5771f2740 4 client.25981 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:58:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.311+0000 7fd5771f2740 1 -- 172.21.15.1:0/4113331413 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56252be0b320 con 0x56252beb6fb0 2024-04-26T01:58:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.311+0000 7fd570e00640 1 --2- 172.21.15.1:0/4113331413 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56252beb6fb0 0x56252beb93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.312+0000 7fd570e00640 1 --2- 172.21.15.1:0/4113331413 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56252beb6fb0 0x56252beb93a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d0db08c474e021a server_cookie=1bccd59c5a8532e8 in_seq=0 out_seq=0 2024-04-26T01:58:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.312+0000 7fd553400640 10 client.25981 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.313+0000 7fd553400640 1 -- 172.21.15.1:0/4113331413 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56252be0b320 con 0x56252beb6fb0 2024-04-26T01:58:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.313+0000 7fd553400640 10 client.25981 handle_command_reply: tid=0 2024-04-26T01:58:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.313+0000 7fd5771f2740 1 client.25981 shutdown 2024-04-26T01:58:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.313+0000 7fd5771f2740 1 -- 172.21.15.1:0/4113331413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9bdc0 msgr2=0x56252bea3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.313+0000 7fd5771f2740 1 --2- 172.21.15.1:0/4113331413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9bdc0 0x56252bea3f90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd5600d0e50 tx=0x7fd560018420 comp rx=0 tx=0).stop 2024-04-26T01:58:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.313+0000 7fd5771f2740 1 -- 172.21.15.1:0/4113331413 shutdown_connections 2024-04-26T01:58:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.313+0000 7fd5771f2740 1 --2- 172.21.15.1:0/4113331413 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56252beb6fb0 0x56252beb93a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.313+0000 7fd5771f2740 1 --2- 172.21.15.1:0/4113331413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56252bea6fa0 0x56252bea4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.313+0000 7fd5771f2740 1 --2- 172.21.15.1:0/4113331413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9c6e0 0x56252bea44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.313+0000 7fd5771f2740 1 --2- 172.21.15.1:0/4113331413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9bdc0 0x56252bea3f90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.313+0000 7fd5771f2740 1 -- 172.21.15.1:0/4113331413 >> 172.21.15.1:0/4113331413 conn(0x56252be982d0 msgr2=0x56252be93020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.313+0000 7fd553400640 0 client.25981 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.313+0000 7fd5771f2740 1 -- 172.21.15.1:0/4113331413 shutdown_connections 2024-04-26T01:58:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.313+0000 7fd5771f2740 1 -- 172.21.15.1:0/4113331413 wait complete. 2024-04-26T01:58:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.314+0000 7fd5771f2740 20 client.25981 trim_cache size 0 max 16384 2024-04-26T01:58:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.334+0000 7fd5771f2740 1 Processor -- start 2024-04-26T01:58:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.334+0000 7fd5771f2740 1 -- start start 2024-04-26T01:58:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.335+0000 7fd5771f2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56252be982d0 0x56252be9e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.335+0000 7fd5771f2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9bdc0 0x56252bea4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.335+0000 7fd5771f2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9c6e0 0x56252bea5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.335+0000 7fd5771f2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c1ae2c0 con 0x56252be982d0 2024-04-26T01:58:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.335+0000 7fd5771f2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c1abb90 con 0x56252be9bdc0 2024-04-26T01:58:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.335+0000 7fd5771f2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c1a9a00 con 0x56252be9c6e0 2024-04-26T01:58:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.335+0000 7fd570e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9bdc0 0x56252bea4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.335+0000 7fd571800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56252be982d0 0x56252be9e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.335+0000 7fd570e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9bdc0 0x56252bea4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:59250/0 (socket says 172.21.15.1:59250) 2024-04-26T01:58:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.335+0000 7fd570e00640 1 -- 172.21.15.1:0/2225527022 learned_addr learned my addr 172.21.15.1:0/2225527022 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.335+0000 7fd572200640 1 --2- 172.21.15.1:0/2225527022 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9c6e0 0x56252bea5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.335+0000 7fd570e00640 1 -- 172.21.15.1:0/2225527022 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9c6e0 msgr2=0x56252bea5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.335+0000 7fd570e00640 1 --2- 172.21.15.1:0/2225527022 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9c6e0 0x56252bea5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.335+0000 7fd570e00640 1 -- 172.21.15.1:0/2225527022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56252be982d0 msgr2=0x56252be9e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.335+0000 7fd570e00640 1 --2- 172.21.15.1:0/2225527022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56252be982d0 0x56252be9e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.335+0000 7fd570e00640 1 -- 172.21.15.1:0/2225527022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56252be9d070 con 0x56252be9bdc0 2024-04-26T01:58:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.336+0000 7fd570e00640 1 --2- 172.21.15.1:0/2225527022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9bdc0 0x56252bea4830 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x56252be99460 tx=0x7fd558005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=573df9e46ecb89a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.336+0000 7fd55e000640 1 -- 172.21.15.1:0/2225527022 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd558005220 con 0x56252be9bdc0 2024-04-26T01:58:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.336+0000 7fd55e000640 1 -- 172.21.15.1:0/2225527022 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd558004c50 con 0x56252be9bdc0 2024-04-26T01:58:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.336+0000 7fd55e000640 1 -- 172.21.15.1:0/2225527022 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd558003b50 con 0x56252be9bdc0 2024-04-26T01:58:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.337+0000 7fd5771f2740 1 -- 172.21.15.1:0/2225527022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9bdc0 msgr2=0x56252bea4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.337+0000 7fd5771f2740 1 --2- 172.21.15.1:0/2225527022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9bdc0 0x56252bea4830 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x56252be99460 tx=0x7fd558005f90 comp rx=0 tx=0).stop 2024-04-26T01:58:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.337+0000 7fd5771f2740 1 -- 172.21.15.1:0/2225527022 shutdown_connections 2024-04-26T01:58:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.337+0000 7fd5771f2740 1 --2- 172.21.15.1:0/2225527022 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9c6e0 0x56252bea5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.337+0000 7fd5771f2740 1 --2- 172.21.15.1:0/2225527022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9bdc0 0x56252bea4830 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.337+0000 7fd5771f2740 1 --2- 172.21.15.1:0/2225527022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56252be982d0 0x56252be9e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.337+0000 7fd5771f2740 1 -- 172.21.15.1:0/2225527022 >> 172.21.15.1:0/2225527022 conn(0x56252be9f960 msgr2=0x56252bea57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.337+0000 7fd5771f2740 1 -- 172.21.15.1:0/2225527022 shutdown_connections 2024-04-26T01:58:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.337+0000 7fd5771f2740 1 -- 172.21.15.1:0/2225527022 wait complete. 2024-04-26T01:58:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.338+0000 7fd5771f2740 1 -- start start 2024-04-26T01:58:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.338+0000 7fd5771f2740 1 Processor -- start 2024-04-26T01:58:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.338+0000 7fd5771f2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56252be982d0 0x56252beac100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.338+0000 7fd5771f2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9bdc0 0x56252beac640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.338+0000 7fd5771f2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9c6e0 0x56252bf1b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.338+0000 7fd5771f2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c1aa050 con 0x56252be982d0 2024-04-26T01:58:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.338+0000 7fd5771f2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c1ab4c0 con 0x56252be9c6e0 2024-04-26T01:58:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.338+0000 7fd5771f2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c1aa720 con 0x56252be9bdc0 2024-04-26T01:58:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.339+0000 7fd5771f2740 10 client.0 _ll_register_callbacks cb 0x7fd560005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.339+0000 7fd570e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9bdc0 0x56252beac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.339+0000 7fd570e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9bdc0 0x56252beac640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:60098/0 (socket says 172.21.15.1:60098) 2024-04-26T01:58:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.339+0000 7fd570e00640 1 -- 172.21.15.1:0/2625997865 learned_addr learned my addr 172.21.15.1:0/2625997865 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.339+0000 7fd572200640 1 --2- 172.21.15.1:0/2625997865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9c6e0 0x56252bf1b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.339+0000 7fd571800640 1 --2- 172.21.15.1:0/2625997865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56252be982d0 0x56252beac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.339+0000 7fd572200640 1 -- 172.21.15.1:0/2625997865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9bdc0 msgr2=0x56252beac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.339+0000 7fd572200640 1 --2- 172.21.15.1:0/2625997865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9bdc0 0x56252beac640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.339+0000 7fd572200640 1 -- 172.21.15.1:0/2625997865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56252be982d0 msgr2=0x56252beac100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.339+0000 7fd572200640 1 --2- 172.21.15.1:0/2625997865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56252be982d0 0x56252beac100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.339+0000 7fd572200640 1 -- 172.21.15.1:0/2625997865 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56252bf1b7e0 con 0x56252be9c6e0 2024-04-26T01:58:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.339+0000 7fd570e00640 1 --2- 172.21.15.1:0/2625997865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9bdc0 0x56252beac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.340+0000 7fd572200640 1 --2- 172.21.15.1:0/2625997865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9c6e0 0x56252bf1b2a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd564005e30 tx=0x7fd56402a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ace1fd029dce28bf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.340+0000 7fd553400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:58:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.340+0000 7fd553400640 1 -- 172.21.15.1:0/2625997865 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564059060 con 0x56252be9c6e0 2024-04-26T01:58:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.340+0000 7fd553400640 1 -- 172.21.15.1:0/2625997865 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fd564043030 con 0x56252be9c6e0 2024-04-26T01:58:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.340+0000 7fd5771f2740 1 -- 172.21.15.1:0/2625997865 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd558004c50 con 0x56252be9c6e0 2024-04-26T01:58:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.340+0000 7fd553400640 1 -- 172.21.15.1:0/2625997865 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd56402f8e0 con 0x56252be9c6e0 2024-04-26T01:58:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.340+0000 7fd5771f2740 10 client.25970 fetch_fsmap learned FSMap version 32 2024-04-26T01:58:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.340+0000 7fd5771f2740 1 -- 172.21.15.1:0/2625997865 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56252bf1cb20 con 0x56252be9c6e0 2024-04-26T01:58:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.341+0000 7fd553400640 1 -- 172.21.15.1:0/2625997865 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd558004c50 con 0x56252be9c6e0 2024-04-26T01:58:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.341+0000 7fd553400640 1 -- 172.21.15.1:0/2625997865 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fd564030d00 con 0x56252be9c6e0 2024-04-26T01:58:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.341+0000 7fd5771f2740 10 client.25970 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:58:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.341+0000 7fd5771f2740 10 client.25970 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:58:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.341+0000 7fd5771f2740 20 client.25970 populate_metadata read hostname 'smithi001' 2024-04-26T01:58:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.341+0000 7fd5771f2740 1 --2- 172.21.15.1:0/2625997865 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56252bea6fa0 0x56252bf1f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.341+0000 7fd5771f2740 4 client.25970 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:58:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.341+0000 7fd5771f2740 1 -- 172.21.15.1:0/2625997865 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56252be05190 con 0x56252bea6fa0 2024-04-26T01:58:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.341+0000 7fd571800640 1 --2- 172.21.15.1:0/2625997865 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56252bea6fa0 0x56252bf1f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.342+0000 7fd571800640 1 --2- 172.21.15.1:0/2625997865 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56252bea6fa0 0x56252bf1f360 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89dcb4f04a657654 server_cookie=70945276f114771f in_seq=0 out_seq=0 2024-04-26T01:58:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.342+0000 7fd553400640 10 client.25970 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.356+0000 7fd553400640 1 -- 172.21.15.1:0/2625997865 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56252be05190 con 0x56252bea6fa0 2024-04-26T01:58:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.356+0000 7fd553400640 10 client.25970 handle_command_reply: tid=0 2024-04-26T01:58:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.356+0000 7fd5771f2740 1 client.25970 shutdown 2024-04-26T01:58:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.357+0000 7fd5771f2740 1 -- 172.21.15.1:0/2625997865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9c6e0 msgr2=0x56252bf1b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.357+0000 7fd5771f2740 1 --2- 172.21.15.1:0/2625997865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9c6e0 0x56252bf1b2a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd564005e30 tx=0x7fd56402a8c0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.357+0000 7fd5771f2740 1 -- 172.21.15.1:0/2625997865 shutdown_connections 2024-04-26T01:58:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.357+0000 7fd5771f2740 1 --2- 172.21.15.1:0/2625997865 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56252bea6fa0 0x56252bf1f360 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.357+0000 7fd5771f2740 1 --2- 172.21.15.1:0/2625997865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56252be9c6e0 0x56252bf1b2a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.357+0000 7fd5771f2740 1 --2- 172.21.15.1:0/2625997865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56252be9bdc0 0x56252beac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.357+0000 7fd5771f2740 1 --2- 172.21.15.1:0/2625997865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56252be982d0 0x56252beac100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.357+0000 7fd5771f2740 1 -- 172.21.15.1:0/2625997865 >> 172.21.15.1:0/2625997865 conn(0x56252be9f960 msgr2=0x56252bea6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.357+0000 7fd553400640 0 client.25970 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.357+0000 7fd5771f2740 1 -- 172.21.15.1:0/2625997865 shutdown_connections 2024-04-26T01:58:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.357+0000 7fd5771f2740 1 -- 172.21.15.1:0/2625997865 wait complete. 2024-04-26T01:58:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.357+0000 7fd5771f2740 20 client.25970 trim_cache size 0 max 16384 2024-04-26T01:58:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:58:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:58:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:58:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:58:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.358+0000 7fd572c00640 1 -- 172.21.15.1:0/2239675371 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd5540a3d80 msgr2=0x7fd5540a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.358+0000 7fd572c00640 1 --2- 172.21.15.1:0/2239675371 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd5540a3d80 0x7fd5540a6240 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd564006010 tx=0x7fd564005d80 comp rx=0 tx=0).stop 2024-04-26T01:58:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.358+0000 7fd572c00640 1 -- 172.21.15.1:0/2239675371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd56c101fe0 msgr2=0x7fd56c198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.358+0000 7fd572c00640 1 --2- 172.21.15.1:0/2239675371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd56c101fe0 0x7fd56c198550 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd56000e9f0 tx=0x7fd56000eeb0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.358+0000 7fd572c00640 1 -- 172.21.15.1:0/2239675371 shutdown_connections 2024-04-26T01:58:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.358+0000 7fd572c00640 1 --2- 172.21.15.1:0/2239675371 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd5540a3d80 0x7fd5540a6240 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.358+0000 7fd572c00640 1 --2- 172.21.15.1:0/2239675371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd56c19ce10 0x7fd56c19d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.358+0000 7fd572c00640 1 --2- 172.21.15.1:0/2239675371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd56c1070b0 0x7fd56c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.358+0000 7fd572c00640 1 --2- 172.21.15.1:0/2239675371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd56c101fe0 0x7fd56c198550 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.358+0000 7fd572c00640 1 -- 172.21.15.1:0/2239675371 >> 172.21.15.1:0/2239675371 conn(0x7fd56c0fd860 msgr2=0x7fd56c0ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.358+0000 7fd572c00640 1 -- 172.21.15.1:0/2239675371 shutdown_connections 2024-04-26T01:58:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:24.358+0000 7fd572c00640 1 -- 172.21.15.1:0/2239675371 wait complete. 2024-04-26T01:58:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:24 smithi097 ceph-mon[32853]: pgmap v208: 97 pgs: 97 active+clean; 49 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.1 MiB/s wr, 335 op/s 2024-04-26T01:58:24.489 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:24 smithi001 ceph-mon[27822]: pgmap v208: 97 pgs: 97 active+clean; 49 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.1 MiB/s wr, 335 op/s 2024-04-26T01:58:24.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:24 smithi173 ceph-mon[32553]: pgmap v208: 97 pgs: 97 active+clean; 49 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.1 MiB/s wr, 335 op/s 2024-04-26T01:58:24.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.853+0000 7fbb3e800700 1 -- 172.21.15.1:0/451581772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb38103450 msgr2=0x7fbb3810f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:24.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.853+0000 7fbb3e800700 1 --2- 172.21.15.1:0/451581772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb38103450 0x7fbb3810f8b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbb30009f90 tx=0x7fbb3000b330 comp rx=0 tx=0).stop 2024-04-26T01:58:24.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.853+0000 7fbb3e800700 1 -- 172.21.15.1:0/451581772 shutdown_connections 2024-04-26T01:58:24.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.853+0000 7fbb3e800700 1 --2- 172.21.15.1:0/451581772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb38103450 0x7fbb3810f8b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.853+0000 7fbb3e800700 1 --2- 172.21.15.1:0/451581772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb38108a50 0x7fbb38108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.853+0000 7fbb3e800700 1 --2- 172.21.15.1:0/451581772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb38102a90 0x7fbb38102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.853+0000 7fbb3e800700 1 -- 172.21.15.1:0/451581772 >> 172.21.15.1:0/451581772 conn(0x7fbb38079ba0 msgr2=0x7fbb38079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:24.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.853+0000 7fbb3e800700 1 -- 172.21.15.1:0/451581772 shutdown_connections 2024-04-26T01:58:24.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.853+0000 7fbb3e800700 1 -- 172.21.15.1:0/451581772 wait complete. 2024-04-26T01:58:24.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.854+0000 7fbb3e800700 1 Processor -- start 2024-04-26T01:58:24.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.855+0000 7fbb3e800700 1 -- start start 2024-04-26T01:58:24.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.855+0000 7fbb3e800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb38102a90 0x7fbb38074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.855+0000 7fbb3e800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb38103450 0x7fbb38075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.855+0000 7fbb3e800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb38108a50 0x7fbb380757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.855+0000 7fbb3e800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbb38112c50 con 0x7fbb38102a90 2024-04-26T01:58:24.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.855+0000 7fbb3e800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbb38112ad0 con 0x7fbb38108a50 2024-04-26T01:58:24.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.855+0000 7fbb3e800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbb38112dd0 con 0x7fbb38103450 2024-04-26T01:58:24.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.855+0000 7fbb37400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb38102a90 0x7fbb38074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.855+0000 7fbb37400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb38102a90 0x7fbb38074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57870/0 (socket says 172.21.15.1:57870) 2024-04-26T01:58:24.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.855+0000 7fbb37400700 1 -- 172.21.15.1:0/447015882 learned_addr learned my addr 172.21.15.1:0/447015882 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:24.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.856+0000 7fbb37e00700 1 --2- 172.21.15.1:0/447015882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb38108a50 0x7fbb380757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.856+0000 7fbb36a00700 1 --2- 172.21.15.1:0/447015882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb38103450 0x7fbb38075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.856+0000 7fbb37400700 1 -- 172.21.15.1:0/447015882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb38103450 msgr2=0x7fbb38075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:24.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.856+0000 7fbb37400700 1 --2- 172.21.15.1:0/447015882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb38103450 0x7fbb38075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.856+0000 7fbb37400700 1 -- 172.21.15.1:0/447015882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb38108a50 msgr2=0x7fbb380757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:24.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.856+0000 7fbb37400700 1 --2- 172.21.15.1:0/447015882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb38108a50 0x7fbb380757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:24.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.856+0000 7fbb37400700 1 -- 172.21.15.1:0/447015882 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb381a7650 con 0x7fbb38102a90 2024-04-26T01:58:24.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.856+0000 7fbb37e00700 1 --2- 172.21.15.1:0/447015882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb38108a50 0x7fbb380757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:24.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.857+0000 7fbb37400700 1 --2- 172.21.15.1:0/447015882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb38102a90 0x7fbb38074d40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800eb30 tx=0x7fbb2800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:24.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.857+0000 7fbb2fe00700 1 -- 172.21.15.1:0/447015882 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2800cbd0 con 0x7fbb38102a90 2024-04-26T01:58:24.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.857+0000 7fbb3e800700 1 -- 172.21.15.1:0/447015882 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb381a7940 con 0x7fbb38102a90 2024-04-26T01:58:24.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.857+0000 7fbb3e800700 1 -- 172.21.15.1:0/447015882 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb381a7ed0 con 0x7fbb38102a90 2024-04-26T01:58:24.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.857+0000 7fbb2fe00700 1 -- 172.21.15.1:0/447015882 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb280044d0 con 0x7fbb38102a90 2024-04-26T01:58:24.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.857+0000 7fbb2fe00700 1 -- 172.21.15.1:0/447015882 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28003e30 con 0x7fbb38102a90 2024-04-26T01:58:24.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.860+0000 7fbb3e800700 1 -- 172.21.15.1:0/447015882 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb18005330 con 0x7fbb38102a90 2024-04-26T01:58:24.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.863+0000 7fbb2fe00700 1 -- 172.21.15.1:0/447015882 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fbb28004000 con 0x7fbb38102a90 2024-04-26T01:58:24.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.865+0000 7fbb2fe00700 1 --2- 172.21.15.1:0/447015882 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fbb1c0a3eb0 0x7fbb1c0a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:24.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.865+0000 7fbb2fe00700 1 -- 172.21.15.1:0/447015882 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fbb280d0a10 con 0x7fbb38102a90 2024-04-26T01:58:24.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.866+0000 7fbb36a00700 1 --2- 172.21.15.1:0/447015882 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fbb1c0a3eb0 0x7fbb1c0a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:24.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.867+0000 7fbb36a00700 1 --2- 172.21.15.1:0/447015882 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fbb1c0a3eb0 0x7fbb1c0a6370 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbb24009750 tx=0x7fbb24009310 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:24.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:24.871+0000 7fbb2fe00700 1 -- 172.21.15.1:0/447015882 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbb280042f0 con 0x7fbb38102a90 2024-04-26T01:58:25.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:25.205+0000 7fbb3e800700 1 -- 172.21.15.1:0/447015882 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb180058f0 con 0x7fbb38102a90 2024-04-26T01:58:25.206 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:58:25.206 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:55:49.645549+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25414":{"gid":25414,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/453427322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":453427322},{"type":"v1","addr":"172.21.15.173:6835","nonce":453427322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24499,24514,24518]},"id":1}]} 2024-04-26T01:58:25.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:25.207+0000 7fbb2fe00700 1 -- 172.21.15.1:0/447015882 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11838 (secure 0 0 0) 0x7fbb2809d020 con 0x7fbb38102a90 2024-04-26T01:58:25.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:25.210+0000 7fbb3e800700 1 -- 172.21.15.1:0/447015882 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fbb1c0a3eb0 msgr2=0x7fbb1c0a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:25.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:25.210+0000 7fbb3e800700 1 --2- 172.21.15.1:0/447015882 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fbb1c0a3eb0 0x7fbb1c0a6370 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbb24009750 tx=0x7fbb24009310 comp rx=0 tx=0).stop 2024-04-26T01:58:25.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:25.210+0000 7fbb3e800700 1 -- 172.21.15.1:0/447015882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb38102a90 msgr2=0x7fbb38074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:25.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:25.210+0000 7fbb3e800700 1 --2- 172.21.15.1:0/447015882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb38102a90 0x7fbb38074d40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800eb30 tx=0x7fbb2800ee40 comp rx=0 tx=0).stop 2024-04-26T01:58:25.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:25.210+0000 7fbb3e800700 1 -- 172.21.15.1:0/447015882 shutdown_connections 2024-04-26T01:58:25.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:25.210+0000 7fbb3e800700 1 --2- 172.21.15.1:0/447015882 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fbb1c0a3eb0 0x7fbb1c0a6370 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:25.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:25.210+0000 7fbb3e800700 1 --2- 172.21.15.1:0/447015882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb38108a50 0x7fbb380757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:25.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:25.210+0000 7fbb3e800700 1 --2- 172.21.15.1:0/447015882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb38102a90 0x7fbb38074d40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:25.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:25.211+0000 7fbb3e800700 1 --2- 172.21.15.1:0/447015882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb38103450 0x7fbb38075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:25.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:25.211+0000 7fbb3e800700 1 -- 172.21.15.1:0/447015882 >> 172.21.15.1:0/447015882 conn(0x7fbb38079ba0 msgr2=0x7fbb38101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:25.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:25.211+0000 7fbb3e800700 1 -- 172.21.15.1:0/447015882 shutdown_connections 2024-04-26T01:58:25.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:25.211+0000 7fbb3e800700 1 -- 172.21.15.1:0/447015882 wait complete. 2024-04-26T01:58:25.212 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 32 2024-04-26T01:58:26.030 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24518 quiesce db / --timeout 60 --expiration 75.9 --await-for 42.0' 2024-04-26T01:58:26.030 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24518 quiesce db / --timeout 60 --expiration 75.9 --await-for 42.0 2024-04-26T01:58:26.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:26 smithi001 ceph-mon[27822]: pgmap v209: 97 pgs: 97 active+clean; 51 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.0 MiB/s wr, 484 op/s 2024-04-26T01:58:26.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:26 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/447015882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:58:26.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:26 smithi097 ceph-mon[32853]: pgmap v209: 97 pgs: 97 active+clean; 51 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.0 MiB/s wr, 484 op/s 2024-04-26T01:58:26.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:26 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/447015882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:58:26.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:26 smithi173 ceph-mon[32553]: pgmap v209: 97 pgs: 97 active+clean; 51 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.0 MiB/s wr, 484 op/s 2024-04-26T01:58:26.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:26 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/447015882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:58:26.622 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:58:27.108 INFO:tasks.workunit.client.0.smithi001.stdout:40000 mapwrite 0x1becad thru 0x1c6a92 (0x7de6 bytes) 2024-04-26T01:58:28.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:28 smithi097 ceph-mon[32853]: pgmap v210: 97 pgs: 97 active+clean; 51 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.8 MiB/s wr, 286 op/s 2024-04-26T01:58:28.492 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:28 smithi001 ceph-mon[27822]: pgmap v210: 97 pgs: 97 active+clean; 51 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.8 MiB/s wr, 286 op/s 2024-04-26T01:58:28.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:28 smithi173 ceph-mon[32553]: pgmap v210: 97 pgs: 97 active+clean; 51 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.8 MiB/s wr, 286 op/s 2024-04-26T01:58:29.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.051+0000 7f6a10c00700 1 -- 172.21.15.1:0/1824797421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0c104410 msgr2=0x7f6a0c1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:29.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.051+0000 7f6a10c00700 1 --2- 172.21.15.1:0/1824797421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0c104410 0x7f6a0c1088c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f69f80099c0 tx=0x7f69f8009cd0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.051+0000 7f6a10c00700 1 -- 172.21.15.1:0/1824797421 shutdown_connections 2024-04-26T01:58:29.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.051+0000 7f6a10c00700 1 --2- 172.21.15.1:0/1824797421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a0c108ff0 0x7f6a0c10f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.051+0000 7f6a10c00700 1 --2- 172.21.15.1:0/1824797421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a0c103a60 0x7f6a0c103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.051+0000 7f6a10c00700 1 --2- 172.21.15.1:0/1824797421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0c104410 0x7f6a0c1088c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.051+0000 7f6a10c00700 1 -- 172.21.15.1:0/1824797421 >> 172.21.15.1:0/1824797421 conn(0x7f6a0c0fd290 msgr2=0x7f6a0c0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:29.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.052+0000 7f6a10c00700 1 -- 172.21.15.1:0/1824797421 shutdown_connections 2024-04-26T01:58:29.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.052+0000 7f6a10c00700 1 -- 172.21.15.1:0/1824797421 wait complete. 2024-04-26T01:58:29.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.053+0000 7f6a10c00700 1 Processor -- start 2024-04-26T01:58:29.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.053+0000 7f6a10c00700 1 -- start start 2024-04-26T01:58:29.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.053+0000 7f6a10c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a0c103a60 0x7f6a0c198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:29.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.053+0000 7f6a10c00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0c104410 0x7f6a0c199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:29.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.053+0000 7f6a10c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a0c108ff0 0x7f6a0c19d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:29.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.053+0000 7f6a10c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c112c60 con 0x7f6a0c108ff0 2024-04-26T01:58:29.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.053+0000 7f6a10c00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c112ae0 con 0x7f6a0c104410 2024-04-26T01:58:29.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.053+0000 7f6a10c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c112de0 con 0x7f6a0c103a60 2024-04-26T01:58:29.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.054+0000 7f6a0b400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a0c103a60 0x7f6a0c198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:29.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.054+0000 7f6a0b400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a0c103a60 0x7f6a0c198ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:60132/0 (socket says 172.21.15.1:60132) 2024-04-26T01:58:29.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.054+0000 7f6a0b400700 1 -- 172.21.15.1:0/1410673810 learned_addr learned my addr 172.21.15.1:0/1410673810 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:29.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.054+0000 7f6a0aa00700 1 --2- 172.21.15.1:0/1410673810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0c104410 0x7f6a0c199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:29.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.054+0000 7f6a0be00700 1 --2- 172.21.15.1:0/1410673810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a0c108ff0 0x7f6a0c19d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:29.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.054+0000 7f6a0b400700 1 -- 172.21.15.1:0/1410673810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0c104410 msgr2=0x7f6a0c199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:29.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.054+0000 7f6a0b400700 1 --2- 172.21.15.1:0/1410673810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0c104410 0x7f6a0c199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.054+0000 7f6a0b400700 1 -- 172.21.15.1:0/1410673810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a0c108ff0 msgr2=0x7f6a0c19d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:29.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.054+0000 7f6a0b400700 1 --2- 172.21.15.1:0/1410673810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a0c108ff0 0x7f6a0c19d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.054+0000 7f6a0b400700 1 -- 172.21.15.1:0/1410673810 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c19e000 con 0x7f6a0c103a60 2024-04-26T01:58:29.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.055+0000 7f6a0b400700 1 --2- 172.21.15.1:0/1410673810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a0c103a60 0x7f6a0c198ff0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f69fc012d80 tx=0x7f69fc00c5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:29.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.055+0000 7f6a03e00700 1 -- 172.21.15.1:0/1410673810 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc004020 con 0x7f6a0c103a60 2024-04-26T01:58:29.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.055+0000 7f6a10c00700 1 -- 172.21.15.1:0/1410673810 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0c19e2f0 con 0x7f6a0c103a60 2024-04-26T01:58:29.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.055+0000 7f6a03e00700 1 -- 172.21.15.1:0/1410673810 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69fc003680 con 0x7f6a0c103a60 2024-04-26T01:58:29.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.055+0000 7f6a03e00700 1 -- 172.21.15.1:0/1410673810 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc01a670 con 0x7f6a0c103a60 2024-04-26T01:58:29.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.055+0000 7f6a10c00700 1 -- 172.21.15.1:0/1410673810 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0c1015f0 con 0x7f6a0c103a60 2024-04-26T01:58:29.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.059+0000 7f6a03e00700 1 -- 172.21.15.1:0/1410673810 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f69fc021020 con 0x7f6a0c103a60 2024-04-26T01:58:29.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.060+0000 7f6a1caa0b80 1 Processor -- start 2024-04-26T01:58:29.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.061+0000 7f6a1caa0b80 1 -- start start 2024-04-26T01:58:29.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.061+0000 7f6a1caa0b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d17b80 0x55aa02d17f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:29.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.061+0000 7f6a1caa0b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d184a0 0x55aa02d22e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:29.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.061+0000 7f6a1caa0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d23350 0x55aa02d25740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:29.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.061+0000 7f6a1caa0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55aa02c6b020 con 0x55aa02d23350 2024-04-26T01:58:29.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.061+0000 7f6a1caa0b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55aa02c118c0 con 0x55aa02d184a0 2024-04-26T01:58:29.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.061+0000 7f6a1caa0b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55aa02c66fe0 con 0x55aa02d17b80 2024-04-26T01:58:29.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.061+0000 7f6a0be00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d17b80 0x55aa02d17f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:29.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.061+0000 7f6a0be00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d17b80 0x55aa02d17f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:60136/0 (socket says 172.21.15.1:60136) 2024-04-26T01:58:29.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.061+0000 7f6a03e00700 1 --2- 172.21.15.1:0/1410673810 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f69f00a3e50 0x7f69f00a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:29.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.061+0000 7f6a0be00700 1 -- 172.21.15.1:0/2828402274 learned_addr learned my addr 172.21.15.1:0/2828402274 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:29.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.061+0000 7f6a0b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d184a0 0x55aa02d22e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:29.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.062+0000 7f6a0aa00700 1 --2- 172.21.15.1:0/2828402274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d23350 0x55aa02d25740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:29.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.062+0000 7f6a03e00700 1 -- 172.21.15.1:0/1410673810 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f69fc0d45d0 con 0x7f6a0c103a60 2024-04-26T01:58:29.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.062+0000 7f6a0be00700 1 -- 172.21.15.1:0/2828402274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d184a0 msgr2=0x55aa02d22e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:29.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.062+0000 7f6a0be00700 1 --2- 172.21.15.1:0/2828402274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d184a0 0x55aa02d22e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.062+0000 7f6a0be00700 1 -- 172.21.15.1:0/2828402274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d23350 msgr2=0x55aa02d25740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:29.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.062+0000 7f6a0be00700 1 --2- 172.21.15.1:0/2828402274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d23350 0x55aa02d25740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.062+0000 7f6a0be00700 1 -- 172.21.15.1:0/2828402274 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa02c07810 con 0x55aa02d17b80 2024-04-26T01:58:29.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.062+0000 7f6a0b400700 1 --2- 172.21.15.1:0/2828402274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d184a0 0x55aa02d22e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:29.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.062+0000 7f6a0be00700 1 --2- 172.21.15.1:0/1410673810 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f69f00a3e50 0x7f69f00a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:29.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.062+0000 7f6a0be00700 1 --2- 172.21.15.1:0/2828402274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d17b80 0x55aa02d17f60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6a04011790 tx=0x7f6a04011aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86c9e7adc7ed1674 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:29.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.063+0000 7f6a00c00700 1 -- 172.21.15.1:0/2828402274 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a04007900 con 0x55aa02d17b80 2024-04-26T01:58:29.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.064+0000 7f6a00c00700 1 -- 172.21.15.1:0/2828402274 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a04017d40 con 0x55aa02d17b80 2024-04-26T01:58:29.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.064+0000 7f6a00c00700 1 -- 172.21.15.1:0/2828402274 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a04022bb0 con 0x55aa02d17b80 2024-04-26T01:58:29.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.065+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/2828402274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d17b80 msgr2=0x55aa02d17f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:29.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.065+0000 7f6a1caa0b80 1 --2- 172.21.15.1:0/2828402274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d17b80 0x55aa02d17f60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6a04011790 tx=0x7f6a04011aa0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.065+0000 7f6a0be00700 1 --2- 172.21.15.1:0/1410673810 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f69f00a3e50 0x7f69f00a6310 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6a04020000 tx=0x7f6a04021000 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:29.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.065+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/2828402274 shutdown_connections 2024-04-26T01:58:29.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.065+0000 7f6a1caa0b80 1 --2- 172.21.15.1:0/2828402274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d23350 0x55aa02d25740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.065+0000 7f6a1caa0b80 1 --2- 172.21.15.1:0/2828402274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d17b80 0x55aa02d17f60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.065+0000 7f6a1caa0b80 1 --2- 172.21.15.1:0/2828402274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d184a0 0x55aa02d22e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.065+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/2828402274 >> 172.21.15.1:0/2828402274 conn(0x55aa02c46730 msgr2=0x55aa02d13fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:29.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.066+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/2828402274 shutdown_connections 2024-04-26T01:58:29.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.067+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/2828402274 wait complete. 2024-04-26T01:58:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.067+0000 7f6a1caa0b80 1 -- start start 2024-04-26T01:58:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.068+0000 7f6a1caa0b80 1 Processor -- start 2024-04-26T01:58:29.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.068+0000 7f6a1caa0b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d17b80 0x55aa02d22a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:29.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.068+0000 7f6a1caa0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d184a0 0x55aa02d1fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:29.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.068+0000 7f6a1caa0b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d23350 0x55aa02d20270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:29.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.068+0000 7f6a1caa0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55aa02d25ed0 con 0x55aa02d184a0 2024-04-26T01:58:29.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.068+0000 7f6a1caa0b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55aa02c249f0 con 0x55aa02d17b80 2024-04-26T01:58:29.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.068+0000 7f6a1caa0b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55aa02d26050 con 0x55aa02d23350 2024-04-26T01:58:29.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.068+0000 7f6a1caa0b80 10 client.0 _ll_register_callbacks cb 0x55aa02c038b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:29.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.069+0000 7f6a0b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d17b80 0x55aa02d22a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:29.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.069+0000 7f6a0b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d17b80 0x55aa02d22a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:59310/0 (socket says 172.21.15.1:59310) 2024-04-26T01:58:29.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.069+0000 7f6a0be00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d23350 0x55aa02d20270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:29.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.069+0000 7f6a0b400700 1 -- 172.21.15.1:0/448512316 learned_addr learned my addr 172.21.15.1:0/448512316 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:29.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.069+0000 7f6a0aa00700 1 --2- 172.21.15.1:0/448512316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d184a0 0x55aa02d1fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:29.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.069+0000 7f6a0b400700 1 -- 172.21.15.1:0/448512316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d23350 msgr2=0x55aa02d20270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:29.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.069+0000 7f6a0b400700 1 --2- 172.21.15.1:0/448512316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d23350 0x55aa02d20270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.069+0000 7f6a0b400700 1 -- 172.21.15.1:0/448512316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d184a0 msgr2=0x55aa02d1fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:29.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.069+0000 7f6a0b400700 1 --2- 172.21.15.1:0/448512316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d184a0 0x55aa02d1fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.069+0000 7f6a0b400700 1 -- 172.21.15.1:0/448512316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa02bb8a60 con 0x55aa02d17b80 2024-04-26T01:58:29.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.070+0000 7f6a0b400700 1 --2- 172.21.15.1:0/448512316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d17b80 0x55aa02d22a30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0a0470 tx=0x7f69fc09ff10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f353a628a4388cef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:29.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.070+0000 7f69f5600700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:58:29.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.070+0000 7f69f5600700 1 -- 172.21.15.1:0/448512316 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc0a0940 con 0x55aa02d17b80 2024-04-26T01:58:29.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.070+0000 7f69f5600700 1 -- 172.21.15.1:0/448512316 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f69fc0a8cc0 con 0x55aa02d17b80 2024-04-26T01:58:29.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.070+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/448512316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a04017d40 con 0x55aa02d17b80 2024-04-26T01:58:29.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.070+0000 7f69f5600700 1 -- 172.21.15.1:0/448512316 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f69fc0a4890 con 0x55aa02d17b80 2024-04-26T01:58:29.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.070+0000 7f6a1caa0b80 10 client.25988 fetch_fsmap learned FSMap version 32 2024-04-26T01:58:29.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.071+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/448512316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55aa02c01c80 con 0x55aa02d17b80 2024-04-26T01:58:29.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.071+0000 7f69f5600700 1 -- 172.21.15.1:0/448512316 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a04017d40 con 0x55aa02d17b80 2024-04-26T01:58:29.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.071+0000 7f69f5600700 1 -- 172.21.15.1:0/448512316 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f69fc0d4050 con 0x55aa02d17b80 2024-04-26T01:58:29.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.071+0000 7f6a1caa0b80 10 client.25988 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:58:29.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.071+0000 7f6a1caa0b80 10 client.25988 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:58:29.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.071+0000 7f6a1caa0b80 20 client.25988 populate_metadata read hostname 'smithi001' 2024-04-26T01:58:29.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.071+0000 7f6a1caa0b80 1 --2- 172.21.15.1:0/448512316 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aa02d3b2d0 0x55aa02d3d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:29.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.071+0000 7f6a1caa0b80 4 client.25988 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:58:29.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.071+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/448512316 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa02c78810 con 0x55aa02d3b2d0 2024-04-26T01:58:29.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.072+0000 7f6a0aa00700 1 --2- 172.21.15.1:0/448512316 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aa02d3b2d0 0x55aa02d3d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:29.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.072+0000 7f6a0aa00700 1 --2- 172.21.15.1:0/448512316 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aa02d3b2d0 0x55aa02d3d6c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22099599be537115 server_cookie=ef0ed4e6313f03d5 in_seq=0 out_seq=0 2024-04-26T01:58:29.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.072+0000 7f69f5600700 10 client.25988 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:29.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.073+0000 7f69f5600700 1 -- 172.21.15.1:0/448512316 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aa02c78810 con 0x55aa02d3b2d0 2024-04-26T01:58:29.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.073+0000 7f69f5600700 10 client.25988 handle_command_reply: tid=0 2024-04-26T01:58:29.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.073+0000 7f6a1caa0b80 1 client.25988 shutdown 2024-04-26T01:58:29.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.073+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/448512316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d17b80 msgr2=0x55aa02d22a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:29.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.073+0000 7f6a1caa0b80 1 --2- 172.21.15.1:0/448512316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d17b80 0x55aa02d22a30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0a0470 tx=0x7f69fc09ff10 comp rx=0 tx=0).stop 2024-04-26T01:58:29.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.074+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/448512316 shutdown_connections 2024-04-26T01:58:29.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.074+0000 7f6a1caa0b80 1 --2- 172.21.15.1:0/448512316 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aa02d3b2d0 0x55aa02d3d6c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.074+0000 7f6a1caa0b80 1 --2- 172.21.15.1:0/448512316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d23350 0x55aa02d20270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.074+0000 7f6a1caa0b80 1 --2- 172.21.15.1:0/448512316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d17b80 0x55aa02d22a30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.074+0000 7f6a1caa0b80 1 --2- 172.21.15.1:0/448512316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d184a0 0x55aa02d1fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.074+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/448512316 >> 172.21.15.1:0/448512316 conn(0x55aa02c46730 msgr2=0x55aa02d19bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.074+0000 7f69f5600700 0 client.25988 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.074+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/448512316 shutdown_connections 2024-04-26T01:58:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.074+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/448512316 wait complete. 2024-04-26T01:58:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.074+0000 7f6a1caa0b80 20 client.25988 trim_cache size 0 max 16384 2024-04-26T01:58:29.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.103+0000 7f6a1caa0b80 1 Processor -- start 2024-04-26T01:58:29.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.103+0000 7f6a1caa0b80 1 -- start start 2024-04-26T01:58:29.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.103+0000 7f6a1caa0b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d17b80 0x55aa02d37620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:29.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.103+0000 7f6a1caa0b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d184a0 0x55aa02d1eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:29.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.104+0000 7f6a1caa0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d23350 0x55aa02d33b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:29.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.104+0000 7f6a1caa0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1ad990 con 0x55aa02d23350 2024-04-26T01:58:29.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.104+0000 7f6a1caa0b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1ab220 con 0x55aa02d17b80 2024-04-26T01:58:29.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.104+0000 7f6a1caa0b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1a9090 con 0x55aa02d184a0 2024-04-26T01:58:29.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.104+0000 7f6a0b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d17b80 0x55aa02d37620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:29.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.104+0000 7f6a0b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d17b80 0x55aa02d37620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:59314/0 (socket says 172.21.15.1:59314) 2024-04-26T01:58:29.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.104+0000 7f6a0aa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d184a0 0x55aa02d1eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:29.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.104+0000 7f6a0b400700 1 -- 172.21.15.1:0/513619382 learned_addr learned my addr 172.21.15.1:0/513619382 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:29.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.104+0000 7f6a0be00700 1 --2- 172.21.15.1:0/513619382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d23350 0x55aa02d33b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:29.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.104+0000 7f6a0b400700 1 -- 172.21.15.1:0/513619382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d184a0 msgr2=0x55aa02d1eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:29.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.104+0000 7f6a0b400700 1 --2- 172.21.15.1:0/513619382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d184a0 0x55aa02d1eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.104+0000 7f6a0b400700 1 -- 172.21.15.1:0/513619382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d23350 msgr2=0x55aa02d33b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:29.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.104+0000 7f6a0b400700 1 --2- 172.21.15.1:0/513619382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d23350 0x55aa02d33b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.104+0000 7f6a0b400700 1 -- 172.21.15.1:0/513619382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa02d1e2a0 con 0x55aa02d17b80 2024-04-26T01:58:29.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.104+0000 7f6a0be00700 1 --2- 172.21.15.1:0/513619382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d23350 0x55aa02d33b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:58:29.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.105+0000 7f6a0b400700 1 --2- 172.21.15.1:0/513619382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d17b80 0x55aa02d37620 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0a01e0 tx=0x7f69fc0a0660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b738541749531e6c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:29.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.105+0000 7f69f7e00700 1 -- 172.21.15.1:0/513619382 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc09f930 con 0x55aa02d17b80 2024-04-26T01:58:29.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.105+0000 7f69f7e00700 1 -- 172.21.15.1:0/513619382 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69fc09fad0 con 0x55aa02d17b80 2024-04-26T01:58:29.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.105+0000 7f69f7e00700 1 -- 172.21.15.1:0/513619382 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc0a09a0 con 0x55aa02d17b80 2024-04-26T01:58:29.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.106+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/513619382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d17b80 msgr2=0x55aa02d37620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:29.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.106+0000 7f6a1caa0b80 1 --2- 172.21.15.1:0/513619382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d17b80 0x55aa02d37620 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0a01e0 tx=0x7f69fc0a0660 comp rx=0 tx=0).stop 2024-04-26T01:58:29.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.106+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/513619382 shutdown_connections 2024-04-26T01:58:29.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.106+0000 7f6a1caa0b80 1 --2- 172.21.15.1:0/513619382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d23350 0x55aa02d33b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.106+0000 7f6a1caa0b80 1 --2- 172.21.15.1:0/513619382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d17b80 0x55aa02d37620 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.106+0000 7f6a1caa0b80 1 --2- 172.21.15.1:0/513619382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d184a0 0x55aa02d1eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.106+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/513619382 >> 172.21.15.1:0/513619382 conn(0x55aa02c46730 msgr2=0x55aa02d21560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:29.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.106+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/513619382 shutdown_connections 2024-04-26T01:58:29.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.106+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/513619382 wait complete. 2024-04-26T01:58:29.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.107+0000 7f6a1caa0b80 1 -- start start 2024-04-26T01:58:29.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.107+0000 7f6a1caa0b80 1 Processor -- start 2024-04-26T01:58:29.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.108+0000 7f6a1caa0b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d17b80 0x55aa02d32cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:29.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.108+0000 7f6a1caa0b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d184a0 0x55aa02d33210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:29.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.108+0000 7f6a1caa0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d23350 0x55aa02da1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.108+0000 7f6a1caa0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1a96e0 con 0x55aa02d23350 2024-04-26T01:58:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.108+0000 7f6a1caa0b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1aab50 con 0x55aa02d17b80 2024-04-26T01:58:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.108+0000 7f6a1caa0b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1a9db0 con 0x55aa02d184a0 2024-04-26T01:58:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.108+0000 7f6a1caa0b80 10 client.0 _ll_register_callbacks cb 0x55aa02c038b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.108+0000 7f6a0aa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d184a0 0x55aa02d33210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.108+0000 7f6a0aa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d184a0 0x55aa02d33210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:60156/0 (socket says 172.21.15.1:60156) 2024-04-26T01:58:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.108+0000 7f6a0aa00700 1 -- 172.21.15.1:0/3863010982 learned_addr learned my addr 172.21.15.1:0/3863010982 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.108+0000 7f6a0b400700 1 --2- 172.21.15.1:0/3863010982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d17b80 0x55aa02d32cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.109+0000 7f6a0be00700 1 --2- 172.21.15.1:0/3863010982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d23350 0x55aa02da1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.109+0000 7f6a0aa00700 1 -- 172.21.15.1:0/3863010982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d17b80 msgr2=0x55aa02d32cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:29.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.109+0000 7f6a0aa00700 1 --2- 172.21.15.1:0/3863010982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d17b80 0x55aa02d32cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.109+0000 7f6a0aa00700 1 -- 172.21.15.1:0/3863010982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d23350 msgr2=0x55aa02da1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:29.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.109+0000 7f6a0aa00700 1 --2- 172.21.15.1:0/3863010982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d23350 0x55aa02da1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:29.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.109+0000 7f6a0aa00700 1 -- 172.21.15.1:0/3863010982 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa02da1d60 con 0x55aa02d184a0 2024-04-26T01:58:29.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.109+0000 7f6a0be00700 1 --2- 172.21.15.1:0/3863010982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d23350 0x55aa02da1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:58:29.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.109+0000 7f6a0aa00700 1 --2- 172.21.15.1:0/3863010982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d184a0 0x55aa02d33210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c06aa00 tx=0x7f69f8005160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eac0202fac055f29 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:29.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.109+0000 7f69f5600700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:58:29.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.110+0000 7f69f5600700 1 -- 172.21.15.1:0/3863010982 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f802e060 con 0x55aa02d184a0 2024-04-26T01:58:29.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.110+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/3863010982 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69fc09fad0 con 0x55aa02d184a0 2024-04-26T01:58:29.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.110+0000 7f69f5600700 1 -- 172.21.15.1:0/3863010982 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f69f80037c0 con 0x55aa02d184a0 2024-04-26T01:58:29.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.110+0000 7f69f5600700 1 -- 172.21.15.1:0/3863010982 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f69f8003a00 con 0x55aa02d184a0 2024-04-26T01:58:29.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.110+0000 7f6a1caa0b80 10 client.26011 fetch_fsmap learned FSMap version 32 2024-04-26T01:58:29.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.110+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/3863010982 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55aa02da3070 con 0x55aa02d184a0 2024-04-26T01:58:29.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.110+0000 7f69f5600700 1 -- 172.21.15.1:0/3863010982 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69fc09fad0 con 0x55aa02d184a0 2024-04-26T01:58:29.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.111+0000 7f69f5600700 1 -- 172.21.15.1:0/3863010982 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f69f80281d0 con 0x55aa02d184a0 2024-04-26T01:58:29.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.111+0000 7f6a1caa0b80 10 client.26011 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:58:29.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.111+0000 7f6a1caa0b80 10 client.26011 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:58:29.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.111+0000 7f6a1caa0b80 20 client.26011 populate_metadata read hostname 'smithi001' 2024-04-26T01:58:29.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.111+0000 7f6a1caa0b80 1 --2- 172.21.15.1:0/3863010982 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aa02da37d0 0x55aa02da5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:29.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.111+0000 7f6a1caa0b80 4 client.26011 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 75.9, "await_for": 42.0}] 2024-04-26T01:58:29.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.111+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/3863010982 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 75.9, "await_for": 42.0}) -- 0x55aa02c728b0 con 0x55aa02da37d0 2024-04-26T01:58:29.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.111+0000 7f6a0b400700 1 --2- 172.21.15.1:0/3863010982 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aa02da37d0 0x55aa02da5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:29.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.112+0000 7f6a0b400700 1 --2- 172.21.15.1:0/3863010982 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aa02da37d0 0x55aa02da5bf0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a15b6e52ede4ce0c server_cookie=6776db083a721aaf in_seq=0 out_seq=0 2024-04-26T01:58:29.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:29.112+0000 7f69f5600700 10 client.26011 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:30.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.030+0000 7f69f5600700 1 -- 172.21.15.1:0/3863010982 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55aa02c728b0 con 0x55aa02da37d0 2024-04-26T01:58:30.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.030+0000 7f69f5600700 10 client.26011 handle_command_reply: tid=0 2024-04-26T01:58:30.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.030+0000 7f6a1caa0b80 1 client.26011 shutdown 2024-04-26T01:58:30.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.031+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/3863010982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d184a0 msgr2=0x55aa02d33210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:30.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.031+0000 7f6a1caa0b80 1 --2- 172.21.15.1:0/3863010982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d184a0 0x55aa02d33210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c06aa00 tx=0x7f69f8005160 comp rx=0 tx=0).stop 2024-04-26T01:58:30.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.031+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/3863010982 shutdown_connections 2024-04-26T01:58:30.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.031+0000 7f6a1caa0b80 1 --2- 172.21.15.1:0/3863010982 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aa02da37d0 0x55aa02da5bf0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:30.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.031+0000 7f6a1caa0b80 1 --2- 172.21.15.1:0/3863010982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aa02d23350 0x55aa02da1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:30.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.031+0000 7f6a1caa0b80 1 --2- 172.21.15.1:0/3863010982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aa02d17b80 0x55aa02d32cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:30.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.031+0000 7f6a1caa0b80 1 --2- 172.21.15.1:0/3863010982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aa02d184a0 0x55aa02d33210 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:30.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.031+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/3863010982 >> 172.21.15.1:0/3863010982 conn(0x55aa02c46730 msgr2=0x55aa02d21560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:30.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.031+0000 7f69f5600700 0 client.26011 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:30.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.032+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/3863010982 shutdown_connections 2024-04-26T01:58:30.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.032+0000 7f6a1caa0b80 1 -- 172.21.15.1:0/3863010982 wait complete. 2024-04-26T01:58:30.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.032+0000 7f6a1caa0b80 20 client.26011 trim_cache size 0 max 16384 2024-04-26T01:58:30.032 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T01:58:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 32, 2024-04-26T01:58:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24518, 2024-04-26T01:58:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 18, 2024-04-26T01:58:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-04-26T01:58:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: "2f800756": { 2024-04-26T01:58:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 18, 2024-04-26T01:58:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-04-26T01:58:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:58:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-04-26T01:58:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:58:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T01:58:30.032 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-04-26T01:58:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 75.9, 2024-04-26T01:58:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-04-26T01:58:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-04-26T01:58:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-04-26T01:58:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:58:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-04-26T01:58:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:58:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:58:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:58:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:58:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:58:30.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:58:30.033 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T01:58:30.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.033+0000 7f6a10c00700 1 -- 172.21.15.1:0/1410673810 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f69f00a3e50 msgr2=0x7f69f00a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:30.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.033+0000 7f6a10c00700 1 --2- 172.21.15.1:0/1410673810 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f69f00a3e50 0x7f69f00a6310 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6a04020000 tx=0x7f6a04021000 comp rx=0 tx=0).stop 2024-04-26T01:58:30.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.033+0000 7f6a10c00700 1 -- 172.21.15.1:0/1410673810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a0c103a60 msgr2=0x7f6a0c198ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:30.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.033+0000 7f6a10c00700 1 --2- 172.21.15.1:0/1410673810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a0c103a60 0x7f6a0c198ff0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f69fc012d80 tx=0x7f69fc00c5b0 comp rx=0 tx=0).stop 2024-04-26T01:58:30.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.034+0000 7f6a10c00700 1 -- 172.21.15.1:0/1410673810 shutdown_connections 2024-04-26T01:58:30.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.034+0000 7f6a10c00700 1 --2- 172.21.15.1:0/1410673810 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f69f00a3e50 0x7f69f00a6310 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:30.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.034+0000 7f6a10c00700 1 --2- 172.21.15.1:0/1410673810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a0c108ff0 0x7f6a0c19d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:30.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.034+0000 7f6a10c00700 1 --2- 172.21.15.1:0/1410673810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a0c103a60 0x7f6a0c198ff0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:30.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.034+0000 7f6a10c00700 1 --2- 172.21.15.1:0/1410673810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0c104410 0x7f6a0c199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:30.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.034+0000 7f6a10c00700 1 -- 172.21.15.1:0/1410673810 >> 172.21.15.1:0/1410673810 conn(0x7f6a0c0fd290 msgr2=0x7f6a0c0fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:30.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.034+0000 7f6a10c00700 1 -- 172.21.15.1:0/1410673810 shutdown_connections 2024-04-26T01:58:30.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:30.034+0000 7f6a10c00700 1 -- 172.21.15.1:0/1410673810 wait complete. 2024-04-26T01:58:30.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:30 smithi001 ceph-mon[27822]: pgmap v211: 97 pgs: 97 active+clean; 51 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.8 MiB/s wr, 286 op/s 2024-04-26T01:58:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:30 smithi097 ceph-mon[32853]: pgmap v211: 97 pgs: 97 active+clean; 51 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.8 MiB/s wr, 286 op/s 2024-04-26T01:58:30.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:30 smithi173 ceph-mon[32553]: pgmap v211: 97 pgs: 97 active+clean; 51 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.8 MiB/s wr, 286 op/s 2024-04-26T01:58:30.870 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2f800756', quiesce took 9.0 seconds. Will release after: 15.9 2024-04-26T01:58:32.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:32 smithi097 ceph-mon[32853]: pgmap v212: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.5 MiB/s wr, 422 op/s 2024-04-26T01:58:32.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:32 smithi173 ceph-mon[32553]: pgmap v212: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.5 MiB/s wr, 422 op/s 2024-04-26T01:58:32.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:32 smithi001 ceph-mon[27822]: pgmap v212: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.5 MiB/s wr, 422 op/s 2024-04-26T01:58:34.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:34 smithi097 ceph-mon[32853]: pgmap v213: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.7 MiB/s wr, 288 op/s 2024-04-26T01:58:34.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:34 smithi173 ceph-mon[32553]: pgmap v213: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.7 MiB/s wr, 288 op/s 2024-04-26T01:58:34.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:34 smithi001 ceph-mon[27822]: pgmap v213: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.7 MiB/s wr, 288 op/s 2024-04-26T01:58:36.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:36 smithi097 ceph-mon[32853]: pgmap v214: 97 pgs: 97 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.7 MiB/s wr, 292 op/s 2024-04-26T01:58:36.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:36 smithi173 ceph-mon[32553]: pgmap v214: 97 pgs: 97 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.7 MiB/s wr, 292 op/s 2024-04-26T01:58:36.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:36 smithi001 ceph-mon[27822]: pgmap v214: 97 pgs: 97 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.7 MiB/s wr, 292 op/s 2024-04-26T01:58:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.509+0000 7f6ec2400640 1 -- 172.21.15.1:0/3095452994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ebc112c30 msgr2=0x7f6ebc115020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.509+0000 7f6ec2400640 1 --2- 172.21.15.1:0/3095452994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ebc112c30 0x7f6ebc115020 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009c10 tx=0x7f6eb402a6f0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.509+0000 7f6ec2400640 1 -- 172.21.15.1:0/3095452994 shutdown_connections 2024-04-26T01:58:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.509+0000 7f6ec2400640 1 --2- 172.21.15.1:0/3095452994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ebc112c30 0x7f6ebc115020 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.509+0000 7f6ec2400640 1 --2- 172.21.15.1:0/3095452994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ebc076da0 0x7f6ebc1126f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.509+0000 7f6ec2400640 1 --2- 172.21.15.1:0/3095452994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ebc076480 0x7f6ebc076860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.509+0000 7f6ec2400640 1 -- 172.21.15.1:0/3095452994 >> 172.21.15.1:0/3095452994 conn(0x7f6ebc0ffa60 msgr2=0x7f6ebc101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.510+0000 7f6ec2400640 1 -- 172.21.15.1:0/3095452994 shutdown_connections 2024-04-26T01:58:37.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.510+0000 7f6ec2400640 1 -- 172.21.15.1:0/3095452994 wait complete. 2024-04-26T01:58:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.511+0000 7f6ec2400640 1 Processor -- start 2024-04-26T01:58:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.511+0000 7f6ec2400640 1 -- start start 2024-04-26T01:58:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.511+0000 7f6ec2400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ebc076480 0x7f6ebc19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.511+0000 7f6ec2400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ebc076da0 0x7f6ebc19f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.511+0000 7f6ec2400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ebc112c30 0x7f6ebc1a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:37.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.511+0000 7f6ec2400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc118620 con 0x7f6ebc076da0 2024-04-26T01:58:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.511+0000 7f6ec2400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1184a0 con 0x7f6ebc076480 2024-04-26T01:58:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.511+0000 7f6ec2400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1187a0 con 0x7f6ebc112c30 2024-04-26T01:58:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.511+0000 7f6ebbe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ebc076da0 0x7f6ebc19f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.511+0000 7f6ebbe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ebc076da0 0x7f6ebc19f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45664/0 (socket says 172.21.15.1:45664) 2024-04-26T01:58:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.511+0000 7f6ebbe00640 1 -- 172.21.15.1:0/3056378979 learned_addr learned my addr 172.21.15.1:0/3056378979 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.511+0000 7f6ec1000640 1 --2- 172.21.15.1:0/3056378979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ebc076480 0x7f6ebc19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.512+0000 7f6ec1a00640 1 --2- 172.21.15.1:0/3056378979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ebc112c30 0x7f6ebc1a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.512+0000 7f6ec1000640 1 -- 172.21.15.1:0/3056378979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ebc112c30 msgr2=0x7f6ebc1a3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.512+0000 7f6ec1000640 1 --2- 172.21.15.1:0/3056378979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ebc112c30 0x7f6ebc1a3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.512+0000 7f6ec1000640 1 -- 172.21.15.1:0/3056378979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ebc076da0 msgr2=0x7f6ebc19f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.512+0000 7f6ec1000640 1 --2- 172.21.15.1:0/3056378979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ebc076da0 0x7f6ebc19f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.512+0000 7f6ec1000640 1 -- 172.21.15.1:0/3056378979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ebc1a3e70 con 0x7f6ebc076480 2024-04-26T01:58:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.512+0000 7f6ec1000640 1 --2- 172.21.15.1:0/3056378979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ebc076480 0x7f6ebc19ee80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0005fd0 tx=0x7f6eb000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.513+0000 7f6eb9600640 1 -- 172.21.15.1:0/3056378979 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb000cd00 con 0x7f6ebc076480 2024-04-26T01:58:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.513+0000 7f6ec2400640 1 -- 172.21.15.1:0/3056378979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ebc1a4160 con 0x7f6ebc076480 2024-04-26T01:58:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.513+0000 7f6eb9600640 1 -- 172.21.15.1:0/3056378979 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb00035f0 con 0x7f6ebc076480 2024-04-26T01:58:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.513+0000 7f6eb9600640 1 -- 172.21.15.1:0/3056378979 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0010690 con 0x7f6ebc076480 2024-04-26T01:58:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.513+0000 7f6ec2400640 1 -- 172.21.15.1:0/3056378979 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ebc1ab9f0 con 0x7f6ebc076480 2024-04-26T01:58:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.516+0000 7f6ec680d740 1 Processor -- start 2024-04-26T01:58:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.516+0000 7f6ec680d740 1 -- start start 2024-04-26T01:58:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.516+0000 7f6eb9600640 1 -- 172.21.15.1:0/3056378979 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f6eb0003a80 con 0x7f6ebc076480 2024-04-26T01:58:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.516+0000 7f6ec680d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a569b0 0x55e351a56d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:37.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.516+0000 7f6ec680d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a572d0 0x55e351a61650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.516+0000 7f6ec680d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e351a61b90 0x55e351a63f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.516+0000 7f6ec680d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e3519bc900 con 0x55e351a569b0 2024-04-26T01:58:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.517+0000 7f6ec680d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e351962ef0 con 0x55e351a61b90 2024-04-26T01:58:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.517+0000 7f6ec680d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e3519b88c0 con 0x55e351a572d0 2024-04-26T01:58:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.517+0000 7f6ec1a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a569b0 0x55e351a56d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.517+0000 7f6ec1a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a569b0 0x55e351a56d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45674/0 (socket says 172.21.15.1:45674) 2024-04-26T01:58:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.517+0000 7f6ec1a00640 1 -- 172.21.15.1:0/2170316562 learned_addr learned my addr 172.21.15.1:0/2170316562 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.517+0000 7f6ec1000640 1 --2- 172.21.15.1:0/2170316562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a572d0 0x55e351a61650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.517+0000 7f6ebbe00640 1 --2- 172.21.15.1:0/2170316562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e351a61b90 0x55e351a63f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.517+0000 7f6ec1a00640 1 -- 172.21.15.1:0/2170316562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a572d0 msgr2=0x55e351a61650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.517+0000 7f6ec1a00640 1 --2- 172.21.15.1:0/2170316562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a572d0 0x55e351a61650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.517+0000 7f6ec1a00640 1 -- 172.21.15.1:0/2170316562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e351a61b90 msgr2=0x55e351a63f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.517+0000 7f6ec1a00640 1 --2- 172.21.15.1:0/2170316562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e351a61b90 0x55e351a63f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.517+0000 7f6ec1a00640 1 -- 172.21.15.1:0/2170316562 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3516cf210 con 0x55e351a569b0 2024-04-26T01:58:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.517+0000 7f6ebbe00640 1 --2- 172.21.15.1:0/2170316562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e351a61b90 0x55e351a63f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.518+0000 7f6ec1a00640 1 --2- 172.21.15.1:0/2170316562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a569b0 0x55e351a56d90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4004af0 tx=0x7f6eb4030430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef58526961de0df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.518+0000 7f6eb9600640 1 --2- 172.21.15.1:0/3056378979 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6ea40a3d80 0x7f6ea40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.518+0000 7f6eae000640 1 -- 172.21.15.1:0/2170316562 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4041030 con 0x55e351a569b0 2024-04-26T01:58:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.518+0000 7f6eae000640 1 -- 172.21.15.1:0/2170316562 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb4003e20 con 0x55e351a569b0 2024-04-26T01:58:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.518+0000 7f6eae000640 1 -- 172.21.15.1:0/2170316562 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4036040 con 0x55e351a569b0 2024-04-26T01:58:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.518+0000 7f6eb9600640 1 -- 172.21.15.1:0/3056378979 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f6eb00d09e0 con 0x7f6ebc076480 2024-04-26T01:58:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.518+0000 7f6ec1000640 1 --2- 172.21.15.1:0/3056378979 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6ea40a3d80 0x7f6ea40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.519+0000 7f6ec680d740 1 -- 172.21.15.1:0/2170316562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a569b0 msgr2=0x55e351a56d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.519+0000 7f6ec680d740 1 --2- 172.21.15.1:0/2170316562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a569b0 0x55e351a56d90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4004af0 tx=0x7f6eb4030430 comp rx=0 tx=0).stop 2024-04-26T01:58:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.519+0000 7f6ec1000640 1 --2- 172.21.15.1:0/3056378979 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6ea40a3d80 0x7f6ea40a6240 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55e351a57f50 tx=0x7f6eb00d6530 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.519+0000 7f6ec680d740 1 -- 172.21.15.1:0/2170316562 shutdown_connections 2024-04-26T01:58:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.519+0000 7f6ec680d740 1 --2- 172.21.15.1:0/2170316562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e351a61b90 0x55e351a63f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.519+0000 7f6ec680d740 1 --2- 172.21.15.1:0/2170316562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a572d0 0x55e351a61650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.519+0000 7f6ec680d740 1 --2- 172.21.15.1:0/2170316562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a569b0 0x55e351a56d90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.519+0000 7f6ec680d740 1 -- 172.21.15.1:0/2170316562 >> 172.21.15.1:0/2170316562 conn(0x55e351a52ec0 msgr2=0x55e351a532a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.519+0000 7f6ec680d740 1 -- 172.21.15.1:0/2170316562 shutdown_connections 2024-04-26T01:58:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.519+0000 7f6ec680d740 1 -- 172.21.15.1:0/2170316562 wait complete. 2024-04-26T01:58:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.520+0000 7f6ec680d740 1 -- start start 2024-04-26T01:58:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.520+0000 7f6ec680d740 1 Processor -- start 2024-04-26T01:58:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.520+0000 7f6ec680d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e351a569b0 0x55e351a5eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.520+0000 7f6ec680d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a572d0 0x55e351a5f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.520+0000 7f6ec680d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a61b90 0x55e351a5f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.520+0000 7f6ec680d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e351a64760 con 0x55e351a61b90 2024-04-26T01:58:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.520+0000 7f6ec680d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e351976310 con 0x55e351a569b0 2024-04-26T01:58:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.521+0000 7f6ec680d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e351a648e0 con 0x55e351a572d0 2024-04-26T01:58:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.521+0000 7f6ec680d740 10 client.0 _ll_register_callbacks cb 0x55e351944cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.521+0000 7f6ec1a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a61b90 0x55e351a5f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.521+0000 7f6ebbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a572d0 0x55e351a5f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.521+0000 7f6ec1a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a61b90 0x55e351a5f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45682/0 (socket says 172.21.15.1:45682) 2024-04-26T01:58:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.521+0000 7f6ec1a00640 1 -- 172.21.15.1:0/3926482387 learned_addr learned my addr 172.21.15.1:0/3926482387 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.521+0000 7f6ec1a00640 1 --2- 172.21.15.1:0/3926482387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e351a569b0 0x55e351a5eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.521+0000 7f6ebbe00640 1 -- 172.21.15.1:0/3926482387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e351a569b0 msgr2=0x55e351a5eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.521+0000 7f6ebbe00640 1 --2- 172.21.15.1:0/3926482387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e351a569b0 0x55e351a5eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.521+0000 7f6ebbe00640 1 -- 172.21.15.1:0/3926482387 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a61b90 msgr2=0x55e351a5f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.521+0000 7f6ebbe00640 1 --2- 172.21.15.1:0/3926482387 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a61b90 0x55e351a5f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.521+0000 7f6ebbe00640 1 -- 172.21.15.1:0/3926482387 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e35179b620 con 0x55e351a572d0 2024-04-26T01:58:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.522+0000 7f6ebbe00640 1 --2- 172.21.15.1:0/3926482387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a572d0 0x55e351a5f020 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8009640 tx=0x7f6ea8012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3cf8ef9902bb5d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.522+0000 7f6ea2a00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:58:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.522+0000 7f6ea2a00640 1 -- 172.21.15.1:0/3926482387 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea800f8f0 con 0x55e351a572d0 2024-04-26T01:58:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.522+0000 7f6ea2a00640 1 -- 172.21.15.1:0/3926482387 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f6ea800fa90 con 0x55e351a572d0 2024-04-26T01:58:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.522+0000 7f6ec680d740 1 -- 172.21.15.1:0/3926482387 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eb4003e20 con 0x55e351a572d0 2024-04-26T01:58:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.522+0000 7f6ea2a00640 1 -- 172.21.15.1:0/3926482387 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6ea800fc90 con 0x55e351a572d0 2024-04-26T01:58:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.522+0000 7f6ec680d740 10 client.26023 fetch_fsmap learned FSMap version 32 2024-04-26T01:58:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.522+0000 7f6ec680d740 1 -- 172.21.15.1:0/3926482387 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e351a70a60 con 0x55e351a572d0 2024-04-26T01:58:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.523+0000 7f6ea2a00640 1 -- 172.21.15.1:0/3926482387 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb4003e20 con 0x55e351a572d0 2024-04-26T01:58:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.523+0000 7f6ea2a00640 1 -- 172.21.15.1:0/3926482387 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f6ea8023070 con 0x55e351a572d0 2024-04-26T01:58:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.523+0000 7f6ec680d740 10 client.26023 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:58:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.523+0000 7f6ec680d740 10 client.26023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:58:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.523+0000 7f6ec680d740 20 client.26023 populate_metadata read hostname 'smithi001' 2024-04-26T01:58:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.523+0000 7f6ec680d740 1 --2- 172.21.15.1:0/3926482387 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e351a71f20 0x55e351a74330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.523+0000 7f6ec680d740 4 client.26023 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:58:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.523+0000 7f6ec680d740 1 -- 172.21.15.1:0/3926482387 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3519d08b0 con 0x55e351a71f20 2024-04-26T01:58:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.523+0000 7f6ec1a00640 1 --2- 172.21.15.1:0/3926482387 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e351a71f20 0x55e351a74330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.524+0000 7f6ec1a00640 1 --2- 172.21.15.1:0/3926482387 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e351a71f20 0x55e351a74330 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8071dde765adb7c server_cookie=a199d5ce6ee2ea7f in_seq=0 out_seq=0 2024-04-26T01:58:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.524+0000 7f6ea2a00640 10 client.26023 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.525+0000 7f6ea2a00640 1 -- 172.21.15.1:0/3926482387 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e3519d08b0 con 0x55e351a71f20 2024-04-26T01:58:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.525+0000 7f6ea2a00640 10 client.26023 handle_command_reply: tid=0 2024-04-26T01:58:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.525+0000 7f6ec680d740 1 client.26023 shutdown 2024-04-26T01:58:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.525+0000 7f6ec680d740 1 -- 172.21.15.1:0/3926482387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a572d0 msgr2=0x55e351a5f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.525+0000 7f6ec680d740 1 --2- 172.21.15.1:0/3926482387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a572d0 0x55e351a5f020 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8009640 tx=0x7f6ea8012b60 comp rx=0 tx=0).stop 2024-04-26T01:58:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.525+0000 7f6ec680d740 1 -- 172.21.15.1:0/3926482387 shutdown_connections 2024-04-26T01:58:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.525+0000 7f6ec680d740 1 --2- 172.21.15.1:0/3926482387 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e351a71f20 0x55e351a74330 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.525+0000 7f6ec680d740 1 --2- 172.21.15.1:0/3926482387 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a61b90 0x55e351a5f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.525+0000 7f6ec680d740 1 --2- 172.21.15.1:0/3926482387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a572d0 0x55e351a5f020 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.525+0000 7f6ec680d740 1 --2- 172.21.15.1:0/3926482387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e351a569b0 0x55e351a5eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.525+0000 7f6ea2a00640 0 client.26023 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.525+0000 7f6ec680d740 1 -- 172.21.15.1:0/3926482387 >> 172.21.15.1:0/3926482387 conn(0x55e351a52ec0 msgr2=0x55e351a58bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.525+0000 7f6ec680d740 1 -- 172.21.15.1:0/3926482387 shutdown_connections 2024-04-26T01:58:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.525+0000 7f6ec680d740 1 -- 172.21.15.1:0/3926482387 wait complete. 2024-04-26T01:58:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.526+0000 7f6ec680d740 20 client.26023 trim_cache size 0 max 16384 2024-04-26T01:58:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.542+0000 7f6ec680d740 1 Processor -- start 2024-04-26T01:58:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.542+0000 7f6ec680d740 1 -- start start 2024-04-26T01:58:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.542+0000 7f6ec680d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a569b0 0x55e351a6d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.542+0000 7f6ec680d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a572d0 0x55e351a79c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.542+0000 7f6ec680d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e351a61b90 0x55e351a7a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.542+0000 7f6ec680d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1b6950 con 0x55e351a569b0 2024-04-26T01:58:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.542+0000 7f6ec680d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1b4220 con 0x55e351a61b90 2024-04-26T01:58:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.542+0000 7f6ec680d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1b2090 con 0x55e351a572d0 2024-04-26T01:58:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.543+0000 7f6ec1a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a569b0 0x55e351a6d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.543+0000 7f6ec1a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a569b0 0x55e351a6d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45694/0 (socket says 172.21.15.1:45694) 2024-04-26T01:58:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.543+0000 7f6ec1a00640 1 -- 172.21.15.1:0/1788898745 learned_addr learned my addr 172.21.15.1:0/1788898745 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.543+0000 7f6ebbe00640 1 --2- 172.21.15.1:0/1788898745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a572d0 0x55e351a79c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.543+0000 7f6ec1a00640 1 --2- 172.21.15.1:0/1788898745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e351a61b90 0x55e351a7a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.543+0000 7f6ebbe00640 1 -- 172.21.15.1:0/1788898745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e351a61b90 msgr2=0x55e351a7a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.543+0000 7f6ebbe00640 1 --2- 172.21.15.1:0/1788898745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e351a61b90 0x55e351a7a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.543+0000 7f6ebbe00640 1 -- 172.21.15.1:0/1788898745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a569b0 msgr2=0x55e351a6d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.543+0000 7f6ebbe00640 1 --2- 172.21.15.1:0/1788898745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a569b0 0x55e351a6d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.543+0000 7f6ebbe00640 1 -- 172.21.15.1:0/1788898745 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e351a570c0 con 0x55e351a572d0 2024-04-26T01:58:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.543+0000 7f6ec1a00640 1 --2- 172.21.15.1:0/1788898745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a569b0 0x55e351a6d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.543+0000 7f6ebbe00640 1 --2- 172.21.15.1:0/1788898745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a572d0 0x55e351a79c40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6ea801dfd0 tx=0x7f6ea8009fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1bac20671482ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.544+0000 7f6ead600640 1 -- 172.21.15.1:0/1788898745 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea8029030 con 0x55e351a572d0 2024-04-26T01:58:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.544+0000 7f6ead600640 1 -- 172.21.15.1:0/1788898745 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ea800f8f0 con 0x55e351a572d0 2024-04-26T01:58:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.544+0000 7f6ead600640 1 -- 172.21.15.1:0/1788898745 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea8005120 con 0x55e351a572d0 2024-04-26T01:58:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.544+0000 7f6ec680d740 1 -- 172.21.15.1:0/1788898745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a572d0 msgr2=0x55e351a79c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.544+0000 7f6ec680d740 1 --2- 172.21.15.1:0/1788898745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a572d0 0x55e351a79c40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6ea801dfd0 tx=0x7f6ea8009fa0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.544+0000 7f6ec680d740 1 -- 172.21.15.1:0/1788898745 shutdown_connections 2024-04-26T01:58:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.544+0000 7f6ec680d740 1 --2- 172.21.15.1:0/1788898745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e351a61b90 0x55e351a7a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.544+0000 7f6ec680d740 1 --2- 172.21.15.1:0/1788898745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a572d0 0x55e351a79c40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.544+0000 7f6ec680d740 1 --2- 172.21.15.1:0/1788898745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a569b0 0x55e351a6d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.544+0000 7f6ec680d740 1 -- 172.21.15.1:0/1788898745 >> 172.21.15.1:0/1788898745 conn(0x55e351a52ec0 msgr2=0x55e351a59d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.544+0000 7f6ec680d740 1 -- 172.21.15.1:0/1788898745 shutdown_connections 2024-04-26T01:58:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.544+0000 7f6ec680d740 1 -- 172.21.15.1:0/1788898745 wait complete. 2024-04-26T01:58:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.545+0000 7f6ec680d740 1 -- start start 2024-04-26T01:58:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.545+0000 7f6ec680d740 1 Processor -- start 2024-04-26T01:58:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.545+0000 7f6ec680d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e351a569b0 0x55e351a69b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.545+0000 7f6ec680d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a572d0 0x55e351a6a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.545+0000 7f6ec680d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a61b90 0x55e351a66fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.545+0000 7f6ec680d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1b26e0 con 0x55e351a572d0 2024-04-26T01:58:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.545+0000 7f6ec680d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1b3b50 con 0x55e351a569b0 2024-04-26T01:58:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.545+0000 7f6ec680d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1b2db0 con 0x55e351a61b90 2024-04-26T01:58:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.545+0000 7f6ec680d740 10 client.0 _ll_register_callbacks cb 0x7f6ea8009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.546+0000 7f6ebbe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a572d0 0x55e351a6a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.546+0000 7f6ebbe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a572d0 0x55e351a6a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45700/0 (socket says 172.21.15.1:45700) 2024-04-26T01:58:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.546+0000 7f6ebbe00640 1 -- 172.21.15.1:0/2268566633 learned_addr learned my addr 172.21.15.1:0/2268566633 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.546+0000 7f6ec1a00640 1 --2- 172.21.15.1:0/2268566633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e351a569b0 0x55e351a69b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.546+0000 7f6ec1a00640 1 --2- 172.21.15.1:0/2268566633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a61b90 0x55e351a66fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.546+0000 7f6ebbe00640 1 -- 172.21.15.1:0/2268566633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a61b90 msgr2=0x55e351a66fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.546+0000 7f6ebbe00640 1 --2- 172.21.15.1:0/2268566633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a61b90 0x55e351a66fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.546+0000 7f6ebbe00640 1 -- 172.21.15.1:0/2268566633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e351a569b0 msgr2=0x55e351a69b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.546+0000 7f6ebbe00640 1 --2- 172.21.15.1:0/2268566633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e351a569b0 0x55e351a69b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.546+0000 7f6ebbe00640 1 -- 172.21.15.1:0/2268566633 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e351a677f0 con 0x55e351a572d0 2024-04-26T01:58:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.546+0000 7f6ebbe00640 1 --2- 172.21.15.1:0/2268566633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a572d0 0x55e351a6a0a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8000c00 tx=0x7f6ea800faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4de82bcc556878 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.546+0000 7f6ea2a00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:58:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.547+0000 7f6ea2a00640 1 -- 172.21.15.1:0/2268566633 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea8033020 con 0x55e351a572d0 2024-04-26T01:58:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.547+0000 7f6ea2a00640 1 -- 172.21.15.1:0/2268566633 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f6ea8010a20 con 0x55e351a572d0 2024-04-26T01:58:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.547+0000 7f6ec680d740 1 -- 172.21.15.1:0/2268566633 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ea800f8f0 con 0x55e351a572d0 2024-04-26T01:58:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.547+0000 7f6ea2a00640 1 -- 172.21.15.1:0/2268566633 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6ea803e050 con 0x55e351a572d0 2024-04-26T01:58:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.547+0000 7f6ec680d740 10 client.15858 fetch_fsmap learned FSMap version 32 2024-04-26T01:58:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.547+0000 7f6ec680d740 1 -- 172.21.15.1:0/2268566633 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e351ad80b0 con 0x55e351a572d0 2024-04-26T01:58:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.547+0000 7f6ea2a00640 1 -- 172.21.15.1:0/2268566633 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ea800f8f0 con 0x55e351a572d0 2024-04-26T01:58:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.548+0000 7f6ea2a00640 1 -- 172.21.15.1:0/2268566633 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f6ea8023040 con 0x55e351a572d0 2024-04-26T01:58:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.548+0000 7f6ec680d740 10 client.15858 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:58:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.548+0000 7f6ec680d740 10 client.15858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:58:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.548+0000 7f6ec680d740 20 client.15858 populate_metadata read hostname 'smithi001' 2024-04-26T01:58:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.548+0000 7f6ec680d740 1 --2- 172.21.15.1:0/2268566633 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e351ad86e0 0x55e351adab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.548+0000 7f6ec680d740 4 client.15858 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:58:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.548+0000 7f6ec680d740 1 -- 172.21.15.1:0/2268566633 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e3519ca320 con 0x55e351ad86e0 2024-04-26T01:58:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.548+0000 7f6ec1a00640 1 --2- 172.21.15.1:0/2268566633 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e351ad86e0 0x55e351adab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.549+0000 7f6ec1a00640 1 --2- 172.21.15.1:0/2268566633 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e351ad86e0 0x55e351adab00 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7bbdc2d9dfcad7d server_cookie=ea489b816093fa63 in_seq=0 out_seq=0 2024-04-26T01:58:37.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.549+0000 7f6ea2a00640 10 client.15858 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.565+0000 7f6ea2a00640 1 -- 172.21.15.1:0/2268566633 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e3519ca320 con 0x55e351ad86e0 2024-04-26T01:58:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.565+0000 7f6ea2a00640 10 client.15858 handle_command_reply: tid=0 2024-04-26T01:58:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.565+0000 7f6ec680d740 1 client.15858 shutdown 2024-04-26T01:58:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.565+0000 7f6ec680d740 1 -- 172.21.15.1:0/2268566633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a572d0 msgr2=0x55e351a6a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.565+0000 7f6ec680d740 1 --2- 172.21.15.1:0/2268566633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a572d0 0x55e351a6a0a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8000c00 tx=0x7f6ea800faa0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.565+0000 7f6ec680d740 1 -- 172.21.15.1:0/2268566633 shutdown_connections 2024-04-26T01:58:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.565+0000 7f6ec680d740 1 --2- 172.21.15.1:0/2268566633 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e351ad86e0 0x55e351adab00 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.565+0000 7f6ec680d740 1 --2- 172.21.15.1:0/2268566633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e351a61b90 0x55e351a66fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.565+0000 7f6ec680d740 1 --2- 172.21.15.1:0/2268566633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e351a572d0 0x55e351a6a0a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.565+0000 7f6ec680d740 1 --2- 172.21.15.1:0/2268566633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e351a569b0 0x55e351a69b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.565+0000 7f6ea2a00640 0 client.15858 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.565+0000 7f6ec680d740 1 -- 172.21.15.1:0/2268566633 >> 172.21.15.1:0/2268566633 conn(0x55e351a52ec0 msgr2=0x55e351a51790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.565+0000 7f6ec680d740 1 -- 172.21.15.1:0/2268566633 shutdown_connections 2024-04-26T01:58:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.565+0000 7f6ec680d740 1 -- 172.21.15.1:0/2268566633 wait complete. 2024-04-26T01:58:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.565+0000 7f6ec680d740 20 client.15858 trim_cache size 0 max 16384 2024-04-26T01:58:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:58:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:58:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:58:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:58:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.566+0000 7f6ec2400640 1 -- 172.21.15.1:0/3056378979 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6ea40a3d80 msgr2=0x7f6ea40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.566+0000 7f6ec2400640 1 --2- 172.21.15.1:0/3056378979 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6ea40a3d80 0x7f6ea40a6240 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55e351a57f50 tx=0x7f6eb00d6530 comp rx=0 tx=0).stop 2024-04-26T01:58:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.566+0000 7f6ec2400640 1 -- 172.21.15.1:0/3056378979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ebc076480 msgr2=0x7f6ebc19ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.566+0000 7f6ec2400640 1 --2- 172.21.15.1:0/3056378979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ebc076480 0x7f6ebc19ee80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0005fd0 tx=0x7f6eb000ee50 comp rx=0 tx=0).stop 2024-04-26T01:58:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.566+0000 7f6ec2400640 1 -- 172.21.15.1:0/3056378979 shutdown_connections 2024-04-26T01:58:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.566+0000 7f6ec2400640 1 --2- 172.21.15.1:0/3056378979 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6ea40a3d80 0x7f6ea40a6240 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.566+0000 7f6ec2400640 1 --2- 172.21.15.1:0/3056378979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ebc112c30 0x7f6ebc1a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.566+0000 7f6ec2400640 1 --2- 172.21.15.1:0/3056378979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ebc076da0 0x7f6ebc19f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.566+0000 7f6ec2400640 1 --2- 172.21.15.1:0/3056378979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ebc076480 0x7f6ebc19ee80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.566+0000 7f6ec2400640 1 -- 172.21.15.1:0/3056378979 >> 172.21.15.1:0/3056378979 conn(0x7f6ebc0ffa60 msgr2=0x7f6ebc101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.566+0000 7f6ec2400640 1 -- 172.21.15.1:0/3056378979 shutdown_connections 2024-04-26T01:58:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:37.566+0000 7f6ec2400640 1 -- 172.21.15.1:0/3056378979 wait complete. 2024-04-26T01:58:38.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:38 smithi097 ceph-mon[32853]: pgmap v215: 97 pgs: 97 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 824 KiB/s wr, 139 op/s 2024-04-26T01:58:38.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:38 smithi173 ceph-mon[32553]: pgmap v215: 97 pgs: 97 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 824 KiB/s wr, 139 op/s 2024-04-26T01:58:38.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:38 smithi001 ceph-mon[27822]: pgmap v215: 97 pgs: 97 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 824 KiB/s wr, 139 op/s 2024-04-26T01:58:40.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:40 smithi173 ceph-mon[32553]: pgmap v216: 97 pgs: 97 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 824 KiB/s wr, 139 op/s 2024-04-26T01:58:40.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:40 smithi001 ceph-mon[27822]: pgmap v216: 97 pgs: 97 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 824 KiB/s wr, 139 op/s 2024-04-26T01:58:40.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:40 smithi097 ceph-mon[32853]: pgmap v216: 97 pgs: 97 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 824 KiB/s wr, 139 op/s 2024-04-26T01:58:42.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:42 smithi173 ceph-mon[32553]: pgmap v217: 97 pgs: 97 active+clean; 49 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 833 KiB/s wr, 145 op/s 2024-04-26T01:58:42.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:42 smithi001 ceph-mon[27822]: pgmap v217: 97 pgs: 97 active+clean; 49 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 833 KiB/s wr, 145 op/s 2024-04-26T01:58:42.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:42 smithi097 ceph-mon[32853]: pgmap v217: 97 pgs: 97 active+clean; 49 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 833 KiB/s wr, 145 op/s 2024-04-26T01:58:44.536 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:44 smithi097 ceph-mon[32853]: pgmap v218: 97 pgs: 97 active+clean; 49 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 36 KiB/s wr, 9 op/s 2024-04-26T01:58:44.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:44 smithi173 ceph-mon[32553]: pgmap v218: 97 pgs: 97 active+clean; 49 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 36 KiB/s wr, 9 op/s 2024-04-26T01:58:44.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:44 smithi001 ceph-mon[27822]: pgmap v218: 97 pgs: 97 active+clean; 49 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 36 KiB/s wr, 9 op/s 2024-04-26T01:58:46.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:46 smithi173 ceph-mon[32553]: pgmap v219: 97 pgs: 97 active+clean; 49 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-04-26T01:58:46.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:46 smithi001 ceph-mon[27822]: pgmap v219: 97 pgs: 97 active+clean; 49 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-04-26T01:58:46.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:46 smithi097 ceph-mon[32853]: pgmap v219: 97 pgs: 97 active+clean; 49 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-04-26T01:58:46.772 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:58:47.256 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:58:48.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:48 smithi001 ceph-mon[27822]: pgmap v220: 97 pgs: 97 active+clean; 49 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.3 KiB/s wr, 8 op/s 2024-04-26T01:58:48.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:48 smithi173 ceph-mon[32553]: pgmap v220: 97 pgs: 97 active+clean; 49 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.3 KiB/s wr, 8 op/s 2024-04-26T01:58:48.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:48 smithi097 ceph-mon[32853]: pgmap v220: 97 pgs: 97 active+clean; 49 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.3 KiB/s wr, 8 op/s 2024-04-26T01:58:49.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:49 smithi001 ceph-mon[27822]: pgmap v221: 97 pgs: 97 active+clean; 49 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.3 KiB/s wr, 8 op/s 2024-04-26T01:58:49.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:49 smithi173 ceph-mon[32553]: pgmap v221: 97 pgs: 97 active+clean; 49 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.3 KiB/s wr, 8 op/s 2024-04-26T01:58:49.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.599+0000 7f9d16800700 1 -- 172.21.15.1:0/3751313349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d101034b0 msgr2=0x7f9d1010f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:49.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.599+0000 7f9d16800700 1 --2- 172.21.15.1:0/3751313349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d101034b0 0x7f9d1010f920 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9d080099c0 tx=0x7f9d08009cd0 comp rx=0 tx=0).stop 2024-04-26T01:58:49.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.599+0000 7f9d16800700 1 -- 172.21.15.1:0/3751313349 shutdown_connections 2024-04-26T01:58:49.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.599+0000 7f9d16800700 1 --2- 172.21.15.1:0/3751313349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d101034b0 0x7f9d1010f920 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:49.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.599+0000 7f9d16800700 1 --2- 172.21.15.1:0/3751313349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d10107940 0x7f9d10107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:49.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.599+0000 7f9d16800700 1 --2- 172.21.15.1:0/3751313349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d10102af0 0x7f9d10102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:49.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.599+0000 7f9d16800700 1 -- 172.21.15.1:0/3751313349 >> 172.21.15.1:0/3751313349 conn(0x7f9d10079ba0 msgr2=0x7f9d10079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:49.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.599+0000 7f9d16800700 1 -- 172.21.15.1:0/3751313349 shutdown_connections 2024-04-26T01:58:49.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.600+0000 7f9d16800700 1 -- 172.21.15.1:0/3751313349 wait complete. 2024-04-26T01:58:49.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.601+0000 7f9d16800700 1 Processor -- start 2024-04-26T01:58:49.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.601+0000 7f9d16800700 1 -- start start 2024-04-26T01:58:49.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.602+0000 7f9d16800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d10102af0 0x7f9d10074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:49.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.602+0000 7f9d16800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d101034b0 0x7f9d10075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:49.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.602+0000 7f9d16800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d10107940 0x7f9d100757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:49.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.602+0000 7f9d16800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10112c50 con 0x7f9d10102af0 2024-04-26T01:58:49.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.602+0000 7f9d16800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10112ad0 con 0x7f9d101034b0 2024-04-26T01:58:49.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.602+0000 7f9d16800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10112dd0 con 0x7f9d10107940 2024-04-26T01:58:49.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.602+0000 7f9d0f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d10102af0 0x7f9d10074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:49.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.602+0000 7f9d0ea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d101034b0 0x7f9d10075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:49.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.602+0000 7f9d0ea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d101034b0 0x7f9d10075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:46074/0 (socket says 172.21.15.1:46074) 2024-04-26T01:58:49.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.602+0000 7f9d0ea00700 1 -- 172.21.15.1:0/985390397 learned_addr learned my addr 172.21.15.1:0/985390397 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:49.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.602+0000 7f9d0fe00700 1 --2- 172.21.15.1:0/985390397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d10107940 0x7f9d100757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:49.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.603+0000 7f9d0ea00700 1 -- 172.21.15.1:0/985390397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d10107940 msgr2=0x7f9d100757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:49.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.603+0000 7f9d0ea00700 1 --2- 172.21.15.1:0/985390397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d10107940 0x7f9d100757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:49.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.603+0000 7f9d0ea00700 1 -- 172.21.15.1:0/985390397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d10102af0 msgr2=0x7f9d10074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:49.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.603+0000 7f9d0ea00700 1 --2- 172.21.15.1:0/985390397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d10102af0 0x7f9d10074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:49.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.603+0000 7f9d0ea00700 1 -- 172.21.15.1:0/985390397 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d101a75d0 con 0x7f9d101034b0 2024-04-26T01:58:49.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.603+0000 7f9d0fe00700 1 --2- 172.21.15.1:0/985390397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d10107940 0x7f9d100757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:49.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.603+0000 7f9d0ea00700 1 --2- 172.21.15.1:0/985390397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d101034b0 0x7f9d10075280 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc00e9a0 tx=0x7f9cfc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:49.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.604+0000 7f9d07e00700 1 -- 172.21.15.1:0/985390397 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc00cbd0 con 0x7f9d101034b0 2024-04-26T01:58:49.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.604+0000 7f9d07e00700 1 -- 172.21.15.1:0/985390397 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cfc00cd70 con 0x7f9d101034b0 2024-04-26T01:58:49.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.604+0000 7f9d16800700 1 -- 172.21.15.1:0/985390397 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d101a78c0 con 0x7f9d101034b0 2024-04-26T01:58:49.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.604+0000 7f9d07e00700 1 -- 172.21.15.1:0/985390397 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc018680 con 0x7f9d101034b0 2024-04-26T01:58:49.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.604+0000 7f9d16800700 1 -- 172.21.15.1:0/985390397 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d101a7e50 con 0x7f9d101034b0 2024-04-26T01:58:49.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.605+0000 7f9d16800700 1 -- 172.21.15.1:0/985390397 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d10069730 con 0x7f9d101034b0 2024-04-26T01:58:49.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.607+0000 7f9d07e00700 1 -- 172.21.15.1:0/985390397 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f9cfc010430 con 0x7f9d101034b0 2024-04-26T01:58:49.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.609+0000 7f9d07e00700 1 --2- 172.21.15.1:0/985390397 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f9cf40a3e30 0x7f9cf40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:49.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.609+0000 7f9d07e00700 1 -- 172.21.15.1:0/985390397 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f9cfc0d09f0 con 0x7f9d101034b0 2024-04-26T01:58:49.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.609+0000 7f9d0f400700 1 --2- 172.21.15.1:0/985390397 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f9cf40a3e30 0x7f9cf40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:49.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.610+0000 7f9d0f400700 1 --2- 172.21.15.1:0/985390397 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f9cf40a3e30 0x7f9cf40a62f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000f540 tx=0x7f9d0000b040 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:49.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.611+0000 7f9d07e00700 1 -- 172.21.15.1:0/985390397 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9cfc09d000 con 0x7f9d101034b0 2024-04-26T01:58:49.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:49 smithi097 ceph-mon[32853]: pgmap v221: 97 pgs: 97 active+clean; 49 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.3 KiB/s wr, 8 op/s 2024-04-26T01:58:49.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.899+0000 7f9d16800700 1 -- 172.21.15.1:0/985390397 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d1002df80 con 0x7f9d101034b0 2024-04-26T01:58:49.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.900+0000 7f9d07e00700 1 -- 172.21.15.1:0/985390397 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11838 (secure 0 0 0) 0x7f9cfc09c900 con 0x7f9d101034b0 2024-04-26T01:58:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:58:49.900 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:55:49.645549+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25414":{"gid":25414,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/453427322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":453427322},{"type":"v1","addr":"172.21.15.173:6835","nonce":453427322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24499,24514,24518]},"id":1}]} 2024-04-26T01:58:49.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.904+0000 7f9d16800700 1 -- 172.21.15.1:0/985390397 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f9cf40a3e30 msgr2=0x7f9cf40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:49.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.904+0000 7f9d16800700 1 --2- 172.21.15.1:0/985390397 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f9cf40a3e30 0x7f9cf40a62f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000f540 tx=0x7f9d0000b040 comp rx=0 tx=0).stop 2024-04-26T01:58:49.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.904+0000 7f9d16800700 1 -- 172.21.15.1:0/985390397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d101034b0 msgr2=0x7f9d10075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:49.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.904+0000 7f9d16800700 1 --2- 172.21.15.1:0/985390397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d101034b0 0x7f9d10075280 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc00e9a0 tx=0x7f9cfc00ecb0 comp rx=0 tx=0).stop 2024-04-26T01:58:49.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.904+0000 7f9d16800700 1 -- 172.21.15.1:0/985390397 shutdown_connections 2024-04-26T01:58:49.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.904+0000 7f9d16800700 1 --2- 172.21.15.1:0/985390397 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f9cf40a3e30 0x7f9cf40a62f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:49.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.904+0000 7f9d16800700 1 --2- 172.21.15.1:0/985390397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d10107940 0x7f9d100757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:49.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.904+0000 7f9d16800700 1 --2- 172.21.15.1:0/985390397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d10102af0 0x7f9d10074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:49.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.904+0000 7f9d16800700 1 --2- 172.21.15.1:0/985390397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d101034b0 0x7f9d10075280 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:49.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.904+0000 7f9d16800700 1 -- 172.21.15.1:0/985390397 >> 172.21.15.1:0/985390397 conn(0x7f9d10079ba0 msgr2=0x7f9d10100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:49.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.904+0000 7f9d16800700 1 -- 172.21.15.1:0/985390397 shutdown_connections 2024-04-26T01:58:49.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:49.904+0000 7f9d16800700 1 -- 172.21.15.1:0/985390397 wait complete. 2024-04-26T01:58:49.904 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 32 2024-04-26T01:58:50.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:50 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/985390397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:58:50.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:50 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/985390397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:58:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.716+0000 7f7061600640 1 -- 172.21.15.1:0/4229479554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705c107280 msgr2=0x7f705c10ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.716+0000 7f7061600640 1 --2- 172.21.15.1:0/4229479554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705c107280 0x7f705c10ee50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7054009ca0 tx=0x7f705402a8c0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.716+0000 7f7061600640 1 -- 172.21.15.1:0/4229479554 shutdown_connections 2024-04-26T01:58:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.716+0000 7f7061600640 1 --2- 172.21.15.1:0/4229479554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705c107280 0x7f705c10ee50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.716+0000 7f7061600640 1 --2- 172.21.15.1:0/4229479554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f705c102b60 0x7f705c106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.716+0000 7f7061600640 1 --2- 172.21.15.1:0/4229479554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705c1021b0 0x7f705c102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.716+0000 7f7061600640 1 -- 172.21.15.1:0/4229479554 >> 172.21.15.1:0/4229479554 conn(0x7f705c0fda10 msgr2=0x7f705c0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.717+0000 7f7061600640 1 -- 172.21.15.1:0/4229479554 shutdown_connections 2024-04-26T01:58:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.717+0000 7f7061600640 1 -- 172.21.15.1:0/4229479554 wait complete. 2024-04-26T01:58:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.718+0000 7f7061600640 1 Processor -- start 2024-04-26T01:58:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.718+0000 7f7061600640 1 -- start start 2024-04-26T01:58:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.719+0000 7f7061600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705c1021b0 0x7f705c1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.719+0000 7f7061600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705c107280 0x7f705c1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.719+0000 7f7061600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f705c19a740 0x7f705c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.719+0000 7f7061600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f705c112220 con 0x7f705c19a740 2024-04-26T01:58:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.719+0000 7f7061600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1120a0 con 0x7f705c1021b0 2024-04-26T01:58:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.719+0000 7f7061600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1123a0 con 0x7f705c107280 2024-04-26T01:58:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.719+0000 7f7053e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705c107280 0x7f705c1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.719+0000 7f705b400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f705c19a740 0x7f705c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.719+0000 7f705aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705c1021b0 0x7f705c1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.719+0000 7f7053e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705c107280 0x7f705c1a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:49360/0 (socket says 172.21.15.1:49360) 2024-04-26T01:58:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.719+0000 7f7053e00640 1 -- 172.21.15.1:0/1730478609 learned_addr learned my addr 172.21.15.1:0/1730478609 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.719+0000 7f7053e00640 1 -- 172.21.15.1:0/1730478609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705c1021b0 msgr2=0x7f705c1a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.719+0000 7f7053e00640 1 --2- 172.21.15.1:0/1730478609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705c1021b0 0x7f705c1a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.719+0000 7f7053e00640 1 -- 172.21.15.1:0/1730478609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f705c19a740 msgr2=0x7f705c19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.719+0000 7f7053e00640 1 --2- 172.21.15.1:0/1730478609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f705c19a740 0x7f705c19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.720+0000 7f7053e00640 1 -- 172.21.15.1:0/1730478609 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705c19b340 con 0x7f705c107280 2024-04-26T01:58:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.720+0000 7f7053e00640 1 --2- 172.21.15.1:0/1730478609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705c107280 0x7f705c1a0bc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f704400e9c0 tx=0x7f704400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.720+0000 7f7053400640 1 -- 172.21.15.1:0/1730478609 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704400cd80 con 0x7f705c107280 2024-04-26T01:58:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.720+0000 7f7053400640 1 -- 172.21.15.1:0/1730478609 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70440054d0 con 0x7f705c107280 2024-04-26T01:58:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.720+0000 7f7053400640 1 -- 172.21.15.1:0/1730478609 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7044010430 con 0x7f705c107280 2024-04-26T01:58:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.720+0000 7f7061600640 1 -- 172.21.15.1:0/1730478609 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705c1a7420 con 0x7f705c107280 2024-04-26T01:58:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.720+0000 7f7061600640 1 -- 172.21.15.1:0/1730478609 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f705c1a77d0 con 0x7f705c107280 2024-04-26T01:58:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.725+0000 7f7053400640 1 -- 172.21.15.1:0/1730478609 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f7044004fd0 con 0x7f705c107280 2024-04-26T01:58:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.725+0000 7f7064388740 1 Processor -- start 2024-04-26T01:58:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.726+0000 7f7064388740 1 -- start start 2024-04-26T01:58:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.726+0000 7f7064388740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674de820 0x5625674dec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.726+0000 7f7064388740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674df140 0x5625674e94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.726+0000 7f7064388740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674e9a00 0x5625674ebdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.726+0000 7f7064388740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562567444900 con 0x5625674e9a00 2024-04-26T01:58:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.726+0000 7f7064388740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5625673eaef0 con 0x5625674df140 2024-04-26T01:58:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.726+0000 7f7064388740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5625674408c0 con 0x5625674de820 2024-04-26T01:58:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.726+0000 7f705aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674df140 0x5625674e94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.726+0000 7f705b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674de820 0x5625674dec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.726+0000 7f705b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674de820 0x5625674dec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:49376/0 (socket says 172.21.15.1:49376) 2024-04-26T01:58:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.726+0000 7f705aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674df140 0x5625674e94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:46110/0 (socket says 172.21.15.1:46110) 2024-04-26T01:58:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.726+0000 7f705b400640 1 -- 172.21.15.1:0/2188992739 learned_addr learned my addr 172.21.15.1:0/2188992739 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.726+0000 7f7053e00640 1 --2- 172.21.15.1:0/2188992739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674e9a00 0x5625674ebdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.727+0000 7f705b400640 1 -- 172.21.15.1:0/2188992739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674df140 msgr2=0x5625674e94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.727+0000 7f705b400640 1 --2- 172.21.15.1:0/2188992739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674df140 0x5625674e94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.727+0000 7f705b400640 1 -- 172.21.15.1:0/2188992739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674e9a00 msgr2=0x5625674ebdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.727+0000 7f705b400640 1 --2- 172.21.15.1:0/2188992739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674e9a00 0x5625674ebdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.727+0000 7f705b400640 1 -- 172.21.15.1:0/2188992739 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562567157210 con 0x5625674de820 2024-04-26T01:58:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.727+0000 7f7053e00640 1 --2- 172.21.15.1:0/2188992739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674e9a00 0x5625674ebdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.727+0000 7f705b400640 1 --2- 172.21.15.1:0/2188992739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674de820 0x5625674dec00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7054009ca0 tx=0x7f70540093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e75101ee516b1834 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.727+0000 7f7053400640 1 --2- 172.21.15.1:0/1730478609 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f70400a3d90 0x7f70400a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.728+0000 7f7053400640 1 -- 172.21.15.1:0/1730478609 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f70440d0b50 con 0x7f705c107280 2024-04-26T01:58:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.728+0000 7f705aa00640 1 --2- 172.21.15.1:0/1730478609 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f70400a3d90 0x7f70400a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.728+0000 7f704b400640 1 -- 172.21.15.1:0/2188992739 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054031cf0 con 0x5625674de820 2024-04-26T01:58:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.728+0000 7f704b400640 1 -- 172.21.15.1:0/2188992739 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7054036030 con 0x5625674de820 2024-04-26T01:58:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.728+0000 7f704b400640 1 -- 172.21.15.1:0/2188992739 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70540042a0 con 0x5625674de820 2024-04-26T01:58:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.729+0000 7f7064388740 1 -- 172.21.15.1:0/2188992739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674de820 msgr2=0x5625674dec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.729+0000 7f7064388740 1 --2- 172.21.15.1:0/2188992739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674de820 0x5625674dec00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7054009ca0 tx=0x7f70540093e0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.729+0000 7f705aa00640 1 --2- 172.21.15.1:0/1730478609 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f70400a3d90 0x7f70400a6250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x5625674dfdc0 tx=0x7f704c005f90 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.729+0000 7f7064388740 1 -- 172.21.15.1:0/2188992739 shutdown_connections 2024-04-26T01:58:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.729+0000 7f7064388740 1 --2- 172.21.15.1:0/2188992739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674e9a00 0x5625674ebdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.729+0000 7f7064388740 1 --2- 172.21.15.1:0/2188992739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674df140 0x5625674e94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.729+0000 7f7064388740 1 --2- 172.21.15.1:0/2188992739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674de820 0x5625674dec00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.729+0000 7f7064388740 1 -- 172.21.15.1:0/2188992739 >> 172.21.15.1:0/2188992739 conn(0x5625674dad30 msgr2=0x5625674db110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.729+0000 7f7064388740 1 -- 172.21.15.1:0/2188992739 shutdown_connections 2024-04-26T01:58:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.729+0000 7f7064388740 1 -- 172.21.15.1:0/2188992739 wait complete. 2024-04-26T01:58:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.729+0000 7f7064388740 1 -- start start 2024-04-26T01:58:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.730+0000 7f7064388740 1 Processor -- start 2024-04-26T01:58:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.730+0000 7f7064388740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674de820 0x5625674e69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.730+0000 7f7064388740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674df140 0x5625674e6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.730+0000 7f7064388740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674e9a00 0x5625674e7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.730+0000 7f7064388740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5625674ec450 con 0x5625674e9a00 2024-04-26T01:58:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.730+0000 7f7064388740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5625673fe310 con 0x5625674df140 2024-04-26T01:58:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.730+0000 7f7064388740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5625674ec5d0 con 0x5625674de820 2024-04-26T01:58:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.730+0000 7f7064388740 10 client.0 _ll_register_callbacks cb 0x5625673cccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.730+0000 7f705aa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674de820 0x5625674e69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.730+0000 7f705b400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674e9a00 0x5625674e7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.731+0000 7f705aa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674de820 0x5625674e69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:49384/0 (socket says 172.21.15.1:49384) 2024-04-26T01:58:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.731+0000 7f705aa00640 1 -- 172.21.15.1:0/3222521358 learned_addr learned my addr 172.21.15.1:0/3222521358 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.731+0000 7f7053e00640 1 --2- 172.21.15.1:0/3222521358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674df140 0x5625674e6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.731+0000 7f705aa00640 1 -- 172.21.15.1:0/3222521358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674df140 msgr2=0x5625674e6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.731+0000 7f705aa00640 1 --2- 172.21.15.1:0/3222521358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674df140 0x5625674e6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.731+0000 7f705aa00640 1 -- 172.21.15.1:0/3222521358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674e9a00 msgr2=0x5625674e7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.731+0000 7f705aa00640 1 --2- 172.21.15.1:0/3222521358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674e9a00 0x5625674e7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.731+0000 7f705aa00640 1 -- 172.21.15.1:0/3222521358 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562567223620 con 0x5625674de820 2024-04-26T01:58:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.731+0000 7f705aa00640 1 --2- 172.21.15.1:0/3222521358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674de820 0x5625674e69f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f704c01e9d0 tx=0x7f704c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dc53b094839afe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.731+0000 7f703fe00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:58:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.732+0000 7f703fe00640 1 -- 172.21.15.1:0/3222521358 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c02e060 con 0x5625674de820 2024-04-26T01:58:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.732+0000 7f7064388740 1 -- 172.21.15.1:0/3222521358 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7054036030 con 0x5625674de820 2024-04-26T01:58:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.732+0000 7f703fe00640 1 -- 172.21.15.1:0/3222521358 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f704c018030 con 0x5625674de820 2024-04-26T01:58:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.732+0000 7f703fe00640 1 -- 172.21.15.1:0/3222521358 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f704c009e60 con 0x5625674de820 2024-04-26T01:58:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.732+0000 7f7064388740 10 client.26047 fetch_fsmap learned FSMap version 32 2024-04-26T01:58:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.732+0000 7f7064388740 1 -- 172.21.15.1:0/3222521358 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5625674f87c0 con 0x5625674de820 2024-04-26T01:58:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.732+0000 7f703fe00640 1 -- 172.21.15.1:0/3222521358 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7054036030 con 0x5625674de820 2024-04-26T01:58:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.733+0000 7f703fe00640 1 -- 172.21.15.1:0/3222521358 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f704c031610 con 0x5625674de820 2024-04-26T01:58:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.733+0000 7f7064388740 10 client.26047 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:58:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.733+0000 7f7064388740 10 client.26047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:58:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.733+0000 7f7064388740 20 client.26047 populate_metadata read hostname 'smithi001' 2024-04-26T01:58:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.733+0000 7f7064388740 1 --2- 172.21.15.1:0/3222521358 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5625674f9c30 0x5625674fc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.733+0000 7f7064388740 4 client.26047 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:58:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.733+0000 7f7064388740 1 -- 172.21.15.1:0/3222521358 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625674588b0 con 0x5625674f9c30 2024-04-26T01:58:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.733+0000 7f7053e00640 1 --2- 172.21.15.1:0/3222521358 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5625674f9c30 0x5625674fc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.734+0000 7f7053e00640 1 --2- 172.21.15.1:0/3222521358 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5625674f9c30 0x5625674fc020 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1cbf1d921b8a6e1 server_cookie=5e922e3fa3ed861b in_seq=0 out_seq=0 2024-04-26T01:58:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.734+0000 7f703fe00640 10 client.26047 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.735+0000 7f703fe00640 1 -- 172.21.15.1:0/3222521358 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5625674588b0 con 0x5625674f9c30 2024-04-26T01:58:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.735+0000 7f703fe00640 10 client.26047 handle_command_reply: tid=0 2024-04-26T01:58:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.735+0000 7f7064388740 1 client.26047 shutdown 2024-04-26T01:58:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.735+0000 7f7064388740 1 -- 172.21.15.1:0/3222521358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674de820 msgr2=0x5625674e69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.735+0000 7f7064388740 1 --2- 172.21.15.1:0/3222521358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674de820 0x5625674e69f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f704c01e9d0 tx=0x7f704c004730 comp rx=0 tx=0).stop 2024-04-26T01:58:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.735+0000 7f7064388740 1 -- 172.21.15.1:0/3222521358 shutdown_connections 2024-04-26T01:58:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.735+0000 7f7064388740 1 --2- 172.21.15.1:0/3222521358 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5625674f9c30 0x5625674fc020 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.735+0000 7f7064388740 1 --2- 172.21.15.1:0/3222521358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674e9a00 0x5625674e7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.735+0000 7f7064388740 1 --2- 172.21.15.1:0/3222521358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674df140 0x5625674e6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.735+0000 7f7064388740 1 --2- 172.21.15.1:0/3222521358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674de820 0x5625674e69f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.735+0000 7f703fe00640 0 client.26047 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.735+0000 7f7064388740 1 -- 172.21.15.1:0/3222521358 >> 172.21.15.1:0/3222521358 conn(0x5625674dad30 msgr2=0x5625674e0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.735+0000 7f7064388740 1 -- 172.21.15.1:0/3222521358 shutdown_connections 2024-04-26T01:58:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.735+0000 7f7064388740 1 -- 172.21.15.1:0/3222521358 wait complete. 2024-04-26T01:58:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.735+0000 7f7064388740 20 client.26047 trim_cache size 0 max 16384 2024-04-26T01:58:50.737 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:50 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/985390397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:58:50.738 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24518 quiesce db --set-id 2f800756 --release --await' 2024-04-26T01:58:50.739 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24518 quiesce db --set-id 2f800756 --release --await 2024-04-26T01:58:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.752+0000 7f7064388740 1 Processor -- start 2024-04-26T01:58:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.752+0000 7f7064388740 1 -- start start 2024-04-26T01:58:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.752+0000 7f7064388740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674de820 0x5625674f5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.752+0000 7f7064388740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674df140 0x5625674e11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.752+0000 7f7064388740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674e9a00 0x562567501910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.752+0000 7f7064388740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1b2780 con 0x5625674de820 2024-04-26T01:58:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.752+0000 7f7064388740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1b0050 con 0x5625674df140 2024-04-26T01:58:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.752+0000 7f7064388740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1adec0 con 0x5625674e9a00 2024-04-26T01:58:50.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.752+0000 7f705aa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674de820 0x5625674f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:50.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.753+0000 7f705aa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674de820 0x5625674f5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34954/0 (socket says 172.21.15.1:34954) 2024-04-26T01:58:50.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.753+0000 7f705aa00640 1 -- 172.21.15.1:0/1728875212 learned_addr learned my addr 172.21.15.1:0/1728875212 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:50.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.753+0000 7f705b400640 1 --2- 172.21.15.1:0/1728875212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674e9a00 0x562567501910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.753+0000 7f7053e00640 1 --2- 172.21.15.1:0/1728875212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674df140 0x5625674e11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.753+0000 7f705aa00640 1 -- 172.21.15.1:0/1728875212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674e9a00 msgr2=0x562567501910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.753+0000 7f705aa00640 1 --2- 172.21.15.1:0/1728875212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674e9a00 0x562567501910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.753+0000 7f705aa00640 1 -- 172.21.15.1:0/1728875212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674df140 msgr2=0x5625674e11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.753+0000 7f705aa00640 1 --2- 172.21.15.1:0/1728875212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674df140 0x5625674e11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.753+0000 7f705aa00640 1 -- 172.21.15.1:0/1728875212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625674ec160 con 0x5625674de820 2024-04-26T01:58:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.753+0000 7f7053e00640 1 --2- 172.21.15.1:0/1728875212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674df140 0x5625674e11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.753+0000 7f705aa00640 1 --2- 172.21.15.1:0/1728875212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674de820 0x5625674f5310 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f704c01e860 tx=0x7f704c031e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2284ef2c020ec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.754+0000 7f704aa00640 1 -- 172.21.15.1:0/1728875212 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c0313e0 con 0x5625674de820 2024-04-26T01:58:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.754+0000 7f704aa00640 1 -- 172.21.15.1:0/1728875212 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f704c031580 con 0x5625674de820 2024-04-26T01:58:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.754+0000 7f704aa00640 1 -- 172.21.15.1:0/1728875212 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c032da0 con 0x5625674de820 2024-04-26T01:58:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.754+0000 7f7064388740 1 -- 172.21.15.1:0/1728875212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674de820 msgr2=0x5625674f5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.754+0000 7f7064388740 1 --2- 172.21.15.1:0/1728875212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674de820 0x5625674f5310 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f704c01e860 tx=0x7f704c031e20 comp rx=0 tx=0).stop 2024-04-26T01:58:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.755+0000 7f7064388740 1 -- 172.21.15.1:0/1728875212 shutdown_connections 2024-04-26T01:58:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.755+0000 7f7064388740 1 --2- 172.21.15.1:0/1728875212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674e9a00 0x562567501910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.755+0000 7f7064388740 1 --2- 172.21.15.1:0/1728875212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674df140 0x5625674e11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.755+0000 7f7064388740 1 --2- 172.21.15.1:0/1728875212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674de820 0x5625674f5310 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.755+0000 7f7064388740 1 -- 172.21.15.1:0/1728875212 >> 172.21.15.1:0/1728875212 conn(0x5625674dad30 msgr2=0x5625674e8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.755+0000 7f7064388740 1 -- 172.21.15.1:0/1728875212 shutdown_connections 2024-04-26T01:58:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.755+0000 7f7064388740 1 -- 172.21.15.1:0/1728875212 wait complete. 2024-04-26T01:58:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.755+0000 7f7064388740 1 -- start start 2024-04-26T01:58:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.755+0000 7f7064388740 1 Processor -- start 2024-04-26T01:58:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.756+0000 7f7064388740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674de820 0x5625674ef8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.756+0000 7f7064388740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674df140 0x5625674efdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.756+0000 7f7064388740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674e9a00 0x56256755f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.756+0000 7f7064388740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1ae510 con 0x5625674de820 2024-04-26T01:58:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.756+0000 7f7064388740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1af980 con 0x5625674e9a00 2024-04-26T01:58:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.756+0000 7f7064388740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1aebe0 con 0x5625674df140 2024-04-26T01:58:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.756+0000 7f7064388740 10 client.0 _ll_register_callbacks cb 0x7f704c01e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.756+0000 7f705aa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674de820 0x5625674ef8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.756+0000 7f7053e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674df140 0x5625674efdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.756+0000 7f705aa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674de820 0x5625674ef8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34964/0 (socket says 172.21.15.1:34964) 2024-04-26T01:58:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.756+0000 7f705aa00640 1 -- 172.21.15.1:0/567336287 learned_addr learned my addr 172.21.15.1:0/567336287 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.756+0000 7f705b400640 1 --2- 172.21.15.1:0/567336287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674e9a00 0x56256755f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.757+0000 7f7053e00640 1 -- 172.21.15.1:0/567336287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674e9a00 msgr2=0x56256755f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.757+0000 7f7053e00640 1 --2- 172.21.15.1:0/567336287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674e9a00 0x56256755f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.757+0000 7f7053e00640 1 -- 172.21.15.1:0/567336287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674de820 msgr2=0x5625674ef8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.757+0000 7f7053e00640 1 --2- 172.21.15.1:0/567336287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674de820 0x5625674ef8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.757+0000 7f7053e00640 1 -- 172.21.15.1:0/567336287 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56256755f6d0 con 0x5625674df140 2024-04-26T01:58:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.757+0000 7f705aa00640 1 --2- 172.21.15.1:0/567336287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674de820 0x5625674ef8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.757+0000 7f7053e00640 1 --2- 172.21.15.1:0/567336287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674df140 0x5625674efdf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f70440ce210 tx=0x7f70440bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcbd75b98204c2e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.757+0000 7f703fe00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:58:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.757+0000 7f703fe00640 1 -- 172.21.15.1:0/567336287 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70440bd0b0 con 0x5625674df140 2024-04-26T01:58:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.757+0000 7f703fe00640 1 -- 172.21.15.1:0/567336287 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f704409ad60 con 0x5625674df140 2024-04-26T01:58:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.757+0000 7f7064388740 1 -- 172.21.15.1:0/567336287 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f704c031580 con 0x5625674df140 2024-04-26T01:58:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.757+0000 7f703fe00640 1 -- 172.21.15.1:0/567336287 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f704409d160 con 0x5625674df140 2024-04-26T01:58:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.757+0000 7f7064388740 10 client.26053 fetch_fsmap learned FSMap version 32 2024-04-26T01:58:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.757+0000 7f7064388740 1 -- 172.21.15.1:0/567336287 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562567560970 con 0x5625674df140 2024-04-26T01:58:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.758+0000 7f703fe00640 1 -- 172.21.15.1:0/567336287 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f704c031580 con 0x5625674df140 2024-04-26T01:58:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.758+0000 7f703fe00640 1 -- 172.21.15.1:0/567336287 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f70440a5240 con 0x5625674df140 2024-04-26T01:58:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.758+0000 7f7064388740 10 client.26053 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:58:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.758+0000 7f7064388740 10 client.26053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:58:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.758+0000 7f7064388740 20 client.26053 populate_metadata read hostname 'smithi001' 2024-04-26T01:58:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.758+0000 7f7064388740 1 --2- 172.21.15.1:0/567336287 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562567561040 0x562567563460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.758+0000 7f7064388740 4 client.26053 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:58:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.758+0000 7f7064388740 1 -- 172.21.15.1:0/567336287 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562567452320 con 0x562567561040 2024-04-26T01:58:50.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.758+0000 7f705aa00640 1 --2- 172.21.15.1:0/567336287 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562567561040 0x562567563460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:50.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.759+0000 7f705aa00640 1 --2- 172.21.15.1:0/567336287 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562567561040 0x562567563460 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9630cd1ee3148d90 server_cookie=da510f9b966e85e0 in_seq=0 out_seq=0 2024-04-26T01:58:50.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.759+0000 7f703fe00640 10 client.26053 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.771+0000 7f703fe00640 1 -- 172.21.15.1:0/567336287 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562567452320 con 0x562567561040 2024-04-26T01:58:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.771+0000 7f703fe00640 10 client.26053 handle_command_reply: tid=0 2024-04-26T01:58:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.771+0000 7f7064388740 1 client.26053 shutdown 2024-04-26T01:58:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.771+0000 7f7064388740 1 -- 172.21.15.1:0/567336287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674df140 msgr2=0x5625674efdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.771+0000 7f7064388740 1 --2- 172.21.15.1:0/567336287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674df140 0x5625674efdf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f70440ce210 tx=0x7f70440bd040 comp rx=0 tx=0).stop 2024-04-26T01:58:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.771+0000 7f7064388740 1 -- 172.21.15.1:0/567336287 shutdown_connections 2024-04-26T01:58:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.771+0000 7f7064388740 1 --2- 172.21.15.1:0/567336287 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562567561040 0x562567563460 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.771+0000 7f7064388740 1 --2- 172.21.15.1:0/567336287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5625674e9a00 0x56256755f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.771+0000 7f7064388740 1 --2- 172.21.15.1:0/567336287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625674df140 0x5625674efdf0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.771+0000 7f703fe00640 0 client.26053 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.771+0000 7f7064388740 1 --2- 172.21.15.1:0/567336287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5625674de820 0x5625674ef8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.772+0000 7f7064388740 1 -- 172.21.15.1:0/567336287 >> 172.21.15.1:0/567336287 conn(0x5625674dad30 msgr2=0x5625674e8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.772+0000 7f7064388740 1 -- 172.21.15.1:0/567336287 shutdown_connections 2024-04-26T01:58:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.772+0000 7f7064388740 1 -- 172.21.15.1:0/567336287 wait complete. 2024-04-26T01:58:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.772+0000 7f7064388740 20 client.26053 trim_cache size 0 max 16384 2024-04-26T01:58:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:58:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:58:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:58:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:58:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.773+0000 7f7061600640 1 -- 172.21.15.1:0/1730478609 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f70400a3d90 msgr2=0x7f70400a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.773+0000 7f7061600640 1 --2- 172.21.15.1:0/1730478609 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f70400a3d90 0x7f70400a6250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x5625674dfdc0 tx=0x7f704c005f90 comp rx=0 tx=0).stop 2024-04-26T01:58:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.773+0000 7f7061600640 1 -- 172.21.15.1:0/1730478609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705c107280 msgr2=0x7f705c1a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.773+0000 7f7061600640 1 --2- 172.21.15.1:0/1730478609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705c107280 0x7f705c1a0bc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f704400e9c0 tx=0x7f704400ee80 comp rx=0 tx=0).stop 2024-04-26T01:58:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.773+0000 7f7061600640 1 -- 172.21.15.1:0/1730478609 shutdown_connections 2024-04-26T01:58:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.773+0000 7f7061600640 1 --2- 172.21.15.1:0/1730478609 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f70400a3d90 0x7f70400a6250 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.773+0000 7f7061600640 1 --2- 172.21.15.1:0/1730478609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f705c19a740 0x7f705c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.773+0000 7f7061600640 1 --2- 172.21.15.1:0/1730478609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705c107280 0x7f705c1a0bc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.773+0000 7f7061600640 1 --2- 172.21.15.1:0/1730478609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705c1021b0 0x7f705c1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.773+0000 7f7061600640 1 -- 172.21.15.1:0/1730478609 >> 172.21.15.1:0/1730478609 conn(0x7f705c0fda10 msgr2=0x7f705c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.773+0000 7f7061600640 1 -- 172.21.15.1:0/1730478609 shutdown_connections 2024-04-26T01:58:50.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:58:50.773+0000 7f7061600640 1 -- 172.21.15.1:0/1730478609 wait complete. 2024-04-26T01:58:51.189 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:58:51.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:51 smithi001 ceph-mon[27822]: pgmap v222: 97 pgs: 97 active+clean; 49 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-04-26T01:58:51.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:51 smithi097 ceph-mon[32853]: pgmap v222: 97 pgs: 97 active+clean; 49 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-04-26T01:58:51.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:51 smithi173 ceph-mon[32553]: pgmap v222: 97 pgs: 97 active+clean; 49 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-04-26T01:58:53.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.554+0000 7fe38b200700 1 -- 172.21.15.1:0/122797028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c076f70 msgr2=0x7fe38c077350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:53.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.554+0000 7fe38b200700 1 --2- 172.21.15.1:0/122797028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c076f70 0x7fe38c077350 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009990 tx=0x7fe37c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.554+0000 7fe38b200700 1 -- 172.21.15.1:0/122797028 shutdown_connections 2024-04-26T01:58:53.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.554+0000 7fe38b200700 1 --2- 172.21.15.1:0/122797028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe38c1137f0 0x7fe38c115be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.554+0000 7fe38b200700 1 --2- 172.21.15.1:0/122797028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c076f70 0x7fe38c077350 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.554+0000 7fe38b200700 1 --2- 172.21.15.1:0/122797028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe38c077890 0x7fe38c1132b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.554+0000 7fe38b200700 1 -- 172.21.15.1:0/122797028 >> 172.21.15.1:0/122797028 conn(0x7fe38c100680 msgr2=0x7fe38c102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:53.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.555+0000 7fe38b200700 1 -- 172.21.15.1:0/122797028 shutdown_connections 2024-04-26T01:58:53.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.555+0000 7fe38b200700 1 -- 172.21.15.1:0/122797028 wait complete. 2024-04-26T01:58:53.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.556+0000 7fe38b200700 1 Processor -- start 2024-04-26T01:58:53.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.556+0000 7fe38b200700 1 -- start start 2024-04-26T01:58:53.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.556+0000 7fe38b200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe38c076f70 0x7fe38c19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:53.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.556+0000 7fe38b200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c077890 0x7fe38c19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:53.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.556+0000 7fe38b200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe38c1137f0 0x7fe38c1a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:53.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.556+0000 7fe38b200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c1191e0 con 0x7fe38c077890 2024-04-26T01:58:53.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.556+0000 7fe38b200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c119060 con 0x7fe38c076f70 2024-04-26T01:58:53.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.556+0000 7fe38b200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c119360 con 0x7fe38c1137f0 2024-04-26T01:58:53.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.557+0000 7fe389400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c077890 0x7fe38c19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:53.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.557+0000 7fe389e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe38c076f70 0x7fe38c19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:53.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.557+0000 7fe389400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c077890 0x7fe38c19fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37412/0 (socket says 172.21.15.1:37412) 2024-04-26T01:58:53.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.557+0000 7fe389400700 1 -- 172.21.15.1:0/275964341 learned_addr learned my addr 172.21.15.1:0/275964341 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:53.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.557+0000 7fe38a800700 1 --2- 172.21.15.1:0/275964341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe38c1137f0 0x7fe38c1a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:53.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.557+0000 7fe389400700 1 -- 172.21.15.1:0/275964341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe38c1137f0 msgr2=0x7fe38c1a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:53.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.557+0000 7fe389400700 1 --2- 172.21.15.1:0/275964341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe38c1137f0 0x7fe38c1a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.557+0000 7fe389400700 1 -- 172.21.15.1:0/275964341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe38c076f70 msgr2=0x7fe38c19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:53.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.557+0000 7fe389400700 1 --2- 172.21.15.1:0/275964341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe38c076f70 0x7fe38c19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.557+0000 7fe389400700 1 -- 172.21.15.1:0/275964341 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe38c1a46c0 con 0x7fe38c077890 2024-04-26T01:58:53.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.558+0000 7fe389400700 1 --2- 172.21.15.1:0/275964341 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c077890 0x7fe38c19fbf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe37800d980 tx=0x7fe37800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:53.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.558+0000 7fe382a00700 1 -- 172.21.15.1:0/275964341 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe378017070 con 0x7fe38c077890 2024-04-26T01:58:53.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.558+0000 7fe382a00700 1 -- 172.21.15.1:0/275964341 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe378005020 con 0x7fe38c077890 2024-04-26T01:58:53.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.558+0000 7fe382a00700 1 -- 172.21.15.1:0/275964341 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37800fa10 con 0x7fe38c077890 2024-04-26T01:58:53.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.558+0000 7fe38b200700 1 -- 172.21.15.1:0/275964341 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe38c1a49b0 con 0x7fe38c077890 2024-04-26T01:58:53.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.558+0000 7fe38b200700 1 -- 172.21.15.1:0/275964341 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe38c1ac230 con 0x7fe38c077890 2024-04-26T01:58:53.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.565+0000 7fe382a00700 1 -- 172.21.15.1:0/275964341 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fe378004a30 con 0x7fe38c077890 2024-04-26T01:58:53.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.565+0000 7fe39ba2bb80 1 Processor -- start 2024-04-26T01:58:53.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.565+0000 7fe39ba2bb80 1 -- start start 2024-04-26T01:58:53.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.565+0000 7fe39ba2bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e57ea0 0x5581d8e58280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:53.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.565+0000 7fe39ba2bb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e587c0 0x5581d8e63130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:53.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.565+0000 7fe39ba2bb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e63670 0x5581d8e65a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:53.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.566+0000 7fe39ba2bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5581d8daaf90 con 0x5581d8e57ea0 2024-04-26T01:58:53.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.566+0000 7fe39ba2bb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5581d8d51830 con 0x5581d8e63670 2024-04-26T01:58:53.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.566+0000 7fe39ba2bb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5581d8da6f50 con 0x5581d8e587c0 2024-04-26T01:58:53.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.566+0000 7fe38a800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e57ea0 0x5581d8e58280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:53.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.566+0000 7fe389400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e63670 0x5581d8e65a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:53.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.566+0000 7fe38a800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e57ea0 0x5581d8e58280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37416/0 (socket says 172.21.15.1:37416) 2024-04-26T01:58:53.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.566+0000 7fe389e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e587c0 0x5581d8e63130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:53.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.566+0000 7fe38a800700 1 -- 172.21.15.1:0/633469514 learned_addr learned my addr 172.21.15.1:0/633469514 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:53.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.566+0000 7fe382a00700 1 --2- 172.21.15.1:0/275964341 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe3700a3e30 0x7fe3700a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:53.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.566+0000 7fe389400700 1 -- 172.21.15.1:0/633469514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e587c0 msgr2=0x5581d8e63130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:53.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.566+0000 7fe389400700 1 --2- 172.21.15.1:0/633469514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e587c0 0x5581d8e63130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.566+0000 7fe389400700 1 -- 172.21.15.1:0/633469514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e57ea0 msgr2=0x5581d8e58280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:53.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.566+0000 7fe389400700 1 --2- 172.21.15.1:0/633469514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e57ea0 0x5581d8e58280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.566+0000 7fe389400700 1 -- 172.21.15.1:0/633469514 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d8d47780 con 0x5581d8e63670 2024-04-26T01:58:53.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.567+0000 7fe38a800700 1 --2- 172.21.15.1:0/275964341 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe3700a3e30 0x7fe3700a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:53.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.567+0000 7fe382a00700 1 -- 172.21.15.1:0/275964341 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fe3780d06c0 con 0x7fe38c077890 2024-04-26T01:58:53.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.567+0000 7fe389400700 1 --2- 172.21.15.1:0/633469514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e63670 0x5581d8e65a60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe3780cdd80 tx=0x7fe3780988f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e50d646490422fa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:53.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.567+0000 7fe377400700 1 -- 172.21.15.1:0/633469514 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37809c3a0 con 0x5581d8e63670 2024-04-26T01:58:53.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.567+0000 7fe377400700 1 -- 172.21.15.1:0/633469514 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe378004d20 con 0x5581d8e63670 2024-04-26T01:58:53.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.567+0000 7fe377400700 1 -- 172.21.15.1:0/633469514 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37809bf30 con 0x5581d8e63670 2024-04-26T01:58:53.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.568+0000 7fe38a800700 1 --2- 172.21.15.1:0/275964341 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe3700a3e30 0x7fe3700a62f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe384009790 tx=0x7fe384004ad0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:53.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.568+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/633469514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e63670 msgr2=0x5581d8e65a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:53.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.568+0000 7fe39ba2bb80 1 --2- 172.21.15.1:0/633469514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e63670 0x5581d8e65a60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe3780cdd80 tx=0x7fe3780988f0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.568+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/633469514 shutdown_connections 2024-04-26T01:58:53.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.568+0000 7fe39ba2bb80 1 --2- 172.21.15.1:0/633469514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e63670 0x5581d8e65a60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.568+0000 7fe39ba2bb80 1 --2- 172.21.15.1:0/633469514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e57ea0 0x5581d8e58280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.568+0000 7fe39ba2bb80 1 --2- 172.21.15.1:0/633469514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e587c0 0x5581d8e63130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.568+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/633469514 >> 172.21.15.1:0/633469514 conn(0x5581d8d87050 msgr2=0x5581d8e543f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:53.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.568+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/633469514 shutdown_connections 2024-04-26T01:58:53.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.568+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/633469514 wait complete. 2024-04-26T01:58:53.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.569+0000 7fe39ba2bb80 1 -- start start 2024-04-26T01:58:53.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.569+0000 7fe39ba2bb80 1 Processor -- start 2024-04-26T01:58:53.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.570+0000 7fe39ba2bb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e57ea0 0x5581d8e62df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:53.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.570+0000 7fe39ba2bb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e587c0 0x5581d8e60160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:53.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.570+0000 7fe39ba2bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e63670 0x5581d8e606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:53.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.570+0000 7fe39ba2bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5581d8e660a0 con 0x5581d8e63670 2024-04-26T01:58:53.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.570+0000 7fe39ba2bb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5581d8d64960 con 0x5581d8e587c0 2024-04-26T01:58:53.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.570+0000 7fe39ba2bb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5581d8e66220 con 0x5581d8e57ea0 2024-04-26T01:58:53.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.570+0000 7fe39ba2bb80 10 client.0 _ll_register_callbacks cb 0x5581d8d43820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:53.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.570+0000 7fe38a800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e63670 0x5581d8e606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:53.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.570+0000 7fe389e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e57ea0 0x5581d8e62df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:53.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.570+0000 7fe389e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e57ea0 0x5581d8e62df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:47354/0 (socket says 172.21.15.1:47354) 2024-04-26T01:58:53.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.570+0000 7fe389e00700 1 -- 172.21.15.1:0/1536504093 learned_addr learned my addr 172.21.15.1:0/1536504093 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:53.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.570+0000 7fe389400700 1 --2- 172.21.15.1:0/1536504093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e587c0 0x5581d8e60160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:53.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.571+0000 7fe389e00700 1 -- 172.21.15.1:0/1536504093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e587c0 msgr2=0x5581d8e60160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:53.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.571+0000 7fe389e00700 1 --2- 172.21.15.1:0/1536504093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e587c0 0x5581d8e60160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.571+0000 7fe389e00700 1 -- 172.21.15.1:0/1536504093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e63670 msgr2=0x5581d8e606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:53.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.571+0000 7fe389e00700 1 --2- 172.21.15.1:0/1536504093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e63670 0x5581d8e606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.571+0000 7fe389e00700 1 -- 172.21.15.1:0/1536504093 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d8cf89d0 con 0x5581d8e57ea0 2024-04-26T01:58:53.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.571+0000 7fe389400700 1 --2- 172.21.15.1:0/1536504093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e587c0 0x5581d8e60160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:53.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.571+0000 7fe38a800700 1 --2- 172.21.15.1:0/1536504093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e63670 0x5581d8e606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:53.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.571+0000 7fe389e00700 1 --2- 172.21.15.1:0/1536504093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e57ea0 0x5581d8e62df0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00bf20 tx=0x7fe37c00bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91930f10553c3a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:53.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.571+0000 7fe36be00700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:58:53.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.571+0000 7fe36be00700 1 -- 172.21.15.1:0/1536504093 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c003a60 con 0x5581d8e57ea0 2024-04-26T01:58:53.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.571+0000 7fe36be00700 1 -- 172.21.15.1:0/1536504093 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fe37c003c00 con 0x5581d8e57ea0 2024-04-26T01:58:53.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.571+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/1536504093 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5581d8db8780 con 0x5581d8e57ea0 2024-04-26T01:58:53.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.572+0000 7fe36be00700 1 -- 172.21.15.1:0/1536504093 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe37c003e00 con 0x5581d8e57ea0 2024-04-26T01:58:53.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.572+0000 7fe39ba2bb80 10 client.26065 fetch_fsmap learned FSMap version 32 2024-04-26T01:58:53.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.572+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/1536504093 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5581d8d41bf0 con 0x5581d8e57ea0 2024-04-26T01:58:53.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.572+0000 7fe36be00700 1 -- 172.21.15.1:0/1536504093 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5581d8db8780 con 0x5581d8e57ea0 2024-04-26T01:58:53.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.573+0000 7fe36be00700 1 -- 172.21.15.1:0/1536504093 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fe37c00f920 con 0x5581d8e57ea0 2024-04-26T01:58:53.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.573+0000 7fe39ba2bb80 10 client.26065 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:58:53.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.573+0000 7fe39ba2bb80 10 client.26065 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:58:53.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.573+0000 7fe39ba2bb80 20 client.26065 populate_metadata read hostname 'smithi001' 2024-04-26T01:58:53.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.573+0000 7fe39ba2bb80 1 --2- 172.21.15.1:0/1536504093 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5581d8e72eb0 0x5581d8e752e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:53.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.573+0000 7fe39ba2bb80 4 client.26065 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:58:53.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.573+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/1536504093 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581d8db2820 con 0x5581d8e72eb0 2024-04-26T01:58:53.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.573+0000 7fe389400700 1 --2- 172.21.15.1:0/1536504093 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5581d8e72eb0 0x5581d8e752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:53.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.574+0000 7fe389400700 1 --2- 172.21.15.1:0/1536504093 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5581d8e72eb0 0x5581d8e752e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c622367547c4f88b server_cookie=45e7a7773fa99bb3 in_seq=0 out_seq=0 2024-04-26T01:58:53.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.574+0000 7fe36be00700 10 client.26065 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:53.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.575+0000 7fe36be00700 1 -- 172.21.15.1:0/1536504093 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581d8db2820 con 0x5581d8e72eb0 2024-04-26T01:58:53.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.575+0000 7fe36be00700 10 client.26065 handle_command_reply: tid=0 2024-04-26T01:58:53.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.575+0000 7fe39ba2bb80 1 client.26065 shutdown 2024-04-26T01:58:53.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.575+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/1536504093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e57ea0 msgr2=0x5581d8e62df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:53.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.575+0000 7fe39ba2bb80 1 --2- 172.21.15.1:0/1536504093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e57ea0 0x5581d8e62df0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00bf20 tx=0x7fe37c00bf50 comp rx=0 tx=0).stop 2024-04-26T01:58:53.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.575+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/1536504093 shutdown_connections 2024-04-26T01:58:53.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.575+0000 7fe39ba2bb80 1 --2- 172.21.15.1:0/1536504093 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5581d8e72eb0 0x5581d8e752e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.575+0000 7fe39ba2bb80 1 --2- 172.21.15.1:0/1536504093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e63670 0x5581d8e606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.575+0000 7fe39ba2bb80 1 --2- 172.21.15.1:0/1536504093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e57ea0 0x5581d8e62df0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.575+0000 7fe39ba2bb80 1 --2- 172.21.15.1:0/1536504093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e587c0 0x5581d8e60160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.575+0000 7fe36be00700 0 client.26065 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:53.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.575+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/1536504093 >> 172.21.15.1:0/1536504093 conn(0x5581d8d87050 msgr2=0x5581d8e53fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:53.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.575+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/1536504093 shutdown_connections 2024-04-26T01:58:53.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.575+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/1536504093 wait complete. 2024-04-26T01:58:53.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.575+0000 7fe39ba2bb80 20 client.26065 trim_cache size 0 max 16384 2024-04-26T01:58:53.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.597+0000 7fe39ba2bb80 1 Processor -- start 2024-04-26T01:58:53.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.597+0000 7fe39ba2bb80 1 -- start start 2024-04-26T01:58:53.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.598+0000 7fe39ba2bb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e57ea0 0x5581d8e6efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:53.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.598+0000 7fe39ba2bb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e587c0 0x5581d8e6b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:53.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.598+0000 7fe39ba2bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e63670 0x5581d8e6ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:53.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.598+0000 7fe39ba2bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c1b5fc0 con 0x5581d8e63670 2024-04-26T01:58:53.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.598+0000 7fe39ba2bb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c1b3850 con 0x5581d8e587c0 2024-04-26T01:58:53.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.598+0000 7fe39ba2bb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c1b16c0 con 0x5581d8e57ea0 2024-04-26T01:58:53.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.598+0000 7fe38a800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e63670 0x5581d8e6ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:53.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.598+0000 7fe389400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e587c0 0x5581d8e6b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:53.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.598+0000 7fe389e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e57ea0 0x5581d8e6efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:53.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.598+0000 7fe389400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e587c0 0x5581d8e6b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:48654/0 (socket says 172.21.15.1:48654) 2024-04-26T01:58:53.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.598+0000 7fe389e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e57ea0 0x5581d8e6efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:47358/0 (socket says 172.21.15.1:47358) 2024-04-26T01:58:53.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.598+0000 7fe389e00700 1 -- 172.21.15.1:0/1586341041 learned_addr learned my addr 172.21.15.1:0/1586341041 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:53.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.598+0000 7fe38a800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e63670 0x5581d8e6ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37442/0 (socket says 172.21.15.1:37442) 2024-04-26T01:58:53.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.599+0000 7fe389e00700 1 -- 172.21.15.1:0/1586341041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e587c0 msgr2=0x5581d8e6b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:53.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.599+0000 7fe389e00700 1 --2- 172.21.15.1:0/1586341041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e587c0 0x5581d8e6b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.599+0000 7fe389e00700 1 -- 172.21.15.1:0/1586341041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e63670 msgr2=0x5581d8e6ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:53.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.599+0000 7fe389e00700 1 --2- 172.21.15.1:0/1586341041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e63670 0x5581d8e6ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.599+0000 7fe389e00700 1 -- 172.21.15.1:0/1586341041 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d8e53100 con 0x5581d8e57ea0 2024-04-26T01:58:53.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.599+0000 7fe389400700 1 --2- 172.21.15.1:0/1586341041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e587c0 0x5581d8e6b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:53.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.599+0000 7fe389e00700 1 --2- 172.21.15.1:0/1586341041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e57ea0 0x5581d8e6efc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009ce0 tx=0x7fe37c00b6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e102e315050d1c03 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:53.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.599+0000 7fe376a00700 1 -- 172.21.15.1:0/1586341041 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c023440 con 0x5581d8e57ea0 2024-04-26T01:58:53.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.599+0000 7fe376a00700 1 -- 172.21.15.1:0/1586341041 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe37c0050e0 con 0x5581d8e57ea0 2024-04-26T01:58:53.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.600+0000 7fe376a00700 1 -- 172.21.15.1:0/1586341041 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c00f730 con 0x5581d8e57ea0 2024-04-26T01:58:53.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.600+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/1586341041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e57ea0 msgr2=0x5581d8e6efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:53.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.600+0000 7fe39ba2bb80 1 --2- 172.21.15.1:0/1586341041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e57ea0 0x5581d8e6efc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009ce0 tx=0x7fe37c00b6f0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.600+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/1586341041 shutdown_connections 2024-04-26T01:58:53.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.600+0000 7fe39ba2bb80 1 --2- 172.21.15.1:0/1586341041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e63670 0x5581d8e6ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.600+0000 7fe39ba2bb80 1 --2- 172.21.15.1:0/1586341041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e57ea0 0x5581d8e6efc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.600+0000 7fe39ba2bb80 1 --2- 172.21.15.1:0/1586341041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e587c0 0x5581d8e6b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.600+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/1586341041 >> 172.21.15.1:0/1586341041 conn(0x5581d8d87050 msgr2=0x5581d8e53560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:53.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.600+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/1586341041 shutdown_connections 2024-04-26T01:58:53.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.600+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/1586341041 wait complete. 2024-04-26T01:58:53.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.601+0000 7fe39ba2bb80 1 -- start start 2024-04-26T01:58:53.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.601+0000 7fe39ba2bb80 1 Processor -- start 2024-04-26T01:58:53.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.601+0000 7fe39ba2bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e57ea0 0x5581d8e6b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:53.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.601+0000 7fe39ba2bb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e587c0 0x5581d8e6b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:53.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.602+0000 7fe39ba2bb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e63670 0x5581d8e69610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:53.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.602+0000 7fe39ba2bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c1b1d10 con 0x5581d8e57ea0 2024-04-26T01:58:53.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.602+0000 7fe39ba2bb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c1b3180 con 0x5581d8e63670 2024-04-26T01:58:53.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.602+0000 7fe39ba2bb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c1b23e0 con 0x5581d8e587c0 2024-04-26T01:58:53.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.602+0000 7fe39ba2bb80 10 client.0 _ll_register_callbacks cb 0x5581d8d43820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:53.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.602+0000 7fe389e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e57ea0 0x5581d8e6b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:53.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.602+0000 7fe38a800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e63670 0x5581d8e69610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:53.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.602+0000 7fe389e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e57ea0 0x5581d8e6b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37450/0 (socket says 172.21.15.1:37450) 2024-04-26T01:58:53.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.602+0000 7fe389e00700 1 -- 172.21.15.1:0/3999425201 learned_addr learned my addr 172.21.15.1:0/3999425201 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:58:53.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.602+0000 7fe389400700 1 --2- 172.21.15.1:0/3999425201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e587c0 0x5581d8e6b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:53.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.602+0000 7fe38a800700 1 -- 172.21.15.1:0/3999425201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e587c0 msgr2=0x5581d8e6b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:53.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.602+0000 7fe38a800700 1 --2- 172.21.15.1:0/3999425201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e587c0 0x5581d8e6b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.602+0000 7fe38a800700 1 -- 172.21.15.1:0/3999425201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e57ea0 msgr2=0x5581d8e6b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:53.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.602+0000 7fe38a800700 1 --2- 172.21.15.1:0/3999425201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e57ea0 0x5581d8e6b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.602+0000 7fe38a800700 1 -- 172.21.15.1:0/3999425201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d8e69d30 con 0x5581d8e63670 2024-04-26T01:58:53.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.603+0000 7fe38a800700 1 --2- 172.21.15.1:0/3999425201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e63670 0x5581d8e69610 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe384009fd0 tx=0x7fe384009130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f4dd88f2253d2cf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:53.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.603+0000 7fe36be00700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:58:53.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.603+0000 7fe36be00700 1 -- 172.21.15.1:0/3999425201 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384017e20 con 0x5581d8e63670 2024-04-26T01:58:53.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.603+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/3999425201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe37c0050e0 con 0x5581d8e63670 2024-04-26T01:58:53.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.603+0000 7fe36be00700 1 -- 172.21.15.1:0/3999425201 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fe384007420 con 0x5581d8e63670 2024-04-26T01:58:53.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.603+0000 7fe36be00700 1 -- 172.21.15.1:0/3999425201 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe384007620 con 0x5581d8e63670 2024-04-26T01:58:53.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.603+0000 7fe39ba2bb80 10 client.26054 fetch_fsmap learned FSMap version 32 2024-04-26T01:58:53.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.603+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/3999425201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5581d8ed9f00 con 0x5581d8e63670 2024-04-26T01:58:53.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.604+0000 7fe36be00700 1 -- 172.21.15.1:0/3999425201 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe37c0050e0 con 0x5581d8e63670 2024-04-26T01:58:53.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.604+0000 7fe36be00700 1 -- 172.21.15.1:0/3999425201 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fe38402e820 con 0x5581d8e63670 2024-04-26T01:58:53.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.604+0000 7fe39ba2bb80 10 client.26054 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:58:53.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.604+0000 7fe39ba2bb80 10 client.26054 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:58:53.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.604+0000 7fe39ba2bb80 20 client.26054 populate_metadata read hostname 'smithi001' 2024-04-26T01:58:53.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.604+0000 7fe39ba2bb80 1 --2- 172.21.15.1:0/3999425201 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5581d8eda660 0x5581d8edca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:53.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.604+0000 7fe39ba2bb80 4 client.26054 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2f800756", "release": true, "await": true}] 2024-04-26T01:58:53.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.604+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/3999425201 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2f800756", "release": true, "await": true}) -- 0x5581d8c2b350 con 0x5581d8eda660 2024-04-26T01:58:53.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.604+0000 7fe389e00700 1 --2- 172.21.15.1:0/3999425201 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5581d8eda660 0x5581d8edca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:53.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.605+0000 7fe389e00700 1 --2- 172.21.15.1:0/3999425201 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5581d8eda660 0x5581d8edca80 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6989ac19f4dec204 server_cookie=b2334ebdc422c0e7 in_seq=0 out_seq=0 2024-04-26T01:58:53.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.605+0000 7fe36be00700 10 client.26054 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:53.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.795+0000 7fe36be00700 1 -- 172.21.15.1:0/3999425201 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5581d8c2b350 con 0x5581d8eda660 2024-04-26T01:58:53.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.795+0000 7fe36be00700 10 client.26054 handle_command_reply: tid=0 2024-04-26T01:58:53.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.795+0000 7fe39ba2bb80 1 client.26054 shutdown 2024-04-26T01:58:53.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.795+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/3999425201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e63670 msgr2=0x5581d8e69610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:53.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.795+0000 7fe39ba2bb80 1 --2- 172.21.15.1:0/3999425201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e63670 0x5581d8e69610 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe384009fd0 tx=0x7fe384009130 comp rx=0 tx=0).stop 2024-04-26T01:58:53.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.796+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/3999425201 shutdown_connections 2024-04-26T01:58:53.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.796+0000 7fe39ba2bb80 1 --2- 172.21.15.1:0/3999425201 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5581d8eda660 0x5581d8edca80 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.796+0000 7fe39ba2bb80 1 --2- 172.21.15.1:0/3999425201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581d8e63670 0x5581d8e69610 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.796+0000 7fe39ba2bb80 1 --2- 172.21.15.1:0/3999425201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581d8e57ea0 0x5581d8e6b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.796+0000 7fe39ba2bb80 1 --2- 172.21.15.1:0/3999425201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5581d8e587c0 0x5581d8e6b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.796+0000 7fe36be00700 0 client.26054 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:58:53.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.796+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/3999425201 >> 172.21.15.1:0/3999425201 conn(0x5581d8d87050 msgr2=0x5581d8e53df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:53.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.797+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/3999425201 shutdown_connections 2024-04-26T01:58:53.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.797+0000 7fe39ba2bb80 1 -- 172.21.15.1:0/3999425201 wait complete. 2024-04-26T01:58:53.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.797+0000 7fe39ba2bb80 20 client.26054 trim_cache size 0 max 16384 2024-04-26T01:58:53.797 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T01:58:53.798 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 32, 2024-04-26T01:58:53.798 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24518, 2024-04-26T01:58:53.798 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 20, 2024-04-26T01:58:53.798 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-04-26T01:58:53.798 INFO:teuthology.orchestra.run.smithi001.stdout: "2f800756": { 2024-04-26T01:58:53.798 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 20, 2024-04-26T01:58:53.798 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-04-26T01:58:53.798 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:58:53.798 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-04-26T01:58:53.798 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:58:53.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T01:58:53.798 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-04-26T01:58:53.798 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 75.9, 2024-04-26T01:58:53.799 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-04-26T01:58:53.799 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-04-26T01:58:53.799 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-04-26T01:58:53.799 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:58:53.799 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-04-26T01:58:53.799 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:58:53.799 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:58:53.799 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:58:53.799 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:58:53.799 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:58:53.799 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:58:53.799 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T01:58:53.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.798+0000 7fe38b200700 1 -- 172.21.15.1:0/275964341 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe3700a3e30 msgr2=0x7fe3700a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:53.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.798+0000 7fe38b200700 1 --2- 172.21.15.1:0/275964341 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe3700a3e30 0x7fe3700a62f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe384009790 tx=0x7fe384004ad0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.798+0000 7fe38b200700 1 -- 172.21.15.1:0/275964341 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c077890 msgr2=0x7fe38c19fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:53.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.798+0000 7fe38b200700 1 --2- 172.21.15.1:0/275964341 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c077890 0x7fe38c19fbf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe37800d980 tx=0x7fe37800dc90 comp rx=0 tx=0).stop 2024-04-26T01:58:53.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.799+0000 7fe38b200700 1 -- 172.21.15.1:0/275964341 shutdown_connections 2024-04-26T01:58:53.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.799+0000 7fe38b200700 1 --2- 172.21.15.1:0/275964341 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe3700a3e30 0x7fe3700a62f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.799+0000 7fe38b200700 1 --2- 172.21.15.1:0/275964341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe38c1137f0 0x7fe38c1a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.799+0000 7fe38b200700 1 --2- 172.21.15.1:0/275964341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe38c076f70 0x7fe38c19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.799+0000 7fe38b200700 1 --2- 172.21.15.1:0/275964341 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c077890 0x7fe38c19fbf0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:53.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.799+0000 7fe38b200700 1 -- 172.21.15.1:0/275964341 >> 172.21.15.1:0/275964341 conn(0x7fe38c100680 msgr2=0x7fe38c114460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:53.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.799+0000 7fe38b200700 1 -- 172.21.15.1:0/275964341 shutdown_connections 2024-04-26T01:58:53.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:58:53.799+0000 7fe38b200700 1 -- 172.21.15.1:0/275964341 wait complete. 2024-04-26T01:58:54.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:53 smithi001 ceph-mon[27822]: pgmap v223: 97 pgs: 97 active+clean; 49 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-04-26T01:58:54.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:53 smithi097 ceph-mon[32853]: pgmap v223: 97 pgs: 97 active+clean; 49 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-04-26T01:58:54.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:53 smithi173 ceph-mon[32553]: pgmap v223: 97 pgs: 97 active+clean; 49 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-04-26T01:58:54.671 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2f800756', seconds elapsed: 32.8 2024-04-26T01:58:56.101 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:55 smithi001 ceph-mon[27822]: pgmap v224: 97 pgs: 97 active+clean; 50 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 161 KiB/s wr, 32 op/s 2024-04-26T01:58:56.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:55 smithi097 ceph-mon[32853]: pgmap v224: 97 pgs: 97 active+clean; 50 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 161 KiB/s wr, 32 op/s 2024-04-26T01:58:56.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:55 smithi173 ceph-mon[32553]: pgmap v224: 97 pgs: 97 active+clean; 50 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 161 KiB/s wr, 32 op/s 2024-04-26T01:58:58.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:57 smithi097 ceph-mon[32853]: pgmap v225: 97 pgs: 97 active+clean; 50 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 161 KiB/s wr, 29 op/s 2024-04-26T01:58:58.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:57 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:58:58.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:57 smithi173 ceph-mon[32553]: pgmap v225: 97 pgs: 97 active+clean; 50 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 161 KiB/s wr, 29 op/s 2024-04-26T01:58:58.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:57 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:58:58.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:57 smithi001 ceph-mon[27822]: pgmap v225: 97 pgs: 97 active+clean; 50 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 161 KiB/s wr, 29 op/s 2024-04-26T01:58:58.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:57 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:58:59.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:58 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:58:59.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:58 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:58:59.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:58 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:58:59.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:58 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:58:59.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:58 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:58:59.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:58 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:58:59.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:58 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:58:59.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:58 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:58:59.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:58 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T01:59:00.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:58:59 smithi097 ceph-mon[32853]: pgmap v226: 97 pgs: 97 active+clean; 50 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 161 KiB/s wr, 29 op/s 2024-04-26T01:59:00.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:58:59 smithi173 ceph-mon[32553]: pgmap v226: 97 pgs: 97 active+clean; 50 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 161 KiB/s wr, 29 op/s 2024-04-26T01:59:00.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:58:59 smithi001 ceph-mon[27822]: pgmap v226: 97 pgs: 97 active+clean; 50 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 161 KiB/s wr, 29 op/s 2024-04-26T01:59:02.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:01 smithi097 ceph-mon[32853]: pgmap v227: 97 pgs: 97 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 975 KiB/s wr, 157 op/s 2024-04-26T01:59:02.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:01 smithi173 ceph-mon[32553]: pgmap v227: 97 pgs: 97 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 975 KiB/s wr, 157 op/s 2024-04-26T01:59:02.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:01 smithi001 ceph-mon[27822]: pgmap v227: 97 pgs: 97 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 975 KiB/s wr, 157 op/s 2024-04-26T01:59:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.938+0000 7efee6200640 1 -- 172.21.15.1:0/3576864141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efee0103ae0 msgr2=0x7efee010bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.938+0000 7efee6200640 1 --2- 172.21.15.1:0/3576864141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efee0103ae0 0x7efee010bfc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efec8009900 tx=0x7efec802a480 comp rx=0 tx=0).stop 2024-04-26T01:59:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.938+0000 7efee6200640 1 -- 172.21.15.1:0/3576864141 shutdown_connections 2024-04-26T01:59:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.938+0000 7efee6200640 1 --2- 172.21.15.1:0/3576864141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee010c590 0x7efee010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.938+0000 7efee6200640 1 --2- 172.21.15.1:0/3576864141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efee0103ae0 0x7efee010bfc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.938+0000 7efee6200640 1 --2- 172.21.15.1:0/3576864141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efee01031c0 0x7efee01035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.938+0000 7efee6200640 1 -- 172.21.15.1:0/3576864141 >> 172.21.15.1:0/3576864141 conn(0x7efee0077f70 msgr2=0x7efee0078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.938+0000 7efee6200640 1 -- 172.21.15.1:0/3576864141 shutdown_connections 2024-04-26T01:59:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.938+0000 7efee6200640 1 -- 172.21.15.1:0/3576864141 wait complete. 2024-04-26T01:59:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.940+0000 7efee6200640 1 Processor -- start 2024-04-26T01:59:03.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.940+0000 7efee6200640 1 -- start start 2024-04-26T01:59:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.941+0000 7efee6200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee01031c0 0x7efee01a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.941+0000 7efee6200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efee0103ae0 0x7efee01a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.941+0000 7efee6200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efee010c590 0x7efee019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.941+0000 7efee6200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efee0112200 con 0x7efee010c590 2024-04-26T01:59:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.941+0000 7efee6200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efee0112080 con 0x7efee01031c0 2024-04-26T01:59:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.941+0000 7efee6200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efee0112380 con 0x7efee0103ae0 2024-04-26T01:59:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.941+0000 7efedf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee01031c0 0x7efee01a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.941+0000 7efed6a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efee0103ae0 0x7efee01a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.941+0000 7efedf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee01031c0 0x7efee01a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:52092/0 (socket says 172.21.15.1:52092) 2024-04-26T01:59:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.941+0000 7efedf400640 1 -- 172.21.15.1:0/803343418 learned_addr learned my addr 172.21.15.1:0/803343418 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.941+0000 7efedfe00640 1 --2- 172.21.15.1:0/803343418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efee010c590 0x7efee019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.942+0000 7efed6a00640 1 -- 172.21.15.1:0/803343418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee01031c0 msgr2=0x7efee01a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.942+0000 7efed6a00640 1 --2- 172.21.15.1:0/803343418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee01031c0 0x7efee01a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.942+0000 7efed6a00640 1 -- 172.21.15.1:0/803343418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efee010c590 msgr2=0x7efee019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.942+0000 7efed6a00640 1 --2- 172.21.15.1:0/803343418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efee010c590 0x7efee019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.942+0000 7efed6a00640 1 -- 172.21.15.1:0/803343418 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee019af90 con 0x7efee0103ae0 2024-04-26T01:59:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.942+0000 7efedfe00640 1 --2- 172.21.15.1:0/803343418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efee010c590 0x7efee019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.942+0000 7efedf400640 1 --2- 172.21.15.1:0/803343418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee01031c0 0x7efee01a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.942+0000 7efed6a00640 1 --2- 172.21.15.1:0/803343418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efee0103ae0 0x7efee01a0b90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efec802aa70 tx=0x7efec8003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.942+0000 7efedcc00640 1 -- 172.21.15.1:0/803343418 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec803f070 con 0x7efee0103ae0 2024-04-26T01:59:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.942+0000 7efee6200640 1 -- 172.21.15.1:0/803343418 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee019b220 con 0x7efee0103ae0 2024-04-26T01:59:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.942+0000 7efedcc00640 1 -- 172.21.15.1:0/803343418 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efec8031b50 con 0x7efee0103ae0 2024-04-26T01:59:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.943+0000 7efedcc00640 1 -- 172.21.15.1:0/803343418 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8035030 con 0x7efee0103ae0 2024-04-26T01:59:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.943+0000 7efee6200640 1 -- 172.21.15.1:0/803343418 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee01a74a0 con 0x7efee0103ae0 2024-04-26T01:59:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.946+0000 7efedcc00640 1 -- 172.21.15.1:0/803343418 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7efec8042070 con 0x7efee0103ae0 2024-04-26T01:59:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.947+0000 7efee9036740 1 Processor -- start 2024-04-26T01:59:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.947+0000 7efee9036740 1 -- start start 2024-04-26T01:59:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.947+0000 7efedcc00640 1 --2- 172.21.15.1:0/803343418 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efec40a3d10 0x7efec40a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.947+0000 7efee9036740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b0059f820 0x563b0059fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.947+0000 7efee9036740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005a0140 0x563b005aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.947+0000 7efedcc00640 1 -- 172.21.15.1:0/803343418 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7efec80f0e30 con 0x7efee0103ae0 2024-04-26T01:59:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.947+0000 7efee9036740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b005aaa00 0x563b005acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.948+0000 7efee9036740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563b00505900 con 0x563b005aaa00 2024-04-26T01:59:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.948+0000 7efee9036740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563b004abef0 con 0x563b0059f820 2024-04-26T01:59:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.948+0000 7efee9036740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563b005018c0 con 0x563b005a0140 2024-04-26T01:59:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.948+0000 7efed6a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005a0140 0x563b005aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.948+0000 7efed6a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005a0140 0x563b005aa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:37628/0 (socket says 172.21.15.1:37628) 2024-04-26T01:59:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.948+0000 7efedf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b0059f820 0x563b0059fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.948+0000 7efed6a00640 1 -- 172.21.15.1:0/2032963934 learned_addr learned my addr 172.21.15.1:0/2032963934 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.948+0000 7efedfe00640 1 --2- 172.21.15.1:0/2032963934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b005aaa00 0x563b005acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.948+0000 7efedfe00640 1 --2- 172.21.15.1:0/803343418 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efec40a3d10 0x7efec40a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.948+0000 7efed6a00640 1 -- 172.21.15.1:0/2032963934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b0059f820 msgr2=0x563b0059fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.948+0000 7efed6a00640 1 --2- 172.21.15.1:0/2032963934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b0059f820 0x563b0059fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.948+0000 7efed6a00640 1 -- 172.21.15.1:0/2032963934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b005aaa00 msgr2=0x563b005acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.948+0000 7efed6a00640 1 --2- 172.21.15.1:0/2032963934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b005aaa00 0x563b005acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.948+0000 7efed6a00640 1 -- 172.21.15.1:0/2032963934 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b00218210 con 0x563b005a0140 2024-04-26T01:59:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.949+0000 7efed6a00640 1 --2- 172.21.15.1:0/2032963934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005a0140 0x563b005aa4c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efec80b9710 tx=0x7efec8031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82a31167b50d4ff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.949+0000 7efedfe00640 1 --2- 172.21.15.1:0/803343418 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efec40a3d10 0x7efec40a61d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x563b0059d150 tx=0x7efed8005f90 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.949+0000 7efecfe00640 1 -- 172.21.15.1:0/2032963934 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8049070 con 0x563b005a0140 2024-04-26T01:59:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.949+0000 7efecfe00640 1 -- 172.21.15.1:0/2032963934 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efec80bcd40 con 0x563b005a0140 2024-04-26T01:59:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.949+0000 7efecfe00640 1 -- 172.21.15.1:0/2032963934 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec80b93b0 con 0x563b005a0140 2024-04-26T01:59:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.950+0000 7efee9036740 1 -- 172.21.15.1:0/2032963934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005a0140 msgr2=0x563b005aa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.950+0000 7efee9036740 1 --2- 172.21.15.1:0/2032963934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005a0140 0x563b005aa4c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efec80b9710 tx=0x7efec8031400 comp rx=0 tx=0).stop 2024-04-26T01:59:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.950+0000 7efee9036740 1 -- 172.21.15.1:0/2032963934 shutdown_connections 2024-04-26T01:59:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.950+0000 7efee9036740 1 --2- 172.21.15.1:0/2032963934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b005aaa00 0x563b005acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.950+0000 7efee9036740 1 --2- 172.21.15.1:0/2032963934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005a0140 0x563b005aa4c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.950+0000 7efee9036740 1 --2- 172.21.15.1:0/2032963934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b0059f820 0x563b0059fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.950+0000 7efee9036740 1 -- 172.21.15.1:0/2032963934 >> 172.21.15.1:0/2032963934 conn(0x563b0059bd30 msgr2=0x563b0059c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.950+0000 7efee9036740 1 -- 172.21.15.1:0/2032963934 shutdown_connections 2024-04-26T01:59:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.950+0000 7efee9036740 1 -- 172.21.15.1:0/2032963934 wait complete. 2024-04-26T01:59:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.951+0000 7efee9036740 1 -- start start 2024-04-26T01:59:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.951+0000 7efee9036740 1 Processor -- start 2024-04-26T01:59:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.951+0000 7efee9036740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b0059f820 0x563b005a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.951+0000 7efee9036740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b005a0140 0x563b005a7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.951+0000 7efee9036740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005aaa00 0x563b005a8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.951+0000 7efee9036740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563b005ad3f0 con 0x563b0059f820 2024-04-26T01:59:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.951+0000 7efee9036740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563b004bf310 con 0x563b005a0140 2024-04-26T01:59:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.951+0000 7efee9036740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563b005ad570 con 0x563b005aaa00 2024-04-26T01:59:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.951+0000 7efee9036740 10 client.0 _ll_register_callbacks cb 0x563b0048dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.952+0000 7efedf400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b0059f820 0x563b005a79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.952+0000 7efed6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b005a0140 0x563b005a7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.952+0000 7efedf400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b0059f820 0x563b005a79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50784/0 (socket says 172.21.15.1:50784) 2024-04-26T01:59:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.952+0000 7efedfe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005aaa00 0x563b005a8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.952+0000 7efedf400640 1 -- 172.21.15.1:0/3875991190 learned_addr learned my addr 172.21.15.1:0/3875991190 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.952+0000 7efed6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b005a0140 0x563b005a7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:52106/0 (socket says 172.21.15.1:52106) 2024-04-26T01:59:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.952+0000 7efedf400640 1 -- 172.21.15.1:0/3875991190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005aaa00 msgr2=0x563b005a8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.952+0000 7efedf400640 1 --2- 172.21.15.1:0/3875991190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005aaa00 0x563b005a8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.952+0000 7efedf400640 1 -- 172.21.15.1:0/3875991190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b005a0140 msgr2=0x563b005a7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.952+0000 7efedf400640 1 --2- 172.21.15.1:0/3875991190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b005a0140 0x563b005a7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.952+0000 7efedf400640 1 -- 172.21.15.1:0/3875991190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b002e4620 con 0x563b0059f820 2024-04-26T01:59:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.952+0000 7efed6a00640 1 --2- 172.21.15.1:0/3875991190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b005a0140 0x563b005a7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.952+0000 7efedf400640 1 --2- 172.21.15.1:0/3875991190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b0059f820 0x563b005a79f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efed000ff90 tx=0x7efed00128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333f1e431cbbc03b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.952+0000 7efeccc00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:59:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.953+0000 7efeccc00640 1 -- 172.21.15.1:0/3875991190 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0010430 con 0x563b0059f820 2024-04-26T01:59:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.953+0000 7efeccc00640 1 -- 172.21.15.1:0/3875991190 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7efed00105d0 con 0x563b0059f820 2024-04-26T01:59:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.953+0000 7efee9036740 1 -- 172.21.15.1:0/3875991190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efec80bcd40 con 0x563b0059f820 2024-04-26T01:59:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.953+0000 7efeccc00640 1 -- 172.21.15.1:0/3875991190 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efed00107d0 con 0x563b0059f820 2024-04-26T01:59:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.953+0000 7efee9036740 10 client.15912 fetch_fsmap learned FSMap version 32 2024-04-26T01:59:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.953+0000 7efee9036740 1 -- 172.21.15.1:0/3875991190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563b005b9640 con 0x563b0059f820 2024-04-26T01:59:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.953+0000 7efeccc00640 1 -- 172.21.15.1:0/3875991190 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efec80bcd40 con 0x563b0059f820 2024-04-26T01:59:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.954+0000 7efeccc00640 1 -- 172.21.15.1:0/3875991190 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7efed00249a0 con 0x563b0059f820 2024-04-26T01:59:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.954+0000 7efee9036740 10 client.15912 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:59:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.954+0000 7efee9036740 10 client.15912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:59:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.954+0000 7efee9036740 20 client.15912 populate_metadata read hostname 'smithi001' 2024-04-26T01:59:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.954+0000 7efee9036740 1 --2- 172.21.15.1:0/3875991190 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563b005bab50 0x563b005bcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.954+0000 7efee9036740 4 client.15912 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:59:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.954+0000 7efee9036740 1 -- 172.21.15.1:0/3875991190 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b005198b0 con 0x563b005bab50 2024-04-26T01:59:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.954+0000 7efed6a00640 1 --2- 172.21.15.1:0/3875991190 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563b005bab50 0x563b005bcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.955+0000 7efed6a00640 1 --2- 172.21.15.1:0/3875991190 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563b005bab50 0x563b005bcf80 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ce5ccd052477d3 server_cookie=d9bc4b7b4b117408 in_seq=0 out_seq=0 2024-04-26T01:59:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.955+0000 7efeccc00640 10 client.15912 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.956+0000 7efeccc00640 1 -- 172.21.15.1:0/3875991190 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b005198b0 con 0x563b005bab50 2024-04-26T01:59:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.956+0000 7efeccc00640 10 client.15912 handle_command_reply: tid=0 2024-04-26T01:59:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.956+0000 7efee9036740 1 client.15912 shutdown 2024-04-26T01:59:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.956+0000 7efee9036740 1 -- 172.21.15.1:0/3875991190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b0059f820 msgr2=0x563b005a79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.956+0000 7efee9036740 1 --2- 172.21.15.1:0/3875991190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b0059f820 0x563b005a79f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efed000ff90 tx=0x7efed00128e0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.956+0000 7efee9036740 1 -- 172.21.15.1:0/3875991190 shutdown_connections 2024-04-26T01:59:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.956+0000 7efee9036740 1 --2- 172.21.15.1:0/3875991190 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563b005bab50 0x563b005bcf80 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.956+0000 7efee9036740 1 --2- 172.21.15.1:0/3875991190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005aaa00 0x563b005a8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.956+0000 7efee9036740 1 --2- 172.21.15.1:0/3875991190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b005a0140 0x563b005a7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.956+0000 7efee9036740 1 --2- 172.21.15.1:0/3875991190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b0059f820 0x563b005a79f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.956+0000 7efee9036740 1 -- 172.21.15.1:0/3875991190 >> 172.21.15.1:0/3875991190 conn(0x563b0059bd30 msgr2=0x563b005a1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.956+0000 7efeccc00640 0 client.15912 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.956+0000 7efee9036740 1 -- 172.21.15.1:0/3875991190 shutdown_connections 2024-04-26T01:59:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.956+0000 7efee9036740 1 -- 172.21.15.1:0/3875991190 wait complete. 2024-04-26T01:59:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.956+0000 7efee9036740 20 client.15912 trim_cache size 0 max 16384 2024-04-26T01:59:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.976+0000 7efee9036740 1 Processor -- start 2024-04-26T01:59:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.977+0000 7efee9036740 1 -- start start 2024-04-26T01:59:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.977+0000 7efee9036740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b0059f820 0x563b005b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.977+0000 7efee9036740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005a0140 0x563b0059b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.977+0000 7efee9036740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b005aaa00 0x563b005c2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.977+0000 7efee9036740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efee01b27d0 con 0x563b0059f820 2024-04-26T01:59:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.977+0000 7efee9036740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efee01b00a0 con 0x563b005aaa00 2024-04-26T01:59:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.977+0000 7efee9036740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efee01adf10 con 0x563b005a0140 2024-04-26T01:59:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.977+0000 7efed6a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005a0140 0x563b0059b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.977+0000 7efed6a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005a0140 0x563b0059b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:37656/0 (socket says 172.21.15.1:37656) 2024-04-26T01:59:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.977+0000 7efed6a00640 1 -- 172.21.15.1:0/3507703456 learned_addr learned my addr 172.21.15.1:0/3507703456 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.977+0000 7efedfe00640 1 --2- 172.21.15.1:0/3507703456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b005aaa00 0x563b005c2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.977+0000 7efedf400640 1 --2- 172.21.15.1:0/3507703456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b0059f820 0x563b005b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.978+0000 7efed6a00640 1 -- 172.21.15.1:0/3507703456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b005aaa00 msgr2=0x563b005c2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.978+0000 7efed6a00640 1 --2- 172.21.15.1:0/3507703456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b005aaa00 0x563b005c2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.978+0000 7efed6a00640 1 -- 172.21.15.1:0/3507703456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b0059f820 msgr2=0x563b005b6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.978+0000 7efed6a00640 1 --2- 172.21.15.1:0/3507703456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b0059f820 0x563b005b6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.978+0000 7efed6a00640 1 -- 172.21.15.1:0/3507703456 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b00597470 con 0x563b005a0140 2024-04-26T01:59:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.978+0000 7efed6a00640 1 --2- 172.21.15.1:0/3507703456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005a0140 0x563b0059b610 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efec80bca40 tx=0x7efec8031e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cd99e5e72f2e488 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.978+0000 7efecf400640 1 -- 172.21.15.1:0/3507703456 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec80da070 con 0x563b005a0140 2024-04-26T01:59:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.978+0000 7efecf400640 1 -- 172.21.15.1:0/3507703456 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efec80bade0 con 0x563b005a0140 2024-04-26T01:59:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.979+0000 7efecf400640 1 -- 172.21.15.1:0/3507703456 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8049070 con 0x563b005a0140 2024-04-26T01:59:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.979+0000 7efee9036740 1 -- 172.21.15.1:0/3507703456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005a0140 msgr2=0x563b0059b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.979+0000 7efee9036740 1 --2- 172.21.15.1:0/3507703456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005a0140 0x563b0059b610 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efec80bca40 tx=0x7efec8031e40 comp rx=0 tx=0).stop 2024-04-26T01:59:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.979+0000 7efee9036740 1 -- 172.21.15.1:0/3507703456 shutdown_connections 2024-04-26T01:59:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.979+0000 7efee9036740 1 --2- 172.21.15.1:0/3507703456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b005aaa00 0x563b005c2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.979+0000 7efee9036740 1 --2- 172.21.15.1:0/3507703456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005a0140 0x563b0059b610 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.979+0000 7efee9036740 1 --2- 172.21.15.1:0/3507703456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b0059f820 0x563b005b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.979+0000 7efee9036740 1 -- 172.21.15.1:0/3507703456 >> 172.21.15.1:0/3507703456 conn(0x563b0059bd30 msgr2=0x563b005a1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.980+0000 7efee9036740 1 -- 172.21.15.1:0/3507703456 shutdown_connections 2024-04-26T01:59:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.980+0000 7efee9036740 1 -- 172.21.15.1:0/3507703456 wait complete. 2024-04-26T01:59:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.980+0000 7efee9036740 1 -- start start 2024-04-26T01:59:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.980+0000 7efee9036740 1 Processor -- start 2024-04-26T01:59:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.981+0000 7efee9036740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b0059f820 0x563b005b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.981+0000 7efee9036740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b005a0140 0x563b005afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.981+0000 7efee9036740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005aaa00 0x563b005b0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.981+0000 7efee9036740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efee01ae560 con 0x563b005a0140 2024-04-26T01:59:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.981+0000 7efee9036740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efee01af9d0 con 0x563b0059f820 2024-04-26T01:59:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.981+0000 7efee9036740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efee01aec30 con 0x563b005aaa00 2024-04-26T01:59:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.981+0000 7efee9036740 10 client.0 _ll_register_callbacks cb 0x7efed000fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.981+0000 7efedf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b0059f820 0x563b005b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.981+0000 7efedf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b0059f820 0x563b005b27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:52128/0 (socket says 172.21.15.1:52128) 2024-04-26T01:59:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.981+0000 7efedf400640 1 -- 172.21.15.1:0/3798244864 learned_addr learned my addr 172.21.15.1:0/3798244864 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.981+0000 7efedfe00640 1 --2- 172.21.15.1:0/3798244864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005aaa00 0x563b005b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.981+0000 7efed6a00640 1 --2- 172.21.15.1:0/3798244864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b005a0140 0x563b005afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.981+0000 7efedf400640 1 -- 172.21.15.1:0/3798244864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005aaa00 msgr2=0x563b005b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.981+0000 7efedf400640 1 --2- 172.21.15.1:0/3798244864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005aaa00 0x563b005b0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.981+0000 7efedf400640 1 -- 172.21.15.1:0/3798244864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b005a0140 msgr2=0x563b005afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.981+0000 7efedf400640 1 --2- 172.21.15.1:0/3798244864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b005a0140 0x563b005afbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.981+0000 7efedf400640 1 -- 172.21.15.1:0/3798244864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b005b06d0 con 0x563b0059f820 2024-04-26T01:59:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.981+0000 7efedf400640 1 --2- 172.21.15.1:0/3798244864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b0059f820 0x563b005b27c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efed000f490 tx=0x7efed00106e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9e61a467bf877c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.981+0000 7efeccc00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:59:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.982+0000 7efeccc00640 1 -- 172.21.15.1:0/3798244864 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0010b80 con 0x563b0059f820 2024-04-26T01:59:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.982+0000 7efee9036740 1 -- 172.21.15.1:0/3798244864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efec80bade0 con 0x563b0059f820 2024-04-26T01:59:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.982+0000 7efeccc00640 1 -- 172.21.15.1:0/3798244864 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7efed0010d20 con 0x563b0059f820 2024-04-26T01:59:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.982+0000 7efeccc00640 1 -- 172.21.15.1:0/3798244864 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efed0003970 con 0x563b0059f820 2024-04-26T01:59:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.982+0000 7efee9036740 10 client.26072 fetch_fsmap learned FSMap version 32 2024-04-26T01:59:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.982+0000 7efee9036740 1 -- 172.21.15.1:0/3798244864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563b00620ed0 con 0x563b0059f820 2024-04-26T01:59:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.982+0000 7efeccc00640 1 -- 172.21.15.1:0/3798244864 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efec80bade0 con 0x563b0059f820 2024-04-26T01:59:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.982+0000 7efeccc00640 1 -- 172.21.15.1:0/3798244864 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7efed00247b0 con 0x563b0059f820 2024-04-26T01:59:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.982+0000 7efee9036740 10 client.26072 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:59:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.982+0000 7efee9036740 10 client.26072 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:59:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.982+0000 7efee9036740 20 client.26072 populate_metadata read hostname 'smithi001' 2024-04-26T01:59:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.982+0000 7efee9036740 1 --2- 172.21.15.1:0/3798244864 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563b00621650 0x563b00623a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.982+0000 7efee9036740 4 client.26072 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:59:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.982+0000 7efee9036740 1 -- 172.21.15.1:0/3798244864 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563b00513320 con 0x563b00621650 2024-04-26T01:59:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.983+0000 7efed6a00640 1 --2- 172.21.15.1:0/3798244864 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563b00621650 0x563b00623a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.983+0000 7efed6a00640 1 --2- 172.21.15.1:0/3798244864 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563b00621650 0x563b00623a70 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c56b6fa70b341644 server_cookie=a0b77fc35b6e6969 in_seq=0 out_seq=0 2024-04-26T01:59:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.983+0000 7efeccc00640 10 client.26072 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.995+0000 7efeccc00640 1 -- 172.21.15.1:0/3798244864 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563b00513320 con 0x563b00621650 2024-04-26T01:59:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.995+0000 7efeccc00640 10 client.26072 handle_command_reply: tid=0 2024-04-26T01:59:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.995+0000 7efee9036740 1 client.26072 shutdown 2024-04-26T01:59:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.995+0000 7efee9036740 1 -- 172.21.15.1:0/3798244864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b0059f820 msgr2=0x563b005b27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.995+0000 7efee9036740 1 --2- 172.21.15.1:0/3798244864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b0059f820 0x563b005b27c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efed000f490 tx=0x7efed00106e0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.995+0000 7efee9036740 1 -- 172.21.15.1:0/3798244864 shutdown_connections 2024-04-26T01:59:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.995+0000 7efee9036740 1 --2- 172.21.15.1:0/3798244864 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563b00621650 0x563b00623a70 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.995+0000 7efee9036740 1 --2- 172.21.15.1:0/3798244864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b005aaa00 0x563b005b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.995+0000 7efee9036740 1 --2- 172.21.15.1:0/3798244864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b005a0140 0x563b005afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.995+0000 7efee9036740 1 --2- 172.21.15.1:0/3798244864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563b0059f820 0x563b005b27c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.995+0000 7efeccc00640 0 client.26072 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.995+0000 7efee9036740 1 -- 172.21.15.1:0/3798244864 >> 172.21.15.1:0/3798244864 conn(0x563b0059bd30 msgr2=0x563b005a7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.996+0000 7efee9036740 1 -- 172.21.15.1:0/3798244864 shutdown_connections 2024-04-26T01:59:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.996+0000 7efee9036740 1 -- 172.21.15.1:0/3798244864 wait complete. 2024-04-26T01:59:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.996+0000 7efee9036740 20 client.26072 trim_cache size 0 max 16384 2024-04-26T01:59:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:59:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:59:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:59:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:59:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.997+0000 7efee6200640 1 -- 172.21.15.1:0/803343418 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efec40a3d10 msgr2=0x7efec40a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.997+0000 7efee6200640 1 --2- 172.21.15.1:0/803343418 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efec40a3d10 0x7efec40a61d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x563b0059d150 tx=0x7efed8005f90 comp rx=0 tx=0).stop 2024-04-26T01:59:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.997+0000 7efee6200640 1 -- 172.21.15.1:0/803343418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efee0103ae0 msgr2=0x7efee01a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.997+0000 7efee6200640 1 --2- 172.21.15.1:0/803343418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efee0103ae0 0x7efee01a0b90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efec802aa70 tx=0x7efec8003b90 comp rx=0 tx=0).stop 2024-04-26T01:59:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.997+0000 7efee6200640 1 -- 172.21.15.1:0/803343418 shutdown_connections 2024-04-26T01:59:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.997+0000 7efee6200640 1 --2- 172.21.15.1:0/803343418 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efec40a3d10 0x7efec40a61d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.997+0000 7efee6200640 1 --2- 172.21.15.1:0/803343418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efee010c590 0x7efee019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.997+0000 7efee6200640 1 --2- 172.21.15.1:0/803343418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efee0103ae0 0x7efee01a0b90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.997+0000 7efee6200640 1 --2- 172.21.15.1:0/803343418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee01031c0 0x7efee01a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.997+0000 7efee6200640 1 -- 172.21.15.1:0/803343418 >> 172.21.15.1:0/803343418 conn(0x7efee0077f70 msgr2=0x7efee0108470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.997+0000 7efee6200640 1 -- 172.21.15.1:0/803343418 shutdown_connections 2024-04-26T01:59:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:03.997+0000 7efee6200640 1 -- 172.21.15.1:0/803343418 wait complete. 2024-04-26T01:59:04.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:03 smithi097 ceph-mon[32853]: pgmap v228: 97 pgs: 97 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 973 KiB/s wr, 154 op/s 2024-04-26T01:59:04.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:03 smithi173 ceph-mon[32553]: pgmap v228: 97 pgs: 97 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 973 KiB/s wr, 154 op/s 2024-04-26T01:59:04.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:03 smithi001 ceph-mon[27822]: pgmap v228: 97 pgs: 97 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 973 KiB/s wr, 154 op/s 2024-04-26T01:59:06.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:05 smithi097 ceph-mon[32853]: pgmap v229: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.8 MiB/s wr, 292 op/s 2024-04-26T01:59:06.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:05 smithi097 ceph-mon[32853]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T01:59:06.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:05 smithi173 ceph-mon[32553]: pgmap v229: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.8 MiB/s wr, 292 op/s 2024-04-26T01:59:06.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:05 smithi173 ceph-mon[32553]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T01:59:06.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:05 smithi001 ceph-mon[27822]: pgmap v229: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.8 MiB/s wr, 292 op/s 2024-04-26T01:59:06.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:05 smithi001 ceph-mon[27822]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T01:59:08.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:07 smithi097 ceph-mon[32853]: pgmap v231: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.0 MiB/s wr, 317 op/s 2024-04-26T01:59:08.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:07 smithi173 ceph-mon[32553]: pgmap v231: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.0 MiB/s wr, 317 op/s 2024-04-26T01:59:08.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:07 smithi001 ceph-mon[27822]: pgmap v231: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.0 MiB/s wr, 317 op/s 2024-04-26T01:59:10.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:09 smithi097 ceph-mon[32853]: pgmap v232: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.0 MiB/s wr, 317 op/s 2024-04-26T01:59:10.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:09 smithi173 ceph-mon[32553]: pgmap v232: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.0 MiB/s wr, 317 op/s 2024-04-26T01:59:10.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:09 smithi001 ceph-mon[27822]: pgmap v232: 97 pgs: 97 active+clean; 52 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.0 MiB/s wr, 317 op/s 2024-04-26T01:59:10.722 INFO:tasks.workunit.client.0.smithi001.stdout:50000 write 0x5304f4 thru 0x53fa11 (0xf51e bytes) 2024-04-26T01:59:10.722 INFO:tasks.workunit.client.0.smithi001.stdout:All 50000 operations completed A-OK! 2024-04-26T01:59:10.734 INFO:tasks.workunit.client.0.smithi001.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2024-04-26T01:59:10.743 INFO:tasks.workunit.client.0.smithi001.stdout:Seed set to 1 2024-04-26T01:59:12.160 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-26T01:59:12.160 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-26T01:59:12.163 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-26T01:59:12.163 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-26T01:59:12.163 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support clone range, disabling! 2024-04-26T01:59:12.165 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-26T01:59:12.168 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support exchange range, disabling! 2024-04-26T01:59:12.168 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero size read 2024-04-26T01:59:12.186 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x1c37b90 2024-04-26T01:59:12.197 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x5bd330d 2024-04-26T01:59:12.226 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0x637bf99 2024-04-26T01:59:12.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:11 smithi097 ceph-mon[32853]: pgmap v233: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 299 op/s 2024-04-26T01:59:12.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:11 smithi173 ceph-mon[32553]: pgmap v233: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 299 op/s 2024-04-26T01:59:12.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:11 smithi001 ceph-mon[27822]: pgmap v233: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 299 op/s 2024-04-26T01:59:12.710 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0x63d9b9c 2024-04-26T01:59:14.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:13 smithi173 ceph-mon[32553]: pgmap v234: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 299 op/s 2024-04-26T01:59:14.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:13 smithi001 ceph-mon[27822]: pgmap v234: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 299 op/s 2024-04-26T01:59:14.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:13 smithi097 ceph-mon[32853]: pgmap v234: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 299 op/s 2024-04-26T01:59:15.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:15 smithi097 ceph-mon[32853]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T01:59:15.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:15 smithi173 ceph-mon[32553]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T01:59:15.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:15 smithi001 ceph-mon[27822]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T01:59:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:16 smithi097 ceph-mon[32853]: pgmap v236: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.3 MiB/s wr, 246 op/s 2024-04-26T01:59:16.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:16 smithi173 ceph-mon[32553]: pgmap v236: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.3 MiB/s wr, 246 op/s 2024-04-26T01:59:16.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:16 smithi001 ceph-mon[27822]: pgmap v236: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.3 MiB/s wr, 246 op/s 2024-04-26T01:59:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.183+0000 7f9044200640 1 -- 172.21.15.1:0/2396564201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f903c1018d0 msgr2=0x7f903c105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.183+0000 7f9044200640 1 --2- 172.21.15.1:0/2396564201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f903c1018d0 0x7f903c105d90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9028009900 tx=0x7f902802a480 comp rx=0 tx=0).stop 2024-04-26T01:59:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.184+0000 7f9044200640 1 -- 172.21.15.1:0/2396564201 shutdown_connections 2024-04-26T01:59:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.184+0000 7f9044200640 1 --2- 172.21.15.1:0/2396564201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f903c1063b0 0x7f903c106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.184+0000 7f9044200640 1 --2- 172.21.15.1:0/2396564201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f903c1018d0 0x7f903c105d90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.184+0000 7f9044200640 1 --2- 172.21.15.1:0/2396564201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f903c100fb0 0x7f903c101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.184+0000 7f9044200640 1 -- 172.21.15.1:0/2396564201 >> 172.21.15.1:0/2396564201 conn(0x7f903c077f70 msgr2=0x7f903c078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.184+0000 7f9044200640 1 -- 172.21.15.1:0/2396564201 shutdown_connections 2024-04-26T01:59:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.184+0000 7f9044200640 1 -- 172.21.15.1:0/2396564201 wait complete. 2024-04-26T01:59:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.185+0000 7f9044200640 1 Processor -- start 2024-04-26T01:59:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.185+0000 7f9044200640 1 -- start start 2024-04-26T01:59:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.185+0000 7f9044200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f903c100fb0 0x7f903c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.185+0000 7f9044200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f903c1018d0 0x7f903c19f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.185+0000 7f9044200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f903c1063b0 0x7f903c1a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.185+0000 7f9044200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1187e0 con 0x7f903c1063b0 2024-04-26T01:59:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.185+0000 7f9044200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f903c118660 con 0x7f903c1018d0 2024-04-26T01:59:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.185+0000 7f9044200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f903c118960 con 0x7f903c100fb0 2024-04-26T01:59:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.186+0000 7f9040e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f903c1018d0 0x7f903c19f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.186+0000 7f9042200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f903c1063b0 0x7f903c1a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.186+0000 7f9040e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f903c1018d0 0x7f903c19f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:59794/0 (socket says 172.21.15.1:59794) 2024-04-26T01:59:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.186+0000 7f9040e00640 1 -- 172.21.15.1:0/2127880146 learned_addr learned my addr 172.21.15.1:0/2127880146 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.186+0000 7f9041800640 1 --2- 172.21.15.1:0/2127880146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f903c100fb0 0x7f903c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.186+0000 7f9040e00640 1 -- 172.21.15.1:0/2127880146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f903c100fb0 msgr2=0x7f903c19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.186+0000 7f9040e00640 1 --2- 172.21.15.1:0/2127880146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f903c100fb0 0x7f903c19ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.186+0000 7f9040e00640 1 -- 172.21.15.1:0/2127880146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f903c1063b0 msgr2=0x7f903c1a36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.186+0000 7f9040e00640 1 --2- 172.21.15.1:0/2127880146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f903c1063b0 0x7f903c1a36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.186+0000 7f9040e00640 1 -- 172.21.15.1:0/2127880146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903c1a3df0 con 0x7f903c1018d0 2024-04-26T01:59:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.186+0000 7f9042200640 1 --2- 172.21.15.1:0/2127880146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f903c1063b0 0x7f903c1a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.187+0000 7f9040e00640 1 --2- 172.21.15.1:0/2127880146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f903c1018d0 0x7f903c19f340 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f902802aa70 tx=0x7f9028003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.187+0000 7f903a000640 1 -- 172.21.15.1:0/2127880146 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902803f070 con 0x7f903c1018d0 2024-04-26T01:59:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.187+0000 7f903a000640 1 -- 172.21.15.1:0/2127880146 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f902803bb20 con 0x7f903c1018d0 2024-04-26T01:59:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.187+0000 7f903a000640 1 -- 172.21.15.1:0/2127880146 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028035030 con 0x7f903c1018d0 2024-04-26T01:59:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.187+0000 7f9044200640 1 -- 172.21.15.1:0/2127880146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903c1a4080 con 0x7f903c1018d0 2024-04-26T01:59:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.187+0000 7f9044200640 1 -- 172.21.15.1:0/2127880146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f903c1ab970 con 0x7f903c1018d0 2024-04-26T01:59:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.190+0000 7f903a000640 1 -- 172.21.15.1:0/2127880146 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f9028042070 con 0x7f903c1018d0 2024-04-26T01:59:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.192+0000 7f9047022740 1 Processor -- start 2024-04-26T01:59:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.192+0000 7f9047022740 1 -- start start 2024-04-26T01:59:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.192+0000 7f9047022740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709bd820 0x55a7709bdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.193+0000 7f9047022740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 0x55a7709c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.193+0000 7f9047022740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709c8a00 0x55a7709cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.193+0000 7f903a000640 1 --2- 172.21.15.1:0/2127880146 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f90240a3d90 0x7f90240a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.193+0000 7f9047022740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a770923900 con 0x55a7709be140 2024-04-26T01:59:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.193+0000 7f9047022740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a7708c9ef0 con 0x55a7709c8a00 2024-04-26T01:59:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.193+0000 7f9047022740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a77091f8c0 con 0x55a7709bd820 2024-04-26T01:59:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.193+0000 7f903a000640 1 -- 172.21.15.1:0/2127880146 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f90280f0ea0 con 0x7f903c1018d0 2024-04-26T01:59:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.193+0000 7f9042200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709bd820 0x55a7709bdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.193+0000 7f9041800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 0x55a7709c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.193+0000 7f9040e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709c8a00 0x55a7709cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.193+0000 7f9040e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709c8a00 0x55a7709cadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:59806/0 (socket says 172.21.15.1:59806) 2024-04-26T01:59:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.193+0000 7f9040e00640 1 -- 172.21.15.1:0/3793423878 learned_addr learned my addr 172.21.15.1:0/3793423878 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.193+0000 7f9040e00640 1 -- 172.21.15.1:0/3793423878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709bd820 msgr2=0x55a7709bdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.193+0000 7f9040e00640 1 --2- 172.21.15.1:0/3793423878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709bd820 0x55a7709bdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.193+0000 7f9042200640 1 --2- 172.21.15.1:0/2127880146 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f90240a3d90 0x7f90240a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.193+0000 7f9040e00640 1 -- 172.21.15.1:0/3793423878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 msgr2=0x55a7709c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.193+0000 7f9040e00640 1 --2- 172.21.15.1:0/3793423878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 0x55a7709c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.193+0000 7f9040e00640 1 -- 172.21.15.1:0/3793423878 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a770636210 con 0x55a7709c8a00 2024-04-26T01:59:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.194+0000 7f9040e00640 1 --2- 172.21.15.1:0/3793423878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709c8a00 0x55a7709cadf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f902802aa40 tx=0x7f902802a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ddf5ec57011dc7a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.194+0000 7f9042200640 1 --2- 172.21.15.1:0/2127880146 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f90240a3d90 0x7f90240a6250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9034005fd0 tx=0x7f9034009500 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.196+0000 7f902ea00640 1 -- 172.21.15.1:0/3793423878 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028049070 con 0x55a7709c8a00 2024-04-26T01:59:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.196+0000 7f902ea00640 1 -- 172.21.15.1:0/3793423878 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9028031430 con 0x55a7709c8a00 2024-04-26T01:59:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.196+0000 7f902ea00640 1 -- 172.21.15.1:0/3793423878 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90280b94d0 con 0x55a7709c8a00 2024-04-26T01:59:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.197+0000 7f9047022740 1 -- 172.21.15.1:0/3793423878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709c8a00 msgr2=0x55a7709cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.197+0000 7f9047022740 1 --2- 172.21.15.1:0/3793423878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709c8a00 0x55a7709cadf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f902802aa40 tx=0x7f902802a480 comp rx=0 tx=0).stop 2024-04-26T01:59:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.197+0000 7f9047022740 1 -- 172.21.15.1:0/3793423878 shutdown_connections 2024-04-26T01:59:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.197+0000 7f9047022740 1 --2- 172.21.15.1:0/3793423878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709c8a00 0x55a7709cadf0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.197+0000 7f9047022740 1 --2- 172.21.15.1:0/3793423878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 0x55a7709c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.197+0000 7f9047022740 1 --2- 172.21.15.1:0/3793423878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709bd820 0x55a7709bdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.197+0000 7f9047022740 1 -- 172.21.15.1:0/3793423878 >> 172.21.15.1:0/3793423878 conn(0x55a7709b9d30 msgr2=0x55a7709ba110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.197+0000 7f9047022740 1 -- 172.21.15.1:0/3793423878 shutdown_connections 2024-04-26T01:59:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.197+0000 7f9047022740 1 -- 172.21.15.1:0/3793423878 wait complete. 2024-04-26T01:59:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.198+0000 7f9047022740 1 -- start start 2024-04-26T01:59:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.198+0000 7f9047022740 1 Processor -- start 2024-04-26T01:59:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.198+0000 7f9047022740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709bd820 0x55a7709c5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.198+0000 7f9047022740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 0x55a7709c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.198+0000 7f9047022740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709c8a00 0x55a7709c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.198+0000 7f9047022740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a7709cb3f0 con 0x55a7709be140 2024-04-26T01:59:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.198+0000 7f9047022740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a7708dd310 con 0x55a7709c8a00 2024-04-26T01:59:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.198+0000 7f9047022740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a7709cb570 con 0x55a7709bd820 2024-04-26T01:59:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.199+0000 7f9047022740 10 client.0 _ll_register_callbacks cb 0x55a7708abcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.199+0000 7f9042200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709c8a00 0x55a7709c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.199+0000 7f9040e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 0x55a7709c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.199+0000 7f9041800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709bd820 0x55a7709c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.199+0000 7f9042200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709c8a00 0x55a7709c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:59818/0 (socket says 172.21.15.1:59818) 2024-04-26T01:59:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.199+0000 7f9042200640 1 -- 172.21.15.1:0/2677619792 learned_addr learned my addr 172.21.15.1:0/2677619792 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.199+0000 7f9040e00640 1 -- 172.21.15.1:0/2677619792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709bd820 msgr2=0x55a7709c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.199+0000 7f9040e00640 1 --2- 172.21.15.1:0/2677619792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709bd820 0x55a7709c5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.199+0000 7f9040e00640 1 -- 172.21.15.1:0/2677619792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709c8a00 msgr2=0x55a7709c63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.199+0000 7f9040e00640 1 --2- 172.21.15.1:0/2677619792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709c8a00 0x55a7709c63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.199+0000 7f9040e00640 1 -- 172.21.15.1:0/2677619792 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a770702620 con 0x55a7709be140 2024-04-26T01:59:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.199+0000 7f9041800640 1 --2- 172.21.15.1:0/2677619792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709bd820 0x55a7709c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.199+0000 7f9042200640 1 --2- 172.21.15.1:0/2677619792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709c8a00 0x55a7709c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.200+0000 7f9040e00640 1 --2- 172.21.15.1:0/2677619792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 0x55a7709c5ea0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55a7709bb150 tx=0x7f90280b8a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b249a0e736c4df6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.200+0000 7f9023400640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:59:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.200+0000 7f9023400640 1 -- 172.21.15.1:0/2677619792 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028049030 con 0x55a7709be140 2024-04-26T01:59:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.200+0000 7f9023400640 1 -- 172.21.15.1:0/2677619792 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f90280ee7e0 con 0x55a7709be140 2024-04-26T01:59:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.200+0000 7f9047022740 1 -- 172.21.15.1:0/2677619792 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a7709378b0 con 0x55a7709be140 2024-04-26T01:59:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.200+0000 7f9023400640 1 -- 172.21.15.1:0/2677619792 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f90280a0050 con 0x55a7709be140 2024-04-26T01:59:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.200+0000 7f9047022740 10 client.15930 fetch_fsmap learned FSMap version 32 2024-04-26T01:59:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.200+0000 7f9047022740 1 -- 172.21.15.1:0/2677619792 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a7709cb3f0 con 0x55a7709be140 2024-04-26T01:59:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.200+0000 7f9023400640 1 -- 172.21.15.1:0/2677619792 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a7709378b0 con 0x55a7709be140 2024-04-26T01:59:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.200+0000 7f9023400640 1 -- 172.21.15.1:0/2677619792 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f90280c5590 con 0x55a7709be140 2024-04-26T01:59:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.201+0000 7f9047022740 10 client.15930 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:59:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.201+0000 7f9047022740 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:59:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.201+0000 7f9047022740 20 client.15930 populate_metadata read hostname 'smithi001' 2024-04-26T01:59:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.201+0000 7f9047022740 1 --2- 172.21.15.1:0/2677619792 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a7709d87d0 0x55a7709dabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.201+0000 7f9047022740 4 client.15930 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:59:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.201+0000 7f9047022740 1 -- 172.21.15.1:0/2677619792 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a770931320 con 0x55a7709d87d0 2024-04-26T01:59:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.201+0000 7f9041800640 1 --2- 172.21.15.1:0/2677619792 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a7709d87d0 0x55a7709dabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.202+0000 7f9041800640 1 --2- 172.21.15.1:0/2677619792 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a7709d87d0 0x55a7709dabc0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99ebc8b7a5e0969e server_cookie=f03d0c91af17df1e in_seq=0 out_seq=0 2024-04-26T01:59:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.202+0000 7f9023400640 10 client.15930 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.202+0000 7f9023400640 1 -- 172.21.15.1:0/2677619792 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a770931320 con 0x55a7709d87d0 2024-04-26T01:59:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.202+0000 7f9023400640 10 client.15930 handle_command_reply: tid=0 2024-04-26T01:59:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.202+0000 7f9047022740 1 client.15930 shutdown 2024-04-26T01:59:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.203+0000 7f9047022740 1 -- 172.21.15.1:0/2677619792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 msgr2=0x55a7709c5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.203+0000 7f9047022740 1 --2- 172.21.15.1:0/2677619792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 0x55a7709c5ea0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55a7709bb150 tx=0x7f90280b8a70 comp rx=0 tx=0).stop 2024-04-26T01:59:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.203+0000 7f9047022740 1 -- 172.21.15.1:0/2677619792 shutdown_connections 2024-04-26T01:59:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.203+0000 7f9047022740 1 --2- 172.21.15.1:0/2677619792 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a7709d87d0 0x55a7709dabc0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.203+0000 7f9047022740 1 --2- 172.21.15.1:0/2677619792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709c8a00 0x55a7709c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.203+0000 7f9047022740 1 --2- 172.21.15.1:0/2677619792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 0x55a7709c5ea0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.203+0000 7f9047022740 1 --2- 172.21.15.1:0/2677619792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709bd820 0x55a7709c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.203+0000 7f9023400640 0 client.15930 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.203+0000 7f9047022740 1 -- 172.21.15.1:0/2677619792 >> 172.21.15.1:0/2677619792 conn(0x55a7709b9d30 msgr2=0x55a7709bfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.203+0000 7f9047022740 1 -- 172.21.15.1:0/2677619792 shutdown_connections 2024-04-26T01:59:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.203+0000 7f9047022740 1 -- 172.21.15.1:0/2677619792 wait complete. 2024-04-26T01:59:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.203+0000 7f9047022740 20 client.15930 trim_cache size 0 max 16384 2024-04-26T01:59:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.220+0000 7f9047022740 1 Processor -- start 2024-04-26T01:59:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.220+0000 7f9047022740 1 -- start start 2024-04-26T01:59:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.221+0000 7f9047022740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709bd820 0x55a7709b9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.221+0000 7f9047022740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 0x55a7709c8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.221+0000 7f9047022740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709c8a00 0x55a7709c6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.221+0000 7f9047022740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1b6c50 con 0x55a7709be140 2024-04-26T01:59:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.221+0000 7f9047022740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1b4520 con 0x55a7709bd820 2024-04-26T01:59:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.221+0000 7f9047022740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1b2370 con 0x55a7709c8a00 2024-04-26T01:59:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.221+0000 7f9040e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 0x55a7709c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.221+0000 7f9042200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709c8a00 0x55a7709c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.221+0000 7f9041800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709bd820 0x55a7709b9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.221+0000 7f9040e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 0x55a7709c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52394/0 (socket says 172.21.15.1:52394) 2024-04-26T01:59:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.221+0000 7f9040e00640 1 -- 172.21.15.1:0/2825826532 learned_addr learned my addr 172.21.15.1:0/2825826532 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.221+0000 7f9041800640 1 -- 172.21.15.1:0/2825826532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709c8a00 msgr2=0x55a7709c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.221+0000 7f9041800640 1 --2- 172.21.15.1:0/2825826532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709c8a00 0x55a7709c6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.221+0000 7f9041800640 1 -- 172.21.15.1:0/2825826532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 msgr2=0x55a7709c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.221+0000 7f9041800640 1 --2- 172.21.15.1:0/2825826532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 0x55a7709c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.221+0000 7f9041800640 1 -- 172.21.15.1:0/2825826532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7709daf30 con 0x55a7709bd820 2024-04-26T01:59:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.222+0000 7f9041800640 1 --2- 172.21.15.1:0/2825826532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709bd820 0x55a7709b9610 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55a7709db1c0 tx=0x7f9030005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=213d4f877b6bf910 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.222+0000 7f902e000640 1 -- 172.21.15.1:0/2825826532 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030020310 con 0x55a7709bd820 2024-04-26T01:59:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.222+0000 7f902e000640 1 -- 172.21.15.1:0/2825826532 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9030010430 con 0x55a7709bd820 2024-04-26T01:59:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.222+0000 7f902e000640 1 -- 172.21.15.1:0/2825826532 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903001e6a0 con 0x55a7709bd820 2024-04-26T01:59:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.223+0000 7f9047022740 1 -- 172.21.15.1:0/2825826532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709bd820 msgr2=0x55a7709b9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.223+0000 7f9047022740 1 --2- 172.21.15.1:0/2825826532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709bd820 0x55a7709b9610 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55a7709db1c0 tx=0x7f9030005f90 comp rx=0 tx=0).stop 2024-04-26T01:59:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.223+0000 7f9047022740 1 -- 172.21.15.1:0/2825826532 shutdown_connections 2024-04-26T01:59:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.223+0000 7f9047022740 1 --2- 172.21.15.1:0/2825826532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709c8a00 0x55a7709c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.223+0000 7f9047022740 1 --2- 172.21.15.1:0/2825826532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 0x55a7709c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.223+0000 7f9047022740 1 --2- 172.21.15.1:0/2825826532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709bd820 0x55a7709b9610 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.223+0000 7f9047022740 1 -- 172.21.15.1:0/2825826532 >> 172.21.15.1:0/2825826532 conn(0x55a7709b9d30 msgr2=0x55a7709d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.223+0000 7f9047022740 1 -- 172.21.15.1:0/2825826532 shutdown_connections 2024-04-26T01:59:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.223+0000 7f9047022740 1 -- 172.21.15.1:0/2825826532 wait complete. 2024-04-26T01:59:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.224+0000 7f9047022740 1 -- start start 2024-04-26T01:59:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.224+0000 7f9047022740 1 Processor -- start 2024-04-26T01:59:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.224+0000 7f9047022740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709bd820 0x55a7709d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.224+0000 7f9047022740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 0x55a7709cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.224+0000 7f9047022740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709c8a00 0x55a7709ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.224+0000 7f9047022740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1b29e0 con 0x55a7709be140 2024-04-26T01:59:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.224+0000 7f9047022740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1b3e50 con 0x55a7709c8a00 2024-04-26T01:59:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.224+0000 7f9047022740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1b30b0 con 0x55a7709bd820 2024-04-26T01:59:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.224+0000 7f9047022740 10 client.0 _ll_register_callbacks cb 0x7f90280ee500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.225+0000 7f9041800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709bd820 0x55a7709d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.225+0000 7f9040e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 0x55a7709cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.225+0000 7f9041800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709bd820 0x55a7709d08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:38988/0 (socket says 172.21.15.1:38988) 2024-04-26T01:59:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.225+0000 7f9041800640 1 -- 172.21.15.1:0/1434353145 learned_addr learned my addr 172.21.15.1:0/1434353145 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.225+0000 7f9042200640 1 --2- 172.21.15.1:0/1434353145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709c8a00 0x55a7709ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.225+0000 7f9040e00640 1 -- 172.21.15.1:0/1434353145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709bd820 msgr2=0x55a7709d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.225+0000 7f9040e00640 1 --2- 172.21.15.1:0/1434353145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709bd820 0x55a7709d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.225+0000 7f9040e00640 1 -- 172.21.15.1:0/1434353145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709c8a00 msgr2=0x55a7709ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.225+0000 7f9040e00640 1 --2- 172.21.15.1:0/1434353145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709c8a00 0x55a7709ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.225+0000 7f9040e00640 1 -- 172.21.15.1:0/1434353145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7709ce750 con 0x55a7709be140 2024-04-26T01:59:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.225+0000 7f9040e00640 1 --2- 172.21.15.1:0/1434353145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 0x55a7709cdc50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f90280bb110 tx=0x7f9028005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c789314ce1f981c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.225+0000 7f9023400640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:59:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.226+0000 7f9023400640 1 -- 172.21.15.1:0/1434353145 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028049070 con 0x55a7709be140 2024-04-26T01:59:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.226+0000 7f9023400640 1 -- 172.21.15.1:0/1434353145 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f90280ee7e0 con 0x55a7709be140 2024-04-26T01:59:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.226+0000 7f9047022740 1 -- 172.21.15.1:0/1434353145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9030010430 con 0x55a7709be140 2024-04-26T01:59:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.226+0000 7f9023400640 1 -- 172.21.15.1:0/1434353145 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f90280bce60 con 0x55a7709be140 2024-04-26T01:59:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.226+0000 7f9047022740 10 client.15936 fetch_fsmap learned FSMap version 32 2024-04-26T01:59:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.226+0000 7f9047022740 1 -- 172.21.15.1:0/1434353145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a7709ce750 con 0x55a7709be140 2024-04-26T01:59:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.226+0000 7f9023400640 1 -- 172.21.15.1:0/1434353145 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9030010430 con 0x55a7709be140 2024-04-26T01:59:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.226+0000 7f9023400640 1 -- 172.21.15.1:0/1434353145 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f90280d6120 con 0x55a7709be140 2024-04-26T01:59:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.226+0000 7f9047022740 10 client.15936 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:59:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.227+0000 7f9047022740 10 client.15936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:59:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.227+0000 7f9047022740 20 client.15936 populate_metadata read hostname 'smithi001' 2024-04-26T01:59:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.227+0000 7f9047022740 1 --2- 172.21.15.1:0/1434353145 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a770a3ef40 0x55a770a41360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.227+0000 7f9047022740 4 client.15936 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:59:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.227+0000 7f9047022740 1 -- 172.21.15.1:0/1434353145 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a77092b190 con 0x55a770a3ef40 2024-04-26T01:59:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.227+0000 7f9041800640 1 --2- 172.21.15.1:0/1434353145 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a770a3ef40 0x55a770a41360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.227+0000 7f9041800640 1 --2- 172.21.15.1:0/1434353145 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a770a3ef40 0x55a770a41360 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=794be1a3e467ea12 server_cookie=7daca51f2c68a2c3 in_seq=0 out_seq=0 2024-04-26T01:59:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.227+0000 7f9023400640 10 client.15936 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.247+0000 7f9023400640 1 -- 172.21.15.1:0/1434353145 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a77092b190 con 0x55a770a3ef40 2024-04-26T01:59:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.247+0000 7f9023400640 10 client.15936 handle_command_reply: tid=0 2024-04-26T01:59:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.247+0000 7f9047022740 1 client.15936 shutdown 2024-04-26T01:59:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.247+0000 7f9047022740 1 -- 172.21.15.1:0/1434353145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 msgr2=0x55a7709cdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.247+0000 7f9047022740 1 --2- 172.21.15.1:0/1434353145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 0x55a7709cdc50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f90280bb110 tx=0x7f9028005e00 comp rx=0 tx=0).stop 2024-04-26T01:59:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.247+0000 7f9047022740 1 -- 172.21.15.1:0/1434353145 shutdown_connections 2024-04-26T01:59:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.247+0000 7f9047022740 1 --2- 172.21.15.1:0/1434353145 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a770a3ef40 0x55a770a41360 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.247+0000 7f9047022740 1 --2- 172.21.15.1:0/1434353145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a7709c8a00 0x55a7709ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.247+0000 7f9047022740 1 --2- 172.21.15.1:0/1434353145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7709be140 0x55a7709cdc50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.247+0000 7f9047022740 1 --2- 172.21.15.1:0/1434353145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7709bd820 0x55a7709d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.247+0000 7f9047022740 1 -- 172.21.15.1:0/1434353145 >> 172.21.15.1:0/1434353145 conn(0x55a7709b9d30 msgr2=0x55a7709b8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.247+0000 7f9023400640 0 client.15936 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.247+0000 7f9047022740 1 -- 172.21.15.1:0/1434353145 shutdown_connections 2024-04-26T01:59:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.247+0000 7f9047022740 1 -- 172.21.15.1:0/1434353145 wait complete. 2024-04-26T01:59:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.247+0000 7f9047022740 20 client.15936 trim_cache size 0 max 16384 2024-04-26T01:59:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:59:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:59:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:59:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:59:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.249+0000 7f9044200640 1 -- 172.21.15.1:0/2127880146 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f90240a3d90 msgr2=0x7f90240a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.249+0000 7f9044200640 1 --2- 172.21.15.1:0/2127880146 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f90240a3d90 0x7f90240a6250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9034005fd0 tx=0x7f9034009500 comp rx=0 tx=0).stop 2024-04-26T01:59:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.249+0000 7f9044200640 1 -- 172.21.15.1:0/2127880146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f903c1018d0 msgr2=0x7f903c19f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.249+0000 7f9044200640 1 --2- 172.21.15.1:0/2127880146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f903c1018d0 0x7f903c19f340 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f902802aa70 tx=0x7f9028003c10 comp rx=0 tx=0).stop 2024-04-26T01:59:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.249+0000 7f9044200640 1 -- 172.21.15.1:0/2127880146 shutdown_connections 2024-04-26T01:59:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.249+0000 7f9044200640 1 --2- 172.21.15.1:0/2127880146 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f90240a3d90 0x7f90240a6250 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.249+0000 7f9044200640 1 --2- 172.21.15.1:0/2127880146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f903c1063b0 0x7f903c1a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.249+0000 7f9044200640 1 --2- 172.21.15.1:0/2127880146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f903c1018d0 0x7f903c19f340 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.249+0000 7f9044200640 1 --2- 172.21.15.1:0/2127880146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f903c100fb0 0x7f903c19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.249+0000 7f9044200640 1 -- 172.21.15.1:0/2127880146 >> 172.21.15.1:0/2127880146 conn(0x7f903c077f70 msgr2=0x7f903c103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.250+0000 7f9044200640 1 -- 172.21.15.1:0/2127880146 shutdown_connections 2024-04-26T01:59:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:17.250+0000 7f9044200640 1 -- 172.21.15.1:0/2127880146 wait complete. 2024-04-26T01:59:18.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:18 smithi097 ceph-mon[32853]: pgmap v237: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.3 MiB/s wr, 243 op/s 2024-04-26T01:59:18.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:18 smithi173 ceph-mon[32553]: pgmap v237: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.3 MiB/s wr, 243 op/s 2024-04-26T01:59:18.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:18 smithi001 ceph-mon[27822]: pgmap v237: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.3 MiB/s wr, 243 op/s 2024-04-26T01:59:20.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:20 smithi097 ceph-mon[32853]: pgmap v238: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.3 MiB/s wr, 243 op/s 2024-04-26T01:59:20.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:20 smithi173 ceph-mon[32553]: pgmap v238: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.3 MiB/s wr, 243 op/s 2024-04-26T01:59:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:20 smithi001 ceph-mon[27822]: pgmap v238: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.3 MiB/s wr, 243 op/s 2024-04-26T01:59:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:22 smithi097 ceph-mon[32853]: pgmap v239: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 322 op/s 2024-04-26T01:59:22.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:22 smithi173 ceph-mon[32553]: pgmap v239: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 322 op/s 2024-04-26T01:59:22.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:22 smithi001 ceph-mon[27822]: pgmap v239: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 322 op/s 2024-04-26T01:59:24.200 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.9 2024-04-26T01:59:24.201 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:59:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:24 smithi097 ceph-mon[32853]: pgmap v240: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 322 op/s 2024-04-26T01:59:24.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:24 smithi173 ceph-mon[32553]: pgmap v240: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 322 op/s 2024-04-26T01:59:24.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:24 smithi001 ceph-mon[27822]: pgmap v240: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 322 op/s 2024-04-26T01:59:24.705 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:59:26.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:26 smithi097 ceph-mon[32853]: pgmap v241: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.6 MiB/s wr, 471 op/s 2024-04-26T01:59:26.526 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:26 smithi001 ceph-mon[27822]: pgmap v241: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.6 MiB/s wr, 471 op/s 2024-04-26T01:59:26.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:26 smithi173 ceph-mon[32553]: pgmap v241: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.6 MiB/s wr, 471 op/s 2024-04-26T01:59:27.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.072+0000 7fd5b5600700 1 -- 172.21.15.1:0/286280312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5b006a750 msgr2=0x7fd5b006ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:27.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.072+0000 7fd5b5600700 1 --2- 172.21.15.1:0/286280312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5b006a750 0x7fd5b006ab30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009f90 tx=0x7fd5a000b330 comp rx=0 tx=0).stop 2024-04-26T01:59:27.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.072+0000 7fd5b5600700 1 -- 172.21.15.1:0/286280312 shutdown_connections 2024-04-26T01:59:27.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.072+0000 7fd5b5600700 1 --2- 172.21.15.1:0/286280312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5b010f4f0 0x7fd5b01118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:27.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.072+0000 7fd5b5600700 1 --2- 172.21.15.1:0/286280312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5b006a750 0x7fd5b006ab30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:27.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.072+0000 7fd5b5600700 1 --2- 172.21.15.1:0/286280312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5b006b070 0x7fd5b010efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:27.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.072+0000 7fd5b5600700 1 -- 172.21.15.1:0/286280312 >> 172.21.15.1:0/286280312 conn(0x7fd5b00788f0 msgr2=0x7fd5b0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:27.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.072+0000 7fd5b5600700 1 -- 172.21.15.1:0/286280312 shutdown_connections 2024-04-26T01:59:27.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.072+0000 7fd5b5600700 1 -- 172.21.15.1:0/286280312 wait complete. 2024-04-26T01:59:27.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.074+0000 7fd5b5600700 1 Processor -- start 2024-04-26T01:59:27.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.074+0000 7fd5b5600700 1 -- start start 2024-04-26T01:59:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.075+0000 7fd5b5600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5b006a750 0x7fd5b019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.075+0000 7fd5b5600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5b006b070 0x7fd5b019b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.075+0000 7fd5b5600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5b010f4f0 0x7fd5b019fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.075+0000 7fd5b5600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b0114e70 con 0x7fd5b006b070 2024-04-26T01:59:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.075+0000 7fd5b5600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b0114cf0 con 0x7fd5b006a750 2024-04-26T01:59:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.075+0000 7fd5b5600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b0114ff0 con 0x7fd5b010f4f0 2024-04-26T01:59:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.075+0000 7fd5aea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5b006a750 0x7fd5b019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.075+0000 7fd5aea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5b006a750 0x7fd5b019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:44140/0 (socket says 172.21.15.1:44140) 2024-04-26T01:59:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.075+0000 7fd5aea00700 1 -- 172.21.15.1:0/3883466765 learned_addr learned my addr 172.21.15.1:0/3883466765 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.075+0000 7fd5a6000700 1 --2- 172.21.15.1:0/3883466765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5b006b070 0x7fd5b019b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.075+0000 7fd5af400700 1 --2- 172.21.15.1:0/3883466765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5b010f4f0 0x7fd5b019fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.075+0000 7fd5aea00700 1 -- 172.21.15.1:0/3883466765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5b010f4f0 msgr2=0x7fd5b019fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.075+0000 7fd5aea00700 1 --2- 172.21.15.1:0/3883466765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5b010f4f0 0x7fd5b019fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.075+0000 7fd5aea00700 1 -- 172.21.15.1:0/3883466765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5b006b070 msgr2=0x7fd5b019b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.075+0000 7fd5aea00700 1 --2- 172.21.15.1:0/3883466765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5b006b070 0x7fd5b019b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.075+0000 7fd5aea00700 1 -- 172.21.15.1:0/3883466765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b01a0300 con 0x7fd5b006a750 2024-04-26T01:59:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.075+0000 7fd5a6000700 1 --2- 172.21.15.1:0/3883466765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5b006b070 0x7fd5b019b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:27.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.076+0000 7fd5aea00700 1 --2- 172.21.15.1:0/3883466765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5b006a750 0x7fd5b019b340 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009f60 tx=0x7fd5a000ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:27.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.076+0000 7fd5a7e00700 1 -- 172.21.15.1:0/3883466765 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0021070 con 0x7fd5b006a750 2024-04-26T01:59:27.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.076+0000 7fd5a7e00700 1 -- 172.21.15.1:0/3883466765 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5a0007ac0 con 0x7fd5b006a750 2024-04-26T01:59:27.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.076+0000 7fd5b5600700 1 -- 172.21.15.1:0/3883466765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b01a0590 con 0x7fd5b006a750 2024-04-26T01:59:27.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.076+0000 7fd5a7e00700 1 -- 172.21.15.1:0/3883466765 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a00052a0 con 0x7fd5b006a750 2024-04-26T01:59:27.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.076+0000 7fd5b5600700 1 -- 172.21.15.1:0/3883466765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b01a7e70 con 0x7fd5b006a750 2024-04-26T01:59:27.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.078+0000 7fd5b5600700 1 -- 172.21.15.1:0/3883466765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b0069730 con 0x7fd5b006a750 2024-04-26T01:59:27.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.081+0000 7fd5a7e00700 1 -- 172.21.15.1:0/3883466765 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fd5a0024070 con 0x7fd5b006a750 2024-04-26T01:59:27.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.082+0000 7fd5a7e00700 1 --2- 172.21.15.1:0/3883466765 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd5940a3db0 0x7fd5940a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:27.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.083+0000 7fd5a7e00700 1 -- 172.21.15.1:0/3883466765 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fd5a000f020 con 0x7fd5b006a750 2024-04-26T01:59:27.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.083+0000 7fd5a6000700 1 --2- 172.21.15.1:0/3883466765 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd5940a3db0 0x7fd5940a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:27.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.084+0000 7fd5a6000700 1 --2- 172.21.15.1:0/3883466765 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd5940a3db0 0x7fd5940a6270 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0102ea0 tx=0x7fd598009310 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.087+0000 7fd5a7e00700 1 -- 172.21.15.1:0/3883466765 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd5a002b020 con 0x7fd5b006a750 2024-04-26T01:59:27.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.425+0000 7fd5b5600700 1 -- 172.21.15.1:0/3883466765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5b002df80 con 0x7fd5b006a750 2024-04-26T01:59:27.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.429+0000 7fd5a7e00700 1 -- 172.21.15.1:0/3883466765 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11838 (secure 0 0 0) 0x7fd5a00a4020 con 0x7fd5b006a750 2024-04-26T01:59:27.430 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:59:27.430 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:55:49.645549+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25414":{"gid":25414,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/453427322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":453427322},{"type":"v1","addr":"172.21.15.173:6835","nonce":453427322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24499,24514,24518]},"id":1}]} 2024-04-26T01:59:27.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.433+0000 7fd5b5600700 1 -- 172.21.15.1:0/3883466765 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd5940a3db0 msgr2=0x7fd5940a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:27.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.433+0000 7fd5b5600700 1 --2- 172.21.15.1:0/3883466765 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd5940a3db0 0x7fd5940a6270 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0102ea0 tx=0x7fd598009310 comp rx=0 tx=0).stop 2024-04-26T01:59:27.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.433+0000 7fd5b5600700 1 -- 172.21.15.1:0/3883466765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5b006a750 msgr2=0x7fd5b019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:27.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.433+0000 7fd5b5600700 1 --2- 172.21.15.1:0/3883466765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5b006a750 0x7fd5b019b340 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009f60 tx=0x7fd5a000ccb0 comp rx=0 tx=0).stop 2024-04-26T01:59:27.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.434+0000 7fd5b5600700 1 -- 172.21.15.1:0/3883466765 shutdown_connections 2024-04-26T01:59:27.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.434+0000 7fd5b5600700 1 --2- 172.21.15.1:0/3883466765 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd5940a3db0 0x7fd5940a6270 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:27.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.434+0000 7fd5b5600700 1 --2- 172.21.15.1:0/3883466765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5b010f4f0 0x7fd5b019fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:27.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.434+0000 7fd5b5600700 1 --2- 172.21.15.1:0/3883466765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5b006a750 0x7fd5b019b340 secure :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009f60 tx=0x7fd5a000ccb0 comp rx=0 tx=0).stop 2024-04-26T01:59:27.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.434+0000 7fd5b5600700 1 --2- 172.21.15.1:0/3883466765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5b006b070 0x7fd5b019b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:27.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.434+0000 7fd5b5600700 1 -- 172.21.15.1:0/3883466765 >> 172.21.15.1:0/3883466765 conn(0x7fd5b00788f0 msgr2=0x7fd5b0110980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:27.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.434+0000 7fd5b5600700 1 -- 172.21.15.1:0/3883466765 shutdown_connections 2024-04-26T01:59:27.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:27.434+0000 7fd5b5600700 1 -- 172.21.15.1:0/3883466765 wait complete. 2024-04-26T01:59:27.435 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 32 2024-04-26T01:59:28.284 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24518 quiesce db / --timeout 60 --expiration 82.9 --await-for 42.0' 2024-04-26T01:59:28.285 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24518 quiesce db / --timeout 60 --expiration 82.9 --await-for 42.0 2024-04-26T01:59:28.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:28 smithi097 ceph-mon[32853]: pgmap v242: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.0 MiB/s wr, 328 op/s 2024-04-26T01:59:28.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:28 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3883466765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:59:28.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:28 smithi173 ceph-mon[32553]: pgmap v242: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.0 MiB/s wr, 328 op/s 2024-04-26T01:59:28.581 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:28 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3883466765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:59:28.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:28 smithi001 ceph-mon[27822]: pgmap v242: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.0 MiB/s wr, 328 op/s 2024-04-26T01:59:28.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:28 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3883466765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:59:28.777 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:59:30.266 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:30 smithi001 ceph-mon[27822]: pgmap v243: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.0 MiB/s wr, 328 op/s 2024-04-26T01:59:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.444+0000 7f0c38200640 1 -- 172.21.15.1:0/2917712850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c30109490 msgr2=0x7f0c30111060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.444+0000 7f0c38200640 1 --2- 172.21.15.1:0/2917712850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c30109490 0x7f0c30111060 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0c28009ca0 tx=0x7f0c2802a8c0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.444+0000 7f0c38200640 1 -- 172.21.15.1:0/2917712850 shutdown_connections 2024-04-26T01:59:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.444+0000 7f0c38200640 1 --2- 172.21.15.1:0/2917712850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c30109490 0x7f0c30111060 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.444+0000 7f0c38200640 1 --2- 172.21.15.1:0/2917712850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c30104d70 0x7f0c30108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.444+0000 7f0c38200640 1 --2- 172.21.15.1:0/2917712850 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c301043c0 0x7f0c301047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.444+0000 7f0c38200640 1 -- 172.21.15.1:0/2917712850 >> 172.21.15.1:0/2917712850 conn(0x7f0c300ffc40 msgr2=0x7f0c30102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.444+0000 7f0c38200640 1 -- 172.21.15.1:0/2917712850 shutdown_connections 2024-04-26T01:59:30.444 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.444+0000 7f0c38200640 1 -- 172.21.15.1:0/2917712850 wait complete. 2024-04-26T01:59:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.445+0000 7f0c38200640 1 Processor -- start 2024-04-26T01:59:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.445+0000 7f0c38200640 1 -- start start 2024-04-26T01:59:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.445+0000 7f0c38200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c301043c0 0x7f0c301a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.446+0000 7f0c38200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c30109490 0x7f0c301a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.446+0000 7f0c38200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c3019a7d0 0x7f0c3019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.446+0000 7f0c38200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0c30114430 con 0x7f0c30109490 2024-04-26T01:59:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.446+0000 7f0c38200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301142b0 con 0x7f0c301043c0 2024-04-26T01:59:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.446+0000 7f0c38200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301145b0 con 0x7f0c3019a7d0 2024-04-26T01:59:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.446+0000 7f0c35800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c301043c0 0x7f0c301a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.446+0000 7f0c35800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c301043c0 0x7f0c301a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:44162/0 (socket says 172.21.15.1:44162) 2024-04-26T01:59:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.446+0000 7f0c35800640 1 -- 172.21.15.1:0/4138450575 learned_addr learned my addr 172.21.15.1:0/4138450575 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.446+0000 7f0c34e00640 1 --2- 172.21.15.1:0/4138450575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c30109490 0x7f0c301a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.446+0000 7f0c36200640 1 --2- 172.21.15.1:0/4138450575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c3019a7d0 0x7f0c3019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.446+0000 7f0c35800640 1 -- 172.21.15.1:0/4138450575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c3019a7d0 msgr2=0x7f0c3019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.446+0000 7f0c35800640 1 --2- 172.21.15.1:0/4138450575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c3019a7d0 0x7f0c3019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.446+0000 7f0c35800640 1 -- 172.21.15.1:0/4138450575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c30109490 msgr2=0x7f0c301a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.446+0000 7f0c35800640 1 --2- 172.21.15.1:0/4138450575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c30109490 0x7f0c301a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.446+0000 7f0c35800640 1 -- 172.21.15.1:0/4138450575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3019b360 con 0x7f0c301043c0 2024-04-26T01:59:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.446+0000 7f0c34e00640 1 --2- 172.21.15.1:0/4138450575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c30109490 0x7f0c301a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.447+0000 7f0c35800640 1 --2- 172.21.15.1:0/4138450575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c301043c0 0x7f0c301a0610 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400e9f0 tx=0x7f0c2400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.447+0000 7f0c2e000640 1 -- 172.21.15.1:0/4138450575 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2400cd80 con 0x7f0c301043c0 2024-04-26T01:59:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.447+0000 7f0c38200640 1 -- 172.21.15.1:0/4138450575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3019b650 con 0x7f0c301043c0 2024-04-26T01:59:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.447+0000 7f0c2e000640 1 -- 172.21.15.1:0/4138450575 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c240108e0 con 0x7f0c301043c0 2024-04-26T01:59:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.447+0000 7f0c2e000640 1 -- 172.21.15.1:0/4138450575 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24018980 con 0x7f0c301043c0 2024-04-26T01:59:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.447+0000 7f0c38200640 1 -- 172.21.15.1:0/4138450575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c301a7830 con 0x7f0c301043c0 2024-04-26T01:59:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.451+0000 7f0c2e000640 1 -- 172.21.15.1:0/4138450575 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f0c24010430 con 0x7f0c301043c0 2024-04-26T01:59:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.451+0000 7f0c3aff3740 1 Processor -- start 2024-04-26T01:59:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.451+0000 7f0c3aff3740 1 -- start start 2024-04-26T01:59:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.451+0000 7f0c3aff3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca71f6820 0x558ca71f6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.451+0000 7f0c3aff3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca71f7140 0x558ca72014c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.452+0000 7f0c3aff3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca7201a00 0x558ca7203df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.452+0000 7f0c3aff3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558ca715c900 con 0x558ca71f7140 2024-04-26T01:59:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.452+0000 7f0c3aff3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558ca7102ef0 con 0x558ca71f6820 2024-04-26T01:59:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.452+0000 7f0c3aff3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558ca71588c0 con 0x558ca7201a00 2024-04-26T01:59:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.452+0000 7f0c35800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca71f7140 0x558ca72014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.452+0000 7f0c34e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca7201a00 0x558ca7203df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.452+0000 7f0c36200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca71f6820 0x558ca71f6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.452+0000 7f0c35800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca71f7140 0x558ca72014c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41852/0 (socket says 172.21.15.1:41852) 2024-04-26T01:59:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.452+0000 7f0c35800640 1 -- 172.21.15.1:0/3404818462 learned_addr learned my addr 172.21.15.1:0/3404818462 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.452+0000 7f0c2e000640 1 --2- 172.21.15.1:0/4138450575 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0c180a3d10 0x7f0c180a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.452+0000 7f0c34e00640 1 -- 172.21.15.1:0/3404818462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca71f6820 msgr2=0x558ca71f6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.452+0000 7f0c34e00640 1 --2- 172.21.15.1:0/3404818462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca71f6820 0x558ca71f6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.453+0000 7f0c34e00640 1 -- 172.21.15.1:0/3404818462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca71f7140 msgr2=0x558ca72014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.453+0000 7f0c34e00640 1 --2- 172.21.15.1:0/3404818462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca71f7140 0x558ca72014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.453+0000 7f0c34e00640 1 -- 172.21.15.1:0/3404818462 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ca6e6f210 con 0x558ca7201a00 2024-04-26T01:59:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.453+0000 7f0c2e000640 1 -- 172.21.15.1:0/4138450575 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f0c240d0c20 con 0x7f0c301043c0 2024-04-26T01:59:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.453+0000 7f0c36200640 1 --2- 172.21.15.1:0/4138450575 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0c180a3d10 0x7f0c180a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.453+0000 7f0c34e00640 1 --2- 172.21.15.1:0/3404818462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca7201a00 0x558ca7203df0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0c3019bad0 tx=0x7f0c1c0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85861ece3a8aa60d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.453+0000 7f0c22a00640 1 -- 172.21.15.1:0/3404818462 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c009e40 con 0x558ca7201a00 2024-04-26T01:59:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.453+0000 7f0c22a00640 1 -- 172.21.15.1:0/3404818462 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c1c012900 con 0x558ca7201a00 2024-04-26T01:59:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.453+0000 7f0c22a00640 1 -- 172.21.15.1:0/3404818462 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c00f950 con 0x558ca7201a00 2024-04-26T01:59:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.454+0000 7f0c3aff3740 1 -- 172.21.15.1:0/3404818462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca7201a00 msgr2=0x558ca7203df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.454+0000 7f0c36200640 1 --2- 172.21.15.1:0/4138450575 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0c180a3d10 0x7f0c180a61d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0c28006010 tx=0x7f0c280095c0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.454+0000 7f0c3aff3740 1 --2- 172.21.15.1:0/3404818462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca7201a00 0x558ca7203df0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0c3019bad0 tx=0x7f0c1c0117b0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.454+0000 7f0c3aff3740 1 -- 172.21.15.1:0/3404818462 shutdown_connections 2024-04-26T01:59:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.454+0000 7f0c3aff3740 1 --2- 172.21.15.1:0/3404818462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca7201a00 0x558ca7203df0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.454+0000 7f0c3aff3740 1 --2- 172.21.15.1:0/3404818462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca71f7140 0x558ca72014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.454+0000 7f0c3aff3740 1 --2- 172.21.15.1:0/3404818462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca71f6820 0x558ca71f6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.454+0000 7f0c3aff3740 1 -- 172.21.15.1:0/3404818462 >> 172.21.15.1:0/3404818462 conn(0x558ca71f2d30 msgr2=0x558ca71f3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.455+0000 7f0c3aff3740 1 -- 172.21.15.1:0/3404818462 shutdown_connections 2024-04-26T01:59:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.455+0000 7f0c3aff3740 1 -- 172.21.15.1:0/3404818462 wait complete. 2024-04-26T01:59:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.455+0000 7f0c3aff3740 1 -- start start 2024-04-26T01:59:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.455+0000 7f0c3aff3740 1 Processor -- start 2024-04-26T01:59:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.456+0000 7f0c3aff3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca71f6820 0x558ca71fe9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.456+0000 7f0c3aff3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca71f7140 0x558ca71feef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.456+0000 7f0c3aff3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca7201a00 0x558ca71ff430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.456+0000 7f0c3aff3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558ca7204630 con 0x558ca71f7140 2024-04-26T01:59:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.456+0000 7f0c3aff3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558ca7116310 con 0x558ca7201a00 2024-04-26T01:59:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.456+0000 7f0c3aff3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558ca72047b0 con 0x558ca71f6820 2024-04-26T01:59:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.456+0000 7f0c3aff3740 10 client.0 _ll_register_callbacks cb 0x558ca70e4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.456+0000 7f0c35800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca71f6820 0x558ca71fe9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.456+0000 7f0c34e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca71f7140 0x558ca71feef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.456+0000 7f0c36200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca7201a00 0x558ca71ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.456+0000 7f0c35800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca71f6820 0x558ca71fe9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:46768/0 (socket says 172.21.15.1:46768) 2024-04-26T01:59:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.456+0000 7f0c35800640 1 -- 172.21.15.1:0/3567029756 learned_addr learned my addr 172.21.15.1:0/3567029756 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.456+0000 7f0c35800640 1 -- 172.21.15.1:0/3567029756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca7201a00 msgr2=0x558ca71ff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.456+0000 7f0c35800640 1 --2- 172.21.15.1:0/3567029756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca7201a00 0x558ca71ff430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.456+0000 7f0c35800640 1 -- 172.21.15.1:0/3567029756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca71f7140 msgr2=0x558ca71feef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.457+0000 7f0c35800640 1 --2- 172.21.15.1:0/3567029756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca71f7140 0x558ca71feef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.457+0000 7f0c35800640 1 -- 172.21.15.1:0/3567029756 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ca6f3b620 con 0x558ca71f6820 2024-04-26T01:59:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.457+0000 7f0c35800640 1 --2- 172.21.15.1:0/3567029756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca71f6820 0x558ca71fe9b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0c24098f90 tx=0x7f0c2400cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46650a548d7f88bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.457+0000 7f0c17400640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:59:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.457+0000 7f0c17400640 1 -- 172.21.15.1:0/3567029756 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2409c490 con 0x558ca71f6820 2024-04-26T01:59:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.457+0000 7f0c17400640 1 -- 172.21.15.1:0/3567029756 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f0c240991a0 con 0x558ca71f6820 2024-04-26T01:59:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.457+0000 7f0c3aff3740 1 -- 172.21.15.1:0/3567029756 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c1c012900 con 0x558ca71f6820 2024-04-26T01:59:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.457+0000 7f0c17400640 1 -- 172.21.15.1:0/3567029756 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0c24010720 con 0x558ca71f6820 2024-04-26T01:59:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.457+0000 7f0c3aff3740 10 client.26125 fetch_fsmap learned FSMap version 32 2024-04-26T01:59:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.457+0000 7f0c3aff3740 1 -- 172.21.15.1:0/3567029756 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558ca7210910 con 0x558ca71f6820 2024-04-26T01:59:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.458+0000 7f0c17400640 1 -- 172.21.15.1:0/3567029756 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c1c012900 con 0x558ca71f6820 2024-04-26T01:59:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.459+0000 7f0c17400640 1 -- 172.21.15.1:0/3567029756 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f0c240ce560 con 0x558ca71f6820 2024-04-26T01:59:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.459+0000 7f0c3aff3740 10 client.26125 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:59:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.459+0000 7f0c3aff3740 10 client.26125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:59:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.459+0000 7f0c3aff3740 20 client.26125 populate_metadata read hostname 'smithi001' 2024-04-26T01:59:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.459+0000 7f0c3aff3740 1 --2- 172.21.15.1:0/3567029756 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x558ca7211d80 0x558ca7214170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.459+0000 7f0c3aff3740 4 client.26125 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:59:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.459+0000 7f0c3aff3740 1 -- 172.21.15.1:0/3567029756 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ca71708b0 con 0x558ca7211d80 2024-04-26T01:59:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.459+0000 7f0c34e00640 1 --2- 172.21.15.1:0/3567029756 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x558ca7211d80 0x558ca7214170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.460+0000 7f0c34e00640 1 --2- 172.21.15.1:0/3567029756 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x558ca7211d80 0x558ca7214170 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72bf7ec1293b73a server_cookie=f08ea42e0742c82f in_seq=0 out_seq=0 2024-04-26T01:59:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.460+0000 7f0c17400640 10 client.26125 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.460+0000 7f0c17400640 1 -- 172.21.15.1:0/3567029756 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ca71708b0 con 0x558ca7211d80 2024-04-26T01:59:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.460+0000 7f0c17400640 10 client.26125 handle_command_reply: tid=0 2024-04-26T01:59:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.460+0000 7f0c3aff3740 1 client.26125 shutdown 2024-04-26T01:59:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.461+0000 7f0c3aff3740 1 -- 172.21.15.1:0/3567029756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca71f6820 msgr2=0x558ca71fe9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.461+0000 7f0c3aff3740 1 --2- 172.21.15.1:0/3567029756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca71f6820 0x558ca71fe9b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0c24098f90 tx=0x7f0c2400cf20 comp rx=0 tx=0).stop 2024-04-26T01:59:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.461+0000 7f0c3aff3740 1 -- 172.21.15.1:0/3567029756 shutdown_connections 2024-04-26T01:59:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.461+0000 7f0c3aff3740 1 --2- 172.21.15.1:0/3567029756 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x558ca7211d80 0x558ca7214170 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.461+0000 7f0c3aff3740 1 --2- 172.21.15.1:0/3567029756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca7201a00 0x558ca71ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.461+0000 7f0c3aff3740 1 --2- 172.21.15.1:0/3567029756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca71f7140 0x558ca71feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.461+0000 7f0c3aff3740 1 --2- 172.21.15.1:0/3567029756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca71f6820 0x558ca71fe9b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.461+0000 7f0c3aff3740 1 -- 172.21.15.1:0/3567029756 >> 172.21.15.1:0/3567029756 conn(0x558ca71f2d30 msgr2=0x558ca71f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.461+0000 7f0c17400640 0 client.26125 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.461+0000 7f0c3aff3740 1 -- 172.21.15.1:0/3567029756 shutdown_connections 2024-04-26T01:59:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.461+0000 7f0c3aff3740 1 -- 172.21.15.1:0/3567029756 wait complete. 2024-04-26T01:59:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.461+0000 7f0c3aff3740 20 client.26125 trim_cache size 0 max 16384 2024-04-26T01:59:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:30 smithi097 ceph-mon[32853]: pgmap v243: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.0 MiB/s wr, 328 op/s 2024-04-26T01:59:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.485+0000 7f0c3aff3740 1 Processor -- start 2024-04-26T01:59:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.486+0000 7f0c3aff3740 1 -- start start 2024-04-26T01:59:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.486+0000 7f0c3aff3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca71f6820 0x558ca720d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.486+0000 7f0c35800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca71f6820 0x558ca720d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.487+0000 7f0c35800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca71f6820 0x558ca720d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:44170/0 (socket says 172.21.15.1:44170) 2024-04-26T01:59:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.488+0000 7f0c3aff3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca71f7140 0x558ca72199f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.488+0000 7f0c3aff3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca7201a00 0x558ca7219f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.488+0000 7f0c3aff3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301b28b0 con 0x558ca7201a00 2024-04-26T01:59:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.488+0000 7f0c3aff3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301b0180 con 0x558ca71f6820 2024-04-26T01:59:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.488+0000 7f0c3aff3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301adff0 con 0x558ca71f7140 2024-04-26T01:59:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.489+0000 7f0c36200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca7201a00 0x558ca7219f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.489+0000 7f0c36200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca7201a00 0x558ca7219f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41870/0 (socket says 172.21.15.1:41870) 2024-04-26T01:59:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.489+0000 7f0c35800640 1 -- 172.21.15.1:0/3758079674 learned_addr learned my addr 172.21.15.1:0/3758079674 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.490+0000 7f0c34e00640 1 --2- 172.21.15.1:0/3758079674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca71f7140 0x558ca72199f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.490+0000 7f0c35800640 1 -- 172.21.15.1:0/3758079674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca71f7140 msgr2=0x558ca72199f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.490+0000 7f0c35800640 1 --2- 172.21.15.1:0/3758079674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca71f7140 0x558ca72199f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.490+0000 7f0c35800640 1 -- 172.21.15.1:0/3758079674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca7201a00 msgr2=0x558ca7219f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.491+0000 7f0c35800640 1 --2- 172.21.15.1:0/3758079674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca7201a00 0x558ca7219f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.491+0000 7f0c35800640 1 -- 172.21.15.1:0/3758079674 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ca71fcf40 con 0x558ca71f6820 2024-04-26T01:59:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.491+0000 7f0c34e00640 1 --2- 172.21.15.1:0/3758079674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca71f7140 0x558ca72199f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:59:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.491+0000 7f0c36200640 1 --2- 172.21.15.1:0/3758079674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca7201a00 0x558ca7219f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.491+0000 7f0c35800640 1 --2- 172.21.15.1:0/3758079674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca71f6820 0x558ca720d4f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0c240097b0 tx=0x7f0c240bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3c7a581816ad7b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.492+0000 7f0c22000640 1 -- 172.21.15.1:0/3758079674 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2409de60 con 0x558ca71f6820 2024-04-26T01:59:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.492+0000 7f0c22000640 1 -- 172.21.15.1:0/3758079674 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c2409d230 con 0x558ca71f6820 2024-04-26T01:59:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.492+0000 7f0c22000640 1 -- 172.21.15.1:0/3758079674 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24010ba0 con 0x558ca71f6820 2024-04-26T01:59:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.492+0000 7f0c3aff3740 1 -- 172.21.15.1:0/3758079674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca71f6820 msgr2=0x558ca720d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.492+0000 7f0c3aff3740 1 --2- 172.21.15.1:0/3758079674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca71f6820 0x558ca720d4f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0c240097b0 tx=0x7f0c240bd040 comp rx=0 tx=0).stop 2024-04-26T01:59:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.493+0000 7f0c3aff3740 1 -- 172.21.15.1:0/3758079674 shutdown_connections 2024-04-26T01:59:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.493+0000 7f0c3aff3740 1 --2- 172.21.15.1:0/3758079674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca7201a00 0x558ca7219f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.493+0000 7f0c3aff3740 1 --2- 172.21.15.1:0/3758079674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca71f7140 0x558ca72199f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.493+0000 7f0c3aff3740 1 --2- 172.21.15.1:0/3758079674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca71f6820 0x558ca720d4f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.493+0000 7f0c3aff3740 1 -- 172.21.15.1:0/3758079674 >> 172.21.15.1:0/3758079674 conn(0x558ca71f2d30 msgr2=0x558ca71f9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.493+0000 7f0c3aff3740 1 -- 172.21.15.1:0/3758079674 shutdown_connections 2024-04-26T01:59:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.493+0000 7f0c3aff3740 1 -- 172.21.15.1:0/3758079674 wait complete. 2024-04-26T01:59:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.493+0000 7f0c3aff3740 1 -- start start 2024-04-26T01:59:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.493+0000 7f0c3aff3740 1 Processor -- start 2024-04-26T01:59:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.494+0000 7f0c3aff3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca71f6820 0x558ca7207470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.494+0000 7f0c3aff3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca71f7140 0x558ca72079b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.494+0000 7f0c3aff3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca7201a00 0x558ca7207ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.494+0000 7f0c3aff3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301ae640 con 0x558ca71f6820 2024-04-26T01:59:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.494+0000 7f0c3aff3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301afab0 con 0x558ca7201a00 2024-04-26T01:59:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.494+0000 7f0c3aff3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0c301aed10 con 0x558ca71f7140 2024-04-26T01:59:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.494+0000 7f0c3aff3740 10 client.0 _ll_register_callbacks cb 0x7f0c240ce280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.494+0000 7f0c34e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca71f7140 0x558ca72079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.494+0000 7f0c36200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca7201a00 0x558ca7207ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.494+0000 7f0c34e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca71f7140 0x558ca72079b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:46788/0 (socket says 172.21.15.1:46788) 2024-04-26T01:59:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.494+0000 7f0c34e00640 1 -- 172.21.15.1:0/1231142931 learned_addr learned my addr 172.21.15.1:0/1231142931 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.494+0000 7f0c35800640 1 --2- 172.21.15.1:0/1231142931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca71f6820 0x558ca7207470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.495+0000 7f0c36200640 1 -- 172.21.15.1:0/1231142931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca71f7140 msgr2=0x558ca72079b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.495+0000 7f0c36200640 1 --2- 172.21.15.1:0/1231142931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca71f7140 0x558ca72079b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.495+0000 7f0c36200640 1 -- 172.21.15.1:0/1231142931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca71f6820 msgr2=0x558ca7207470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.495+0000 7f0c36200640 1 --2- 172.21.15.1:0/1231142931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca71f6820 0x558ca7207470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.495+0000 7f0c36200640 1 -- 172.21.15.1:0/1231142931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ca7277270 con 0x558ca7201a00 2024-04-26T01:59:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.495+0000 7f0c35800640 1 --2- 172.21.15.1:0/1231142931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca71f6820 0x558ca7207470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.495+0000 7f0c36200640 1 --2- 172.21.15.1:0/1231142931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca7201a00 0x558ca7207ef0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0c28005e30 tx=0x7f0c2802a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37eb8db9b39de26d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.495+0000 7f0c17400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:59:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.495+0000 7f0c17400640 1 -- 172.21.15.1:0/1231142931 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2802fa20 con 0x558ca7201a00 2024-04-26T01:59:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.495+0000 7f0c17400640 1 -- 172.21.15.1:0/1231142931 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f0c2802fbc0 con 0x558ca7201a00 2024-04-26T01:59:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.495+0000 7f0c3aff3740 1 -- 172.21.15.1:0/1231142931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c2409d230 con 0x558ca7201a00 2024-04-26T01:59:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.496+0000 7f0c17400640 1 -- 172.21.15.1:0/1231142931 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0c28062050 con 0x558ca7201a00 2024-04-26T01:59:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.496+0000 7f0c3aff3740 10 client.26114 fetch_fsmap learned FSMap version 32 2024-04-26T01:59:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.496+0000 7f0c3aff3740 1 -- 172.21.15.1:0/1231142931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558ca7278470 con 0x558ca7201a00 2024-04-26T01:59:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.496+0000 7f0c17400640 1 -- 172.21.15.1:0/1231142931 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c2409d230 con 0x558ca7201a00 2024-04-26T01:59:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.496+0000 7f0c17400640 1 -- 172.21.15.1:0/1231142931 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f0c28054480 con 0x558ca7201a00 2024-04-26T01:59:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.497+0000 7f0c3aff3740 10 client.26114 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:59:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.497+0000 7f0c3aff3740 10 client.26114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:59:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.497+0000 7f0c3aff3740 20 client.26114 populate_metadata read hostname 'smithi001' 2024-04-26T01:59:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.497+0000 7f0c3aff3740 1 --2- 172.21.15.1:0/1231142931 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x558ca7278bf0 0x558ca727b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.497+0000 7f0c3aff3740 4 client.26114 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:59:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.497+0000 7f0c3aff3740 1 -- 172.21.15.1:0/1231142931 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558ca716a320 con 0x558ca7278bf0 2024-04-26T01:59:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.497+0000 7f0c35800640 1 --2- 172.21.15.1:0/1231142931 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x558ca7278bf0 0x558ca727b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.498+0000 7f0c35800640 1 --2- 172.21.15.1:0/1231142931 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x558ca7278bf0 0x558ca727b010 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=570f9a97a69fd7d7 server_cookie=a5d075909a6da48b in_seq=0 out_seq=0 2024-04-26T01:59:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.498+0000 7f0c17400640 10 client.26114 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.510+0000 7f0c17400640 1 -- 172.21.15.1:0/1231142931 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558ca716a320 con 0x558ca7278bf0 2024-04-26T01:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.510+0000 7f0c17400640 10 client.26114 handle_command_reply: tid=0 2024-04-26T01:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.510+0000 7f0c3aff3740 1 client.26114 shutdown 2024-04-26T01:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.511+0000 7f0c3aff3740 1 -- 172.21.15.1:0/1231142931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca7201a00 msgr2=0x558ca7207ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.511+0000 7f0c3aff3740 1 --2- 172.21.15.1:0/1231142931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca7201a00 0x558ca7207ef0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0c28005e30 tx=0x7f0c2802a8c0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.511+0000 7f0c3aff3740 1 -- 172.21.15.1:0/1231142931 shutdown_connections 2024-04-26T01:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.511+0000 7f0c3aff3740 1 --2- 172.21.15.1:0/1231142931 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x558ca7278bf0 0x558ca727b010 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.511+0000 7f0c3aff3740 1 --2- 172.21.15.1:0/1231142931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ca7201a00 0x558ca7207ef0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.511+0000 7f0c3aff3740 1 --2- 172.21.15.1:0/1231142931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ca71f7140 0x558ca72079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.511+0000 7f0c3aff3740 1 --2- 172.21.15.1:0/1231142931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ca71f6820 0x558ca7207470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.511+0000 7f0c3aff3740 1 -- 172.21.15.1:0/1231142931 >> 172.21.15.1:0/1231142931 conn(0x558ca71f2d30 msgr2=0x558ca71f1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.511+0000 7f0c17400640 0 client.26114 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.511+0000 7f0c3aff3740 1 -- 172.21.15.1:0/1231142931 shutdown_connections 2024-04-26T01:59:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.511+0000 7f0c3aff3740 1 -- 172.21.15.1:0/1231142931 wait complete. 2024-04-26T01:59:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.511+0000 7f0c3aff3740 20 client.26114 trim_cache size 0 max 16384 2024-04-26T01:59:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:59:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:59:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:59:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.512+0000 7f0c38200640 1 -- 172.21.15.1:0/4138450575 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0c180a3d10 msgr2=0x7f0c180a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.512+0000 7f0c38200640 1 --2- 172.21.15.1:0/4138450575 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0c180a3d10 0x7f0c180a61d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0c28006010 tx=0x7f0c280095c0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.512+0000 7f0c38200640 1 -- 172.21.15.1:0/4138450575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c301043c0 msgr2=0x7f0c301a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.512+0000 7f0c38200640 1 --2- 172.21.15.1:0/4138450575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c301043c0 0x7f0c301a0610 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400e9f0 tx=0x7f0c2400eeb0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.512+0000 7f0c38200640 1 -- 172.21.15.1:0/4138450575 shutdown_connections 2024-04-26T01:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.512+0000 7f0c38200640 1 --2- 172.21.15.1:0/4138450575 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0c180a3d10 0x7f0c180a61d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.512+0000 7f0c38200640 1 --2- 172.21.15.1:0/4138450575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c3019a7d0 0x7f0c3019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.512+0000 7f0c38200640 1 --2- 172.21.15.1:0/4138450575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c30109490 0x7f0c301a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.512+0000 7f0c38200640 1 --2- 172.21.15.1:0/4138450575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c301043c0 0x7f0c301a0610 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.512+0000 7f0c38200640 1 -- 172.21.15.1:0/4138450575 >> 172.21.15.1:0/4138450575 conn(0x7f0c300ffc40 msgr2=0x7f0c30101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.513+0000 7f0c38200640 1 -- 172.21.15.1:0/4138450575 shutdown_connections 2024-04-26T01:59:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:30.513+0000 7f0c38200640 1 -- 172.21.15.1:0/4138450575 wait complete. 2024-04-26T01:59:30.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:30 smithi173 ceph-mon[32553]: pgmap v243: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.0 MiB/s wr, 328 op/s 2024-04-26T01:59:31.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.172+0000 7f973b000700 1 -- 172.21.15.1:0/4208396098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9734108a80 msgr2=0x7f9734108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:31.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.172+0000 7f973b000700 1 --2- 172.21.15.1:0/4208396098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9734108a80 0x7f9734108e60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9724009990 tx=0x7f9724009ca0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.172+0000 7f973b000700 1 -- 172.21.15.1:0/4208396098 shutdown_connections 2024-04-26T01:59:31.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.172+0000 7f973b000700 1 --2- 172.21.15.1:0/4208396098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97341034b0 0x7f973410f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.172+0000 7f973b000700 1 --2- 172.21.15.1:0/4208396098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9734108a80 0x7f9734108e60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.172+0000 7f973b000700 1 --2- 172.21.15.1:0/4208396098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9734102af0 0x7f9734102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.172+0000 7f973b000700 1 -- 172.21.15.1:0/4208396098 >> 172.21.15.1:0/4208396098 conn(0x7f9734079b20 msgr2=0x7f9734079f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:31.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.172+0000 7f973b000700 1 -- 172.21.15.1:0/4208396098 shutdown_connections 2024-04-26T01:59:31.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.172+0000 7f973b000700 1 -- 172.21.15.1:0/4208396098 wait complete. 2024-04-26T01:59:31.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.173+0000 7f973b000700 1 Processor -- start 2024-04-26T01:59:31.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.173+0000 7f973b000700 1 -- start start 2024-04-26T01:59:31.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.173+0000 7f973b000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9734102af0 0x7f973419b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:31.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.173+0000 7f973b000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97341034b0 0x7f973419b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:31.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.173+0000 7f973b000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9734108a80 0x7f973419fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:31.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.173+0000 7f973b000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9734112c10 con 0x7f97341034b0 2024-04-26T01:59:31.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.174+0000 7f973b000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9734112a90 con 0x7f9734108a80 2024-04-26T01:59:31.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.174+0000 7f973b000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9734112d90 con 0x7f9734102af0 2024-04-26T01:59:31.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.174+0000 7f9733400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97341034b0 0x7f973419b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:31.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.174+0000 7f9733e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9734102af0 0x7f973419b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:31.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.174+0000 7f9733400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97341034b0 0x7f973419b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41890/0 (socket says 172.21.15.1:41890) 2024-04-26T01:59:31.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.174+0000 7f9733e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9734102af0 0x7f973419b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:46814/0 (socket says 172.21.15.1:46814) 2024-04-26T01:59:31.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.174+0000 7f9733400700 1 -- 172.21.15.1:0/3143756022 learned_addr learned my addr 172.21.15.1:0/3143756022 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:31.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.174+0000 7f9739000700 1 --2- 172.21.15.1:0/3143756022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9734108a80 0x7f973419fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:31.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.174+0000 7f9733e00700 1 -- 172.21.15.1:0/3143756022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9734108a80 msgr2=0x7f973419fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:31.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.174+0000 7f9733e00700 1 --2- 172.21.15.1:0/3143756022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9734108a80 0x7f973419fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.174+0000 7f9733e00700 1 -- 172.21.15.1:0/3143756022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97341034b0 msgr2=0x7f973419b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:31.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.174+0000 7f9733e00700 1 --2- 172.21.15.1:0/3143756022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97341034b0 0x7f973419b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.175+0000 7f9733e00700 1 -- 172.21.15.1:0/3143756022 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97341a0200 con 0x7f9734102af0 2024-04-26T01:59:31.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.175+0000 7f9733400700 1 --2- 172.21.15.1:0/3143756022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97341034b0 0x7f973419b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:31.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.175+0000 7f9733e00700 1 --2- 172.21.15.1:0/3143756022 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9734102af0 0x7f973419b1f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f97240059d0 tx=0x7f97240047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:31.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.175+0000 7f9730c00700 1 -- 172.21.15.1:0/3143756022 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724027070 con 0x7f9734102af0 2024-04-26T01:59:31.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.175+0000 7f9730c00700 1 -- 172.21.15.1:0/3143756022 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9724022470 con 0x7f9734102af0 2024-04-26T01:59:31.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.175+0000 7f9730c00700 1 -- 172.21.15.1:0/3143756022 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724017650 con 0x7f9734102af0 2024-04-26T01:59:31.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.175+0000 7f973b000700 1 -- 172.21.15.1:0/3143756022 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97341a0490 con 0x7f9734102af0 2024-04-26T01:59:31.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.175+0000 7f973b000700 1 -- 172.21.15.1:0/3143756022 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97341a08f0 con 0x7f9734102af0 2024-04-26T01:59:31.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.178+0000 7f9730c00700 1 -- 172.21.15.1:0/3143756022 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f972402a070 con 0x7f9734102af0 2024-04-26T01:59:31.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.180+0000 7f9730c00700 1 --2- 172.21.15.1:0/3143756022 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f97180a3ed0 0x7f97180a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:31.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.180+0000 7f9730c00700 1 -- 172.21.15.1:0/3143756022 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f97240d13d0 con 0x7f9734102af0 2024-04-26T01:59:31.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.180+0000 7f974596fb80 1 Processor -- start 2024-04-26T01:59:31.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.180+0000 7f9733400700 1 --2- 172.21.15.1:0/3143756022 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f97180a3ed0 0x7f97180a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:31.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.180+0000 7f974596fb80 1 -- start start 2024-04-26T01:59:31.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.181+0000 7f974596fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef1998dd0 0x563ef19991b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:31.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.181+0000 7f974596fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef19996f0 0x563ef19ac830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:31.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.181+0000 7f974596fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef19acd70 0x563ef19af160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:31.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.181+0000 7f974596fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563ef18ec020 con 0x563ef19996f0 2024-04-26T01:59:31.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.181+0000 7f974596fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563ef18928c0 con 0x563ef19acd70 2024-04-26T01:59:31.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.181+0000 7f974596fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563ef18e7fe0 con 0x563ef1998dd0 2024-04-26T01:59:31.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.181+0000 7f9733400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef19996f0 0x563ef19ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:31.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.181+0000 7f9739000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef19acd70 0x563ef19af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:31.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.181+0000 7f9733e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef1998dd0 0x563ef19991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:31.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.181+0000 7f9739000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef19acd70 0x563ef19af160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:44188/0 (socket says 172.21.15.1:44188) 2024-04-26T01:59:31.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.181+0000 7f9739000700 1 -- 172.21.15.1:0/494430513 learned_addr learned my addr 172.21.15.1:0/494430513 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:31.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.182+0000 7f9733e00700 1 -- 172.21.15.1:0/494430513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef19acd70 msgr2=0x563ef19af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:31.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.182+0000 7f9733e00700 1 --2- 172.21.15.1:0/494430513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef19acd70 0x563ef19af160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.182+0000 7f9733e00700 1 -- 172.21.15.1:0/494430513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef19996f0 msgr2=0x563ef19ac830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:31.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.182+0000 7f9733e00700 1 --2- 172.21.15.1:0/494430513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef19996f0 0x563ef19ac830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.182+0000 7f9733e00700 1 -- 172.21.15.1:0/494430513 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ef1888810 con 0x563ef1998dd0 2024-04-26T01:59:31.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.182+0000 7f9733400700 1 --2- 172.21.15.1:0/494430513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef19996f0 0x563ef19ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:59:31.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.182+0000 7f9733e00700 1 --2- 172.21.15.1:0/494430513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef1998dd0 0x563ef19991b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f97240cea90 tx=0x7f972409cf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4eeaf3e50ae184a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:31.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.183+0000 7f9733400700 1 --2- 172.21.15.1:0/3143756022 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f97180a3ed0 0x7f97180a6390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f973407a3c0 tx=0x7f9720011950 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:31.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.183+0000 7f9729600700 1 -- 172.21.15.1:0/494430513 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972409da70 con 0x563ef1998dd0 2024-04-26T01:59:31.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.183+0000 7f9729600700 1 -- 172.21.15.1:0/494430513 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f972409d370 con 0x563ef1998dd0 2024-04-26T01:59:31.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.183+0000 7f9729600700 1 -- 172.21.15.1:0/494430513 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972409d5d0 con 0x563ef1998dd0 2024-04-26T01:59:31.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.184+0000 7f974596fb80 1 -- 172.21.15.1:0/494430513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef1998dd0 msgr2=0x563ef19991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:31.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.184+0000 7f974596fb80 1 --2- 172.21.15.1:0/494430513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef1998dd0 0x563ef19991b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f97240cea90 tx=0x7f972409cf30 comp rx=0 tx=0).stop 2024-04-26T01:59:31.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.184+0000 7f974596fb80 1 -- 172.21.15.1:0/494430513 shutdown_connections 2024-04-26T01:59:31.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.184+0000 7f974596fb80 1 --2- 172.21.15.1:0/494430513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef19acd70 0x563ef19af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.184+0000 7f974596fb80 1 --2- 172.21.15.1:0/494430513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef1998dd0 0x563ef19991b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.184+0000 7f974596fb80 1 --2- 172.21.15.1:0/494430513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef19996f0 0x563ef19ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.184+0000 7f974596fb80 1 -- 172.21.15.1:0/494430513 >> 172.21.15.1:0/494430513 conn(0x563ef18c80e0 msgr2=0x563ef19952e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:31.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.184+0000 7f974596fb80 1 -- 172.21.15.1:0/494430513 shutdown_connections 2024-04-26T01:59:31.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.184+0000 7f974596fb80 1 -- 172.21.15.1:0/494430513 wait complete. 2024-04-26T01:59:31.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.185+0000 7f974596fb80 1 -- start start 2024-04-26T01:59:31.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.185+0000 7f974596fb80 1 Processor -- start 2024-04-26T01:59:31.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.185+0000 7f974596fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef1998dd0 0x563ef19ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:31.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.185+0000 7f974596fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef19996f0 0x563ef19a97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:31.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.185+0000 7f974596fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef19acd70 0x563ef19a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:31.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.185+0000 7f974596fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563ef18e7fe0 con 0x563ef19996f0 2024-04-26T01:59:31.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.185+0000 7f974596fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563ef18a59f0 con 0x563ef1998dd0 2024-04-26T01:59:31.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.185+0000 7f974596fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563ef19af6a0 con 0x563ef19acd70 2024-04-26T01:59:31.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.185+0000 7f974596fb80 10 client.0 _ll_register_callbacks cb 0x563ef18848b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:31.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.186+0000 7f9733400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef19996f0 0x563ef19a97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:31.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.186+0000 7f9733e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef1998dd0 0x563ef19ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:31.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.186+0000 7f9733400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef19996f0 0x563ef19a97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41906/0 (socket says 172.21.15.1:41906) 2024-04-26T01:59:31.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.186+0000 7f9733e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef1998dd0 0x563ef19ac4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:44194/0 (socket says 172.21.15.1:44194) 2024-04-26T01:59:31.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.186+0000 7f9733400700 1 -- 172.21.15.1:0/693240467 learned_addr learned my addr 172.21.15.1:0/693240467 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:31.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.186+0000 7f9739000700 1 --2- 172.21.15.1:0/693240467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef19acd70 0x563ef19a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:31.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.186+0000 7f9733e00700 1 -- 172.21.15.1:0/693240467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef19acd70 msgr2=0x563ef19a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:31.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.186+0000 7f9733e00700 1 --2- 172.21.15.1:0/693240467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef19acd70 0x563ef19a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.186+0000 7f9733e00700 1 -- 172.21.15.1:0/693240467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef19996f0 msgr2=0x563ef19a97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:31.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.186+0000 7f9733e00700 1 --2- 172.21.15.1:0/693240467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef19996f0 0x563ef19a97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.186+0000 7f9733e00700 1 -- 172.21.15.1:0/693240467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ef1839a60 con 0x563ef1998dd0 2024-04-26T01:59:31.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.186+0000 7f9733400700 1 --2- 172.21.15.1:0/693240467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef19996f0 0x563ef19a97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:31.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.186+0000 7f9739000700 1 --2- 172.21.15.1:0/693240467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef19acd70 0x563ef19a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:31.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.187+0000 7f9733e00700 1 --2- 172.21.15.1:0/693240467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef1998dd0 0x563ef19ac4e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f97240a6110 tx=0x7f972409cfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13602afe4fccfe98 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:31.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.187+0000 7f971e000700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:59:31.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.188+0000 7f971e000700 1 -- 172.21.15.1:0/693240467 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97240a1720 con 0x563ef1998dd0 2024-04-26T01:59:31.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.188+0000 7f971e000700 1 -- 172.21.15.1:0/693240467 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f972409d5d0 con 0x563ef1998dd0 2024-04-26T01:59:31.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.188+0000 7f974596fb80 1 -- 172.21.15.1:0/693240467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f972409d370 con 0x563ef1998dd0 2024-04-26T01:59:31.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.188+0000 7f971e000700 1 -- 172.21.15.1:0/693240467 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f972409d7d0 con 0x563ef1998dd0 2024-04-26T01:59:31.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.188+0000 7f974596fb80 10 client.26126 fetch_fsmap learned FSMap version 32 2024-04-26T01:59:31.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.188+0000 7f974596fb80 1 -- 172.21.15.1:0/693240467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563ef1882c80 con 0x563ef1998dd0 2024-04-26T01:59:31.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.188+0000 7f971e000700 1 -- 172.21.15.1:0/693240467 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f972409d370 con 0x563ef1998dd0 2024-04-26T01:59:31.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.189+0000 7f971e000700 1 -- 172.21.15.1:0/693240467 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f972409c7c0 con 0x563ef1998dd0 2024-04-26T01:59:31.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.189+0000 7f974596fb80 10 client.26126 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:59:31.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.189+0000 7f974596fb80 10 client.26126 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:59:31.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.189+0000 7f974596fb80 20 client.26126 populate_metadata read hostname 'smithi001' 2024-04-26T01:59:31.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.189+0000 7f974596fb80 1 --2- 172.21.15.1:0/693240467 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563ef19bc040 0x563ef19be430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:31.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.189+0000 7f974596fb80 4 client.26126 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:59:31.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.189+0000 7f974596fb80 1 -- 172.21.15.1:0/693240467 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ef18f9810 con 0x563ef19bc040 2024-04-26T01:59:31.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.189+0000 7f9733400700 1 --2- 172.21.15.1:0/693240467 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563ef19bc040 0x563ef19be430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:31.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.190+0000 7f9733400700 1 --2- 172.21.15.1:0/693240467 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563ef19bc040 0x563ef19be430 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d4b1d9ae7de4619 server_cookie=3ac82e5d6d98c42 in_seq=0 out_seq=0 2024-04-26T01:59:31.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.190+0000 7f971e000700 10 client.26126 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:31.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.191+0000 7f971e000700 1 -- 172.21.15.1:0/693240467 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ef18f9810 con 0x563ef19bc040 2024-04-26T01:59:31.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.191+0000 7f971e000700 10 client.26126 handle_command_reply: tid=0 2024-04-26T01:59:31.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.191+0000 7f974596fb80 1 client.26126 shutdown 2024-04-26T01:59:31.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.192+0000 7f974596fb80 1 -- 172.21.15.1:0/693240467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef1998dd0 msgr2=0x563ef19ac4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:31.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.192+0000 7f974596fb80 1 --2- 172.21.15.1:0/693240467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef1998dd0 0x563ef19ac4e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f97240a6110 tx=0x7f972409cfc0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.192+0000 7f974596fb80 1 -- 172.21.15.1:0/693240467 shutdown_connections 2024-04-26T01:59:31.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.192+0000 7f974596fb80 1 --2- 172.21.15.1:0/693240467 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563ef19bc040 0x563ef19be430 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.192+0000 7f974596fb80 1 --2- 172.21.15.1:0/693240467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef19acd70 0x563ef19a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.192+0000 7f974596fb80 1 --2- 172.21.15.1:0/693240467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef1998dd0 0x563ef19ac4e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.192+0000 7f974596fb80 1 --2- 172.21.15.1:0/693240467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef19996f0 0x563ef19a97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.192+0000 7f974596fb80 1 -- 172.21.15.1:0/693240467 >> 172.21.15.1:0/693240467 conn(0x563ef18c80e0 msgr2=0x563ef1994ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:31.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.192+0000 7f971e000700 0 client.26126 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:31.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.192+0000 7f974596fb80 1 -- 172.21.15.1:0/693240467 shutdown_connections 2024-04-26T01:59:31.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.192+0000 7f974596fb80 1 -- 172.21.15.1:0/693240467 wait complete. 2024-04-26T01:59:31.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.192+0000 7f974596fb80 20 client.26126 trim_cache size 0 max 16384 2024-04-26T01:59:31.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.219+0000 7f974596fb80 1 Processor -- start 2024-04-26T01:59:31.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.219+0000 7f974596fb80 1 -- start start 2024-04-26T01:59:31.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.219+0000 7f974596fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef1998dd0 0x563ef19a5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:31.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.219+0000 7f974596fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef19996f0 0x563ef19a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:31.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.219+0000 7f974596fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef19acd70 0x563ef19ab720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:31.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.219+0000 7f974596fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f97341ad950 con 0x563ef1998dd0 2024-04-26T01:59:31.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.219+0000 7f974596fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f97341ab1e0 con 0x563ef19996f0 2024-04-26T01:59:31.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.219+0000 7f974596fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f97341a9050 con 0x563ef19acd70 2024-04-26T01:59:31.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.220+0000 7f9733400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef19996f0 0x563ef19a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:31.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.220+0000 7f9733e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef1998dd0 0x563ef19a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:31.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.220+0000 7f9733400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef19996f0 0x563ef19a9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:44200/0 (socket says 172.21.15.1:44200) 2024-04-26T01:59:31.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.220+0000 7f9733400700 1 -- 172.21.15.1:0/3443062919 learned_addr learned my addr 172.21.15.1:0/3443062919 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:31.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.220+0000 7f9739000700 1 --2- 172.21.15.1:0/3443062919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef19acd70 0x563ef19ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:31.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.220+0000 7f9733400700 1 -- 172.21.15.1:0/3443062919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef19acd70 msgr2=0x563ef19ab720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:31.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.220+0000 7f9733400700 1 --2- 172.21.15.1:0/3443062919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef19acd70 0x563ef19ab720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.220+0000 7f9733400700 1 -- 172.21.15.1:0/3443062919 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef1998dd0 msgr2=0x563ef19a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:31.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.220+0000 7f9733400700 1 --2- 172.21.15.1:0/3443062919 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef1998dd0 0x563ef19a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.220+0000 7f9733400700 1 -- 172.21.15.1:0/3443062919 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ef199ac40 con 0x563ef19996f0 2024-04-26T01:59:31.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.220+0000 7f9733e00700 1 --2- 172.21.15.1:0/3443062919 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef1998dd0 0x563ef19a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:31.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.221+0000 7f9733400700 1 --2- 172.21.15.1:0/3443062919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef19996f0 0x563ef19a9c50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f97200094f0 tx=0x7f9720009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef584e4fdf9d5ac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:31.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.221+0000 7f9728c00700 1 -- 172.21.15.1:0/3443062919 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720017040 con 0x563ef19996f0 2024-04-26T01:59:31.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.221+0000 7f9728c00700 1 -- 172.21.15.1:0/3443062919 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9720019050 con 0x563ef19996f0 2024-04-26T01:59:31.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.221+0000 7f9728c00700 1 -- 172.21.15.1:0/3443062919 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972002b130 con 0x563ef19996f0 2024-04-26T01:59:31.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.222+0000 7f974596fb80 1 -- 172.21.15.1:0/3443062919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef19996f0 msgr2=0x563ef19a9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:31.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.222+0000 7f974596fb80 1 --2- 172.21.15.1:0/3443062919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef19996f0 0x563ef19a9c50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f97200094f0 tx=0x7f9720009260 comp rx=0 tx=0).stop 2024-04-26T01:59:31.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.222+0000 7f974596fb80 1 -- 172.21.15.1:0/3443062919 shutdown_connections 2024-04-26T01:59:31.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.222+0000 7f974596fb80 1 --2- 172.21.15.1:0/3443062919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef19acd70 0x563ef19ab720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.222+0000 7f974596fb80 1 --2- 172.21.15.1:0/3443062919 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef1998dd0 0x563ef19a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.222+0000 7f974596fb80 1 --2- 172.21.15.1:0/3443062919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef19996f0 0x563ef19a9c50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.222+0000 7f974596fb80 1 -- 172.21.15.1:0/3443062919 >> 172.21.15.1:0/3443062919 conn(0x563ef18c80e0 msgr2=0x563ef19ab010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:31.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.222+0000 7f974596fb80 1 -- 172.21.15.1:0/3443062919 shutdown_connections 2024-04-26T01:59:31.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.222+0000 7f974596fb80 1 -- 172.21.15.1:0/3443062919 wait complete. 2024-04-26T01:59:31.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.223+0000 7f974596fb80 1 -- start start 2024-04-26T01:59:31.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.223+0000 7f974596fb80 1 Processor -- start 2024-04-26T01:59:31.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.224+0000 7f974596fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef1998dd0 0x563ef19b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:31.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.224+0000 7f974596fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef19996f0 0x563ef19b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:31.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.224+0000 7f974596fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef19acd70 0x563ef1a22490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:31.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.224+0000 7f974596fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f97341a96a0 con 0x563ef19996f0 2024-04-26T01:59:31.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.224+0000 7f974596fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f97341aab10 con 0x563ef19acd70 2024-04-26T01:59:31.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.224+0000 7f974596fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f97341a9d70 con 0x563ef1998dd0 2024-04-26T01:59:31.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.224+0000 7f974596fb80 10 client.0 _ll_register_callbacks cb 0x563ef18848b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:31.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.224+0000 7f9739000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef19acd70 0x563ef1a22490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:31.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.224+0000 7f9733e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef1998dd0 0x563ef19b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:31.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.224+0000 7f9739000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef19acd70 0x563ef1a22490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:44202/0 (socket says 172.21.15.1:44202) 2024-04-26T01:59:31.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.224+0000 7f9739000700 1 -- 172.21.15.1:0/1583889336 learned_addr learned my addr 172.21.15.1:0/1583889336 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:31.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.224+0000 7f9733400700 1 --2- 172.21.15.1:0/1583889336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef19996f0 0x563ef19b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:31.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.224+0000 7f9739000700 1 -- 172.21.15.1:0/1583889336 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef1998dd0 msgr2=0x563ef19b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:31.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.225+0000 7f9739000700 1 --2- 172.21.15.1:0/1583889336 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef1998dd0 0x563ef19b1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.225+0000 7f9739000700 1 -- 172.21.15.1:0/1583889336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef19996f0 msgr2=0x563ef19b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:31.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.225+0000 7f9739000700 1 --2- 172.21.15.1:0/1583889336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef19996f0 0x563ef19b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.225+0000 7f9739000700 1 -- 172.21.15.1:0/1583889336 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ef19b2730 con 0x563ef19acd70 2024-04-26T01:59:31.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.225+0000 7f9733400700 1 --2- 172.21.15.1:0/1583889336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef19996f0 0x563ef19b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:31.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.225+0000 7f9739000700 1 --2- 172.21.15.1:0/1583889336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef19acd70 0x563ef1a22490 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f972c007f00 tx=0x7f972c0176f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d200f1042f9e1f0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:31.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.225+0000 7f971e000700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T01:59:31.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.225+0000 7f971e000700 1 -- 172.21.15.1:0/1583889336 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c003c00 con 0x563ef19acd70 2024-04-26T01:59:31.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.225+0000 7f971e000700 1 -- 172.21.15.1:0/1583889336 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f972c003da0 con 0x563ef19acd70 2024-04-26T01:59:31.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.225+0000 7f974596fb80 1 -- 172.21.15.1:0/1583889336 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9720019050 con 0x563ef19acd70 2024-04-26T01:59:31.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.226+0000 7f971e000700 1 -- 172.21.15.1:0/1583889336 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f972c0145c0 con 0x563ef19acd70 2024-04-26T01:59:31.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.226+0000 7f974596fb80 10 client.26132 fetch_fsmap learned FSMap version 32 2024-04-26T01:59:31.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.226+0000 7f974596fb80 1 -- 172.21.15.1:0/1583889336 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563ef1a23b40 con 0x563ef19acd70 2024-04-26T01:59:31.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.226+0000 7f971e000700 1 -- 172.21.15.1:0/1583889336 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9720019050 con 0x563ef19acd70 2024-04-26T01:59:31.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.226+0000 7f971e000700 1 -- 172.21.15.1:0/1583889336 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f972c026070 con 0x563ef19acd70 2024-04-26T01:59:31.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.226+0000 7f974596fb80 10 client.26132 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:59:31.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.226+0000 7f974596fb80 10 client.26132 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T01:59:31.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.226+0000 7f974596fb80 20 client.26132 populate_metadata read hostname 'smithi001' 2024-04-26T01:59:31.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.226+0000 7f974596fb80 1 --2- 172.21.15.1:0/1583889336 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563ef1a242f0 0x563ef1a26710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:31.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.226+0000 7f974596fb80 4 client.26132 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 82.9, "await_for": 42.0}] 2024-04-26T01:59:31.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.226+0000 7f974596fb80 1 -- 172.21.15.1:0/1583889336 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 82.9, "await_for": 42.0}) -- 0x563ef18f38b0 con 0x563ef1a242f0 2024-04-26T01:59:31.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.227+0000 7f9733e00700 1 --2- 172.21.15.1:0/1583889336 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563ef1a242f0 0x563ef1a26710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:31.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.228+0000 7f9733e00700 1 --2- 172.21.15.1:0/1583889336 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563ef1a242f0 0x563ef1a26710 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71f6d03c3de8e422 server_cookie=5a5e24b8cc02abc5 in_seq=0 out_seq=0 2024-04-26T01:59:31.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:31.228+0000 7f971e000700 10 client.26132 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:32.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.157+0000 7f971e000700 1 -- 172.21.15.1:0/1583889336 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x563ef18f38b0 con 0x563ef1a242f0 2024-04-26T01:59:32.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.157+0000 7f971e000700 10 client.26132 handle_command_reply: tid=0 2024-04-26T01:59:32.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.157+0000 7f974596fb80 1 client.26132 shutdown 2024-04-26T01:59:32.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.158+0000 7f974596fb80 1 -- 172.21.15.1:0/1583889336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef19acd70 msgr2=0x563ef1a22490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:32.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.158+0000 7f974596fb80 1 --2- 172.21.15.1:0/1583889336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef19acd70 0x563ef1a22490 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f972c007f00 tx=0x7f972c0176f0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.158+0000 7f974596fb80 1 -- 172.21.15.1:0/1583889336 shutdown_connections 2024-04-26T01:59:32.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.158+0000 7f974596fb80 1 --2- 172.21.15.1:0/1583889336 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563ef1a242f0 0x563ef1a26710 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.158+0000 7f974596fb80 1 --2- 172.21.15.1:0/1583889336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563ef19acd70 0x563ef1a22490 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.158+0000 7f974596fb80 1 --2- 172.21.15.1:0/1583889336 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563ef1998dd0 0x563ef19b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.158+0000 7f974596fb80 1 --2- 172.21.15.1:0/1583889336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563ef19996f0 0x563ef19b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.158+0000 7f974596fb80 1 -- 172.21.15.1:0/1583889336 >> 172.21.15.1:0/1583889336 conn(0x563ef18c80e0 msgr2=0x563ef19ab8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:32.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.158+0000 7f971e000700 0 client.26132 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:32.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.158+0000 7f974596fb80 1 -- 172.21.15.1:0/1583889336 shutdown_connections 2024-04-26T01:59:32.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.158+0000 7f974596fb80 1 -- 172.21.15.1:0/1583889336 wait complete. 2024-04-26T01:59:32.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.159+0000 7f974596fb80 20 client.26132 trim_cache size 0 max 16384 2024-04-26T01:59:32.158 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T01:59:32.159 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 32, 2024-04-26T01:59:32.159 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24518, 2024-04-26T01:59:32.159 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 22, 2024-04-26T01:59:32.159 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-04-26T01:59:32.159 INFO:teuthology.orchestra.run.smithi001.stdout: "a9fc8138": { 2024-04-26T01:59:32.159 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 22, 2024-04-26T01:59:32.159 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-04-26T01:59:32.159 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:59:32.159 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-04-26T01:59:32.159 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:59:32.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T01:59:32.159 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-04-26T01:59:32.160 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 82.9, 2024-04-26T01:59:32.160 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-04-26T01:59:32.160 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-04-26T01:59:32.160 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-04-26T01:59:32.160 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T01:59:32.160 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-04-26T01:59:32.160 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T01:59:32.160 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:59:32.160 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:59:32.160 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:59:32.160 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:59:32.160 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T01:59:32.160 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T01:59:32.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.160+0000 7f973b000700 1 -- 172.21.15.1:0/3143756022 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f97180a3ed0 msgr2=0x7f97180a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:32.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.160+0000 7f973b000700 1 --2- 172.21.15.1:0/3143756022 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f97180a3ed0 0x7f97180a6390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f973407a3c0 tx=0x7f9720011950 comp rx=0 tx=0).stop 2024-04-26T01:59:32.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.160+0000 7f973b000700 1 -- 172.21.15.1:0/3143756022 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9734102af0 msgr2=0x7f973419b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:32.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.160+0000 7f973b000700 1 --2- 172.21.15.1:0/3143756022 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9734102af0 0x7f973419b1f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f97240059d0 tx=0x7f97240047b0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.161+0000 7f973b000700 1 -- 172.21.15.1:0/3143756022 shutdown_connections 2024-04-26T01:59:32.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.161+0000 7f973b000700 1 --2- 172.21.15.1:0/3143756022 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f97180a3ed0 0x7f97180a6390 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.161+0000 7f973b000700 1 --2- 172.21.15.1:0/3143756022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9734108a80 0x7f973419fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.161+0000 7f973b000700 1 --2- 172.21.15.1:0/3143756022 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9734102af0 0x7f973419b1f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.161+0000 7f973b000700 1 --2- 172.21.15.1:0/3143756022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97341034b0 0x7f973419b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.161+0000 7f973b000700 1 -- 172.21.15.1:0/3143756022 >> 172.21.15.1:0/3143756022 conn(0x7f9734079b20 msgr2=0x7f973410de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:32.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.161+0000 7f973b000700 1 -- 172.21.15.1:0/3143756022 shutdown_connections 2024-04-26T01:59:32.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:32.161+0000 7f973b000700 1 -- 172.21.15.1:0/3143756022 wait complete. 2024-04-26T01:59:32.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:32 smithi097 ceph-mon[32853]: pgmap v244: 97 pgs: 97 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.6 MiB/s wr, 539 op/s 2024-04-26T01:59:32.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:32 smithi173 ceph-mon[32553]: pgmap v244: 97 pgs: 97 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.6 MiB/s wr, 539 op/s 2024-04-26T01:59:32.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:32 smithi001 ceph-mon[27822]: pgmap v244: 97 pgs: 97 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.6 MiB/s wr, 539 op/s 2024-04-26T01:59:33.120 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a9fc8138', quiesce took 8.9 seconds. Will release after: 22.9 2024-04-26T01:59:34.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:34 smithi173 ceph-mon[32553]: pgmap v245: 97 pgs: 97 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.0 MiB/s wr, 362 op/s 2024-04-26T01:59:34.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:34 smithi001 ceph-mon[27822]: pgmap v245: 97 pgs: 97 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.0 MiB/s wr, 362 op/s 2024-04-26T01:59:34.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:34 smithi097 ceph-mon[32853]: pgmap v245: 97 pgs: 97 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.0 MiB/s wr, 362 op/s 2024-04-26T01:59:36.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:36 smithi173 ceph-mon[32553]: pgmap v246: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.2 MiB/s wr, 424 op/s 2024-04-26T01:59:36.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:36 smithi001 ceph-mon[27822]: pgmap v246: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.2 MiB/s wr, 424 op/s 2024-04-26T01:59:36.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:36 smithi097 ceph-mon[32853]: pgmap v246: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.2 MiB/s wr, 424 op/s 2024-04-26T01:59:38.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:38 smithi173 ceph-mon[32553]: pgmap v247: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 735 KiB/s wr, 273 op/s 2024-04-26T01:59:38.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:38 smithi001 ceph-mon[27822]: pgmap v247: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 735 KiB/s wr, 273 op/s 2024-04-26T01:59:38.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:38 smithi097 ceph-mon[32853]: pgmap v247: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 735 KiB/s wr, 273 op/s 2024-04-26T01:59:40.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:40 smithi173 ceph-mon[32553]: pgmap v248: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 735 KiB/s wr, 273 op/s 2024-04-26T01:59:40.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:40 smithi001 ceph-mon[27822]: pgmap v248: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 735 KiB/s wr, 273 op/s 2024-04-26T01:59:40.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:40 smithi097 ceph-mon[32853]: pgmap v248: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 735 KiB/s wr, 273 op/s 2024-04-26T01:59:42.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:42 smithi173 ceph-mon[32553]: pgmap v249: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 737 KiB/s wr, 276 op/s 2024-04-26T01:59:42.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:42 smithi001 ceph-mon[27822]: pgmap v249: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 737 KiB/s wr, 276 op/s 2024-04-26T01:59:42.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:42 smithi097 ceph-mon[32853]: pgmap v249: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 737 KiB/s wr, 276 op/s 2024-04-26T01:59:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.661+0000 7f7755400640 1 -- 172.21.15.1:0/3937731224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77501092f0 msgr2=0x7f7750110ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.661+0000 7f7755400640 1 --2- 172.21.15.1:0/3937731224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77501092f0 0x7f7750110ec0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7748009ca0 tx=0x7f774802a8c0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.661+0000 7f7755400640 1 -- 172.21.15.1:0/3937731224 shutdown_connections 2024-04-26T01:59:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.661+0000 7f7755400640 1 --2- 172.21.15.1:0/3937731224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77501092f0 0x7f7750110ec0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.661+0000 7f7755400640 1 --2- 172.21.15.1:0/3937731224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7750104bd0 0x7f7750108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.661+0000 7f7755400640 1 --2- 172.21.15.1:0/3937731224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7750104220 0x7f7750104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.661+0000 7f7755400640 1 -- 172.21.15.1:0/3937731224 >> 172.21.15.1:0/3937731224 conn(0x7f77500ffa60 msgr2=0x7f7750101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.661+0000 7f7755400640 1 -- 172.21.15.1:0/3937731224 shutdown_connections 2024-04-26T01:59:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.661+0000 7f7755400640 1 -- 172.21.15.1:0/3937731224 wait complete. 2024-04-26T01:59:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.662+0000 7f7755400640 1 Processor -- start 2024-04-26T01:59:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.662+0000 7f7755400640 1 -- start start 2024-04-26T01:59:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.663+0000 7f7755400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7750104220 0x7f77501a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.663+0000 7f7755400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77501092f0 0x7f77501a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.663+0000 7f7755400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f775019a5e0 0x7f775019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.663+0000 7f7755400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7750114240 con 0x7f775019a5e0 2024-04-26T01:59:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.663+0000 7f7755400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f77501140c0 con 0x7f77501092f0 2024-04-26T01:59:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.663+0000 7f7755400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f77501143c0 con 0x7f7750104220 2024-04-26T01:59:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.663+0000 7f7754a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f775019a5e0 0x7f775019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.663+0000 7f7754a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f775019a5e0 0x7f775019aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46786/0 (socket says 172.21.15.1:46786) 2024-04-26T01:59:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.663+0000 7f7754a00640 1 -- 172.21.15.1:0/1967747896 learned_addr learned my addr 172.21.15.1:0/1967747896 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.663+0000 7f774fe00640 1 --2- 172.21.15.1:0/1967747896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7750104220 0x7f77501a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.663+0000 7f7747e00640 1 --2- 172.21.15.1:0/1967747896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77501092f0 0x7f77501a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.664+0000 7f7754a00640 1 -- 172.21.15.1:0/1967747896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7750104220 msgr2=0x7f77501a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.664+0000 7f7754a00640 1 --2- 172.21.15.1:0/1967747896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7750104220 0x7f77501a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.664+0000 7f7754a00640 1 -- 172.21.15.1:0/1967747896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77501092f0 msgr2=0x7f77501a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.664+0000 7f7754a00640 1 --2- 172.21.15.1:0/1967747896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77501092f0 0x7f77501a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.664+0000 7f7754a00640 1 -- 172.21.15.1:0/1967747896 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775019b100 con 0x7f775019a5e0 2024-04-26T01:59:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.664+0000 7f774fe00640 1 --2- 172.21.15.1:0/1967747896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7750104220 0x7f77501a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.664+0000 7f7747e00640 1 --2- 172.21.15.1:0/1967747896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77501092f0 0x7f77501a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.664+0000 7f7754a00640 1 --2- 172.21.15.1:0/1967747896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f775019a5e0 0x7f775019aa90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7748005cc0 tx=0x7f7748031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.664+0000 7f774d600640 1 -- 172.21.15.1:0/1967747896 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748003b70 con 0x7f775019a5e0 2024-04-26T01:59:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.664+0000 7f774d600640 1 -- 172.21.15.1:0/1967747896 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77480318f0 con 0x7f775019a5e0 2024-04-26T01:59:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.664+0000 7f774d600640 1 -- 172.21.15.1:0/1967747896 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774802f950 con 0x7f775019a5e0 2024-04-26T01:59:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.664+0000 7f7755400640 1 -- 172.21.15.1:0/1967747896 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f775019b390 con 0x7f775019a5e0 2024-04-26T01:59:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.665+0000 7f7755400640 1 -- 172.21.15.1:0/1967747896 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77501a7540 con 0x7f775019a5e0 2024-04-26T01:59:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.668+0000 7f77597fb740 1 Processor -- start 2024-04-26T01:59:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.668+0000 7f774d600640 1 -- 172.21.15.1:0/1967747896 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f774802fb30 con 0x7f775019a5e0 2024-04-26T01:59:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.668+0000 7f77597fb740 1 -- start start 2024-04-26T01:59:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.668+0000 7f77597fb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1cc9b0 0x562c7e1ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.668+0000 7f77597fb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cd2d0 0x562c7e1d7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.668+0000 7f77597fb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1d7b90 0x562c7e1d9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.668+0000 7f77597fb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562c7e132900 con 0x562c7e1d7b90 2024-04-26T01:59:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.668+0000 7f77597fb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562c7e0d8ef0 con 0x562c7e1cc9b0 2024-04-26T01:59:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.668+0000 7f77597fb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562c7e12e8c0 con 0x562c7e1cd2d0 2024-04-26T01:59:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.669+0000 7f774fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cd2d0 0x562c7e1d7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.669+0000 7f774fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cd2d0 0x562c7e1d7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:56616/0 (socket says 172.21.15.1:56616) 2024-04-26T01:59:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.669+0000 7f774fe00640 1 -- 172.21.15.1:0/4236714839 learned_addr learned my addr 172.21.15.1:0/4236714839 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.669+0000 7f7747e00640 1 --2- 172.21.15.1:0/4236714839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1cc9b0 0x562c7e1ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.669+0000 7f7747e00640 1 --2- 172.21.15.1:0/4236714839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1d7b90 0x562c7e1d9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.669+0000 7f774fe00640 1 -- 172.21.15.1:0/4236714839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1cc9b0 msgr2=0x562c7e1ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.669+0000 7f774fe00640 1 --2- 172.21.15.1:0/4236714839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1cc9b0 0x562c7e1ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.669+0000 7f774fe00640 1 -- 172.21.15.1:0/4236714839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1d7b90 msgr2=0x562c7e1d9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.669+0000 7f774fe00640 1 --2- 172.21.15.1:0/4236714839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1d7b90 0x562c7e1d9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.669+0000 7f774fe00640 1 -- 172.21.15.1:0/4236714839 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c7de45210 con 0x562c7e1cd2d0 2024-04-26T01:59:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.669+0000 7f7747e00640 1 --2- 172.21.15.1:0/4236714839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1d7b90 0x562c7e1d9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.669+0000 7f774d600640 1 --2- 172.21.15.1:0/1967747896 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f77340a3d00 0x7f77340a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.670+0000 7f774fe00640 1 --2- 172.21.15.1:0/4236714839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cd2d0 0x562c7e1d7650 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f77400115f0 tx=0x7f7740011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9748cec3fcdc7761 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.670+0000 7f774d600640 1 -- 172.21.15.1:0/1967747896 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f77480f1a70 con 0x7f775019a5e0 2024-04-26T01:59:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.670+0000 7f7747e00640 1 --2- 172.21.15.1:0/1967747896 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f77340a3d00 0x7f77340a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.670+0000 7f7745200640 1 -- 172.21.15.1:0/4236714839 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774001e070 con 0x562c7e1cd2d0 2024-04-26T01:59:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.670+0000 7f7745200640 1 -- 172.21.15.1:0/4236714839 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f774000faf0 con 0x562c7e1cd2d0 2024-04-26T01:59:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.670+0000 7f7745200640 1 -- 172.21.15.1:0/4236714839 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7740019bc0 con 0x562c7e1cd2d0 2024-04-26T01:59:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.671+0000 7f77597fb740 1 -- 172.21.15.1:0/4236714839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cd2d0 msgr2=0x562c7e1d7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.671+0000 7f7747e00640 1 --2- 172.21.15.1:0/1967747896 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f77340a3d00 0x7f77340a61c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x562c7e1ca2e0 tx=0x7f7738013040 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.671+0000 7f77597fb740 1 --2- 172.21.15.1:0/4236714839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cd2d0 0x562c7e1d7650 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f77400115f0 tx=0x7f7740011ab0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.671+0000 7f77597fb740 1 -- 172.21.15.1:0/4236714839 shutdown_connections 2024-04-26T01:59:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.671+0000 7f77597fb740 1 --2- 172.21.15.1:0/4236714839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1d7b90 0x562c7e1d9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.671+0000 7f77597fb740 1 --2- 172.21.15.1:0/4236714839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cd2d0 0x562c7e1d7650 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.671+0000 7f77597fb740 1 --2- 172.21.15.1:0/4236714839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1cc9b0 0x562c7e1ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.671+0000 7f77597fb740 1 -- 172.21.15.1:0/4236714839 >> 172.21.15.1:0/4236714839 conn(0x562c7e1c8ec0 msgr2=0x562c7e1c92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.671+0000 7f77597fb740 1 -- 172.21.15.1:0/4236714839 shutdown_connections 2024-04-26T01:59:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.671+0000 7f77597fb740 1 -- 172.21.15.1:0/4236714839 wait complete. 2024-04-26T01:59:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.672+0000 7f77597fb740 1 -- start start 2024-04-26T01:59:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.672+0000 7f77597fb740 1 Processor -- start 2024-04-26T01:59:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.672+0000 7f77597fb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1cc9b0 0x562c7e1d4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.672+0000 7f77597fb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cd2d0 0x562c7e1d5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.672+0000 7f77597fb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1d7b90 0x562c7e1d5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.672+0000 7f77597fb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562c7e1da7c0 con 0x562c7e1cc9b0 2024-04-26T01:59:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.672+0000 7f77597fb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562c7e0ec310 con 0x562c7e1d7b90 2024-04-26T01:59:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.672+0000 7f77597fb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562c7e1da940 con 0x562c7e1cd2d0 2024-04-26T01:59:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.672+0000 7f77597fb740 10 client.0 _ll_register_callbacks cb 0x562c7e0bacb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.673+0000 7f774fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1cc9b0 0x562c7e1d4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.673+0000 7f7747e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cd2d0 0x562c7e1d5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.673+0000 7f774fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1cc9b0 0x562c7e1d4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46804/0 (socket says 172.21.15.1:46804) 2024-04-26T01:59:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.673+0000 7f7747e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cd2d0 0x562c7e1d5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:56624/0 (socket says 172.21.15.1:56624) 2024-04-26T01:59:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.673+0000 7f774fe00640 1 -- 172.21.15.1:0/1042957087 learned_addr learned my addr 172.21.15.1:0/1042957087 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.673+0000 7f7754a00640 1 --2- 172.21.15.1:0/1042957087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1d7b90 0x562c7e1d5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.673+0000 7f774fe00640 1 -- 172.21.15.1:0/1042957087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cd2d0 msgr2=0x562c7e1d5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.673+0000 7f774fe00640 1 --2- 172.21.15.1:0/1042957087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cd2d0 0x562c7e1d5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.673+0000 7f774fe00640 1 -- 172.21.15.1:0/1042957087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1d7b90 msgr2=0x562c7e1d5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.673+0000 7f774fe00640 1 --2- 172.21.15.1:0/1042957087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1d7b90 0x562c7e1d5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.673+0000 7f774fe00640 1 -- 172.21.15.1:0/1042957087 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c7df11620 con 0x562c7e1cc9b0 2024-04-26T01:59:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.673+0000 7f7754a00640 1 --2- 172.21.15.1:0/1042957087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1d7b90 0x562c7e1d5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.673+0000 7f774fe00640 1 --2- 172.21.15.1:0/1042957087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1cc9b0 0x562c7e1d4bc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f774001c040 tx=0x7f7740019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc476141937610f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.673+0000 7f773d600640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:59:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.674+0000 7f773d600640 1 -- 172.21.15.1:0/1042957087 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7740027060 con 0x562c7e1cc9b0 2024-04-26T01:59:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.674+0000 7f773d600640 1 -- 172.21.15.1:0/1042957087 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f774000f640 con 0x562c7e1cc9b0 2024-04-26T01:59:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.674+0000 7f77597fb740 1 -- 172.21.15.1:0/1042957087 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f774000faf0 con 0x562c7e1cc9b0 2024-04-26T01:59:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.674+0000 7f773d600640 1 -- 172.21.15.1:0/1042957087 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f774001e040 con 0x562c7e1cc9b0 2024-04-26T01:59:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.674+0000 7f77597fb740 10 client.15987 fetch_fsmap learned FSMap version 32 2024-04-26T01:59:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.674+0000 7f77597fb740 1 -- 172.21.15.1:0/1042957087 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562c7e1e6ab0 con 0x562c7e1cc9b0 2024-04-26T01:59:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.674+0000 7f773d600640 1 -- 172.21.15.1:0/1042957087 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f774000faf0 con 0x562c7e1cc9b0 2024-04-26T01:59:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.675+0000 7f773d600640 1 -- 172.21.15.1:0/1042957087 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f7740021030 con 0x562c7e1cc9b0 2024-04-26T01:59:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.675+0000 7f77597fb740 10 client.15987 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:59:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.675+0000 7f77597fb740 10 client.15987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:59:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.675+0000 7f77597fb740 20 client.15987 populate_metadata read hostname 'smithi001' 2024-04-26T01:59:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.675+0000 7f77597fb740 1 --2- 172.21.15.1:0/1042957087 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562c7e1e7eb0 0x562c7e1ea2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.675+0000 7f77597fb740 4 client.15987 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:59:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.675+0000 7f77597fb740 1 -- 172.21.15.1:0/1042957087 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c7e1468b0 con 0x562c7e1e7eb0 2024-04-26T01:59:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.675+0000 7f7747e00640 1 --2- 172.21.15.1:0/1042957087 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562c7e1e7eb0 0x562c7e1ea2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.676+0000 7f7747e00640 1 --2- 172.21.15.1:0/1042957087 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562c7e1e7eb0 0x562c7e1ea2a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1352b63ae9293387 server_cookie=40700eadf7e84f58 in_seq=0 out_seq=0 2024-04-26T01:59:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.676+0000 7f773d600640 10 client.15987 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.677+0000 7f773d600640 1 -- 172.21.15.1:0/1042957087 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c7e1468b0 con 0x562c7e1e7eb0 2024-04-26T01:59:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.677+0000 7f773d600640 10 client.15987 handle_command_reply: tid=0 2024-04-26T01:59:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.677+0000 7f77597fb740 1 client.15987 shutdown 2024-04-26T01:59:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.677+0000 7f77597fb740 1 -- 172.21.15.1:0/1042957087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1cc9b0 msgr2=0x562c7e1d4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.677+0000 7f77597fb740 1 --2- 172.21.15.1:0/1042957087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1cc9b0 0x562c7e1d4bc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f774001c040 tx=0x7f7740019640 comp rx=0 tx=0).stop 2024-04-26T01:59:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.677+0000 7f77597fb740 1 -- 172.21.15.1:0/1042957087 shutdown_connections 2024-04-26T01:59:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.677+0000 7f77597fb740 1 --2- 172.21.15.1:0/1042957087 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562c7e1e7eb0 0x562c7e1ea2a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.677+0000 7f77597fb740 1 --2- 172.21.15.1:0/1042957087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1d7b90 0x562c7e1d5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.677+0000 7f77597fb740 1 --2- 172.21.15.1:0/1042957087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cd2d0 0x562c7e1d5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.677+0000 7f773d600640 0 client.15987 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.677+0000 7f77597fb740 1 --2- 172.21.15.1:0/1042957087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1cc9b0 0x562c7e1d4bc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.677+0000 7f77597fb740 1 -- 172.21.15.1:0/1042957087 >> 172.21.15.1:0/1042957087 conn(0x562c7e1c8ec0 msgr2=0x562c7e1cebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.677+0000 7f77597fb740 1 -- 172.21.15.1:0/1042957087 shutdown_connections 2024-04-26T01:59:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.678+0000 7f77597fb740 1 -- 172.21.15.1:0/1042957087 wait complete. 2024-04-26T01:59:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.678+0000 7f77597fb740 20 client.15987 trim_cache size 0 max 16384 2024-04-26T01:59:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.694+0000 7f77597fb740 1 Processor -- start 2024-04-26T01:59:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.694+0000 7f77597fb740 1 -- start start 2024-04-26T01:59:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.694+0000 7f77597fb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cc9b0 0x562c7e1e3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.694+0000 7f77597fb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1cd2d0 0x562c7e1d5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.694+0000 7f77597fb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1d7b90 0x562c7e1d57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.694+0000 7f77597fb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f77501b2660 con 0x562c7e1d7b90 2024-04-26T01:59:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.694+0000 7f77597fb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f77501aff30 con 0x562c7e1cd2d0 2024-04-26T01:59:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.694+0000 7f77597fb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f77501adda0 con 0x562c7e1cc9b0 2024-04-26T01:59:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.695+0000 7f774fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cc9b0 0x562c7e1e3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.695+0000 7f7747e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1cd2d0 0x562c7e1d5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.695+0000 7f7754a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1d7b90 0x562c7e1d57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.695+0000 7f7747e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1cd2d0 0x562c7e1d5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:44074/0 (socket says 172.21.15.1:44074) 2024-04-26T01:59:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.695+0000 7f774fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cc9b0 0x562c7e1e3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:56638/0 (socket says 172.21.15.1:56638) 2024-04-26T01:59:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.695+0000 7f7747e00640 1 -- 172.21.15.1:0/3106484903 learned_addr learned my addr 172.21.15.1:0/3106484903 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.695+0000 7f774fe00640 1 -- 172.21.15.1:0/3106484903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1cd2d0 msgr2=0x562c7e1d5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.695+0000 7f774fe00640 1 --2- 172.21.15.1:0/3106484903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1cd2d0 0x562c7e1d5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.695+0000 7f774fe00640 1 -- 172.21.15.1:0/3106484903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1d7b90 msgr2=0x562c7e1d57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.695+0000 7f774fe00640 1 --2- 172.21.15.1:0/3106484903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1d7b90 0x562c7e1d57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.695+0000 7f774fe00640 1 -- 172.21.15.1:0/3106484903 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c7e1d3130 con 0x562c7e1cc9b0 2024-04-26T01:59:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.695+0000 7f7747e00640 1 --2- 172.21.15.1:0/3106484903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1cd2d0 0x562c7e1d5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.696+0000 7f774fe00640 1 --2- 172.21.15.1:0/3106484903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cc9b0 0x562c7e1e3680 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7740011af0 tx=0x7f7740009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7690266209510030 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.696+0000 7f773fe00640 1 -- 172.21.15.1:0/3106484903 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7740027070 con 0x562c7e1cc9b0 2024-04-26T01:59:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.696+0000 7f773fe00640 1 -- 172.21.15.1:0/3106484903 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7740005080 con 0x562c7e1cc9b0 2024-04-26T01:59:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.696+0000 7f773fe00640 1 -- 172.21.15.1:0/3106484903 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77400035f0 con 0x562c7e1cc9b0 2024-04-26T01:59:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.696+0000 7f77597fb740 1 -- 172.21.15.1:0/3106484903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cc9b0 msgr2=0x562c7e1e3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.696+0000 7f77597fb740 1 --2- 172.21.15.1:0/3106484903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cc9b0 0x562c7e1e3680 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7740011af0 tx=0x7f7740009b50 comp rx=0 tx=0).stop 2024-04-26T01:59:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.696+0000 7f77597fb740 1 -- 172.21.15.1:0/3106484903 shutdown_connections 2024-04-26T01:59:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.696+0000 7f77597fb740 1 --2- 172.21.15.1:0/3106484903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1d7b90 0x562c7e1d57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.696+0000 7f77597fb740 1 --2- 172.21.15.1:0/3106484903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1cd2d0 0x562c7e1d5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.696+0000 7f77597fb740 1 --2- 172.21.15.1:0/3106484903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cc9b0 0x562c7e1e3680 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.696+0000 7f77597fb740 1 -- 172.21.15.1:0/3106484903 >> 172.21.15.1:0/3106484903 conn(0x562c7e1c8ec0 msgr2=0x562c7e1d4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.696+0000 7f77597fb740 1 -- 172.21.15.1:0/3106484903 shutdown_connections 2024-04-26T01:59:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.697+0000 7f77597fb740 1 -- 172.21.15.1:0/3106484903 wait complete. 2024-04-26T01:59:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.697+0000 7f77597fb740 1 -- start start 2024-04-26T01:59:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.697+0000 7f77597fb740 1 Processor -- start 2024-04-26T01:59:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.698+0000 7f77597fb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cc9b0 0x562c7e1dce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.698+0000 7f77597fb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1cd2d0 0x562c7e1dd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.698+0000 7f77597fb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1d7b90 0x562c7e24ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.698+0000 7f77597fb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f77501ae3f0 con 0x562c7e1d7b90 2024-04-26T01:59:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.698+0000 7f77597fb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f77501af860 con 0x562c7e1cd2d0 2024-04-26T01:59:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.698+0000 7f77597fb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f77501aeac0 con 0x562c7e1cc9b0 2024-04-26T01:59:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.698+0000 7f77597fb740 10 client.0 _ll_register_callbacks cb 0x7f7740011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.698+0000 7f7754a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1d7b90 0x562c7e24ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.698+0000 7f7747e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1cd2d0 0x562c7e1dd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.698+0000 7f7747e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1cd2d0 0x562c7e1dd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:44076/0 (socket says 172.21.15.1:44076) 2024-04-26T01:59:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.698+0000 7f774fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cc9b0 0x562c7e1dce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.698+0000 7f7747e00640 1 -- 172.21.15.1:0/3728212627 learned_addr learned my addr 172.21.15.1:0/3728212627 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.698+0000 7f7754a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1d7b90 0x562c7e24ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46820/0 (socket says 172.21.15.1:46820) 2024-04-26T01:59:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.698+0000 7f774fe00640 1 -- 172.21.15.1:0/3728212627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1cd2d0 msgr2=0x562c7e1dd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.698+0000 7f774fe00640 1 --2- 172.21.15.1:0/3728212627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1cd2d0 0x562c7e1dd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.698+0000 7f774fe00640 1 -- 172.21.15.1:0/3728212627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1d7b90 msgr2=0x562c7e24ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.698+0000 7f774fe00640 1 --2- 172.21.15.1:0/3728212627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1d7b90 0x562c7e24ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.698+0000 7f774fe00640 1 -- 172.21.15.1:0/3728212627 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c7e24f070 con 0x562c7e1cc9b0 2024-04-26T01:59:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.698+0000 7f7754a00640 1 --2- 172.21.15.1:0/3728212627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1d7b90 0x562c7e24ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.699+0000 7f774fe00640 1 --2- 172.21.15.1:0/3728212627 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cc9b0 0x562c7e1dce30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f77400059f0 tx=0x7f7740009d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fc32124085fb179 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.699+0000 7f773d600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T01:59:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.699+0000 7f773d600640 1 -- 172.21.15.1:0/3728212627 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7740027040 con 0x562c7e1cc9b0 2024-04-26T01:59:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.699+0000 7f77597fb740 1 -- 172.21.15.1:0/3728212627 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7740005080 con 0x562c7e1cc9b0 2024-04-26T01:59:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.699+0000 7f773d600640 1 -- 172.21.15.1:0/3728212627 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f7740019620 con 0x562c7e1cc9b0 2024-04-26T01:59:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.699+0000 7f773d600640 1 -- 172.21.15.1:0/3728212627 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7740043050 con 0x562c7e1cc9b0 2024-04-26T01:59:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.699+0000 7f77597fb740 10 client.26164 fetch_fsmap learned FSMap version 32 2024-04-26T01:59:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.699+0000 7f77597fb740 1 -- 172.21.15.1:0/3728212627 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562c7e2563f0 con 0x562c7e1cc9b0 2024-04-26T01:59:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.700+0000 7f773d600640 1 -- 172.21.15.1:0/3728212627 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7740005080 con 0x562c7e1cc9b0 2024-04-26T01:59:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.700+0000 7f773d600640 1 -- 172.21.15.1:0/3728212627 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f7740032040 con 0x562c7e1cc9b0 2024-04-26T01:59:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.700+0000 7f77597fb740 10 client.26164 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:59:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.700+0000 7f77597fb740 10 client.26164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:59:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.700+0000 7f77597fb740 20 client.26164 populate_metadata read hostname 'smithi001' 2024-04-26T01:59:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.700+0000 7f77597fb740 1 --2- 172.21.15.1:0/3728212627 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562c7e256a50 0x562c7e258e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.700+0000 7f77597fb740 4 client.26164 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:59:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.700+0000 7f77597fb740 1 -- 172.21.15.1:0/3728212627 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c7e140320 con 0x562c7e256a50 2024-04-26T01:59:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.700+0000 7f7747e00640 1 --2- 172.21.15.1:0/3728212627 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562c7e256a50 0x562c7e258e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.701+0000 7f7747e00640 1 --2- 172.21.15.1:0/3728212627 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562c7e256a50 0x562c7e258e70 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69559a5440b94c92 server_cookie=68ff5efd076ea014 in_seq=0 out_seq=0 2024-04-26T01:59:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.701+0000 7f773d600640 10 client.26164 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.716+0000 7f773d600640 1 -- 172.21.15.1:0/3728212627 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562c7e140320 con 0x562c7e256a50 2024-04-26T01:59:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.716+0000 7f773d600640 10 client.26164 handle_command_reply: tid=0 2024-04-26T01:59:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.716+0000 7f77597fb740 1 client.26164 shutdown 2024-04-26T01:59:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.716+0000 7f77597fb740 1 -- 172.21.15.1:0/3728212627 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cc9b0 msgr2=0x562c7e1dce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.716+0000 7f77597fb740 1 --2- 172.21.15.1:0/3728212627 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cc9b0 0x562c7e1dce30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f77400059f0 tx=0x7f7740009d80 comp rx=0 tx=0).stop 2024-04-26T01:59:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.716+0000 7f77597fb740 1 -- 172.21.15.1:0/3728212627 shutdown_connections 2024-04-26T01:59:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.716+0000 7f77597fb740 1 --2- 172.21.15.1:0/3728212627 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x562c7e256a50 0x562c7e258e70 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.716+0000 7f77597fb740 1 --2- 172.21.15.1:0/3728212627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c7e1d7b90 0x562c7e24ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.716+0000 7f77597fb740 1 --2- 172.21.15.1:0/3728212627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562c7e1cd2d0 0x562c7e1dd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.716+0000 7f77597fb740 1 --2- 172.21.15.1:0/3728212627 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562c7e1cc9b0 0x562c7e1dce30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.716+0000 7f77597fb740 1 -- 172.21.15.1:0/3728212627 >> 172.21.15.1:0/3728212627 conn(0x562c7e1c8ec0 msgr2=0x562c7e1d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.716+0000 7f773d600640 0 client.26164 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.717+0000 7f77597fb740 1 -- 172.21.15.1:0/3728212627 shutdown_connections 2024-04-26T01:59:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.717+0000 7f77597fb740 1 -- 172.21.15.1:0/3728212627 wait complete. 2024-04-26T01:59:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.717+0000 7f77597fb740 20 client.26164 trim_cache size 0 max 16384 2024-04-26T01:59:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:59:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:59:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:59:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:59:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.717+0000 7f7755400640 1 -- 172.21.15.1:0/1967747896 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f77340a3d00 msgr2=0x7f77340a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.717+0000 7f7755400640 1 --2- 172.21.15.1:0/1967747896 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f77340a3d00 0x7f77340a61c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x562c7e1ca2e0 tx=0x7f7738013040 comp rx=0 tx=0).stop 2024-04-26T01:59:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.717+0000 7f7755400640 1 -- 172.21.15.1:0/1967747896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f775019a5e0 msgr2=0x7f775019aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.717+0000 7f7755400640 1 --2- 172.21.15.1:0/1967747896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f775019a5e0 0x7f775019aa90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7748005cc0 tx=0x7f7748031700 comp rx=0 tx=0).stop 2024-04-26T01:59:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.718+0000 7f7755400640 1 -- 172.21.15.1:0/1967747896 shutdown_connections 2024-04-26T01:59:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.718+0000 7f7755400640 1 --2- 172.21.15.1:0/1967747896 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f77340a3d00 0x7f77340a61c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.718+0000 7f7755400640 1 --2- 172.21.15.1:0/1967747896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f775019a5e0 0x7f775019aa90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.718+0000 7f7755400640 1 --2- 172.21.15.1:0/1967747896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77501092f0 0x7f77501a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.718+0000 7f7755400640 1 --2- 172.21.15.1:0/1967747896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7750104220 0x7f77501a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.718+0000 7f7755400640 1 -- 172.21.15.1:0/1967747896 >> 172.21.15.1:0/1967747896 conn(0x7f77500ffa60 msgr2=0x7f7750101dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.718+0000 7f7755400640 1 -- 172.21.15.1:0/1967747896 shutdown_connections 2024-04-26T01:59:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:43.718+0000 7f7755400640 1 -- 172.21.15.1:0/1967747896 wait complete. 2024-04-26T01:59:44.536 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:44 smithi097 ceph-mon[32853]: pgmap v250: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 178 KiB/s wr, 65 op/s 2024-04-26T01:59:44.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:44 smithi173 ceph-mon[32553]: pgmap v250: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 178 KiB/s wr, 65 op/s 2024-04-26T01:59:44.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:44 smithi001 ceph-mon[27822]: pgmap v250: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 178 KiB/s wr, 65 op/s 2024-04-26T01:59:46.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:46 smithi173 ceph-mon[32553]: pgmap v251: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 190 KiB/s wr, 71 op/s 2024-04-26T01:59:46.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:46 smithi001 ceph-mon[27822]: pgmap v251: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 190 KiB/s wr, 71 op/s 2024-04-26T01:59:46.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:46 smithi097 ceph-mon[32853]: pgmap v251: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 190 KiB/s wr, 71 op/s 2024-04-26T01:59:47.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:47 smithi173 ceph-mon[32553]: pgmap v252: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-04-26T01:59:47.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:47 smithi001 ceph-mon[27822]: pgmap v252: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-04-26T01:59:47.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:47 smithi097 ceph-mon[32853]: pgmap v252: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-04-26T01:59:50.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:49 smithi097 ceph-mon[32853]: pgmap v253: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-04-26T01:59:50.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:49 smithi173 ceph-mon[32553]: pgmap v253: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-04-26T01:59:50.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:49 smithi001 ceph-mon[27822]: pgmap v253: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-04-26T01:59:52.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:51 smithi097 ceph-mon[32853]: pgmap v254: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 82 KiB/s wr, 13 op/s 2024-04-26T01:59:52.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:51 smithi173 ceph-mon[32553]: pgmap v254: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 82 KiB/s wr, 13 op/s 2024-04-26T01:59:52.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:51 smithi001 ceph-mon[27822]: pgmap v254: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 82 KiB/s wr, 13 op/s 2024-04-26T01:59:54.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:53 smithi097 ceph-mon[32853]: pgmap v255: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 80 KiB/s wr, 10 op/s 2024-04-26T01:59:54.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:53 smithi173 ceph-mon[32553]: pgmap v255: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 80 KiB/s wr, 10 op/s 2024-04-26T01:59:54.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:53 smithi001 ceph-mon[27822]: pgmap v255: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 80 KiB/s wr, 10 op/s 2024-04-26T01:59:56.022 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:59:56.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:55 smithi097 ceph-mon[32853]: pgmap v256: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 83 KiB/s wr, 13 op/s 2024-04-26T01:59:56.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:55 smithi173 ceph-mon[32553]: pgmap v256: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 83 KiB/s wr, 13 op/s 2024-04-26T01:59:56.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:55 smithi001 ceph-mon[27822]: pgmap v256: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 83 KiB/s wr, 13 op/s 2024-04-26T01:59:56.504 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:59:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.864+0000 7f9058a00640 1 -- 172.21.15.1:0/4063461421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9054103b00 msgr2=0x7f905410bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.864+0000 7f9058a00640 1 --2- 172.21.15.1:0/4063461421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9054103b00 0x7f905410bfe0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9040009860 tx=0x7f904002a3d0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.865+0000 7f9058a00640 1 -- 172.21.15.1:0/4063461421 shutdown_connections 2024-04-26T01:59:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.865+0000 7f9058a00640 1 --2- 172.21.15.1:0/4063461421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f905410c5b0 0x7f905410e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.865+0000 7f9058a00640 1 --2- 172.21.15.1:0/4063461421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9054103b00 0x7f905410bfe0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.865+0000 7f9058a00640 1 --2- 172.21.15.1:0/4063461421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90541031e0 0x7f90541035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.865+0000 7f9058a00640 1 -- 172.21.15.1:0/4063461421 >> 172.21.15.1:0/4063461421 conn(0x7f9054077fd0 msgr2=0x7f90540783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.865+0000 7f9058a00640 1 -- 172.21.15.1:0/4063461421 shutdown_connections 2024-04-26T01:59:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.865+0000 7f9058a00640 1 -- 172.21.15.1:0/4063461421 wait complete. 2024-04-26T01:59:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.866+0000 7f9058a00640 1 Processor -- start 2024-04-26T01:59:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.866+0000 7f9058a00640 1 -- start start 2024-04-26T01:59:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.867+0000 7f9058a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90541031e0 0x7f90541a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.867+0000 7f9058a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9054103b00 0x7f90541a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.867+0000 7f9058a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f905410c5b0 0x7f905419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.867+0000 7f9058a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9054112240 con 0x7f905410c5b0 2024-04-26T01:59:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.867+0000 7f9058a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f90541120c0 con 0x7f9054103b00 2024-04-26T01:59:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.867+0000 7f9058a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f90541123c0 con 0x7f90541031e0 2024-04-26T01:59:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.867+0000 7f9052800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f905410c5b0 0x7f905419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.867+0000 7f9051400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9054103b00 0x7f90541a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.867+0000 7f9052800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f905410c5b0 0x7f905419a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49548/0 (socket says 172.21.15.1:49548) 2024-04-26T01:59:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.867+0000 7f9052800640 1 -- 172.21.15.1:0/1949298237 learned_addr learned my addr 172.21.15.1:0/1949298237 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.867+0000 7f9051e00640 1 --2- 172.21.15.1:0/1949298237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90541031e0 0x7f90541a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.868+0000 7f9052800640 1 -- 172.21.15.1:0/1949298237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90541031e0 msgr2=0x7f90541a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.868+0000 7f9052800640 1 --2- 172.21.15.1:0/1949298237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90541031e0 0x7f90541a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.868+0000 7f9052800640 1 -- 172.21.15.1:0/1949298237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9054103b00 msgr2=0x7f90541a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.868+0000 7f9052800640 1 --2- 172.21.15.1:0/1949298237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9054103b00 0x7f90541a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.868+0000 7f9052800640 1 -- 172.21.15.1:0/1949298237 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905419aff0 con 0x7f905410c5b0 2024-04-26T01:59:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.868+0000 7f9052800640 1 --2- 172.21.15.1:0/1949298237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f905410c5b0 0x7f905419a820 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f904c00c3f0 tx=0x7f904c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.868+0000 7f904aa00640 1 -- 172.21.15.1:0/1949298237 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c003970 con 0x7f905410c5b0 2024-04-26T01:59:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.868+0000 7f9058a00640 1 -- 172.21.15.1:0/1949298237 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f905419b2e0 con 0x7f905410c5b0 2024-04-26T01:59:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.868+0000 7f904aa00640 1 -- 172.21.15.1:0/1949298237 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f904c00b690 con 0x7f905410c5b0 2024-04-26T01:59:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.868+0000 7f904aa00640 1 -- 172.21.15.1:0/1949298237 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c004d80 con 0x7f905410c5b0 2024-04-26T01:59:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.868+0000 7f9058a00640 1 -- 172.21.15.1:0/1949298237 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90541a7500 con 0x7f905410c5b0 2024-04-26T01:59:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.872+0000 7f904aa00640 1 -- 172.21.15.1:0/1949298237 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f904c01d020 con 0x7f905410c5b0 2024-04-26T01:59:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.872+0000 7f905b7e5740 1 Processor -- start 2024-04-26T01:59:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.872+0000 7f905b7e5740 1 -- start start 2024-04-26T01:59:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.873+0000 7f905b7e5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db1f67820 0x559db1f67c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.873+0000 7f905b7e5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f68140 0x559db1f724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.873+0000 7f905b7e5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f72a00 0x559db1f74df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.873+0000 7f905b7e5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559db1ecd900 con 0x559db1f72a00 2024-04-26T01:59:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.873+0000 7f905b7e5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559db1e73ef0 con 0x559db1f67820 2024-04-26T01:59:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.873+0000 7f905b7e5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559db1ec98c0 con 0x559db1f68140 2024-04-26T01:59:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.873+0000 7f9051400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db1f67820 0x559db1f67c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.873+0000 7f9051e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f68140 0x559db1f724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.873+0000 7f9051400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db1f67820 0x559db1f67c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:46950/0 (socket says 172.21.15.1:46950) 2024-04-26T01:59:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.873+0000 7f9051400640 1 -- 172.21.15.1:0/4011224493 learned_addr learned my addr 172.21.15.1:0/4011224493 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.873+0000 7f9051400640 1 --2- 172.21.15.1:0/4011224493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f72a00 0x559db1f74df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.874+0000 7f9051e00640 1 -- 172.21.15.1:0/4011224493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db1f67820 msgr2=0x559db1f67c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.874+0000 7f9051e00640 1 --2- 172.21.15.1:0/4011224493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db1f67820 0x559db1f67c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.874+0000 7f904aa00640 1 --2- 172.21.15.1:0/1949298237 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f90380a3d10 0x7f90380a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.874+0000 7f9051e00640 1 -- 172.21.15.1:0/4011224493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f72a00 msgr2=0x559db1f74df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.874+0000 7f9051e00640 1 --2- 172.21.15.1:0/4011224493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f72a00 0x559db1f74df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.874+0000 7f9051e00640 1 -- 172.21.15.1:0/4011224493 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559db1be0210 con 0x559db1f68140 2024-04-26T01:59:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.874+0000 7f9051400640 1 --2- 172.21.15.1:0/4011224493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f72a00 0x559db1f74df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.874+0000 7f9052800640 1 --2- 172.21.15.1:0/1949298237 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f90380a3d10 0x7f90380a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.874+0000 7f904aa00640 1 -- 172.21.15.1:0/1949298237 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f904c0d0710 con 0x7f905410c5b0 2024-04-26T01:59:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.874+0000 7f9051e00640 1 --2- 172.21.15.1:0/4011224493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f68140 0x559db1f724c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9044009c00 tx=0x7f9044011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3b5e06a646bd1c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.875+0000 7f903f000640 1 -- 172.21.15.1:0/4011224493 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044004230 con 0x559db1f68140 2024-04-26T01:59:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.875+0000 7f903f000640 1 -- 172.21.15.1:0/4011224493 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90440043d0 con 0x559db1f68140 2024-04-26T01:59:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.875+0000 7f903f000640 1 -- 172.21.15.1:0/4011224493 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044004c70 con 0x559db1f68140 2024-04-26T01:59:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.875+0000 7f905b7e5740 1 -- 172.21.15.1:0/4011224493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f68140 msgr2=0x559db1f724c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.875+0000 7f905b7e5740 1 --2- 172.21.15.1:0/4011224493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f68140 0x559db1f724c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9044009c00 tx=0x7f9044011810 comp rx=0 tx=0).stop 2024-04-26T01:59:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.875+0000 7f9052800640 1 --2- 172.21.15.1:0/1949298237 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f90380a3d10 0x7f90380a61d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f904c00cdb0 tx=0x7f904c01c000 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.876+0000 7f905b7e5740 1 -- 172.21.15.1:0/4011224493 shutdown_connections 2024-04-26T01:59:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.876+0000 7f905b7e5740 1 --2- 172.21.15.1:0/4011224493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f72a00 0x559db1f74df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.876+0000 7f905b7e5740 1 --2- 172.21.15.1:0/4011224493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f68140 0x559db1f724c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.876+0000 7f905b7e5740 1 --2- 172.21.15.1:0/4011224493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db1f67820 0x559db1f67c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.876+0000 7f905b7e5740 1 -- 172.21.15.1:0/4011224493 >> 172.21.15.1:0/4011224493 conn(0x559db1f63d30 msgr2=0x559db1f64110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.876+0000 7f905b7e5740 1 -- 172.21.15.1:0/4011224493 shutdown_connections 2024-04-26T01:59:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.876+0000 7f905b7e5740 1 -- 172.21.15.1:0/4011224493 wait complete. 2024-04-26T01:59:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.876+0000 7f905b7e5740 1 -- start start 2024-04-26T01:59:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.876+0000 7f905b7e5740 1 Processor -- start 2024-04-26T01:59:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.877+0000 7f905b7e5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f67820 0x559db1f6f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.877+0000 7f905b7e5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f68140 0x559db1f6ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.877+0000 7f905b7e5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db1f72a00 0x559db1f70440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.877+0000 7f905b7e5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559db1f75630 con 0x559db1f68140 2024-04-26T01:59:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.877+0000 7f905b7e5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559db1e87310 con 0x559db1f72a00 2024-04-26T01:59:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.877+0000 7f905b7e5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559db1f757b0 con 0x559db1f67820 2024-04-26T01:59:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.877+0000 7f905b7e5740 10 client.0 _ll_register_callbacks cb 0x559db1e55cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.877+0000 7f9051e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f68140 0x559db1f6ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.877+0000 7f9051400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f67820 0x559db1f6f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.877+0000 7f9051e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f68140 0x559db1f6ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49578/0 (socket says 172.21.15.1:49578) 2024-04-26T01:59:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.877+0000 7f9051e00640 1 -- 172.21.15.1:0/730549562 learned_addr learned my addr 172.21.15.1:0/730549562 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.878+0000 7f9051400640 1 --2- 172.21.15.1:0/730549562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db1f72a00 0x559db1f70440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.878+0000 7f9051e00640 1 -- 172.21.15.1:0/730549562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f67820 msgr2=0x559db1f6f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.878+0000 7f9051e00640 1 --2- 172.21.15.1:0/730549562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f67820 0x559db1f6f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.878+0000 7f9051e00640 1 -- 172.21.15.1:0/730549562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db1f72a00 msgr2=0x559db1f70440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.878+0000 7f9051e00640 1 --2- 172.21.15.1:0/730549562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db1f72a00 0x559db1f70440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.878+0000 7f9051e00640 1 -- 172.21.15.1:0/730549562 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559db1cac620 con 0x559db1f68140 2024-04-26T01:59:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.878+0000 7f9051400640 1 --2- 172.21.15.1:0/730549562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f67820 0x559db1f6f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.878+0000 7f9051e00640 1 --2- 172.21.15.1:0/730549562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f68140 0x559db1f6ff00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9044004910 tx=0x7f9044019b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a18f58ad11c8d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.878+0000 7f9033400640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:59:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.879+0000 7f9033400640 1 -- 172.21.15.1:0/730549562 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044028020 con 0x559db1f68140 2024-04-26T01:59:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.879+0000 7f9033400640 1 -- 172.21.15.1:0/730549562 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f9044024620 con 0x559db1f68140 2024-04-26T01:59:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.879+0000 7f905b7e5740 1 -- 172.21.15.1:0/730549562 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90440043d0 con 0x559db1f68140 2024-04-26T01:59:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.879+0000 7f9033400640 1 -- 172.21.15.1:0/730549562 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9044033050 con 0x559db1f68140 2024-04-26T01:59:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.879+0000 7f905b7e5740 10 client.16005 fetch_fsmap learned FSMap version 32 2024-04-26T01:59:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.879+0000 7f905b7e5740 1 -- 172.21.15.1:0/730549562 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559db1f818c0 con 0x559db1f68140 2024-04-26T01:59:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.879+0000 7f9033400640 1 -- 172.21.15.1:0/730549562 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90440043d0 con 0x559db1f68140 2024-04-26T01:59:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.879+0000 7f9033400640 1 -- 172.21.15.1:0/730549562 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f9044018070 con 0x559db1f68140 2024-04-26T01:59:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.880+0000 7f905b7e5740 10 client.16005 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:59:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.880+0000 7f905b7e5740 10 client.16005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:59:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.880+0000 7f905b7e5740 20 client.16005 populate_metadata read hostname 'smithi001' 2024-04-26T01:59:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.880+0000 7f905b7e5740 1 --2- 172.21.15.1:0/730549562 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559db1f82cc0 0x559db1f850b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.880+0000 7f905b7e5740 4 client.16005 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:59:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.880+0000 7f905b7e5740 1 -- 172.21.15.1:0/730549562 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559db1ee18b0 con 0x559db1f82cc0 2024-04-26T01:59:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.880+0000 7f9051400640 1 --2- 172.21.15.1:0/730549562 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559db1f82cc0 0x559db1f850b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.881+0000 7f9051400640 1 --2- 172.21.15.1:0/730549562 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559db1f82cc0 0x559db1f850b0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87c24d092b983666 server_cookie=9c45e2b1ef37094b in_seq=0 out_seq=0 2024-04-26T01:59:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.881+0000 7f9033400640 10 client.16005 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.881+0000 7f9033400640 1 -- 172.21.15.1:0/730549562 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559db1ee18b0 con 0x559db1f82cc0 2024-04-26T01:59:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.881+0000 7f9033400640 10 client.16005 handle_command_reply: tid=0 2024-04-26T01:59:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.882+0000 7f905b7e5740 1 client.16005 shutdown 2024-04-26T01:59:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.882+0000 7f905b7e5740 1 -- 172.21.15.1:0/730549562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f68140 msgr2=0x559db1f6ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.882+0000 7f905b7e5740 1 --2- 172.21.15.1:0/730549562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f68140 0x559db1f6ff00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9044004910 tx=0x7f9044019b00 comp rx=0 tx=0).stop 2024-04-26T01:59:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.882+0000 7f905b7e5740 1 -- 172.21.15.1:0/730549562 shutdown_connections 2024-04-26T01:59:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.882+0000 7f905b7e5740 1 --2- 172.21.15.1:0/730549562 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559db1f82cc0 0x559db1f850b0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.882+0000 7f905b7e5740 1 --2- 172.21.15.1:0/730549562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db1f72a00 0x559db1f70440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.882+0000 7f905b7e5740 1 --2- 172.21.15.1:0/730549562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f68140 0x559db1f6ff00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.882+0000 7f905b7e5740 1 --2- 172.21.15.1:0/730549562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f67820 0x559db1f6f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.882+0000 7f9033400640 0 client.16005 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.882+0000 7f905b7e5740 1 -- 172.21.15.1:0/730549562 >> 172.21.15.1:0/730549562 conn(0x559db1f63d30 msgr2=0x559db1f69a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.882+0000 7f905b7e5740 1 -- 172.21.15.1:0/730549562 shutdown_connections 2024-04-26T01:59:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.882+0000 7f905b7e5740 1 -- 172.21.15.1:0/730549562 wait complete. 2024-04-26T01:59:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.882+0000 7f905b7e5740 20 client.16005 trim_cache size 0 max 16384 2024-04-26T01:59:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.899+0000 7f905b7e5740 1 Processor -- start 2024-04-26T01:59:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.899+0000 7f905b7e5740 1 -- start start 2024-04-26T01:59:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.899+0000 7f905b7e5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db1f67820 0x559db1f6a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.899+0000 7f905b7e5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f68140 0x559db1f8a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.899+0000 7f905b7e5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f72a00 0x559db1f8ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.899+0000 7f905b7e5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f90541b2740 con 0x559db1f72a00 2024-04-26T01:59:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.899+0000 7f905b7e5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f90541b0010 con 0x559db1f67820 2024-04-26T01:59:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.899+0000 7f905b7e5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f90541ade80 con 0x559db1f68140 2024-04-26T01:59:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.900+0000 7f9051e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f68140 0x559db1f8a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.900+0000 7f9051e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f68140 0x559db1f8a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:35074/0 (socket says 172.21.15.1:35074) 2024-04-26T01:59:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.900+0000 7f9051e00640 1 -- 172.21.15.1:0/1424296542 learned_addr learned my addr 172.21.15.1:0/1424296542 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.900+0000 7f9051400640 1 --2- 172.21.15.1:0/1424296542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db1f67820 0x559db1f6a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.900+0000 7f9051400640 1 --2- 172.21.15.1:0/1424296542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f72a00 0x559db1f8ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.900+0000 7f9051e00640 1 -- 172.21.15.1:0/1424296542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db1f67820 msgr2=0x559db1f6a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.900+0000 7f9051e00640 1 --2- 172.21.15.1:0/1424296542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db1f67820 0x559db1f6a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.900+0000 7f9051e00640 1 -- 172.21.15.1:0/1424296542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f72a00 msgr2=0x559db1f8ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.900+0000 7f9051e00640 1 --2- 172.21.15.1:0/1424296542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f72a00 0x559db1f8ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.900+0000 7f9051e00640 1 -- 172.21.15.1:0/1424296542 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559db1f85670 con 0x559db1f68140 2024-04-26T01:59:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.900+0000 7f9051400640 1 --2- 172.21.15.1:0/1424296542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f72a00 0x559db1f8ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.900+0000 7f9051e00640 1 --2- 172.21.15.1:0/1424296542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f68140 0x559db1f8a8a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9044012fd0 tx=0x7f90440059b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c99f24c8b7d69ac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.901+0000 7f903e600640 1 -- 172.21.15.1:0/1424296542 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044028070 con 0x559db1f68140 2024-04-26T01:59:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.901+0000 7f903e600640 1 -- 172.21.15.1:0/1424296542 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f904401f040 con 0x559db1f68140 2024-04-26T01:59:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.901+0000 7f903e600640 1 -- 172.21.15.1:0/1424296542 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044024420 con 0x559db1f68140 2024-04-26T01:59:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.901+0000 7f905b7e5740 1 -- 172.21.15.1:0/1424296542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f68140 msgr2=0x559db1f8a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.901+0000 7f905b7e5740 1 --2- 172.21.15.1:0/1424296542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f68140 0x559db1f8a8a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9044012fd0 tx=0x7f90440059b0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.902+0000 7f905b7e5740 1 -- 172.21.15.1:0/1424296542 shutdown_connections 2024-04-26T01:59:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.902+0000 7f905b7e5740 1 --2- 172.21.15.1:0/1424296542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f72a00 0x559db1f8ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.902+0000 7f905b7e5740 1 --2- 172.21.15.1:0/1424296542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f68140 0x559db1f8a8a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.902+0000 7f905b7e5740 1 --2- 172.21.15.1:0/1424296542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db1f67820 0x559db1f6a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.902+0000 7f905b7e5740 1 -- 172.21.15.1:0/1424296542 >> 172.21.15.1:0/1424296542 conn(0x559db1f63d30 msgr2=0x559db1f62600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.902+0000 7f905b7e5740 1 -- 172.21.15.1:0/1424296542 shutdown_connections 2024-04-26T01:59:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.902+0000 7f905b7e5740 1 -- 172.21.15.1:0/1424296542 wait complete. 2024-04-26T01:59:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.902+0000 7f905b7e5740 1 -- start start 2024-04-26T01:59:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.902+0000 7f905b7e5740 1 Processor -- start 2024-04-26T01:59:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.903+0000 7f905b7e5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f67820 0x559db1f786a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.903+0000 7f905b7e5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f68140 0x559db1f78be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.903+0000 7f905b7e5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db1f72a00 0x559db1f79120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.903+0000 7f905b7e5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f90541ae4d0 con 0x559db1f67820 2024-04-26T01:59:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.903+0000 7f905b7e5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f90541af940 con 0x559db1f72a00 2024-04-26T01:59:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.903+0000 7f905b7e5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f90541aeba0 con 0x559db1f68140 2024-04-26T01:59:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.903+0000 7f905b7e5740 10 client.0 _ll_register_callbacks cb 0x7f9044019790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.903+0000 7f9051400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f67820 0x559db1f786a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.903+0000 7f9051400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f67820 0x559db1f786a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49600/0 (socket says 172.21.15.1:49600) 2024-04-26T01:59:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.903+0000 7f9051400640 1 -- 172.21.15.1:0/2641285539 learned_addr learned my addr 172.21.15.1:0/2641285539 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.903+0000 7f9051e00640 1 --2- 172.21.15.1:0/2641285539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f68140 0x559db1f78be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.903+0000 7f9051400640 1 --2- 172.21.15.1:0/2641285539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db1f72a00 0x559db1f79120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.904+0000 7f9051400640 1 -- 172.21.15.1:0/2641285539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f68140 msgr2=0x559db1f78be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.904+0000 7f9051400640 1 --2- 172.21.15.1:0/2641285539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f68140 0x559db1f78be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.904+0000 7f9051400640 1 -- 172.21.15.1:0/2641285539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db1f72a00 msgr2=0x559db1f79120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.904+0000 7f9051400640 1 --2- 172.21.15.1:0/2641285539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db1f72a00 0x559db1f79120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.904+0000 7f9051400640 1 -- 172.21.15.1:0/2641285539 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559db1fe8370 con 0x559db1f67820 2024-04-26T01:59:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.904+0000 7f9051e00640 1 --2- 172.21.15.1:0/2641285539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f68140 0x559db1f78be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.904+0000 7f9051400640 1 --2- 172.21.15.1:0/2641285539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f67820 0x559db1f786a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f904002a410 tx=0x7f90400058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6d9cbf0ce87178 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.904+0000 7f9033400640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T01:59:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.904+0000 7f9033400640 1 -- 172.21.15.1:0/2641285539 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9040048040 con 0x559db1f67820 2024-04-26T01:59:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.904+0000 7f9033400640 1 -- 172.21.15.1:0/2641285539 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f9040030a40 con 0x559db1f67820 2024-04-26T01:59:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.904+0000 7f905b7e5740 1 -- 172.21.15.1:0/2641285539 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f904401f040 con 0x559db1f67820 2024-04-26T01:59:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.905+0000 7f9033400640 1 -- 172.21.15.1:0/2641285539 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9040030c40 con 0x559db1f67820 2024-04-26T01:59:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.905+0000 7f905b7e5740 10 client.16011 fetch_fsmap learned FSMap version 32 2024-04-26T01:59:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.905+0000 7f905b7e5740 1 -- 172.21.15.1:0/2641285539 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559db1fe9680 con 0x559db1f67820 2024-04-26T01:59:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.905+0000 7f9033400640 1 -- 172.21.15.1:0/2641285539 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f904401f040 con 0x559db1f67820 2024-04-26T01:59:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.905+0000 7f9033400640 1 -- 172.21.15.1:0/2641285539 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f904004e6d0 con 0x559db1f67820 2024-04-26T01:59:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.905+0000 7f905b7e5740 10 client.16011 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T01:59:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.905+0000 7f905b7e5740 10 client.16011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T01:59:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.905+0000 7f905b7e5740 20 client.16011 populate_metadata read hostname 'smithi001' 2024-04-26T01:59:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.905+0000 7f905b7e5740 1 --2- 172.21.15.1:0/2641285539 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559db1fe9d80 0x559db1fec1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.905+0000 7f905b7e5740 4 client.16011 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:59:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.905+0000 7f905b7e5740 1 -- 172.21.15.1:0/2641285539 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559db1edb320 con 0x559db1fe9d80 2024-04-26T01:59:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.906+0000 7f9051e00640 1 --2- 172.21.15.1:0/2641285539 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559db1fe9d80 0x559db1fec1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.906+0000 7f9051e00640 1 --2- 172.21.15.1:0/2641285539 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559db1fe9d80 0x559db1fec1a0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cca25ba6bbd81405 server_cookie=816246cf30a6b518 in_seq=0 out_seq=0 2024-04-26T01:59:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.906+0000 7f9033400640 10 client.16011 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.925+0000 7f9033400640 1 -- 172.21.15.1:0/2641285539 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559db1edb320 con 0x559db1fe9d80 2024-04-26T01:59:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.925+0000 7f9033400640 10 client.16011 handle_command_reply: tid=0 2024-04-26T01:59:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.925+0000 7f905b7e5740 1 client.16011 shutdown 2024-04-26T01:59:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.925+0000 7f905b7e5740 1 -- 172.21.15.1:0/2641285539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f67820 msgr2=0x559db1f786a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.925+0000 7f905b7e5740 1 --2- 172.21.15.1:0/2641285539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f67820 0x559db1f786a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f904002a410 tx=0x7f90400058b0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.925+0000 7f905b7e5740 1 -- 172.21.15.1:0/2641285539 shutdown_connections 2024-04-26T01:59:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.925+0000 7f905b7e5740 1 --2- 172.21.15.1:0/2641285539 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x559db1fe9d80 0x559db1fec1a0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.925+0000 7f905b7e5740 1 --2- 172.21.15.1:0/2641285539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559db1f72a00 0x559db1f79120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.925+0000 7f905b7e5740 1 --2- 172.21.15.1:0/2641285539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559db1f68140 0x559db1f78be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.925+0000 7f905b7e5740 1 --2- 172.21.15.1:0/2641285539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559db1f67820 0x559db1f786a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.925+0000 7f905b7e5740 1 -- 172.21.15.1:0/2641285539 >> 172.21.15.1:0/2641285539 conn(0x559db1f63d30 msgr2=0x559db1f62e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.925+0000 7f9033400640 0 client.16011 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T01:59:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.925+0000 7f905b7e5740 1 -- 172.21.15.1:0/2641285539 shutdown_connections 2024-04-26T01:59:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.925+0000 7f905b7e5740 1 -- 172.21.15.1:0/2641285539 wait complete. 2024-04-26T01:59:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.925+0000 7f905b7e5740 20 client.16011 trim_cache size 0 max 16384 2024-04-26T01:59:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T01:59:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T01:59:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T01:59:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T01:59:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.926+0000 7f9058a00640 1 -- 172.21.15.1:0/1949298237 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f90380a3d10 msgr2=0x7f90380a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.926+0000 7f9058a00640 1 --2- 172.21.15.1:0/1949298237 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f90380a3d10 0x7f90380a61d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f904c00cdb0 tx=0x7f904c01c000 comp rx=0 tx=0).stop 2024-04-26T01:59:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.926+0000 7f9058a00640 1 -- 172.21.15.1:0/1949298237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f905410c5b0 msgr2=0x7f905419a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.926+0000 7f9058a00640 1 --2- 172.21.15.1:0/1949298237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f905410c5b0 0x7f905419a820 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f904c00c3f0 tx=0x7f904c00c8b0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.926+0000 7f9058a00640 1 -- 172.21.15.1:0/1949298237 shutdown_connections 2024-04-26T01:59:56.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.926+0000 7f9058a00640 1 --2- 172.21.15.1:0/1949298237 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f90380a3d10 0x7f90380a61d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.926+0000 7f9058a00640 1 --2- 172.21.15.1:0/1949298237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f905410c5b0 0x7f905419a820 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.926+0000 7f9058a00640 1 --2- 172.21.15.1:0/1949298237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9054103b00 0x7f90541a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.926+0000 7f9058a00640 1 --2- 172.21.15.1:0/1949298237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90541031e0 0x7f90541a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.926+0000 7f9058a00640 1 -- 172.21.15.1:0/1949298237 >> 172.21.15.1:0/1949298237 conn(0x7f9054077fd0 msgr2=0x7f9054108490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.926+0000 7f9058a00640 1 -- 172.21.15.1:0/1949298237 shutdown_connections 2024-04-26T01:59:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T01:59:56.927+0000 7f9058a00640 1 -- 172.21.15.1:0/1949298237 wait complete. 2024-04-26T01:59:58.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:57 smithi001 ceph-mon[27822]: pgmap v257: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 71 KiB/s wr, 7 op/s 2024-04-26T01:59:58.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:57 smithi097 ceph-mon[32853]: pgmap v257: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 71 KiB/s wr, 7 op/s 2024-04-26T01:59:58.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:57 smithi173 ceph-mon[32553]: pgmap v257: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 71 KiB/s wr, 7 op/s 2024-04-26T01:59:58.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.774+0000 7fad34e00700 1 -- 172.21.15.1:0/1420003433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad301092d0 msgr2=0x7fad301096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:58.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.774+0000 7fad34e00700 1 --2- 172.21.15.1:0/1420003433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad301092d0 0x7fad301096b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fad18009990 tx=0x7fad18009ca0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.774+0000 7fad34e00700 1 -- 172.21.15.1:0/1420003433 shutdown_connections 2024-04-26T01:59:58.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.774+0000 7fad34e00700 1 --2- 172.21.15.1:0/1420003433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3010e3b0 0x7fad30115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.774+0000 7fad34e00700 1 --2- 172.21.15.1:0/1420003433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad301092d0 0x7fad301096b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.774+0000 7fad34e00700 1 --2- 172.21.15.1:0/1420003433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad30109c80 0x7fad3010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.774+0000 7fad34e00700 1 -- 172.21.15.1:0/1420003433 >> 172.21.15.1:0/1420003433 conn(0x7fad30076210 msgr2=0x7fad30107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:58.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.774+0000 7fad34e00700 1 -- 172.21.15.1:0/1420003433 shutdown_connections 2024-04-26T01:59:58.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.775+0000 7fad34e00700 1 -- 172.21.15.1:0/1420003433 wait complete. 2024-04-26T01:59:58.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.776+0000 7fad34e00700 1 Processor -- start 2024-04-26T01:59:58.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.776+0000 7fad34e00700 1 -- start start 2024-04-26T01:59:58.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.776+0000 7fad34e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad301092d0 0x7fad301a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.776+0000 7fad34e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad30109c80 0x7fad301a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.776+0000 7fad34e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3010e3b0 0x7fad3019f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.776+0000 7fad34e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fad301192b0 con 0x7fad3010e3b0 2024-04-26T01:59:58.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.776+0000 7fad34e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fad30119130 con 0x7fad301092d0 2024-04-26T01:59:58.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.776+0000 7fad34e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fad30119430 con 0x7fad30109c80 2024-04-26T01:59:58.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.777+0000 7fad2de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad301092d0 0x7fad301a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:58.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.777+0000 7fad2de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad301092d0 0x7fad301a5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:46988/0 (socket says 172.21.15.1:46988) 2024-04-26T01:59:58.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.777+0000 7fad2de00700 1 -- 172.21.15.1:0/687841586 learned_addr learned my addr 172.21.15.1:0/687841586 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T01:59:58.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.777+0000 7fad2e800700 1 --2- 172.21.15.1:0/687841586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3010e3b0 0x7fad3019f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:58.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.777+0000 7fad27e00700 1 --2- 172.21.15.1:0/687841586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad30109c80 0x7fad301a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:58.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.777+0000 7fad2de00700 1 -- 172.21.15.1:0/687841586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad30109c80 msgr2=0x7fad301a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:58.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.777+0000 7fad2de00700 1 --2- 172.21.15.1:0/687841586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad30109c80 0x7fad301a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.777+0000 7fad2de00700 1 -- 172.21.15.1:0/687841586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3010e3b0 msgr2=0x7fad3019f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:58.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.777+0000 7fad2de00700 1 --2- 172.21.15.1:0/687841586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3010e3b0 0x7fad3019f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.777+0000 7fad2de00700 1 -- 172.21.15.1:0/687841586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3019fd70 con 0x7fad301092d0 2024-04-26T01:59:58.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.777+0000 7fad2de00700 1 --2- 172.21.15.1:0/687841586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad301092d0 0x7fad301a5400 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fad18005910 tx=0x7fad180047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:58.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.778+0000 7fad26a00700 1 -- 172.21.15.1:0/687841586 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18027070 con 0x7fad301092d0 2024-04-26T01:59:58.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.778+0000 7fad26a00700 1 -- 172.21.15.1:0/687841586 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad1800b610 con 0x7fad301092d0 2024-04-26T01:59:58.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.778+0000 7fad34e00700 1 -- 172.21.15.1:0/687841586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad301a0000 con 0x7fad301092d0 2024-04-26T01:59:58.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.778+0000 7fad26a00700 1 -- 172.21.15.1:0/687841586 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad180173f0 con 0x7fad301092d0 2024-04-26T01:59:58.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.778+0000 7fad34e00700 1 -- 172.21.15.1:0/687841586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad301ac2d0 con 0x7fad301092d0 2024-04-26T01:59:58.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.779+0000 7fad34e00700 1 -- 172.21.15.1:0/687841586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad30069730 con 0x7fad301092d0 2024-04-26T01:59:58.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.781+0000 7fad26a00700 1 -- 172.21.15.1:0/687841586 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fad1802a070 con 0x7fad301092d0 2024-04-26T01:59:58.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.782+0000 7fad26a00700 1 --2- 172.21.15.1:0/687841586 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fad140a3e30 0x7fad140a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.783+0000 7fad26a00700 1 -- 172.21.15.1:0/687841586 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fad180d1780 con 0x7fad301092d0 2024-04-26T01:59:58.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.783+0000 7fad27e00700 1 --2- 172.21.15.1:0/687841586 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fad140a3e30 0x7fad140a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:58.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.784+0000 7fad27e00700 1 --2- 172.21.15.1:0/687841586 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fad140a3e30 0x7fad140a62f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fad30108910 tx=0x7fad20009260 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:58.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:58.785+0000 7fad26a00700 1 -- 172.21.15.1:0/687841586 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad18013070 con 0x7fad301092d0 2024-04-26T01:59:59.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:59.122+0000 7fad34e00700 1 -- 172.21.15.1:0/687841586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad3002df80 con 0x7fad301092d0 2024-04-26T01:59:59.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:59.123+0000 7fad26a00700 1 -- 172.21.15.1:0/687841586 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11838 (secure 0 0 0) 0x7fad1809de20 con 0x7fad301092d0 2024-04-26T01:59:59.122 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T01:59:59.122 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:55:49.645549+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25414":{"gid":25414,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/453427322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":453427322},{"type":"v1","addr":"172.21.15.173:6835","nonce":453427322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24499,24514,24518]},"id":1}]} 2024-04-26T01:59:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:59.126+0000 7fad34e00700 1 -- 172.21.15.1:0/687841586 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fad140a3e30 msgr2=0x7fad140a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:59.126+0000 7fad34e00700 1 --2- 172.21.15.1:0/687841586 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fad140a3e30 0x7fad140a62f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fad30108910 tx=0x7fad20009260 comp rx=0 tx=0).stop 2024-04-26T01:59:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:59.126+0000 7fad34e00700 1 -- 172.21.15.1:0/687841586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad301092d0 msgr2=0x7fad301a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:59.126+0000 7fad34e00700 1 --2- 172.21.15.1:0/687841586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad301092d0 0x7fad301a5400 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fad18005910 tx=0x7fad180047b0 comp rx=0 tx=0).stop 2024-04-26T01:59:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:59.126+0000 7fad34e00700 1 -- 172.21.15.1:0/687841586 shutdown_connections 2024-04-26T01:59:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:59.126+0000 7fad34e00700 1 --2- 172.21.15.1:0/687841586 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fad140a3e30 0x7fad140a62f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:59.126+0000 7fad34e00700 1 --2- 172.21.15.1:0/687841586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3010e3b0 0x7fad3019f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:59.126+0000 7fad34e00700 1 --2- 172.21.15.1:0/687841586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad301092d0 0x7fad301a5400 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:59.126+0000 7fad34e00700 1 --2- 172.21.15.1:0/687841586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad30109c80 0x7fad301a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:59.126+0000 7fad34e00700 1 -- 172.21.15.1:0/687841586 >> 172.21.15.1:0/687841586 conn(0x7fad30076210 msgr2=0x7fad30077e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:59.126+0000 7fad34e00700 1 -- 172.21.15.1:0/687841586 shutdown_connections 2024-04-26T01:59:59.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T01:59:59.126+0000 7fad34e00700 1 -- 172.21.15.1:0/687841586 wait complete. 2024-04-26T01:59:59.126 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 32 2024-04-26T01:59:59.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:58 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:59:59.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:58 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:59:59.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:58 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:00:00.034 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24518 quiesce db --set-id a9fc8138 --release --await' 2024-04-26T02:00:00.034 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24518 quiesce db --set-id a9fc8138 --release --await 2024-04-26T02:00:00.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:59 smithi173 ceph-mon[32553]: pgmap v258: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 71 KiB/s wr, 7 op/s 2024-04-26T02:00:00.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 01:59:59 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/687841586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:00:00.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:59 smithi001 ceph-mon[27822]: pgmap v258: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 71 KiB/s wr, 7 op/s 2024-04-26T02:00:00.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 01:59:59 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/687841586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:00:00.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:59 smithi097 ceph-mon[32853]: pgmap v258: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 71 KiB/s wr, 7 op/s 2024-04-26T02:00:00.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 01:59:59 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/687841586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:00:00.509 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:00:01.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:00 smithi173 ceph-mon[32553]: overall HEALTH_OK 2024-04-26T02:00:01.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:00 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:00:01.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:00 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:00:01.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:00 smithi001 ceph-mon[27822]: overall HEALTH_OK 2024-04-26T02:00:01.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:00 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:00:01.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:00 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:00:01.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:00 smithi097 ceph-mon[32853]: overall HEALTH_OK 2024-04-26T02:00:01.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:00 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:00:01.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:00 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:00:02.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:02 smithi173 ceph-mon[32553]: pgmap v259: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 77 KiB/s wr, 12 op/s 2024-04-26T02:00:02.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:02 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:00:02.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:02 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:00:02.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:02 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:00:02.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:02 smithi001 ceph-mon[27822]: pgmap v259: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 77 KiB/s wr, 12 op/s 2024-04-26T02:00:02.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:02 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:00:02.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:02 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:00:02.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:02 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:00:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:02 smithi097 ceph-mon[32853]: pgmap v259: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 77 KiB/s wr, 12 op/s 2024-04-26T02:00:02.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:02 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:00:02.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:02 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:00:02.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:02 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:00:02.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.740+0000 7f0786600700 1 -- 172.21.15.1:0/3058618427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0780102af0 msgr2=0x7f0780102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:02.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.740+0000 7f0786600700 1 --2- 172.21.15.1:0/3058618427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0780102af0 0x7f0780102f70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f076c0099c0 tx=0x7f076c009cd0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.741+0000 7f0786600700 1 -- 172.21.15.1:0/3058618427 shutdown_connections 2024-04-26T02:00:02.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.741+0000 7f0786600700 1 --2- 172.21.15.1:0/3058618427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07801034b0 0x7f078010f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.741+0000 7f0786600700 1 --2- 172.21.15.1:0/3058618427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0780107940 0x7f0780107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.741+0000 7f0786600700 1 --2- 172.21.15.1:0/3058618427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0780102af0 0x7f0780102f70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.741+0000 7f0786600700 1 -- 172.21.15.1:0/3058618427 >> 172.21.15.1:0/3058618427 conn(0x7f0780079ba0 msgr2=0x7f0780079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:02.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.741+0000 7f0786600700 1 -- 172.21.15.1:0/3058618427 shutdown_connections 2024-04-26T02:00:02.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.741+0000 7f0786600700 1 -- 172.21.15.1:0/3058618427 wait complete. 2024-04-26T02:00:02.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.742+0000 7f0786600700 1 Processor -- start 2024-04-26T02:00:02.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.742+0000 7f0786600700 1 -- start start 2024-04-26T02:00:02.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.743+0000 7f0786600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0780102af0 0x7f078019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:02.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.743+0000 7f0786600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07801034b0 0x7f078019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:02.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.743+0000 7f0786600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0780107940 0x7f078019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:02.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.743+0000 7f0786600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0780112c90 con 0x7f0780102af0 2024-04-26T02:00:02.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.743+0000 7f0786600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0780112b10 con 0x7f0780107940 2024-04-26T02:00:02.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.743+0000 7f0786600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0780112e10 con 0x7f07801034b0 2024-04-26T02:00:02.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.743+0000 7f077f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0780102af0 0x7f078019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:02.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.743+0000 7f077f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0780102af0 0x7f078019b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57846/0 (socket says 172.21.15.1:57846) 2024-04-26T02:00:02.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.743+0000 7f077f400700 1 -- 172.21.15.1:0/3122744346 learned_addr learned my addr 172.21.15.1:0/3122744346 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:02.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.743+0000 7f077fe00700 1 --2- 172.21.15.1:0/3122744346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0780107940 0x7f078019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:02.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.743+0000 7f077ea00700 1 --2- 172.21.15.1:0/3122744346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07801034b0 0x7f078019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:02.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.744+0000 7f077f400700 1 -- 172.21.15.1:0/3122744346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07801034b0 msgr2=0x7f078019b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:02.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.744+0000 7f077f400700 1 --2- 172.21.15.1:0/3122744346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07801034b0 0x7f078019b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.744+0000 7f077f400700 1 -- 172.21.15.1:0/3122744346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0780107940 msgr2=0x7f078019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:02.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.744+0000 7f077f400700 1 --2- 172.21.15.1:0/3122744346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0780107940 0x7f078019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.744+0000 7f077f400700 1 -- 172.21.15.1:0/3122744346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07801a0400 con 0x7f0780102af0 2024-04-26T02:00:02.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.744+0000 7f077f400700 1 --2- 172.21.15.1:0/3122744346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0780102af0 0x7f078019b3f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f077000e9a0 tx=0x7f077000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:02.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.744+0000 7f0777e00700 1 -- 172.21.15.1:0/3122744346 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077000ca10 con 0x7f0780102af0 2024-04-26T02:00:02.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.744+0000 7f0777e00700 1 -- 172.21.15.1:0/3122744346 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f077000cbb0 con 0x7f0780102af0 2024-04-26T02:00:02.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.744+0000 7f0786600700 1 -- 172.21.15.1:0/3122744346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07801a06f0 con 0x7f0780102af0 2024-04-26T02:00:02.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.745+0000 7f0777e00700 1 -- 172.21.15.1:0/3122744346 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770018680 con 0x7f0780102af0 2024-04-26T02:00:02.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.745+0000 7f0786600700 1 -- 172.21.15.1:0/3122744346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07801a7f70 con 0x7f0780102af0 2024-04-26T02:00:02.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.748+0000 7f0777e00700 1 -- 172.21.15.1:0/3122744346 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f0770010430 con 0x7f0780102af0 2024-04-26T02:00:02.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.749+0000 7f0790fe3b80 1 Processor -- start 2024-04-26T02:00:02.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.750+0000 7f0790fe3b80 1 -- start start 2024-04-26T02:00:02.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.750+0000 7f0790fe3b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfaaeae920 0x55dfaaeaed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:02.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.750+0000 7f0790fe3b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeaf240 0x55dfaaeb9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:02.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.750+0000 7f0790fe3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeba0f0 0x55dfaaebc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:02.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.750+0000 7f0790fe3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55dfaae01f90 con 0x55dfaaeba0f0 2024-04-26T02:00:02.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.750+0000 7f0790fe3b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dfaada8830 con 0x55dfaaeae920 2024-04-26T02:00:02.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.750+0000 7f0790fe3b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55dfaadfdf50 con 0x55dfaaeaf240 2024-04-26T02:00:02.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.750+0000 7f077f400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeaf240 0x55dfaaeb9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:02.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.750+0000 7f077ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeba0f0 0x55dfaaebc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:02.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.750+0000 7f077fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfaaeae920 0x55dfaaeaed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:02.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.750+0000 7f077fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfaaeae920 0x55dfaaeaed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:57186/0 (socket says 172.21.15.1:57186) 2024-04-26T02:00:02.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.750+0000 7f077fe00700 1 -- 172.21.15.1:0/2214614561 learned_addr learned my addr 172.21.15.1:0/2214614561 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:02.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.751+0000 7f0777e00700 1 --2- 172.21.15.1:0/3122744346 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f07640a3db0 0x7f07640a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:02.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.751+0000 7f077f400700 1 -- 172.21.15.1:0/2214614561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfaaeae920 msgr2=0x55dfaaeaed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:02.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.751+0000 7f077f400700 1 --2- 172.21.15.1:0/2214614561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfaaeae920 0x55dfaaeaed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.751+0000 7f077f400700 1 -- 172.21.15.1:0/2214614561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeba0f0 msgr2=0x55dfaaebc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:02.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.751+0000 7f077f400700 1 --2- 172.21.15.1:0/2214614561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeba0f0 0x55dfaaebc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.751+0000 7f077f400700 1 -- 172.21.15.1:0/2214614561 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfaad9e780 con 0x55dfaaeaf240 2024-04-26T02:00:02.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.751+0000 7f077fe00700 1 --2- 172.21.15.1:0/3122744346 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f07640a3db0 0x7f07640a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:02.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.751+0000 7f077f400700 1 --2- 172.21.15.1:0/2214614561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeaf240 0x55dfaaeb9bb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f07700ce060 tx=0x7f0770014600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1ba7939abf330cc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:02.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.751+0000 7f0777e00700 1 -- 172.21.15.1:0/3122744346 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f07700d09a0 con 0x7f0780102af0 2024-04-26T02:00:02.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.752+0000 7f0774c00700 1 -- 172.21.15.1:0/2214614561 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077009c3f0 con 0x55dfaaeaf240 2024-04-26T02:00:02.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.752+0000 7f0774c00700 1 -- 172.21.15.1:0/2214614561 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07700673c0 con 0x55dfaaeaf240 2024-04-26T02:00:02.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.752+0000 7f0774c00700 1 -- 172.21.15.1:0/2214614561 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077009c210 con 0x55dfaaeaf240 2024-04-26T02:00:02.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.753+0000 7f0790fe3b80 1 -- 172.21.15.1:0/2214614561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeaf240 msgr2=0x55dfaaeb9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:02.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.753+0000 7f077fe00700 1 --2- 172.21.15.1:0/3122744346 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f07640a3db0 0x7f07640a6270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0778009fd0 tx=0x7f0778004ad0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:02.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.753+0000 7f0790fe3b80 1 --2- 172.21.15.1:0/2214614561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeaf240 0x55dfaaeb9bb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f07700ce060 tx=0x7f0770014600 comp rx=0 tx=0).stop 2024-04-26T02:00:02.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.753+0000 7f0790fe3b80 1 -- 172.21.15.1:0/2214614561 shutdown_connections 2024-04-26T02:00:02.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.753+0000 7f0790fe3b80 1 --2- 172.21.15.1:0/2214614561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeba0f0 0x55dfaaebc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.753+0000 7f0790fe3b80 1 --2- 172.21.15.1:0/2214614561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfaaeae920 0x55dfaaeaed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.753+0000 7f0790fe3b80 1 --2- 172.21.15.1:0/2214614561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeaf240 0x55dfaaeb9bb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.753+0000 7f0790fe3b80 1 -- 172.21.15.1:0/2214614561 >> 172.21.15.1:0/2214614561 conn(0x55dfaaddd6a0 msgr2=0x55dfaaeaadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:02.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.753+0000 7f0790fe3b80 1 -- 172.21.15.1:0/2214614561 shutdown_connections 2024-04-26T02:00:02.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.753+0000 7f0790fe3b80 1 -- 172.21.15.1:0/2214614561 wait complete. 2024-04-26T02:00:02.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.753+0000 7f0790fe3b80 1 -- start start 2024-04-26T02:00:02.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.754+0000 7f0790fe3b80 1 Processor -- start 2024-04-26T02:00:02.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.754+0000 7f0790fe3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeae920 0x55dfaaeb97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:02.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.754+0000 7f0790fe3b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeaf240 0x55dfaaeb6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:02.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.754+0000 7f0790fe3b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfaaeba0f0 0x55dfaaeb6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:02.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.754+0000 7f0790fe3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55dfaaebcbb0 con 0x55dfaaeae920 2024-04-26T02:00:02.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.754+0000 7f0790fe3b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dfaadbb960 con 0x55dfaaeba0f0 2024-04-26T02:00:02.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.754+0000 7f0790fe3b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55dfaaebcd30 con 0x55dfaaeaf240 2024-04-26T02:00:02.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.754+0000 7f0790fe3b80 10 client.0 _ll_register_callbacks cb 0x55dfaad9a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:02.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.755+0000 7f077f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeae920 0x55dfaaeb97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:02.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.755+0000 7f077ea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeaf240 0x55dfaaeb6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:02.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.755+0000 7f077f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeae920 0x55dfaaeb97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57874/0 (socket says 172.21.15.1:57874) 2024-04-26T02:00:02.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.755+0000 7f077f400700 1 -- 172.21.15.1:0/1220065433 learned_addr learned my addr 172.21.15.1:0/1220065433 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:02.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.755+0000 7f077ea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeaf240 0x55dfaaeb6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:49158/0 (socket says 172.21.15.1:49158) 2024-04-26T02:00:02.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.755+0000 7f077ea00700 1 -- 172.21.15.1:0/1220065433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfaaeba0f0 msgr2=0x55dfaaeb6ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T02:00:02.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.755+0000 7f077ea00700 1 --2- 172.21.15.1:0/1220065433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfaaeba0f0 0x55dfaaeb6ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.755+0000 7f077ea00700 1 -- 172.21.15.1:0/1220065433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeae920 msgr2=0x55dfaaeb97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:02.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.755+0000 7f077ea00700 1 --2- 172.21.15.1:0/1220065433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeae920 0x55dfaaeb97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.755+0000 7f077ea00700 1 -- 172.21.15.1:0/1220065433 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfaad4f9d0 con 0x55dfaaeaf240 2024-04-26T02:00:02.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.755+0000 7f077f400700 1 --2- 172.21.15.1:0/1220065433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeae920 0x55dfaaeb97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:02.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.755+0000 7f077ea00700 1 --2- 172.21.15.1:0/1220065433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeaf240 0x55dfaaeb6ab0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f076c00b580 tx=0x7f076c0057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6f2e4530d103806 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:02.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.755+0000 7f0769600700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T02:00:02.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.756+0000 7f0769600700 1 -- 172.21.15.1:0/1220065433 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c0039c0 con 0x55dfaaeaf240 2024-04-26T02:00:02.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.756+0000 7f0790fe3b80 1 -- 172.21.15.1:0/1220065433 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07700673c0 con 0x55dfaaeaf240 2024-04-26T02:00:02.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.756+0000 7f0769600700 1 -- 172.21.15.1:0/1220065433 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f076c003b60 con 0x55dfaaeaf240 2024-04-26T02:00:02.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.756+0000 7f0769600700 1 -- 172.21.15.1:0/1220065433 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f076c032050 con 0x55dfaaeaf240 2024-04-26T02:00:02.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.756+0000 7f0790fe3b80 10 client.26200 fetch_fsmap learned FSMap version 32 2024-04-26T02:00:02.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.756+0000 7f0790fe3b80 1 -- 172.21.15.1:0/1220065433 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dfaad98bf0 con 0x55dfaaeaf240 2024-04-26T02:00:02.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.756+0000 7f0769600700 1 -- 172.21.15.1:0/1220065433 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07700673c0 con 0x55dfaaeaf240 2024-04-26T02:00:02.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.757+0000 7f0769600700 1 -- 172.21.15.1:0/1220065433 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f076c00f460 con 0x55dfaaeaf240 2024-04-26T02:00:02.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.757+0000 7f0790fe3b80 10 client.26200 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:00:02.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.757+0000 7f0790fe3b80 10 client.26200 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T02:00:02.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.757+0000 7f0790fe3b80 20 client.26200 populate_metadata read hostname 'smithi001' 2024-04-26T02:00:02.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.757+0000 7f0790fe3b80 1 --2- 172.21.15.1:0/1220065433 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55dfaaec97f0 0x55dfaaecbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:02.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.757+0000 7f0790fe3b80 4 client.26200 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:00:02.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.757+0000 7f0790fe3b80 1 -- 172.21.15.1:0/1220065433 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfaae0f780 con 0x55dfaaec97f0 2024-04-26T02:00:02.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.757+0000 7f077f400700 1 --2- 172.21.15.1:0/1220065433 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55dfaaec97f0 0x55dfaaecbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:02.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.758+0000 7f077f400700 1 --2- 172.21.15.1:0/1220065433 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55dfaaec97f0 0x55dfaaecbbe0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51d5404b478c92dd server_cookie=71c1671ed9ae68e4 in_seq=0 out_seq=0 2024-04-26T02:00:02.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.758+0000 7f0769600700 10 client.26200 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:02.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.759+0000 7f0769600700 1 -- 172.21.15.1:0/1220065433 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dfaae0f780 con 0x55dfaaec97f0 2024-04-26T02:00:02.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.759+0000 7f0769600700 10 client.26200 handle_command_reply: tid=0 2024-04-26T02:00:02.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.759+0000 7f0790fe3b80 1 client.26200 shutdown 2024-04-26T02:00:02.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.759+0000 7f0790fe3b80 1 -- 172.21.15.1:0/1220065433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeaf240 msgr2=0x55dfaaeb6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:02.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.759+0000 7f0790fe3b80 1 --2- 172.21.15.1:0/1220065433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeaf240 0x55dfaaeb6ab0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f076c00b580 tx=0x7f076c0057a0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.759+0000 7f0790fe3b80 1 -- 172.21.15.1:0/1220065433 shutdown_connections 2024-04-26T02:00:02.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.759+0000 7f0790fe3b80 1 --2- 172.21.15.1:0/1220065433 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55dfaaec97f0 0x55dfaaecbbe0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.759+0000 7f0790fe3b80 1 --2- 172.21.15.1:0/1220065433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfaaeba0f0 0x55dfaaeb6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.759+0000 7f0790fe3b80 1 --2- 172.21.15.1:0/1220065433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeae920 0x55dfaaeb97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.759+0000 7f0790fe3b80 1 --2- 172.21.15.1:0/1220065433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeaf240 0x55dfaaeb6ab0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.759+0000 7f0769600700 0 client.26200 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:02.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.759+0000 7f0790fe3b80 1 -- 172.21.15.1:0/1220065433 >> 172.21.15.1:0/1220065433 conn(0x55dfaaddd6a0 msgr2=0x55dfaaeaa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:02.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.759+0000 7f0790fe3b80 1 -- 172.21.15.1:0/1220065433 shutdown_connections 2024-04-26T02:00:02.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.759+0000 7f0790fe3b80 1 -- 172.21.15.1:0/1220065433 wait complete. 2024-04-26T02:00:02.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.760+0000 7f0790fe3b80 20 client.26200 trim_cache size 0 max 16384 2024-04-26T02:00:02.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.784+0000 7f0790fe3b80 1 Processor -- start 2024-04-26T02:00:02.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.784+0000 7f0790fe3b80 1 -- start start 2024-04-26T02:00:02.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.785+0000 7f0790fe3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeae920 0x55dfaaec5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:02.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.785+0000 7f0790fe3b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfaaeaf240 0x55dfaaec1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:02.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.785+0000 7f0790fe3b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeba0f0 0x55dfaaec2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:02.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.785+0000 7f0790fe3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f07801b1f30 con 0x55dfaaeae920 2024-04-26T02:00:02.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.785+0000 7f0790fe3b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f07801af7c0 con 0x55dfaaeaf240 2024-04-26T02:00:02.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.785+0000 7f0790fe3b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f07801ad630 con 0x55dfaaeba0f0 2024-04-26T02:00:02.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.785+0000 7f077f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeae920 0x55dfaaec5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:02.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.785+0000 7f077f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeae920 0x55dfaaec5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57878/0 (socket says 172.21.15.1:57878) 2024-04-26T02:00:02.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.785+0000 7f077f400700 1 -- 172.21.15.1:0/1595045026 learned_addr learned my addr 172.21.15.1:0/1595045026 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:02.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.785+0000 7f077fe00700 1 --2- 172.21.15.1:0/1595045026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeba0f0 0x55dfaaec2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:02.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.785+0000 7f077ea00700 1 --2- 172.21.15.1:0/1595045026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfaaeaf240 0x55dfaaec1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:02.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.786+0000 7f077f400700 1 -- 172.21.15.1:0/1595045026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeba0f0 msgr2=0x55dfaaec2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:02.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.786+0000 7f077f400700 1 --2- 172.21.15.1:0/1595045026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeba0f0 0x55dfaaec2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.786+0000 7f077f400700 1 -- 172.21.15.1:0/1595045026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfaaeaf240 msgr2=0x55dfaaec1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:02.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.786+0000 7f077f400700 1 --2- 172.21.15.1:0/1595045026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfaaeaf240 0x55dfaaec1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.786+0000 7f077f400700 1 -- 172.21.15.1:0/1595045026 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfaaecc260 con 0x55dfaaeae920 2024-04-26T02:00:02.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.786+0000 7f077ea00700 1 --2- 172.21.15.1:0/1595045026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfaaeaf240 0x55dfaaec1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:00:02.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.786+0000 7f077f400700 1 --2- 172.21.15.1:0/1595045026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeae920 0x55dfaaec5ae0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0770099280 tx=0x7f07700d64b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b9bb7a5ca3b020 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:02.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.786+0000 7f076be00700 1 -- 172.21.15.1:0/1595045026 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07700a0c60 con 0x55dfaaeae920 2024-04-26T02:00:02.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.786+0000 7f076be00700 1 -- 172.21.15.1:0/1595045026 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07700a5520 con 0x55dfaaeae920 2024-04-26T02:00:02.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.787+0000 7f076be00700 1 -- 172.21.15.1:0/1595045026 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07700ce2e0 con 0x55dfaaeae920 2024-04-26T02:00:02.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.787+0000 7f0790fe3b80 1 -- 172.21.15.1:0/1595045026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeae920 msgr2=0x55dfaaec5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:02.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.787+0000 7f0790fe3b80 1 --2- 172.21.15.1:0/1595045026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeae920 0x55dfaaec5ae0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0770099280 tx=0x7f07700d64b0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.787+0000 7f0790fe3b80 1 -- 172.21.15.1:0/1595045026 shutdown_connections 2024-04-26T02:00:02.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.787+0000 7f0790fe3b80 1 --2- 172.21.15.1:0/1595045026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeba0f0 0x55dfaaec2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.787+0000 7f0790fe3b80 1 --2- 172.21.15.1:0/1595045026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeae920 0x55dfaaec5ae0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.787+0000 7f0790fe3b80 1 --2- 172.21.15.1:0/1595045026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfaaeaf240 0x55dfaaec1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.787+0000 7f0790fe3b80 1 -- 172.21.15.1:0/1595045026 >> 172.21.15.1:0/1595045026 conn(0x55dfaaddd6a0 msgr2=0x55dfaaea9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:02.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.788+0000 7f0790fe3b80 1 -- 172.21.15.1:0/1595045026 shutdown_connections 2024-04-26T02:00:02.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.788+0000 7f0790fe3b80 1 -- 172.21.15.1:0/1595045026 wait complete. 2024-04-26T02:00:02.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.788+0000 7f0790fe3b80 1 -- start start 2024-04-26T02:00:02.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.788+0000 7f0790fe3b80 1 Processor -- start 2024-04-26T02:00:02.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.789+0000 7f0790fe3b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeae920 0x55dfaaec23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:02.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.789+0000 7f0790fe3b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfaaeaf240 0x55dfaaeb08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:02.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.789+0000 7f0790fe3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeba0f0 0x55dfaaeb0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:02.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.789+0000 7f0790fe3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f07801adc80 con 0x55dfaaeba0f0 2024-04-26T02:00:02.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.789+0000 7f0790fe3b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f07801af0f0 con 0x55dfaaeaf240 2024-04-26T02:00:02.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.789+0000 7f0790fe3b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f07801ae350 con 0x55dfaaeae920 2024-04-26T02:00:02.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.789+0000 7f0790fe3b80 10 client.0 _ll_register_callbacks cb 0x55dfaad9a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:02.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.789+0000 7f077fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeba0f0 0x55dfaaeb0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:02.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.789+0000 7f077fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeba0f0 0x55dfaaeb0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57890/0 (socket says 172.21.15.1:57890) 2024-04-26T02:00:02.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.789+0000 7f077fe00700 1 -- 172.21.15.1:0/494483741 learned_addr learned my addr 172.21.15.1:0/494483741 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:02.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.789+0000 7f077f400700 1 --2- 172.21.15.1:0/494483741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeae920 0x55dfaaec23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:02.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.789+0000 7f077ea00700 1 --2- 172.21.15.1:0/494483741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfaaeaf240 0x55dfaaeb08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:02.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.790+0000 7f077fe00700 1 -- 172.21.15.1:0/494483741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeae920 msgr2=0x55dfaaec23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:02.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.790+0000 7f077fe00700 1 --2- 172.21.15.1:0/494483741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeae920 0x55dfaaec23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.790+0000 7f077fe00700 1 -- 172.21.15.1:0/494483741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfaaeaf240 msgr2=0x55dfaaeb08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:02.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.790+0000 7f077fe00700 1 --2- 172.21.15.1:0/494483741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfaaeaf240 0x55dfaaeb08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.790+0000 7f077fe00700 1 -- 172.21.15.1:0/494483741 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfaaeb1600 con 0x55dfaaeba0f0 2024-04-26T02:00:02.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.790+0000 7f077fe00700 1 --2- 172.21.15.1:0/494483741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeba0f0 0x55dfaaeb0e00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0778017e10 tx=0x7f077800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a154fd45202e379f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:02.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.790+0000 7f0769600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T02:00:02.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.790+0000 7f0769600700 1 -- 172.21.15.1:0/494483741 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0778029070 con 0x55dfaaeba0f0 2024-04-26T02:00:02.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.790+0000 7f0790fe3b80 1 -- 172.21.15.1:0/494483741 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07700a5520 con 0x55dfaaeba0f0 2024-04-26T02:00:02.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.791+0000 7f0769600700 1 -- 172.21.15.1:0/494483741 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f0778016c90 con 0x55dfaaeba0f0 2024-04-26T02:00:02.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.791+0000 7f0769600700 1 -- 172.21.15.1:0/494483741 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0778025400 con 0x55dfaaeba0f0 2024-04-26T02:00:02.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.791+0000 7f0790fe3b80 10 client.16035 fetch_fsmap learned FSMap version 32 2024-04-26T02:00:02.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.791+0000 7f0790fe3b80 1 -- 172.21.15.1:0/494483741 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dfaaf30ea0 con 0x55dfaaeba0f0 2024-04-26T02:00:02.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.791+0000 7f0769600700 1 -- 172.21.15.1:0/494483741 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07700a5520 con 0x55dfaaeba0f0 2024-04-26T02:00:02.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.791+0000 7f0769600700 1 -- 172.21.15.1:0/494483741 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f077802d8e0 con 0x55dfaaeba0f0 2024-04-26T02:00:02.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.792+0000 7f0790fe3b80 10 client.16035 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:00:02.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.792+0000 7f0790fe3b80 10 client.16035 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T02:00:02.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.792+0000 7f0790fe3b80 20 client.16035 populate_metadata read hostname 'smithi001' 2024-04-26T02:00:02.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.792+0000 7f0790fe3b80 1 --2- 172.21.15.1:0/494483741 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55dfaaf31610 0x55dfaaf33a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:02.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.792+0000 7f0790fe3b80 4 client.16035 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a9fc8138", "release": true, "await": true}] 2024-04-26T02:00:02.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.792+0000 7f0790fe3b80 1 -- 172.21.15.1:0/494483741 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a9fc8138", "release": true, "await": true}) -- 0x55dfaae09820 con 0x55dfaaf31610 2024-04-26T02:00:02.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.792+0000 7f077f400700 1 --2- 172.21.15.1:0/494483741 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55dfaaf31610 0x55dfaaf33a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:02.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.792+0000 7f077f400700 1 --2- 172.21.15.1:0/494483741 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55dfaaf31610 0x55dfaaf33a30 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4ddce7fc8ccfe4f server_cookie=3d21b6dbec523199 in_seq=0 out_seq=0 2024-04-26T02:00:02.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.792+0000 7f0769600700 10 client.16035 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:02.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.981+0000 7f0769600700 1 -- 172.21.15.1:0/494483741 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55dfaae09820 con 0x55dfaaf31610 2024-04-26T02:00:02.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.981+0000 7f0769600700 10 client.16035 handle_command_reply: tid=0 2024-04-26T02:00:02.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.981+0000 7f0790fe3b80 1 client.16035 shutdown 2024-04-26T02:00:02.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.982+0000 7f0790fe3b80 1 -- 172.21.15.1:0/494483741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeba0f0 msgr2=0x55dfaaeb0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:02.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.982+0000 7f0790fe3b80 1 --2- 172.21.15.1:0/494483741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeba0f0 0x55dfaaeb0e00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0778017e10 tx=0x7f077800b040 comp rx=0 tx=0).stop 2024-04-26T02:00:02.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.982+0000 7f0790fe3b80 1 -- 172.21.15.1:0/494483741 shutdown_connections 2024-04-26T02:00:02.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.982+0000 7f0790fe3b80 1 --2- 172.21.15.1:0/494483741 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55dfaaf31610 0x55dfaaf33a30 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.982+0000 7f0790fe3b80 1 --2- 172.21.15.1:0/494483741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfaaeba0f0 0x55dfaaeb0e00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.982+0000 7f0790fe3b80 1 --2- 172.21.15.1:0/494483741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dfaaeae920 0x55dfaaec23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.982+0000 7f0790fe3b80 1 --2- 172.21.15.1:0/494483741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dfaaeaf240 0x55dfaaeb08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.982+0000 7f0769600700 0 client.16035 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:02.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.982+0000 7f0790fe3b80 1 -- 172.21.15.1:0/494483741 >> 172.21.15.1:0/494483741 conn(0x55dfaaddd6a0 msgr2=0x55dfaaeaa790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:02.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.982+0000 7f0790fe3b80 1 -- 172.21.15.1:0/494483741 shutdown_connections 2024-04-26T02:00:02.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.983+0000 7f0790fe3b80 1 -- 172.21.15.1:0/494483741 wait complete. 2024-04-26T02:00:02.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.983+0000 7f0790fe3b80 20 client.16035 trim_cache size 0 max 16384 2024-04-26T02:00:02.983 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T02:00:02.983 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 32, 2024-04-26T02:00:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24518, 2024-04-26T02:00:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 24, 2024-04-26T02:00:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-04-26T02:00:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: "a9fc8138": { 2024-04-26T02:00:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 24, 2024-04-26T02:00:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-04-26T02:00:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T02:00:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-04-26T02:00:02.984 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T02:00:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:00:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-04-26T02:00:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 82.9, 2024-04-26T02:00:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-04-26T02:00:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-04-26T02:00:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-04-26T02:00:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T02:00:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-04-26T02:00:02.985 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T02:00:02.986 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:00:02.986 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:00:02.986 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:00:02.986 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:00:02.986 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:00:02.986 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T02:00:02.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.985+0000 7f0786600700 1 -- 172.21.15.1:0/3122744346 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f07640a3db0 msgr2=0x7f07640a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:02.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.985+0000 7f0786600700 1 --2- 172.21.15.1:0/3122744346 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f07640a3db0 0x7f07640a6270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0778009fd0 tx=0x7f0778004ad0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.985+0000 7f0786600700 1 -- 172.21.15.1:0/3122744346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0780102af0 msgr2=0x7f078019b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:02.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.985+0000 7f0786600700 1 --2- 172.21.15.1:0/3122744346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0780102af0 0x7f078019b3f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f077000e9a0 tx=0x7f077000ecb0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.985+0000 7f0786600700 1 -- 172.21.15.1:0/3122744346 shutdown_connections 2024-04-26T02:00:02.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.985+0000 7f0786600700 1 --2- 172.21.15.1:0/3122744346 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f07640a3db0 0x7f07640a6270 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.985+0000 7f0786600700 1 --2- 172.21.15.1:0/3122744346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0780107940 0x7f078019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.985+0000 7f0786600700 1 --2- 172.21.15.1:0/3122744346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0780102af0 0x7f078019b3f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.985+0000 7f0786600700 1 --2- 172.21.15.1:0/3122744346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07801034b0 0x7f078019b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:02.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.985+0000 7f0786600700 1 -- 172.21.15.1:0/3122744346 >> 172.21.15.1:0/3122744346 conn(0x7f0780079ba0 msgr2=0x7f0780100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:02.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.985+0000 7f0786600700 1 -- 172.21.15.1:0/3122744346 shutdown_connections 2024-04-26T02:00:02.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:02.986+0000 7f0786600700 1 -- 172.21.15.1:0/3122744346 wait complete. 2024-04-26T02:00:04.111 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a9fc8138', seconds elapsed: 39.9 2024-04-26T02:00:04.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:04 smithi173 ceph-mon[32553]: pgmap v260: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.9 KiB/s wr, 7 op/s 2024-04-26T02:00:04.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:04 smithi001 ceph-mon[27822]: pgmap v260: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.9 KiB/s wr, 7 op/s 2024-04-26T02:00:04.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:04 smithi097 ceph-mon[32853]: pgmap v260: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.9 KiB/s wr, 7 op/s 2024-04-26T02:00:06.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:06 smithi097 ceph-mon[32853]: pgmap v261: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 230 KiB/s wr, 78 op/s 2024-04-26T02:00:06.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:06 smithi097 ceph-mon[32853]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T02:00:06.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:06 smithi173 ceph-mon[32553]: pgmap v261: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 230 KiB/s wr, 78 op/s 2024-04-26T02:00:06.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:06 smithi173 ceph-mon[32553]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T02:00:06.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:06 smithi001 ceph-mon[27822]: pgmap v261: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 230 KiB/s wr, 78 op/s 2024-04-26T02:00:06.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:06 smithi001 ceph-mon[27822]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T02:00:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:08 smithi097 ceph-mon[32853]: pgmap v263: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 273 KiB/s wr, 91 op/s 2024-04-26T02:00:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:08 smithi173 ceph-mon[32553]: pgmap v263: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 273 KiB/s wr, 91 op/s 2024-04-26T02:00:08.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:08 smithi001 ceph-mon[27822]: pgmap v263: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 273 KiB/s wr, 91 op/s 2024-04-26T02:00:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.082+0000 7f583a200640 1 -- 172.21.15.1:0/1579886149 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5834074ca0 msgr2=0x7f583410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.082+0000 7f583a200640 1 --2- 172.21.15.1:0/1579886149 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5834074ca0 0x7f583410e240 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f581c009900 tx=0x7f581c02a480 comp rx=0 tx=0).stop 2024-04-26T02:00:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.082+0000 7f583a200640 1 -- 172.21.15.1:0/1579886149 shutdown_connections 2024-04-26T02:00:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.082+0000 7f583a200640 1 --2- 172.21.15.1:0/1579886149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f583410e780 0x7f5834110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.082+0000 7f583a200640 1 --2- 172.21.15.1:0/1579886149 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5834074ca0 0x7f583410e240 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.082+0000 7f583a200640 1 --2- 172.21.15.1:0/1579886149 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5834074380 0x7f5834074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.082+0000 7f583a200640 1 -- 172.21.15.1:0/1579886149 >> 172.21.15.1:0/1579886149 conn(0x7f58340fda10 msgr2=0x7f58340ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.083+0000 7f583a200640 1 -- 172.21.15.1:0/1579886149 shutdown_connections 2024-04-26T02:00:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.083+0000 7f583a200640 1 -- 172.21.15.1:0/1579886149 wait complete. 2024-04-26T02:00:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.084+0000 7f583a200640 1 Processor -- start 2024-04-26T02:00:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.084+0000 7f583a200640 1 -- start start 2024-04-26T02:00:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.084+0000 7f583a200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5834074380 0x7f5834101ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.084+0000 7f583a200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5834074ca0 0x7f58341023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.084+0000 7f583a200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f583410e780 0x7f5834102920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.084+0000 7f583a200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f58341143f0 con 0x7f5834074380 2024-04-26T02:00:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.084+0000 7f583a200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5834114270 con 0x7f5834074ca0 2024-04-26T02:00:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.084+0000 7f583a200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5834114570 con 0x7f583410e780 2024-04-26T02:00:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.085+0000 7f5833e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f583410e780 0x7f5834102920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.085+0000 7f5833400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5834074380 0x7f5834101ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.085+0000 7f582aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5834074ca0 0x7f58341023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.085+0000 7f5833e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f583410e780 0x7f5834102920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:49198/0 (socket says 172.21.15.1:49198) 2024-04-26T02:00:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.085+0000 7f5833e00640 1 -- 172.21.15.1:0/2866797758 learned_addr learned my addr 172.21.15.1:0/2866797758 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.085+0000 7f582aa00640 1 -- 172.21.15.1:0/2866797758 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f583410e780 msgr2=0x7f5834102920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.085+0000 7f582aa00640 1 --2- 172.21.15.1:0/2866797758 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f583410e780 0x7f5834102920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.085+0000 7f582aa00640 1 -- 172.21.15.1:0/2866797758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5834074380 msgr2=0x7f5834101ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.085+0000 7f582aa00640 1 --2- 172.21.15.1:0/2866797758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5834074380 0x7f5834101ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.085+0000 7f582aa00640 1 -- 172.21.15.1:0/2866797758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58341aafe0 con 0x7f5834074ca0 2024-04-26T02:00:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.085+0000 7f5833e00640 1 --2- 172.21.15.1:0/2866797758 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f583410e780 0x7f5834102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.085+0000 7f5833400640 1 --2- 172.21.15.1:0/2866797758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5834074380 0x7f5834101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.086+0000 7f582aa00640 1 --2- 172.21.15.1:0/2866797758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5834074ca0 0x7f58341023e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f581c02aa70 tx=0x7f581c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.086+0000 7f5830c00640 1 -- 172.21.15.1:0/2866797758 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c03f070 con 0x7f5834074ca0 2024-04-26T02:00:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.086+0000 7f5830c00640 1 -- 172.21.15.1:0/2866797758 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f581c031b50 con 0x7f5834074ca0 2024-04-26T02:00:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.086+0000 7f5830c00640 1 -- 172.21.15.1:0/2866797758 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c035030 con 0x7f5834074ca0 2024-04-26T02:00:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.086+0000 7f583a200640 1 -- 172.21.15.1:0/2866797758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58341ab270 con 0x7f5834074ca0 2024-04-26T02:00:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.086+0000 7f583a200640 1 -- 172.21.15.1:0/2866797758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58341ab7b0 con 0x7f5834074ca0 2024-04-26T02:00:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.090+0000 7f5830c00640 1 -- 172.21.15.1:0/2866797758 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f581c042070 con 0x7f5834074ca0 2024-04-26T02:00:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.090+0000 7f583d026740 1 Processor -- start 2024-04-26T02:00:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.090+0000 7f583d026740 1 -- start start 2024-04-26T02:00:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.091+0000 7f583d026740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a789820 0x55dd0a789c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.091+0000 7f583d026740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a78a140 0x55dd0a7944c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.091+0000 7f583d026740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a794a00 0x55dd0a796df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.091+0000 7f583d026740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55dd0a6ef900 con 0x55dd0a794a00 2024-04-26T02:00:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.091+0000 7f583d026740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dd0a695ef0 con 0x55dd0a78a140 2024-04-26T02:00:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.091+0000 7f583d026740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55dd0a6eb8c0 con 0x55dd0a789820 2024-04-26T02:00:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.091+0000 7f5833e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a789820 0x55dd0a789c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.091+0000 7f5833e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a789820 0x55dd0a789c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:49200/0 (socket says 172.21.15.1:49200) 2024-04-26T02:00:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.091+0000 7f5833400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a78a140 0x55dd0a7944c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.091+0000 7f5833e00640 1 -- 172.21.15.1:0/130989044 learned_addr learned my addr 172.21.15.1:0/130989044 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.092+0000 7f5833400640 1 -- 172.21.15.1:0/130989044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a789820 msgr2=0x55dd0a789c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.092+0000 7f5833400640 1 --2- 172.21.15.1:0/130989044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a789820 0x55dd0a789c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.092+0000 7f5833400640 1 -- 172.21.15.1:0/130989044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a794a00 msgr2=0x55dd0a796df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.092+0000 7f5833400640 1 --2- 172.21.15.1:0/130989044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a794a00 0x55dd0a796df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.092+0000 7f5833400640 1 -- 172.21.15.1:0/130989044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd0a402210 con 0x55dd0a78a140 2024-04-26T02:00:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.092+0000 7f5833e00640 1 --2- 172.21.15.1:0/130989044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a789820 0x55dd0a789c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.092+0000 7f5833400640 1 --2- 172.21.15.1:0/130989044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a78a140 0x55dd0a7944c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5824005fd0 tx=0x7f5824011c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6290c194911a5fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.092+0000 7f5830c00640 1 --2- 172.21.15.1:0/2866797758 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f58180a3d90 0x7f58180a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.092+0000 7f5823e00640 1 -- 172.21.15.1:0/130989044 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824003c30 con 0x55dd0a78a140 2024-04-26T02:00:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.092+0000 7f5823e00640 1 -- 172.21.15.1:0/130989044 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f582400fe50 con 0x55dd0a78a140 2024-04-26T02:00:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.092+0000 7f5823e00640 1 -- 172.21.15.1:0/130989044 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58240193e0 con 0x55dd0a78a140 2024-04-26T02:00:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.092+0000 7f5830c00640 1 -- 172.21.15.1:0/2866797758 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8615+0+0 (secure 0 0 0) 0x7f581c0f0a00 con 0x7f5834074ca0 2024-04-26T02:00:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.093+0000 7f5833e00640 1 --2- 172.21.15.1:0/2866797758 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f58180a3d90 0x7f58180a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.094+0000 7f583d026740 1 -- 172.21.15.1:0/130989044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a78a140 msgr2=0x55dd0a7944c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.094+0000 7f5833e00640 1 --2- 172.21.15.1:0/2866797758 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f58180a3d90 0x7f58180a6250 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f582c011890 tx=0x7f582c00b420 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.094+0000 7f583d026740 1 --2- 172.21.15.1:0/130989044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a78a140 0x55dd0a7944c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5824005fd0 tx=0x7f5824011c70 comp rx=0 tx=0).stop 2024-04-26T02:00:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.094+0000 7f583d026740 1 -- 172.21.15.1:0/130989044 shutdown_connections 2024-04-26T02:00:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.095+0000 7f583d026740 1 --2- 172.21.15.1:0/130989044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a794a00 0x55dd0a796df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.095+0000 7f583d026740 1 --2- 172.21.15.1:0/130989044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a78a140 0x55dd0a7944c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.095+0000 7f583d026740 1 --2- 172.21.15.1:0/130989044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a789820 0x55dd0a789c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.095+0000 7f583d026740 1 -- 172.21.15.1:0/130989044 >> 172.21.15.1:0/130989044 conn(0x55dd0a785d30 msgr2=0x55dd0a786110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.095+0000 7f583d026740 1 -- 172.21.15.1:0/130989044 shutdown_connections 2024-04-26T02:00:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.096+0000 7f583d026740 1 -- 172.21.15.1:0/130989044 wait complete. 2024-04-26T02:00:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.096+0000 7f583d026740 1 -- start start 2024-04-26T02:00:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.097+0000 7f583d026740 1 Processor -- start 2024-04-26T02:00:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.097+0000 7f583d026740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a789820 0x55dd0a791980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.097+0000 7f583d026740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a78a140 0x55dd0a791ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.097+0000 7f583d026740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a794a00 0x55dd0a792400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.097+0000 7f583d026740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55dd0a797630 con 0x55dd0a789820 2024-04-26T02:00:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.097+0000 7f583d026740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dd0a6a9310 con 0x55dd0a794a00 2024-04-26T02:00:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.097+0000 7f583d026740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55dd0a7977b0 con 0x55dd0a78a140 2024-04-26T02:00:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.097+0000 7f583d026740 10 client.0 _ll_register_callbacks cb 0x55dd0a677cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.098+0000 7f5833e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a794a00 0x55dd0a792400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.098+0000 7f582aa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a78a140 0x55dd0a791ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.098+0000 7f5833e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a794a00 0x55dd0a792400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:57244/0 (socket says 172.21.15.1:57244) 2024-04-26T02:00:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.098+0000 7f5833e00640 1 -- 172.21.15.1:0/2758289725 learned_addr learned my addr 172.21.15.1:0/2758289725 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.098+0000 7f5833400640 1 --2- 172.21.15.1:0/2758289725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a789820 0x55dd0a791980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.098+0000 7f5833e00640 1 -- 172.21.15.1:0/2758289725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a78a140 msgr2=0x55dd0a791ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.098+0000 7f5833e00640 1 --2- 172.21.15.1:0/2758289725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a78a140 0x55dd0a791ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.098+0000 7f5833e00640 1 -- 172.21.15.1:0/2758289725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a789820 msgr2=0x55dd0a791980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.098+0000 7f5833e00640 1 --2- 172.21.15.1:0/2758289725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a789820 0x55dd0a791980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.098+0000 7f5833e00640 1 -- 172.21.15.1:0/2758289725 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd0a4ce620 con 0x55dd0a794a00 2024-04-26T02:00:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.098+0000 7f582aa00640 1 --2- 172.21.15.1:0/2758289725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a78a140 0x55dd0a791ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.098+0000 7f5833400640 1 --2- 172.21.15.1:0/2758289725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a789820 0x55dd0a791980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:00:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.099+0000 7f5833e00640 1 --2- 172.21.15.1:0/2758289725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a794a00 0x55dd0a792400 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f582c005fd0 tx=0x7f582c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa65ab8dc82fbb60 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.099+0000 7f5820c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:00:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.099+0000 7f5820c00640 1 -- 172.21.15.1:0/2758289725 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c02c060 con 0x55dd0a794a00 2024-04-26T02:00:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.099+0000 7f5820c00640 1 -- 172.21.15.1:0/2758289725 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8615+0+0 (secure 0 0 0) 0x7f582c017030 con 0x55dd0a794a00 2024-04-26T02:00:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.099+0000 7f583d026740 1 -- 172.21.15.1:0/2758289725 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f582400fe50 con 0x55dd0a794a00 2024-04-26T02:00:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.099+0000 7f5820c00640 1 -- 172.21.15.1:0/2758289725 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f582c01fbc0 con 0x55dd0a794a00 2024-04-26T02:00:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.099+0000 7f583d026740 10 client.26201 fetch_fsmap learned FSMap version 32 2024-04-26T02:00:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.099+0000 7f583d026740 1 -- 172.21.15.1:0/2758289725 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dd0a7a38d0 con 0x55dd0a794a00 2024-04-26T02:00:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.100+0000 7f5820c00640 1 -- 172.21.15.1:0/2758289725 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f582400fe50 con 0x55dd0a794a00 2024-04-26T02:00:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.100+0000 7f5820c00640 1 -- 172.21.15.1:0/2758289725 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f582c027410 con 0x55dd0a794a00 2024-04-26T02:00:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.100+0000 7f583d026740 10 client.26201 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:00:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.100+0000 7f583d026740 10 client.26201 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:00:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.100+0000 7f583d026740 20 client.26201 populate_metadata read hostname 'smithi001' 2024-04-26T02:00:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.100+0000 7f583d026740 1 --2- 172.21.15.1:0/2758289725 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55dd0a7a4d90 0x55dd0a7a7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.100+0000 7f583d026740 4 client.26201 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:00:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.100+0000 7f583d026740 1 -- 172.21.15.1:0/2758289725 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd0a7038b0 con 0x55dd0a7a4d90 2024-04-26T02:00:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.101+0000 7f5833400640 1 --2- 172.21.15.1:0/2758289725 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55dd0a7a4d90 0x55dd0a7a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.101+0000 7f5833400640 1 --2- 172.21.15.1:0/2758289725 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55dd0a7a4d90 0x55dd0a7a7180 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebd25f73d1abb790 server_cookie=be9c7f35ae583852 in_seq=0 out_seq=0 2024-04-26T02:00:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.101+0000 7f5820c00640 10 client.26201 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.102+0000 7f5820c00640 1 -- 172.21.15.1:0/2758289725 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd0a7038b0 con 0x55dd0a7a4d90 2024-04-26T02:00:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.102+0000 7f5820c00640 10 client.26201 handle_command_reply: tid=0 2024-04-26T02:00:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.102+0000 7f583d026740 1 client.26201 shutdown 2024-04-26T02:00:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.102+0000 7f583d026740 1 -- 172.21.15.1:0/2758289725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a794a00 msgr2=0x55dd0a792400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.102+0000 7f583d026740 1 --2- 172.21.15.1:0/2758289725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a794a00 0x55dd0a792400 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f582c005fd0 tx=0x7f582c004730 comp rx=0 tx=0).stop 2024-04-26T02:00:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.103+0000 7f583d026740 1 -- 172.21.15.1:0/2758289725 shutdown_connections 2024-04-26T02:00:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.103+0000 7f583d026740 1 --2- 172.21.15.1:0/2758289725 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55dd0a7a4d90 0x55dd0a7a7180 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.103+0000 7f583d026740 1 --2- 172.21.15.1:0/2758289725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a794a00 0x55dd0a792400 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.103+0000 7f583d026740 1 --2- 172.21.15.1:0/2758289725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a78a140 0x55dd0a791ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.103+0000 7f583d026740 1 --2- 172.21.15.1:0/2758289725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a789820 0x55dd0a791980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.103+0000 7f583d026740 1 -- 172.21.15.1:0/2758289725 >> 172.21.15.1:0/2758289725 conn(0x55dd0a785d30 msgr2=0x55dd0a78ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.103+0000 7f5820c00640 0 client.26201 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.107+0000 7f583d026740 1 -- 172.21.15.1:0/2758289725 shutdown_connections 2024-04-26T02:00:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.107+0000 7f583d026740 1 -- 172.21.15.1:0/2758289725 wait complete. 2024-04-26T02:00:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.107+0000 7f583d026740 20 client.26201 trim_cache size 0 max 16384 2024-04-26T02:00:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.125+0000 7f583d026740 1 Processor -- start 2024-04-26T02:00:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.125+0000 7f583d026740 1 -- start start 2024-04-26T02:00:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.126+0000 7f583d026740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a789820 0x55dd0a7a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.126+0000 7f583d026740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a78a140 0x55dd0a792220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.126+0000 7f583d026740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a794a00 0x55dd0a7aca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.126+0000 7f583d026740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f58341b6bc0 con 0x55dd0a78a140 2024-04-26T02:00:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.126+0000 7f583d026740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f58341b4490 con 0x55dd0a789820 2024-04-26T02:00:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.126+0000 7f583d026740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f58341b2300 con 0x55dd0a794a00 2024-04-26T02:00:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.126+0000 7f582aa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a78a140 0x55dd0a792220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.126+0000 7f5833400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a789820 0x55dd0a7a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.126+0000 7f582aa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a78a140 0x55dd0a792220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57942/0 (socket says 172.21.15.1:57942) 2024-04-26T02:00:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.126+0000 7f582aa00640 1 -- 172.21.15.1:0/116428092 learned_addr learned my addr 172.21.15.1:0/116428092 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.126+0000 7f5833e00640 1 --2- 172.21.15.1:0/116428092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a794a00 0x55dd0a7aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.127+0000 7f582aa00640 1 -- 172.21.15.1:0/116428092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a794a00 msgr2=0x55dd0a7aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.127+0000 7f582aa00640 1 --2- 172.21.15.1:0/116428092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a794a00 0x55dd0a7aca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.127+0000 7f582aa00640 1 -- 172.21.15.1:0/116428092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a789820 msgr2=0x55dd0a7a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.127+0000 7f582aa00640 1 --2- 172.21.15.1:0/116428092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a789820 0x55dd0a7a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.127+0000 7f582aa00640 1 -- 172.21.15.1:0/116428092 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd0a78fed0 con 0x55dd0a78a140 2024-04-26T02:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.127+0000 7f5833400640 1 --2- 172.21.15.1:0/116428092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a789820 0x55dd0a7a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.127+0000 7f582aa00640 1 --2- 172.21.15.1:0/116428092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a78a140 0x55dd0a792220 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f581c0ee6f0 tx=0x7f581c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891624a3f528db91 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.127+0000 7f5823400640 1 -- 172.21.15.1:0/116428092 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c049070 con 0x55dd0a78a140 2024-04-26T02:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.127+0000 7f5823400640 1 -- 172.21.15.1:0/116428092 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f581c0c0cf0 con 0x55dd0a78a140 2024-04-26T02:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.127+0000 7f5823400640 1 -- 172.21.15.1:0/116428092 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c0ee500 con 0x55dd0a78a140 2024-04-26T02:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.128+0000 7f583d026740 1 -- 172.21.15.1:0/116428092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a78a140 msgr2=0x55dd0a792220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.128+0000 7f583d026740 1 --2- 172.21.15.1:0/116428092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a78a140 0x55dd0a792220 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f581c0ee6f0 tx=0x7f581c031400 comp rx=0 tx=0).stop 2024-04-26T02:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.128+0000 7f583d026740 1 -- 172.21.15.1:0/116428092 shutdown_connections 2024-04-26T02:00:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.128+0000 7f583d026740 1 --2- 172.21.15.1:0/116428092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a794a00 0x55dd0a7aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.128+0000 7f583d026740 1 --2- 172.21.15.1:0/116428092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a78a140 0x55dd0a792220 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.128+0000 7f583d026740 1 --2- 172.21.15.1:0/116428092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a789820 0x55dd0a7a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.128+0000 7f583d026740 1 -- 172.21.15.1:0/116428092 >> 172.21.15.1:0/116428092 conn(0x55dd0a785d30 msgr2=0x55dd0a786110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.128+0000 7f583d026740 1 -- 172.21.15.1:0/116428092 shutdown_connections 2024-04-26T02:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.128+0000 7f583d026740 1 -- 172.21.15.1:0/116428092 wait complete. 2024-04-26T02:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.129+0000 7f583d026740 1 -- start start 2024-04-26T02:00:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.129+0000 7f583d026740 1 Processor -- start 2024-04-26T02:00:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.129+0000 7f583d026740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a789820 0x55dd0a803660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.129+0000 7f583d026740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a78a140 0x55dd0a803ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.129+0000 7f583d026740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a794a00 0x55dd0a799da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.129+0000 7f583d026740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f58341b2950 con 0x55dd0a789820 2024-04-26T02:00:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.129+0000 7f583d026740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f58341b3dc0 con 0x55dd0a78a140 2024-04-26T02:00:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.129+0000 7f583d026740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f58341b3020 con 0x55dd0a794a00 2024-04-26T02:00:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.129+0000 7f583d026740 10 client.0 _ll_register_callbacks cb 0x7f582c00be00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.130+0000 7f5833400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a789820 0x55dd0a803660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.130+0000 7f5833400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a789820 0x55dd0a803660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57946/0 (socket says 172.21.15.1:57946) 2024-04-26T02:00:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.130+0000 7f5833400640 1 -- 172.21.15.1:0/933255672 learned_addr learned my addr 172.21.15.1:0/933255672 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.130+0000 7f582aa00640 1 --2- 172.21.15.1:0/933255672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a78a140 0x55dd0a803ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.130+0000 7f5833e00640 1 --2- 172.21.15.1:0/933255672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a794a00 0x55dd0a799da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.130+0000 7f5833400640 1 -- 172.21.15.1:0/933255672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a794a00 msgr2=0x55dd0a799da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.130+0000 7f5833400640 1 --2- 172.21.15.1:0/933255672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a794a00 0x55dd0a799da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.130+0000 7f5833400640 1 -- 172.21.15.1:0/933255672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a78a140 msgr2=0x55dd0a803ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.130+0000 7f5833400640 1 --2- 172.21.15.1:0/933255672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a78a140 0x55dd0a803ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.130+0000 7f5833400640 1 -- 172.21.15.1:0/933255672 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd0a79a490 con 0x55dd0a789820 2024-04-26T02:00:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.130+0000 7f582aa00640 1 --2- 172.21.15.1:0/933255672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a78a140 0x55dd0a803ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.130+0000 7f5833400640 1 --2- 172.21.15.1:0/933255672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a789820 0x55dd0a803660 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f582400f790 tx=0x7f5824009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3dbf182aa2cf09d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.130+0000 7f5820c00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T02:00:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.131+0000 7f5820c00640 1 -- 172.21.15.1:0/933255672 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582402f060 con 0x55dd0a789820 2024-04-26T02:00:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.131+0000 7f5820c00640 1 -- 172.21.15.1:0/933255672 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8615+0+0 (secure 0 0 0) 0x7f5824011250 con 0x55dd0a789820 2024-04-26T02:00:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.131+0000 7f583d026740 1 -- 172.21.15.1:0/933255672 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f581c0c0cf0 con 0x55dd0a789820 2024-04-26T02:00:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.131+0000 7f5820c00640 1 -- 172.21.15.1:0/933255672 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f582402ae60 con 0x55dd0a789820 2024-04-26T02:00:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.131+0000 7f583d026740 10 client.16050 fetch_fsmap learned FSMap version 32 2024-04-26T02:00:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.131+0000 7f583d026740 1 -- 172.21.15.1:0/933255672 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dd0a80adf0 con 0x55dd0a789820 2024-04-26T02:00:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.131+0000 7f5820c00640 1 -- 172.21.15.1:0/933255672 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f581c0c0cf0 con 0x55dd0a789820 2024-04-26T02:00:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.132+0000 7f5820c00640 1 -- 172.21.15.1:0/933255672 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f5824018070 con 0x55dd0a789820 2024-04-26T02:00:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.132+0000 7f583d026740 10 client.16050 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:00:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.132+0000 7f583d026740 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:00:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.132+0000 7f583d026740 20 client.16050 populate_metadata read hostname 'smithi001' 2024-04-26T02:00:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.132+0000 7f583d026740 1 --2- 172.21.15.1:0/933255672 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55dd0a80b330 0x55dd0a80d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.132+0000 7f583d026740 4 client.16050 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:00:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.132+0000 7f583d026740 1 -- 172.21.15.1:0/933255672 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dd0a6fd320 con 0x55dd0a80b330 2024-04-26T02:00:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.133+0000 7f582aa00640 1 --2- 172.21.15.1:0/933255672 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55dd0a80b330 0x55dd0a80d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.133+0000 7f582aa00640 1 --2- 172.21.15.1:0/933255672 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55dd0a80b330 0x55dd0a80d720 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b727ee9de0a89cbc server_cookie=12123e390f1d7baa in_seq=0 out_seq=0 2024-04-26T02:00:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.133+0000 7f5820c00640 10 client.16050 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.145+0000 7f5820c00640 1 -- 172.21.15.1:0/933255672 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dd0a6fd320 con 0x55dd0a80b330 2024-04-26T02:00:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.145+0000 7f5820c00640 10 client.16050 handle_command_reply: tid=0 2024-04-26T02:00:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.145+0000 7f583d026740 1 client.16050 shutdown 2024-04-26T02:00:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.145+0000 7f583d026740 1 -- 172.21.15.1:0/933255672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a789820 msgr2=0x55dd0a803660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.145+0000 7f583d026740 1 --2- 172.21.15.1:0/933255672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a789820 0x55dd0a803660 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f582400f790 tx=0x7f5824009e20 comp rx=0 tx=0).stop 2024-04-26T02:00:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.146+0000 7f583d026740 1 -- 172.21.15.1:0/933255672 shutdown_connections 2024-04-26T02:00:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.146+0000 7f583d026740 1 --2- 172.21.15.1:0/933255672 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55dd0a80b330 0x55dd0a80d720 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.146+0000 7f583d026740 1 --2- 172.21.15.1:0/933255672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd0a794a00 0x55dd0a799da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.146+0000 7f583d026740 1 --2- 172.21.15.1:0/933255672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dd0a78a140 0x55dd0a803ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.146+0000 7f583d026740 1 --2- 172.21.15.1:0/933255672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dd0a789820 0x55dd0a803660 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.146+0000 7f583d026740 1 -- 172.21.15.1:0/933255672 >> 172.21.15.1:0/933255672 conn(0x55dd0a785d30 msgr2=0x55dd0a78b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.146+0000 7f5820c00640 0 client.16050 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.146+0000 7f583d026740 1 -- 172.21.15.1:0/933255672 shutdown_connections 2024-04-26T02:00:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.146+0000 7f583d026740 1 -- 172.21.15.1:0/933255672 wait complete. 2024-04-26T02:00:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.146+0000 7f583d026740 20 client.16050 trim_cache size 0 max 16384 2024-04-26T02:00:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T02:00:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T02:00:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T02:00:10.146 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T02:00:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.147+0000 7f583a200640 1 -- 172.21.15.1:0/2866797758 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f58180a3d90 msgr2=0x7f58180a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.147+0000 7f583a200640 1 --2- 172.21.15.1:0/2866797758 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f58180a3d90 0x7f58180a6250 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f582c011890 tx=0x7f582c00b420 comp rx=0 tx=0).stop 2024-04-26T02:00:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.147+0000 7f583a200640 1 -- 172.21.15.1:0/2866797758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5834074ca0 msgr2=0x7f58341023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.147+0000 7f583a200640 1 --2- 172.21.15.1:0/2866797758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5834074ca0 0x7f58341023e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f581c02aa70 tx=0x7f581c003b90 comp rx=0 tx=0).stop 2024-04-26T02:00:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.147+0000 7f583a200640 1 -- 172.21.15.1:0/2866797758 shutdown_connections 2024-04-26T02:00:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.147+0000 7f583a200640 1 --2- 172.21.15.1:0/2866797758 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f58180a3d90 0x7f58180a6250 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.147+0000 7f583a200640 1 --2- 172.21.15.1:0/2866797758 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f583410e780 0x7f5834102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.147+0000 7f583a200640 1 --2- 172.21.15.1:0/2866797758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5834074ca0 0x7f58341023e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.147+0000 7f583a200640 1 --2- 172.21.15.1:0/2866797758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5834074380 0x7f5834101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.147+0000 7f583a200640 1 -- 172.21.15.1:0/2866797758 >> 172.21.15.1:0/2866797758 conn(0x7f58340fda10 msgr2=0x7f58340ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.147+0000 7f583a200640 1 -- 172.21.15.1:0/2866797758 shutdown_connections 2024-04-26T02:00:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:10.147+0000 7f583a200640 1 -- 172.21.15.1:0/2866797758 wait complete. 2024-04-26T02:00:10.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:10 smithi097 ceph-mon[32853]: pgmap v264: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 273 KiB/s wr, 91 op/s 2024-04-26T02:00:10.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:10 smithi173 ceph-mon[32553]: pgmap v264: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 273 KiB/s wr, 91 op/s 2024-04-26T02:00:10.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:10 smithi001 ceph-mon[27822]: pgmap v264: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 273 KiB/s wr, 91 op/s 2024-04-26T02:00:12.367 INFO:tasks.workunit.client.0.smithi001.stdout:10000 mapread 0x33cc4e3 thru 0x33cec16 (0x2734 bytes) 2024-04-26T02:00:12.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:12 smithi097 ceph-mon[32853]: pgmap v265: 97 pgs: 97 active+clean; 106 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 890 KiB/s wr, 256 op/s 2024-04-26T02:00:12.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:12 smithi173 ceph-mon[32553]: pgmap v265: 97 pgs: 97 active+clean; 106 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 890 KiB/s wr, 256 op/s 2024-04-26T02:00:12.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:12 smithi001 ceph-mon[27822]: pgmap v265: 97 pgs: 97 active+clean; 106 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 890 KiB/s wr, 256 op/s 2024-04-26T02:00:14.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:14 smithi097 ceph-mon[32853]: pgmap v266: 97 pgs: 97 active+clean; 106 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 890 KiB/s wr, 256 op/s 2024-04-26T02:00:14.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:14 smithi097 ceph-mon[32853]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T02:00:14.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:14 smithi173 ceph-mon[32553]: pgmap v266: 97 pgs: 97 active+clean; 106 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 890 KiB/s wr, 256 op/s 2024-04-26T02:00:14.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:14 smithi173 ceph-mon[32553]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T02:00:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:14 smithi001 ceph-mon[27822]: pgmap v266: 97 pgs: 97 active+clean; 106 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 890 KiB/s wr, 256 op/s 2024-04-26T02:00:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:14 smithi001 ceph-mon[27822]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T02:00:16.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:16 smithi173 ceph-mon[32553]: pgmap v268: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.3 MiB/s wr, 384 op/s 2024-04-26T02:00:16.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:16 smithi001 ceph-mon[27822]: pgmap v268: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.3 MiB/s wr, 384 op/s 2024-04-26T02:00:16.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:16 smithi097 ceph-mon[32853]: pgmap v268: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.3 MiB/s wr, 384 op/s 2024-04-26T02:00:18.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:18 smithi173 ceph-mon[32553]: pgmap v269: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.2 MiB/s wr, 373 op/s 2024-04-26T02:00:18.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:18 smithi001 ceph-mon[27822]: pgmap v269: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.2 MiB/s wr, 373 op/s 2024-04-26T02:00:18.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:18 smithi097 ceph-mon[32853]: pgmap v269: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.2 MiB/s wr, 373 op/s 2024-04-26T02:00:20.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:20 smithi173 ceph-mon[32553]: pgmap v270: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.2 MiB/s wr, 373 op/s 2024-04-26T02:00:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:20 smithi001 ceph-mon[27822]: pgmap v270: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.2 MiB/s wr, 373 op/s 2024-04-26T02:00:20.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:20 smithi097 ceph-mon[32853]: pgmap v270: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.2 MiB/s wr, 373 op/s 2024-04-26T02:00:22.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:22 smithi173 ceph-mon[32553]: pgmap v271: 97 pgs: 97 active+clean; 104 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.2 MiB/s wr, 407 op/s 2024-04-26T02:00:22.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:22 smithi001 ceph-mon[27822]: pgmap v271: 97 pgs: 97 active+clean; 104 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.2 MiB/s wr, 407 op/s 2024-04-26T02:00:22.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:22 smithi097 ceph-mon[32853]: pgmap v271: 97 pgs: 97 active+clean; 104 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.2 MiB/s wr, 407 op/s 2024-04-26T02:00:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.309+0000 7fdfab200640 1 -- 172.21.15.1:0/625855197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfac109000 msgr2=0x7fdfac1093e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.309+0000 7fdfab200640 1 --2- 172.21.15.1:0/625855197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfac109000 0x7fdfac1093e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009900 tx=0x7fdf9c02a410 comp rx=0 tx=0).stop 2024-04-26T02:00:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.309+0000 7fdfab200640 1 -- 172.21.15.1:0/625855197 shutdown_connections 2024-04-26T02:00:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.309+0000 7fdfab200640 1 --2- 172.21.15.1:0/625855197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfac10d700 0x7fdfac1152d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.309+0000 7fdfab200640 1 --2- 172.21.15.1:0/625855197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfac1099b0 0x7fdfac10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.309+0000 7fdfab200640 1 --2- 172.21.15.1:0/625855197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfac109000 0x7fdfac1093e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.309+0000 7fdfab200640 1 -- 172.21.15.1:0/625855197 >> 172.21.15.1:0/625855197 conn(0x7fdfac077940 msgr2=0x7fdfac079d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.310+0000 7fdfab200640 1 -- 172.21.15.1:0/625855197 shutdown_connections 2024-04-26T02:00:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.310+0000 7fdfab200640 1 -- 172.21.15.1:0/625855197 wait complete. 2024-04-26T02:00:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.311+0000 7fdfab200640 1 Processor -- start 2024-04-26T02:00:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.311+0000 7fdfab200640 1 -- start start 2024-04-26T02:00:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.311+0000 7fdfab200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfac109000 0x7fdfac1a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.311+0000 7fdfab200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfac10d700 0x7fdfac1a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.311+0000 7fdfab200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfac19ea80 0x7fdfac19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.311+0000 7fdfab200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac1186e0 con 0x7fdfac10d700 2024-04-26T02:00:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.311+0000 7fdfab200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac118560 con 0x7fdfac19ea80 2024-04-26T02:00:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.311+0000 7fdfab200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac118860 con 0x7fdfac109000 2024-04-26T02:00:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.311+0000 7fdfaa800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfac19ea80 0x7fdfac19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.311+0000 7fdfaa800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfac19ea80 0x7fdfac19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:53536/0 (socket says 172.21.15.1:53536) 2024-04-26T02:00:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.311+0000 7fdfaa800640 1 -- 172.21.15.1:0/2669298687 learned_addr learned my addr 172.21.15.1:0/2669298687 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.311+0000 7fdfa9e00640 1 --2- 172.21.15.1:0/2669298687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfac109000 0x7fdfac1a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.312+0000 7fdfa9400640 1 --2- 172.21.15.1:0/2669298687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfac10d700 0x7fdfac1a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.312+0000 7fdfaa800640 1 -- 172.21.15.1:0/2669298687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfac109000 msgr2=0x7fdfac1a48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.312+0000 7fdfaa800640 1 --2- 172.21.15.1:0/2669298687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfac109000 0x7fdfac1a48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.312+0000 7fdfaa800640 1 -- 172.21.15.1:0/2669298687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfac10d700 msgr2=0x7fdfac1a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.312+0000 7fdfaa800640 1 --2- 172.21.15.1:0/2669298687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfac10d700 0x7fdfac1a4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.312+0000 7fdfaa800640 1 -- 172.21.15.1:0/2669298687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfac19f5a0 con 0x7fdfac19ea80 2024-04-26T02:00:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.312+0000 7fdfaa800640 1 --2- 172.21.15.1:0/2669298687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfac19ea80 0x7fdfac19ef30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400ea60 tx=0x7fdfa400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.312+0000 7fdfa2a00640 1 -- 172.21.15.1:0/2669298687 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa400ce40 con 0x7fdfac19ea80 2024-04-26T02:00:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.313+0000 7fdfab200640 1 -- 172.21.15.1:0/2669298687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfac19f890 con 0x7fdfac19ea80 2024-04-26T02:00:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.313+0000 7fdfa2a00640 1 -- 172.21.15.1:0/2669298687 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfa4005020 con 0x7fdfac19ea80 2024-04-26T02:00:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.313+0000 7fdfa2a00640 1 -- 172.21.15.1:0/2669298687 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4010670 con 0x7fdfac19ea80 2024-04-26T02:00:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.313+0000 7fdfab200640 1 -- 172.21.15.1:0/2669298687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfac1abae0 con 0x7fdfac19ea80 2024-04-26T02:00:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.316+0000 7fdfa2a00640 1 -- 172.21.15.1:0/2669298687 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fdfa4010810 con 0x7fdfac19ea80 2024-04-26T02:00:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.316+0000 7fdfb3dff740 1 Processor -- start 2024-04-26T02:00:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.317+0000 7fdfb3dff740 1 -- start start 2024-04-26T02:00:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.317+0000 7fdfb3dff740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264d9b0 0x55c4b264dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.317+0000 7fdfb3dff740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264e2d0 0x55c4b2658650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.317+0000 7fdfb3dff740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 0x55c4b265af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.317+0000 7fdfb3dff740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c4b25b3900 con 0x55c4b2658b90 2024-04-26T02:00:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.317+0000 7fdfb3dff740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c4b2559ef0 con 0x55c4b264d9b0 2024-04-26T02:00:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.317+0000 7fdfb3dff740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c4b25af8c0 con 0x55c4b264e2d0 2024-04-26T02:00:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.317+0000 7fdfa9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264e2d0 0x55c4b2658650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.317+0000 7fdfa9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264d9b0 0x55c4b264dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.317+0000 7fdfa9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264e2d0 0x55c4b2658650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:33580/0 (socket says 172.21.15.1:33580) 2024-04-26T02:00:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.317+0000 7fdfa9e00640 1 -- 172.21.15.1:0/812531195 learned_addr learned my addr 172.21.15.1:0/812531195 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.318+0000 7fdfa9400640 1 --2- 172.21.15.1:0/812531195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 0x55c4b265af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.318+0000 7fdfa9e00640 1 -- 172.21.15.1:0/812531195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264d9b0 msgr2=0x55c4b264dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.318+0000 7fdfa9e00640 1 --2- 172.21.15.1:0/812531195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264d9b0 0x55c4b264dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.318+0000 7fdfa9e00640 1 -- 172.21.15.1:0/812531195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 msgr2=0x55c4b265af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.318+0000 7fdfa9e00640 1 --2- 172.21.15.1:0/812531195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 0x55c4b265af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.318+0000 7fdfa9e00640 1 -- 172.21.15.1:0/812531195 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4b22c6210 con 0x55c4b264e2d0 2024-04-26T02:00:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.318+0000 7fdfa9400640 1 --2- 172.21.15.1:0/812531195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264d9b0 0x55c4b264dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.318+0000 7fdfa2a00640 1 --2- 172.21.15.1:0/2669298687 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fdf900a3d60 0x7fdf900a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.318+0000 7fdfa9e00640 1 --2- 172.21.15.1:0/812531195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264e2d0 0x55c4b2658650 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009230 tx=0x7fdf9c004180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=584a13b26d7f6a42 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.319+0000 7fdfa2a00640 1 -- 172.21.15.1:0/2669298687 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fdfa40d0d70 con 0x7fdfac19ea80 2024-04-26T02:00:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.319+0000 7fdfa9400640 1 --2- 172.21.15.1:0/2669298687 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fdf900a3d60 0x7fdf900a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.319+0000 7fdf97400640 1 -- 172.21.15.1:0/812531195 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c035040 con 0x55c4b264e2d0 2024-04-26T02:00:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.319+0000 7fdf97400640 1 -- 172.21.15.1:0/812531195 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf9c0054f0 con 0x55c4b264e2d0 2024-04-26T02:00:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.319+0000 7fdf97400640 1 -- 172.21.15.1:0/812531195 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c0035f0 con 0x55c4b264e2d0 2024-04-26T02:00:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.320+0000 7fdfb3dff740 1 -- 172.21.15.1:0/812531195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264e2d0 msgr2=0x55c4b2658650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.320+0000 7fdfa9400640 1 --2- 172.21.15.1:0/2669298687 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fdf900a3d60 0x7fdf900a6220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55c4b264b2e0 tx=0x7fdf98004730 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.320+0000 7fdfb3dff740 1 --2- 172.21.15.1:0/812531195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264e2d0 0x55c4b2658650 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009230 tx=0x7fdf9c004180 comp rx=0 tx=0).stop 2024-04-26T02:00:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.320+0000 7fdfb3dff740 1 -- 172.21.15.1:0/812531195 shutdown_connections 2024-04-26T02:00:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.320+0000 7fdfb3dff740 1 --2- 172.21.15.1:0/812531195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 0x55c4b265af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.320+0000 7fdfb3dff740 1 --2- 172.21.15.1:0/812531195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264e2d0 0x55c4b2658650 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.320+0000 7fdfb3dff740 1 --2- 172.21.15.1:0/812531195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264d9b0 0x55c4b264dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.320+0000 7fdfb3dff740 1 -- 172.21.15.1:0/812531195 >> 172.21.15.1:0/812531195 conn(0x55c4b2649ec0 msgr2=0x55c4b264a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.320+0000 7fdfb3dff740 1 -- 172.21.15.1:0/812531195 shutdown_connections 2024-04-26T02:00:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.320+0000 7fdfb3dff740 1 -- 172.21.15.1:0/812531195 wait complete. 2024-04-26T02:00:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.321+0000 7fdfb3dff740 1 -- start start 2024-04-26T02:00:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.321+0000 7fdfb3dff740 1 Processor -- start 2024-04-26T02:00:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.321+0000 7fdfb3dff740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264d9b0 0x55c4b2655ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.321+0000 7fdfb3dff740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264e2d0 0x55c4b2656020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.321+0000 7fdfb3dff740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 0x55c4b2656560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.321+0000 7fdfb3dff740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c4b265b760 con 0x55c4b2658b90 2024-04-26T02:00:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.321+0000 7fdfb3dff740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c4b256d310 con 0x55c4b264e2d0 2024-04-26T02:00:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.321+0000 7fdfb3dff740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c4b265b8e0 con 0x55c4b264d9b0 2024-04-26T02:00:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.321+0000 7fdfb3dff740 10 client.0 _ll_register_callbacks cb 0x55c4b253bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.322+0000 7fdfaa800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 0x55c4b2656560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.322+0000 7fdfa9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264d9b0 0x55c4b2655ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.322+0000 7fdfa9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264e2d0 0x55c4b2656020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.322+0000 7fdfa9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264e2d0 0x55c4b2656020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:53554/0 (socket says 172.21.15.1:53554) 2024-04-26T02:00:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.322+0000 7fdfa9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264d9b0 0x55c4b2655ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:33584/0 (socket says 172.21.15.1:33584) 2024-04-26T02:00:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.322+0000 7fdfa9400640 1 -- 172.21.15.1:0/3941571278 learned_addr learned my addr 172.21.15.1:0/3941571278 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.322+0000 7fdfa9400640 1 -- 172.21.15.1:0/3941571278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264d9b0 msgr2=0x55c4b2655ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.322+0000 7fdfa9400640 1 --2- 172.21.15.1:0/3941571278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264d9b0 0x55c4b2655ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.322+0000 7fdfa9400640 1 -- 172.21.15.1:0/3941571278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 msgr2=0x55c4b2656560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.322+0000 7fdfa9400640 1 --2- 172.21.15.1:0/3941571278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 0x55c4b2656560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.322+0000 7fdfa9400640 1 -- 172.21.15.1:0/3941571278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4b2392620 con 0x55c4b264e2d0 2024-04-26T02:00:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.322+0000 7fdfa9400640 1 --2- 172.21.15.1:0/3941571278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264e2d0 0x55c4b2656020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdfac07c020 tx=0x7fdf9800c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81d0d7f7e4e464be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.322+0000 7fdf8be00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:00:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.323+0000 7fdf8be00640 1 -- 172.21.15.1:0/3941571278 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98024070 con 0x55c4b264e2d0 2024-04-26T02:00:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.323+0000 7fdf8be00640 1 -- 172.21.15.1:0/3941571278 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fdf98029460 con 0x55c4b264e2d0 2024-04-26T02:00:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.323+0000 7fdfb3dff740 1 -- 172.21.15.1:0/3941571278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf9c0054f0 con 0x55c4b264e2d0 2024-04-26T02:00:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.323+0000 7fdf8be00640 1 -- 172.21.15.1:0/3941571278 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdf98029660 con 0x55c4b264e2d0 2024-04-26T02:00:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.323+0000 7fdfb3dff740 10 client.26219 fetch_fsmap learned FSMap version 32 2024-04-26T02:00:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.323+0000 7fdfb3dff740 1 -- 172.21.15.1:0/3941571278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c4b2667a10 con 0x55c4b264e2d0 2024-04-26T02:00:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.323+0000 7fdf8be00640 1 -- 172.21.15.1:0/3941571278 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf9c0054f0 con 0x55c4b264e2d0 2024-04-26T02:00:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.324+0000 7fdf8be00640 1 -- 172.21.15.1:0/3941571278 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fdf98030070 con 0x55c4b264e2d0 2024-04-26T02:00:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.324+0000 7fdfb3dff740 10 client.26219 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:00:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.324+0000 7fdfb3dff740 10 client.26219 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:00:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.324+0000 7fdfb3dff740 20 client.26219 populate_metadata read hostname 'smithi001' 2024-04-26T02:00:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.324+0000 7fdfb3dff740 1 --2- 172.21.15.1:0/3941571278 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c4b2668f20 0x55c4b266b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.324+0000 7fdfb3dff740 4 client.26219 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:00:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.324+0000 7fdfb3dff740 1 -- 172.21.15.1:0/3941571278 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4b25c78b0 con 0x55c4b2668f20 2024-04-26T02:00:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.324+0000 7fdfa9e00640 1 --2- 172.21.15.1:0/3941571278 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c4b2668f20 0x55c4b266b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.325+0000 7fdfa9e00640 1 --2- 172.21.15.1:0/3941571278 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c4b2668f20 0x55c4b266b310 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d63708ae155cf884 server_cookie=2c077de38f967eb9 in_seq=0 out_seq=0 2024-04-26T02:00:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.325+0000 7fdf8be00640 10 client.26219 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.325+0000 7fdf8be00640 1 -- 172.21.15.1:0/3941571278 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c4b25c78b0 con 0x55c4b2668f20 2024-04-26T02:00:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.325+0000 7fdf8be00640 10 client.26219 handle_command_reply: tid=0 2024-04-26T02:00:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.326+0000 7fdfb3dff740 1 client.26219 shutdown 2024-04-26T02:00:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.326+0000 7fdfb3dff740 1 -- 172.21.15.1:0/3941571278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264e2d0 msgr2=0x55c4b2656020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.326+0000 7fdfb3dff740 1 --2- 172.21.15.1:0/3941571278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264e2d0 0x55c4b2656020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdfac07c020 tx=0x7fdf9800c430 comp rx=0 tx=0).stop 2024-04-26T02:00:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.326+0000 7fdfb3dff740 1 -- 172.21.15.1:0/3941571278 shutdown_connections 2024-04-26T02:00:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.326+0000 7fdfb3dff740 1 --2- 172.21.15.1:0/3941571278 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c4b2668f20 0x55c4b266b310 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.326+0000 7fdfb3dff740 1 --2- 172.21.15.1:0/3941571278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 0x55c4b2656560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.326+0000 7fdfb3dff740 1 --2- 172.21.15.1:0/3941571278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264e2d0 0x55c4b2656020 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.326+0000 7fdfb3dff740 1 --2- 172.21.15.1:0/3941571278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264d9b0 0x55c4b2655ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.326+0000 7fdf8be00640 0 client.26219 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.326+0000 7fdfb3dff740 1 -- 172.21.15.1:0/3941571278 >> 172.21.15.1:0/3941571278 conn(0x55c4b2649ec0 msgr2=0x55c4b264fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.326+0000 7fdfb3dff740 1 -- 172.21.15.1:0/3941571278 shutdown_connections 2024-04-26T02:00:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.326+0000 7fdfb3dff740 1 -- 172.21.15.1:0/3941571278 wait complete. 2024-04-26T02:00:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.326+0000 7fdfb3dff740 20 client.26219 trim_cache size 0 max 16384 2024-04-26T02:00:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.345+0000 7fdfb3dff740 1 Processor -- start 2024-04-26T02:00:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.345+0000 7fdfb3dff740 1 -- start start 2024-04-26T02:00:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.345+0000 7fdfb3dff740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264d9b0 0x55c4b2664620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.345+0000 7fdfb3dff740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264e2d0 0x55c4b2670c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.345+0000 7fdfb3dff740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 0x55c4b2671160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.345+0000 7fdfb3dff740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac1b6ac0 con 0x55c4b2658b90 2024-04-26T02:00:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.345+0000 7fdfb3dff740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac1b4390 con 0x55c4b264e2d0 2024-04-26T02:00:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.345+0000 7fdfb3dff740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac1b2200 con 0x55c4b264d9b0 2024-04-26T02:00:23.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.346+0000 7fdfa9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264e2d0 0x55c4b2670c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.346+0000 7fdfa9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264d9b0 0x55c4b2664620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.346+0000 7fdfa9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264e2d0 0x55c4b2670c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:53570/0 (socket says 172.21.15.1:53570) 2024-04-26T02:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.346+0000 7fdfa9400640 1 -- 172.21.15.1:0/287331273 learned_addr learned my addr 172.21.15.1:0/287331273 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.346+0000 7fdfa9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264d9b0 0x55c4b2664620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:33598/0 (socket says 172.21.15.1:33598) 2024-04-26T02:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.346+0000 7fdfaa800640 1 --2- 172.21.15.1:0/287331273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 0x55c4b2671160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.346+0000 7fdfa9400640 1 -- 172.21.15.1:0/287331273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264d9b0 msgr2=0x55c4b2664620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.346+0000 7fdfa9400640 1 --2- 172.21.15.1:0/287331273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264d9b0 0x55c4b2664620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.346+0000 7fdfa9400640 1 -- 172.21.15.1:0/287331273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 msgr2=0x55c4b2671160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.346+0000 7fdfa9400640 1 --2- 172.21.15.1:0/287331273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 0x55c4b2671160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.346+0000 7fdfa9400640 1 -- 172.21.15.1:0/287331273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4b264b0f0 con 0x55c4b264e2d0 2024-04-26T02:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.346+0000 7fdfa9e00640 1 --2- 172.21.15.1:0/287331273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264d9b0 0x55c4b2664620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.346+0000 7fdfaa800640 1 --2- 172.21.15.1:0/287331273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 0x55c4b2671160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:00:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.346+0000 7fdfa9400640 1 --2- 172.21.15.1:0/287331273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264e2d0 0x55c4b2670c20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdf980058c0 tx=0x7fdf9800cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=275b8ebf3a83abf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.347+0000 7fdf96a00640 1 -- 172.21.15.1:0/287331273 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98036070 con 0x55c4b264e2d0 2024-04-26T02:00:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.347+0000 7fdf96a00640 1 -- 172.21.15.1:0/287331273 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf980203f0 con 0x55c4b264e2d0 2024-04-26T02:00:23.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.347+0000 7fdf96a00640 1 -- 172.21.15.1:0/287331273 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98024040 con 0x55c4b264e2d0 2024-04-26T02:00:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.348+0000 7fdfb3dff740 1 -- 172.21.15.1:0/287331273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264e2d0 msgr2=0x55c4b2670c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.348+0000 7fdfb3dff740 1 --2- 172.21.15.1:0/287331273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264e2d0 0x55c4b2670c20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdf980058c0 tx=0x7fdf9800cb30 comp rx=0 tx=0).stop 2024-04-26T02:00:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.348+0000 7fdfb3dff740 1 -- 172.21.15.1:0/287331273 shutdown_connections 2024-04-26T02:00:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.348+0000 7fdfb3dff740 1 --2- 172.21.15.1:0/287331273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 0x55c4b2671160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.348+0000 7fdfb3dff740 1 --2- 172.21.15.1:0/287331273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264e2d0 0x55c4b2670c20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.348+0000 7fdfb3dff740 1 --2- 172.21.15.1:0/287331273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264d9b0 0x55c4b2664620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.348+0000 7fdfb3dff740 1 -- 172.21.15.1:0/287331273 >> 172.21.15.1:0/287331273 conn(0x55c4b2649ec0 msgr2=0x55c4b264a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.348+0000 7fdfb3dff740 1 -- 172.21.15.1:0/287331273 shutdown_connections 2024-04-26T02:00:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.348+0000 7fdfb3dff740 1 -- 172.21.15.1:0/287331273 wait complete. 2024-04-26T02:00:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.348+0000 7fdfb3dff740 1 -- start start 2024-04-26T02:00:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.349+0000 7fdfb3dff740 1 Processor -- start 2024-04-26T02:00:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.349+0000 7fdfb3dff740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264d9b0 0x55c4b2661080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.349+0000 7fdfb3dff740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264e2d0 0x55c4b265e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.349+0000 7fdfb3dff740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 0x55c4b265eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.349+0000 7fdfb3dff740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac1b2850 con 0x55c4b2658b90 2024-04-26T02:00:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.349+0000 7fdfb3dff740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac1b3cc0 con 0x55c4b264e2d0 2024-04-26T02:00:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.349+0000 7fdfb3dff740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac1b2f20 con 0x55c4b264d9b0 2024-04-26T02:00:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.349+0000 7fdfb3dff740 10 client.0 _ll_register_callbacks cb 0x7fdf98011d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.349+0000 7fdfaa800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 0x55c4b265eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.349+0000 7fdfa9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264d9b0 0x55c4b2661080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.349+0000 7fdfa9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264e2d0 0x55c4b265e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.349+0000 7fdfa9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264d9b0 0x55c4b2661080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:33600/0 (socket says 172.21.15.1:33600) 2024-04-26T02:00:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.349+0000 7fdfa9e00640 1 -- 172.21.15.1:0/1289914845 learned_addr learned my addr 172.21.15.1:0/1289914845 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:23.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.349+0000 7fdfa9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264e2d0 0x55c4b265e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:53574/0 (socket says 172.21.15.1:53574) 2024-04-26T02:00:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.349+0000 7fdfaa800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 0x55c4b265eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54168/0 (socket says 172.21.15.1:54168) 2024-04-26T02:00:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.350+0000 7fdfa9e00640 1 -- 172.21.15.1:0/1289914845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264e2d0 msgr2=0x55c4b265e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.350+0000 7fdfa9e00640 1 --2- 172.21.15.1:0/1289914845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264e2d0 0x55c4b265e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.350+0000 7fdfa9e00640 1 -- 172.21.15.1:0/1289914845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 msgr2=0x55c4b265eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.350+0000 7fdfa9e00640 1 --2- 172.21.15.1:0/1289914845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 0x55c4b265eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.350+0000 7fdfa9e00640 1 -- 172.21.15.1:0/1289914845 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4b26ce4a0 con 0x55c4b264d9b0 2024-04-26T02:00:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.350+0000 7fdfa9400640 1 --2- 172.21.15.1:0/1289914845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264e2d0 0x55c4b265e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.350+0000 7fdfa9e00640 1 --2- 172.21.15.1:0/1289914845 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264d9b0 0x55c4b2661080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c004ba0 tx=0x7fdf9c004b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b260546cdb0ef2ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:23.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.350+0000 7fdf8be00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T02:00:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.351+0000 7fdf8be00640 1 -- 172.21.15.1:0/1289914845 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c035040 con 0x55c4b264d9b0 2024-04-26T02:00:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.351+0000 7fdfb3dff740 1 -- 172.21.15.1:0/1289914845 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf980203f0 con 0x55c4b264d9b0 2024-04-26T02:00:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.351+0000 7fdf8be00640 1 -- 172.21.15.1:0/1289914845 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fdf9c004470 con 0x55c4b264d9b0 2024-04-26T02:00:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.351+0000 7fdf8be00640 1 -- 172.21.15.1:0/1289914845 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdf9c060050 con 0x55c4b264d9b0 2024-04-26T02:00:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.351+0000 7fdfb3dff740 10 client.26242 fetch_fsmap learned FSMap version 32 2024-04-26T02:00:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.351+0000 7fdfb3dff740 1 -- 172.21.15.1:0/1289914845 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c4b26cf650 con 0x55c4b264d9b0 2024-04-26T02:00:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.351+0000 7fdf8be00640 1 -- 172.21.15.1:0/1289914845 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf980203f0 con 0x55c4b264d9b0 2024-04-26T02:00:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.351+0000 7fdf8be00640 1 -- 172.21.15.1:0/1289914845 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fdf9c052350 con 0x55c4b264d9b0 2024-04-26T02:00:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.351+0000 7fdfb3dff740 10 client.26242 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:00:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.351+0000 7fdfb3dff740 10 client.26242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:00:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.351+0000 7fdfb3dff740 20 client.26242 populate_metadata read hostname 'smithi001' 2024-04-26T02:00:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.351+0000 7fdfb3dff740 1 --2- 172.21.15.1:0/1289914845 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c4b26cf990 0x55c4b26d1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.351+0000 7fdfb3dff740 4 client.26242 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:00:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.351+0000 7fdfb3dff740 1 -- 172.21.15.1:0/1289914845 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c4b25c1320 con 0x55c4b26cf990 2024-04-26T02:00:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.352+0000 7fdfa9400640 1 --2- 172.21.15.1:0/1289914845 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c4b26cf990 0x55c4b26d1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.352+0000 7fdfa9400640 1 --2- 172.21.15.1:0/1289914845 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c4b26cf990 0x55c4b26d1d80 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9154534028fe800 server_cookie=ea8b893fb394ed93 in_seq=0 out_seq=0 2024-04-26T02:00:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.352+0000 7fdf8be00640 10 client.26242 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.364+0000 7fdf8be00640 1 -- 172.21.15.1:0/1289914845 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c4b25c1320 con 0x55c4b26cf990 2024-04-26T02:00:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.364+0000 7fdf8be00640 10 client.26242 handle_command_reply: tid=0 2024-04-26T02:00:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.365+0000 7fdfb3dff740 1 client.26242 shutdown 2024-04-26T02:00:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.365+0000 7fdfb3dff740 1 -- 172.21.15.1:0/1289914845 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264d9b0 msgr2=0x55c4b2661080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.365+0000 7fdfb3dff740 1 --2- 172.21.15.1:0/1289914845 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264d9b0 0x55c4b2661080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c004ba0 tx=0x7fdf9c004b30 comp rx=0 tx=0).stop 2024-04-26T02:00:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.365+0000 7fdfb3dff740 1 -- 172.21.15.1:0/1289914845 shutdown_connections 2024-04-26T02:00:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.365+0000 7fdfb3dff740 1 --2- 172.21.15.1:0/1289914845 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c4b26cf990 0x55c4b26d1d80 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.365+0000 7fdfb3dff740 1 --2- 172.21.15.1:0/1289914845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4b2658b90 0x55c4b265eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.365+0000 7fdfb3dff740 1 --2- 172.21.15.1:0/1289914845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4b264e2d0 0x55c4b265e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.365+0000 7fdfb3dff740 1 --2- 172.21.15.1:0/1289914845 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c4b264d9b0 0x55c4b2661080 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.365+0000 7fdfb3dff740 1 -- 172.21.15.1:0/1289914845 >> 172.21.15.1:0/1289914845 conn(0x55c4b2649ec0 msgr2=0x55c4b2655aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.365+0000 7fdf8be00640 0 client.26242 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.365+0000 7fdfb3dff740 1 -- 172.21.15.1:0/1289914845 shutdown_connections 2024-04-26T02:00:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.365+0000 7fdfb3dff740 1 -- 172.21.15.1:0/1289914845 wait complete. 2024-04-26T02:00:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.365+0000 7fdfb3dff740 20 client.26242 trim_cache size 0 max 16384 2024-04-26T02:00:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T02:00:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T02:00:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T02:00:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T02:00:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.367+0000 7fdfab200640 1 -- 172.21.15.1:0/2669298687 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fdf900a3d60 msgr2=0x7fdf900a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.367+0000 7fdfab200640 1 --2- 172.21.15.1:0/2669298687 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fdf900a3d60 0x7fdf900a6220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55c4b264b2e0 tx=0x7fdf98004730 comp rx=0 tx=0).stop 2024-04-26T02:00:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.367+0000 7fdfab200640 1 -- 172.21.15.1:0/2669298687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfac19ea80 msgr2=0x7fdfac19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.367+0000 7fdfab200640 1 --2- 172.21.15.1:0/2669298687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfac19ea80 0x7fdfac19ef30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400ea60 tx=0x7fdfa400ef20 comp rx=0 tx=0).stop 2024-04-26T02:00:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.368+0000 7fdfab200640 1 -- 172.21.15.1:0/2669298687 shutdown_connections 2024-04-26T02:00:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.368+0000 7fdfab200640 1 --2- 172.21.15.1:0/2669298687 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fdf900a3d60 0x7fdf900a6220 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.368+0000 7fdfab200640 1 --2- 172.21.15.1:0/2669298687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfac19ea80 0x7fdfac19ef30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.368+0000 7fdfab200640 1 --2- 172.21.15.1:0/2669298687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfac10d700 0x7fdfac1a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.368+0000 7fdfab200640 1 --2- 172.21.15.1:0/2669298687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfac109000 0x7fdfac1a48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.368+0000 7fdfab200640 1 -- 172.21.15.1:0/2669298687 >> 172.21.15.1:0/2669298687 conn(0x7fdfac077940 msgr2=0x7fdfac10bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.368+0000 7fdfab200640 1 -- 172.21.15.1:0/2669298687 shutdown_connections 2024-04-26T02:00:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:23.368+0000 7fdfab200640 1 -- 172.21.15.1:0/2669298687 wait complete. 2024-04-26T02:00:24.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:24 smithi173 ceph-mon[32553]: pgmap v272: 97 pgs: 97 active+clean; 104 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.2 MiB/s wr, 407 op/s 2024-04-26T02:00:24.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:24 smithi001 ceph-mon[27822]: pgmap v272: 97 pgs: 97 active+clean; 104 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.2 MiB/s wr, 407 op/s 2024-04-26T02:00:24.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:24 smithi097 ceph-mon[32853]: pgmap v272: 97 pgs: 97 active+clean; 104 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.2 MiB/s wr, 407 op/s 2024-04-26T02:00:26.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:26 smithi173 ceph-mon[32553]: pgmap v273: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.6 MiB/s wr, 561 op/s 2024-04-26T02:00:26.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:26 smithi001 ceph-mon[27822]: pgmap v273: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.6 MiB/s wr, 561 op/s 2024-04-26T02:00:26.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:26 smithi097 ceph-mon[32853]: pgmap v273: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.6 MiB/s wr, 561 op/s 2024-04-26T02:00:28.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:28 smithi173 ceph-mon[32553]: pgmap v274: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 970 KiB/s wr, 332 op/s 2024-04-26T02:00:28.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:28 smithi001 ceph-mon[27822]: pgmap v274: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 970 KiB/s wr, 332 op/s 2024-04-26T02:00:28.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:28 smithi097 ceph-mon[32853]: pgmap v274: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 970 KiB/s wr, 332 op/s 2024-04-26T02:00:29.913 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x63e5d2a 2024-04-26T02:00:30.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:30 smithi173 ceph-mon[32553]: pgmap v275: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 970 KiB/s wr, 332 op/s 2024-04-26T02:00:30.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:30 smithi001 ceph-mon[27822]: pgmap v275: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 970 KiB/s wr, 332 op/s 2024-04-26T02:00:30.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:30 smithi097 ceph-mon[32853]: pgmap v275: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 970 KiB/s wr, 332 op/s 2024-04-26T02:00:32.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:32 smithi173 ceph-mon[32553]: pgmap v276: 97 pgs: 97 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 492 op/s 2024-04-26T02:00:32.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:32 smithi001 ceph-mon[27822]: pgmap v276: 97 pgs: 97 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 492 op/s 2024-04-26T02:00:32.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:32 smithi097 ceph-mon[32853]: pgmap v276: 97 pgs: 97 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 492 op/s 2024-04-26T02:00:34.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:34 smithi173 ceph-mon[32553]: pgmap v277: 97 pgs: 97 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 997 KiB/s wr, 321 op/s 2024-04-26T02:00:34.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:34 smithi001 ceph-mon[27822]: pgmap v277: 97 pgs: 97 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 997 KiB/s wr, 321 op/s 2024-04-26T02:00:34.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:34 smithi097 ceph-mon[32853]: pgmap v277: 97 pgs: 97 active+clean; 108 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 997 KiB/s wr, 321 op/s 2024-04-26T02:00:35.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:35 smithi001 ceph-mon[27822]: pgmap v278: 97 pgs: 97 active+clean; 122 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 473 op/s 2024-04-26T02:00:35.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:35 smithi097 ceph-mon[32853]: pgmap v278: 97 pgs: 97 active+clean; 122 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 473 op/s 2024-04-26T02:00:35.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:35 smithi173 ceph-mon[32553]: pgmap v278: 97 pgs: 97 active+clean; 122 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 473 op/s 2024-04-26T02:00:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.524+0000 7f0ef0000640 1 -- 172.21.15.1:0/697881712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ee810e750 msgr2=0x7f0ee8110bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.524+0000 7f0ef0000640 1 --2- 172.21.15.1:0/697881712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ee810e750 0x7f0ee8110bc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0009ca0 tx=0x7f0ee002a950 comp rx=0 tx=0).stop 2024-04-26T02:00:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.525+0000 7f0ef0000640 1 -- 172.21.15.1:0/697881712 shutdown_connections 2024-04-26T02:00:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.525+0000 7f0ef0000640 1 --2- 172.21.15.1:0/697881712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ee810e750 0x7f0ee8110bc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.525+0000 7f0ef0000640 1 --2- 172.21.15.1:0/697881712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ee806aad0 0x7f0ee806af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.525+0000 7f0ef0000640 1 --2- 172.21.15.1:0/697881712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ee8102ef0 0x7f0ee806a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.525+0000 7f0ef0000640 1 -- 172.21.15.1:0/697881712 >> 172.21.15.1:0/697881712 conn(0x7f0ee8077d10 msgr2=0x7f0ee8078120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.525+0000 7f0ef0000640 1 -- 172.21.15.1:0/697881712 shutdown_connections 2024-04-26T02:00:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.525+0000 7f0ef0000640 1 -- 172.21.15.1:0/697881712 wait complete. 2024-04-26T02:00:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.526+0000 7f0ef0000640 1 Processor -- start 2024-04-26T02:00:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.526+0000 7f0ef0000640 1 -- start start 2024-04-26T02:00:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.527+0000 7f0ef0000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ee806aad0 0x7f0ee81a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.527+0000 7f0ef0000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ee8102ef0 0x7f0ee81a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.527+0000 7f0ef0000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ee810e750 0x7f0ee819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.527+0000 7f0ef0000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee8114400 con 0x7f0ee810e750 2024-04-26T02:00:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.527+0000 7f0ef0000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee8114280 con 0x7f0ee8102ef0 2024-04-26T02:00:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.527+0000 7f0ef0000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee8114580 con 0x7f0ee806aad0 2024-04-26T02:00:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.527+0000 7f0eed600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ee806aad0 0x7f0ee81a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.527+0000 7f0eed600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ee806aad0 0x7f0ee81a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:53064/0 (socket says 172.21.15.1:53064) 2024-04-26T02:00:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.527+0000 7f0eed600640 1 -- 172.21.15.1:0/3351243009 learned_addr learned my addr 172.21.15.1:0/3351243009 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.527+0000 7f0eee000640 1 --2- 172.21.15.1:0/3351243009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ee810e750 0x7f0ee819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.527+0000 7f0eecc00640 1 --2- 172.21.15.1:0/3351243009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ee8102ef0 0x7f0ee81a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.528+0000 7f0eed600640 1 -- 172.21.15.1:0/3351243009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ee8102ef0 msgr2=0x7f0ee81a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.528+0000 7f0eed600640 1 --2- 172.21.15.1:0/3351243009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ee8102ef0 0x7f0ee81a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.528+0000 7f0eed600640 1 -- 172.21.15.1:0/3351243009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ee810e750 msgr2=0x7f0ee819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.528+0000 7f0eed600640 1 --2- 172.21.15.1:0/3351243009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ee810e750 0x7f0ee819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.528+0000 7f0eed600640 1 -- 172.21.15.1:0/3351243009 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee819af60 con 0x7f0ee806aad0 2024-04-26T02:00:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.528+0000 7f0eed600640 1 --2- 172.21.15.1:0/3351243009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ee806aad0 0x7f0ee81a05e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00d950 tx=0x7f0edc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.528+0000 7f0ee6000640 1 -- 172.21.15.1:0/3351243009 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc017070 con 0x7f0ee806aad0 2024-04-26T02:00:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.528+0000 7f0ee6000640 1 -- 172.21.15.1:0/3351243009 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0edc009c30 con 0x7f0ee806aad0 2024-04-26T02:00:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.528+0000 7f0ef0000640 1 -- 172.21.15.1:0/3351243009 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee819b250 con 0x7f0ee806aad0 2024-04-26T02:00:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.528+0000 7f0ee6000640 1 -- 172.21.15.1:0/3351243009 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc00f620 con 0x7f0ee806aad0 2024-04-26T02:00:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.529+0000 7f0ef0000640 1 -- 172.21.15.1:0/3351243009 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee81a7560 con 0x7f0ee806aad0 2024-04-26T02:00:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.532+0000 7f0ee6000640 1 -- 172.21.15.1:0/3351243009 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f0edc00f7c0 con 0x7f0ee806aad0 2024-04-26T02:00:36.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.532+0000 7f0ef2e69740 1 Processor -- start 2024-04-26T02:00:36.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.532+0000 7f0ef2e69740 1 -- start start 2024-04-26T02:00:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.533+0000 7f0ef2e69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac7546c820 0x55ac7546cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.533+0000 7f0ef2e69740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac7546d140 0x55ac754774c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.533+0000 7f0ef2e69740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac75477a00 0x55ac75479df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.533+0000 7f0ef2e69740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ac753d2900 con 0x55ac7546d140 2024-04-26T02:00:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.533+0000 7f0ef2e69740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ac75378ef0 con 0x55ac75477a00 2024-04-26T02:00:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.533+0000 7f0ef2e69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ac753ce8c0 con 0x55ac7546c820 2024-04-26T02:00:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.533+0000 7f0eed600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac7546d140 0x55ac754774c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.533+0000 7f0eee000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac7546c820 0x55ac7546cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.533+0000 7f0eed600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac7546d140 0x55ac754774c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40802/0 (socket says 172.21.15.1:40802) 2024-04-26T02:00:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.533+0000 7f0eed600640 1 -- 172.21.15.1:0/1524557956 learned_addr learned my addr 172.21.15.1:0/1524557956 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.533+0000 7f0eecc00640 1 --2- 172.21.15.1:0/1524557956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac75477a00 0x55ac75479df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.534+0000 7f0eee000640 1 -- 172.21.15.1:0/1524557956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac75477a00 msgr2=0x55ac75479df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.534+0000 7f0eee000640 1 --2- 172.21.15.1:0/1524557956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac75477a00 0x55ac75479df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.534+0000 7f0eee000640 1 -- 172.21.15.1:0/1524557956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac7546d140 msgr2=0x55ac754774c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.534+0000 7f0eee000640 1 --2- 172.21.15.1:0/1524557956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac7546d140 0x55ac754774c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.534+0000 7f0eee000640 1 -- 172.21.15.1:0/1524557956 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac750e5210 con 0x55ac7546c820 2024-04-26T02:00:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.534+0000 7f0ee6000640 1 --2- 172.21.15.1:0/3351243009 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0ed00a3d10 0x7f0ed00a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.534+0000 7f0ee6000640 1 -- 172.21.15.1:0/3351243009 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f0edc0d0420 con 0x7f0ee806aad0 2024-04-26T02:00:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.534+0000 7f0eecc00640 1 --2- 172.21.15.1:0/3351243009 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0ed00a3d10 0x7f0ed00a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.534+0000 7f0eee000640 1 --2- 172.21.15.1:0/1524557956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac7546c820 0x55ac7546cc00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0005e30 tx=0x7f0ee002f450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a843b6eeca31a63a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.534+0000 7f0eda600640 1 -- 172.21.15.1:0/1524557956 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0048070 con 0x55ac7546c820 2024-04-26T02:00:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.534+0000 7f0eda600640 1 -- 172.21.15.1:0/1524557956 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee0003e20 con 0x55ac7546c820 2024-04-26T02:00:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.534+0000 7f0eda600640 1 -- 172.21.15.1:0/1524557956 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0034040 con 0x55ac7546c820 2024-04-26T02:00:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.535+0000 7f0ef2e69740 1 -- 172.21.15.1:0/1524557956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac7546c820 msgr2=0x55ac7546cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.535+0000 7f0ef2e69740 1 --2- 172.21.15.1:0/1524557956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac7546c820 0x55ac7546cc00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0005e30 tx=0x7f0ee002f450 comp rx=0 tx=0).stop 2024-04-26T02:00:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.535+0000 7f0eecc00640 1 --2- 172.21.15.1:0/3351243009 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0ed00a3d10 0x7f0ed00a61d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55ac7546a150 tx=0x7f0ed4005f90 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.535+0000 7f0ef2e69740 1 -- 172.21.15.1:0/1524557956 shutdown_connections 2024-04-26T02:00:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.535+0000 7f0ef2e69740 1 --2- 172.21.15.1:0/1524557956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac75477a00 0x55ac75479df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.535+0000 7f0ef2e69740 1 --2- 172.21.15.1:0/1524557956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac7546d140 0x55ac754774c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.535+0000 7f0ef2e69740 1 --2- 172.21.15.1:0/1524557956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac7546c820 0x55ac7546cc00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.535+0000 7f0ef2e69740 1 -- 172.21.15.1:0/1524557956 >> 172.21.15.1:0/1524557956 conn(0x55ac75468d30 msgr2=0x55ac75469110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.535+0000 7f0ef2e69740 1 -- 172.21.15.1:0/1524557956 shutdown_connections 2024-04-26T02:00:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.535+0000 7f0ef2e69740 1 -- 172.21.15.1:0/1524557956 wait complete. 2024-04-26T02:00:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.536+0000 7f0ef2e69740 1 -- start start 2024-04-26T02:00:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.536+0000 7f0ef2e69740 1 Processor -- start 2024-04-26T02:00:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.536+0000 7f0ef2e69740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac7546c820 0x55ac75474a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.536+0000 7f0ef2e69740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac7546d140 0x55ac75474f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.536+0000 7f0ef2e69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac75477a00 0x55ac754754b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.537+0000 7f0ef2e69740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ac7547a5d0 con 0x55ac7546d140 2024-04-26T02:00:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.537+0000 7f0ef2e69740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ac7538c310 con 0x55ac7546c820 2024-04-26T02:00:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.537+0000 7f0ef2e69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ac7547a750 con 0x55ac75477a00 2024-04-26T02:00:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.537+0000 7f0ef2e69740 10 client.0 _ll_register_callbacks cb 0x55ac7535acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.537+0000 7f0eed600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac7546c820 0x55ac75474a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.537+0000 7f0eed600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac7546c820 0x55ac75474a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:43086/0 (socket says 172.21.15.1:43086) 2024-04-26T02:00:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.537+0000 7f0eed600640 1 -- 172.21.15.1:0/2752140577 learned_addr learned my addr 172.21.15.1:0/2752140577 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.537+0000 7f0eecc00640 1 --2- 172.21.15.1:0/2752140577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac7546d140 0x55ac75474f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.537+0000 7f0eee000640 1 --2- 172.21.15.1:0/2752140577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac75477a00 0x55ac754754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.537+0000 7f0eed600640 1 -- 172.21.15.1:0/2752140577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac75477a00 msgr2=0x55ac754754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.537+0000 7f0eed600640 1 --2- 172.21.15.1:0/2752140577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac75477a00 0x55ac754754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.537+0000 7f0eed600640 1 -- 172.21.15.1:0/2752140577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac7546d140 msgr2=0x55ac75474f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.537+0000 7f0eed600640 1 --2- 172.21.15.1:0/2752140577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac7546d140 0x55ac75474f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.537+0000 7f0eed600640 1 -- 172.21.15.1:0/2752140577 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac751b1620 con 0x55ac7546c820 2024-04-26T02:00:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.537+0000 7f0eee000640 1 --2- 172.21.15.1:0/2752140577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac75477a00 0x55ac754754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:00:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.537+0000 7f0eecc00640 1 --2- 172.21.15.1:0/2752140577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac7546d140 0x55ac75474f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:00:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.537+0000 7f0eed600640 1 --2- 172.21.15.1:0/2752140577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac7546c820 0x55ac75474a30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0058c0 tx=0x7f0edc0d3a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd64f845aa3f09d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.537+0000 7f0ecea00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:00:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.538+0000 7f0ecea00640 1 -- 172.21.15.1:0/2752140577 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc020050 con 0x55ac7546c820 2024-04-26T02:00:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.538+0000 7f0ecea00640 1 -- 172.21.15.1:0/2752140577 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f0edc0a01f0 con 0x55ac7546c820 2024-04-26T02:00:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.538+0000 7f0ef2e69740 1 -- 172.21.15.1:0/2752140577 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ee0003e20 con 0x55ac7546c820 2024-04-26T02:00:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.538+0000 7f0ecea00640 1 -- 172.21.15.1:0/2752140577 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0edc09bf20 con 0x55ac7546c820 2024-04-26T02:00:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.538+0000 7f0ef2e69740 10 client.26237 fetch_fsmap learned FSMap version 32 2024-04-26T02:00:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.538+0000 7f0ef2e69740 1 -- 172.21.15.1:0/2752140577 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ac75486920 con 0x55ac7546c820 2024-04-26T02:00:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.538+0000 7f0ecea00640 1 -- 172.21.15.1:0/2752140577 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee0003e20 con 0x55ac7546c820 2024-04-26T02:00:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.538+0000 7f0ecea00640 1 -- 172.21.15.1:0/2752140577 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f0edc098130 con 0x55ac7546c820 2024-04-26T02:00:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.538+0000 7f0ef2e69740 10 client.26237 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:00:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.538+0000 7f0ef2e69740 10 client.26237 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:00:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.539+0000 7f0ef2e69740 20 client.26237 populate_metadata read hostname 'smithi001' 2024-04-26T02:00:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.539+0000 7f0ef2e69740 1 --2- 172.21.15.1:0/2752140577 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ac75487e30 0x55ac7548a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.539+0000 7f0ef2e69740 4 client.26237 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:00:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.539+0000 7f0ef2e69740 1 -- 172.21.15.1:0/2752140577 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac753e68b0 con 0x55ac75487e30 2024-04-26T02:00:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.539+0000 7f0eecc00640 1 --2- 172.21.15.1:0/2752140577 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ac75487e30 0x55ac7548a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.539+0000 7f0eecc00640 1 --2- 172.21.15.1:0/2752140577 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ac75487e30 0x55ac7548a260 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca1c8784859b599b server_cookie=420deea73927bba0 in_seq=0 out_seq=0 2024-04-26T02:00:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.539+0000 7f0ecea00640 10 client.26237 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.540+0000 7f0ecea00640 1 -- 172.21.15.1:0/2752140577 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ac753e68b0 con 0x55ac75487e30 2024-04-26T02:00:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.540+0000 7f0ecea00640 10 client.26237 handle_command_reply: tid=0 2024-04-26T02:00:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.540+0000 7f0ef2e69740 1 client.26237 shutdown 2024-04-26T02:00:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.540+0000 7f0ef2e69740 1 -- 172.21.15.1:0/2752140577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac7546c820 msgr2=0x55ac75474a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.540+0000 7f0ef2e69740 1 --2- 172.21.15.1:0/2752140577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac7546c820 0x55ac75474a30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0058c0 tx=0x7f0edc0d3a00 comp rx=0 tx=0).stop 2024-04-26T02:00:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.541+0000 7f0ef2e69740 1 -- 172.21.15.1:0/2752140577 shutdown_connections 2024-04-26T02:00:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.541+0000 7f0ef2e69740 1 --2- 172.21.15.1:0/2752140577 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ac75487e30 0x55ac7548a260 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.541+0000 7f0ef2e69740 1 --2- 172.21.15.1:0/2752140577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac75477a00 0x55ac754754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.541+0000 7f0ef2e69740 1 --2- 172.21.15.1:0/2752140577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac7546d140 0x55ac75474f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.541+0000 7f0ef2e69740 1 --2- 172.21.15.1:0/2752140577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac7546c820 0x55ac75474a30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.541+0000 7f0ecea00640 0 client.26237 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.541+0000 7f0ef2e69740 1 -- 172.21.15.1:0/2752140577 >> 172.21.15.1:0/2752140577 conn(0x55ac75468d30 msgr2=0x55ac7546eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.541+0000 7f0ef2e69740 1 -- 172.21.15.1:0/2752140577 shutdown_connections 2024-04-26T02:00:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.541+0000 7f0ef2e69740 1 -- 172.21.15.1:0/2752140577 wait complete. 2024-04-26T02:00:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.541+0000 7f0ef2e69740 20 client.26237 trim_cache size 0 max 16384 2024-04-26T02:00:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.558+0000 7f0ef2e69740 1 Processor -- start 2024-04-26T02:00:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.558+0000 7f0ef2e69740 1 -- start start 2024-04-26T02:00:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.558+0000 7f0ef2e69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac7546c820 0x55ac75483490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.558+0000 7f0ef2e69740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac7546d140 0x55ac7546e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.558+0000 7f0ef2e69740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac75477a00 0x55ac7548fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.558+0000 7f0ef2e69740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee81b26d0 con 0x55ac7546d140 2024-04-26T02:00:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.558+0000 7f0ef2e69740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee81affa0 con 0x55ac75477a00 2024-04-26T02:00:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.558+0000 7f0ef2e69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee81ade10 con 0x55ac7546c820 2024-04-26T02:00:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.559+0000 7f0eed600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac7546c820 0x55ac75483490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.559+0000 7f0eee000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac75477a00 0x55ac7548fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.559+0000 7f0eed600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac7546c820 0x55ac75483490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:53086/0 (socket says 172.21.15.1:53086) 2024-04-26T02:00:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.559+0000 7f0eed600640 1 -- 172.21.15.1:0/1930370934 learned_addr learned my addr 172.21.15.1:0/1930370934 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.559+0000 7f0eecc00640 1 --2- 172.21.15.1:0/1930370934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac7546d140 0x55ac7546e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.559+0000 7f0eee000640 1 -- 172.21.15.1:0/1930370934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac7546c820 msgr2=0x55ac75483490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.559+0000 7f0eee000640 1 --2- 172.21.15.1:0/1930370934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac7546c820 0x55ac75483490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.559+0000 7f0eee000640 1 -- 172.21.15.1:0/1930370934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac7546d140 msgr2=0x55ac7546e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.559+0000 7f0eee000640 1 --2- 172.21.15.1:0/1930370934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac7546d140 0x55ac7546e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.559+0000 7f0eee000640 1 -- 172.21.15.1:0/1930370934 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac7548a8e0 con 0x55ac75477a00 2024-04-26T02:00:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.560+0000 7f0eee000640 1 --2- 172.21.15.1:0/1930370934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac75477a00 0x55ac7548fb70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0ee002af40 tx=0x7f0ee002b050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9338413b57a6b041 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.560+0000 7f0ed9c00640 1 -- 172.21.15.1:0/1930370934 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0034050 con 0x55ac75477a00 2024-04-26T02:00:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.560+0000 7f0ed9c00640 1 -- 172.21.15.1:0/1930370934 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee0003e20 con 0x55ac75477a00 2024-04-26T02:00:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.560+0000 7f0ed9c00640 1 -- 172.21.15.1:0/1930370934 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0048070 con 0x55ac75477a00 2024-04-26T02:00:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.561+0000 7f0ef2e69740 1 -- 172.21.15.1:0/1930370934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac75477a00 msgr2=0x55ac7548fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.561+0000 7f0ef2e69740 1 --2- 172.21.15.1:0/1930370934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac75477a00 0x55ac7548fb70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0ee002af40 tx=0x7f0ee002b050 comp rx=0 tx=0).stop 2024-04-26T02:00:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.561+0000 7f0ef2e69740 1 -- 172.21.15.1:0/1930370934 shutdown_connections 2024-04-26T02:00:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.561+0000 7f0ef2e69740 1 --2- 172.21.15.1:0/1930370934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac75477a00 0x55ac7548fb70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.561+0000 7f0ef2e69740 1 --2- 172.21.15.1:0/1930370934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac7546d140 0x55ac7546e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.561+0000 7f0ef2e69740 1 --2- 172.21.15.1:0/1930370934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac7546c820 0x55ac75483490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.561+0000 7f0ef2e69740 1 -- 172.21.15.1:0/1930370934 >> 172.21.15.1:0/1930370934 conn(0x55ac75468d30 msgr2=0x55ac7546e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.562+0000 7f0ef2e69740 1 -- 172.21.15.1:0/1930370934 shutdown_connections 2024-04-26T02:00:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.562+0000 7f0ef2e69740 1 -- 172.21.15.1:0/1930370934 wait complete. 2024-04-26T02:00:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.562+0000 7f0ef2e69740 1 -- start start 2024-04-26T02:00:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.563+0000 7f0ef2e69740 1 Processor -- start 2024-04-26T02:00:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.563+0000 7f0ef2e69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac7546c820 0x55ac7547d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.563+0000 7f0ef2e69740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac7546d140 0x55ac7547da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.563+0000 7f0ef2e69740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac75477a00 0x55ac7547df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.563+0000 7f0ef2e69740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee81ae460 con 0x55ac75477a00 2024-04-26T02:00:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.563+0000 7f0ef2e69740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee81af8d0 con 0x55ac7546d140 2024-04-26T02:00:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.563+0000 7f0ef2e69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee81aeb30 con 0x55ac7546c820 2024-04-26T02:00:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.563+0000 7f0ef2e69740 10 client.0 _ll_register_callbacks cb 0x7f0edc0d3bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.564+0000 7f0eed600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac7546c820 0x55ac7547d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.564+0000 7f0eed600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac7546c820 0x55ac7547d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:53088/0 (socket says 172.21.15.1:53088) 2024-04-26T02:00:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.564+0000 7f0eed600640 1 -- 172.21.15.1:0/2067001790 learned_addr learned my addr 172.21.15.1:0/2067001790 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.564+0000 7f0eecc00640 1 --2- 172.21.15.1:0/2067001790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac7546d140 0x55ac7547da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.564+0000 7f0eed600640 1 -- 172.21.15.1:0/2067001790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac7546d140 msgr2=0x55ac7547da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.564+0000 7f0eed600640 1 --2- 172.21.15.1:0/2067001790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac7546d140 0x55ac7547da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.564+0000 7f0eed600640 1 -- 172.21.15.1:0/2067001790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac75477a00 msgr2=0x55ac7547df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.564+0000 7f0eed600640 1 --2- 172.21.15.1:0/2067001790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac75477a00 0x55ac7547df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.564+0000 7f0eed600640 1 -- 172.21.15.1:0/2067001790 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac754ed4f0 con 0x55ac7546c820 2024-04-26T02:00:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.564+0000 7f0eed600640 1 --2- 172.21.15.1:0/2067001790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac7546c820 0x55ac7547d4f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0edc09c0e0 tx=0x7f0edc0d3af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b29836dd8415f8e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.564+0000 7f0ecea00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T02:00:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.565+0000 7f0ecea00640 1 -- 172.21.15.1:0/2067001790 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc014030 con 0x55ac7546c820 2024-04-26T02:00:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.565+0000 7f0ecea00640 1 -- 172.21.15.1:0/2067001790 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f0edc0cdd60 con 0x55ac7546c820 2024-04-26T02:00:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.565+0000 7f0ef2e69740 1 -- 172.21.15.1:0/2067001790 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ee0003e20 con 0x55ac7546c820 2024-04-26T02:00:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.565+0000 7f0ecea00640 1 -- 172.21.15.1:0/2067001790 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0edc020040 con 0x55ac7546c820 2024-04-26T02:00:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.565+0000 7f0ef2e69740 10 client.26260 fetch_fsmap learned FSMap version 32 2024-04-26T02:00:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.565+0000 7f0ef2e69740 1 -- 172.21.15.1:0/2067001790 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ac754ee800 con 0x55ac7546c820 2024-04-26T02:00:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.565+0000 7f0ecea00640 1 -- 172.21.15.1:0/2067001790 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee0003e20 con 0x55ac7546c820 2024-04-26T02:00:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.565+0000 7f0ecea00640 1 -- 172.21.15.1:0/2067001790 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f0edc09b2a0 con 0x55ac7546c820 2024-04-26T02:00:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.565+0000 7f0ef2e69740 10 client.26260 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:00:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.565+0000 7f0ef2e69740 10 client.26260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:00:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.565+0000 7f0ef2e69740 20 client.26260 populate_metadata read hostname 'smithi001' 2024-04-26T02:00:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.565+0000 7f0ef2e69740 1 --2- 172.21.15.1:0/2067001790 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ac754eef00 0x55ac754f1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.566+0000 7f0ef2e69740 4 client.26260 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:00:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.566+0000 7f0ef2e69740 1 -- 172.21.15.1:0/2067001790 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac75468b80 con 0x55ac754eef00 2024-04-26T02:00:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.566+0000 7f0eecc00640 1 --2- 172.21.15.1:0/2067001790 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ac754eef00 0x55ac754f1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.566+0000 7f0eecc00640 1 --2- 172.21.15.1:0/2067001790 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ac754eef00 0x55ac754f1320 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d07e071ec229cfa server_cookie=c6e11557aaff4e48 in_seq=0 out_seq=0 2024-04-26T02:00:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.566+0000 7f0ecea00640 10 client.26260 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.580+0000 7f0ecea00640 1 -- 172.21.15.1:0/2067001790 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac75468b80 con 0x55ac754eef00 2024-04-26T02:00:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.580+0000 7f0ecea00640 10 client.26260 handle_command_reply: tid=0 2024-04-26T02:00:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.580+0000 7f0ef2e69740 1 client.26260 shutdown 2024-04-26T02:00:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.580+0000 7f0ef2e69740 1 -- 172.21.15.1:0/2067001790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac7546c820 msgr2=0x55ac7547d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.580+0000 7f0ef2e69740 1 --2- 172.21.15.1:0/2067001790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac7546c820 0x55ac7547d4f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0edc09c0e0 tx=0x7f0edc0d3af0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.581+0000 7f0ef2e69740 1 -- 172.21.15.1:0/2067001790 shutdown_connections 2024-04-26T02:00:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.581+0000 7f0ef2e69740 1 --2- 172.21.15.1:0/2067001790 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ac754eef00 0x55ac754f1320 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.581+0000 7f0ef2e69740 1 --2- 172.21.15.1:0/2067001790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac75477a00 0x55ac7547df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.581+0000 7f0ef2e69740 1 --2- 172.21.15.1:0/2067001790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac7546d140 0x55ac7547da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.581+0000 7f0ef2e69740 1 --2- 172.21.15.1:0/2067001790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac7546c820 0x55ac7547d4f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.581+0000 7f0ecea00640 0 client.26260 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.581+0000 7f0ef2e69740 1 -- 172.21.15.1:0/2067001790 >> 172.21.15.1:0/2067001790 conn(0x55ac75468d30 msgr2=0x55ac75475280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.581+0000 7f0ef2e69740 1 -- 172.21.15.1:0/2067001790 shutdown_connections 2024-04-26T02:00:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.581+0000 7f0ef2e69740 1 -- 172.21.15.1:0/2067001790 wait complete. 2024-04-26T02:00:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.581+0000 7f0ef2e69740 20 client.26260 trim_cache size 0 max 16384 2024-04-26T02:00:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T02:00:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T02:00:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T02:00:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T02:00:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.582+0000 7f0ef0000640 1 -- 172.21.15.1:0/3351243009 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0ed00a3d10 msgr2=0x7f0ed00a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.582+0000 7f0ef0000640 1 --2- 172.21.15.1:0/3351243009 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0ed00a3d10 0x7f0ed00a61d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55ac7546a150 tx=0x7f0ed4005f90 comp rx=0 tx=0).stop 2024-04-26T02:00:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.582+0000 7f0ef0000640 1 -- 172.21.15.1:0/3351243009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ee806aad0 msgr2=0x7f0ee81a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.582+0000 7f0ef0000640 1 --2- 172.21.15.1:0/3351243009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ee806aad0 0x7f0ee81a05e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00d950 tx=0x7f0edc00de10 comp rx=0 tx=0).stop 2024-04-26T02:00:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.582+0000 7f0ef0000640 1 -- 172.21.15.1:0/3351243009 shutdown_connections 2024-04-26T02:00:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.582+0000 7f0ef0000640 1 --2- 172.21.15.1:0/3351243009 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0ed00a3d10 0x7f0ed00a61d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.582+0000 7f0ef0000640 1 --2- 172.21.15.1:0/3351243009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ee810e750 0x7f0ee819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.582+0000 7f0ef0000640 1 --2- 172.21.15.1:0/3351243009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ee8102ef0 0x7f0ee81a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.582+0000 7f0ef0000640 1 --2- 172.21.15.1:0/3351243009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ee806aad0 0x7f0ee81a05e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.582+0000 7f0ef0000640 1 -- 172.21.15.1:0/3351243009 >> 172.21.15.1:0/3351243009 conn(0x7f0ee8077d10 msgr2=0x7f0ee810f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.582+0000 7f0ef0000640 1 -- 172.21.15.1:0/3351243009 shutdown_connections 2024-04-26T02:00:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:36.582+0000 7f0ef0000640 1 -- 172.21.15.1:0/3351243009 wait complete. 2024-04-26T02:00:36.973 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x63e9b52 2024-04-26T02:00:38.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:37 smithi097 ceph-mon[32853]: pgmap v279: 97 pgs: 97 active+clean; 122 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 994 KiB/s wr, 312 op/s 2024-04-26T02:00:38.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:37 smithi173 ceph-mon[32553]: pgmap v279: 97 pgs: 97 active+clean; 122 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 994 KiB/s wr, 312 op/s 2024-04-26T02:00:38.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:37 smithi001 ceph-mon[27822]: pgmap v279: 97 pgs: 97 active+clean; 122 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 994 KiB/s wr, 312 op/s 2024-04-26T02:00:40.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:39 smithi097 ceph-mon[32853]: pgmap v280: 97 pgs: 97 active+clean; 122 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 995 KiB/s wr, 312 op/s 2024-04-26T02:00:40.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:39 smithi173 ceph-mon[32553]: pgmap v280: 97 pgs: 97 active+clean; 122 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 995 KiB/s wr, 312 op/s 2024-04-26T02:00:40.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:39 smithi001 ceph-mon[27822]: pgmap v280: 97 pgs: 97 active+clean; 122 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 995 KiB/s wr, 312 op/s 2024-04-26T02:00:42.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:41 smithi173 ceph-mon[32553]: pgmap v281: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 468 op/s 2024-04-26T02:00:42.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:41 smithi001 ceph-mon[27822]: pgmap v281: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 468 op/s 2024-04-26T02:00:42.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:41 smithi097 ceph-mon[32853]: pgmap v281: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 468 op/s 2024-04-26T02:00:42.607 INFO:tasks.workunit.client.0.smithi001.stdout:20000 read 0xa6cbf1 thru 0xa6e358 (0x1768 bytes) 2024-04-26T02:00:44.285 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:43 smithi097 ceph-mon[32853]: pgmap v282: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 968 KiB/s wr, 308 op/s 2024-04-26T02:00:44.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:43 smithi173 ceph-mon[32553]: pgmap v282: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 968 KiB/s wr, 308 op/s 2024-04-26T02:00:44.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:43 smithi001 ceph-mon[27822]: pgmap v282: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 968 KiB/s wr, 308 op/s 2024-04-26T02:00:46.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:45 smithi173 ceph-mon[32553]: pgmap v283: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 1.4 MiB/s wr, 458 op/s 2024-04-26T02:00:46.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:45 smithi001 ceph-mon[27822]: pgmap v283: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 1.4 MiB/s wr, 458 op/s 2024-04-26T02:00:46.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:45 smithi097 ceph-mon[32853]: pgmap v283: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 1.4 MiB/s wr, 458 op/s 2024-04-26T02:00:46.640 INFO:tasks.quiescer.fs.[cephfs]:Total duration (44.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-26T02:00:46.641 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.1 2024-04-26T02:00:46.641 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:00:47.123 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:00:47.806 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0x63ed6b9 2024-04-26T02:00:48.252 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:48 smithi001 ceph-mon[27822]: pgmap v284: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 905 KiB/s wr, 305 op/s 2024-04-26T02:00:48.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:48 smithi173 ceph-mon[32553]: pgmap v284: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 905 KiB/s wr, 305 op/s 2024-04-26T02:00:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:48 smithi097 ceph-mon[32853]: pgmap v284: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 905 KiB/s wr, 305 op/s 2024-04-26T02:00:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.742+0000 7f0bbc000640 1 -- 172.21.15.1:0/337154407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bb410e7d0 msgr2=0x7f0bb410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.742+0000 7f0bbc000640 1 --2- 172.21.15.1:0/337154407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bb410e7d0 0x7f0bb410ec30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009900 tx=0x7f0ba002a410 comp rx=0 tx=0).stop 2024-04-26T02:00:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.743+0000 7f0bbc000640 1 -- 172.21.15.1:0/337154407 shutdown_connections 2024-04-26T02:00:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.743+0000 7f0bbc000640 1 --2- 172.21.15.1:0/337154407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bb411b730 0x7f0bb411db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.743+0000 7f0bbc000640 1 --2- 172.21.15.1:0/337154407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bb410e7d0 0x7f0bb410ec30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.743+0000 7f0bbc000640 1 --2- 172.21.15.1:0/337154407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bb41106f0 0x7f0bb410e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.743+0000 7f0bbc000640 1 -- 172.21.15.1:0/337154407 >> 172.21.15.1:0/337154407 conn(0x7f0bb406d030 msgr2=0x7f0bb406d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.743+0000 7f0bbc000640 1 -- 172.21.15.1:0/337154407 shutdown_connections 2024-04-26T02:00:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.743+0000 7f0bbc000640 1 -- 172.21.15.1:0/337154407 wait complete. 2024-04-26T02:00:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.744+0000 7f0bbc000640 1 Processor -- start 2024-04-26T02:00:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.744+0000 7f0bbc000640 1 -- start start 2024-04-26T02:00:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.744+0000 7f0bbc000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bb410e7d0 0x7f0bb41a7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.744+0000 7f0bbc000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bb41106f0 0x7f0bb41a7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.745+0000 7f0bbc000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bb411b730 0x7f0bb41abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.745+0000 7f0bbc000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb4121020 con 0x7f0bb410e7d0 2024-04-26T02:00:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.745+0000 7f0bbc000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb4120ea0 con 0x7f0bb41106f0 2024-04-26T02:00:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.745+0000 7f0bbc000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb41211a0 con 0x7f0bb411b730 2024-04-26T02:00:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.745+0000 7f0bb9600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bb410e7d0 0x7f0bb41a7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.745+0000 7f0bb8c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bb41106f0 0x7f0bb41a7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.745+0000 7f0bb8c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bb41106f0 0x7f0bb41a7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:36896/0 (socket says 172.21.15.1:36896) 2024-04-26T02:00:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.745+0000 7f0bb8c00640 1 -- 172.21.15.1:0/3811047200 learned_addr learned my addr 172.21.15.1:0/3811047200 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.745+0000 7f0bba000640 1 --2- 172.21.15.1:0/3811047200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bb411b730 0x7f0bb41abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.745+0000 7f0bb8c00640 1 -- 172.21.15.1:0/3811047200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bb411b730 msgr2=0x7f0bb41abde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.745+0000 7f0bb8c00640 1 --2- 172.21.15.1:0/3811047200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bb411b730 0x7f0bb41abde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.745+0000 7f0bb8c00640 1 -- 172.21.15.1:0/3811047200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bb410e7d0 msgr2=0x7f0bb41a7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.745+0000 7f0bb8c00640 1 --2- 172.21.15.1:0/3811047200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bb410e7d0 0x7f0bb41a7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.745+0000 7f0bb8c00640 1 -- 172.21.15.1:0/3811047200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb41ac510 con 0x7f0bb41106f0 2024-04-26T02:00:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.746+0000 7f0bb8c00640 1 --2- 172.21.15.1:0/3811047200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bb41106f0 0x7f0bb41a7a60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0005ab0 tx=0x7f0ba0003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.746+0000 7f0bb2000640 1 -- 172.21.15.1:0/3811047200 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba003f070 con 0x7f0bb41106f0 2024-04-26T02:00:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.746+0000 7f0bb2000640 1 -- 172.21.15.1:0/3811047200 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ba0031b60 con 0x7f0bb41106f0 2024-04-26T02:00:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.746+0000 7f0bbc000640 1 -- 172.21.15.1:0/3811047200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb41ac7a0 con 0x7f0bb41106f0 2024-04-26T02:00:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.746+0000 7f0bb2000640 1 -- 172.21.15.1:0/3811047200 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0035030 con 0x7f0bb41106f0 2024-04-26T02:00:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.746+0000 7f0bbc000640 1 -- 172.21.15.1:0/3811047200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb41b4090 con 0x7f0bb41106f0 2024-04-26T02:00:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.749+0000 7f0bb2000640 1 -- 172.21.15.1:0/3811047200 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f0ba0042070 con 0x7f0bb41106f0 2024-04-26T02:00:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.751+0000 7f0bbedf3740 1 Processor -- start 2024-04-26T02:00:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.751+0000 7f0bbedf3740 1 -- start start 2024-04-26T02:00:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.751+0000 7f0bbedf3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7db357820 0x55c7db357c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.751+0000 7f0bbedf3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 0x55c7db3624c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.751+0000 7f0bbedf3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 0x55c7db364df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.751+0000 7f0bbedf3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c7db2bd900 con 0x55c7db357820 2024-04-26T02:00:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.751+0000 7f0bbedf3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c7db263ef0 con 0x55c7db362a00 2024-04-26T02:00:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.751+0000 7f0bbedf3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c7db2b98c0 con 0x55c7db358140 2024-04-26T02:00:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.752+0000 7f0bb9600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 0x55c7db3624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.752+0000 7f0bb8c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 0x55c7db364df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.752+0000 7f0bb9600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 0x55c7db3624c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:47910/0 (socket says 172.21.15.1:47910) 2024-04-26T02:00:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.752+0000 7f0bb9600640 1 -- 172.21.15.1:0/4192813810 learned_addr learned my addr 172.21.15.1:0/4192813810 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.752+0000 7f0bba000640 1 --2- 172.21.15.1:0/4192813810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7db357820 0x55c7db357c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.752+0000 7f0bb2000640 1 --2- 172.21.15.1:0/3811047200 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0b9c0a3d90 0x7f0b9c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.752+0000 7f0bb2000640 1 -- 172.21.15.1:0/3811047200 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f0ba00f0b70 con 0x7f0bb41106f0 2024-04-26T02:00:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.752+0000 7f0bb8c00640 1 -- 172.21.15.1:0/4192813810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 msgr2=0x55c7db3624c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.752+0000 7f0bb8c00640 1 --2- 172.21.15.1:0/4192813810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 0x55c7db3624c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.752+0000 7f0bb8c00640 1 -- 172.21.15.1:0/4192813810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7db357820 msgr2=0x55c7db357c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.752+0000 7f0bb8c00640 1 --2- 172.21.15.1:0/4192813810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7db357820 0x55c7db357c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.752+0000 7f0bb8c00640 1 -- 172.21.15.1:0/4192813810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7dafd0210 con 0x55c7db362a00 2024-04-26T02:00:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.753+0000 7f0bba000640 1 --2- 172.21.15.1:0/3811047200 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0b9c0a3d90 0x7f0b9c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.753+0000 7f0bb8c00640 1 --2- 172.21.15.1:0/4192813810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 0x55c7db364df0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0ba00ee230 tx=0x7f0ba0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=276b8d4eee66f192 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.753+0000 7f0ba6600640 1 -- 172.21.15.1:0/4192813810 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0049030 con 0x55c7db362a00 2024-04-26T02:00:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.753+0000 7f0ba6600640 1 -- 172.21.15.1:0/4192813810 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ba00dd0f0 con 0x55c7db362a00 2024-04-26T02:00:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.753+0000 7f0ba6600640 1 -- 172.21.15.1:0/4192813810 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba00baec0 con 0x55c7db362a00 2024-04-26T02:00:49.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.752+0000 7f2a74e00700 1 -- 172.21.15.1:0/4184103772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a70102a90 msgr2=0x7f2a70102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:49.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.752+0000 7f2a74e00700 1 --2- 172.21.15.1:0/4184103772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a70102a90 0x7f2a70102f10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0098f0 tx=0x7f2a5c009c00 comp rx=0 tx=0).stop 2024-04-26T02:00:49.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.753+0000 7f2a74e00700 1 -- 172.21.15.1:0/4184103772 shutdown_connections 2024-04-26T02:00:49.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.753+0000 7f2a74e00700 1 --2- 172.21.15.1:0/4184103772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a70103450 0x7f2a7010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.753+0000 7f2a74e00700 1 --2- 172.21.15.1:0/4184103772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a70108a50 0x7f2a70108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.753+0000 7f2a74e00700 1 --2- 172.21.15.1:0/4184103772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a70102a90 0x7f2a70102f10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.753+0000 7f2a74e00700 1 -- 172.21.15.1:0/4184103772 >> 172.21.15.1:0/4184103772 conn(0x7f2a70079ba0 msgr2=0x7f2a70079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:49.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.753+0000 7f2a74e00700 1 -- 172.21.15.1:0/4184103772 shutdown_connections 2024-04-26T02:00:49.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.753+0000 7f2a74e00700 1 -- 172.21.15.1:0/4184103772 wait complete. 2024-04-26T02:00:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.754+0000 7f0bbedf3740 1 -- 172.21.15.1:0/4192813810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 msgr2=0x55c7db364df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.754+0000 7f0bbedf3740 1 --2- 172.21.15.1:0/4192813810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 0x55c7db364df0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0ba00ee230 tx=0x7f0ba0009260 comp rx=0 tx=0).stop 2024-04-26T02:00:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.754+0000 7f0bba000640 1 --2- 172.21.15.1:0/3811047200 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0b9c0a3d90 0x7f0b9c0a6250 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009710 tx=0x7f0ba80046c0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.754+0000 7f0bbedf3740 1 -- 172.21.15.1:0/4192813810 shutdown_connections 2024-04-26T02:00:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.754+0000 7f0bbedf3740 1 --2- 172.21.15.1:0/4192813810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 0x55c7db364df0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.754+0000 7f0bbedf3740 1 --2- 172.21.15.1:0/4192813810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 0x55c7db3624c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.754+0000 7f0bbedf3740 1 --2- 172.21.15.1:0/4192813810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7db357820 0x55c7db357c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.754+0000 7f0bbedf3740 1 -- 172.21.15.1:0/4192813810 >> 172.21.15.1:0/4192813810 conn(0x55c7db353d30 msgr2=0x55c7db354110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.754+0000 7f0bbedf3740 1 -- 172.21.15.1:0/4192813810 shutdown_connections 2024-04-26T02:00:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.754+0000 7f0bbedf3740 1 -- 172.21.15.1:0/4192813810 wait complete. 2024-04-26T02:00:49.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.755+0000 7f2a74e00700 1 Processor -- start 2024-04-26T02:00:49.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.755+0000 7f2a74e00700 1 -- start start 2024-04-26T02:00:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.755+0000 7f0bbedf3740 1 -- start start 2024-04-26T02:00:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.755+0000 7f0bbedf3740 1 Processor -- start 2024-04-26T02:00:49.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.755+0000 7f2a74e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a70102a90 0x7f2a7019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.755+0000 7f2a74e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a70103450 0x7f2a7019b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.755+0000 7f2a74e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a70108a50 0x7f2a7019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.755+0000 7f2a74e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2a70112be0 con 0x7f2a70103450 2024-04-26T02:00:49.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.755+0000 7f2a74e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2a70112a60 con 0x7f2a70108a50 2024-04-26T02:00:49.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.755+0000 7f2a74e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2a70112d60 con 0x7f2a70102a90 2024-04-26T02:00:49.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.756+0000 7f2a6e800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a70108a50 0x7f2a7019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:49.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.756+0000 7f2a6e800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a70108a50 0x7f2a7019fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:36924/0 (socket says 172.21.15.1:36924) 2024-04-26T02:00:49.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.756+0000 7f2a6e800700 1 -- 172.21.15.1:0/918812264 learned_addr learned my addr 172.21.15.1:0/918812264 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:49.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.756+0000 7f2a6de00700 1 --2- 172.21.15.1:0/918812264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a70102a90 0x7f2a7019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:49.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.756+0000 7f2a6d400700 1 --2- 172.21.15.1:0/918812264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a70103450 0x7f2a7019b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.756+0000 7f0bbedf3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7db357820 0x55c7db35f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.756+0000 7f0bbedf3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 0x55c7db35fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.756+0000 7f0bbedf3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 0x55c7db3603e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.756+0000 7f0bbedf3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c7db263ef0 con 0x55c7db357820 2024-04-26T02:00:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.756+0000 7f0bbedf3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c7db277310 con 0x55c7db362a00 2024-04-26T02:00:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.756+0000 7f0bbedf3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c7db3653f0 con 0x55c7db358140 2024-04-26T02:00:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.757+0000 7f0bbedf3740 10 client.0 _ll_register_callbacks cb 0x55c7db245cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:49.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.756+0000 7f2a6de00700 1 -- 172.21.15.1:0/918812264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a70108a50 msgr2=0x7f2a7019fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:49.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.756+0000 7f2a6de00700 1 --2- 172.21.15.1:0/918812264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a70108a50 0x7f2a7019fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.756+0000 7f2a6de00700 1 -- 172.21.15.1:0/918812264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a70103450 msgr2=0x7f2a7019b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:49.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.756+0000 7f2a6de00700 1 --2- 172.21.15.1:0/918812264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a70103450 0x7f2a7019b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.756+0000 7f2a6de00700 1 -- 172.21.15.1:0/918812264 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a701a0360 con 0x7f2a70102a90 2024-04-26T02:00:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.757+0000 7f0bba000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 0x55c7db3603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.757+0000 7f0bba000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 0x55c7db3603e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:36934/0 (socket says 172.21.15.1:36934) 2024-04-26T02:00:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.757+0000 7f0bba000640 1 -- 172.21.15.1:0/373426841 learned_addr learned my addr 172.21.15.1:0/373426841 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.757+0000 7f0bba000640 1 -- 172.21.15.1:0/373426841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 msgr2=0x55c7db35fea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T02:00:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.757+0000 7f0bba000640 1 --2- 172.21.15.1:0/373426841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 0x55c7db35fea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.757+0000 7f0bba000640 1 -- 172.21.15.1:0/373426841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7db357820 msgr2=0x55c7db35f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.757+0000 7f0bb9600640 1 --2- 172.21.15.1:0/373426841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7db357820 0x55c7db35f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:49.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.757+0000 7f2a6de00700 1 --2- 172.21.15.1:0/918812264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a70102a90 0x7f2a7019b350 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000eb90 tx=0x7f2a6000eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.757+0000 7f0bba000640 1 --2- 172.21.15.1:0/373426841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7db357820 0x55c7db35f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.757+0000 7f0bba000640 1 -- 172.21.15.1:0/373426841 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7db09c620 con 0x55c7db362a00 2024-04-26T02:00:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.757+0000 7f0bbedf3740 1 -- 172.21.15.1:0/373426841 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ba00dd0f0 con 0x55c7db362a00 2024-04-26T02:00:49.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.757+0000 7f2a66a00700 1 -- 172.21.15.1:0/918812264 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6000cc90 con 0x7f2a70102a90 2024-04-26T02:00:49.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.758+0000 7f2a66a00700 1 -- 172.21.15.1:0/918812264 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a6000ce30 con 0x7f2a70102a90 2024-04-26T02:00:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.758+0000 7f0bba000640 1 --2- 172.21.15.1:0/373426841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 0x55c7db3603e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80216f0 tx=0x7f0ba8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79ad7b22f22a462c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.758+0000 7f0b9aa00640 10 client.26261 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:00:49.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.758+0000 7f2a66a00700 1 -- 172.21.15.1:0/918812264 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a60010430 con 0x7f2a70102a90 2024-04-26T02:00:49.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.758+0000 7f2a74e00700 1 -- 172.21.15.1:0/918812264 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a701a0650 con 0x7f2a70102a90 2024-04-26T02:00:49.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.758+0000 7f2a74e00700 1 -- 172.21.15.1:0/918812264 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a701a7ed0 con 0x7f2a70102a90 2024-04-26T02:00:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.758+0000 7f0b9aa00640 1 -- 172.21.15.1:0/373426841 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8010780 con 0x55c7db362a00 2024-04-26T02:00:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.758+0000 7f0b9aa00640 1 -- 172.21.15.1:0/373426841 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f0ba8010920 con 0x55c7db362a00 2024-04-26T02:00:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.758+0000 7f0b9aa00640 1 -- 172.21.15.1:0/373426841 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0ba8010b60 con 0x55c7db362a00 2024-04-26T02:00:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.759+0000 7f0bbedf3740 10 client.26261 fetch_fsmap learned FSMap version 32 2024-04-26T02:00:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.759+0000 7f0bbedf3740 1 -- 172.21.15.1:0/373426841 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55c7db3713a0 con 0x55c7db362a00 2024-04-26T02:00:49.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.759+0000 7f2a74e00700 1 -- 172.21.15.1:0/918812264 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a70104180 con 0x7f2a70102a90 2024-04-26T02:00:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.760+0000 7f0b9aa00640 1 -- 172.21.15.1:0/373426841 <== mon.2 v2:172.21.15.97:3300/0 4 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f0ba8028020 con 0x55c7db362a00 2024-04-26T02:00:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.760+0000 7f0bbedf3740 10 client.26261 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:00:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.760+0000 7f0bbedf3740 10 client.26261 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:00:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.760+0000 7f0bbedf3740 20 client.26261 populate_metadata read hostname 'smithi001' 2024-04-26T02:00:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.761+0000 7f0bbedf3740 1 --2- 172.21.15.1:0/373426841 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c7db372750 0x55c7db374b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.761+0000 7f0bbedf3740 4 client.26261 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:00:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.761+0000 7f0bbedf3740 1 -- 172.21.15.1:0/373426841 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7db2d18b0 con 0x55c7db372750 2024-04-26T02:00:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.761+0000 7f0bb9600640 1 --2- 172.21.15.1:0/373426841 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c7db372750 0x55c7db374b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:49.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.761+0000 7f2a66a00700 1 -- 172.21.15.1:0/918812264 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f2a60003ef0 con 0x7f2a70102a90 2024-04-26T02:00:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.763+0000 7f0bb9600640 1 --2- 172.21.15.1:0/373426841 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c7db372750 0x55c7db374b40 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c76c571c15f2a46 server_cookie=773b4b194e93846b in_seq=0 out_seq=0 2024-04-26T02:00:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.763+0000 7f0b9aa00640 10 client.26261 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.764+0000 7f0b9aa00640 1 -- 172.21.15.1:0/373426841 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c7db2d18b0 con 0x55c7db372750 2024-04-26T02:00:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.764+0000 7f0b9aa00640 10 client.26261 handle_command_reply: tid=0 2024-04-26T02:00:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.764+0000 7f0bbedf3740 1 client.26261 shutdown 2024-04-26T02:00:49.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.763+0000 7f2a66a00700 1 --2- 172.21.15.1:0/918812264 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2a540a3e30 0x7f2a540a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.764+0000 7f2a66a00700 1 -- 172.21.15.1:0/918812264 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f2a600d0a80 con 0x7f2a70102a90 2024-04-26T02:00:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.764+0000 7f0bbedf3740 1 -- 172.21.15.1:0/373426841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 msgr2=0x55c7db3603e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.764+0000 7f0bbedf3740 1 --2- 172.21.15.1:0/373426841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 0x55c7db3603e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80216f0 tx=0x7f0ba8005f90 comp rx=0 tx=0).stop 2024-04-26T02:00:49.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.764+0000 7f2a6d400700 1 --2- 172.21.15.1:0/918812264 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2a540a3e30 0x7f2a540a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.765+0000 7f0bbedf3740 1 -- 172.21.15.1:0/373426841 shutdown_connections 2024-04-26T02:00:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.765+0000 7f0bbedf3740 1 --2- 172.21.15.1:0/373426841 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c7db372750 0x55c7db374b40 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.765+0000 7f0bbedf3740 1 --2- 172.21.15.1:0/373426841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 0x55c7db3603e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.765+0000 7f0bbedf3740 1 --2- 172.21.15.1:0/373426841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 0x55c7db35fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.765+0000 7f0bbedf3740 1 --2- 172.21.15.1:0/373426841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7db357820 0x55c7db35f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.765+0000 7f0bbedf3740 1 -- 172.21.15.1:0/373426841 >> 172.21.15.1:0/373426841 conn(0x55c7db353d30 msgr2=0x55c7db359ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.765+0000 7f0b9aa00640 0 client.26261 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.765+0000 7f0bbedf3740 1 -- 172.21.15.1:0/373426841 shutdown_connections 2024-04-26T02:00:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.766+0000 7f0bbedf3740 1 -- 172.21.15.1:0/373426841 wait complete. 2024-04-26T02:00:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.766+0000 7f0bbedf3740 20 client.26261 trim_cache size 0 max 16384 2024-04-26T02:00:49.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.767+0000 7f2a6d400700 1 --2- 172.21.15.1:0/918812264 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2a540a3e30 0x7f2a540a62f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009fd0 tx=0x7f2a5c019040 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:49.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:49.768+0000 7f2a66a00700 1 -- 172.21.15.1:0/918812264 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a600d2e20 con 0x7f2a70102a90 2024-04-26T02:00:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.785+0000 7f0bbedf3740 1 Processor -- start 2024-04-26T02:00:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.785+0000 7f0bbedf3740 1 -- start start 2024-04-26T02:00:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.785+0000 7f0bbedf3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7db357820 0x55c7db35a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.785+0000 7f0bbedf3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 0x55c7db360200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.785+0000 7f0bbedf3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 0x55c7db37a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.785+0000 7f0bbedf3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb41bf4c0 con 0x55c7db357820 2024-04-26T02:00:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.785+0000 7f0bbedf3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb41bcd90 con 0x55c7db362a00 2024-04-26T02:00:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.785+0000 7f0bbedf3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb41bac00 con 0x55c7db358140 2024-04-26T02:00:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.786+0000 7f0bb9600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7db357820 0x55c7db35a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.786+0000 7f0bba000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 0x55c7db37a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.786+0000 7f0bb8c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 0x55c7db360200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.786+0000 7f0bb9600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7db357820 0x55c7db35a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57666/0 (socket says 172.21.15.1:57666) 2024-04-26T02:00:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.786+0000 7f0bb9600640 1 -- 172.21.15.1:0/3719553002 learned_addr learned my addr 172.21.15.1:0/3719553002 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.786+0000 7f0bba000640 1 -- 172.21.15.1:0/3719553002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 msgr2=0x55c7db360200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.786+0000 7f0bba000640 1 --2- 172.21.15.1:0/3719553002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 0x55c7db360200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.786+0000 7f0bba000640 1 -- 172.21.15.1:0/3719553002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7db357820 msgr2=0x55c7db35a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.786+0000 7f0bba000640 1 --2- 172.21.15.1:0/3719553002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7db357820 0x55c7db35a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.786+0000 7f0bba000640 1 -- 172.21.15.1:0/3719553002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7db375190 con 0x55c7db362a00 2024-04-26T02:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.786+0000 7f0bb8c00640 1 --2- 172.21.15.1:0/3719553002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 0x55c7db360200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.787+0000 7f0bba000640 1 --2- 172.21.15.1:0/3719553002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 0x55c7db37a420 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8005c20 tx=0x7f0ba8023040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e79a4949661d8b73 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.787+0000 7f0ba5c00640 1 -- 172.21.15.1:0/3719553002 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8010980 con 0x55c7db362a00 2024-04-26T02:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.787+0000 7f0ba5c00640 1 -- 172.21.15.1:0/3719553002 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ba00dd0f0 con 0x55c7db362a00 2024-04-26T02:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.787+0000 7f0ba5c00640 1 -- 172.21.15.1:0/3719553002 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba802a830 con 0x55c7db362a00 2024-04-26T02:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.788+0000 7f0bbedf3740 1 -- 172.21.15.1:0/3719553002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 msgr2=0x55c7db37a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.788+0000 7f0bbedf3740 1 --2- 172.21.15.1:0/3719553002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 0x55c7db37a420 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8005c20 tx=0x7f0ba8023040 comp rx=0 tx=0).stop 2024-04-26T02:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.788+0000 7f0bbedf3740 1 -- 172.21.15.1:0/3719553002 shutdown_connections 2024-04-26T02:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.788+0000 7f0bbedf3740 1 --2- 172.21.15.1:0/3719553002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 0x55c7db37a420 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.788+0000 7f0bbedf3740 1 --2- 172.21.15.1:0/3719553002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 0x55c7db360200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.788+0000 7f0bbedf3740 1 --2- 172.21.15.1:0/3719553002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7db357820 0x55c7db35a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.788+0000 7f0bbedf3740 1 -- 172.21.15.1:0/3719553002 >> 172.21.15.1:0/3719553002 conn(0x55c7db353d30 msgr2=0x55c7db352e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.788+0000 7f0bbedf3740 1 -- 172.21.15.1:0/3719553002 shutdown_connections 2024-04-26T02:00:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.788+0000 7f0bbedf3740 1 -- 172.21.15.1:0/3719553002 wait complete. 2024-04-26T02:00:49.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.789+0000 7f0bbedf3740 1 -- start start 2024-04-26T02:00:49.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.789+0000 7f0bbedf3740 1 Processor -- start 2024-04-26T02:00:49.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.789+0000 7f0bbedf3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7db357820 0x55c7db36a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.789+0000 7f0bbedf3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 0x55c7db367b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.789+0000 7f0bbedf3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 0x55c7db3680d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.790+0000 7f0bbedf3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb41bb250 con 0x55c7db357820 2024-04-26T02:00:49.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.790+0000 7f0bbedf3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb41bc6c0 con 0x55c7db362a00 2024-04-26T02:00:49.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.790+0000 7f0bbedf3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb41bb920 con 0x55c7db358140 2024-04-26T02:00:49.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.790+0000 7f0bbedf3740 10 client.0 _ll_register_callbacks cb 0x7f0ba8003ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:49.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.790+0000 7f0bb8c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 0x55c7db367b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:49.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.790+0000 7f0bb8c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 0x55c7db367b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:47944/0 (socket says 172.21.15.1:47944) 2024-04-26T02:00:49.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.790+0000 7f0bb8c00640 1 -- 172.21.15.1:0/351399710 learned_addr learned my addr 172.21.15.1:0/351399710 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.790+0000 7f0bba000640 1 --2- 172.21.15.1:0/351399710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 0x55c7db3680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.790+0000 7f0bb9600640 1 --2- 172.21.15.1:0/351399710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7db357820 0x55c7db36a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.790+0000 7f0bb8c00640 1 -- 172.21.15.1:0/351399710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 msgr2=0x55c7db3680d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.790+0000 7f0bb8c00640 1 --2- 172.21.15.1:0/351399710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 0x55c7db3680d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.790+0000 7f0bb8c00640 1 -- 172.21.15.1:0/351399710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7db357820 msgr2=0x55c7db36a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.790+0000 7f0bb8c00640 1 --2- 172.21.15.1:0/351399710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7db357820 0x55c7db36a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.790+0000 7f0bb8c00640 1 -- 172.21.15.1:0/351399710 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7db368740 con 0x55c7db358140 2024-04-26T02:00:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.791+0000 7f0bb8c00640 1 --2- 172.21.15.1:0/351399710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 0x55c7db367b90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0ba00bc830 tx=0x7f0ba002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dced39d141334ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.791+0000 7f0b9aa00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T02:00:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.791+0000 7f0b9aa00640 1 -- 172.21.15.1:0/351399710 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0049040 con 0x55c7db358140 2024-04-26T02:00:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.791+0000 7f0b9aa00640 1 -- 172.21.15.1:0/351399710 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f0ba00b91a0 con 0x55c7db358140 2024-04-26T02:00:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.791+0000 7f0bbedf3740 1 -- 172.21.15.1:0/351399710 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ba00dd0f0 con 0x55c7db358140 2024-04-26T02:00:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.791+0000 7f0b9aa00640 1 -- 172.21.15.1:0/351399710 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0ba00b8dd0 con 0x55c7db358140 2024-04-26T02:00:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.791+0000 7f0bbedf3740 10 client.26281 fetch_fsmap learned FSMap version 32 2024-04-26T02:00:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.791+0000 7f0bbedf3740 1 -- 172.21.15.1:0/351399710 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c7db3d8b40 con 0x55c7db358140 2024-04-26T02:00:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.791+0000 7f0b9aa00640 1 -- 172.21.15.1:0/351399710 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ba00dd0f0 con 0x55c7db358140 2024-04-26T02:00:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.792+0000 7f0b9aa00640 1 -- 172.21.15.1:0/351399710 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f0ba00ee4b0 con 0x55c7db358140 2024-04-26T02:00:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.792+0000 7f0bbedf3740 10 client.26281 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:00:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.792+0000 7f0bbedf3740 10 client.26281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:00:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.792+0000 7f0bbedf3740 20 client.26281 populate_metadata read hostname 'smithi001' 2024-04-26T02:00:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.792+0000 7f0bbedf3740 1 --2- 172.21.15.1:0/351399710 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c7db3d91c0 0x55c7db3db5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.792+0000 7f0bbedf3740 4 client.26281 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:00:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.792+0000 7f0bbedf3740 1 -- 172.21.15.1:0/351399710 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c7db2cb320 con 0x55c7db3d91c0 2024-04-26T02:00:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.792+0000 7f0bb9600640 1 --2- 172.21.15.1:0/351399710 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c7db3d91c0 0x55c7db3db5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.793+0000 7f0bb9600640 1 --2- 172.21.15.1:0/351399710 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c7db3d91c0 0x55c7db3db5e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62dd67f060318907 server_cookie=32d50ad73622203c in_seq=0 out_seq=0 2024-04-26T02:00:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.793+0000 7f0b9aa00640 10 client.26281 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.810+0000 7f0b9aa00640 1 -- 172.21.15.1:0/351399710 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c7db2cb320 con 0x55c7db3d91c0 2024-04-26T02:00:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.810+0000 7f0b9aa00640 10 client.26281 handle_command_reply: tid=0 2024-04-26T02:00:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.810+0000 7f0bbedf3740 1 client.26281 shutdown 2024-04-26T02:00:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.811+0000 7f0bbedf3740 1 -- 172.21.15.1:0/351399710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 msgr2=0x55c7db367b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.811+0000 7f0bbedf3740 1 --2- 172.21.15.1:0/351399710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 0x55c7db367b90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0ba00bc830 tx=0x7f0ba002a410 comp rx=0 tx=0).stop 2024-04-26T02:00:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.812+0000 7f0bbedf3740 1 -- 172.21.15.1:0/351399710 shutdown_connections 2024-04-26T02:00:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.812+0000 7f0bbedf3740 1 --2- 172.21.15.1:0/351399710 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c7db3d91c0 0x55c7db3db5e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.812+0000 7f0bbedf3740 1 --2- 172.21.15.1:0/351399710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c7db362a00 0x55c7db3680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.812+0000 7f0bbedf3740 1 --2- 172.21.15.1:0/351399710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7db358140 0x55c7db367b90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.812+0000 7f0b9aa00640 0 client.26281 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.812+0000 7f0bbedf3740 1 --2- 172.21.15.1:0/351399710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7db357820 0x55c7db36a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.812+0000 7f0bbedf3740 1 -- 172.21.15.1:0/351399710 >> 172.21.15.1:0/351399710 conn(0x55c7db353d30 msgr2=0x55c7db361190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.813+0000 7f0bbedf3740 1 -- 172.21.15.1:0/351399710 shutdown_connections 2024-04-26T02:00:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.814+0000 7f0bbedf3740 1 -- 172.21.15.1:0/351399710 wait complete. 2024-04-26T02:00:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.815+0000 7f0bbedf3740 20 client.26281 trim_cache size 0 max 16384 2024-04-26T02:00:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T02:00:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T02:00:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T02:00:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T02:00:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.816+0000 7f0bbc000640 1 -- 172.21.15.1:0/3811047200 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0b9c0a3d90 msgr2=0x7f0b9c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.816+0000 7f0bbc000640 1 --2- 172.21.15.1:0/3811047200 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0b9c0a3d90 0x7f0b9c0a6250 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009710 tx=0x7f0ba80046c0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.816+0000 7f0bbc000640 1 -- 172.21.15.1:0/3811047200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bb41106f0 msgr2=0x7f0bb41a7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.816+0000 7f0bbc000640 1 --2- 172.21.15.1:0/3811047200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bb41106f0 0x7f0bb41a7a60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0005ab0 tx=0x7f0ba0003c10 comp rx=0 tx=0).stop 2024-04-26T02:00:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.819+0000 7f0bbc000640 1 -- 172.21.15.1:0/3811047200 shutdown_connections 2024-04-26T02:00:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.819+0000 7f0bbc000640 1 --2- 172.21.15.1:0/3811047200 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0b9c0a3d90 0x7f0b9c0a6250 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.819+0000 7f0bbc000640 1 --2- 172.21.15.1:0/3811047200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bb411b730 0x7f0bb41abde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.819+0000 7f0bbc000640 1 --2- 172.21.15.1:0/3811047200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bb41106f0 0x7f0bb41a7a60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.819+0000 7f0bbc000640 1 --2- 172.21.15.1:0/3811047200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bb410e7d0 0x7f0bb41a7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.819+0000 7f0bbc000640 1 -- 172.21.15.1:0/3811047200 >> 172.21.15.1:0/3811047200 conn(0x7f0bb406d030 msgr2=0x7f0bb4072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.819+0000 7f0bbc000640 1 -- 172.21.15.1:0/3811047200 shutdown_connections 2024-04-26T02:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:00:49.819+0000 7f0bbc000640 1 -- 172.21.15.1:0/3811047200 wait complete. 2024-04-26T02:00:50.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:50.099+0000 7f2a74e00700 1 -- 172.21.15.1:0/918812264 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a7002df80 con 0x7f2a70102a90 2024-04-26T02:00:50.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:50.100+0000 7f2a66a00700 1 -- 172.21.15.1:0/918812264 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11838 (secure 0 0 0) 0x7f2a6009d090 con 0x7f2a70102a90 2024-04-26T02:00:50.099 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T02:00:50.099 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:55:49.645549+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25414":{"gid":25414,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/453427322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":453427322},{"type":"v1","addr":"172.21.15.173:6835","nonce":453427322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24499,24514,24518]},"id":1}]} 2024-04-26T02:00:50.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:50.103+0000 7f2a74e00700 1 -- 172.21.15.1:0/918812264 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2a540a3e30 msgr2=0x7f2a540a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:50.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:50.103+0000 7f2a74e00700 1 --2- 172.21.15.1:0/918812264 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2a540a3e30 0x7f2a540a62f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009fd0 tx=0x7f2a5c019040 comp rx=0 tx=0).stop 2024-04-26T02:00:50.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:50.103+0000 7f2a74e00700 1 -- 172.21.15.1:0/918812264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a70102a90 msgr2=0x7f2a7019b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:50.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:50.103+0000 7f2a74e00700 1 --2- 172.21.15.1:0/918812264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a70102a90 0x7f2a7019b350 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000eb90 tx=0x7f2a6000eea0 comp rx=0 tx=0).stop 2024-04-26T02:00:50.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:50.103+0000 7f2a74e00700 1 -- 172.21.15.1:0/918812264 shutdown_connections 2024-04-26T02:00:50.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:50.103+0000 7f2a74e00700 1 --2- 172.21.15.1:0/918812264 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2a540a3e30 0x7f2a540a62f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:50.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:50.103+0000 7f2a74e00700 1 --2- 172.21.15.1:0/918812264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a70108a50 0x7f2a7019fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:50.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:50.103+0000 7f2a74e00700 1 --2- 172.21.15.1:0/918812264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a70102a90 0x7f2a7019b350 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:50.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:50.103+0000 7f2a74e00700 1 --2- 172.21.15.1:0/918812264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a70103450 0x7f2a7019b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:50.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:50.103+0000 7f2a74e00700 1 -- 172.21.15.1:0/918812264 >> 172.21.15.1:0/918812264 conn(0x7f2a70079ba0 msgr2=0x7f2a7010e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:50.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:50.103+0000 7f2a74e00700 1 -- 172.21.15.1:0/918812264 shutdown_connections 2024-04-26T02:00:50.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:50.103+0000 7f2a74e00700 1 -- 172.21.15.1:0/918812264 wait complete. 2024-04-26T02:00:50.103 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 32 2024-04-26T02:00:50.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:50 smithi173 ceph-mon[32553]: pgmap v285: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 906 KiB/s wr, 305 op/s 2024-04-26T02:00:50.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:50 smithi001 ceph-mon[27822]: pgmap v285: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 906 KiB/s wr, 305 op/s 2024-04-26T02:00:50.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:50 smithi097 ceph-mon[32853]: pgmap v285: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 906 KiB/s wr, 305 op/s 2024-04-26T02:00:50.983 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24518 quiesce db / --timeout 60 --expiration 82.1 --await-for 42.0' 2024-04-26T02:00:50.983 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24518 quiesce db / --timeout 60 --expiration 82.1 --await-for 42.0 2024-04-26T02:00:51.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:51 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/918812264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:00:51.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:51 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/918812264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:00:51.483 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:00:51.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:51 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/918812264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:00:52.170 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:52 smithi001 ceph-mon[27822]: pgmap v286: 97 pgs: 97 active+clean; 105 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.4 MiB/s wr, 461 op/s 2024-04-26T02:00:52.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:52 smithi173 ceph-mon[32553]: pgmap v286: 97 pgs: 97 active+clean; 105 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.4 MiB/s wr, 461 op/s 2024-04-26T02:00:52.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:52 smithi097 ceph-mon[32853]: pgmap v286: 97 pgs: 97 active+clean; 105 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.4 MiB/s wr, 461 op/s 2024-04-26T02:00:54.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.099+0000 7f6014a00700 1 -- 172.21.15.1:0/550036377 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6010109fb0 msgr2=0x7f6010111b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:54.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.099+0000 7f6014a00700 1 --2- 172.21.15.1:0/550036377 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6010109fb0 0x7f6010111b80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f600800b0d0 tx=0x7f600800b3e0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.099+0000 7f6014a00700 1 -- 172.21.15.1:0/550036377 shutdown_connections 2024-04-26T02:00:54.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.099+0000 7f6014a00700 1 --2- 172.21.15.1:0/550036377 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6010109fb0 0x7f6010111b80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.099+0000 7f6014a00700 1 --2- 172.21.15.1:0/550036377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6010104ed0 0x7f60101052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.099+0000 7f6014a00700 1 --2- 172.21.15.1:0/550036377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010105880 0x7f60101098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.099+0000 7f6014a00700 1 -- 172.21.15.1:0/550036377 >> 172.21.15.1:0/550036377 conn(0x7f6010100740 msgr2=0x7f6010102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:54.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.099+0000 7f6014a00700 1 -- 172.21.15.1:0/550036377 shutdown_connections 2024-04-26T02:00:54.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.099+0000 7f6014a00700 1 -- 172.21.15.1:0/550036377 wait complete. 2024-04-26T02:00:54.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.100+0000 7f6014a00700 1 Processor -- start 2024-04-26T02:00:54.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.100+0000 7f6014a00700 1 -- start start 2024-04-26T02:00:54.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.101+0000 7f6014a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010104ed0 0x7f601019b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:54.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.101+0000 7f6014a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6010105880 0x7f601019b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:54.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.101+0000 7f6014a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6010109fb0 0x7f601019fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:54.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.101+0000 7f6014a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6010114e50 con 0x7f6010105880 2024-04-26T02:00:54.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.101+0000 7f6014a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6010114cd0 con 0x7f6010104ed0 2024-04-26T02:00:54.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.101+0000 7f6014a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6010114fd0 con 0x7f6010109fb0 2024-04-26T02:00:54.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.101+0000 7f600de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010104ed0 0x7f601019b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:54.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.101+0000 7f600de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010104ed0 0x7f601019b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:50606/0 (socket says 172.21.15.1:50606) 2024-04-26T02:00:54.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.101+0000 7f600de00700 1 -- 172.21.15.1:0/4262656136 learned_addr learned my addr 172.21.15.1:0/4262656136 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:54.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.101+0000 7f600d400700 1 --2- 172.21.15.1:0/4262656136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6010105880 0x7f601019b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:54.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.101+0000 7f600e800700 1 --2- 172.21.15.1:0/4262656136 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6010109fb0 0x7f601019fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:54.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.102+0000 7f600de00700 1 -- 172.21.15.1:0/4262656136 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6010109fb0 msgr2=0x7f601019fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:54.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.102+0000 7f600de00700 1 --2- 172.21.15.1:0/4262656136 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6010109fb0 0x7f601019fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.102+0000 7f600de00700 1 -- 172.21.15.1:0/4262656136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6010105880 msgr2=0x7f601019b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:54.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.102+0000 7f600de00700 1 --2- 172.21.15.1:0/4262656136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6010105880 0x7f601019b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.102+0000 7f600de00700 1 -- 172.21.15.1:0/4262656136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60101a01d0 con 0x7f6010104ed0 2024-04-26T02:00:54.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.102+0000 7f600de00700 1 --2- 172.21.15.1:0/4262656136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010104ed0 0x7f601019b1c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f600000e9a0 tx=0x7f600000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:54.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.102+0000 7f6006a00700 1 -- 172.21.15.1:0/4262656136 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600000cbd0 con 0x7f6010104ed0 2024-04-26T02:00:54.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.103+0000 7f6006a00700 1 -- 172.21.15.1:0/4262656136 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f600000cd70 con 0x7f6010104ed0 2024-04-26T02:00:54.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.103+0000 7f6014a00700 1 -- 172.21.15.1:0/4262656136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60101a04c0 con 0x7f6010104ed0 2024-04-26T02:00:54.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.103+0000 7f6006a00700 1 -- 172.21.15.1:0/4262656136 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000018680 con 0x7f6010104ed0 2024-04-26T02:00:54.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.103+0000 7f6014a00700 1 -- 172.21.15.1:0/4262656136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60101a7d40 con 0x7f6010104ed0 2024-04-26T02:00:54.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.107+0000 7f6006a00700 1 -- 172.21.15.1:0/4262656136 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f6000010430 con 0x7f6010104ed0 2024-04-26T02:00:54.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.108+0000 7f601f37eb80 1 Processor -- start 2024-04-26T02:00:54.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.108+0000 7f601f37eb80 1 -- start start 2024-04-26T02:00:54.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.108+0000 7f601f37eb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e2bfae39b0 0x55e2bfae3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:54.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.108+0000 7f601f37eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfae42d0 0x55e2bfaeec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:54.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.108+0000 7f601f37eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfaef180 0x55e2bfaf1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:54.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.108+0000 7f601f37eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e2bfa37020 con 0x55e2bfaef180 2024-04-26T02:00:54.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.108+0000 7f601f37eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e2bf9dd8c0 con 0x55e2bfae42d0 2024-04-26T02:00:54.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.108+0000 7f601f37eb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e2bfa32fe0 con 0x55e2bfae39b0 2024-04-26T02:00:54.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.109+0000 7f600d400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfaef180 0x55e2bfaf1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:54.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.109+0000 7f600de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfae42d0 0x55e2bfaeec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:54.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.109+0000 7f600de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfae42d0 0x55e2bfaeec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:50612/0 (socket says 172.21.15.1:50612) 2024-04-26T02:00:54.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.109+0000 7f600de00700 1 -- 172.21.15.1:0/3358116616 learned_addr learned my addr 172.21.15.1:0/3358116616 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:54.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.109+0000 7f600de00700 1 -- 172.21.15.1:0/3358116616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e2bfae39b0 msgr2=0x55e2bfae3d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T02:00:54.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.109+0000 7f600de00700 1 --2- 172.21.15.1:0/3358116616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e2bfae39b0 0x55e2bfae3d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.109+0000 7f600de00700 1 -- 172.21.15.1:0/3358116616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfaef180 msgr2=0x55e2bfaf1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:54.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.109+0000 7f600de00700 1 --2- 172.21.15.1:0/3358116616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfaef180 0x55e2bfaf1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.109+0000 7f600de00700 1 -- 172.21.15.1:0/3358116616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2bf9d3810 con 0x55e2bfae42d0 2024-04-26T02:00:54.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.110+0000 7f600de00700 1 --2- 172.21.15.1:0/3358116616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfae42d0 0x55e2bfaeec40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f60000ce0c0 tx=0x7f60000d4f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dcace6b64431af9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:54.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.110+0000 7f6006a00700 1 --2- 172.21.15.1:0/4262656136 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f5ff40a3e50 0x7f5ff40a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:54.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.110+0000 7f5ffb400700 1 -- 172.21.15.1:0/3358116616 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000098c70 con 0x55e2bfae42d0 2024-04-26T02:00:54.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.110+0000 7f600e800700 1 --2- 172.21.15.1:0/4262656136 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f5ff40a3e50 0x7f5ff40a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:54.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.110+0000 7f5ffb400700 1 -- 172.21.15.1:0/3358116616 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f600009d0d0 con 0x55e2bfae42d0 2024-04-26T02:00:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.110+0000 7f6006a00700 1 -- 172.21.15.1:0/4262656136 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f60000d0ac0 con 0x7f6010104ed0 2024-04-26T02:00:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.110+0000 7f5ffb400700 1 -- 172.21.15.1:0/3358116616 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600009be20 con 0x55e2bfae42d0 2024-04-26T02:00:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.111+0000 7f601f37eb80 1 -- 172.21.15.1:0/3358116616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfae42d0 msgr2=0x55e2bfaeec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.111+0000 7f601f37eb80 1 --2- 172.21.15.1:0/3358116616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfae42d0 0x55e2bfaeec40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f60000ce0c0 tx=0x7f60000d4f10 comp rx=0 tx=0).stop 2024-04-26T02:00:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.111+0000 7f600e800700 1 --2- 172.21.15.1:0/4262656136 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f5ff40a3e50 0x7f5ff40a6310 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6008009fd0 tx=0x7f600800b040 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.111+0000 7f601f37eb80 1 -- 172.21.15.1:0/3358116616 shutdown_connections 2024-04-26T02:00:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.111+0000 7f601f37eb80 1 --2- 172.21.15.1:0/3358116616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfaef180 0x55e2bfaf1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.111+0000 7f601f37eb80 1 --2- 172.21.15.1:0/3358116616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e2bfae39b0 0x55e2bfae3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.111+0000 7f601f37eb80 1 --2- 172.21.15.1:0/3358116616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfae42d0 0x55e2bfaeec40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.111+0000 7f601f37eb80 1 -- 172.21.15.1:0/3358116616 >> 172.21.15.1:0/3358116616 conn(0x55e2bfa12730 msgr2=0x55e2bfadfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.111+0000 7f601f37eb80 1 -- 172.21.15.1:0/3358116616 shutdown_connections 2024-04-26T02:00:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.111+0000 7f601f37eb80 1 -- 172.21.15.1:0/3358116616 wait complete. 2024-04-26T02:00:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.112+0000 7f601f37eb80 1 -- start start 2024-04-26T02:00:54.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.112+0000 7f601f37eb80 1 Processor -- start 2024-04-26T02:00:54.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.113+0000 7f601f37eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfae39b0 0x55e2bfaee850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:54.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.113+0000 7f601f37eb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e2bfae42d0 0x55e2bfaebbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:54.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.113+0000 7f601f37eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfaef180 0x55e2bfaec100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:54.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.113+0000 7f601f37eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e2bfaf1ca0 con 0x55e2bfae39b0 2024-04-26T02:00:54.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.113+0000 7f601f37eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e2bf9f09f0 con 0x55e2bfaef180 2024-04-26T02:00:54.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.113+0000 7f601f37eb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e2bfaf1e20 con 0x55e2bfae42d0 2024-04-26T02:00:54.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.113+0000 7f601f37eb80 10 client.0 _ll_register_callbacks cb 0x55e2bf9cf8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:54.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.113+0000 7f600de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfae39b0 0x55e2bfaee850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:54.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.113+0000 7f600e800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfaef180 0x55e2bfaec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:54.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.113+0000 7f600de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfae39b0 0x55e2bfaee850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40742/0 (socket says 172.21.15.1:40742) 2024-04-26T02:00:54.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.113+0000 7f600de00700 1 -- 172.21.15.1:0/1470707707 learned_addr learned my addr 172.21.15.1:0/1470707707 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:54.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.113+0000 7f600e800700 1 --2- 172.21.15.1:0/1470707707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfaef180 0x55e2bfaec100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:50628/0 (socket says 172.21.15.1:50628) 2024-04-26T02:00:54.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.113+0000 7f600d400700 1 --2- 172.21.15.1:0/1470707707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e2bfae42d0 0x55e2bfaebbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:54.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.113+0000 7f600e800700 1 -- 172.21.15.1:0/1470707707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e2bfae42d0 msgr2=0x55e2bfaebbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:54.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.113+0000 7f600e800700 1 --2- 172.21.15.1:0/1470707707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e2bfae42d0 0x55e2bfaebbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.114+0000 7f600e800700 1 -- 172.21.15.1:0/1470707707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfae39b0 msgr2=0x55e2bfaee850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:54.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.114+0000 7f600e800700 1 --2- 172.21.15.1:0/1470707707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfae39b0 0x55e2bfaee850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.114+0000 7f600e800700 1 -- 172.21.15.1:0/1470707707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2bf984a60 con 0x55e2bfaef180 2024-04-26T02:00:54.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.114+0000 7f600e800700 1 --2- 172.21.15.1:0/1470707707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfaef180 0x55e2bfaec100 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f600800b0d0 tx=0x7f600800b3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=888122c50a213382 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:54.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.114+0000 7f5fefe00700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:00:54.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.114+0000 7f5fefe00700 1 -- 172.21.15.1:0/1470707707 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600802c070 con 0x55e2bfaef180 2024-04-26T02:00:54.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.115+0000 7f5fefe00700 1 -- 172.21.15.1:0/1470707707 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f6008007d50 con 0x55e2bfaef180 2024-04-26T02:00:54.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.115+0000 7f601f37eb80 1 -- 172.21.15.1:0/1470707707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f600009d0d0 con 0x55e2bfaef180 2024-04-26T02:00:54.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.115+0000 7f5fefe00700 1 -- 172.21.15.1:0/1470707707 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6008022070 con 0x55e2bfaef180 2024-04-26T02:00:54.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.115+0000 7f601f37eb80 10 client.26279 fetch_fsmap learned FSMap version 32 2024-04-26T02:00:54.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.115+0000 7f601f37eb80 1 -- 172.21.15.1:0/1470707707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e2bf9cdc80 con 0x55e2bfaef180 2024-04-26T02:00:54.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.115+0000 7f5fefe00700 1 -- 172.21.15.1:0/1470707707 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f600009d0d0 con 0x55e2bfaef180 2024-04-26T02:00:54.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.115+0000 7f5fefe00700 1 -- 172.21.15.1:0/1470707707 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f6008011070 con 0x55e2bfaef180 2024-04-26T02:00:54.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.116+0000 7f601f37eb80 10 client.26279 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:00:54.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.116+0000 7f601f37eb80 10 client.26279 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T02:00:54.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.116+0000 7f601f37eb80 20 client.26279 populate_metadata read hostname 'smithi001' 2024-04-26T02:00:54.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.116+0000 7f601f37eb80 1 --2- 172.21.15.1:0/1470707707 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e2bfafe960 0x55e2bfb00d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:54.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.116+0000 7f601f37eb80 4 client.26279 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:00:54.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.116+0000 7f601f37eb80 1 -- 172.21.15.1:0/1470707707 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2bfa44810 con 0x55e2bfafe960 2024-04-26T02:00:54.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.116+0000 7f600de00700 1 --2- 172.21.15.1:0/1470707707 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e2bfafe960 0x55e2bfb00d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:54.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.116+0000 7f600de00700 1 --2- 172.21.15.1:0/1470707707 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e2bfafe960 0x55e2bfb00d80 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7629d0f38e3c267 server_cookie=8b75b19e09eb8c44 in_seq=0 out_seq=0 2024-04-26T02:00:54.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.116+0000 7f5fefe00700 10 client.26279 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:54.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.117+0000 7f5fefe00700 1 -- 172.21.15.1:0/1470707707 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e2bfa44810 con 0x55e2bfafe960 2024-04-26T02:00:54.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.117+0000 7f5fefe00700 10 client.26279 handle_command_reply: tid=0 2024-04-26T02:00:54.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.117+0000 7f601f37eb80 1 client.26279 shutdown 2024-04-26T02:00:54.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.117+0000 7f601f37eb80 1 -- 172.21.15.1:0/1470707707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfaef180 msgr2=0x55e2bfaec100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:54.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.118+0000 7f601f37eb80 1 --2- 172.21.15.1:0/1470707707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfaef180 0x55e2bfaec100 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f600800b0d0 tx=0x7f600800b3e0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.118+0000 7f601f37eb80 1 -- 172.21.15.1:0/1470707707 shutdown_connections 2024-04-26T02:00:54.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.118+0000 7f601f37eb80 1 --2- 172.21.15.1:0/1470707707 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e2bfafe960 0x55e2bfb00d80 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.118+0000 7f601f37eb80 1 --2- 172.21.15.1:0/1470707707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfaef180 0x55e2bfaec100 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.118+0000 7f601f37eb80 1 --2- 172.21.15.1:0/1470707707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfae39b0 0x55e2bfaee850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.118+0000 7f601f37eb80 1 --2- 172.21.15.1:0/1470707707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e2bfae42d0 0x55e2bfaebbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.118+0000 7f601f37eb80 1 -- 172.21.15.1:0/1470707707 >> 172.21.15.1:0/1470707707 conn(0x55e2bfa12730 msgr2=0x55e2bfadfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:54.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.118+0000 7f5fefe00700 0 client.26279 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:54.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.118+0000 7f601f37eb80 1 -- 172.21.15.1:0/1470707707 shutdown_connections 2024-04-26T02:00:54.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.118+0000 7f601f37eb80 1 -- 172.21.15.1:0/1470707707 wait complete. 2024-04-26T02:00:54.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.118+0000 7f601f37eb80 20 client.26279 trim_cache size 0 max 16384 2024-04-26T02:00:54.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.142+0000 7f601f37eb80 1 Processor -- start 2024-04-26T02:00:54.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.142+0000 7f601f37eb80 1 -- start start 2024-04-26T02:00:54.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.143+0000 7f601f37eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfae39b0 0x55e2bfaec0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:54.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.143+0000 7f601f37eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfae42d0 0x55e2bfb06650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:54.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.143+0000 7f601f37eb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e2bfaef180 0x55e2bfb06b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:54.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.143+0000 7f601f37eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f60101b1db0 con 0x55e2bfae42d0 2024-04-26T02:00:54.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.143+0000 7f601f37eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f60101af640 con 0x55e2bfae39b0 2024-04-26T02:00:54.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.143+0000 7f601f37eb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f60101ad4b0 con 0x55e2bfaef180 2024-04-26T02:00:54.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.143+0000 7f600de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfae39b0 0x55e2bfaec0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:54.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.143+0000 7f600de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfae39b0 0x55e2bfaec0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:50634/0 (socket says 172.21.15.1:50634) 2024-04-26T02:00:54.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.143+0000 7f600de00700 1 -- 172.21.15.1:0/4251124992 learned_addr learned my addr 172.21.15.1:0/4251124992 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:54.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.143+0000 7f600e800700 1 --2- 172.21.15.1:0/4251124992 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e2bfaef180 0x55e2bfb06b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:54.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.143+0000 7f600d400700 1 --2- 172.21.15.1:0/4251124992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfae42d0 0x55e2bfb06650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:54.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.144+0000 7f600de00700 1 -- 172.21.15.1:0/4251124992 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e2bfaef180 msgr2=0x55e2bfb06b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:54.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.144+0000 7f600de00700 1 --2- 172.21.15.1:0/4251124992 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e2bfaef180 0x55e2bfb06b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.144+0000 7f600de00700 1 -- 172.21.15.1:0/4251124992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfae42d0 msgr2=0x55e2bfb06650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:54.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.144+0000 7f600de00700 1 --2- 172.21.15.1:0/4251124992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfae42d0 0x55e2bfb06650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.144+0000 7f600de00700 1 -- 172.21.15.1:0/4251124992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2bfaea130 con 0x55e2bfae39b0 2024-04-26T02:00:54.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.144+0000 7f600d400700 1 --2- 172.21.15.1:0/4251124992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfae42d0 0x55e2bfb06650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:00:54.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.144+0000 7f600de00700 1 --2- 172.21.15.1:0/4251124992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfae39b0 0x55e2bfaec0b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f600009c6d0 tx=0x7f60000d4570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a7248219ff3a334 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:54.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.144+0000 7f5ffaa00700 1 -- 172.21.15.1:0/4251124992 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600009c150 con 0x55e2bfae39b0 2024-04-26T02:00:54.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.144+0000 7f5ffaa00700 1 -- 172.21.15.1:0/4251124992 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f600009d0d0 con 0x55e2bfae39b0 2024-04-26T02:00:54.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.144+0000 7f5ffaa00700 1 -- 172.21.15.1:0/4251124992 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600009c330 con 0x55e2bfae39b0 2024-04-26T02:00:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.145+0000 7f601f37eb80 1 -- 172.21.15.1:0/4251124992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfae39b0 msgr2=0x55e2bfaec0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.145+0000 7f601f37eb80 1 --2- 172.21.15.1:0/4251124992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfae39b0 0x55e2bfaec0b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f600009c6d0 tx=0x7f60000d4570 comp rx=0 tx=0).stop 2024-04-26T02:00:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.145+0000 7f601f37eb80 1 -- 172.21.15.1:0/4251124992 shutdown_connections 2024-04-26T02:00:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.145+0000 7f601f37eb80 1 --2- 172.21.15.1:0/4251124992 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e2bfaef180 0x55e2bfb06b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.145+0000 7f601f37eb80 1 --2- 172.21.15.1:0/4251124992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfae39b0 0x55e2bfaec0b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.145+0000 7f601f37eb80 1 --2- 172.21.15.1:0/4251124992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfae42d0 0x55e2bfb06650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.145+0000 7f601f37eb80 1 -- 172.21.15.1:0/4251124992 >> 172.21.15.1:0/4251124992 conn(0x55e2bfa12730 msgr2=0x55e2bfaeb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.145+0000 7f601f37eb80 1 -- 172.21.15.1:0/4251124992 shutdown_connections 2024-04-26T02:00:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.145+0000 7f601f37eb80 1 -- 172.21.15.1:0/4251124992 wait complete. 2024-04-26T02:00:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.146+0000 7f601f37eb80 1 -- start start 2024-04-26T02:00:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.146+0000 7f601f37eb80 1 Processor -- start 2024-04-26T02:00:54.146 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:54 smithi001 ceph-mon[27822]: pgmap v287: 97 pgs: 97 active+clean; 105 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 976 KiB/s wr, 305 op/s 2024-04-26T02:00:54.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.146+0000 7f601f37eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfae39b0 0x55e2bfaf40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:54.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.147+0000 7f601f37eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfae42d0 0x55e2bfaf45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:54.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.147+0000 7f601f37eb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e2bfaef180 0x55e2bfaf4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:54.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.147+0000 7f601f37eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f60101adb00 con 0x55e2bfae42d0 2024-04-26T02:00:54.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.147+0000 7f601f37eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f60101aef70 con 0x55e2bfae39b0 2024-04-26T02:00:54.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.147+0000 7f601f37eb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f60101ae1d0 con 0x55e2bfaef180 2024-04-26T02:00:54.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.147+0000 7f601f37eb80 10 client.0 _ll_register_callbacks cb 0x55e2bf9cf8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:54.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.147+0000 7f600de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfae39b0 0x55e2bfaf40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:54.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.147+0000 7f600de00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfae39b0 0x55e2bfaf40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:50646/0 (socket says 172.21.15.1:50646) 2024-04-26T02:00:54.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.147+0000 7f600de00700 1 -- 172.21.15.1:0/608528719 learned_addr learned my addr 172.21.15.1:0/608528719 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:00:54.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.147+0000 7f600d400700 1 --2- 172.21.15.1:0/608528719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfae42d0 0x55e2bfaf45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:54.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.147+0000 7f600e800700 1 --2- 172.21.15.1:0/608528719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e2bfaef180 0x55e2bfaf4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:54.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.147+0000 7f600de00700 1 -- 172.21.15.1:0/608528719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e2bfaef180 msgr2=0x55e2bfaf4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:54.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.147+0000 7f600de00700 1 --2- 172.21.15.1:0/608528719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e2bfaef180 0x55e2bfaf4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.147+0000 7f600de00700 1 -- 172.21.15.1:0/608528719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfae42d0 msgr2=0x55e2bfaf45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:54.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.147+0000 7f600de00700 1 --2- 172.21.15.1:0/608528719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfae42d0 0x55e2bfaf45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:54.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.147+0000 7f600de00700 1 -- 172.21.15.1:0/608528719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2bfb6d6e0 con 0x55e2bfae39b0 2024-04-26T02:00:54.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.148+0000 7f600e800700 1 --2- 172.21.15.1:0/608528719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e2bfaef180 0x55e2bfaf4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:00:54.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.148+0000 7f600de00700 1 --2- 172.21.15.1:0/608528719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfae39b0 0x55e2bfaf40b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6000000c00 tx=0x7f600000efd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef5cdbd43558cef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:54.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.148+0000 7f5fefe00700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:00:54.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.148+0000 7f5fefe00700 1 -- 172.21.15.1:0/608528719 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600009c510 con 0x55e2bfae39b0 2024-04-26T02:00:54.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.148+0000 7f5fefe00700 1 -- 172.21.15.1:0/608528719 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f6000099040 con 0x55e2bfae39b0 2024-04-26T02:00:54.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.148+0000 7f601f37eb80 1 -- 172.21.15.1:0/608528719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f600009d0d0 con 0x55e2bfae39b0 2024-04-26T02:00:54.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.148+0000 7f5fefe00700 1 -- 172.21.15.1:0/608528719 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f600009c7a0 con 0x55e2bfae39b0 2024-04-26T02:00:54.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.148+0000 7f601f37eb80 10 client.26285 fetch_fsmap learned FSMap version 32 2024-04-26T02:00:54.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.148+0000 7f601f37eb80 1 -- 172.21.15.1:0/608528719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e2bfb6e830 con 0x55e2bfae39b0 2024-04-26T02:00:54.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.149+0000 7f5fefe00700 1 -- 172.21.15.1:0/608528719 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f600009d0d0 con 0x55e2bfae39b0 2024-04-26T02:00:54.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.149+0000 7f5fefe00700 1 -- 172.21.15.1:0/608528719 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f60000ce400 con 0x55e2bfae39b0 2024-04-26T02:00:54.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.149+0000 7f601f37eb80 10 client.26285 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:00:54.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.149+0000 7f601f37eb80 10 client.26285 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T02:00:54.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.149+0000 7f601f37eb80 20 client.26285 populate_metadata read hostname 'smithi001' 2024-04-26T02:00:54.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.149+0000 7f601f37eb80 1 --2- 172.21.15.1:0/608528719 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e2bfb6eab0 0x55e2bfb70ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:54.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.149+0000 7f601f37eb80 4 client.26285 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 82.1, "await_for": 42.0}] 2024-04-26T02:00:54.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.149+0000 7f601f37eb80 1 -- 172.21.15.1:0/608528719 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 82.1, "await_for": 42.0}) -- 0x55e2bfa3e8b0 con 0x55e2bfb6eab0 2024-04-26T02:00:54.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.149+0000 7f600d400700 1 --2- 172.21.15.1:0/608528719 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e2bfb6eab0 0x55e2bfb70ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:54.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.150+0000 7f600d400700 1 --2- 172.21.15.1:0/608528719 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e2bfb6eab0 0x55e2bfb70ea0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8e4aa46b53efc7e server_cookie=bdeef7b02fd5ba89 in_seq=0 out_seq=0 2024-04-26T02:00:54.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:54.150+0000 7f5fefe00700 10 client.26285 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:54.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:54 smithi173 ceph-mon[32553]: pgmap v287: 97 pgs: 97 active+clean; 105 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 976 KiB/s wr, 305 op/s 2024-04-26T02:00:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:54 smithi097 ceph-mon[32853]: pgmap v287: 97 pgs: 97 active+clean; 105 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 976 KiB/s wr, 305 op/s 2024-04-26T02:00:55.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.077+0000 7f5fefe00700 1 -- 172.21.15.1:0/608528719 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55e2bfa3e8b0 con 0x55e2bfb6eab0 2024-04-26T02:00:55.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.077+0000 7f5fefe00700 10 client.26285 handle_command_reply: tid=0 2024-04-26T02:00:55.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.077+0000 7f601f37eb80 1 client.26285 shutdown 2024-04-26T02:00:55.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.078+0000 7f601f37eb80 1 -- 172.21.15.1:0/608528719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfae39b0 msgr2=0x55e2bfaf40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:55.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.078+0000 7f601f37eb80 1 --2- 172.21.15.1:0/608528719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfae39b0 0x55e2bfaf40b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6000000c00 tx=0x7f600000efd0 comp rx=0 tx=0).stop 2024-04-26T02:00:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.079+0000 7f601f37eb80 1 -- 172.21.15.1:0/608528719 shutdown_connections 2024-04-26T02:00:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.079+0000 7f601f37eb80 1 --2- 172.21.15.1:0/608528719 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55e2bfb6eab0 0x55e2bfb70ea0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.079+0000 7f601f37eb80 1 --2- 172.21.15.1:0/608528719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e2bfaef180 0x55e2bfaf4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.079+0000 7f601f37eb80 1 --2- 172.21.15.1:0/608528719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e2bfae39b0 0x55e2bfaf40b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.079+0000 7f601f37eb80 1 --2- 172.21.15.1:0/608528719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2bfae42d0 0x55e2bfaf45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.079+0000 7f601f37eb80 1 -- 172.21.15.1:0/608528719 >> 172.21.15.1:0/608528719 conn(0x55e2bfa12730 msgr2=0x55e2bfaed3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.079+0000 7f5fefe00700 0 client.26285 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:00:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.079+0000 7f601f37eb80 1 -- 172.21.15.1:0/608528719 shutdown_connections 2024-04-26T02:00:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.079+0000 7f601f37eb80 1 -- 172.21.15.1:0/608528719 wait complete. 2024-04-26T02:00:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.079+0000 7f601f37eb80 20 client.26285 trim_cache size 0 max 16384 2024-04-26T02:00:55.079 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T02:00:55.079 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 32, 2024-04-26T02:00:55.079 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24518, 2024-04-26T02:00:55.080 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 26, 2024-04-26T02:00:55.080 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-04-26T02:00:55.080 INFO:teuthology.orchestra.run.smithi001.stdout: "7a5cdd8c": { 2024-04-26T02:00:55.080 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 26, 2024-04-26T02:00:55.080 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-04-26T02:00:55.080 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T02:00:55.080 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-04-26T02:00:55.080 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T02:00:55.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:00:55.080 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-04-26T02:00:55.080 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 82.1, 2024-04-26T02:00:55.080 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-04-26T02:00:55.080 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-04-26T02:00:55.080 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-04-26T02:00:55.080 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T02:00:55.081 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-04-26T02:00:55.081 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T02:00:55.081 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:00:55.081 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:00:55.081 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:00:55.081 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:00:55.081 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:00:55.081 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T02:00:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.081+0000 7f6014a00700 1 -- 172.21.15.1:0/4262656136 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f5ff40a3e50 msgr2=0x7f5ff40a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.081+0000 7f6014a00700 1 --2- 172.21.15.1:0/4262656136 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f5ff40a3e50 0x7f5ff40a6310 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6008009fd0 tx=0x7f600800b040 comp rx=0 tx=0).stop 2024-04-26T02:00:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.081+0000 7f6014a00700 1 -- 172.21.15.1:0/4262656136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010104ed0 msgr2=0x7f601019b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.081+0000 7f6014a00700 1 --2- 172.21.15.1:0/4262656136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010104ed0 0x7f601019b1c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f600000e9a0 tx=0x7f600000ecb0 comp rx=0 tx=0).stop 2024-04-26T02:00:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.081+0000 7f6014a00700 1 -- 172.21.15.1:0/4262656136 shutdown_connections 2024-04-26T02:00:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.081+0000 7f6014a00700 1 --2- 172.21.15.1:0/4262656136 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f5ff40a3e50 0x7f5ff40a6310 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.081+0000 7f6014a00700 1 --2- 172.21.15.1:0/4262656136 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6010109fb0 0x7f601019fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.081+0000 7f6014a00700 1 --2- 172.21.15.1:0/4262656136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6010104ed0 0x7f601019b1c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.081+0000 7f6014a00700 1 --2- 172.21.15.1:0/4262656136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6010105880 0x7f601019b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.081+0000 7f6014a00700 1 -- 172.21.15.1:0/4262656136 >> 172.21.15.1:0/4262656136 conn(0x7f6010100740 msgr2=0x7f6010101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.082+0000 7f6014a00700 1 -- 172.21.15.1:0/4262656136 shutdown_connections 2024-04-26T02:00:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:00:55.082+0000 7f6014a00700 1 -- 172.21.15.1:0/4262656136 wait complete. 2024-04-26T02:00:56.005 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7a5cdd8c', quiesce took 9.4 seconds. Will release after: 22.1 2024-04-26T02:00:56.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:56 smithi173 ceph-mon[32553]: pgmap v288: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.5 MiB/s wr, 461 op/s 2024-04-26T02:00:56.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:56 smithi001 ceph-mon[27822]: pgmap v288: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.5 MiB/s wr, 461 op/s 2024-04-26T02:00:56.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:56 smithi097 ceph-mon[32853]: pgmap v288: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.5 MiB/s wr, 461 op/s 2024-04-26T02:00:58.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:00:58 smithi173 ceph-mon[32553]: pgmap v289: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.1 MiB/s wr, 311 op/s 2024-04-26T02:00:58.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:00:58 smithi001 ceph-mon[27822]: pgmap v289: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.1 MiB/s wr, 311 op/s 2024-04-26T02:00:58.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:00:58 smithi097 ceph-mon[32853]: pgmap v289: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.1 MiB/s wr, 311 op/s 2024-04-26T02:01:00.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:00 smithi173 ceph-mon[32553]: pgmap v290: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.1 MiB/s wr, 311 op/s 2024-04-26T02:01:00.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:00 smithi001 ceph-mon[27822]: pgmap v290: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.1 MiB/s wr, 311 op/s 2024-04-26T02:01:00.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:00 smithi097 ceph-mon[32853]: pgmap v290: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.1 MiB/s wr, 311 op/s 2024-04-26T02:01:02.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:02 smithi173 ceph-mon[32553]: pgmap v291: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.1 MiB/s wr, 314 op/s 2024-04-26T02:01:02.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:02 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:01:02.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:02 smithi001 ceph-mon[27822]: pgmap v291: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.1 MiB/s wr, 314 op/s 2024-04-26T02:01:02.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:02 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:01:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:02 smithi097 ceph-mon[32853]: pgmap v291: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.1 MiB/s wr, 314 op/s 2024-04-26T02:01:02.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:02 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:01:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.970+0000 7feafa000640 1 -- 172.21.15.1:0/693082984 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec0318a0 con 0x7feafc0742f0 2024-04-26T02:01:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.971+0000 7feafbe00640 1 -- 172.21.15.1:0/693082984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feafc0742f0 msgr2=0x7feafc0746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.971+0000 7feafbe00640 1 --2- 172.21.15.1:0/693082984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feafc0742f0 0x7feafc0746d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7feaec009900 tx=0x7feaec02a480 comp rx=0 tx=0).stop 2024-04-26T02:01:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.971+0000 7feafbe00640 1 -- 172.21.15.1:0/693082984 shutdown_connections 2024-04-26T02:01:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.971+0000 7feafbe00640 1 --2- 172.21.15.1:0/693082984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feafc112d80 0x7feafc115170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.971+0000 7feafbe00640 1 --2- 172.21.15.1:0/693082984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feafc074c10 0x7feafc112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.971+0000 7feafbe00640 1 --2- 172.21.15.1:0/693082984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feafc0742f0 0x7feafc0746d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.971+0000 7feafbe00640 1 -- 172.21.15.1:0/693082984 >> 172.21.15.1:0/693082984 conn(0x7feafc0fd860 msgr2=0x7feafc0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.971+0000 7feafbe00640 1 -- 172.21.15.1:0/693082984 shutdown_connections 2024-04-26T02:01:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.971+0000 7feafbe00640 1 -- 172.21.15.1:0/693082984 wait complete. 2024-04-26T02:01:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.972+0000 7feafbe00640 1 Processor -- start 2024-04-26T02:01:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.972+0000 7feafbe00640 1 -- start start 2024-04-26T02:01:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.973+0000 7feafbe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feafc0742f0 0x7feafc1a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.973+0000 7feafbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feafc074c10 0x7feafc1a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.973+0000 7feafbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feafc112d80 0x7feafc19ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.973+0000 7feafbe00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feafc118620 con 0x7feafc0742f0 2024-04-26T02:01:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.973+0000 7feafbe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1184a0 con 0x7feafc112d80 2024-04-26T02:01:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.973+0000 7feafbe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1187a0 con 0x7feafc074c10 2024-04-26T02:01:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.973+0000 7feafaa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feafc0742f0 0x7feafc1a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.973+0000 7feafaa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feafc0742f0 0x7feafc1a4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50128/0 (socket says 172.21.15.1:50128) 2024-04-26T02:01:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.973+0000 7feaf3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feafc074c10 0x7feafc1a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.973+0000 7feafaa00640 1 -- 172.21.15.1:0/1883690707 learned_addr learned my addr 172.21.15.1:0/1883690707 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.973+0000 7feafb400640 1 --2- 172.21.15.1:0/1883690707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feafc112d80 0x7feafc19ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.974+0000 7feaf3e00640 1 -- 172.21.15.1:0/1883690707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feafc112d80 msgr2=0x7feafc19ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.974+0000 7feaf3e00640 1 --2- 172.21.15.1:0/1883690707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feafc112d80 0x7feafc19ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.974+0000 7feaf3e00640 1 -- 172.21.15.1:0/1883690707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feafc0742f0 msgr2=0x7feafc1a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.974+0000 7feaf3e00640 1 --2- 172.21.15.1:0/1883690707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feafc0742f0 0x7feafc1a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.974+0000 7feaf3e00640 1 -- 172.21.15.1:0/1883690707 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feafc19f180 con 0x7feafc074c10 2024-04-26T02:01:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.974+0000 7feaf3e00640 1 --2- 172.21.15.1:0/1883690707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feafc074c10 0x7feafc1a4d90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7feae400e9f0 tx=0x7feae400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.974+0000 7feaf3400640 1 -- 172.21.15.1:0/1883690707 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae400ccc0 con 0x7feafc074c10 2024-04-26T02:01:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.974+0000 7feaf3400640 1 -- 172.21.15.1:0/1883690707 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feae4005020 con 0x7feafc074c10 2024-04-26T02:01:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.974+0000 7feafbe00640 1 -- 172.21.15.1:0/1883690707 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feafc19f470 con 0x7feafc074c10 2024-04-26T02:01:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.974+0000 7feaf3400640 1 -- 172.21.15.1:0/1883690707 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae4004a00 con 0x7feafc074c10 2024-04-26T02:01:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.975+0000 7feafbe00640 1 -- 172.21.15.1:0/1883690707 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feafc1ab710 con 0x7feafc074c10 2024-04-26T02:01:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.978+0000 7feaf3400640 1 -- 172.21.15.1:0/1883690707 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7feae4004ba0 con 0x7feafc074c10 2024-04-26T02:01:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.978+0000 7feb04592740 1 Processor -- start 2024-04-26T02:01:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.978+0000 7feb04592740 1 -- start start 2024-04-26T02:01:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.978+0000 7feb04592740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6e5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.979+0000 7feb04592740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 0x55ac2d6f0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.979+0000 7feb04592740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6f0b90 0x55ac2d6f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.979+0000 7feb04592740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ac2d64b900 con 0x55ac2d6e62d0 2024-04-26T02:01:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.979+0000 7feb04592740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ac2d5f1ef0 con 0x55ac2d6f0b90 2024-04-26T02:01:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.979+0000 7feb04592740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ac2d6478c0 con 0x55ac2d6e59b0 2024-04-26T02:01:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.979+0000 7feaf3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6f0b90 0x55ac2d6f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.979+0000 7feafb400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6e5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.979+0000 7feafb400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6e5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:54724/0 (socket says 172.21.15.1:54724) 2024-04-26T02:01:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.979+0000 7feafb400640 1 -- 172.21.15.1:0/3802553794 learned_addr learned my addr 172.21.15.1:0/3802553794 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.979+0000 7feafaa00640 1 --2- 172.21.15.1:0/3802553794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 0x55ac2d6f0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.979+0000 7feafb400640 1 -- 172.21.15.1:0/3802553794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6f0b90 msgr2=0x55ac2d6f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.979+0000 7feafb400640 1 --2- 172.21.15.1:0/3802553794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6f0b90 0x55ac2d6f2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.980+0000 7feafb400640 1 -- 172.21.15.1:0/3802553794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 msgr2=0x55ac2d6f0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.980+0000 7feafb400640 1 --2- 172.21.15.1:0/3802553794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 0x55ac2d6f0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.980+0000 7feafb400640 1 -- 172.21.15.1:0/3802553794 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac2d35e210 con 0x55ac2d6e59b0 2024-04-26T02:01:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.980+0000 7feaf3e00640 1 --2- 172.21.15.1:0/3802553794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6f0b90 0x55ac2d6f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.980+0000 7feaf3400640 1 --2- 172.21.15.1:0/1883690707 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7feae00a3d00 0x7feae00a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.980+0000 7feafb400640 1 --2- 172.21.15.1:0/3802553794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6e5d90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009e30 tx=0x7feaf4010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12bbe52490923eeb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.980+0000 7feaf3400640 1 -- 172.21.15.1:0/1883690707 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7feae40d08d0 con 0x7feafc074c10 2024-04-26T02:01:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.980+0000 7feafaa00640 1 --2- 172.21.15.1:0/1883690707 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7feae00a3d00 0x7feae00a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.980+0000 7feaebe00640 1 -- 172.21.15.1:0/3802553794 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4016070 con 0x55ac2d6e59b0 2024-04-26T02:01:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.980+0000 7feaebe00640 1 -- 172.21.15.1:0/3802553794 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf4004d20 con 0x55ac2d6e59b0 2024-04-26T02:01:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.980+0000 7feaebe00640 1 -- 172.21.15.1:0/3802553794 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf40127c0 con 0x55ac2d6e59b0 2024-04-26T02:01:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.982+0000 7feafaa00640 1 --2- 172.21.15.1:0/1883690707 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7feae00a3d00 0x7feae00a61c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7feaec009230 tx=0x7feaec0058d0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.982+0000 7feb04592740 1 -- 172.21.15.1:0/3802553794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6e59b0 msgr2=0x55ac2d6e5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.982+0000 7feb04592740 1 --2- 172.21.15.1:0/3802553794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6e5d90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009e30 tx=0x7feaf4010740 comp rx=0 tx=0).stop 2024-04-26T02:01:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.982+0000 7feb04592740 1 -- 172.21.15.1:0/3802553794 shutdown_connections 2024-04-26T02:01:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.982+0000 7feb04592740 1 --2- 172.21.15.1:0/3802553794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6f0b90 0x55ac2d6f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.982+0000 7feb04592740 1 --2- 172.21.15.1:0/3802553794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 0x55ac2d6f0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.982+0000 7feb04592740 1 --2- 172.21.15.1:0/3802553794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6e5d90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.982+0000 7feb04592740 1 -- 172.21.15.1:0/3802553794 >> 172.21.15.1:0/3802553794 conn(0x55ac2d6e1ec0 msgr2=0x55ac2d6e22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.982+0000 7feb04592740 1 -- 172.21.15.1:0/3802553794 shutdown_connections 2024-04-26T02:01:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.982+0000 7feb04592740 1 -- 172.21.15.1:0/3802553794 wait complete. 2024-04-26T02:01:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.983+0000 7feb04592740 1 -- start start 2024-04-26T02:01:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.983+0000 7feb04592740 1 Processor -- start 2024-04-26T02:01:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.984+0000 7feb04592740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6edb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.984+0000 7feb04592740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 0x55ac2d6ee0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.984+0000 7feb04592740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6f0b90 0x55ac2d6ee610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.984+0000 7feb04592740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ac2d6f37c0 con 0x55ac2d6e62d0 2024-04-26T02:01:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.984+0000 7feb04592740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ac2d605310 con 0x55ac2d6e59b0 2024-04-26T02:01:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.984+0000 7feb04592740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ac2d6f3940 con 0x55ac2d6f0b90 2024-04-26T02:01:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.984+0000 7feb04592740 10 client.0 _ll_register_callbacks cb 0x55ac2d5d3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.984+0000 7feaf3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 0x55ac2d6ee0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.984+0000 7feaf3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 0x55ac2d6ee0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50148/0 (socket says 172.21.15.1:50148) 2024-04-26T02:01:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.984+0000 7feaf3e00640 1 -- 172.21.15.1:0/360774885 learned_addr learned my addr 172.21.15.1:0/360774885 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.984+0000 7feafaa00640 1 --2- 172.21.15.1:0/360774885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6edb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.984+0000 7feafb400640 1 --2- 172.21.15.1:0/360774885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6f0b90 0x55ac2d6ee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.985+0000 7feaf3e00640 1 -- 172.21.15.1:0/360774885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6f0b90 msgr2=0x55ac2d6ee610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.985+0000 7feaf3e00640 1 --2- 172.21.15.1:0/360774885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6f0b90 0x55ac2d6ee610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.985+0000 7feaf3e00640 1 -- 172.21.15.1:0/360774885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6e59b0 msgr2=0x55ac2d6edb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.985+0000 7feaf3e00640 1 --2- 172.21.15.1:0/360774885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6edb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.985+0000 7feaf3e00640 1 -- 172.21.15.1:0/360774885 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac2d42a620 con 0x55ac2d6e62d0 2024-04-26T02:01:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.985+0000 7feafaa00640 1 --2- 172.21.15.1:0/360774885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6edb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.985+0000 7feafb400640 1 --2- 172.21.15.1:0/360774885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6f0b90 0x55ac2d6ee610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:01:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.985+0000 7feaf3e00640 1 --2- 172.21.15.1:0/360774885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 0x55ac2d6ee0d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7feae40cdff0 tx=0x7feae40096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38e17ad1d342af8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.985+0000 7feae8c00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T02:01:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.985+0000 7feae8c00640 1 -- 172.21.15.1:0/360774885 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae401c030 con 0x55ac2d6e62d0 2024-04-26T02:01:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.985+0000 7feae8c00640 1 -- 172.21.15.1:0/360774885 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7feae40054f0 con 0x55ac2d6e62d0 2024-04-26T02:01:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.985+0000 7feb04592740 1 -- 172.21.15.1:0/360774885 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaf4004d20 con 0x55ac2d6e62d0 2024-04-26T02:01:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.986+0000 7feae8c00640 1 -- 172.21.15.1:0/360774885 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feae409c320 con 0x55ac2d6e62d0 2024-04-26T02:01:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.986+0000 7feb04592740 10 client.16131 fetch_fsmap learned FSMap version 32 2024-04-26T02:01:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.986+0000 7feb04592740 1 -- 172.21.15.1:0/360774885 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ac2d6ffae0 con 0x55ac2d6e62d0 2024-04-26T02:01:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.986+0000 7feae8c00640 1 -- 172.21.15.1:0/360774885 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf4004d20 con 0x55ac2d6e62d0 2024-04-26T02:01:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.986+0000 7feae8c00640 1 -- 172.21.15.1:0/360774885 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7feae4098e50 con 0x55ac2d6e62d0 2024-04-26T02:01:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.986+0000 7feb04592740 10 client.16131 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:01:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.986+0000 7feb04592740 10 client.16131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:01:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.986+0000 7feb04592740 20 client.16131 populate_metadata read hostname 'smithi001' 2024-04-26T02:01:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.986+0000 7feb04592740 1 --2- 172.21.15.1:0/360774885 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ac2d700fa0 0x55ac2d703390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.986+0000 7feb04592740 4 client.16131 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:01:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.986+0000 7feb04592740 1 -- 172.21.15.1:0/360774885 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac2d65f8b0 con 0x55ac2d700fa0 2024-04-26T02:01:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.987+0000 7feafaa00640 1 --2- 172.21.15.1:0/360774885 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ac2d700fa0 0x55ac2d703390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.987+0000 7feafaa00640 1 --2- 172.21.15.1:0/360774885 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ac2d700fa0 0x55ac2d703390 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e18a681f0555a29 server_cookie=9f8883e578f70006 in_seq=0 out_seq=0 2024-04-26T02:01:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.987+0000 7feae8c00640 10 client.16131 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.988+0000 7feae8c00640 1 -- 172.21.15.1:0/360774885 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ac2d65f8b0 con 0x55ac2d700fa0 2024-04-26T02:01:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.988+0000 7feae8c00640 10 client.16131 handle_command_reply: tid=0 2024-04-26T02:01:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.988+0000 7feb04592740 1 client.16131 shutdown 2024-04-26T02:01:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.988+0000 7feb04592740 1 -- 172.21.15.1:0/360774885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 msgr2=0x55ac2d6ee0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.988+0000 7feb04592740 1 --2- 172.21.15.1:0/360774885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 0x55ac2d6ee0d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7feae40cdff0 tx=0x7feae40096e0 comp rx=0 tx=0).stop 2024-04-26T02:01:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.988+0000 7feb04592740 1 -- 172.21.15.1:0/360774885 shutdown_connections 2024-04-26T02:01:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.988+0000 7feb04592740 1 --2- 172.21.15.1:0/360774885 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ac2d700fa0 0x55ac2d703390 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.988+0000 7feb04592740 1 --2- 172.21.15.1:0/360774885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6f0b90 0x55ac2d6ee610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.988+0000 7feb04592740 1 --2- 172.21.15.1:0/360774885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 0x55ac2d6ee0d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.988+0000 7feb04592740 1 --2- 172.21.15.1:0/360774885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6edb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.988+0000 7feb04592740 1 -- 172.21.15.1:0/360774885 >> 172.21.15.1:0/360774885 conn(0x55ac2d6e1ec0 msgr2=0x55ac2d6e7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.988+0000 7feae8c00640 0 client.16131 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.989+0000 7feb04592740 1 -- 172.21.15.1:0/360774885 shutdown_connections 2024-04-26T02:01:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.989+0000 7feb04592740 1 -- 172.21.15.1:0/360774885 wait complete. 2024-04-26T02:01:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:02.989+0000 7feb04592740 20 client.16131 trim_cache size 0 max 16384 2024-04-26T02:01:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.006+0000 7feb04592740 1 Processor -- start 2024-04-26T02:01:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.006+0000 7feb04592740 1 -- start start 2024-04-26T02:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.006+0000 7feb04592740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6fc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.006+0000 7feb04592740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 0x55ac2d6efad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.006+0000 7feb04592740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6f0b90 0x55ac2d708c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.006+0000 7feb04592740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1b6a20 con 0x55ac2d6e62d0 2024-04-26T02:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.006+0000 7feb04592740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1b42f0 con 0x55ac2d6f0b90 2024-04-26T02:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.006+0000 7feb04592740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1b2160 con 0x55ac2d6e59b0 2024-04-26T02:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.007+0000 7feafaa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.007+0000 7feaf3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 0x55ac2d6efad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.007+0000 7feafb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6f0b90 0x55ac2d708c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.007+0000 7feafaa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6fc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:54732/0 (socket says 172.21.15.1:54732) 2024-04-26T02:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.007+0000 7feafaa00640 1 -- 172.21.15.1:0/1426461372 learned_addr learned my addr 172.21.15.1:0/1426461372 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.007+0000 7feafb400640 1 -- 172.21.15.1:0/1426461372 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6e59b0 msgr2=0x55ac2d6fc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.007+0000 7feafb400640 1 --2- 172.21.15.1:0/1426461372 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6fc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.007+0000 7feafb400640 1 -- 172.21.15.1:0/1426461372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 msgr2=0x55ac2d6efad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.007+0000 7feafb400640 1 --2- 172.21.15.1:0/1426461372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 0x55ac2d6efad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.007+0000 7feafb400640 1 -- 172.21.15.1:0/1426461372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac2d703700 con 0x55ac2d6f0b90 2024-04-26T02:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.007+0000 7feafaa00640 1 --2- 172.21.15.1:0/1426461372 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.007+0000 7feaf3e00640 1 --2- 172.21.15.1:0/1426461372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 0x55ac2d6efad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.008+0000 7feafb400640 1 --2- 172.21.15.1:0/1426461372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6f0b90 0x55ac2d708c60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7feaf4010c40 tx=0x7feaf4010f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52e4a1d974ea70ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.008+0000 7feaeb400640 1 -- 172.21.15.1:0/1426461372 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4016050 con 0x55ac2d6f0b90 2024-04-26T02:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.008+0000 7feaeb400640 1 -- 172.21.15.1:0/1426461372 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf4027070 con 0x55ac2d6f0b90 2024-04-26T02:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.008+0000 7feaeb400640 1 -- 172.21.15.1:0/1426461372 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4022780 con 0x55ac2d6f0b90 2024-04-26T02:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.009+0000 7feb04592740 1 -- 172.21.15.1:0/1426461372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6f0b90 msgr2=0x55ac2d708c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.009+0000 7feb04592740 1 --2- 172.21.15.1:0/1426461372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6f0b90 0x55ac2d708c60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7feaf4010c40 tx=0x7feaf4010f90 comp rx=0 tx=0).stop 2024-04-26T02:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.009+0000 7feb04592740 1 -- 172.21.15.1:0/1426461372 shutdown_connections 2024-04-26T02:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.009+0000 7feb04592740 1 --2- 172.21.15.1:0/1426461372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6f0b90 0x55ac2d708c60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.009+0000 7feb04592740 1 --2- 172.21.15.1:0/1426461372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 0x55ac2d6efad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.009+0000 7feb04592740 1 --2- 172.21.15.1:0/1426461372 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.009+0000 7feb04592740 1 -- 172.21.15.1:0/1426461372 >> 172.21.15.1:0/1426461372 conn(0x55ac2d6e1ec0 msgr2=0x55ac2d6ef3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.009+0000 7feb04592740 1 -- 172.21.15.1:0/1426461372 shutdown_connections 2024-04-26T02:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.009+0000 7feb04592740 1 -- 172.21.15.1:0/1426461372 wait complete. 2024-04-26T02:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.009+0000 7feb04592740 1 -- start start 2024-04-26T02:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.010+0000 7feb04592740 1 Processor -- start 2024-04-26T02:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.010+0000 7feb04592740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6f8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.010+0000 7feb04592740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 0x55ac2d6f6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.010+0000 7feb04592740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6f0b90 0x55ac2d6f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.011+0000 7feb04592740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1b27b0 con 0x55ac2d6e62d0 2024-04-26T02:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.011+0000 7feb04592740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1b3c20 con 0x55ac2d6f0b90 2024-04-26T02:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.011+0000 7feb04592740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1b2e80 con 0x55ac2d6e59b0 2024-04-26T02:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.011+0000 7feb04592740 10 client.0 _ll_register_callbacks cb 0x7feae4004e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.011+0000 7feafaa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6f8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.011+0000 7feaf3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 0x55ac2d6f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.011+0000 7feafaa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6f8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:54742/0 (socket says 172.21.15.1:54742) 2024-04-26T02:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.011+0000 7feafaa00640 1 -- 172.21.15.1:0/2314168122 learned_addr learned my addr 172.21.15.1:0/2314168122 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.011+0000 7feafb400640 1 --2- 172.21.15.1:0/2314168122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6f0b90 0x55ac2d6f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.011+0000 7feafaa00640 1 -- 172.21.15.1:0/2314168122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6f0b90 msgr2=0x55ac2d6f67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.011+0000 7feafaa00640 1 --2- 172.21.15.1:0/2314168122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6f0b90 0x55ac2d6f67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.011+0000 7feafaa00640 1 -- 172.21.15.1:0/2314168122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 msgr2=0x55ac2d6f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.011+0000 7feafaa00640 1 --2- 172.21.15.1:0/2314168122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 0x55ac2d6f6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.011+0000 7feafaa00640 1 -- 172.21.15.1:0/2314168122 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac2d6f6fe0 con 0x55ac2d6e59b0 2024-04-26T02:01:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.012+0000 7feafaa00640 1 --2- 172.21.15.1:0/2314168122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6f8f50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feaec004770 tx=0x7feaec009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6aec4f45d5126d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.012+0000 7feae8c00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T02:01:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.012+0000 7feae8c00640 1 -- 172.21.15.1:0/2314168122 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec031e60 con 0x55ac2d6e59b0 2024-04-26T02:01:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.012+0000 7feb04592740 1 -- 172.21.15.1:0/2314168122 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaf4027070 con 0x55ac2d6e59b0 2024-04-26T02:01:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.012+0000 7feae8c00640 1 -- 172.21.15.1:0/2314168122 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7feaec053500 con 0x55ac2d6e59b0 2024-04-26T02:01:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.012+0000 7feae8c00640 1 -- 172.21.15.1:0/2314168122 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feaec035040 con 0x55ac2d6e59b0 2024-04-26T02:01:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.012+0000 7feb04592740 10 client.26314 fetch_fsmap learned FSMap version 32 2024-04-26T02:01:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.012+0000 7feb04592740 1 -- 172.21.15.1:0/2314168122 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ac2d767520 con 0x55ac2d6e59b0 2024-04-26T02:01:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.013+0000 7feae8c00640 1 -- 172.21.15.1:0/2314168122 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf4027070 con 0x55ac2d6e59b0 2024-04-26T02:01:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.013+0000 7feae8c00640 1 -- 172.21.15.1:0/2314168122 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7feaec05c470 con 0x55ac2d6e59b0 2024-04-26T02:01:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.013+0000 7feb04592740 10 client.26314 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:01:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.013+0000 7feb04592740 10 client.26314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:01:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.013+0000 7feb04592740 20 client.26314 populate_metadata read hostname 'smithi001' 2024-04-26T02:01:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.013+0000 7feb04592740 1 --2- 172.21.15.1:0/2314168122 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ac2d767c50 0x55ac2d76a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.013+0000 7feb04592740 4 client.26314 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:01:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.013+0000 7feb04592740 1 -- 172.21.15.1:0/2314168122 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac2d659320 con 0x55ac2d767c50 2024-04-26T02:01:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.013+0000 7feaf3e00640 1 --2- 172.21.15.1:0/2314168122 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ac2d767c50 0x55ac2d76a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.014+0000 7feaf3e00640 1 --2- 172.21.15.1:0/2314168122 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ac2d767c50 0x55ac2d76a070 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bae71d4a4acf28f server_cookie=1772ac0a752f743d in_seq=0 out_seq=0 2024-04-26T02:01:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.014+0000 7feae8c00640 10 client.26314 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.029+0000 7feae8c00640 1 -- 172.21.15.1:0/2314168122 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac2d659320 con 0x55ac2d767c50 2024-04-26T02:01:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.029+0000 7feae8c00640 10 client.26314 handle_command_reply: tid=0 2024-04-26T02:01:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.029+0000 7feb04592740 1 client.26314 shutdown 2024-04-26T02:01:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.029+0000 7feb04592740 1 -- 172.21.15.1:0/2314168122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6e59b0 msgr2=0x55ac2d6f8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.029+0000 7feb04592740 1 --2- 172.21.15.1:0/2314168122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6f8f50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feaec004770 tx=0x7feaec009260 comp rx=0 tx=0).stop 2024-04-26T02:01:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.029+0000 7feb04592740 1 -- 172.21.15.1:0/2314168122 shutdown_connections 2024-04-26T02:01:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.029+0000 7feb04592740 1 --2- 172.21.15.1:0/2314168122 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ac2d767c50 0x55ac2d76a070 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.029+0000 7feb04592740 1 --2- 172.21.15.1:0/2314168122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ac2d6f0b90 0x55ac2d6f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.029+0000 7feb04592740 1 --2- 172.21.15.1:0/2314168122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ac2d6e62d0 0x55ac2d6f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.029+0000 7feb04592740 1 --2- 172.21.15.1:0/2314168122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac2d6e59b0 0x55ac2d6f8f50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.029+0000 7feae8c00640 0 client.26314 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.029+0000 7feb04592740 1 -- 172.21.15.1:0/2314168122 >> 172.21.15.1:0/2314168122 conn(0x55ac2d6e1ec0 msgr2=0x55ac2d6efc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.030+0000 7feb04592740 1 -- 172.21.15.1:0/2314168122 shutdown_connections 2024-04-26T02:01:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.030+0000 7feb04592740 1 -- 172.21.15.1:0/2314168122 wait complete. 2024-04-26T02:01:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.030+0000 7feb04592740 20 client.26314 trim_cache size 0 max 16384 2024-04-26T02:01:03.030 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T02:01:03.030 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T02:01:03.030 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T02:01:03.030 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T02:01:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.031+0000 7feafbe00640 1 -- 172.21.15.1:0/1883690707 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7feae00a3d00 msgr2=0x7feae00a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.031+0000 7feafbe00640 1 --2- 172.21.15.1:0/1883690707 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7feae00a3d00 0x7feae00a61c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7feaec009230 tx=0x7feaec0058d0 comp rx=0 tx=0).stop 2024-04-26T02:01:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.031+0000 7feafbe00640 1 -- 172.21.15.1:0/1883690707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feafc074c10 msgr2=0x7feafc1a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.031+0000 7feafbe00640 1 --2- 172.21.15.1:0/1883690707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feafc074c10 0x7feafc1a4d90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7feae400e9f0 tx=0x7feae400eeb0 comp rx=0 tx=0).stop 2024-04-26T02:01:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.031+0000 7feafbe00640 1 -- 172.21.15.1:0/1883690707 shutdown_connections 2024-04-26T02:01:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.031+0000 7feafbe00640 1 --2- 172.21.15.1:0/1883690707 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7feae00a3d00 0x7feae00a61c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.031+0000 7feafbe00640 1 --2- 172.21.15.1:0/1883690707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feafc112d80 0x7feafc19ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.031+0000 7feafbe00640 1 --2- 172.21.15.1:0/1883690707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feafc074c10 0x7feafc1a4d90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.031+0000 7feafbe00640 1 --2- 172.21.15.1:0/1883690707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feafc0742f0 0x7feafc1a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.031+0000 7feafbe00640 1 -- 172.21.15.1:0/1883690707 >> 172.21.15.1:0/1883690707 conn(0x7feafc0fd860 msgr2=0x7feafc0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.031+0000 7feafbe00640 1 -- 172.21.15.1:0/1883690707 shutdown_connections 2024-04-26T02:01:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:03.031+0000 7feafbe00640 1 -- 172.21.15.1:0/1883690707 wait complete. 2024-04-26T02:01:03.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:03 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:01:03.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:03 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:01:03.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:03 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:01:03.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:03 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:01:03.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:03 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:01:03.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:03 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:01:03.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:03 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:01:03.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:03 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:01:03.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:03 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:01:04.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:04 smithi173 ceph-mon[32553]: pgmap v292: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 580 KiB/s wr, 158 op/s 2024-04-26T02:01:04.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:04 smithi001 ceph-mon[27822]: pgmap v292: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 580 KiB/s wr, 158 op/s 2024-04-26T02:01:04.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:04 smithi097 ceph-mon[32853]: pgmap v292: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 580 KiB/s wr, 158 op/s 2024-04-26T02:01:06.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:06 smithi173 ceph-mon[32553]: pgmap v293: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 632 KiB/s wr, 181 op/s 2024-04-26T02:01:06.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:06 smithi173 ceph-mon[32553]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T02:01:06.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:06 smithi001 ceph-mon[27822]: pgmap v293: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 632 KiB/s wr, 181 op/s 2024-04-26T02:01:06.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:06 smithi001 ceph-mon[27822]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T02:01:06.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:06 smithi097 ceph-mon[32853]: pgmap v293: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 632 KiB/s wr, 181 op/s 2024-04-26T02:01:06.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:06 smithi097 ceph-mon[32853]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T02:01:08.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:08 smithi173 ceph-mon[32553]: pgmap v295: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 66 KiB/s wr, 30 op/s 2024-04-26T02:01:08.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:08 smithi001 ceph-mon[27822]: pgmap v295: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 66 KiB/s wr, 30 op/s 2024-04-26T02:01:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:08 smithi097 ceph-mon[32853]: pgmap v295: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 66 KiB/s wr, 30 op/s 2024-04-26T02:01:10.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:10 smithi001 ceph-mon[27822]: pgmap v296: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 66 KiB/s wr, 30 op/s 2024-04-26T02:01:10.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:10 smithi097 ceph-mon[32853]: pgmap v296: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 66 KiB/s wr, 30 op/s 2024-04-26T02:01:10.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:10 smithi173 ceph-mon[32553]: pgmap v296: 97 pgs: 97 active+clean; 115 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 66 KiB/s wr, 30 op/s 2024-04-26T02:01:12.370 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:12 smithi173 ceph-mon[32553]: pgmap v297: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 83 KiB/s wr, 30 op/s 2024-04-26T02:01:12.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:12 smithi001 ceph-mon[27822]: pgmap v297: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 83 KiB/s wr, 30 op/s 2024-04-26T02:01:12.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:12 smithi097 ceph-mon[32853]: pgmap v297: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 83 KiB/s wr, 30 op/s 2024-04-26T02:01:14.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:14 smithi001 ceph-mon[27822]: pgmap v298: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 83 KiB/s wr, 30 op/s 2024-04-26T02:01:14.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:14 smithi097 ceph-mon[32853]: pgmap v298: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 83 KiB/s wr, 30 op/s 2024-04-26T02:01:14.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:14 smithi173 ceph-mon[32553]: pgmap v298: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 83 KiB/s wr, 30 op/s 2024-04-26T02:01:15.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:15 smithi097 ceph-mon[32853]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T02:01:15.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:15 smithi173 ceph-mon[32553]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T02:01:15.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:15 smithi001 ceph-mon[27822]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T02:01:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.181+0000 7f8293200640 1 -- 172.21.15.1:0/3885462308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8294074c10 msgr2=0x7f8294112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.181+0000 7f8293200640 1 --2- 172.21.15.1:0/3885462308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8294074c10 0x7f8294112840 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f827c009900 tx=0x7f827c02a480 comp rx=0 tx=0).stop 2024-04-26T02:01:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.181+0000 7f8293200640 1 -- 172.21.15.1:0/3885462308 shutdown_connections 2024-04-26T02:01:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.181+0000 7f8293200640 1 --2- 172.21.15.1:0/3885462308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8294112d80 0x7f8294115170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.181+0000 7f8293200640 1 --2- 172.21.15.1:0/3885462308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8294074c10 0x7f8294112840 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.181+0000 7f8293200640 1 --2- 172.21.15.1:0/3885462308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82940742f0 0x7f82940746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.181+0000 7f8293200640 1 -- 172.21.15.1:0/3885462308 >> 172.21.15.1:0/3885462308 conn(0x7f82940fd860 msgr2=0x7f82940ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.181+0000 7f8293200640 1 -- 172.21.15.1:0/3885462308 shutdown_connections 2024-04-26T02:01:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.182+0000 7f8293200640 1 -- 172.21.15.1:0/3885462308 wait complete. 2024-04-26T02:01:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.182+0000 7f8293200640 1 Processor -- start 2024-04-26T02:01:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f8293200640 1 -- start start 2024-04-26T02:01:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f8293200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82940742f0 0x7f82941a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f8293200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8294074c10 0x7f82941a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f8293200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8294112d80 0x7f829419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f8293200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8294118650 con 0x7f8294074c10 2024-04-26T02:01:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f8293200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f82941184d0 con 0x7f82940742f0 2024-04-26T02:01:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f8293200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f82941187d0 con 0x7f8294112d80 2024-04-26T02:01:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f8291e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82940742f0 0x7f82941a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f8291e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82940742f0 0x7f82941a4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:40738/0 (socket says 172.21.15.1:40738) 2024-04-26T02:01:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f8291e00640 1 -- 172.21.15.1:0/1769011504 learned_addr learned my addr 172.21.15.1:0/1769011504 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f8289400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8294074c10 0x7f82941a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f8292800640 1 --2- 172.21.15.1:0/1769011504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8294112d80 0x7f829419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f8291e00640 1 -- 172.21.15.1:0/1769011504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8294112d80 msgr2=0x7f829419e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f8291e00640 1 --2- 172.21.15.1:0/1769011504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8294112d80 0x7f829419e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f8291e00640 1 -- 172.21.15.1:0/1769011504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8294074c10 msgr2=0x7f82941a4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f8291e00640 1 --2- 172.21.15.1:0/1769011504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8294074c10 0x7f82941a4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f8291e00640 1 -- 172.21.15.1:0/1769011504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829419f130 con 0x7f82940742f0 2024-04-26T02:01:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f8291e00640 1 --2- 172.21.15.1:0/1769011504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82940742f0 0x7f82941a4800 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f828400e9f0 tx=0x7f828400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f828b400640 1 -- 172.21.15.1:0/1769011504 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828400cd80 con 0x7f82940742f0 2024-04-26T02:01:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f8293200640 1 -- 172.21.15.1:0/1769011504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829419f420 con 0x7f82940742f0 2024-04-26T02:01:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f828b400640 1 -- 172.21.15.1:0/1769011504 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82840108e0 con 0x7f82940742f0 2024-04-26T02:01:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f828b400640 1 -- 172.21.15.1:0/1769011504 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8284018980 con 0x7f82940742f0 2024-04-26T02:01:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.183+0000 7f8293200640 1 -- 172.21.15.1:0/1769011504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82941ab6c0 con 0x7f82940742f0 2024-04-26T02:01:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.188+0000 7f829bd7b740 1 Processor -- start 2024-04-26T02:01:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.188+0000 7f829bd7b740 1 -- start start 2024-04-26T02:01:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.188+0000 7f828b400640 1 -- 172.21.15.1:0/1769011504 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f8284010430 con 0x7f82940742f0 2024-04-26T02:01:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.188+0000 7f829bd7b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e269b0 0x561161e26d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.188+0000 7f829bd7b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e272d0 0x561161e31650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.188+0000 7f829bd7b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561161e31b90 0x561161e33f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.188+0000 7f829bd7b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561161d8c900 con 0x561161e272d0 2024-04-26T02:01:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.188+0000 7f829bd7b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561161d32ef0 con 0x561161e31b90 2024-04-26T02:01:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.188+0000 7f829bd7b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561161d888c0 con 0x561161e269b0 2024-04-26T02:01:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.188+0000 7f8291e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e272d0 0x561161e31650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.188+0000 7f8292800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e269b0 0x561161e26d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.188+0000 7f8292800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e269b0 0x561161e26d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:57330/0 (socket says 172.21.15.1:57330) 2024-04-26T02:01:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.188+0000 7f8292800640 1 -- 172.21.15.1:0/37062164 learned_addr learned my addr 172.21.15.1:0/37062164 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.188+0000 7f8289400640 1 --2- 172.21.15.1:0/37062164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561161e31b90 0x561161e33f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.188+0000 7f8292800640 1 -- 172.21.15.1:0/37062164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561161e31b90 msgr2=0x561161e33f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.188+0000 7f8292800640 1 --2- 172.21.15.1:0/37062164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561161e31b90 0x561161e33f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.188+0000 7f8292800640 1 -- 172.21.15.1:0/37062164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e272d0 msgr2=0x561161e31650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.188+0000 7f8292800640 1 --2- 172.21.15.1:0/37062164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e272d0 0x561161e31650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.189 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.188+0000 7f8292800640 1 -- 172.21.15.1:0/37062164 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561161a9f210 con 0x561161e269b0 2024-04-26T02:01:16.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.188+0000 7f8292800640 1 --2- 172.21.15.1:0/37062164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e269b0 0x561161e26d90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f828c009f30 tx=0x7f828c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54289bc51b7432de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.188+0000 7f8283400640 1 -- 172.21.15.1:0/37062164 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c004d20 con 0x561161e269b0 2024-04-26T02:01:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.190+0000 7f8283400640 1 -- 172.21.15.1:0/37062164 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f828c004050 con 0x561161e269b0 2024-04-26T02:01:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.190+0000 7f828b400640 1 --2- 172.21.15.1:0/1769011504 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f82780a3d80 0x7f82780a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.190+0000 7f8283400640 1 -- 172.21.15.1:0/37062164 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c005280 con 0x561161e269b0 2024-04-26T02:01:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.190+0000 7f828b400640 1 -- 172.21.15.1:0/1769011504 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f82840d0b60 con 0x7f82940742f0 2024-04-26T02:01:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.190+0000 7f8291e00640 1 --2- 172.21.15.1:0/1769011504 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f82780a3d80 0x7f82780a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.190+0000 7f829bd7b740 1 -- 172.21.15.1:0/37062164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e269b0 msgr2=0x561161e26d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.190+0000 7f829bd7b740 1 --2- 172.21.15.1:0/37062164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e269b0 0x561161e26d90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f828c009f30 tx=0x7f828c0118d0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.190+0000 7f829bd7b740 1 -- 172.21.15.1:0/37062164 shutdown_connections 2024-04-26T02:01:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.190+0000 7f829bd7b740 1 --2- 172.21.15.1:0/37062164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561161e31b90 0x561161e33f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.190+0000 7f829bd7b740 1 --2- 172.21.15.1:0/37062164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e272d0 0x561161e31650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.190+0000 7f829bd7b740 1 --2- 172.21.15.1:0/37062164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e269b0 0x561161e26d90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.190+0000 7f829bd7b740 1 -- 172.21.15.1:0/37062164 >> 172.21.15.1:0/37062164 conn(0x561161e22ec0 msgr2=0x561161e232a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.191+0000 7f829bd7b740 1 -- 172.21.15.1:0/37062164 shutdown_connections 2024-04-26T02:01:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.191+0000 7f829bd7b740 1 -- 172.21.15.1:0/37062164 wait complete. 2024-04-26T02:01:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.191+0000 7f8291e00640 1 --2- 172.21.15.1:0/1769011504 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f82780a3d80 0x7f82780a6240 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x561161e27f50 tx=0x7f82840d4610 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.191+0000 7f829bd7b740 1 -- start start 2024-04-26T02:01:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.191+0000 7f829bd7b740 1 Processor -- start 2024-04-26T02:01:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.192+0000 7f829bd7b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e269b0 0x561161e2eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.192+0000 7f829bd7b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e272d0 0x561161e2f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.192+0000 7f829bd7b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561161e31b90 0x561161e2f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.192+0000 7f829bd7b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561161e347c0 con 0x561161e269b0 2024-04-26T02:01:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.192+0000 7f829bd7b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561161d46310 con 0x561161e31b90 2024-04-26T02:01:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.192+0000 7f829bd7b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561161e34940 con 0x561161e272d0 2024-04-26T02:01:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.192+0000 7f829bd7b740 10 client.0 _ll_register_callbacks cb 0x561161d14cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.192+0000 7f8292800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e269b0 0x561161e2eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.192+0000 7f8292800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e269b0 0x561161e2eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55022/0 (socket says 172.21.15.1:55022) 2024-04-26T02:01:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.192+0000 7f8292800640 1 -- 172.21.15.1:0/1845235035 learned_addr learned my addr 172.21.15.1:0/1845235035 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.192+0000 7f8289400640 1 --2- 172.21.15.1:0/1845235035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e272d0 0x561161e2f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.192+0000 7f8292800640 1 --2- 172.21.15.1:0/1845235035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561161e31b90 0x561161e2f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.193+0000 7f8292800640 1 -- 172.21.15.1:0/1845235035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e272d0 msgr2=0x561161e2f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.193+0000 7f8292800640 1 --2- 172.21.15.1:0/1845235035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e272d0 0x561161e2f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.193+0000 7f8292800640 1 -- 172.21.15.1:0/1845235035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561161e31b90 msgr2=0x561161e2f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.193+0000 7f8292800640 1 --2- 172.21.15.1:0/1845235035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561161e31b90 0x561161e2f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.193+0000 7f8292800640 1 -- 172.21.15.1:0/1845235035 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561161b6b620 con 0x561161e269b0 2024-04-26T02:01:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.193+0000 7f8289400640 1 --2- 172.21.15.1:0/1845235035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e272d0 0x561161e2f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.193+0000 7f8292800640 1 --2- 172.21.15.1:0/1845235035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e269b0 0x561161e2eb20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f828c009d40 tx=0x7f828c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4a7ae46a6d3178 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.193+0000 7f8277e00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T02:01:16.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.193+0000 7f8277e00640 1 -- 172.21.15.1:0/1845235035 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c028450 con 0x561161e269b0 2024-04-26T02:01:16.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.194+0000 7f8277e00640 1 -- 172.21.15.1:0/1845235035 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f828c0285f0 con 0x561161e269b0 2024-04-26T02:01:16.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.194+0000 7f829bd7b740 1 -- 172.21.15.1:0/1845235035 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f828c004050 con 0x561161e269b0 2024-04-26T02:01:16.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.194+0000 7f8277e00640 1 -- 172.21.15.1:0/1845235035 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f828c037050 con 0x561161e269b0 2024-04-26T02:01:16.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.194+0000 7f829bd7b740 10 client.16149 fetch_fsmap learned FSMap version 32 2024-04-26T02:01:16.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.194+0000 7f829bd7b740 1 -- 172.21.15.1:0/1845235035 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561161e40a20 con 0x561161e269b0 2024-04-26T02:01:16.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.194+0000 7f8277e00640 1 -- 172.21.15.1:0/1845235035 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f828c004050 con 0x561161e269b0 2024-04-26T02:01:16.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.194+0000 7f8277e00640 1 -- 172.21.15.1:0/1845235035 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f828c025040 con 0x561161e269b0 2024-04-26T02:01:16.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.194+0000 7f829bd7b740 10 client.16149 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:01:16.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.194+0000 7f829bd7b740 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:01:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.195+0000 7f829bd7b740 20 client.16149 populate_metadata read hostname 'smithi001' 2024-04-26T02:01:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.195+0000 7f829bd7b740 1 --2- 172.21.15.1:0/1845235035 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561161e41e20 0x561161e44210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.195+0000 7f829bd7b740 4 client.16149 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:01:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.195+0000 7f829bd7b740 1 -- 172.21.15.1:0/1845235035 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561161da08b0 con 0x561161e41e20 2024-04-26T02:01:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.195+0000 7f8289400640 1 --2- 172.21.15.1:0/1845235035 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561161e41e20 0x561161e44210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.195+0000 7f8289400640 1 --2- 172.21.15.1:0/1845235035 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561161e41e20 0x561161e44210 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c5dbc8faa969060 server_cookie=595fdff465523c73 in_seq=0 out_seq=0 2024-04-26T02:01:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.195+0000 7f8277e00640 10 client.16149 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.195+0000 7f8277e00640 1 -- 172.21.15.1:0/1845235035 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561161da08b0 con 0x561161e41e20 2024-04-26T02:01:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.195+0000 7f8277e00640 10 client.16149 handle_command_reply: tid=0 2024-04-26T02:01:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.195+0000 7f829bd7b740 1 client.16149 shutdown 2024-04-26T02:01:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.195+0000 7f829bd7b740 1 -- 172.21.15.1:0/1845235035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e269b0 msgr2=0x561161e2eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.195+0000 7f829bd7b740 1 --2- 172.21.15.1:0/1845235035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e269b0 0x561161e2eb20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f828c009d40 tx=0x7f828c0096c0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.197+0000 7f829bd7b740 1 -- 172.21.15.1:0/1845235035 shutdown_connections 2024-04-26T02:01:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.197+0000 7f829bd7b740 1 --2- 172.21.15.1:0/1845235035 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561161e41e20 0x561161e44210 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.197+0000 7f829bd7b740 1 --2- 172.21.15.1:0/1845235035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561161e31b90 0x561161e2f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.197+0000 7f829bd7b740 1 --2- 172.21.15.1:0/1845235035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e272d0 0x561161e2f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.197+0000 7f829bd7b740 1 --2- 172.21.15.1:0/1845235035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e269b0 0x561161e2eb20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.197+0000 7f8277e00640 0 client.16149 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.197+0000 7f829bd7b740 1 -- 172.21.15.1:0/1845235035 >> 172.21.15.1:0/1845235035 conn(0x561161e22ec0 msgr2=0x561161e28c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.197+0000 7f829bd7b740 1 -- 172.21.15.1:0/1845235035 shutdown_connections 2024-04-26T02:01:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.197+0000 7f829bd7b740 1 -- 172.21.15.1:0/1845235035 wait complete. 2024-04-26T02:01:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.197+0000 7f829bd7b740 20 client.16149 trim_cache size 0 max 16384 2024-04-26T02:01:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.213+0000 7f829bd7b740 1 Processor -- start 2024-04-26T02:01:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.213+0000 7f829bd7b740 1 -- start start 2024-04-26T02:01:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.213+0000 7f829bd7b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e269b0 0x561161e30e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.214+0000 7f829bd7b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e272d0 0x561161e2e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.214+0000 7f829bd7b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561161e31b90 0x561161e227a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.214+0000 7f829bd7b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f82941b6a80 con 0x561161e272d0 2024-04-26T02:01:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.214+0000 7f829bd7b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f82941b4350 con 0x561161e31b90 2024-04-26T02:01:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.214+0000 7f829bd7b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f82941b21c0 con 0x561161e269b0 2024-04-26T02:01:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.214+0000 7f8289400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e272d0 0x561161e2e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.214+0000 7f8289400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e272d0 0x561161e2e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55026/0 (socket says 172.21.15.1:55026) 2024-04-26T02:01:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.214+0000 7f8289400640 1 -- 172.21.15.1:0/1793320374 learned_addr learned my addr 172.21.15.1:0/1793320374 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.214+0000 7f8292800640 1 --2- 172.21.15.1:0/1793320374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561161e31b90 0x561161e227a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.214+0000 7f8292800640 1 --2- 172.21.15.1:0/1793320374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e269b0 0x561161e30e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.214+0000 7f8289400640 1 -- 172.21.15.1:0/1793320374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e269b0 msgr2=0x561161e30e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.214+0000 7f8289400640 1 --2- 172.21.15.1:0/1793320374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e269b0 0x561161e30e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.214+0000 7f8289400640 1 -- 172.21.15.1:0/1793320374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561161e31b90 msgr2=0x561161e227a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.214+0000 7f8289400640 1 --2- 172.21.15.1:0/1793320374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561161e31b90 0x561161e227a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.214+0000 7f8289400640 1 -- 172.21.15.1:0/1793320374 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561161e23a90 con 0x561161e272d0 2024-04-26T02:01:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.214+0000 7f8292800640 1 --2- 172.21.15.1:0/1793320374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561161e31b90 0x561161e227a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:01:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.215+0000 7f8289400640 1 --2- 172.21.15.1:0/1793320374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e272d0 0x561161e2e460 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f827c009230 tx=0x7f827c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6763fc94113226 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.215+0000 7f8282a00640 1 -- 172.21.15.1:0/1793320374 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c035040 con 0x561161e272d0 2024-04-26T02:01:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.215+0000 7f8282a00640 1 -- 172.21.15.1:0/1793320374 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f827c045830 con 0x561161e272d0 2024-04-26T02:01:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.215+0000 7f8282a00640 1 -- 172.21.15.1:0/1793320374 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c005330 con 0x561161e272d0 2024-04-26T02:01:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.215+0000 7f829bd7b740 1 -- 172.21.15.1:0/1793320374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e272d0 msgr2=0x561161e2e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:16.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.215+0000 7f829bd7b740 1 --2- 172.21.15.1:0/1793320374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e272d0 0x561161e2e460 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f827c009230 tx=0x7f827c005a20 comp rx=0 tx=0).stop 2024-04-26T02:01:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.215+0000 7f829bd7b740 1 -- 172.21.15.1:0/1793320374 shutdown_connections 2024-04-26T02:01:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.215+0000 7f829bd7b740 1 --2- 172.21.15.1:0/1793320374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561161e31b90 0x561161e227a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.215+0000 7f829bd7b740 1 --2- 172.21.15.1:0/1793320374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e272d0 0x561161e2e460 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.215+0000 7f829bd7b740 1 --2- 172.21.15.1:0/1793320374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e269b0 0x561161e30e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.215+0000 7f829bd7b740 1 -- 172.21.15.1:0/1793320374 >> 172.21.15.1:0/1793320374 conn(0x561161e22ec0 msgr2=0x561161e28bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.215+0000 7f829bd7b740 1 -- 172.21.15.1:0/1793320374 shutdown_connections 2024-04-26T02:01:16.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.215+0000 7f829bd7b740 1 -- 172.21.15.1:0/1793320374 wait complete. 2024-04-26T02:01:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.215+0000 7f829bd7b740 1 -- start start 2024-04-26T02:01:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.215+0000 7f829bd7b740 1 Processor -- start 2024-04-26T02:01:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f829bd7b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561161e269b0 0x561161e39b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f829bd7b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e272d0 0x561161e3a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f829bd7b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e31b90 0x561161e36fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f829bd7b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f82941b2810 con 0x561161e272d0 2024-04-26T02:01:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f829bd7b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f82941b3c80 con 0x561161e269b0 2024-04-26T02:01:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f829bd7b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f82941b2ee0 con 0x561161e31b90 2024-04-26T02:01:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f829bd7b740 10 client.0 _ll_register_callbacks cb 0x7f828c003e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f8289400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e272d0 0x561161e3a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f8289400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e272d0 0x561161e3a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55032/0 (socket says 172.21.15.1:55032) 2024-04-26T02:01:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f8289400640 1 -- 172.21.15.1:0/3090517304 learned_addr learned my addr 172.21.15.1:0/3090517304 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f8292800640 1 --2- 172.21.15.1:0/3090517304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561161e269b0 0x561161e39b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f8292800640 1 --2- 172.21.15.1:0/3090517304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e31b90 0x561161e36fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f8289400640 1 -- 172.21.15.1:0/3090517304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e31b90 msgr2=0x561161e36fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f8289400640 1 --2- 172.21.15.1:0/3090517304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e31b90 0x561161e36fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f8289400640 1 -- 172.21.15.1:0/3090517304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561161e269b0 msgr2=0x561161e39b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f8289400640 1 --2- 172.21.15.1:0/3090517304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561161e269b0 0x561161e39b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f8289400640 1 -- 172.21.15.1:0/3090517304 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561161e377d0 con 0x561161e272d0 2024-04-26T02:01:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f8289400640 1 --2- 172.21.15.1:0/3090517304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e272d0 0x561161e3a080 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f827c02a9a0 tx=0x7f827c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0122c6c31fedd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f8277e00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T02:01:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f8277e00640 1 -- 172.21.15.1:0/3090517304 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c035040 con 0x561161e272d0 2024-04-26T02:01:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f829bd7b740 1 -- 172.21.15.1:0/3090517304 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f827c045830 con 0x561161e272d0 2024-04-26T02:01:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.217+0000 7f8277e00640 1 -- 172.21.15.1:0/3090517304 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f827c0459e0 con 0x561161e272d0 2024-04-26T02:01:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.219+0000 7f8277e00640 1 -- 172.21.15.1:0/3090517304 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f827c05d050 con 0x561161e272d0 2024-04-26T02:01:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.219+0000 7f829bd7b740 10 client.16155 fetch_fsmap learned FSMap version 32 2024-04-26T02:01:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.219+0000 7f829bd7b740 1 -- 172.21.15.1:0/3090517304 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561161ea7dc0 con 0x561161e272d0 2024-04-26T02:01:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.219+0000 7f8277e00640 1 -- 172.21.15.1:0/3090517304 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f827c045830 con 0x561161e272d0 2024-04-26T02:01:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.219+0000 7f8277e00640 1 -- 172.21.15.1:0/3090517304 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f827c0048c0 con 0x561161e272d0 2024-04-26T02:01:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.219+0000 7f829bd7b740 10 client.16155 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:01:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.219+0000 7f829bd7b740 10 client.16155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:01:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.219+0000 7f829bd7b740 20 client.16155 populate_metadata read hostname 'smithi001' 2024-04-26T02:01:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.219+0000 7f829bd7b740 1 --2- 172.21.15.1:0/3090517304 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561161ea8510 0x561161eaa930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.219+0000 7f829bd7b740 4 client.16155 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:01:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.219+0000 7f829bd7b740 1 -- 172.21.15.1:0/3090517304 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561161d9a320 con 0x561161ea8510 2024-04-26T02:01:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.220+0000 7f8292800640 1 --2- 172.21.15.1:0/3090517304 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561161ea8510 0x561161eaa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.220+0000 7f8292800640 1 --2- 172.21.15.1:0/3090517304 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561161ea8510 0x561161eaa930 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc06877b313a2a42 server_cookie=397c1daca110d946 in_seq=0 out_seq=0 2024-04-26T02:01:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.220+0000 7f8277e00640 10 client.16155 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.233+0000 7f8277e00640 1 -- 172.21.15.1:0/3090517304 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561161d9a320 con 0x561161ea8510 2024-04-26T02:01:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.233+0000 7f8277e00640 10 client.16155 handle_command_reply: tid=0 2024-04-26T02:01:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.233+0000 7f829bd7b740 1 client.16155 shutdown 2024-04-26T02:01:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.233+0000 7f829bd7b740 1 -- 172.21.15.1:0/3090517304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e272d0 msgr2=0x561161e3a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.233+0000 7f829bd7b740 1 --2- 172.21.15.1:0/3090517304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e272d0 0x561161e3a080 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f827c02a9a0 tx=0x7f827c000bc0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.233+0000 7f829bd7b740 1 -- 172.21.15.1:0/3090517304 shutdown_connections 2024-04-26T02:01:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.233+0000 7f829bd7b740 1 --2- 172.21.15.1:0/3090517304 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x561161ea8510 0x561161eaa930 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.233+0000 7f829bd7b740 1 --2- 172.21.15.1:0/3090517304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561161e31b90 0x561161e36fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.233+0000 7f829bd7b740 1 --2- 172.21.15.1:0/3090517304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561161e272d0 0x561161e3a080 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.233+0000 7f829bd7b740 1 --2- 172.21.15.1:0/3090517304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561161e269b0 0x561161e39b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.233+0000 7f829bd7b740 1 -- 172.21.15.1:0/3090517304 >> 172.21.15.1:0/3090517304 conn(0x561161e22ec0 msgr2=0x561161e2eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.233+0000 7f8277e00640 0 client.16155 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.233+0000 7f829bd7b740 1 -- 172.21.15.1:0/3090517304 shutdown_connections 2024-04-26T02:01:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.233+0000 7f829bd7b740 1 -- 172.21.15.1:0/3090517304 wait complete. 2024-04-26T02:01:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.233+0000 7f829bd7b740 20 client.16155 trim_cache size 0 max 16384 2024-04-26T02:01:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T02:01:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T02:01:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T02:01:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T02:01:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.233+0000 7f8293200640 1 -- 172.21.15.1:0/1769011504 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f82780a3d80 msgr2=0x7f82780a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.233+0000 7f8293200640 1 --2- 172.21.15.1:0/1769011504 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f82780a3d80 0x7f82780a6240 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x561161e27f50 tx=0x7f82840d4610 comp rx=0 tx=0).stop 2024-04-26T02:01:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.233+0000 7f8293200640 1 -- 172.21.15.1:0/1769011504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82940742f0 msgr2=0x7f82941a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.233+0000 7f8293200640 1 --2- 172.21.15.1:0/1769011504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82940742f0 0x7f82941a4800 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f828400e9f0 tx=0x7f828400eeb0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.235+0000 7f8293200640 1 -- 172.21.15.1:0/1769011504 shutdown_connections 2024-04-26T02:01:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.235+0000 7f8293200640 1 --2- 172.21.15.1:0/1769011504 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f82780a3d80 0x7f82780a6240 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.235+0000 7f8293200640 1 --2- 172.21.15.1:0/1769011504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8294112d80 0x7f829419e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.235+0000 7f8293200640 1 --2- 172.21.15.1:0/1769011504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8294074c10 0x7f82941a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.235+0000 7f8293200640 1 --2- 172.21.15.1:0/1769011504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82940742f0 0x7f82941a4800 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.235+0000 7f8293200640 1 -- 172.21.15.1:0/1769011504 >> 172.21.15.1:0/1769011504 conn(0x7f82940fd860 msgr2=0x7f82940ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.235+0000 7f8293200640 1 -- 172.21.15.1:0/1769011504 shutdown_connections 2024-04-26T02:01:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:16.235+0000 7f8293200640 1 -- 172.21.15.1:0/1769011504 wait complete. 2024-04-26T02:01:16.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:16 smithi173 ceph-mon[32553]: pgmap v300: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-04-26T02:01:16.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:16 smithi001 ceph-mon[27822]: pgmap v300: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-04-26T02:01:16.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:16 smithi097 ceph-mon[32853]: pgmap v300: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-04-26T02:01:18.106 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:01:18.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:18 smithi173 ceph-mon[32553]: pgmap v301: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-04-26T02:01:18.594 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:01:18.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:18 smithi001 ceph-mon[27822]: pgmap v301: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-04-26T02:01:18.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:18 smithi097 ceph-mon[32853]: pgmap v301: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-04-26T02:01:20.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:20 smithi001 ceph-mon[27822]: pgmap v302: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-04-26T02:01:20.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:20 smithi173 ceph-mon[32553]: pgmap v302: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-04-26T02:01:20.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:20 smithi097 ceph-mon[32853]: pgmap v302: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-04-26T02:01:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.763+0000 7f04f6800700 1 -- 172.21.15.1:0/3610146836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04f006a750 msgr2=0x7f04f006ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.763+0000 7f04f6800700 1 --2- 172.21.15.1:0/3610146836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04f006a750 0x7f04f006ab30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f04e0009990 tx=0x7f04e0009ca0 comp rx=0 tx=0).stop 2024-04-26T02:01:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.763+0000 7f04f6800700 1 -- 172.21.15.1:0/3610146836 shutdown_connections 2024-04-26T02:01:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.763+0000 7f04f6800700 1 --2- 172.21.15.1:0/3610146836 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04f010f4f0 0x7f04f01118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.763+0000 7f04f6800700 1 --2- 172.21.15.1:0/3610146836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04f006a750 0x7f04f006ab30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.763+0000 7f04f6800700 1 --2- 172.21.15.1:0/3610146836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04f006b070 0x7f04f010efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.763+0000 7f04f6800700 1 -- 172.21.15.1:0/3610146836 >> 172.21.15.1:0/3610146836 conn(0x7f04f00788f0 msgr2=0x7f04f0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.763+0000 7f04f6800700 1 -- 172.21.15.1:0/3610146836 shutdown_connections 2024-04-26T02:01:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.763+0000 7f04f6800700 1 -- 172.21.15.1:0/3610146836 wait complete. 2024-04-26T02:01:20.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.765+0000 7f04f6800700 1 Processor -- start 2024-04-26T02:01:20.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.765+0000 7f04f6800700 1 -- start start 2024-04-26T02:01:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.766+0000 7f04f6800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04f006a750 0x7f04f019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.766+0000 7f04f6800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04f006b070 0x7f04f019b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.766+0000 7f04f6800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04f010f4f0 0x7f04f019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.766+0000 7f04f6800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f04f0114e70 con 0x7f04f006a750 2024-04-26T02:01:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.766+0000 7f04f6800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f04f0114cf0 con 0x7f04f010f4f0 2024-04-26T02:01:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.766+0000 7f04f6800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f04f0114ff0 con 0x7f04f006b070 2024-04-26T02:01:20.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.766+0000 7f04ef400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04f006a750 0x7f04f019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:20.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.766+0000 7f04eea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04f006b070 0x7f04f019b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:20.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.766+0000 7f04ef400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04f006a750 0x7f04f019b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55046/0 (socket says 172.21.15.1:55046) 2024-04-26T02:01:20.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.766+0000 7f04ef400700 1 -- 172.21.15.1:0/582827312 learned_addr learned my addr 172.21.15.1:0/582827312 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:20.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.766+0000 7f04efe00700 1 --2- 172.21.15.1:0/582827312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04f010f4f0 0x7f04f019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:20.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.767+0000 7f04ef400700 1 -- 172.21.15.1:0/582827312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04f006b070 msgr2=0x7f04f019b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:20.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.767+0000 7f04ef400700 1 --2- 172.21.15.1:0/582827312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04f006b070 0x7f04f019b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:20.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.767+0000 7f04ef400700 1 -- 172.21.15.1:0/582827312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04f010f4f0 msgr2=0x7f04f019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:20.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.767+0000 7f04ef400700 1 --2- 172.21.15.1:0/582827312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04f010f4f0 0x7f04f019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:20.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.767+0000 7f04ef400700 1 -- 172.21.15.1:0/582827312 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f01a0370 con 0x7f04f006a750 2024-04-26T02:01:20.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.767+0000 7f04eea00700 1 --2- 172.21.15.1:0/582827312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04f006b070 0x7f04f019b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:20.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.767+0000 7f04ef400700 1 --2- 172.21.15.1:0/582827312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04f006a750 0x7f04f019b360 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f04e0000c00 tx=0x7f04e00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:20.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.768+0000 7f04e7e00700 1 -- 172.21.15.1:0/582827312 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0027070 con 0x7f04f006a750 2024-04-26T02:01:20.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.768+0000 7f04f6800700 1 -- 172.21.15.1:0/582827312 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f01a0600 con 0x7f04f006a750 2024-04-26T02:01:20.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.768+0000 7f04e7e00700 1 -- 172.21.15.1:0/582827312 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04e0022470 con 0x7f04f006a750 2024-04-26T02:01:20.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.768+0000 7f04e7e00700 1 -- 172.21.15.1:0/582827312 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0017650 con 0x7f04f006a750 2024-04-26T02:01:20.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.768+0000 7f04f6800700 1 -- 172.21.15.1:0/582827312 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f01a7ee0 con 0x7f04f006a750 2024-04-26T02:01:20.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.769+0000 7f04f6800700 1 -- 172.21.15.1:0/582827312 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f0101f60 con 0x7f04f006a750 2024-04-26T02:01:20.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.771+0000 7f04e7e00700 1 -- 172.21.15.1:0/582827312 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f04e002a070 con 0x7f04f006a750 2024-04-26T02:01:20.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.773+0000 7f04e7e00700 1 --2- 172.21.15.1:0/582827312 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f04d40a3db0 0x7f04d40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:20.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.773+0000 7f04e7e00700 1 -- 172.21.15.1:0/582827312 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f04e00d14f0 con 0x7f04f006a750 2024-04-26T02:01:20.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.773+0000 7f04eea00700 1 --2- 172.21.15.1:0/582827312 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f04d40a3db0 0x7f04d40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:20.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.774+0000 7f04eea00700 1 --2- 172.21.15.1:0/582827312 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f04d40a3db0 0x7f04d40a6270 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f04f0102f00 tx=0x7f04dc009260 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:20.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:20.778+0000 7f04e7e00700 1 -- 172.21.15.1:0/582827312 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04e0013070 con 0x7f04f006a750 2024-04-26T02:01:21.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:21.058+0000 7f04f6800700 1 -- 172.21.15.1:0/582827312 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04f002df80 con 0x7f04f006a750 2024-04-26T02:01:21.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:21.059+0000 7f04e7e00700 1 -- 172.21.15.1:0/582827312 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11838 (secure 0 0 0) 0x7f04e009db90 con 0x7f04f006a750 2024-04-26T02:01:21.058 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T02:01:21.058 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:55:49.645549+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25414":{"gid":25414,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/453427322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":453427322},{"type":"v1","addr":"172.21.15.173:6835","nonce":453427322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24499,24514,24518]},"id":1}]} 2024-04-26T02:01:21.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:21.062+0000 7f04f6800700 1 -- 172.21.15.1:0/582827312 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f04d40a3db0 msgr2=0x7f04d40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:21.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:21.062+0000 7f04f6800700 1 --2- 172.21.15.1:0/582827312 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f04d40a3db0 0x7f04d40a6270 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f04f0102f00 tx=0x7f04dc009260 comp rx=0 tx=0).stop 2024-04-26T02:01:21.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:21.062+0000 7f04f6800700 1 -- 172.21.15.1:0/582827312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04f006a750 msgr2=0x7f04f019b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:21.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:21.062+0000 7f04f6800700 1 --2- 172.21.15.1:0/582827312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04f006a750 0x7f04f019b360 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f04e0000c00 tx=0x7f04e00057c0 comp rx=0 tx=0).stop 2024-04-26T02:01:21.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:21.062+0000 7f04f6800700 1 -- 172.21.15.1:0/582827312 shutdown_connections 2024-04-26T02:01:21.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:21.062+0000 7f04f6800700 1 --2- 172.21.15.1:0/582827312 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f04d40a3db0 0x7f04d40a6270 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:21.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:21.062+0000 7f04f6800700 1 --2- 172.21.15.1:0/582827312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04f010f4f0 0x7f04f019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:21.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:21.062+0000 7f04f6800700 1 --2- 172.21.15.1:0/582827312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04f006a750 0x7f04f019b360 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:21.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:21.062+0000 7f04f6800700 1 --2- 172.21.15.1:0/582827312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04f006b070 0x7f04f019b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:21.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:21.062+0000 7f04f6800700 1 -- 172.21.15.1:0/582827312 >> 172.21.15.1:0/582827312 conn(0x7f04f00788f0 msgr2=0x7f04f01109a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:21.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:21.062+0000 7f04f6800700 1 -- 172.21.15.1:0/582827312 shutdown_connections 2024-04-26T02:01:21.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:21.062+0000 7f04f6800700 1 -- 172.21.15.1:0/582827312 wait complete. 2024-04-26T02:01:21.062 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 32 2024-04-26T02:01:21.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:21 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/582827312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:01:21.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:21 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/582827312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:01:21.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:21 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/582827312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:01:21.859 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24518 quiesce db --set-id 7a5cdd8c --release --await' 2024-04-26T02:01:21.860 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24518 quiesce db --set-id 7a5cdd8c --release --await 2024-04-26T02:01:22.290 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:01:22.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:22 smithi173 ceph-mon[32553]: pgmap v303: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-04-26T02:01:22.625 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:22 smithi001 ceph-mon[27822]: pgmap v303: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-04-26T02:01:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:22 smithi097 ceph-mon[32853]: pgmap v303: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-04-26T02:01:24.311 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:24 smithi001 ceph-mon[27822]: pgmap v304: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-04-26T02:01:24.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:24 smithi173 ceph-mon[32553]: pgmap v304: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-04-26T02:01:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.613+0000 7feeb0e00700 1 -- 172.21.15.1:0/274483391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeac109e90 msgr2=0x7feeac111a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.613+0000 7feeb0e00700 1 --2- 172.21.15.1:0/274483391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeac109e90 0x7feeac111a60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7feea40098f0 tx=0x7feea4009c00 comp rx=0 tx=0).stop 2024-04-26T02:01:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.614+0000 7feeb0e00700 1 -- 172.21.15.1:0/274483391 shutdown_connections 2024-04-26T02:01:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.614+0000 7feeb0e00700 1 --2- 172.21.15.1:0/274483391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeac109e90 0x7feeac111a60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.614+0000 7feeb0e00700 1 --2- 172.21.15.1:0/274483391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feeac104db0 0x7feeac105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.614+0000 7feeb0e00700 1 --2- 172.21.15.1:0/274483391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feeac105760 0x7feeac1097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.614+0000 7feeb0e00700 1 -- 172.21.15.1:0/274483391 >> 172.21.15.1:0/274483391 conn(0x7feeac100620 msgr2=0x7feeac102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.614+0000 7feeb0e00700 1 -- 172.21.15.1:0/274483391 shutdown_connections 2024-04-26T02:01:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.614+0000 7feeb0e00700 1 -- 172.21.15.1:0/274483391 wait complete. 2024-04-26T02:01:24.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.615+0000 7feeb0e00700 1 Processor -- start 2024-04-26T02:01:24.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.615+0000 7feeb0e00700 1 -- start start 2024-04-26T02:01:24.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.616+0000 7feeb0e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feeac104db0 0x7feeac19b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.616+0000 7feeb0e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feeac105760 0x7feeac19b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.616+0000 7feeb0e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeac109e90 0x7feeac19fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.616+0000 7feeb0e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feeac114d20 con 0x7feeac104db0 2024-04-26T02:01:24.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.616+0000 7feeb0e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feeac114ba0 con 0x7feeac109e90 2024-04-26T02:01:24.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.616+0000 7feeb0e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feeac114ea0 con 0x7feeac105760 2024-04-26T02:01:24.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.616+0000 7feeab400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feeac104db0 0x7feeac19b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.616+0000 7feea2a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feeac105760 0x7feeac19b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.616+0000 7feea2a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feeac105760 0x7feeac19b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:39566/0 (socket says 172.21.15.1:39566) 2024-04-26T02:01:24.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.616+0000 7feea2a00700 1 -- 172.21.15.1:0/581691262 learned_addr learned my addr 172.21.15.1:0/581691262 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:24.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.617+0000 7feeabe00700 1 --2- 172.21.15.1:0/581691262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeac109e90 0x7feeac19fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.617+0000 7feea2a00700 1 -- 172.21.15.1:0/581691262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeac109e90 msgr2=0x7feeac19fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:24.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.617+0000 7feea2a00700 1 --2- 172.21.15.1:0/581691262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeac109e90 0x7feeac19fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.617+0000 7feea2a00700 1 -- 172.21.15.1:0/581691262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feeac104db0 msgr2=0x7feeac19b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:24.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.617+0000 7feea2a00700 1 --2- 172.21.15.1:0/581691262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feeac104db0 0x7feeac19b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.617+0000 7feea2a00700 1 -- 172.21.15.1:0/581691262 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeac1a0220 con 0x7feeac105760 2024-04-26T02:01:24.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.617+0000 7feeab400700 1 --2- 172.21.15.1:0/581691262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feeac104db0 0x7feeac19b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:24.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.617+0000 7feea2a00700 1 --2- 172.21.15.1:0/581691262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feeac105760 0x7feeac19b750 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fee9400e3f0 tx=0x7fee9400e700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:24.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.618+0000 7feea8c00700 1 -- 172.21.15.1:0/581691262 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9400ee70 con 0x7feeac105760 2024-04-26T02:01:24.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.618+0000 7feeb0e00700 1 -- 172.21.15.1:0/581691262 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeac1a04b0 con 0x7feeac105760 2024-04-26T02:01:24.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.618+0000 7feea8c00700 1 -- 172.21.15.1:0/581691262 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee94003a00 con 0x7feeac105760 2024-04-26T02:01:24.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.618+0000 7feea8c00700 1 -- 172.21.15.1:0/581691262 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94004610 con 0x7feeac105760 2024-04-26T02:01:24.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.618+0000 7feeb0e00700 1 -- 172.21.15.1:0/581691262 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeac1a7d90 con 0x7feeac105760 2024-04-26T02:01:24.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.621+0000 7feea8c00700 1 -- 172.21.15.1:0/581691262 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fee9401b030 con 0x7feeac105760 2024-04-26T02:01:24.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.622+0000 7feebcdceb80 1 Processor -- start 2024-04-26T02:01:24.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.623+0000 7feea8c00700 1 --2- 172.21.15.1:0/581691262 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fee900a3e30 0x7fee900a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.623+0000 7feebcdceb80 1 -- start start 2024-04-26T02:01:24.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.623+0000 7feea8c00700 1 -- 172.21.15.1:0/581691262 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7fee940cffe0 con 0x7feeac105760 2024-04-26T02:01:24.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.623+0000 7feeabe00700 1 --2- 172.21.15.1:0/581691262 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fee900a3e30 0x7fee900a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.623+0000 7feebcdceb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8aea0 0x5621efb8b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.623+0000 7feebcdceb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb8b7c0 0x5621efb96130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.623+0000 7feebcdceb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb96670 0x5621efb98a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.623+0000 7feebcdceb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5621efaddf90 con 0x5621efb8aea0 2024-04-26T02:01:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.623+0000 7feebcdceb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5621efa84830 con 0x5621efb96670 2024-04-26T02:01:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.623+0000 7feebcdceb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5621efad9f50 con 0x5621efb8b7c0 2024-04-26T02:01:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.624+0000 7feeab400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8aea0 0x5621efb8b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.624+0000 7feeabe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb96670 0x5621efb98a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.624+0000 7feeab400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8aea0 0x5621efb8b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54472/0 (socket says 172.21.15.1:54472) 2024-04-26T02:01:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.624+0000 7feeab400700 1 -- 172.21.15.1:0/4089561384 learned_addr learned my addr 172.21.15.1:0/4089561384 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.624+0000 7feea2a00700 1 --2- 172.21.15.1:0/4089561384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb8b7c0 0x5621efb96130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.624+0000 7feea2a00700 1 -- 172.21.15.1:0/4089561384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb96670 msgr2=0x5621efb98a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.624+0000 7feea2a00700 1 --2- 172.21.15.1:0/4089561384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb96670 0x5621efb98a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.624+0000 7feea2a00700 1 -- 172.21.15.1:0/4089561384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8aea0 msgr2=0x5621efb8b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.624+0000 7feea2a00700 1 --2- 172.21.15.1:0/4089561384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8aea0 0x5621efb8b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.624+0000 7feea2a00700 1 -- 172.21.15.1:0/4089561384 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621efa7a780 con 0x5621efb8b7c0 2024-04-26T02:01:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.624+0000 7feeab400700 1 --2- 172.21.15.1:0/4089561384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8aea0 0x5621efb8b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.624+0000 7feeabe00700 1 --2- 172.21.15.1:0/581691262 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fee900a3e30 0x7fee900a62f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7feea40098c0 tx=0x7feea400f760 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.625+0000 7feea2a00700 1 --2- 172.21.15.1:0/4089561384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb8b7c0 0x5621efb96130 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fee940cd6e0 tx=0x7fee9409bb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2730847f28107a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.625+0000 7feea0c00700 1 -- 172.21.15.1:0/4089561384 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee940b5070 con 0x5621efb8b7c0 2024-04-26T02:01:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.625+0000 7feea0c00700 1 -- 172.21.15.1:0/4089561384 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee9409c6c0 con 0x5621efb8b7c0 2024-04-26T02:01:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.625+0000 7feea0c00700 1 -- 172.21.15.1:0/4089561384 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94098630 con 0x5621efb8b7c0 2024-04-26T02:01:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.626+0000 7feebcdceb80 1 -- 172.21.15.1:0/4089561384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb8b7c0 msgr2=0x5621efb96130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.626+0000 7feebcdceb80 1 --2- 172.21.15.1:0/4089561384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb8b7c0 0x5621efb96130 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fee940cd6e0 tx=0x7fee9409bb00 comp rx=0 tx=0).stop 2024-04-26T02:01:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.626+0000 7feebcdceb80 1 -- 172.21.15.1:0/4089561384 shutdown_connections 2024-04-26T02:01:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.626+0000 7feebcdceb80 1 --2- 172.21.15.1:0/4089561384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb96670 0x5621efb98a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.626+0000 7feebcdceb80 1 --2- 172.21.15.1:0/4089561384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8aea0 0x5621efb8b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.626+0000 7feebcdceb80 1 --2- 172.21.15.1:0/4089561384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb8b7c0 0x5621efb96130 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.626+0000 7feebcdceb80 1 -- 172.21.15.1:0/4089561384 >> 172.21.15.1:0/4089561384 conn(0x5621efaba050 msgr2=0x5621efb873f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.626+0000 7feebcdceb80 1 -- 172.21.15.1:0/4089561384 shutdown_connections 2024-04-26T02:01:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.626+0000 7feebcdceb80 1 -- 172.21.15.1:0/4089561384 wait complete. 2024-04-26T02:01:24.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.627+0000 7feebcdceb80 1 -- start start 2024-04-26T02:01:24.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.627+0000 7feebcdceb80 1 Processor -- start 2024-04-26T02:01:24.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.627+0000 7feebcdceb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb8aea0 0x5621efb95ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.627+0000 7feebcdceb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8b7c0 0x5621efb931e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.627+0000 7feebcdceb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb96670 0x5621efb93720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.627+0000 7feebcdceb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5621efb99100 con 0x5621efb8b7c0 2024-04-26T02:01:24.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.627+0000 7feebcdceb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5621efa97960 con 0x5621efb8aea0 2024-04-26T02:01:24.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.627+0000 7feebcdceb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5621efb99280 con 0x5621efb96670 2024-04-26T02:01:24.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.627+0000 7feebcdceb80 10 client.0 _ll_register_callbacks cb 0x5621efa76820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:24.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.628+0000 7feea2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8b7c0 0x5621efb931e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.628+0000 7feeab400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb8aea0 0x5621efb95ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.628+0000 7feea2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8b7c0 0x5621efb931e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54474/0 (socket says 172.21.15.1:54474) 2024-04-26T02:01:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.628+0000 7feeab400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb8aea0 0x5621efb95ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:50090/0 (socket says 172.21.15.1:50090) 2024-04-26T02:01:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.628+0000 7feea2a00700 1 -- 172.21.15.1:0/2258716168 learned_addr learned my addr 172.21.15.1:0/2258716168 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.628+0000 7feeabe00700 1 --2- 172.21.15.1:0/2258716168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb96670 0x5621efb93720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.628+0000 7feeab400700 1 -- 172.21.15.1:0/2258716168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb96670 msgr2=0x5621efb93720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.628+0000 7feeab400700 1 --2- 172.21.15.1:0/2258716168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb96670 0x5621efb93720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.628+0000 7feeab400700 1 -- 172.21.15.1:0/2258716168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8b7c0 msgr2=0x5621efb931e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.628+0000 7feeab400700 1 --2- 172.21.15.1:0/2258716168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8b7c0 0x5621efb931e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.628+0000 7feeab400700 1 -- 172.21.15.1:0/2258716168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621efa2b9d0 con 0x5621efb8aea0 2024-04-26T02:01:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.628+0000 7feea2a00700 1 --2- 172.21.15.1:0/2258716168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8b7c0 0x5621efb931e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.628+0000 7feeabe00700 1 --2- 172.21.15.1:0/2258716168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb96670 0x5621efb93720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:01:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.629+0000 7feeab400700 1 --2- 172.21.15.1:0/2258716168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb8aea0 0x5621efb95ee0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0119d0 tx=0x7fee9c011ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67cafebce10a0c58 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.629+0000 7fee99600700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:01:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.629+0000 7fee99600700 1 -- 172.21.15.1:0/2258716168 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c010b70 con 0x5621efb8aea0 2024-04-26T02:01:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.629+0000 7fee99600700 1 -- 172.21.15.1:0/2258716168 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7fee9c010d10 con 0x5621efb8aea0 2024-04-26T02:01:24.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.629+0000 7feebcdceb80 1 -- 172.21.15.1:0/2258716168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee9409c6c0 con 0x5621efb8aea0 2024-04-26T02:01:24.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.629+0000 7fee99600700 1 -- 172.21.15.1:0/2258716168 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fee9c023430 con 0x5621efb8aea0 2024-04-26T02:01:24.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.629+0000 7feebcdceb80 10 client.26339 fetch_fsmap learned FSMap version 32 2024-04-26T02:01:24.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.629+0000 7feebcdceb80 1 -- 172.21.15.1:0/2258716168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5621efa74bf0 con 0x5621efb8aea0 2024-04-26T02:01:24.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.630+0000 7fee99600700 1 -- 172.21.15.1:0/2258716168 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee9409c6c0 con 0x5621efb8aea0 2024-04-26T02:01:24.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.630+0000 7fee99600700 1 -- 172.21.15.1:0/2258716168 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fee9c01b650 con 0x5621efb8aea0 2024-04-26T02:01:24.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.630+0000 7feebcdceb80 10 client.26339 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:01:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.630+0000 7feebcdceb80 10 client.26339 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T02:01:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.630+0000 7feebcdceb80 20 client.26339 populate_metadata read hostname 'smithi001' 2024-04-26T02:01:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.630+0000 7feebcdceb80 1 --2- 172.21.15.1:0/2258716168 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5621efba5f40 0x5621efba8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.630+0000 7feebcdceb80 4 client.26339 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:01:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.630+0000 7feebcdceb80 1 -- 172.21.15.1:0/2258716168 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621efaeb780 con 0x5621efba5f40 2024-04-26T02:01:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.631+0000 7feea2a00700 1 --2- 172.21.15.1:0/2258716168 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5621efba5f40 0x5621efba8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.631+0000 7feea2a00700 1 --2- 172.21.15.1:0/2258716168 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5621efba5f40 0x5621efba8330 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b45794aef2694dc3 server_cookie=2f2cb80c851c4ccd in_seq=0 out_seq=0 2024-04-26T02:01:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.631+0000 7fee99600700 10 client.26339 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.632+0000 7fee99600700 1 -- 172.21.15.1:0/2258716168 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5621efaeb780 con 0x5621efba5f40 2024-04-26T02:01:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.632+0000 7fee99600700 10 client.26339 handle_command_reply: tid=0 2024-04-26T02:01:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.632+0000 7feebcdceb80 1 client.26339 shutdown 2024-04-26T02:01:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.632+0000 7feebcdceb80 1 -- 172.21.15.1:0/2258716168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb8aea0 msgr2=0x5621efb95ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.632+0000 7feebcdceb80 1 --2- 172.21.15.1:0/2258716168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb8aea0 0x5621efb95ee0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0119d0 tx=0x7fee9c011ce0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.632+0000 7feebcdceb80 1 -- 172.21.15.1:0/2258716168 shutdown_connections 2024-04-26T02:01:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.632+0000 7feebcdceb80 1 --2- 172.21.15.1:0/2258716168 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5621efba5f40 0x5621efba8330 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.632+0000 7feebcdceb80 1 --2- 172.21.15.1:0/2258716168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb96670 0x5621efb93720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.632+0000 7feebcdceb80 1 --2- 172.21.15.1:0/2258716168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb8aea0 0x5621efb95ee0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.632+0000 7feebcdceb80 1 --2- 172.21.15.1:0/2258716168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8b7c0 0x5621efb931e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.632+0000 7fee99600700 0 client.26339 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.632+0000 7feebcdceb80 1 -- 172.21.15.1:0/2258716168 >> 172.21.15.1:0/2258716168 conn(0x5621efaba050 msgr2=0x5621efb86fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.632+0000 7feebcdceb80 1 -- 172.21.15.1:0/2258716168 shutdown_connections 2024-04-26T02:01:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.633+0000 7feebcdceb80 1 -- 172.21.15.1:0/2258716168 wait complete. 2024-04-26T02:01:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.633+0000 7feebcdceb80 20 client.26339 trim_cache size 0 max 16384 2024-04-26T02:01:24.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.654+0000 7feebcdceb80 1 Processor -- start 2024-04-26T02:01:24.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.654+0000 7feebcdceb80 1 -- start start 2024-04-26T02:01:24.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.654+0000 7feebcdceb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb8aea0 0x5621efb87570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.654+0000 7feebcdceb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8b7c0 0x5621efba1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.654+0000 7feebcdceb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb96670 0x5621efb9e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.654+0000 7feebcdceb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feeac1b1e00 con 0x5621efb8b7c0 2024-04-26T02:01:24.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.654+0000 7feebcdceb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feeac1af690 con 0x5621efb96670 2024-04-26T02:01:24.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.654+0000 7feebcdceb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feeac1ad500 con 0x5621efb8aea0 2024-04-26T02:01:24.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.655+0000 7feea2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8b7c0 0x5621efba1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.655+0000 7feea2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8b7c0 0x5621efba1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54478/0 (socket says 172.21.15.1:54478) 2024-04-26T02:01:24.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.655+0000 7feea2a00700 1 -- 172.21.15.1:0/2819039537 learned_addr learned my addr 172.21.15.1:0/2819039537 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:24.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.655+0000 7feeab400700 1 --2- 172.21.15.1:0/2819039537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb8aea0 0x5621efb87570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.655+0000 7feeabe00700 1 --2- 172.21.15.1:0/2819039537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb96670 0x5621efb9e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.655+0000 7feeab400700 1 -- 172.21.15.1:0/2819039537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb96670 msgr2=0x5621efb9e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:24.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.655+0000 7feeab400700 1 --2- 172.21.15.1:0/2819039537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb96670 0x5621efb9e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.655+0000 7feeab400700 1 -- 172.21.15.1:0/2819039537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8b7c0 msgr2=0x5621efba1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:24.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.655+0000 7feeab400700 1 --2- 172.21.15.1:0/2819039537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8b7c0 0x5621efba1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.655+0000 7feeab400700 1 -- 172.21.15.1:0/2819039537 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621efb8ccf0 con 0x5621efb8aea0 2024-04-26T02:01:24.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.655+0000 7feeabe00700 1 --2- 172.21.15.1:0/2819039537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb96670 0x5621efb9e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:01:24.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.655+0000 7feea2a00700 1 --2- 172.21.15.1:0/2819039537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8b7c0 0x5621efba1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:24.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.656+0000 7feeab400700 1 --2- 172.21.15.1:0/2819039537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb8aea0 0x5621efb87570 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fee9c01bb40 tx=0x7fee9c010d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4bd0f3a7d752396 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:24.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.656+0000 7fee9be00700 1 -- 172.21.15.1:0/2819039537 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c01b420 con 0x5621efb8aea0 2024-04-26T02:01:24.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.656+0000 7fee9be00700 1 -- 172.21.15.1:0/2819039537 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee9c005040 con 0x5621efb8aea0 2024-04-26T02:01:24.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.656+0000 7fee9be00700 1 -- 172.21.15.1:0/2819039537 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c0038e0 con 0x5621efb8aea0 2024-04-26T02:01:24.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.657+0000 7feebcdceb80 1 -- 172.21.15.1:0/2819039537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb8aea0 msgr2=0x5621efb87570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:24.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.657+0000 7feebcdceb80 1 --2- 172.21.15.1:0/2819039537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb8aea0 0x5621efb87570 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fee9c01bb40 tx=0x7fee9c010d90 comp rx=0 tx=0).stop 2024-04-26T02:01:24.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.657+0000 7feebcdceb80 1 -- 172.21.15.1:0/2819039537 shutdown_connections 2024-04-26T02:01:24.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.657+0000 7feebcdceb80 1 --2- 172.21.15.1:0/2819039537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb96670 0x5621efb9e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.657+0000 7feebcdceb80 1 --2- 172.21.15.1:0/2819039537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb8aea0 0x5621efb87570 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.657+0000 7feebcdceb80 1 --2- 172.21.15.1:0/2819039537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8b7c0 0x5621efba1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.657+0000 7feebcdceb80 1 -- 172.21.15.1:0/2819039537 >> 172.21.15.1:0/2819039537 conn(0x5621efaba050 msgr2=0x5621efb952a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:24.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.657+0000 7feebcdceb80 1 -- 172.21.15.1:0/2819039537 shutdown_connections 2024-04-26T02:01:24.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.657+0000 7feebcdceb80 1 -- 172.21.15.1:0/2819039537 wait complete. 2024-04-26T02:01:24.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.657+0000 7feebcdceb80 1 -- start start 2024-04-26T02:01:24.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.658+0000 7feebcdceb80 1 Processor -- start 2024-04-26T02:01:24.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.658+0000 7feebcdceb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb8aea0 0x5621efb9d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.658+0000 7feebcdceb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8b7c0 0x5621efb9dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.658+0000 7feebcdceb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb96670 0x5621efc0c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.658+0000 7feebcdceb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feeac1adb50 con 0x5621efb8b7c0 2024-04-26T02:01:24.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.658+0000 7feebcdceb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feeac1aefc0 con 0x5621efb96670 2024-04-26T02:01:24.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.658+0000 7feebcdceb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feeac1ae220 con 0x5621efb8aea0 2024-04-26T02:01:24.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.658+0000 7feebcdceb80 10 client.0 _ll_register_callbacks cb 0x5621efa76820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:24.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.659+0000 7feea2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8b7c0 0x5621efb9dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.659+0000 7feeab400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb8aea0 0x5621efb9d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.659+0000 7feea2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8b7c0 0x5621efb9dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54488/0 (socket says 172.21.15.1:54488) 2024-04-26T02:01:24.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.659+0000 7feea2a00700 1 -- 172.21.15.1:0/3928520618 learned_addr learned my addr 172.21.15.1:0/3928520618 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:24.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.659+0000 7feeabe00700 1 --2- 172.21.15.1:0/3928520618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb96670 0x5621efc0c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.659+0000 7feeab400700 1 -- 172.21.15.1:0/3928520618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb96670 msgr2=0x5621efc0c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:24.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.659+0000 7feeab400700 1 --2- 172.21.15.1:0/3928520618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb96670 0x5621efc0c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.659+0000 7feeab400700 1 -- 172.21.15.1:0/3928520618 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8b7c0 msgr2=0x5621efb9dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:24.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.659+0000 7feeab400700 1 --2- 172.21.15.1:0/3928520618 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8b7c0 0x5621efb9dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.659+0000 7feeab400700 1 -- 172.21.15.1:0/3928520618 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621efb87ec0 con 0x5621efb8aea0 2024-04-26T02:01:24.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.659+0000 7feeab400700 1 --2- 172.21.15.1:0/3928520618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb8aea0 0x5621efb9d6d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fee9c000c00 tx=0x7fee9c005420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83c6e07f5c60346c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:24.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.659+0000 7fee99600700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T02:01:24.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.660+0000 7fee99600700 1 -- 172.21.15.1:0/3928520618 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c01b630 con 0x5621efb8aea0 2024-04-26T02:01:24.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.660+0000 7fee99600700 1 -- 172.21.15.1:0/3928520618 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7fee9c01b7d0 con 0x5621efb8aea0 2024-04-26T02:01:24.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.660+0000 7feebcdceb80 1 -- 172.21.15.1:0/3928520618 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee9c005040 con 0x5621efb8aea0 2024-04-26T02:01:24.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.660+0000 7fee99600700 1 -- 172.21.15.1:0/3928520618 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fee9c003680 con 0x5621efb8aea0 2024-04-26T02:01:24.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.660+0000 7feebcdceb80 10 client.26356 fetch_fsmap learned FSMap version 32 2024-04-26T02:01:24.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.660+0000 7feebcdceb80 1 -- 172.21.15.1:0/3928520618 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5621efc0dbc0 con 0x5621efb8aea0 2024-04-26T02:01:24.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.660+0000 7fee99600700 1 -- 172.21.15.1:0/3928520618 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee9c005040 con 0x5621efb8aea0 2024-04-26T02:01:24.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.660+0000 7fee99600700 1 -- 172.21.15.1:0/3928520618 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fee9c02b030 con 0x5621efb8aea0 2024-04-26T02:01:24.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.660+0000 7feebcdceb80 10 client.26356 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:01:24.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.661+0000 7feebcdceb80 10 client.26356 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T02:01:24.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.661+0000 7feebcdceb80 20 client.26356 populate_metadata read hostname 'smithi001' 2024-04-26T02:01:24.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.661+0000 7feebcdceb80 1 --2- 172.21.15.1:0/3928520618 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5621efc0e100 0x5621efc10520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:24.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.661+0000 7feebcdceb80 4 client.26356 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7a5cdd8c", "release": true, "await": true}] 2024-04-26T02:01:24.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.661+0000 7feebcdceb80 1 -- 172.21.15.1:0/3928520618 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7a5cdd8c", "release": true, "await": true}) -- 0x5621efae5820 con 0x5621efc0e100 2024-04-26T02:01:24.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.661+0000 7feea2a00700 1 --2- 172.21.15.1:0/3928520618 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5621efc0e100 0x5621efc10520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:24.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.661+0000 7feea2a00700 1 --2- 172.21.15.1:0/3928520618 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5621efc0e100 0x5621efc10520 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c13c90c6d99ac394 server_cookie=ddcd11e80d75e30 in_seq=0 out_seq=0 2024-04-26T02:01:24.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.661+0000 7fee99600700 10 client.26356 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:24.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:24 smithi097 ceph-mon[32853]: pgmap v304: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-04-26T02:01:24.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.849+0000 7fee99600700 1 -- 172.21.15.1:0/3928520618 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5621efae5820 con 0x5621efc0e100 2024-04-26T02:01:24.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.849+0000 7fee99600700 10 client.26356 handle_command_reply: tid=0 2024-04-26T02:01:24.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.849+0000 7feebcdceb80 1 client.26356 shutdown 2024-04-26T02:01:24.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.850+0000 7feebcdceb80 1 -- 172.21.15.1:0/3928520618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb8aea0 msgr2=0x5621efb9d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:24.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.850+0000 7feebcdceb80 1 --2- 172.21.15.1:0/3928520618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb8aea0 0x5621efb9d6d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fee9c000c00 tx=0x7fee9c005420 comp rx=0 tx=0).stop 2024-04-26T02:01:24.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.850+0000 7feebcdceb80 1 -- 172.21.15.1:0/3928520618 shutdown_connections 2024-04-26T02:01:24.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.850+0000 7feebcdceb80 1 --2- 172.21.15.1:0/3928520618 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5621efc0e100 0x5621efc10520 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.850+0000 7feebcdceb80 1 --2- 172.21.15.1:0/3928520618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5621efb96670 0x5621efc0c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.850+0000 7feebcdceb80 1 --2- 172.21.15.1:0/3928520618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5621efb8aea0 0x5621efb9d6d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.850+0000 7feebcdceb80 1 --2- 172.21.15.1:0/3928520618 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621efb8b7c0 0x5621efb9dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.850+0000 7feebcdceb80 1 -- 172.21.15.1:0/3928520618 >> 172.21.15.1:0/3928520618 conn(0x5621efaba050 msgr2=0x5621efb952a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:24.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.850+0000 7fee99600700 0 client.26356 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:24.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.851+0000 7feebcdceb80 1 -- 172.21.15.1:0/3928520618 shutdown_connections 2024-04-26T02:01:24.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.851+0000 7feebcdceb80 1 -- 172.21.15.1:0/3928520618 wait complete. 2024-04-26T02:01:24.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.851+0000 7feebcdceb80 20 client.26356 trim_cache size 0 max 16384 2024-04-26T02:01:24.852 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T02:01:24.852 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 32, 2024-04-26T02:01:24.852 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24518, 2024-04-26T02:01:24.852 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 28, 2024-04-26T02:01:24.852 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-04-26T02:01:24.852 INFO:teuthology.orchestra.run.smithi001.stdout: "7a5cdd8c": { 2024-04-26T02:01:24.852 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 28, 2024-04-26T02:01:24.852 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-04-26T02:01:24.852 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T02:01:24.852 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-04-26T02:01:24.853 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T02:01:24.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:01:24.853 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-04-26T02:01:24.853 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 82.1, 2024-04-26T02:01:24.853 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-04-26T02:01:24.853 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-04-26T02:01:24.853 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-04-26T02:01:24.853 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T02:01:24.853 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-04-26T02:01:24.853 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T02:01:24.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:01:24.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:01:24.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:01:24.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:01:24.854 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:01:24.854 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T02:01:24.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.855+0000 7feeb0e00700 1 -- 172.21.15.1:0/581691262 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fee900a3e30 msgr2=0x7fee900a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:24.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.855+0000 7feeb0e00700 1 --2- 172.21.15.1:0/581691262 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fee900a3e30 0x7fee900a62f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7feea40098c0 tx=0x7feea400f760 comp rx=0 tx=0).stop 2024-04-26T02:01:24.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.855+0000 7feeb0e00700 1 -- 172.21.15.1:0/581691262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feeac105760 msgr2=0x7feeac19b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:24.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.855+0000 7feeb0e00700 1 --2- 172.21.15.1:0/581691262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feeac105760 0x7feeac19b750 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fee9400e3f0 tx=0x7fee9400e700 comp rx=0 tx=0).stop 2024-04-26T02:01:24.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.855+0000 7feeb0e00700 1 -- 172.21.15.1:0/581691262 shutdown_connections 2024-04-26T02:01:24.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.855+0000 7feeb0e00700 1 --2- 172.21.15.1:0/581691262 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fee900a3e30 0x7fee900a62f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.855+0000 7feeb0e00700 1 --2- 172.21.15.1:0/581691262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeac109e90 0x7feeac19fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.855+0000 7feeb0e00700 1 --2- 172.21.15.1:0/581691262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feeac104db0 0x7feeac19b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.855+0000 7feeb0e00700 1 --2- 172.21.15.1:0/581691262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feeac105760 0x7feeac19b750 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:24.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.855+0000 7feeb0e00700 1 -- 172.21.15.1:0/581691262 >> 172.21.15.1:0/581691262 conn(0x7feeac100620 msgr2=0x7feeac102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:24.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.855+0000 7feeb0e00700 1 -- 172.21.15.1:0/581691262 shutdown_connections 2024-04-26T02:01:24.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:24.855+0000 7feeb0e00700 1 -- 172.21.15.1:0/581691262 wait complete. 2024-04-26T02:01:25.851 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7a5cdd8c', seconds elapsed: 39.2 2024-04-26T02:01:25.851 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.1 2024-04-26T02:01:25.851 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:01:26.343 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:01:26.499 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:26 smithi001 ceph-mon[27822]: pgmap v305: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-26T02:01:26.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:26 smithi173 ceph-mon[32553]: pgmap v305: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-26T02:01:26.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:26 smithi097 ceph-mon[32853]: pgmap v305: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-26T02:01:28.533 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:28 smithi001 ceph-mon[27822]: pgmap v306: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 7 op/s 2024-04-26T02:01:28.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:28 smithi173 ceph-mon[32553]: pgmap v306: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 7 op/s 2024-04-26T02:01:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.533+0000 7f6bb9000700 1 -- 172.21.15.1:0/1999452501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bb4105860 msgr2=0x7f6bb41098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.533+0000 7f6bb9000700 1 --2- 172.21.15.1:0/1999452501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bb4105860 0x7f6bb41098b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00099c0 tx=0x7f6ba0009cd0 comp rx=0 tx=0).stop 2024-04-26T02:01:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.533+0000 7f6bb9000700 1 -- 172.21.15.1:0/1999452501 shutdown_connections 2024-04-26T02:01:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.533+0000 7f6bb9000700 1 --2- 172.21.15.1:0/1999452501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bb4109f90 0x7f6bb4111b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.533+0000 7f6bb9000700 1 --2- 172.21.15.1:0/1999452501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bb4104eb0 0x7f6bb4105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.533+0000 7f6bb9000700 1 --2- 172.21.15.1:0/1999452501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bb4105860 0x7f6bb41098b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.533+0000 7f6bb9000700 1 -- 172.21.15.1:0/1999452501 >> 172.21.15.1:0/1999452501 conn(0x7f6bb4100740 msgr2=0x7f6bb4102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.533+0000 7f6bb9000700 1 -- 172.21.15.1:0/1999452501 shutdown_connections 2024-04-26T02:01:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.533+0000 7f6bb9000700 1 -- 172.21.15.1:0/1999452501 wait complete. 2024-04-26T02:01:28.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.535+0000 7f6bb9000700 1 Processor -- start 2024-04-26T02:01:28.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.535+0000 7f6bb9000700 1 -- start start 2024-04-26T02:01:28.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.535+0000 7f6bb9000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bb4104eb0 0x7f6bb419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:28.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.535+0000 7f6bb9000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bb4105860 0x7f6bb419b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:28.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.535+0000 7f6bb9000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bb4109f90 0x7f6bb419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:28.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.535+0000 7f6bb9000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb4114e50 con 0x7f6bb4105860 2024-04-26T02:01:28.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.535+0000 7f6bb9000700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb4114cd0 con 0x7f6bb4104eb0 2024-04-26T02:01:28.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.535+0000 7f6bb9000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb4114fd0 con 0x7f6bb4109f90 2024-04-26T02:01:28.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.536+0000 7f6bb1e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bb4104eb0 0x7f6bb419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:28.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.536+0000 7f6bb2800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bb4109f90 0x7f6bb419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:28.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.536+0000 7f6bb1e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bb4104eb0 0x7f6bb419b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:50138/0 (socket says 172.21.15.1:50138) 2024-04-26T02:01:28.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.536+0000 7f6bb1e00700 1 -- 172.21.15.1:0/1927774952 learned_addr learned my addr 172.21.15.1:0/1927774952 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:28.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.536+0000 7f6bb1400700 1 --2- 172.21.15.1:0/1927774952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bb4105860 0x7f6bb419b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:28.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.536+0000 7f6bb1e00700 1 -- 172.21.15.1:0/1927774952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bb4109f90 msgr2=0x7f6bb419fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:28.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.536+0000 7f6bb1e00700 1 --2- 172.21.15.1:0/1927774952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bb4109f90 0x7f6bb419fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:28.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.536+0000 7f6bb1e00700 1 -- 172.21.15.1:0/1927774952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bb4105860 msgr2=0x7f6bb419b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:28.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.536+0000 7f6bb1e00700 1 --2- 172.21.15.1:0/1927774952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bb4105860 0x7f6bb419b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:28.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.536+0000 7f6bb1e00700 1 -- 172.21.15.1:0/1927774952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb41a0360 con 0x7f6bb4104eb0 2024-04-26T02:01:28.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.536+0000 7f6bb2800700 1 --2- 172.21.15.1:0/1927774952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bb4109f90 0x7f6bb419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:28.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.536+0000 7f6bb1e00700 1 --2- 172.21.15.1:0/1927774952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bb4104eb0 0x7f6bb419b3a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400e9a0 tx=0x7f6ba400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:28.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.537+0000 7f6baaa00700 1 -- 172.21.15.1:0/1927774952 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba400c9c0 con 0x7f6bb4104eb0 2024-04-26T02:01:28.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.537+0000 7f6baaa00700 1 -- 172.21.15.1:0/1927774952 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ba400cb60 con 0x7f6bb4104eb0 2024-04-26T02:01:28.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.537+0000 7f6baaa00700 1 -- 172.21.15.1:0/1927774952 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4018680 con 0x7f6bb4104eb0 2024-04-26T02:01:28.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.537+0000 7f6bb9000700 1 -- 172.21.15.1:0/1927774952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb41a0650 con 0x7f6bb4104eb0 2024-04-26T02:01:28.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.537+0000 7f6bb9000700 1 -- 172.21.15.1:0/1927774952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb41a7ed0 con 0x7f6bb4104eb0 2024-04-26T02:01:28.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.538+0000 7f6bb9000700 1 -- 172.21.15.1:0/1927774952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bb4106390 con 0x7f6bb4104eb0 2024-04-26T02:01:28.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.541+0000 7f6baaa00700 1 -- 172.21.15.1:0/1927774952 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f6ba4010430 con 0x7f6bb4104eb0 2024-04-26T02:01:28.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.543+0000 7f6baaa00700 1 --2- 172.21.15.1:0/1927774952 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6b980a3e30 0x7f6b980a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:28.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.543+0000 7f6baaa00700 1 -- 172.21.15.1:0/1927774952 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f6ba40d0c00 con 0x7f6bb4104eb0 2024-04-26T02:01:28.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.546+0000 7f6bb1400700 1 --2- 172.21.15.1:0/1927774952 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6b980a3e30 0x7f6b980a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:28.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.547+0000 7f6baaa00700 1 -- 172.21.15.1:0/1927774952 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ba409d210 con 0x7f6bb4104eb0 2024-04-26T02:01:28.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.548+0000 7f6bb1400700 1 --2- 172.21.15.1:0/1927774952 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6b980a3e30 0x7f6b980a62f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0000c00 tx=0x7f6ba0005fb0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:28.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:28 smithi097 ceph-mon[32853]: pgmap v306: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 7 op/s 2024-04-26T02:01:28.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.864+0000 7f6bb9000700 1 -- 172.21.15.1:0/1927774952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bb402df80 con 0x7f6bb4104eb0 2024-04-26T02:01:28.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.866+0000 7f6baaa00700 1 -- 172.21.15.1:0/1927774952 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11838 (secure 0 0 0) 0x7f6ba409cb10 con 0x7f6bb4104eb0 2024-04-26T02:01:28.865 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T02:01:28.865 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:55:49.645549+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25414":{"gid":25414,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/453427322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":453427322},{"type":"v1","addr":"172.21.15.173:6835","nonce":453427322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24499,24514,24518]},"id":1}]} 2024-04-26T02:01:28.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.869+0000 7f6bb9000700 1 -- 172.21.15.1:0/1927774952 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6b980a3e30 msgr2=0x7f6b980a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:28.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.869+0000 7f6bb9000700 1 --2- 172.21.15.1:0/1927774952 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6b980a3e30 0x7f6b980a62f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0000c00 tx=0x7f6ba0005fb0 comp rx=0 tx=0).stop 2024-04-26T02:01:28.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.869+0000 7f6bb9000700 1 -- 172.21.15.1:0/1927774952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bb4104eb0 msgr2=0x7f6bb419b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:28.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.869+0000 7f6bb9000700 1 --2- 172.21.15.1:0/1927774952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bb4104eb0 0x7f6bb419b3a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400e9a0 tx=0x7f6ba400ecb0 comp rx=0 tx=0).stop 2024-04-26T02:01:28.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.869+0000 7f6bb9000700 1 -- 172.21.15.1:0/1927774952 shutdown_connections 2024-04-26T02:01:28.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.869+0000 7f6bb9000700 1 --2- 172.21.15.1:0/1927774952 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6b980a3e30 0x7f6b980a62f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:28.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.869+0000 7f6bb9000700 1 --2- 172.21.15.1:0/1927774952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bb4109f90 0x7f6bb419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:28.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.869+0000 7f6bb9000700 1 --2- 172.21.15.1:0/1927774952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bb4104eb0 0x7f6bb419b3a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:28.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.869+0000 7f6bb9000700 1 --2- 172.21.15.1:0/1927774952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bb4105860 0x7f6bb419b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:28.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.869+0000 7f6bb9000700 1 -- 172.21.15.1:0/1927774952 >> 172.21.15.1:0/1927774952 conn(0x7f6bb4100740 msgr2=0x7f6bb4102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:28.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.870+0000 7f6bb9000700 1 -- 172.21.15.1:0/1927774952 shutdown_connections 2024-04-26T02:01:28.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:28.870+0000 7f6bb9000700 1 -- 172.21.15.1:0/1927774952 wait complete. 2024-04-26T02:01:28.870 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 32 2024-04-26T02:01:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.397+0000 7ff493a00640 1 -- 172.21.15.1:0/2005563187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff48c1043c0 msgr2=0x7ff48c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.397+0000 7ff493a00640 1 --2- 172.21.15.1:0/2005563187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff48c1043c0 0x7ff48c1047a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff480009900 tx=0x7ff48002a410 comp rx=0 tx=0).stop 2024-04-26T02:01:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.397+0000 7ff493a00640 1 -- 172.21.15.1:0/2005563187 shutdown_connections 2024-04-26T02:01:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.397+0000 7ff493a00640 1 --2- 172.21.15.1:0/2005563187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff48c109490 0x7ff48c111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.397+0000 7ff493a00640 1 --2- 172.21.15.1:0/2005563187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48c104d70 0x7ff48c108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.397+0000 7ff493a00640 1 --2- 172.21.15.1:0/2005563187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff48c1043c0 0x7ff48c1047a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.397+0000 7ff493a00640 1 -- 172.21.15.1:0/2005563187 >> 172.21.15.1:0/2005563187 conn(0x7ff48c0ffc40 msgr2=0x7ff48c102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.397+0000 7ff493a00640 1 -- 172.21.15.1:0/2005563187 shutdown_connections 2024-04-26T02:01:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.398+0000 7ff493a00640 1 -- 172.21.15.1:0/2005563187 wait complete. 2024-04-26T02:01:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.399+0000 7ff493a00640 1 Processor -- start 2024-04-26T02:01:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.399+0000 7ff493a00640 1 -- start start 2024-04-26T02:01:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.399+0000 7ff493a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff48c1043c0 0x7ff48c1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.399+0000 7ff493a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48c109490 0x7ff48c1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.399+0000 7ff493a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff48c19a7c0 0x7ff48c19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.399+0000 7ff493a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c114420 con 0x7ff48c19a7c0 2024-04-26T02:01:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.399+0000 7ff493a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c1142a0 con 0x7ff48c1043c0 2024-04-26T02:01:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.399+0000 7ff493a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c1145a0 con 0x7ff48c109490 2024-04-26T02:01:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.400+0000 7ff491000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff48c1043c0 0x7ff48c1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.400+0000 7ff48be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48c109490 0x7ff48c1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.400+0000 7ff491000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff48c1043c0 0x7ff48c1a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:50160/0 (socket says 172.21.15.1:50160) 2024-04-26T02:01:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.400+0000 7ff491000640 1 -- 172.21.15.1:0/1779870334 learned_addr learned my addr 172.21.15.1:0/1779870334 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.400+0000 7ff491a00640 1 --2- 172.21.15.1:0/1779870334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff48c19a7c0 0x7ff48c19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.400+0000 7ff491000640 1 -- 172.21.15.1:0/1779870334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48c109490 msgr2=0x7ff48c1a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.400+0000 7ff491000640 1 --2- 172.21.15.1:0/1779870334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48c109490 0x7ff48c1a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.400+0000 7ff491000640 1 -- 172.21.15.1:0/1779870334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff48c19a7c0 msgr2=0x7ff48c19ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.400+0000 7ff491000640 1 --2- 172.21.15.1:0/1779870334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff48c19a7c0 0x7ff48c19ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.400+0000 7ff491000640 1 -- 172.21.15.1:0/1779870334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48c19b350 con 0x7ff48c1043c0 2024-04-26T02:01:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.400+0000 7ff48be00640 1 --2- 172.21.15.1:0/1779870334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48c109490 0x7ff48c1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.400+0000 7ff491000640 1 --2- 172.21.15.1:0/1779870334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff48c1043c0 0x7ff48c1a0650 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff480005ab0 tx=0x7ff480003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.401+0000 7ff489600640 1 -- 172.21.15.1:0/1779870334 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48003f070 con 0x7ff48c1043c0 2024-04-26T02:01:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.401+0000 7ff489600640 1 -- 172.21.15.1:0/1779870334 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff480031b60 con 0x7ff48c1043c0 2024-04-26T02:01:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.401+0000 7ff493a00640 1 -- 172.21.15.1:0/1779870334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff48c19b5e0 con 0x7ff48c1043c0 2024-04-26T02:01:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.401+0000 7ff489600640 1 -- 172.21.15.1:0/1779870334 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480035030 con 0x7ff48c1043c0 2024-04-26T02:01:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.401+0000 7ff493a00640 1 -- 172.21.15.1:0/1779870334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff48c1a77c0 con 0x7ff48c1043c0 2024-04-26T02:01:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.404+0000 7ff489600640 1 -- 172.21.15.1:0/1779870334 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7ff480042070 con 0x7ff48c1043c0 2024-04-26T02:01:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.404+0000 7ff496854740 1 Processor -- start 2024-04-26T02:01:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.405+0000 7ff496854740 1 -- start start 2024-04-26T02:01:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.405+0000 7ff496854740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76565820 0x55da76565c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.405+0000 7ff496854740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76566140 0x55da765704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.405+0000 7ff496854740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76570a00 0x55da76572df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.405+0000 7ff496854740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55da764cb900 con 0x55da76565820 2024-04-26T02:01:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.405+0000 7ff496854740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55da76471ef0 con 0x55da76570a00 2024-04-26T02:01:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.405+0000 7ff496854740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55da764c78c0 con 0x55da76566140 2024-04-26T02:01:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.405+0000 7ff491a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76565820 0x55da76565c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.405+0000 7ff491a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76565820 0x55da76565c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54542/0 (socket says 172.21.15.1:54542) 2024-04-26T02:01:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.405+0000 7ff491a00640 1 -- 172.21.15.1:0/3684595121 learned_addr learned my addr 172.21.15.1:0/3684595121 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.405+0000 7ff491000640 1 --2- 172.21.15.1:0/3684595121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76566140 0x55da765704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.405+0000 7ff48be00640 1 --2- 172.21.15.1:0/3684595121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76570a00 0x55da76572df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.406+0000 7ff491000640 1 -- 172.21.15.1:0/3684595121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76570a00 msgr2=0x55da76572df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.406+0000 7ff491000640 1 --2- 172.21.15.1:0/3684595121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76570a00 0x55da76572df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.406+0000 7ff491000640 1 -- 172.21.15.1:0/3684595121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76565820 msgr2=0x55da76565c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.406+0000 7ff491000640 1 --2- 172.21.15.1:0/3684595121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76565820 0x55da76565c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.406+0000 7ff491000640 1 -- 172.21.15.1:0/3684595121 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da761de210 con 0x55da76566140 2024-04-26T02:01:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.406+0000 7ff491000640 1 --2- 172.21.15.1:0/3684595121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76566140 0x55da765704c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff4800ee5a0 tx=0x7ff4800f91a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=275ab1aec924539b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.406+0000 7ff489600640 1 --2- 172.21.15.1:0/1779870334 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ff4740a3d90 0x7ff4740a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.407+0000 7ff47e000640 1 -- 172.21.15.1:0/3684595121 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4800b9090 con 0x55da76566140 2024-04-26T02:01:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.407+0000 7ff47e000640 1 -- 172.21.15.1:0/3684595121 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4800fe070 con 0x55da76566140 2024-04-26T02:01:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.407+0000 7ff47e000640 1 -- 172.21.15.1:0/3684595121 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4800bc240 con 0x55da76566140 2024-04-26T02:01:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.407+0000 7ff491a00640 1 --2- 172.21.15.1:0/1779870334 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ff4740a3d90 0x7ff4740a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:29.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.407+0000 7ff489600640 1 -- 172.21.15.1:0/1779870334 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7ff4800f0ee0 con 0x7ff48c1043c0 2024-04-26T02:01:29.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.408+0000 7ff491a00640 1 --2- 172.21.15.1:0/1779870334 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ff4740a3d90 0x7ff4740a6250 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff484009fd0 tx=0x7ff484009500 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:29.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.408+0000 7ff496854740 1 -- 172.21.15.1:0/3684595121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76566140 msgr2=0x55da765704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:29.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.408+0000 7ff496854740 1 --2- 172.21.15.1:0/3684595121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76566140 0x55da765704c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff4800ee5a0 tx=0x7ff4800f91a0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.408+0000 7ff496854740 1 -- 172.21.15.1:0/3684595121 shutdown_connections 2024-04-26T02:01:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.408+0000 7ff496854740 1 --2- 172.21.15.1:0/3684595121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76570a00 0x55da76572df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.408+0000 7ff496854740 1 --2- 172.21.15.1:0/3684595121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76566140 0x55da765704c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.408+0000 7ff496854740 1 --2- 172.21.15.1:0/3684595121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76565820 0x55da76565c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.408+0000 7ff496854740 1 -- 172.21.15.1:0/3684595121 >> 172.21.15.1:0/3684595121 conn(0x55da76561d30 msgr2=0x55da76562110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.408+0000 7ff496854740 1 -- 172.21.15.1:0/3684595121 shutdown_connections 2024-04-26T02:01:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.408+0000 7ff496854740 1 -- 172.21.15.1:0/3684595121 wait complete. 2024-04-26T02:01:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.409+0000 7ff496854740 1 -- start start 2024-04-26T02:01:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.409+0000 7ff496854740 1 Processor -- start 2024-04-26T02:01:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.409+0000 7ff496854740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76565820 0x55da7656d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.409+0000 7ff496854740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76566140 0x55da7656de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.409+0000 7ff496854740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76570a00 0x55da7656e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.409+0000 7ff496854740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55da765733f0 con 0x55da76565820 2024-04-26T02:01:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.409+0000 7ff496854740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55da76485310 con 0x55da76570a00 2024-04-26T02:01:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.409+0000 7ff496854740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55da76573570 con 0x55da76566140 2024-04-26T02:01:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.410+0000 7ff496854740 10 client.0 _ll_register_callbacks cb 0x55da76453cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.410+0000 7ff48be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76566140 0x55da7656de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.410+0000 7ff491000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76565820 0x55da7656d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.410+0000 7ff48be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76566140 0x55da7656de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:39662/0 (socket says 172.21.15.1:39662) 2024-04-26T02:01:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.410+0000 7ff48be00640 1 -- 172.21.15.1:0/2145820780 learned_addr learned my addr 172.21.15.1:0/2145820780 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.410+0000 7ff491a00640 1 --2- 172.21.15.1:0/2145820780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76570a00 0x55da7656e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.410+0000 7ff48be00640 1 -- 172.21.15.1:0/2145820780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76570a00 msgr2=0x55da7656e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.410+0000 7ff48be00640 1 --2- 172.21.15.1:0/2145820780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76570a00 0x55da7656e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.410+0000 7ff48be00640 1 -- 172.21.15.1:0/2145820780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76565820 msgr2=0x55da7656d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.410+0000 7ff48be00640 1 --2- 172.21.15.1:0/2145820780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76565820 0x55da7656d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.410+0000 7ff48be00640 1 -- 172.21.15.1:0/2145820780 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da762aa620 con 0x55da76566140 2024-04-26T02:01:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.410+0000 7ff491000640 1 --2- 172.21.15.1:0/2145820780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76565820 0x55da7656d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.411+0000 7ff48be00640 1 --2- 172.21.15.1:0/2145820780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76566140 0x55da7656de00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff478012cd0 tx=0x7ff47800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1676bd30c1231bdc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.411+0000 7ff472a00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T02:01:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.411+0000 7ff472a00640 1 -- 172.21.15.1:0/2145820780 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff478012110 con 0x55da76566140 2024-04-26T02:01:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.411+0000 7ff472a00640 1 -- 172.21.15.1:0/2145820780 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7ff478004030 con 0x55da76566140 2024-04-26T02:01:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.411+0000 7ff496854740 1 -- 172.21.15.1:0/2145820780 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4800fe070 con 0x55da76566140 2024-04-26T02:01:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.411+0000 7ff472a00640 1 -- 172.21.15.1:0/2145820780 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff4780122b0 con 0x55da76566140 2024-04-26T02:01:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.411+0000 7ff496854740 10 client.26374 fetch_fsmap learned FSMap version 32 2024-04-26T02:01:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.411+0000 7ff496854740 1 -- 172.21.15.1:0/2145820780 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55da7657f590 con 0x55da76566140 2024-04-26T02:01:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.412+0000 7ff472a00640 1 -- 172.21.15.1:0/2145820780 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4800fe070 con 0x55da76566140 2024-04-26T02:01:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.412+0000 7ff472a00640 1 -- 172.21.15.1:0/2145820780 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7ff478023070 con 0x55da76566140 2024-04-26T02:01:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.412+0000 7ff496854740 10 client.26374 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:01:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.412+0000 7ff496854740 10 client.26374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:01:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.412+0000 7ff496854740 20 client.26374 populate_metadata read hostname 'smithi001' 2024-04-26T02:01:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.412+0000 7ff496854740 1 --2- 172.21.15.1:0/2145820780 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55da76580a50 0x55da76582e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.412+0000 7ff496854740 4 client.26374 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:01:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.412+0000 7ff496854740 1 -- 172.21.15.1:0/2145820780 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da764df8b0 con 0x55da76580a50 2024-04-26T02:01:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.412+0000 7ff491000640 1 --2- 172.21.15.1:0/2145820780 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55da76580a50 0x55da76582e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.413+0000 7ff491000640 1 --2- 172.21.15.1:0/2145820780 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55da76580a50 0x55da76582e80 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ee5367db9249bc4 server_cookie=44509886fd5a78b2 in_seq=0 out_seq=0 2024-04-26T02:01:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.413+0000 7ff472a00640 10 client.26374 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.414+0000 7ff472a00640 1 -- 172.21.15.1:0/2145820780 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da764df8b0 con 0x55da76580a50 2024-04-26T02:01:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.414+0000 7ff472a00640 10 client.26374 handle_command_reply: tid=0 2024-04-26T02:01:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.414+0000 7ff496854740 1 client.26374 shutdown 2024-04-26T02:01:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.414+0000 7ff496854740 1 -- 172.21.15.1:0/2145820780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76566140 msgr2=0x55da7656de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.414+0000 7ff496854740 1 --2- 172.21.15.1:0/2145820780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76566140 0x55da7656de00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff478012cd0 tx=0x7ff47800f790 comp rx=0 tx=0).stop 2024-04-26T02:01:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.414+0000 7ff496854740 1 -- 172.21.15.1:0/2145820780 shutdown_connections 2024-04-26T02:01:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.414+0000 7ff496854740 1 --2- 172.21.15.1:0/2145820780 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55da76580a50 0x55da76582e80 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.414+0000 7ff496854740 1 --2- 172.21.15.1:0/2145820780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76570a00 0x55da7656e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.414+0000 7ff496854740 1 --2- 172.21.15.1:0/2145820780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76566140 0x55da7656de00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.414+0000 7ff472a00640 0 client.26374 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.414+0000 7ff496854740 1 --2- 172.21.15.1:0/2145820780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76565820 0x55da7656d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.414+0000 7ff496854740 1 -- 172.21.15.1:0/2145820780 >> 172.21.15.1:0/2145820780 conn(0x55da76561d30 msgr2=0x55da76567ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.414+0000 7ff496854740 1 -- 172.21.15.1:0/2145820780 shutdown_connections 2024-04-26T02:01:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.414+0000 7ff496854740 1 -- 172.21.15.1:0/2145820780 wait complete. 2024-04-26T02:01:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.414+0000 7ff496854740 20 client.26374 trim_cache size 0 max 16384 2024-04-26T02:01:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.432+0000 7ff496854740 1 Processor -- start 2024-04-26T02:01:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.432+0000 7ff496854740 1 -- start start 2024-04-26T02:01:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.433+0000 7ff496854740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76565820 0x55da7657c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.433+0000 7ff496854740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76566140 0x55da76588790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.433+0000 7ff496854740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76570a00 0x55da76588cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.433+0000 7ff496854740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c1b27f0 con 0x55da76570a00 2024-04-26T02:01:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.433+0000 7ff496854740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c1b00c0 con 0x55da76566140 2024-04-26T02:01:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.433+0000 7ff496854740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c1adf30 con 0x55da76565820 2024-04-26T02:01:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.433+0000 7ff48be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76566140 0x55da76588790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.433+0000 7ff48be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76566140 0x55da76588790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:50192/0 (socket says 172.21.15.1:50192) 2024-04-26T02:01:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.433+0000 7ff491a00640 1 --2- 172.21.15.1:0/3340325467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76570a00 0x55da76588cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.433+0000 7ff48be00640 1 -- 172.21.15.1:0/3340325467 learned_addr learned my addr 172.21.15.1:0/3340325467 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.433+0000 7ff491000640 1 --2- 172.21.15.1:0/3340325467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76565820 0x55da7657c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.433+0000 7ff491000640 1 -- 172.21.15.1:0/3340325467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76566140 msgr2=0x55da76588790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.433+0000 7ff491000640 1 --2- 172.21.15.1:0/3340325467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76566140 0x55da76588790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.434+0000 7ff491000640 1 -- 172.21.15.1:0/3340325467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76570a00 msgr2=0x55da76588cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.434+0000 7ff491000640 1 --2- 172.21.15.1:0/3340325467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76570a00 0x55da76588cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.434+0000 7ff491000640 1 -- 172.21.15.1:0/3340325467 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da76561b60 con 0x55da76565820 2024-04-26T02:01:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.434+0000 7ff491000640 1 --2- 172.21.15.1:0/3340325467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76565820 0x55da7657c1b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff4800ee5d0 tx=0x7ff480000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d32eab1ded349d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.434+0000 7ff47d600640 1 -- 172.21.15.1:0/3340325467 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480049070 con 0x55da76565820 2024-04-26T02:01:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.434+0000 7ff47d600640 1 -- 172.21.15.1:0/3340325467 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4800fe070 con 0x55da76565820 2024-04-26T02:01:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.434+0000 7ff47d600640 1 -- 172.21.15.1:0/3340325467 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4800bc570 con 0x55da76565820 2024-04-26T02:01:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.435+0000 7ff496854740 1 -- 172.21.15.1:0/3340325467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76565820 msgr2=0x55da7657c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.435+0000 7ff496854740 1 --2- 172.21.15.1:0/3340325467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76565820 0x55da7657c1b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff4800ee5d0 tx=0x7ff480000bc0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.435+0000 7ff496854740 1 -- 172.21.15.1:0/3340325467 shutdown_connections 2024-04-26T02:01:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.435+0000 7ff496854740 1 --2- 172.21.15.1:0/3340325467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76570a00 0x55da76588cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.435+0000 7ff496854740 1 --2- 172.21.15.1:0/3340325467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76566140 0x55da76588790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.435+0000 7ff496854740 1 --2- 172.21.15.1:0/3340325467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76565820 0x55da7657c1b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.435+0000 7ff496854740 1 -- 172.21.15.1:0/3340325467 >> 172.21.15.1:0/3340325467 conn(0x55da76561d30 msgr2=0x55da76568ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.435+0000 7ff496854740 1 -- 172.21.15.1:0/3340325467 shutdown_connections 2024-04-26T02:01:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.435+0000 7ff496854740 1 -- 172.21.15.1:0/3340325467 wait complete. 2024-04-26T02:01:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.436+0000 7ff496854740 1 -- start start 2024-04-26T02:01:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.436+0000 7ff496854740 1 Processor -- start 2024-04-26T02:01:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.436+0000 7ff496854740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76565820 0x55da765786e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.436+0000 7ff496854740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76566140 0x55da76578c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.436+0000 7ff496854740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76570a00 0x55da76575ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.436+0000 7ff496854740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c1ae580 con 0x55da76570a00 2024-04-26T02:01:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.436+0000 7ff496854740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c1af9f0 con 0x55da76566140 2024-04-26T02:01:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.436+0000 7ff496854740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c1aec50 con 0x55da76565820 2024-04-26T02:01:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.437+0000 7ff496854740 10 client.0 _ll_register_callbacks cb 0x7ff478012910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.437+0000 7ff491a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76570a00 0x55da76575ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.437+0000 7ff491a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76570a00 0x55da76575ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54568/0 (socket says 172.21.15.1:54568) 2024-04-26T02:01:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.437+0000 7ff491a00640 1 -- 172.21.15.1:0/3139607916 learned_addr learned my addr 172.21.15.1:0/3139607916 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.437+0000 7ff491000640 1 --2- 172.21.15.1:0/3139607916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76565820 0x55da765786e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.437+0000 7ff48be00640 1 --2- 172.21.15.1:0/3139607916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76566140 0x55da76578c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.437+0000 7ff491a00640 1 -- 172.21.15.1:0/3139607916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76565820 msgr2=0x55da765786e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.437+0000 7ff491a00640 1 --2- 172.21.15.1:0/3139607916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76565820 0x55da765786e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.437+0000 7ff491a00640 1 -- 172.21.15.1:0/3139607916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76566140 msgr2=0x55da76578c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.437+0000 7ff491a00640 1 --2- 172.21.15.1:0/3139607916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76566140 0x55da76578c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.437+0000 7ff491a00640 1 -- 172.21.15.1:0/3139607916 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da76568690 con 0x55da76570a00 2024-04-26T02:01:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.437+0000 7ff48be00640 1 --2- 172.21.15.1:0/3139607916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76566140 0x55da76578c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.438+0000 7ff491a00640 1 --2- 172.21.15.1:0/3139607916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76570a00 0x55da76575ca0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff48401e9b0 tx=0x7ff484004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360e9854982c00ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.438+0000 7ff472a00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T02:01:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.438+0000 7ff472a00640 1 -- 172.21.15.1:0/3139607916 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484004b20 con 0x55da76570a00 2024-04-26T02:01:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.438+0000 7ff472a00640 1 -- 172.21.15.1:0/3139607916 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7ff484004cc0 con 0x55da76570a00 2024-04-26T02:01:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.438+0000 7ff496854740 1 -- 172.21.15.1:0/3139607916 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4800fe070 con 0x55da76570a00 2024-04-26T02:01:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.438+0000 7ff472a00640 1 -- 172.21.15.1:0/3139607916 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff48401d620 con 0x55da76570a00 2024-04-26T02:01:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.438+0000 7ff496854740 10 client.16203 fetch_fsmap learned FSMap version 32 2024-04-26T02:01:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.438+0000 7ff496854740 1 -- 172.21.15.1:0/3139607916 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55da765e6ac0 con 0x55da76570a00 2024-04-26T02:01:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.439+0000 7ff472a00640 1 -- 172.21.15.1:0/3139607916 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4800fe070 con 0x55da76570a00 2024-04-26T02:01:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.439+0000 7ff472a00640 1 -- 172.21.15.1:0/3139607916 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7ff4840053c0 con 0x55da76570a00 2024-04-26T02:01:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.439+0000 7ff496854740 10 client.16203 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:01:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.439+0000 7ff496854740 10 client.16203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:01:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.439+0000 7ff496854740 20 client.16203 populate_metadata read hostname 'smithi001' 2024-04-26T02:01:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.439+0000 7ff496854740 1 --2- 172.21.15.1:0/3139607916 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55da765e72c0 0x55da765e96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.439+0000 7ff496854740 4 client.16203 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:01:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.439+0000 7ff496854740 1 -- 172.21.15.1:0/3139607916 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55da764d9320 con 0x55da765e72c0 2024-04-26T02:01:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.440+0000 7ff491000640 1 --2- 172.21.15.1:0/3139607916 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55da765e72c0 0x55da765e96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.440+0000 7ff491000640 1 --2- 172.21.15.1:0/3139607916 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55da765e72c0 0x55da765e96e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24d19fdd6966f5f2 server_cookie=ec5ad1fb4bb7e821 in_seq=0 out_seq=0 2024-04-26T02:01:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.440+0000 7ff472a00640 10 client.16203 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.450+0000 7ff472a00640 1 -- 172.21.15.1:0/3139607916 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55da764d9320 con 0x55da765e72c0 2024-04-26T02:01:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.450+0000 7ff472a00640 10 client.16203 handle_command_reply: tid=0 2024-04-26T02:01:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.450+0000 7ff496854740 1 client.16203 shutdown 2024-04-26T02:01:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.450+0000 7ff496854740 1 -- 172.21.15.1:0/3139607916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76570a00 msgr2=0x55da76575ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.450+0000 7ff496854740 1 --2- 172.21.15.1:0/3139607916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76570a00 0x55da76575ca0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff48401e9b0 tx=0x7ff484004a40 comp rx=0 tx=0).stop 2024-04-26T02:01:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.450+0000 7ff496854740 1 -- 172.21.15.1:0/3139607916 shutdown_connections 2024-04-26T02:01:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.450+0000 7ff496854740 1 --2- 172.21.15.1:0/3139607916 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55da765e72c0 0x55da765e96e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.450+0000 7ff496854740 1 --2- 172.21.15.1:0/3139607916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da76570a00 0x55da76575ca0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.450+0000 7ff496854740 1 --2- 172.21.15.1:0/3139607916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da76566140 0x55da76578c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.450+0000 7ff496854740 1 --2- 172.21.15.1:0/3139607916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55da76565820 0x55da765786e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.450+0000 7ff496854740 1 -- 172.21.15.1:0/3139607916 >> 172.21.15.1:0/3139607916 conn(0x55da76561d30 msgr2=0x55da7656d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.450+0000 7ff472a00640 0 client.16203 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.451+0000 7ff496854740 1 -- 172.21.15.1:0/3139607916 shutdown_connections 2024-04-26T02:01:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.451+0000 7ff496854740 1 -- 172.21.15.1:0/3139607916 wait complete. 2024-04-26T02:01:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.451+0000 7ff496854740 20 client.16203 trim_cache size 0 max 16384 2024-04-26T02:01:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T02:01:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T02:01:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T02:01:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T02:01:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.452+0000 7ff493a00640 1 -- 172.21.15.1:0/1779870334 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ff4740a3d90 msgr2=0x7ff4740a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.452+0000 7ff493a00640 1 --2- 172.21.15.1:0/1779870334 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ff4740a3d90 0x7ff4740a6250 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff484009fd0 tx=0x7ff484009500 comp rx=0 tx=0).stop 2024-04-26T02:01:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.452+0000 7ff493a00640 1 -- 172.21.15.1:0/1779870334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff48c1043c0 msgr2=0x7ff48c1a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.452+0000 7ff493a00640 1 --2- 172.21.15.1:0/1779870334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff48c1043c0 0x7ff48c1a0650 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff480005ab0 tx=0x7ff480003c10 comp rx=0 tx=0).stop 2024-04-26T02:01:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.452+0000 7ff493a00640 1 -- 172.21.15.1:0/1779870334 shutdown_connections 2024-04-26T02:01:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.452+0000 7ff493a00640 1 --2- 172.21.15.1:0/1779870334 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ff4740a3d90 0x7ff4740a6250 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.452+0000 7ff493a00640 1 --2- 172.21.15.1:0/1779870334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff48c19a7c0 0x7ff48c19ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.452+0000 7ff493a00640 1 --2- 172.21.15.1:0/1779870334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48c109490 0x7ff48c1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.452+0000 7ff493a00640 1 --2- 172.21.15.1:0/1779870334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff48c1043c0 0x7ff48c1a0650 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.452+0000 7ff493a00640 1 -- 172.21.15.1:0/1779870334 >> 172.21.15.1:0/1779870334 conn(0x7ff48c0ffc40 msgr2=0x7ff48c101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.452+0000 7ff493a00640 1 -- 172.21.15.1:0/1779870334 shutdown_connections 2024-04-26T02:01:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:29.452+0000 7ff493a00640 1 -- 172.21.15.1:0/1779870334 wait complete. 2024-04-26T02:01:29.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:29 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1927774952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:01:29.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:29 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1927774952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:01:29.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:29 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1927774952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:01:29.739 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24518 quiesce db / --timeout 60 --expiration 82.1 --await-for 42.0' 2024-04-26T02:01:29.739 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24518 quiesce db / --timeout 60 --expiration 82.1 --await-for 42.0 2024-04-26T02:01:30.265 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:01:30.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:30 smithi173 ceph-mon[32553]: pgmap v307: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 7 op/s 2024-04-26T02:01:30.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:30 smithi001 ceph-mon[27822]: pgmap v307: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 7 op/s 2024-04-26T02:01:30.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:30 smithi097 ceph-mon[32853]: pgmap v307: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 7 op/s 2024-04-26T02:01:31.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:31 smithi173 ceph-mon[32553]: pgmap v308: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 548 KiB/s wr, 200 op/s 2024-04-26T02:01:31.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:31 smithi001 ceph-mon[27822]: pgmap v308: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 548 KiB/s wr, 200 op/s 2024-04-26T02:01:31.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:31 smithi097 ceph-mon[32853]: pgmap v308: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 548 KiB/s wr, 200 op/s 2024-04-26T02:01:32.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.677+0000 7ffb13200700 1 -- 172.21.15.1:0/4153609894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb1410aa90 msgr2=0x7ffb1410ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:32.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.677+0000 7ffb13200700 1 --2- 172.21.15.1:0/4153609894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb1410aa90 0x7ffb1410ae70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ffb04009990 tx=0x7ffb04009ca0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.677+0000 7ffb13200700 1 -- 172.21.15.1:0/4153609894 shutdown_connections 2024-04-26T02:01:32.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.677+0000 7ffb13200700 1 --2- 172.21.15.1:0/4153609894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb14105450 0x7ffb14111960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.677+0000 7ffb13200700 1 --2- 172.21.15.1:0/4153609894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb1410aa90 0x7ffb1410ae70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.677+0000 7ffb13200700 1 --2- 172.21.15.1:0/4153609894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb14104a90 0x7ffb14104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.677+0000 7ffb13200700 1 -- 172.21.15.1:0/4153609894 >> 172.21.15.1:0/4153609894 conn(0x7ffb14100580 msgr2=0x7ffb141029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:32.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.678+0000 7ffb13200700 1 -- 172.21.15.1:0/4153609894 shutdown_connections 2024-04-26T02:01:32.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.678+0000 7ffb13200700 1 -- 172.21.15.1:0/4153609894 wait complete. 2024-04-26T02:01:32.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.679+0000 7ffb13200700 1 Processor -- start 2024-04-26T02:01:32.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.679+0000 7ffb13200700 1 -- start start 2024-04-26T02:01:32.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.680+0000 7ffb13200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb14104a90 0x7ffb1419b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:32.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.680+0000 7ffb13200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb14105450 0x7ffb1419b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:32.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.680+0000 7ffb13200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb1410aa90 0x7ffb1419f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:32.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.680+0000 7ffb13200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffb14114d70 con 0x7ffb14105450 2024-04-26T02:01:32.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.680+0000 7ffb13200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffb14114bf0 con 0x7ffb14104a90 2024-04-26T02:01:32.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.680+0000 7ffb13200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffb14114ef0 con 0x7ffb1410aa90 2024-04-26T02:01:32.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.680+0000 7ffb11e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb14104a90 0x7ffb1419b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:32.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.680+0000 7ffb11e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb14104a90 0x7ffb1419b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:34340/0 (socket says 172.21.15.1:34340) 2024-04-26T02:01:32.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.680+0000 7ffb11e00700 1 -- 172.21.15.1:0/1443741648 learned_addr learned my addr 172.21.15.1:0/1443741648 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:32.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.680+0000 7ffb12800700 1 --2- 172.21.15.1:0/1443741648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb1410aa90 0x7ffb1419f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:32.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.681+0000 7ffb11400700 1 --2- 172.21.15.1:0/1443741648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb14105450 0x7ffb1419b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.681+0000 7ffb11e00700 1 -- 172.21.15.1:0/1443741648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb1410aa90 msgr2=0x7ffb1419f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.681+0000 7ffb11e00700 1 --2- 172.21.15.1:0/1443741648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb1410aa90 0x7ffb1419f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.681+0000 7ffb11e00700 1 -- 172.21.15.1:0/1443741648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb14105450 msgr2=0x7ffb1419b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.681+0000 7ffb11e00700 1 --2- 172.21.15.1:0/1443741648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb14105450 0x7ffb1419b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.681+0000 7ffb11e00700 1 -- 172.21.15.1:0/1443741648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb141a0130 con 0x7ffb14104a90 2024-04-26T02:01:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.681+0000 7ffb11400700 1 --2- 172.21.15.1:0/1443741648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb14105450 0x7ffb1419b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:01:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.681+0000 7ffb11e00700 1 --2- 172.21.15.1:0/1443741648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb14104a90 0x7ffb1419b120 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ffb040098e0 tx=0x7ffb040057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.681+0000 7ffb0aa00700 1 -- 172.21.15.1:0/1443741648 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04026070 con 0x7ffb14104a90 2024-04-26T02:01:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.682+0000 7ffb13200700 1 -- 172.21.15.1:0/1443741648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb141a03c0 con 0x7ffb14104a90 2024-04-26T02:01:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.682+0000 7ffb0aa00700 1 -- 172.21.15.1:0/1443741648 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb04021410 con 0x7ffb14104a90 2024-04-26T02:01:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.682+0000 7ffb0aa00700 1 -- 172.21.15.1:0/1443741648 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04017650 con 0x7ffb14104a90 2024-04-26T02:01:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.682+0000 7ffb13200700 1 -- 172.21.15.1:0/1443741648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb141a7ca0 con 0x7ffb14104a90 2024-04-26T02:01:32.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.687+0000 7ffb234eab80 1 Processor -- start 2024-04-26T02:01:32.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.687+0000 7ffb0aa00700 1 -- 172.21.15.1:0/1443741648 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7ffb04029070 con 0x7ffb14104a90 2024-04-26T02:01:32.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.687+0000 7ffb234eab80 1 -- start start 2024-04-26T02:01:32.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.687+0000 7ffb234eab80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197bdb80 0x55c3197bdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:32.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.687+0000 7ffb234eab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197be4a0 0x55c3197c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:32.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.687+0000 7ffb234eab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197c9350 0x55c3197cb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:32.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.687+0000 7ffb234eab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c319711020 con 0x55c3197c9350 2024-04-26T02:01:32.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.687+0000 7ffb234eab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c3196b78c0 con 0x55c3197be4a0 2024-04-26T02:01:32.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.687+0000 7ffb234eab80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c31970cfe0 con 0x55c3197bdb80 2024-04-26T02:01:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.688+0000 7ffb11e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197be4a0 0x55c3197c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.688+0000 7ffb11e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197be4a0 0x55c3197c8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:34352/0 (socket says 172.21.15.1:34352) 2024-04-26T02:01:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.688+0000 7ffb11e00700 1 -- 172.21.15.1:0/598404565 learned_addr learned my addr 172.21.15.1:0/598404565 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.688+0000 7ffb12800700 1 --2- 172.21.15.1:0/598404565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197bdb80 0x55c3197bdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.688+0000 7ffb11400700 1 --2- 172.21.15.1:0/598404565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197c9350 0x55c3197cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.688+0000 7ffb12800700 1 -- 172.21.15.1:0/598404565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197be4a0 msgr2=0x55c3197c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.688+0000 7ffb12800700 1 --2- 172.21.15.1:0/598404565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197be4a0 0x55c3197c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.688+0000 7ffb12800700 1 -- 172.21.15.1:0/598404565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197c9350 msgr2=0x55c3197cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.688+0000 7ffb12800700 1 --2- 172.21.15.1:0/598404565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197c9350 0x55c3197cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.688+0000 7ffb12800700 1 -- 172.21.15.1:0/598404565 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3196ad810 con 0x55c3197bdb80 2024-04-26T02:01:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.689+0000 7ffb0aa00700 1 --2- 172.21.15.1:0/1443741648 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ffaf80a3ea0 0x7ffaf80a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.689+0000 7ffb12800700 1 --2- 172.21.15.1:0/598404565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197bdb80 0x55c3197bdf60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009f70 tx=0x7ffb0c009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e29e7b909b1fd40 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.689+0000 7ffb0aa00700 1 -- 172.21.15.1:0/1443741648 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7ffb040d1520 con 0x7ffb14104a90 2024-04-26T02:01:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.689+0000 7ffb11e00700 1 --2- 172.21.15.1:0/1443741648 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ffaf80a3ea0 0x7ffaf80a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.689+0000 7ffaff400700 1 -- 172.21.15.1:0/598404565 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c014620 con 0x55c3197bdb80 2024-04-26T02:01:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.689+0000 7ffaff400700 1 -- 172.21.15.1:0/598404565 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb0c019070 con 0x55c3197bdb80 2024-04-26T02:01:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.689+0000 7ffaff400700 1 -- 172.21.15.1:0/598404565 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c013980 con 0x55c3197bdb80 2024-04-26T02:01:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.690+0000 7ffb11e00700 1 --2- 172.21.15.1:0/1443741648 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ffaf80a3ea0 0x7ffaf80a6360 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffb04009ce0 tx=0x7ffb04021f90 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.690+0000 7ffb234eab80 1 -- 172.21.15.1:0/598404565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197bdb80 msgr2=0x55c3197bdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.690+0000 7ffb234eab80 1 --2- 172.21.15.1:0/598404565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197bdb80 0x55c3197bdf60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009f70 tx=0x7ffb0c009650 comp rx=0 tx=0).stop 2024-04-26T02:01:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.690+0000 7ffb234eab80 1 -- 172.21.15.1:0/598404565 shutdown_connections 2024-04-26T02:01:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.690+0000 7ffb234eab80 1 --2- 172.21.15.1:0/598404565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197c9350 0x55c3197cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.690+0000 7ffb234eab80 1 --2- 172.21.15.1:0/598404565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197bdb80 0x55c3197bdf60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.690+0000 7ffb234eab80 1 --2- 172.21.15.1:0/598404565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197be4a0 0x55c3197c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.690+0000 7ffb234eab80 1 -- 172.21.15.1:0/598404565 >> 172.21.15.1:0/598404565 conn(0x55c3196ec730 msgr2=0x55c3197b9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.691+0000 7ffb234eab80 1 -- 172.21.15.1:0/598404565 shutdown_connections 2024-04-26T02:01:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.691+0000 7ffb234eab80 1 -- 172.21.15.1:0/598404565 wait complete. 2024-04-26T02:01:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.691+0000 7ffb234eab80 1 -- start start 2024-04-26T02:01:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.691+0000 7ffb234eab80 1 Processor -- start 2024-04-26T02:01:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.692+0000 7ffb234eab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197bdb80 0x55c3197c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:32.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.692+0000 7ffb234eab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197be4a0 0x55c3197c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:32.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.692+0000 7ffb234eab80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197c9350 0x55c3197c6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:32.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.692+0000 7ffb234eab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c3197cbed0 con 0x55c3197be4a0 2024-04-26T02:01:32.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.692+0000 7ffb234eab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c3196ca9f0 con 0x55c3197bdb80 2024-04-26T02:01:32.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.692+0000 7ffb234eab80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c3197cc050 con 0x55c3197c9350 2024-04-26T02:01:32.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.692+0000 7ffb234eab80 10 client.0 _ll_register_callbacks cb 0x55c3196a98b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:32.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.692+0000 7ffb12800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197c9350 0x55c3197c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:32.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.692+0000 7ffb11400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197be4a0 0x55c3197c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:32.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.692+0000 7ffb11400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197be4a0 0x55c3197c5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49764/0 (socket says 172.21.15.1:49764) 2024-04-26T02:01:32.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.692+0000 7ffb12800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197c9350 0x55c3197c6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:54950/0 (socket says 172.21.15.1:54950) 2024-04-26T02:01:32.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.693+0000 7ffb11400700 1 -- 172.21.15.1:0/1349893318 learned_addr learned my addr 172.21.15.1:0/1349893318 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:32.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.693+0000 7ffb12800700 1 --2- 172.21.15.1:0/1349893318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197bdb80 0x55c3197c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:32.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.693+0000 7ffb11400700 1 -- 172.21.15.1:0/1349893318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197c9350 msgr2=0x55c3197c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:32.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.693+0000 7ffb11400700 1 --2- 172.21.15.1:0/1349893318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197c9350 0x55c3197c6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.693+0000 7ffb11400700 1 -- 172.21.15.1:0/1349893318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197bdb80 msgr2=0x55c3197c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:32.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.693+0000 7ffb11400700 1 --2- 172.21.15.1:0/1349893318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197bdb80 0x55c3197c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.693+0000 7ffb11400700 1 -- 172.21.15.1:0/1349893318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c31965ea60 con 0x55c3197be4a0 2024-04-26T02:01:32.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.693+0000 7ffb11400700 1 --2- 172.21.15.1:0/1349893318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197be4a0 0x55c3197c5dd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009d40 tx=0x7ffb00003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b26455a6e8fc659 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:32.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.693+0000 7ffaf3e00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T02:01:32.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.694+0000 7ffaf3e00700 1 -- 172.21.15.1:0/1349893318 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb00012650 con 0x55c3197be4a0 2024-04-26T02:01:32.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.694+0000 7ffaf3e00700 1 -- 172.21.15.1:0/1349893318 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7ffb000127f0 con 0x55c3197be4a0 2024-04-26T02:01:32.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.694+0000 7ffb234eab80 1 -- 172.21.15.1:0/1349893318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb0c019070 con 0x55c3197be4a0 2024-04-26T02:01:32.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.694+0000 7ffaf3e00700 1 -- 172.21.15.1:0/1349893318 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ffb00012a30 con 0x55c3197be4a0 2024-04-26T02:01:32.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.694+0000 7ffb234eab80 10 client.16212 fetch_fsmap learned FSMap version 32 2024-04-26T02:01:32.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.694+0000 7ffb234eab80 1 -- 172.21.15.1:0/1349893318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c3196a7c80 con 0x55c3197be4a0 2024-04-26T02:01:32.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.694+0000 7ffaf3e00700 1 -- 172.21.15.1:0/1349893318 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb0c019070 con 0x55c3197be4a0 2024-04-26T02:01:32.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.695+0000 7ffaf3e00700 1 -- 172.21.15.1:0/1349893318 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7ffb0001a020 con 0x55c3197be4a0 2024-04-26T02:01:32.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.695+0000 7ffb234eab80 10 client.16212 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:01:32.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.695+0000 7ffb234eab80 10 client.16212 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T02:01:32.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.695+0000 7ffb234eab80 20 client.16212 populate_metadata read hostname 'smithi001' 2024-04-26T02:01:32.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.695+0000 7ffb234eab80 1 --2- 172.21.15.1:0/1349893318 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c3197d8c70 0x55c3197db060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:32.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.695+0000 7ffb234eab80 4 client.16212 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:01:32.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.695+0000 7ffb234eab80 1 -- 172.21.15.1:0/1349893318 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c31971e810 con 0x55c3197d8c70 2024-04-26T02:01:32.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.695+0000 7ffb12800700 1 --2- 172.21.15.1:0/1349893318 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c3197d8c70 0x55c3197db060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:32.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.696+0000 7ffb12800700 1 --2- 172.21.15.1:0/1349893318 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c3197d8c70 0x55c3197db060 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=641ca30a17b6edd3 server_cookie=1f071ef01ce46480 in_seq=0 out_seq=0 2024-04-26T02:01:32.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.696+0000 7ffaf3e00700 10 client.16212 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:32.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.697+0000 7ffaf3e00700 1 -- 172.21.15.1:0/1349893318 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c31971e810 con 0x55c3197d8c70 2024-04-26T02:01:32.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.697+0000 7ffaf3e00700 10 client.16212 handle_command_reply: tid=0 2024-04-26T02:01:32.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.697+0000 7ffb234eab80 1 client.16212 shutdown 2024-04-26T02:01:32.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.697+0000 7ffb234eab80 1 -- 172.21.15.1:0/1349893318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197be4a0 msgr2=0x55c3197c5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:32.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.697+0000 7ffb234eab80 1 --2- 172.21.15.1:0/1349893318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197be4a0 0x55c3197c5dd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009d40 tx=0x7ffb00003960 comp rx=0 tx=0).stop 2024-04-26T02:01:32.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.697+0000 7ffb234eab80 1 -- 172.21.15.1:0/1349893318 shutdown_connections 2024-04-26T02:01:32.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.697+0000 7ffb234eab80 1 --2- 172.21.15.1:0/1349893318 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c3197d8c70 0x55c3197db060 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.697+0000 7ffb234eab80 1 --2- 172.21.15.1:0/1349893318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197c9350 0x55c3197c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.697+0000 7ffb234eab80 1 --2- 172.21.15.1:0/1349893318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197bdb80 0x55c3197c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.697+0000 7ffb234eab80 1 --2- 172.21.15.1:0/1349893318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197be4a0 0x55c3197c5dd0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.697+0000 7ffaf3e00700 0 client.16212 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:32.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.697+0000 7ffb234eab80 1 -- 172.21.15.1:0/1349893318 >> 172.21.15.1:0/1349893318 conn(0x55c3196ec730 msgr2=0x55c3197bfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:32.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.698+0000 7ffb234eab80 1 -- 172.21.15.1:0/1349893318 shutdown_connections 2024-04-26T02:01:32.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.698+0000 7ffb234eab80 1 -- 172.21.15.1:0/1349893318 wait complete. 2024-04-26T02:01:32.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.698+0000 7ffb234eab80 20 client.16212 trim_cache size 0 max 16384 2024-04-26T02:01:32.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.723+0000 7ffb234eab80 1 Processor -- start 2024-04-26T02:01:32.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.723+0000 7ffb234eab80 1 -- start start 2024-04-26T02:01:32.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.723+0000 7ffb234eab80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197bdb80 0x55c3197b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:32.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.723+0000 7ffb234eab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197be4a0 0x55c3197d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:32.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.723+0000 7ffb234eab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197c9350 0x55c3197d1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:32.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.723+0000 7ffb234eab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141b1b70 con 0x55c3197c9350 2024-04-26T02:01:32.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.723+0000 7ffb234eab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141af400 con 0x55c3197be4a0 2024-04-26T02:01:32.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.723+0000 7ffb234eab80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141ad270 con 0x55c3197bdb80 2024-04-26T02:01:32.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.724+0000 7ffb11400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197be4a0 0x55c3197d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:32.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.724+0000 7ffb11400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197be4a0 0x55c3197d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:34360/0 (socket says 172.21.15.1:34360) 2024-04-26T02:01:32.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.724+0000 7ffb11400700 1 -- 172.21.15.1:0/2649361600 learned_addr learned my addr 172.21.15.1:0/2649361600 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:32.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.724+0000 7ffb12800700 1 --2- 172.21.15.1:0/2649361600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197bdb80 0x55c3197b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:32.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.724+0000 7ffb12800700 1 --2- 172.21.15.1:0/2649361600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197c9350 0x55c3197d1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:32.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.724+0000 7ffb12800700 1 -- 172.21.15.1:0/2649361600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197be4a0 msgr2=0x55c3197d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:32.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.724+0000 7ffb12800700 1 --2- 172.21.15.1:0/2649361600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197be4a0 0x55c3197d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.724+0000 7ffb12800700 1 -- 172.21.15.1:0/2649361600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197c9350 msgr2=0x55c3197d1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:32.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.724+0000 7ffb12800700 1 --2- 172.21.15.1:0/2649361600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197c9350 0x55c3197d1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.724+0000 7ffb12800700 1 -- 172.21.15.1:0/2649361600 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3197db6c0 con 0x55c3197bdb80 2024-04-26T02:01:32.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.725+0000 7ffb12800700 1 --2- 172.21.15.1:0/2649361600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197bdb80 0x55c3197b3310 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55c3197db8f0 tx=0x7ffb0c00d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f14ce89f9b48a641 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:32.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.725+0000 7ffafea00700 1 -- 172.21.15.1:0/2649361600 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c013af0 con 0x55c3197bdb80 2024-04-26T02:01:32.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.725+0000 7ffafea00700 1 -- 172.21.15.1:0/2649361600 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb0c019070 con 0x55c3197bdb80 2024-04-26T02:01:32.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.725+0000 7ffafea00700 1 -- 172.21.15.1:0/2649361600 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0144f0 con 0x55c3197bdb80 2024-04-26T02:01:32.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.726+0000 7ffb234eab80 1 -- 172.21.15.1:0/2649361600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197bdb80 msgr2=0x55c3197b3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:32.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.726+0000 7ffb234eab80 1 --2- 172.21.15.1:0/2649361600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197bdb80 0x55c3197b3310 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55c3197db8f0 tx=0x7ffb0c00d040 comp rx=0 tx=0).stop 2024-04-26T02:01:32.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.726+0000 7ffb234eab80 1 -- 172.21.15.1:0/2649361600 shutdown_connections 2024-04-26T02:01:32.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.726+0000 7ffb234eab80 1 --2- 172.21.15.1:0/2649361600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197c9350 0x55c3197d1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.726+0000 7ffb234eab80 1 --2- 172.21.15.1:0/2649361600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197bdb80 0x55c3197b3310 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.726+0000 7ffb234eab80 1 --2- 172.21.15.1:0/2649361600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197be4a0 0x55c3197d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.726+0000 7ffb234eab80 1 -- 172.21.15.1:0/2649361600 >> 172.21.15.1:0/2649361600 conn(0x55c3196ec730 msgr2=0x55c3197d3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:32.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.726+0000 7ffb234eab80 1 -- 172.21.15.1:0/2649361600 shutdown_connections 2024-04-26T02:01:32.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.726+0000 7ffb234eab80 1 -- 172.21.15.1:0/2649361600 wait complete. 2024-04-26T02:01:32.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.727+0000 7ffb234eab80 1 -- start start 2024-04-26T02:01:32.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.727+0000 7ffb234eab80 1 Processor -- start 2024-04-26T02:01:32.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.727+0000 7ffb234eab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197bdb80 0x55c319838580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:32.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.727+0000 7ffb234eab80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197be4a0 0x55c319838ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:32.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.727+0000 7ffb234eab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197c9350 0x55c3197d06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:32.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.727+0000 7ffb234eab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141ad8c0 con 0x55c3197c9350 2024-04-26T02:01:32.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.727+0000 7ffb234eab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141aed30 con 0x55c3197bdb80 2024-04-26T02:01:32.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.727+0000 7ffb234eab80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141adf90 con 0x55c3197be4a0 2024-04-26T02:01:32.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.727+0000 7ffb234eab80 10 client.0 _ll_register_callbacks cb 0x55c3196a98b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:32.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.728+0000 7ffb11400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197be4a0 0x55c319838ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:32.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.728+0000 7ffb11400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197be4a0 0x55c319838ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:54956/0 (socket says 172.21.15.1:54956) 2024-04-26T02:01:32.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.728+0000 7ffb11400700 1 -- 172.21.15.1:0/2340567103 learned_addr learned my addr 172.21.15.1:0/2340567103 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:32.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.728+0000 7ffb12800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197bdb80 0x55c319838580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:32.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.728+0000 7ffb12800700 1 --2- 172.21.15.1:0/2340567103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197c9350 0x55c3197d06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:32.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.728+0000 7ffb11400700 1 -- 172.21.15.1:0/2340567103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197bdb80 msgr2=0x55c319838580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:32.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.728+0000 7ffb11400700 1 --2- 172.21.15.1:0/2340567103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197bdb80 0x55c319838580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.728+0000 7ffb11400700 1 -- 172.21.15.1:0/2340567103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197c9350 msgr2=0x55c3197d06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:32.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.728+0000 7ffb11400700 1 --2- 172.21.15.1:0/2340567103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197c9350 0x55c3197d06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:32.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.728+0000 7ffb11400700 1 -- 172.21.15.1:0/2340567103 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3197d0dc0 con 0x55c3197be4a0 2024-04-26T02:01:32.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.728+0000 7ffb12800700 1 --2- 172.21.15.1:0/2340567103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197c9350 0x55c3197d06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:01:32.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.729+0000 7ffb11400700 1 --2- 172.21.15.1:0/2340567103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197be4a0 0x55c319838ac0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ffb0001ce90 tx=0x7ffb00003fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6506651df999448b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:32.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.729+0000 7ffaf3e00700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T02:01:32.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.729+0000 7ffaf3e00700 1 -- 172.21.15.1:0/2340567103 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb00033060 con 0x55c3197be4a0 2024-04-26T02:01:32.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.729+0000 7ffb234eab80 1 -- 172.21.15.1:0/2340567103 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb0c019070 con 0x55c3197be4a0 2024-04-26T02:01:32.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.729+0000 7ffaf3e00700 1 -- 172.21.15.1:0/2340567103 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7ffb00012450 con 0x55c3197be4a0 2024-04-26T02:01:32.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.729+0000 7ffaf3e00700 1 -- 172.21.15.1:0/2340567103 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ffb00028040 con 0x55c3197be4a0 2024-04-26T02:01:32.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.729+0000 7ffb234eab80 10 client.26398 fetch_fsmap learned FSMap version 32 2024-04-26T02:01:32.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.729+0000 7ffb234eab80 1 -- 172.21.15.1:0/2340567103 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c319840e70 con 0x55c3197be4a0 2024-04-26T02:01:32.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.730+0000 7ffaf3e00700 1 -- 172.21.15.1:0/2340567103 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb0c019070 con 0x55c3197be4a0 2024-04-26T02:01:32.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.730+0000 7ffaf3e00700 1 -- 172.21.15.1:0/2340567103 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7ffb0001a040 con 0x55c3197be4a0 2024-04-26T02:01:32.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.730+0000 7ffb234eab80 10 client.26398 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:01:32.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.730+0000 7ffb234eab80 10 client.26398 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T02:01:32.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.730+0000 7ffb234eab80 20 client.26398 populate_metadata read hostname 'smithi001' 2024-04-26T02:01:32.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.730+0000 7ffb234eab80 1 --2- 172.21.15.1:0/2340567103 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c319841650 0x55c319843a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:32.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.730+0000 7ffb234eab80 4 client.26398 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 82.1, "await_for": 42.0}] 2024-04-26T02:01:32.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.730+0000 7ffb234eab80 1 -- 172.21.15.1:0/2340567103 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 82.1, "await_for": 42.0}) -- 0x55c3197188b0 con 0x55c319841650 2024-04-26T02:01:32.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.731+0000 7ffb12800700 1 --2- 172.21.15.1:0/2340567103 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c319841650 0x55c319843a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:32.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.731+0000 7ffb12800700 1 --2- 172.21.15.1:0/2340567103 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c319841650 0x55c319843a70 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c51c6f80876d793 server_cookie=2b8c03913c4ceae1 in_seq=0 out_seq=0 2024-04-26T02:01:32.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:32.731+0000 7ffaf3e00700 10 client.26398 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:33.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.628+0000 7ffaf3e00700 1 -- 172.21.15.1:0/2340567103 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55c3197188b0 con 0x55c319841650 2024-04-26T02:01:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.628+0000 7ffaf3e00700 10 client.26398 handle_command_reply: tid=0 2024-04-26T02:01:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.628+0000 7ffb234eab80 1 client.26398 shutdown 2024-04-26T02:01:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.629+0000 7ffb234eab80 1 -- 172.21.15.1:0/2340567103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197be4a0 msgr2=0x55c319838ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.629+0000 7ffb234eab80 1 --2- 172.21.15.1:0/2340567103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197be4a0 0x55c319838ac0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ffb0001ce90 tx=0x7ffb00003fe0 comp rx=0 tx=0).stop 2024-04-26T02:01:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.629+0000 7ffb234eab80 1 -- 172.21.15.1:0/2340567103 shutdown_connections 2024-04-26T02:01:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.629+0000 7ffb234eab80 1 --2- 172.21.15.1:0/2340567103 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c319841650 0x55c319843a70 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.629+0000 7ffb234eab80 1 --2- 172.21.15.1:0/2340567103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3197c9350 0x55c3197d06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.629+0000 7ffb234eab80 1 --2- 172.21.15.1:0/2340567103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3197bdb80 0x55c319838580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.629+0000 7ffaf3e00700 0 client.26398 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.629+0000 7ffb234eab80 1 --2- 172.21.15.1:0/2340567103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3197be4a0 0x55c319838ac0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.629+0000 7ffb234eab80 1 -- 172.21.15.1:0/2340567103 >> 172.21.15.1:0/2340567103 conn(0x55c3196ec730 msgr2=0x55c3197d4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.630+0000 7ffb234eab80 1 -- 172.21.15.1:0/2340567103 shutdown_connections 2024-04-26T02:01:33.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.630+0000 7ffb234eab80 1 -- 172.21.15.1:0/2340567103 wait complete. 2024-04-26T02:01:33.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.630+0000 7ffb234eab80 20 client.26398 trim_cache size 0 max 16384 2024-04-26T02:01:33.631 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T02:01:33.631 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 32, 2024-04-26T02:01:33.631 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24518, 2024-04-26T02:01:33.631 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 30, 2024-04-26T02:01:33.632 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-04-26T02:01:33.632 INFO:teuthology.orchestra.run.smithi001.stdout: "b515cac0": { 2024-04-26T02:01:33.632 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 30, 2024-04-26T02:01:33.632 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-04-26T02:01:33.632 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T02:01:33.632 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-04-26T02:01:33.633 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T02:01:33.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:01:33.633 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-04-26T02:01:33.633 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 82.1, 2024-04-26T02:01:33.633 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-04-26T02:01:33.633 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-04-26T02:01:33.633 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-04-26T02:01:33.633 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T02:01:33.633 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-04-26T02:01:33.633 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T02:01:33.633 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:01:33.633 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:01:33.633 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:01:33.634 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:01:33.634 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:01:33.634 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T02:01:33.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.632+0000 7ffb13200700 1 -- 172.21.15.1:0/1443741648 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ffaf80a3ea0 msgr2=0x7ffaf80a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:33.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.632+0000 7ffb13200700 1 --2- 172.21.15.1:0/1443741648 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ffaf80a3ea0 0x7ffaf80a6360 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffb04009ce0 tx=0x7ffb04021f90 comp rx=0 tx=0).stop 2024-04-26T02:01:33.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.632+0000 7ffb13200700 1 -- 172.21.15.1:0/1443741648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb14104a90 msgr2=0x7ffb1419b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:33.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.632+0000 7ffb13200700 1 --2- 172.21.15.1:0/1443741648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb14104a90 0x7ffb1419b120 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ffb040098e0 tx=0x7ffb040057c0 comp rx=0 tx=0).stop 2024-04-26T02:01:33.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.632+0000 7ffb13200700 1 -- 172.21.15.1:0/1443741648 shutdown_connections 2024-04-26T02:01:33.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.632+0000 7ffb13200700 1 --2- 172.21.15.1:0/1443741648 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7ffaf80a3ea0 0x7ffaf80a6360 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:33.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.632+0000 7ffb13200700 1 --2- 172.21.15.1:0/1443741648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb1410aa90 0x7ffb1419f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:33.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.632+0000 7ffb13200700 1 --2- 172.21.15.1:0/1443741648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb14104a90 0x7ffb1419b120 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:33.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.632+0000 7ffb13200700 1 --2- 172.21.15.1:0/1443741648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb14105450 0x7ffb1419b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:33.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.632+0000 7ffb13200700 1 -- 172.21.15.1:0/1443741648 >> 172.21.15.1:0/1443741648 conn(0x7ffb14100580 msgr2=0x7ffb14100a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:33.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.633+0000 7ffb13200700 1 -- 172.21.15.1:0/1443741648 shutdown_connections 2024-04-26T02:01:33.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:33.633+0000 7ffb13200700 1 -- 172.21.15.1:0/1443741648 wait complete. 2024-04-26T02:01:34.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:34 smithi173 ceph-mon[32553]: pgmap v309: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 541 KiB/s wr, 195 op/s 2024-04-26T02:01:34.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:34 smithi001 ceph-mon[27822]: pgmap v309: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 541 KiB/s wr, 195 op/s 2024-04-26T02:01:34.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:34 smithi097 ceph-mon[32853]: pgmap v309: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 541 KiB/s wr, 195 op/s 2024-04-26T02:01:34.492 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b515cac0', quiesce took 8.6 seconds. Will release after: 22.1 2024-04-26T02:01:36.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:36 smithi173 ceph-mon[32553]: pgmap v310: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 850 KiB/s wr, 312 op/s 2024-04-26T02:01:36.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:36 smithi001 ceph-mon[27822]: pgmap v310: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 850 KiB/s wr, 312 op/s 2024-04-26T02:01:36.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:36 smithi097 ceph-mon[32853]: pgmap v310: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 850 KiB/s wr, 312 op/s 2024-04-26T02:01:38.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:38 smithi173 ceph-mon[32553]: pgmap v311: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 850 KiB/s wr, 310 op/s 2024-04-26T02:01:38.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:38 smithi001 ceph-mon[27822]: pgmap v311: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 850 KiB/s wr, 310 op/s 2024-04-26T02:01:38.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:38 smithi097 ceph-mon[32853]: pgmap v311: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 850 KiB/s wr, 310 op/s 2024-04-26T02:01:40.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:40 smithi173 ceph-mon[32553]: pgmap v312: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 850 KiB/s wr, 310 op/s 2024-04-26T02:01:40.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:40 smithi001 ceph-mon[27822]: pgmap v312: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 850 KiB/s wr, 310 op/s 2024-04-26T02:01:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:40 smithi097 ceph-mon[32853]: pgmap v312: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 850 KiB/s wr, 310 op/s 2024-04-26T02:01:42.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:42 smithi173 ceph-mon[32553]: pgmap v313: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 852 KiB/s wr, 313 op/s 2024-04-26T02:01:42.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:42 smithi001 ceph-mon[27822]: pgmap v313: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 852 KiB/s wr, 313 op/s 2024-04-26T02:01:42.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:42 smithi097 ceph-mon[32853]: pgmap v313: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 852 KiB/s wr, 313 op/s 2024-04-26T02:01:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.604+0000 7efcb3e00640 1 -- 172.21.15.1:0/2779859362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcb41014e0 msgr2=0x7efcb410e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.604+0000 7efcb3e00640 1 --2- 172.21.15.1:0/2779859362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcb41014e0 0x7efcb410e110 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009900 tx=0x7efc9c02a410 comp rx=0 tx=0).stop 2024-04-26T02:01:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.604+0000 7efcb3e00640 1 -- 172.21.15.1:0/2779859362 shutdown_connections 2024-04-26T02:01:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.604+0000 7efcb3e00640 1 --2- 172.21.15.1:0/2779859362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcb410e6e0 0x7efcb4110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.604+0000 7efcb3e00640 1 --2- 172.21.15.1:0/2779859362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcb41014e0 0x7efcb410e110 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.604+0000 7efcb3e00640 1 --2- 172.21.15.1:0/2779859362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efcb4100b30 0x7efcb4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.604+0000 7efcb3e00640 1 -- 172.21.15.1:0/2779859362 >> 172.21.15.1:0/2779859362 conn(0x7efcb40fc760 msgr2=0x7efcb40feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.604+0000 7efcb3e00640 1 -- 172.21.15.1:0/2779859362 shutdown_connections 2024-04-26T02:01:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.604+0000 7efcb3e00640 1 -- 172.21.15.1:0/2779859362 wait complete. 2024-04-26T02:01:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.605+0000 7efcb3e00640 1 Processor -- start 2024-04-26T02:01:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.605+0000 7efcb3e00640 1 -- start start 2024-04-26T02:01:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.606+0000 7efcb3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcb4100b30 0x7efcb41a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.606+0000 7efcb3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efcb41014e0 0x7efcb41a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.606+0000 7efcb3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcb410e6e0 0x7efcb419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.606+0000 7efcb3e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4114210 con 0x7efcb4100b30 2024-04-26T02:01:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.606+0000 7efcb3e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4114090 con 0x7efcb410e6e0 2024-04-26T02:01:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.606+0000 7efcb3e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4114390 con 0x7efcb41014e0 2024-04-26T02:01:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.606+0000 7efcb2a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcb4100b30 0x7efcb41a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.606+0000 7efcb2a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcb4100b30 0x7efcb41a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49804/0 (socket says 172.21.15.1:49804) 2024-04-26T02:01:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.606+0000 7efcb2a00640 1 -- 172.21.15.1:0/697580624 learned_addr learned my addr 172.21.15.1:0/697580624 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.606+0000 7efcaa000640 1 --2- 172.21.15.1:0/697580624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efcb41014e0 0x7efcb41a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.606+0000 7efcb3400640 1 --2- 172.21.15.1:0/697580624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcb410e6e0 0x7efcb419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.606+0000 7efcb2a00640 1 -- 172.21.15.1:0/697580624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efcb41014e0 msgr2=0x7efcb41a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.606+0000 7efcb2a00640 1 --2- 172.21.15.1:0/697580624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efcb41014e0 0x7efcb41a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.606+0000 7efcb2a00640 1 -- 172.21.15.1:0/697580624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcb410e6e0 msgr2=0x7efcb419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.606+0000 7efcb2a00640 1 --2- 172.21.15.1:0/697580624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcb410e6e0 0x7efcb419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.606+0000 7efcb2a00640 1 -- 172.21.15.1:0/697580624 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb419ada0 con 0x7efcb4100b30 2024-04-26T02:01:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.607+0000 7efcb2a00640 1 --2- 172.21.15.1:0/697580624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcb4100b30 0x7efcb41a0470 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efca400d950 tx=0x7efca400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.607+0000 7efcabe00640 1 -- 172.21.15.1:0/697580624 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4017030 con 0x7efcb4100b30 2024-04-26T02:01:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.607+0000 7efcb3e00640 1 -- 172.21.15.1:0/697580624 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb419b090 con 0x7efcb4100b30 2024-04-26T02:01:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.607+0000 7efcabe00640 1 -- 172.21.15.1:0/697580624 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efca4004020 con 0x7efcb4100b30 2024-04-26T02:01:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.607+0000 7efcabe00640 1 -- 172.21.15.1:0/697580624 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4005640 con 0x7efcb4100b30 2024-04-26T02:01:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.607+0000 7efcb3e00640 1 -- 172.21.15.1:0/697580624 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb41a7330 con 0x7efcb4100b30 2024-04-26T02:01:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.612+0000 7efcbc7a7740 1 Processor -- start 2024-04-26T02:01:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.612+0000 7efcbc7a7740 1 -- start start 2024-04-26T02:01:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.612+0000 7efcabe00640 1 -- 172.21.15.1:0/697580624 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7efca4009a30 con 0x7efcb4100b30 2024-04-26T02:01:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.612+0000 7efcbc7a7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad7944d9b0 0x55ad7944dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.612+0000 7efcbc7a7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 0x55ad79458650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.612+0000 7efcbc7a7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad79458b90 0x55ad7945af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.612+0000 7efcbc7a7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ad793b3900 con 0x55ad79458b90 2024-04-26T02:01:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.612+0000 7efcbc7a7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ad79359ef0 con 0x55ad7944d9b0 2024-04-26T02:01:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.612+0000 7efcbc7a7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ad793af8c0 con 0x55ad7944e2d0 2024-04-26T02:01:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.613+0000 7efcaa000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad79458b90 0x55ad7945af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.613+0000 7efcb2a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 0x55ad79458650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.613+0000 7efcb3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad7944d9b0 0x55ad7944dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.613+0000 7efcb3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad7944d9b0 0x55ad7944dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:41730/0 (socket says 172.21.15.1:41730) 2024-04-26T02:01:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.613+0000 7efcb2a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 0x55ad79458650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:42824/0 (socket says 172.21.15.1:42824) 2024-04-26T02:01:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.613+0000 7efcb3400640 1 -- 172.21.15.1:0/2456599348 learned_addr learned my addr 172.21.15.1:0/2456599348 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.613+0000 7efcb3400640 1 -- 172.21.15.1:0/2456599348 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 msgr2=0x55ad79458650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.613+0000 7efcb3400640 1 --2- 172.21.15.1:0/2456599348 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 0x55ad79458650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.613+0000 7efcb3400640 1 -- 172.21.15.1:0/2456599348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad79458b90 msgr2=0x55ad7945af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.613+0000 7efcb3400640 1 --2- 172.21.15.1:0/2456599348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad79458b90 0x55ad7945af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.613+0000 7efcb3400640 1 -- 172.21.15.1:0/2456599348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad790c6210 con 0x55ad7944d9b0 2024-04-26T02:01:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.614+0000 7efcb3400640 1 --2- 172.21.15.1:0/2456599348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad7944d9b0 0x55ad7944dd90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efcac009f30 tx=0x7efcac0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=400243ae294ae28d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.614+0000 7efca3e00640 1 -- 172.21.15.1:0/2456599348 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac004d20 con 0x55ad7944d9b0 2024-04-26T02:01:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.614+0000 7efca3e00640 1 -- 172.21.15.1:0/2456599348 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcac004050 con 0x55ad7944d9b0 2024-04-26T02:01:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.614+0000 7efca3e00640 1 -- 172.21.15.1:0/2456599348 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac005260 con 0x55ad7944d9b0 2024-04-26T02:01:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.614+0000 7efcabe00640 1 --2- 172.21.15.1:0/697580624 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efc980a3ea0 0x7efc980a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.614+0000 7efcabe00640 1 -- 172.21.15.1:0/697580624 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7efca40d0160 con 0x7efcb4100b30 2024-04-26T02:01:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.614+0000 7efcb2a00640 1 --2- 172.21.15.1:0/697580624 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efc980a3ea0 0x7efc980a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.615+0000 7efcbc7a7740 1 -- 172.21.15.1:0/2456599348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad7944d9b0 msgr2=0x55ad7944dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.615+0000 7efcbc7a7740 1 --2- 172.21.15.1:0/2456599348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad7944d9b0 0x55ad7944dd90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efcac009f30 tx=0x7efcac0118d0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.615+0000 7efcbc7a7740 1 -- 172.21.15.1:0/2456599348 shutdown_connections 2024-04-26T02:01:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.615+0000 7efcbc7a7740 1 --2- 172.21.15.1:0/2456599348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad79458b90 0x55ad7945af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.615+0000 7efcbc7a7740 1 --2- 172.21.15.1:0/2456599348 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 0x55ad79458650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.615+0000 7efcbc7a7740 1 --2- 172.21.15.1:0/2456599348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad7944d9b0 0x55ad7944dd90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.615+0000 7efcbc7a7740 1 -- 172.21.15.1:0/2456599348 >> 172.21.15.1:0/2456599348 conn(0x55ad79449ec0 msgr2=0x55ad7944a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.615+0000 7efcbc7a7740 1 -- 172.21.15.1:0/2456599348 shutdown_connections 2024-04-26T02:01:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.615+0000 7efcbc7a7740 1 -- 172.21.15.1:0/2456599348 wait complete. 2024-04-26T02:01:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.615+0000 7efcb2a00640 1 --2- 172.21.15.1:0/697580624 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efc980a3ea0 0x7efc980a6360 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55ad7944ef50 tx=0x7efca40d3c10 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.616+0000 7efcbc7a7740 1 -- start start 2024-04-26T02:01:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.616+0000 7efcbc7a7740 1 Processor -- start 2024-04-26T02:01:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.616+0000 7efcbc7a7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad7944d9b0 0x55ad79455b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.616+0000 7efcbc7a7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 0x55ad79456040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.616+0000 7efcbc7a7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad79458b90 0x55ad79456580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.616+0000 7efcbc7a7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ad7945b7c0 con 0x55ad7944d9b0 2024-04-26T02:01:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.616+0000 7efcbc7a7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ad7936d310 con 0x55ad79458b90 2024-04-26T02:01:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.616+0000 7efcbc7a7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ad7945b940 con 0x55ad7944e2d0 2024-04-26T02:01:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.616+0000 7efcbc7a7740 10 client.0 _ll_register_callbacks cb 0x55ad7933bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.617+0000 7efcb3400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad7944d9b0 0x55ad79455b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.617+0000 7efcaa000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 0x55ad79456040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.617+0000 7efcaa000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 0x55ad79456040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:42830/0 (socket says 172.21.15.1:42830) 2024-04-26T02:01:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.617+0000 7efcaa000640 1 -- 172.21.15.1:0/1291038026 learned_addr learned my addr 172.21.15.1:0/1291038026 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.617+0000 7efcb3400640 1 --2- 172.21.15.1:0/1291038026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad79458b90 0x55ad79456580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.617+0000 7efcaa000640 1 -- 172.21.15.1:0/1291038026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad79458b90 msgr2=0x55ad79456580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.617+0000 7efcaa000640 1 --2- 172.21.15.1:0/1291038026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad79458b90 0x55ad79456580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.617+0000 7efcaa000640 1 -- 172.21.15.1:0/1291038026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad7944d9b0 msgr2=0x55ad79455b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.617+0000 7efcaa000640 1 --2- 172.21.15.1:0/1291038026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad7944d9b0 0x55ad79455b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.617+0000 7efcaa000640 1 -- 172.21.15.1:0/1291038026 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad79192620 con 0x55ad7944e2d0 2024-04-26T02:01:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.617+0000 7efcaa000640 1 --2- 172.21.15.1:0/1291038026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 0x55ad79456040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efc9c033000 tx=0x7efc9c0314a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e619ba2a2db70c4d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.617+0000 7efca0c00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T02:01:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.618+0000 7efca0c00640 1 -- 172.21.15.1:0/1291038026 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c035040 con 0x55ad7944e2d0 2024-04-26T02:01:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.618+0000 7efca0c00640 1 -- 172.21.15.1:0/1291038026 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7efc9c004050 con 0x55ad7944e2d0 2024-04-26T02:01:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.618+0000 7efcbc7a7740 1 -- 172.21.15.1:0/1291038026 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcac004050 con 0x55ad7944e2d0 2024-04-26T02:01:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.618+0000 7efca0c00640 1 -- 172.21.15.1:0/1291038026 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efc9c004250 con 0x55ad7944e2d0 2024-04-26T02:01:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.618+0000 7efcbc7a7740 10 client.26410 fetch_fsmap learned FSMap version 32 2024-04-26T02:01:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.618+0000 7efcbc7a7740 1 -- 172.21.15.1:0/1291038026 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ad79467a00 con 0x55ad7944e2d0 2024-04-26T02:01:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.618+0000 7efca0c00640 1 -- 172.21.15.1:0/1291038026 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcac004050 con 0x55ad7944e2d0 2024-04-26T02:01:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.619+0000 7efca0c00640 1 -- 172.21.15.1:0/1291038026 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7efc9c004a20 con 0x55ad7944e2d0 2024-04-26T02:01:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.619+0000 7efcbc7a7740 10 client.26410 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:01:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.619+0000 7efcbc7a7740 10 client.26410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:01:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.619+0000 7efcbc7a7740 20 client.26410 populate_metadata read hostname 'smithi001' 2024-04-26T02:01:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.619+0000 7efcbc7a7740 1 --2- 172.21.15.1:0/1291038026 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ad79468ec0 0x55ad7946b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.619+0000 7efcbc7a7740 4 client.26410 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:01:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.619+0000 7efcbc7a7740 1 -- 172.21.15.1:0/1291038026 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad793c78b0 con 0x55ad79468ec0 2024-04-26T02:01:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.619+0000 7efcb3400640 1 --2- 172.21.15.1:0/1291038026 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ad79468ec0 0x55ad7946b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.620+0000 7efcb3400640 1 --2- 172.21.15.1:0/1291038026 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ad79468ec0 0x55ad7946b2b0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=425dd3593246ed66 server_cookie=6a5a9e679d4275c1 in_seq=0 out_seq=0 2024-04-26T02:01:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.620+0000 7efca0c00640 10 client.26410 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.621+0000 7efca0c00640 1 -- 172.21.15.1:0/1291038026 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad793c78b0 con 0x55ad79468ec0 2024-04-26T02:01:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.621+0000 7efca0c00640 10 client.26410 handle_command_reply: tid=0 2024-04-26T02:01:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.621+0000 7efcbc7a7740 1 client.26410 shutdown 2024-04-26T02:01:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.621+0000 7efcbc7a7740 1 -- 172.21.15.1:0/1291038026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 msgr2=0x55ad79456040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.621+0000 7efcbc7a7740 1 --2- 172.21.15.1:0/1291038026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 0x55ad79456040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efc9c033000 tx=0x7efc9c0314a0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.621+0000 7efcbc7a7740 1 -- 172.21.15.1:0/1291038026 shutdown_connections 2024-04-26T02:01:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.621+0000 7efcbc7a7740 1 --2- 172.21.15.1:0/1291038026 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ad79468ec0 0x55ad7946b2b0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.621+0000 7efcbc7a7740 1 --2- 172.21.15.1:0/1291038026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad79458b90 0x55ad79456580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.621+0000 7efcbc7a7740 1 --2- 172.21.15.1:0/1291038026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 0x55ad79456040 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.621+0000 7efca0c00640 0 client.26410 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.621+0000 7efcbc7a7740 1 --2- 172.21.15.1:0/1291038026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad7944d9b0 0x55ad79455b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.621+0000 7efcbc7a7740 1 -- 172.21.15.1:0/1291038026 >> 172.21.15.1:0/1291038026 conn(0x55ad79449ec0 msgr2=0x55ad7944fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.621+0000 7efcbc7a7740 1 -- 172.21.15.1:0/1291038026 shutdown_connections 2024-04-26T02:01:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.621+0000 7efcbc7a7740 1 -- 172.21.15.1:0/1291038026 wait complete. 2024-04-26T02:01:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.621+0000 7efcbc7a7740 20 client.26410 trim_cache size 0 max 16384 2024-04-26T02:01:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.638+0000 7efcbc7a7740 1 Processor -- start 2024-04-26T02:01:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.638+0000 7efcbc7a7740 1 -- start start 2024-04-26T02:01:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.638+0000 7efcbc7a7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad7944d9b0 0x55ad79464680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.638+0000 7efcbc7a7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 0x55ad79448ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.638+0000 7efcbc7a7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad79458b90 0x55ad79470ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.638+0000 7efcbc7a7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41b25e0 con 0x55ad7944d9b0 2024-04-26T02:01:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.638+0000 7efcbc7a7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41afeb0 con 0x55ad79458b90 2024-04-26T02:01:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.638+0000 7efcbc7a7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41add20 con 0x55ad7944e2d0 2024-04-26T02:01:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.639+0000 7efcb3400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad7944d9b0 0x55ad79464680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.639+0000 7efcb3400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad7944d9b0 0x55ad79464680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50054/0 (socket says 172.21.15.1:50054) 2024-04-26T02:01:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.639+0000 7efcb3400640 1 -- 172.21.15.1:0/2090173894 learned_addr learned my addr 172.21.15.1:0/2090173894 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.639+0000 7efcaa000640 1 --2- 172.21.15.1:0/2090173894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 0x55ad79448ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.639+0000 7efcb3400640 1 --2- 172.21.15.1:0/2090173894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad79458b90 0x55ad79470ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.639+0000 7efcaa000640 1 -- 172.21.15.1:0/2090173894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad79458b90 msgr2=0x55ad79470ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.639+0000 7efcaa000640 1 --2- 172.21.15.1:0/2090173894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad79458b90 0x55ad79470ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.639+0000 7efcaa000640 1 -- 172.21.15.1:0/2090173894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad7944d9b0 msgr2=0x55ad79464680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.639+0000 7efcaa000640 1 --2- 172.21.15.1:0/2090173894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad7944d9b0 0x55ad79464680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.639+0000 7efcaa000640 1 -- 172.21.15.1:0/2090173894 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad79445820 con 0x55ad7944e2d0 2024-04-26T02:01:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.639+0000 7efcb3400640 1 --2- 172.21.15.1:0/2090173894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad7944d9b0 0x55ad79464680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.639+0000 7efcaa000640 1 --2- 172.21.15.1:0/2090173894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 0x55ad79448ea0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efc9c004df0 tx=0x7efc9c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53b48d25966c59cc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.640+0000 7efca3400640 1 -- 172.21.15.1:0/2090173894 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c035030 con 0x55ad7944e2d0 2024-04-26T02:01:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.640+0000 7efca3400640 1 -- 172.21.15.1:0/2090173894 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc9c041070 con 0x55ad7944e2d0 2024-04-26T02:01:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.640+0000 7efca3400640 1 -- 172.21.15.1:0/2090173894 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c051070 con 0x55ad7944e2d0 2024-04-26T02:01:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.640+0000 7efcbc7a7740 1 -- 172.21.15.1:0/2090173894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 msgr2=0x55ad79448ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.640+0000 7efcbc7a7740 1 --2- 172.21.15.1:0/2090173894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 0x55ad79448ea0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efc9c004df0 tx=0x7efc9c005fd0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.640+0000 7efcbc7a7740 1 -- 172.21.15.1:0/2090173894 shutdown_connections 2024-04-26T02:01:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.640+0000 7efcbc7a7740 1 --2- 172.21.15.1:0/2090173894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad79458b90 0x55ad79470ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.640+0000 7efcbc7a7740 1 --2- 172.21.15.1:0/2090173894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 0x55ad79448ea0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.640+0000 7efcbc7a7740 1 --2- 172.21.15.1:0/2090173894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad7944d9b0 0x55ad79464680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.640+0000 7efcbc7a7740 1 -- 172.21.15.1:0/2090173894 >> 172.21.15.1:0/2090173894 conn(0x55ad79449ec0 msgr2=0x55ad79448790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.640+0000 7efcbc7a7740 1 -- 172.21.15.1:0/2090173894 shutdown_connections 2024-04-26T02:01:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.641+0000 7efcbc7a7740 1 -- 172.21.15.1:0/2090173894 wait complete. 2024-04-26T02:01:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.641+0000 7efcbc7a7740 1 -- start start 2024-04-26T02:01:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.641+0000 7efcbc7a7740 1 Processor -- start 2024-04-26T02:01:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.642+0000 7efcbc7a7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad7944d9b0 0x55ad7945e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.642+0000 7efcbc7a7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 0x55ad7945ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.642+0000 7efcbc7a7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad79458b90 0x55ad7945ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.642+0000 7efcbc7a7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41ae370 con 0x55ad7944d9b0 2024-04-26T02:01:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.642+0000 7efcbc7a7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41af7e0 con 0x55ad79458b90 2024-04-26T02:01:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.642+0000 7efcbc7a7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efcb41aea40 con 0x55ad7944e2d0 2024-04-26T02:01:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.642+0000 7efcbc7a7740 10 client.0 _ll_register_callbacks cb 0x7efc9c032c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.642+0000 7efcb3400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad7944d9b0 0x55ad7945e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.642+0000 7efcb3400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad7944d9b0 0x55ad7945e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50058/0 (socket says 172.21.15.1:50058) 2024-04-26T02:01:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.642+0000 7efcb3400640 1 -- 172.21.15.1:0/766601449 learned_addr learned my addr 172.21.15.1:0/766601449 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.642+0000 7efcb3400640 1 --2- 172.21.15.1:0/766601449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad79458b90 0x55ad7945ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.642+0000 7efcaa000640 1 --2- 172.21.15.1:0/766601449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 0x55ad7945ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.642+0000 7efcb3400640 1 -- 172.21.15.1:0/766601449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 msgr2=0x55ad7945ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.642+0000 7efcb3400640 1 --2- 172.21.15.1:0/766601449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 0x55ad7945ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.643+0000 7efcb3400640 1 -- 172.21.15.1:0/766601449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad79458b90 msgr2=0x55ad7945ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.643+0000 7efcb3400640 1 --2- 172.21.15.1:0/766601449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad79458b90 0x55ad7945ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.643+0000 7efcb3400640 1 -- 172.21.15.1:0/766601449 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad794ce420 con 0x55ad7944d9b0 2024-04-26T02:01:42.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.643+0000 7efcb3400640 1 --2- 172.21.15.1:0/766601449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad7944d9b0 0x55ad7945e4e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efcac01b040 tx=0x7efcac004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7352cf1935eba869 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:42.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.643+0000 7efca0c00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T02:01:42.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.643+0000 7efca0c00640 1 -- 172.21.15.1:0/766601449 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac005750 con 0x55ad7944d9b0 2024-04-26T02:01:42.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.643+0000 7efcbc7a7740 1 -- 172.21.15.1:0/766601449 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc9c041070 con 0x55ad7944d9b0 2024-04-26T02:01:42.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.643+0000 7efca0c00640 1 -- 172.21.15.1:0/766601449 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7efcac0058f0 con 0x55ad7944d9b0 2024-04-26T02:01:42.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.643+0000 7efca0c00640 1 -- 172.21.15.1:0/766601449 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efcac028450 con 0x55ad7944d9b0 2024-04-26T02:01:42.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.643+0000 7efcbc7a7740 10 client.16236 fetch_fsmap learned FSMap version 32 2024-04-26T02:01:42.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.643+0000 7efcbc7a7740 1 -- 172.21.15.1:0/766601449 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ad794cf630 con 0x55ad7944d9b0 2024-04-26T02:01:42.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.644+0000 7efca0c00640 1 -- 172.21.15.1:0/766601449 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc9c041070 con 0x55ad7944d9b0 2024-04-26T02:01:42.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.644+0000 7efca0c00640 1 -- 172.21.15.1:0/766601449 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7efcac025040 con 0x55ad7944d9b0 2024-04-26T02:01:42.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.644+0000 7efcbc7a7740 10 client.16236 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:01:42.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.644+0000 7efcbc7a7740 10 client.16236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:01:42.645 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.644+0000 7efcbc7a7740 20 client.16236 populate_metadata read hostname 'smithi001' 2024-04-26T02:01:42.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.644+0000 7efcbc7a7740 1 --2- 172.21.15.1:0/766601449 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ad794cfb70 0x55ad794d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:42.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.644+0000 7efcbc7a7740 4 client.16236 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:01:42.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.644+0000 7efcbc7a7740 1 -- 172.21.15.1:0/766601449 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ad793c1320 con 0x55ad794cfb70 2024-04-26T02:01:42.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.644+0000 7efcaa000640 1 --2- 172.21.15.1:0/766601449 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ad794cfb70 0x55ad794d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:42.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.645+0000 7efcaa000640 1 --2- 172.21.15.1:0/766601449 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ad794cfb70 0x55ad794d1f90 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=178e304111cb8f9a server_cookie=176213d5f280977f in_seq=0 out_seq=0 2024-04-26T02:01:42.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.645+0000 7efca0c00640 10 client.16236 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.657+0000 7efca0c00640 1 -- 172.21.15.1:0/766601449 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ad793c1320 con 0x55ad794cfb70 2024-04-26T02:01:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.657+0000 7efca0c00640 10 client.16236 handle_command_reply: tid=0 2024-04-26T02:01:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.657+0000 7efcbc7a7740 1 client.16236 shutdown 2024-04-26T02:01:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.657+0000 7efcbc7a7740 1 -- 172.21.15.1:0/766601449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad7944d9b0 msgr2=0x55ad7945e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.657+0000 7efcbc7a7740 1 --2- 172.21.15.1:0/766601449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad7944d9b0 0x55ad7945e4e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efcac01b040 tx=0x7efcac004ac0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.657+0000 7efcbc7a7740 1 -- 172.21.15.1:0/766601449 shutdown_connections 2024-04-26T02:01:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.657+0000 7efcbc7a7740 1 --2- 172.21.15.1:0/766601449 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55ad794cfb70 0x55ad794d1f90 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.657+0000 7efcbc7a7740 1 --2- 172.21.15.1:0/766601449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ad79458b90 0x55ad7945ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.657+0000 7efcbc7a7740 1 --2- 172.21.15.1:0/766601449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ad7944e2d0 0x55ad7945ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.657+0000 7efcbc7a7740 1 --2- 172.21.15.1:0/766601449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad7944d9b0 0x55ad7945e4e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.657+0000 7efca0c00640 0 client.16236 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.657+0000 7efcbc7a7740 1 -- 172.21.15.1:0/766601449 >> 172.21.15.1:0/766601449 conn(0x55ad79449ec0 msgr2=0x55ad79455aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.658+0000 7efcbc7a7740 1 -- 172.21.15.1:0/766601449 shutdown_connections 2024-04-26T02:01:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.658+0000 7efcbc7a7740 1 -- 172.21.15.1:0/766601449 wait complete. 2024-04-26T02:01:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.658+0000 7efcbc7a7740 20 client.16236 trim_cache size 0 max 16384 2024-04-26T02:01:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T02:01:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T02:01:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T02:01:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T02:01:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.659+0000 7efcb3e00640 1 -- 172.21.15.1:0/697580624 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efc980a3ea0 msgr2=0x7efc980a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.659+0000 7efcb3e00640 1 --2- 172.21.15.1:0/697580624 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efc980a3ea0 0x7efc980a6360 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55ad7944ef50 tx=0x7efca40d3c10 comp rx=0 tx=0).stop 2024-04-26T02:01:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.659+0000 7efcb3e00640 1 -- 172.21.15.1:0/697580624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcb4100b30 msgr2=0x7efcb41a0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.659+0000 7efcb3e00640 1 --2- 172.21.15.1:0/697580624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcb4100b30 0x7efcb41a0470 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efca400d950 tx=0x7efca400de10 comp rx=0 tx=0).stop 2024-04-26T02:01:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.659+0000 7efcb3e00640 1 -- 172.21.15.1:0/697580624 shutdown_connections 2024-04-26T02:01:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.659+0000 7efcb3e00640 1 --2- 172.21.15.1:0/697580624 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efc980a3ea0 0x7efc980a6360 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.659+0000 7efcb3e00640 1 --2- 172.21.15.1:0/697580624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcb410e6e0 0x7efcb419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.659+0000 7efcb3e00640 1 --2- 172.21.15.1:0/697580624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efcb41014e0 0x7efcb41a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.659+0000 7efcb3e00640 1 --2- 172.21.15.1:0/697580624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcb4100b30 0x7efcb41a0470 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.659+0000 7efcb3e00640 1 -- 172.21.15.1:0/697580624 >> 172.21.15.1:0/697580624 conn(0x7efcb40fc760 msgr2=0x7efcb40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.659+0000 7efcb3e00640 1 -- 172.21.15.1:0/697580624 shutdown_connections 2024-04-26T02:01:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:42.659+0000 7efcb3e00640 1 -- 172.21.15.1:0/697580624 wait complete. 2024-04-26T02:01:44.285 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:44 smithi097 ceph-mon[32853]: pgmap v314: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 311 KiB/s wr, 120 op/s 2024-04-26T02:01:44.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:44 smithi173 ceph-mon[32553]: pgmap v314: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 311 KiB/s wr, 120 op/s 2024-04-26T02:01:44.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:44 smithi001 ceph-mon[27822]: pgmap v314: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 311 KiB/s wr, 120 op/s 2024-04-26T02:01:46.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:46 smithi173 ceph-mon[32553]: pgmap v315: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 319 KiB/s wr, 125 op/s 2024-04-26T02:01:46.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:46 smithi001 ceph-mon[27822]: pgmap v315: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 319 KiB/s wr, 125 op/s 2024-04-26T02:01:46.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:46 smithi097 ceph-mon[32853]: pgmap v315: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 319 KiB/s wr, 125 op/s 2024-04-26T02:01:48.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:48 smithi173 ceph-mon[32553]: pgmap v316: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-04-26T02:01:48.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:48 smithi001 ceph-mon[27822]: pgmap v316: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-04-26T02:01:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:48 smithi097 ceph-mon[32853]: pgmap v316: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-04-26T02:01:50.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:50 smithi173 ceph-mon[32553]: pgmap v317: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-04-26T02:01:50.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:50 smithi001 ceph-mon[27822]: pgmap v317: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-04-26T02:01:50.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:50 smithi097 ceph-mon[32853]: pgmap v317: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-04-26T02:01:52.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:52 smithi173 ceph-mon[32553]: pgmap v318: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 61 KiB/s wr, 11 op/s 2024-04-26T02:01:52.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:52 smithi001 ceph-mon[27822]: pgmap v318: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 61 KiB/s wr, 11 op/s 2024-04-26T02:01:52.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:52 smithi097 ceph-mon[32853]: pgmap v318: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 61 KiB/s wr, 11 op/s 2024-04-26T02:01:54.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:54 smithi173 ceph-mon[32553]: pgmap v319: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 59 KiB/s wr, 8 op/s 2024-04-26T02:01:54.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:54 smithi001 ceph-mon[27822]: pgmap v319: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 59 KiB/s wr, 8 op/s 2024-04-26T02:01:54.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:54 smithi097 ceph-mon[32853]: pgmap v319: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 59 KiB/s wr, 8 op/s 2024-04-26T02:01:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.807+0000 7fc3a7200640 1 -- 172.21.15.1:0/2480334030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3a0104d90 msgr2=0x7fc3a0108dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.807+0000 7fc3a7200640 1 --2- 172.21.15.1:0/2480334030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3a0104d90 0x7fc3a0108dd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc388009910 tx=0x7fc38802a3f0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.807+0000 7fc3a7200640 1 -- 172.21.15.1:0/2480334030 shutdown_connections 2024-04-26T02:01:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.807+0000 7fc3a7200640 1 --2- 172.21.15.1:0/2480334030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3a01094b0 0x7fc3a0111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.807+0000 7fc3a7200640 1 --2- 172.21.15.1:0/2480334030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3a0104d90 0x7fc3a0108dd0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.807+0000 7fc3a7200640 1 --2- 172.21.15.1:0/2480334030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3a01043e0 0x7fc3a01047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.807+0000 7fc3a7200640 1 -- 172.21.15.1:0/2480334030 >> 172.21.15.1:0/2480334030 conn(0x7fc3a00ffc00 msgr2=0x7fc3a0102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.808+0000 7fc3a7200640 1 -- 172.21.15.1:0/2480334030 shutdown_connections 2024-04-26T02:01:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.808+0000 7fc3a7200640 1 -- 172.21.15.1:0/2480334030 wait complete. 2024-04-26T02:01:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.809+0000 7fc3a7200640 1 Processor -- start 2024-04-26T02:01:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.809+0000 7fc3a7200640 1 -- start start 2024-04-26T02:01:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.809+0000 7fc3a7200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3a01043e0 0x7fc3a01a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.809+0000 7fc3a7200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3a01094b0 0x7fc3a01a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.809+0000 7fc3a7200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3a019a7b0 0x7fc3a019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.809+0000 7fc3a7200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a01143d0 con 0x7fc3a019a7b0 2024-04-26T02:01:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.809+0000 7fc3a7200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a0114250 con 0x7fc3a01043e0 2024-04-26T02:01:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.809+0000 7fc3a7200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a0114550 con 0x7fc3a01094b0 2024-04-26T02:01:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.810+0000 7fc39fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3a01043e0 0x7fc3a01a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.810+0000 7fc39f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3a01094b0 0x7fc3a01a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.810+0000 7fc39fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3a01043e0 0x7fc3a01a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:58752/0 (socket says 172.21.15.1:58752) 2024-04-26T02:01:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.810+0000 7fc39fe00640 1 -- 172.21.15.1:0/1240808895 learned_addr learned my addr 172.21.15.1:0/1240808895 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.810+0000 7fc3a5200640 1 --2- 172.21.15.1:0/1240808895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3a019a7b0 0x7fc3a019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.810+0000 7fc39fe00640 1 -- 172.21.15.1:0/1240808895 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3a01094b0 msgr2=0x7fc3a01a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.810+0000 7fc39fe00640 1 --2- 172.21.15.1:0/1240808895 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3a01094b0 0x7fc3a01a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.810+0000 7fc39fe00640 1 -- 172.21.15.1:0/1240808895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3a019a7b0 msgr2=0x7fc3a019ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.810+0000 7fc39fe00640 1 --2- 172.21.15.1:0/1240808895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3a019a7b0 0x7fc3a019ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.810+0000 7fc39fe00640 1 -- 172.21.15.1:0/1240808895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a019b340 con 0x7fc3a01043e0 2024-04-26T02:01:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.811+0000 7fc39fe00640 1 --2- 172.21.15.1:0/1240808895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3a01043e0 0x7fc3a01a0640 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc39000e9f0 tx=0x7fc39000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.811+0000 7fc39cc00640 1 -- 172.21.15.1:0/1240808895 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39000cd80 con 0x7fc3a01043e0 2024-04-26T02:01:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.811+0000 7fc3a7200640 1 -- 172.21.15.1:0/1240808895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a019b630 con 0x7fc3a01043e0 2024-04-26T02:01:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.811+0000 7fc39cc00640 1 -- 172.21.15.1:0/1240808895 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3900108e0 con 0x7fc3a01043e0 2024-04-26T02:01:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.811+0000 7fc39cc00640 1 -- 172.21.15.1:0/1240808895 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390018980 con 0x7fc3a01043e0 2024-04-26T02:01:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.811+0000 7fc3a7200640 1 -- 172.21.15.1:0/1240808895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a01a7830 con 0x7fc3a01043e0 2024-04-26T02:01:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.814+0000 7fc3a9f95740 1 Processor -- start 2024-04-26T02:01:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.814+0000 7fc3a9f95740 1 -- start start 2024-04-26T02:01:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.815+0000 7fc3a9f95740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33e2820 0x55c0a33e2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.815+0000 7fc3a9f95740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0a33e3140 0x55c0a33ed4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.815+0000 7fc3a9f95740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33eda00 0x55c0a33efdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.815+0000 7fc3a9f95740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c0a3348900 con 0x55c0a33eda00 2024-04-26T02:01:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.815+0000 7fc3a9f95740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c0a32eeef0 con 0x55c0a33e2820 2024-04-26T02:01:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.815+0000 7fc3a9f95740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c0a33448c0 con 0x55c0a33e3140 2024-04-26T02:01:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.815+0000 7fc39cc00640 1 -- 172.21.15.1:0/1240808895 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fc390010430 con 0x7fc3a01043e0 2024-04-26T02:01:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.815+0000 7fc39f400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33eda00 0x55c0a33efdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.815+0000 7fc3a5200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33e2820 0x55c0a33e2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.815+0000 7fc39f400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33eda00 0x55c0a33efdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44582/0 (socket says 172.21.15.1:44582) 2024-04-26T02:01:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.815+0000 7fc39f400640 1 -- 172.21.15.1:0/3318767955 learned_addr learned my addr 172.21.15.1:0/3318767955 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.815+0000 7fc3a5200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33e2820 0x55c0a33e2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:58762/0 (socket says 172.21.15.1:58762) 2024-04-26T02:01:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.815+0000 7fc39fe00640 1 --2- 172.21.15.1:0/3318767955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0a33e3140 0x55c0a33ed4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.815+0000 7fc3a5200640 1 -- 172.21.15.1:0/3318767955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0a33e3140 msgr2=0x55c0a33ed4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.816+0000 7fc3a5200640 1 --2- 172.21.15.1:0/3318767955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0a33e3140 0x55c0a33ed4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.816+0000 7fc3a5200640 1 -- 172.21.15.1:0/3318767955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33eda00 msgr2=0x55c0a33efdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.816+0000 7fc3a5200640 1 --2- 172.21.15.1:0/3318767955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33eda00 0x55c0a33efdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.816+0000 7fc3a5200640 1 -- 172.21.15.1:0/3318767955 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0a305b210 con 0x55c0a33e2820 2024-04-26T02:01:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.816+0000 7fc39f400640 1 --2- 172.21.15.1:0/3318767955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33eda00 0x55c0a33efdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.816+0000 7fc3a5200640 1 --2- 172.21.15.1:0/3318767955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33e2820 0x55c0a33e2c00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc398009f00 tx=0x7fc398011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aac87a5b7492ed52 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.816+0000 7fc395600640 1 -- 172.21.15.1:0/3318767955 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398004d20 con 0x55c0a33e2820 2024-04-26T02:01:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.816+0000 7fc395600640 1 -- 172.21.15.1:0/3318767955 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc398004050 con 0x55c0a33e2820 2024-04-26T02:01:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.816+0000 7fc395600640 1 -- 172.21.15.1:0/3318767955 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398005280 con 0x55c0a33e2820 2024-04-26T02:01:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.817+0000 7fc39cc00640 1 --2- 172.21.15.1:0/1240808895 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc3840a3d10 0x7fc3840a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.817+0000 7fc39cc00640 1 -- 172.21.15.1:0/1240808895 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7fc3900d0b70 con 0x7fc3a01043e0 2024-04-26T02:01:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.817+0000 7fc39fe00640 1 --2- 172.21.15.1:0/1240808895 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc3840a3d10 0x7fc3840a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.817+0000 7fc3a9f95740 1 -- 172.21.15.1:0/3318767955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33e2820 msgr2=0x55c0a33e2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.817+0000 7fc3a9f95740 1 --2- 172.21.15.1:0/3318767955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33e2820 0x55c0a33e2c00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc398009f00 tx=0x7fc398011810 comp rx=0 tx=0).stop 2024-04-26T02:01:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.817+0000 7fc3a9f95740 1 -- 172.21.15.1:0/3318767955 shutdown_connections 2024-04-26T02:01:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.817+0000 7fc3a9f95740 1 --2- 172.21.15.1:0/3318767955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33eda00 0x55c0a33efdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.817+0000 7fc3a9f95740 1 --2- 172.21.15.1:0/3318767955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0a33e3140 0x55c0a33ed4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.817+0000 7fc3a9f95740 1 --2- 172.21.15.1:0/3318767955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33e2820 0x55c0a33e2c00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.817+0000 7fc3a9f95740 1 -- 172.21.15.1:0/3318767955 >> 172.21.15.1:0/3318767955 conn(0x55c0a33ded30 msgr2=0x55c0a33df110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.817+0000 7fc3a9f95740 1 -- 172.21.15.1:0/3318767955 shutdown_connections 2024-04-26T02:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.817+0000 7fc3a9f95740 1 -- 172.21.15.1:0/3318767955 wait complete. 2024-04-26T02:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.818+0000 7fc39fe00640 1 --2- 172.21.15.1:0/1240808895 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc3840a3d10 0x7fc3840a61d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55c0a33e3dc0 tx=0x7fc3900d4620 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.818+0000 7fc3a9f95740 1 -- start start 2024-04-26T02:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.818+0000 7fc3a9f95740 1 Processor -- start 2024-04-26T02:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.819+0000 7fc3a9f95740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33e2820 0x55c0a33ea9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.819+0000 7fc3a9f95740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0a33e3140 0x55c0a33eaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.819+0000 7fc3a9f95740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33eda00 0x55c0a33eb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.819+0000 7fc3a9f95740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c0a33f0630 con 0x55c0a33eda00 2024-04-26T02:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.819+0000 7fc3a9f95740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c0a3302310 con 0x55c0a33e2820 2024-04-26T02:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.819+0000 7fc3a9f95740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c0a33f07b0 con 0x55c0a33e3140 2024-04-26T02:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.819+0000 7fc3a9f95740 10 client.0 _ll_register_callbacks cb 0x55c0a32d0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.819+0000 7fc3a5200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33eda00 0x55c0a33eb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.819+0000 7fc3a5200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33eda00 0x55c0a33eb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44588/0 (socket says 172.21.15.1:44588) 2024-04-26T02:01:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.819+0000 7fc3a5200640 1 -- 172.21.15.1:0/3789788677 learned_addr learned my addr 172.21.15.1:0/3789788677 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.819+0000 7fc3a5200640 1 --2- 172.21.15.1:0/3789788677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33e2820 0x55c0a33ea9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.819+0000 7fc39f400640 1 --2- 172.21.15.1:0/3789788677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0a33e3140 0x55c0a33eaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.820+0000 7fc3a5200640 1 -- 172.21.15.1:0/3789788677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0a33e3140 msgr2=0x55c0a33eaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.820+0000 7fc3a5200640 1 --2- 172.21.15.1:0/3789788677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0a33e3140 0x55c0a33eaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.820+0000 7fc3a5200640 1 -- 172.21.15.1:0/3789788677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33e2820 msgr2=0x55c0a33ea9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.820+0000 7fc3a5200640 1 --2- 172.21.15.1:0/3789788677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33e2820 0x55c0a33ea9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.820+0000 7fc3a5200640 1 -- 172.21.15.1:0/3789788677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0a3127620 con 0x55c0a33eda00 2024-04-26T02:01:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.820+0000 7fc3a5200640 1 --2- 172.21.15.1:0/3789788677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33eda00 0x55c0a33eb460 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc3980046e0 tx=0x7fc398004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce541936ad744ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.820+0000 7fc38e000640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T02:01:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.820+0000 7fc38e000640 1 -- 172.21.15.1:0/3789788677 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39802d060 con 0x55c0a33eda00 2024-04-26T02:01:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.820+0000 7fc38e000640 1 -- 172.21.15.1:0/3789788677 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7fc398004a40 con 0x55c0a33eda00 2024-04-26T02:01:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.820+0000 7fc3a9f95740 1 -- 172.21.15.1:0/3789788677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc398004050 con 0x55c0a33eda00 2024-04-26T02:01:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.820+0000 7fc38e000640 1 -- 172.21.15.1:0/3789788677 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc398003cf0 con 0x55c0a33eda00 2024-04-26T02:01:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.820+0000 7fc3a9f95740 10 client.16248 fetch_fsmap learned FSMap version 32 2024-04-26T02:01:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.821+0000 7fc3a9f95740 1 -- 172.21.15.1:0/3789788677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c0a33fc8e0 con 0x55c0a33eda00 2024-04-26T02:01:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.821+0000 7fc38e000640 1 -- 172.21.15.1:0/3789788677 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc398004050 con 0x55c0a33eda00 2024-04-26T02:01:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.821+0000 7fc38e000640 1 -- 172.21.15.1:0/3789788677 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fc398027840 con 0x55c0a33eda00 2024-04-26T02:01:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.821+0000 7fc3a9f95740 10 client.16248 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:01:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.821+0000 7fc3a9f95740 10 client.16248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:01:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.821+0000 7fc3a9f95740 20 client.16248 populate_metadata read hostname 'smithi001' 2024-04-26T02:01:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.821+0000 7fc3a9f95740 1 --2- 172.21.15.1:0/3789788677 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c0a33fdce0 0x55c0a34000d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.821+0000 7fc3a9f95740 4 client.16248 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:01:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.821+0000 7fc3a9f95740 1 -- 172.21.15.1:0/3789788677 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0a335c8b0 con 0x55c0a33fdce0 2024-04-26T02:01:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.822+0000 7fc39f400640 1 --2- 172.21.15.1:0/3789788677 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c0a33fdce0 0x55c0a34000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.822+0000 7fc39f400640 1 --2- 172.21.15.1:0/3789788677 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c0a33fdce0 0x55c0a34000d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=579607e7176ade36 server_cookie=4e8018063d535ada in_seq=0 out_seq=0 2024-04-26T02:01:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.822+0000 7fc38e000640 10 client.16248 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.823+0000 7fc38e000640 1 -- 172.21.15.1:0/3789788677 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c0a335c8b0 con 0x55c0a33fdce0 2024-04-26T02:01:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.823+0000 7fc38e000640 10 client.16248 handle_command_reply: tid=0 2024-04-26T02:01:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.823+0000 7fc3a9f95740 1 client.16248 shutdown 2024-04-26T02:01:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.823+0000 7fc3a9f95740 1 -- 172.21.15.1:0/3789788677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33eda00 msgr2=0x55c0a33eb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.823+0000 7fc3a9f95740 1 --2- 172.21.15.1:0/3789788677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33eda00 0x55c0a33eb460 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc3980046e0 tx=0x7fc398004990 comp rx=0 tx=0).stop 2024-04-26T02:01:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.823+0000 7fc3a9f95740 1 -- 172.21.15.1:0/3789788677 shutdown_connections 2024-04-26T02:01:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.823+0000 7fc3a9f95740 1 --2- 172.21.15.1:0/3789788677 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c0a33fdce0 0x55c0a34000d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.823+0000 7fc3a9f95740 1 --2- 172.21.15.1:0/3789788677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33eda00 0x55c0a33eb460 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.823+0000 7fc3a9f95740 1 --2- 172.21.15.1:0/3789788677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0a33e3140 0x55c0a33eaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.823+0000 7fc3a9f95740 1 --2- 172.21.15.1:0/3789788677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33e2820 0x55c0a33ea9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.823+0000 7fc38e000640 0 client.16248 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.823+0000 7fc3a9f95740 1 -- 172.21.15.1:0/3789788677 >> 172.21.15.1:0/3789788677 conn(0x55c0a33ded30 msgr2=0x55c0a33e4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.823+0000 7fc3a9f95740 1 -- 172.21.15.1:0/3789788677 shutdown_connections 2024-04-26T02:01:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.824+0000 7fc3a9f95740 1 -- 172.21.15.1:0/3789788677 wait complete. 2024-04-26T02:01:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.824+0000 7fc3a9f95740 20 client.16248 trim_cache size 0 max 16384 2024-04-26T02:01:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.840+0000 7fc3a9f95740 1 Processor -- start 2024-04-26T02:01:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.840+0000 7fc3a9f95740 1 -- start start 2024-04-26T02:01:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.841+0000 7fc3a9f95740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33e2820 0x55c0a33de610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.841+0000 7fc3a9f95740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0a33e3140 0x55c0a33e51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.841+0000 7fc3a9f95740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33eda00 0x55c0a3405780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.841+0000 7fc3a9f95740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a01b27e0 con 0x55c0a33eda00 2024-04-26T02:01:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.841+0000 7fc3a9f95740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a01b00b0 con 0x55c0a33e2820 2024-04-26T02:01:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.841+0000 7fc3a9f95740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a01adf20 con 0x55c0a33e3140 2024-04-26T02:01:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.841+0000 7fc3a5200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33e2820 0x55c0a33de610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.841+0000 7fc39f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0a33e3140 0x55c0a33e51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.841+0000 7fc3a5200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33e2820 0x55c0a33de610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:58790/0 (socket says 172.21.15.1:58790) 2024-04-26T02:01:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.841+0000 7fc3a5200640 1 -- 172.21.15.1:0/2820188110 learned_addr learned my addr 172.21.15.1:0/2820188110 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.841+0000 7fc3a5200640 1 --2- 172.21.15.1:0/2820188110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33eda00 0x55c0a3405780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.842+0000 7fc39f400640 1 -- 172.21.15.1:0/2820188110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33e2820 msgr2=0x55c0a33de610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.842+0000 7fc39f400640 1 --2- 172.21.15.1:0/2820188110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33e2820 0x55c0a33de610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.842+0000 7fc39f400640 1 -- 172.21.15.1:0/2820188110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33eda00 msgr2=0x55c0a3405780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.842+0000 7fc39f400640 1 --2- 172.21.15.1:0/2820188110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33eda00 0x55c0a3405780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.842+0000 7fc39f400640 1 -- 172.21.15.1:0/2820188110 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0a33deb50 con 0x55c0a33e3140 2024-04-26T02:01:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.842+0000 7fc39f400640 1 --2- 172.21.15.1:0/2820188110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0a33e3140 0x55c0a33e51e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc388005d60 tx=0x7fc388005cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e961a7c46f3172b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.842+0000 7fc394c00640 1 -- 172.21.15.1:0/2820188110 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388004310 con 0x55c0a33e3140 2024-04-26T02:01:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.843+0000 7fc394c00640 1 -- 172.21.15.1:0/2820188110 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc38804d070 con 0x55c0a33e3140 2024-04-26T02:01:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.843+0000 7fc394c00640 1 -- 172.21.15.1:0/2820188110 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388005040 con 0x55c0a33e3140 2024-04-26T02:01:55.845 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.843+0000 7fc3a9f95740 1 -- 172.21.15.1:0/2820188110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0a33e3140 msgr2=0x55c0a33e51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:55.845 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.843+0000 7fc3a9f95740 1 --2- 172.21.15.1:0/2820188110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0a33e3140 0x55c0a33e51e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc388005d60 tx=0x7fc388005cf0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.845 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.843+0000 7fc3a9f95740 1 -- 172.21.15.1:0/2820188110 shutdown_connections 2024-04-26T02:01:55.845 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.843+0000 7fc3a9f95740 1 --2- 172.21.15.1:0/2820188110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33eda00 0x55c0a3405780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.845 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.843+0000 7fc3a9f95740 1 --2- 172.21.15.1:0/2820188110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0a33e3140 0x55c0a33e51e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.845 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.843+0000 7fc3a9f95740 1 --2- 172.21.15.1:0/2820188110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33e2820 0x55c0a33de610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.843+0000 7fc3a9f95740 1 -- 172.21.15.1:0/2820188110 >> 172.21.15.1:0/2820188110 conn(0x55c0a33ded30 msgr2=0x55c0a33ecaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.843+0000 7fc3a9f95740 1 -- 172.21.15.1:0/2820188110 shutdown_connections 2024-04-26T02:01:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.843+0000 7fc3a9f95740 1 -- 172.21.15.1:0/2820188110 wait complete. 2024-04-26T02:01:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.844+0000 7fc3a9f95740 1 -- start start 2024-04-26T02:01:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.844+0000 7fc3a9f95740 1 Processor -- start 2024-04-26T02:01:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.844+0000 7fc3a9f95740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33e2820 0x55c0a33f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.844+0000 7fc3a9f95740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0a33e3140 0x55c0a33f2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.844+0000 7fc3a9f95740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33eda00 0x55c0a33f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.844+0000 7fc3a9f95740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a01ae570 con 0x55c0a33e2820 2024-04-26T02:01:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.844+0000 7fc3a9f95740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a01af9e0 con 0x55c0a33eda00 2024-04-26T02:01:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.844+0000 7fc3a9f95740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a01aec40 con 0x55c0a33e3140 2024-04-26T02:01:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.844+0000 7fc3a9f95740 10 client.0 _ll_register_callbacks cb 0x7fc3980045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.845+0000 7fc3a5200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33e2820 0x55c0a33f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.845+0000 7fc39f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0a33e3140 0x55c0a33f2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.845+0000 7fc3a5200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33e2820 0x55c0a33f5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44602/0 (socket says 172.21.15.1:44602) 2024-04-26T02:01:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.845+0000 7fc3a5200640 1 -- 172.21.15.1:0/2567302238 learned_addr learned my addr 172.21.15.1:0/2567302238 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.845+0000 7fc3a5200640 1 --2- 172.21.15.1:0/2567302238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33eda00 0x55c0a33f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.845+0000 7fc3a5200640 1 -- 172.21.15.1:0/2567302238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0a33e3140 msgr2=0x55c0a33f2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.845+0000 7fc3a5200640 1 --2- 172.21.15.1:0/2567302238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0a33e3140 0x55c0a33f2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.845+0000 7fc3a5200640 1 -- 172.21.15.1:0/2567302238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33eda00 msgr2=0x55c0a33f34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.845+0000 7fc3a5200640 1 --2- 172.21.15.1:0/2567302238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33eda00 0x55c0a33f34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.845+0000 7fc3a5200640 1 -- 172.21.15.1:0/2567302238 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0a33f3a20 con 0x55c0a33e2820 2024-04-26T02:01:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.845+0000 7fc3a5200640 1 --2- 172.21.15.1:0/2567302238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33e2820 0x55c0a33f5b40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc398027c10 tx=0x7fc3980048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd278520668ff2b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.845+0000 7fc38e000640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T02:01:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.846+0000 7fc38e000640 1 -- 172.21.15.1:0/2567302238 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39802d070 con 0x55c0a33e2820 2024-04-26T02:01:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.846+0000 7fc3a9f95740 1 -- 172.21.15.1:0/2567302238 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc38804d070 con 0x55c0a33e2820 2024-04-26T02:01:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.846+0000 7fc38e000640 1 -- 172.21.15.1:0/2567302238 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7fc398003af0 con 0x55c0a33e2820 2024-04-26T02:01:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.846+0000 7fc38e000640 1 -- 172.21.15.1:0/2567302238 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc398027e80 con 0x55c0a33e2820 2024-04-26T02:01:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.846+0000 7fc3a9f95740 10 client.16254 fetch_fsmap learned FSMap version 32 2024-04-26T02:01:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.846+0000 7fc3a9f95740 1 -- 172.21.15.1:0/2567302238 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c0a3463d90 con 0x55c0a33e2820 2024-04-26T02:01:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.846+0000 7fc38e000640 1 -- 172.21.15.1:0/2567302238 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc38804d070 con 0x55c0a33e2820 2024-04-26T02:01:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.846+0000 7fc38e000640 1 -- 172.21.15.1:0/2567302238 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fc3980197b0 con 0x55c0a33e2820 2024-04-26T02:01:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.846+0000 7fc3a9f95740 10 client.16254 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:01:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.846+0000 7fc3a9f95740 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:01:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.846+0000 7fc3a9f95740 20 client.16254 populate_metadata read hostname 'smithi001' 2024-04-26T02:01:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.846+0000 7fc3a9f95740 1 --2- 172.21.15.1:0/2567302238 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c0a3464450 0x55c0a3466840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.846+0000 7fc3a9f95740 4 client.16254 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:01:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.846+0000 7fc3a9f95740 1 -- 172.21.15.1:0/2567302238 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0a3356320 con 0x55c0a3464450 2024-04-26T02:01:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.847+0000 7fc39f400640 1 --2- 172.21.15.1:0/2567302238 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c0a3464450 0x55c0a3466840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.847+0000 7fc39f400640 1 --2- 172.21.15.1:0/2567302238 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c0a3464450 0x55c0a3466840 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=688a7bfa157a77be server_cookie=43c275ef9890f1d5 in_seq=0 out_seq=0 2024-04-26T02:01:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.847+0000 7fc38e000640 10 client.16254 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.856+0000 7fc38e000640 1 -- 172.21.15.1:0/2567302238 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c0a3356320 con 0x55c0a3464450 2024-04-26T02:01:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.856+0000 7fc38e000640 10 client.16254 handle_command_reply: tid=0 2024-04-26T02:01:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.856+0000 7fc3a9f95740 1 client.16254 shutdown 2024-04-26T02:01:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.856+0000 7fc3a9f95740 1 -- 172.21.15.1:0/2567302238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33e2820 msgr2=0x55c0a33f5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.856+0000 7fc3a9f95740 1 --2- 172.21.15.1:0/2567302238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33e2820 0x55c0a33f5b40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc398027c10 tx=0x7fc3980048f0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.857+0000 7fc3a9f95740 1 -- 172.21.15.1:0/2567302238 shutdown_connections 2024-04-26T02:01:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.857+0000 7fc3a9f95740 1 --2- 172.21.15.1:0/2567302238 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55c0a3464450 0x55c0a3466840 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.857+0000 7fc3a9f95740 1 --2- 172.21.15.1:0/2567302238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c0a33eda00 0x55c0a33f34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.857+0000 7fc3a9f95740 1 --2- 172.21.15.1:0/2567302238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0a33e3140 0x55c0a33f2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.857+0000 7fc3a9f95740 1 --2- 172.21.15.1:0/2567302238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c0a33e2820 0x55c0a33f5b40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.857+0000 7fc3a9f95740 1 -- 172.21.15.1:0/2567302238 >> 172.21.15.1:0/2567302238 conn(0x55c0a33ded30 msgr2=0x55c0a33ea9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.857+0000 7fc38e000640 0 client.16254 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:01:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.857+0000 7fc3a9f95740 1 -- 172.21.15.1:0/2567302238 shutdown_connections 2024-04-26T02:01:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.857+0000 7fc3a9f95740 1 -- 172.21.15.1:0/2567302238 wait complete. 2024-04-26T02:01:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.857+0000 7fc3a9f95740 20 client.16254 trim_cache size 0 max 16384 2024-04-26T02:01:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T02:01:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T02:01:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T02:01:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T02:01:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.858+0000 7fc3a7200640 1 -- 172.21.15.1:0/1240808895 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc3840a3d10 msgr2=0x7fc3840a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.858+0000 7fc3a7200640 1 --2- 172.21.15.1:0/1240808895 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc3840a3d10 0x7fc3840a61d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55c0a33e3dc0 tx=0x7fc3900d4620 comp rx=0 tx=0).stop 2024-04-26T02:01:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.858+0000 7fc3a7200640 1 -- 172.21.15.1:0/1240808895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3a01043e0 msgr2=0x7fc3a01a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.858+0000 7fc3a7200640 1 --2- 172.21.15.1:0/1240808895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3a01043e0 0x7fc3a01a0640 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc39000e9f0 tx=0x7fc39000eeb0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.858+0000 7fc3a7200640 1 -- 172.21.15.1:0/1240808895 shutdown_connections 2024-04-26T02:01:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.858+0000 7fc3a7200640 1 --2- 172.21.15.1:0/1240808895 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fc3840a3d10 0x7fc3840a61d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.858+0000 7fc3a7200640 1 --2- 172.21.15.1:0/1240808895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc3a019a7b0 0x7fc3a019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.858+0000 7fc3a7200640 1 --2- 172.21.15.1:0/1240808895 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3a01094b0 0x7fc3a01a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.858+0000 7fc3a7200640 1 --2- 172.21.15.1:0/1240808895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3a01043e0 0x7fc3a01a0640 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.858+0000 7fc3a7200640 1 -- 172.21.15.1:0/1240808895 >> 172.21.15.1:0/1240808895 conn(0x7fc3a00ffc00 msgr2=0x7fc3a0100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.858+0000 7fc3a7200640 1 -- 172.21.15.1:0/1240808895 shutdown_connections 2024-04-26T02:01:55.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:01:55.858+0000 7fc3a7200640 1 -- 172.21.15.1:0/1240808895 wait complete. 2024-04-26T02:01:56.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:56 smithi173 ceph-mon[32553]: pgmap v320: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 59 KiB/s wr, 11 op/s 2024-04-26T02:01:56.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:56 smithi001 ceph-mon[27822]: pgmap v320: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 59 KiB/s wr, 11 op/s 2024-04-26T02:01:56.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:56 smithi097 ceph-mon[32853]: pgmap v320: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 59 KiB/s wr, 11 op/s 2024-04-26T02:01:56.593 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:01:57.059 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:01:58.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:01:58 smithi173 ceph-mon[32553]: pgmap v321: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 51 KiB/s wr, 6 op/s 2024-04-26T02:01:58.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:01:58 smithi001 ceph-mon[27822]: pgmap v321: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 51 KiB/s wr, 6 op/s 2024-04-26T02:01:58.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:01:58 smithi097 ceph-mon[32853]: pgmap v321: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 51 KiB/s wr, 6 op/s 2024-04-26T02:01:59.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.359+0000 7f60ce200700 1 -- 172.21.15.1:0/2533680644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60c8105780 msgr2=0x7f60c81097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:59.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.359+0000 7f60ce200700 1 --2- 172.21.15.1:0/2533680644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60c8105780 0x7f60c81097d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f60b40098f0 tx=0x7f60b4009c00 comp rx=0 tx=0).stop 2024-04-26T02:01:59.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.360+0000 7f60ce200700 1 -- 172.21.15.1:0/2533680644 shutdown_connections 2024-04-26T02:01:59.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.360+0000 7f60ce200700 1 --2- 172.21.15.1:0/2533680644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60c8109eb0 0x7f60c8111a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:59.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.360+0000 7f60ce200700 1 --2- 172.21.15.1:0/2533680644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60c8104dd0 0x7f60c81051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:59.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.360+0000 7f60ce200700 1 --2- 172.21.15.1:0/2533680644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60c8105780 0x7f60c81097d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:59.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.360+0000 7f60ce200700 1 -- 172.21.15.1:0/2533680644 >> 172.21.15.1:0/2533680644 conn(0x7f60c8100660 msgr2=0x7f60c8102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:59.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.360+0000 7f60ce200700 1 -- 172.21.15.1:0/2533680644 shutdown_connections 2024-04-26T02:01:59.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.360+0000 7f60ce200700 1 -- 172.21.15.1:0/2533680644 wait complete. 2024-04-26T02:01:59.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.362+0000 7f60ce200700 1 Processor -- start 2024-04-26T02:01:59.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.362+0000 7f60ce200700 1 -- start start 2024-04-26T02:01:59.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.363+0000 7f60ce200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60c8104dd0 0x7f60c819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:59.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.363+0000 7f60ce200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60c8105780 0x7f60c819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:59.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.363+0000 7f60ce200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60c8109eb0 0x7f60c819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:59.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.363+0000 7f60ce200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f60c8114da0 con 0x7f60c8104dd0 2024-04-26T02:01:59.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.363+0000 7f60ce200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f60c8114c20 con 0x7f60c8109eb0 2024-04-26T02:01:59.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.363+0000 7f60ce200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f60c8114f20 con 0x7f60c8105780 2024-04-26T02:01:59.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.363+0000 7f60cce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60c8104dd0 0x7f60c819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:59.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.363+0000 7f60cce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60c8104dd0 0x7f60c819b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44614/0 (socket says 172.21.15.1:44614) 2024-04-26T02:01:59.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.363+0000 7f60cce00700 1 -- 172.21.15.1:0/3850193503 learned_addr learned my addr 172.21.15.1:0/3850193503 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:01:59.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.363+0000 7f60c7e00700 1 --2- 172.21.15.1:0/3850193503 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60c8105780 0x7f60c819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:59.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.363+0000 7f60cd800700 1 --2- 172.21.15.1:0/3850193503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60c8109eb0 0x7f60c819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:59.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.364+0000 7f60cce00700 1 -- 172.21.15.1:0/3850193503 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60c8105780 msgr2=0x7f60c819b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:59.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.364+0000 7f60cce00700 1 --2- 172.21.15.1:0/3850193503 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60c8105780 0x7f60c819b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:59.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.364+0000 7f60cce00700 1 -- 172.21.15.1:0/3850193503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60c8109eb0 msgr2=0x7f60c819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:59.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.364+0000 7f60cce00700 1 --2- 172.21.15.1:0/3850193503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60c8109eb0 0x7f60c819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:59.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.364+0000 7f60cce00700 1 -- 172.21.15.1:0/3850193503 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c81a0320 con 0x7f60c8104dd0 2024-04-26T02:01:59.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.364+0000 7f60cce00700 1 --2- 172.21.15.1:0/3850193503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60c8104dd0 0x7f60c819b310 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00e9c0 tx=0x7f60bc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:59.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.364+0000 7f60c5600700 1 -- 172.21.15.1:0/3850193503 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc00cbd0 con 0x7f60c8104dd0 2024-04-26T02:01:59.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.364+0000 7f60c5600700 1 -- 172.21.15.1:0/3850193503 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60bc00cd70 con 0x7f60c8104dd0 2024-04-26T02:01:59.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.364+0000 7f60c5600700 1 -- 172.21.15.1:0/3850193503 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc005610 con 0x7f60c8104dd0 2024-04-26T02:01:59.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.364+0000 7f60ce200700 1 -- 172.21.15.1:0/3850193503 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c81a0610 con 0x7f60c8104dd0 2024-04-26T02:01:59.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.365+0000 7f60ce200700 1 -- 172.21.15.1:0/3850193503 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c81a7e90 con 0x7f60c8104dd0 2024-04-26T02:01:59.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.366+0000 7f60ce200700 1 -- 172.21.15.1:0/3850193503 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60c8069730 con 0x7f60c8104dd0 2024-04-26T02:01:59.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.368+0000 7f60c5600700 1 -- 172.21.15.1:0/3850193503 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f60bc004750 con 0x7f60c8104dd0 2024-04-26T02:01:59.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.369+0000 7f60c5600700 1 --2- 172.21.15.1:0/3850193503 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f60b00a3db0 0x7f60b00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:59.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.370+0000 7f60c5600700 1 -- 172.21.15.1:0/3850193503 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f60bc0d0bd0 con 0x7f60c8104dd0 2024-04-26T02:01:59.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.370+0000 7f60c7e00700 1 --2- 172.21.15.1:0/3850193503 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f60b00a3db0 0x7f60b00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:59.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.370+0000 7f60c7e00700 1 --2- 172.21.15.1:0/3850193503 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f60b00a3db0 0x7f60b00a6270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009fd0 tx=0x7f60b4019040 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:59.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.374+0000 7f60c5600700 1 -- 172.21.15.1:0/3850193503 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60bc09d1e0 con 0x7f60c8104dd0 2024-04-26T02:01:59.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.659+0000 7f60ce200700 1 -- 172.21.15.1:0/3850193503 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60c819c670 con 0x7f60c8104dd0 2024-04-26T02:01:59.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.661+0000 7f60c5600700 1 -- 172.21.15.1:0/3850193503 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11838 (secure 0 0 0) 0x7f60bc09cae0 con 0x7f60c8104dd0 2024-04-26T02:01:59.660 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T02:01:59.660 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T01:55:49.645549+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25414":{"gid":25414,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/453427322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":453427322},{"type":"v1","addr":"172.21.15.173:6835","nonce":453427322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24499,24514,24518]},"id":1}]} 2024-04-26T02:01:59.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.664+0000 7f60ce200700 1 -- 172.21.15.1:0/3850193503 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f60b00a3db0 msgr2=0x7f60b00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:59.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.664+0000 7f60ce200700 1 --2- 172.21.15.1:0/3850193503 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f60b00a3db0 0x7f60b00a6270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009fd0 tx=0x7f60b4019040 comp rx=0 tx=0).stop 2024-04-26T02:01:59.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.664+0000 7f60ce200700 1 -- 172.21.15.1:0/3850193503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60c8104dd0 msgr2=0x7f60c819b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:59.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.664+0000 7f60ce200700 1 --2- 172.21.15.1:0/3850193503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60c8104dd0 0x7f60c819b310 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00e9c0 tx=0x7f60bc00ecd0 comp rx=0 tx=0).stop 2024-04-26T02:01:59.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.664+0000 7f60ce200700 1 -- 172.21.15.1:0/3850193503 shutdown_connections 2024-04-26T02:01:59.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.664+0000 7f60ce200700 1 --2- 172.21.15.1:0/3850193503 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f60b00a3db0 0x7f60b00a6270 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:59.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.664+0000 7f60ce200700 1 --2- 172.21.15.1:0/3850193503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60c8109eb0 0x7f60c819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:59.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.664+0000 7f60ce200700 1 --2- 172.21.15.1:0/3850193503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60c8104dd0 0x7f60c819b310 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:59.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.664+0000 7f60ce200700 1 --2- 172.21.15.1:0/3850193503 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60c8105780 0x7f60c819b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:59.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.664+0000 7f60ce200700 1 -- 172.21.15.1:0/3850193503 >> 172.21.15.1:0/3850193503 conn(0x7f60c8100660 msgr2=0x7f60c8102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:59.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.665+0000 7f60ce200700 1 -- 172.21.15.1:0/3850193503 shutdown_connections 2024-04-26T02:01:59.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:01:59.665+0000 7f60ce200700 1 -- 172.21.15.1:0/3850193503 wait complete. 2024-04-26T02:01:59.665 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 32 2024-04-26T02:02:00.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:00 smithi001 ceph-mon[27822]: pgmap v322: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 51 KiB/s wr, 6 op/s 2024-04-26T02:02:00.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:00 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3850193503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:02:00.482 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24518 quiesce db --set-id b515cac0 --release --await' 2024-04-26T02:02:00.482 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24518 quiesce db --set-id b515cac0 --release --await 2024-04-26T02:02:00.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:00 smithi097 ceph-mon[32853]: pgmap v322: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 51 KiB/s wr, 6 op/s 2024-04-26T02:02:00.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:00 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3850193503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:02:00.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:00 smithi173 ceph-mon[32553]: pgmap v322: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 51 KiB/s wr, 6 op/s 2024-04-26T02:02:00.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:00 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3850193503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:02:00.978 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:02:02.176 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:02 smithi001 ceph-mon[27822]: pgmap v323: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 55 KiB/s wr, 10 op/s 2024-04-26T02:02:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:02 smithi097 ceph-mon[32853]: pgmap v323: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 55 KiB/s wr, 10 op/s 2024-04-26T02:02:02.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:02 smithi173 ceph-mon[32553]: pgmap v323: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 55 KiB/s wr, 10 op/s 2024-04-26T02:02:03.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.218+0000 7f3516600700 1 -- 172.21.15.1:0/1255427345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3510108aa0 msgr2=0x7f3510108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:03.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.218+0000 7f3516600700 1 --2- 172.21.15.1:0/1255427345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3510108aa0 0x7f3510108e80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f35000098f0 tx=0x7f3500009c00 comp rx=0 tx=0).stop 2024-04-26T02:02:03.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.218+0000 7f3516600700 1 -- 172.21.15.1:0/1255427345 shutdown_connections 2024-04-26T02:02:03.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.218+0000 7f3516600700 1 --2- 172.21.15.1:0/1255427345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35101034b0 0x7f351010f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.218+0000 7f3516600700 1 --2- 172.21.15.1:0/1255427345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3510108aa0 0x7f3510108e80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.218+0000 7f3516600700 1 --2- 172.21.15.1:0/1255427345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3510102af0 0x7f3510102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.218+0000 7f3516600700 1 -- 172.21.15.1:0/1255427345 >> 172.21.15.1:0/1255427345 conn(0x7f3510079b90 msgr2=0x7f3510079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:03.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.218+0000 7f3516600700 1 -- 172.21.15.1:0/1255427345 shutdown_connections 2024-04-26T02:02:03.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.218+0000 7f3516600700 1 -- 172.21.15.1:0/1255427345 wait complete. 2024-04-26T02:02:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.220+0000 7f3516600700 1 Processor -- start 2024-04-26T02:02:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.220+0000 7f3516600700 1 -- start start 2024-04-26T02:02:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.220+0000 7f3516600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3510102af0 0x7f351019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.221+0000 7f3516600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35101034b0 0x7f351019b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.221+0000 7f3516600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3510108aa0 0x7f351019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.221+0000 7f3516600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3510112cf0 con 0x7f35101034b0 2024-04-26T02:02:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.221+0000 7f3516600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3510112b70 con 0x7f3510108aa0 2024-04-26T02:02:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.221+0000 7f3516600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3510112e70 con 0x7f3510102af0 2024-04-26T02:02:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.221+0000 7f3506a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35101034b0 0x7f351019b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.221+0000 7f350f400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3510102af0 0x7f351019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.221+0000 7f350fe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3510108aa0 0x7f351019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.221+0000 7f3506a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35101034b0 0x7f351019b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45558/0 (socket says 172.21.15.1:45558) 2024-04-26T02:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.221+0000 7f3506a00700 1 -- 172.21.15.1:0/2895728308 learned_addr learned my addr 172.21.15.1:0/2895728308 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.221+0000 7f350f400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3510102af0 0x7f351019b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:47948/0 (socket says 172.21.15.1:47948) 2024-04-26T02:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.221+0000 7f3506a00700 1 -- 172.21.15.1:0/2895728308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3510102af0 msgr2=0x7f351019b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.221+0000 7f3506a00700 1 --2- 172.21.15.1:0/2895728308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3510102af0 0x7f351019b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.221+0000 7f3506a00700 1 -- 172.21.15.1:0/2895728308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3510108aa0 msgr2=0x7f351019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.221+0000 7f3506a00700 1 --2- 172.21.15.1:0/2895728308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3510108aa0 0x7f351019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.222+0000 7f3506a00700 1 -- 172.21.15.1:0/2895728308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35101a0450 con 0x7f35101034b0 2024-04-26T02:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.222+0000 7f3506a00700 1 --2- 172.21.15.1:0/2895728308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35101034b0 0x7f351019b980 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f34f800ec10 tx=0x7f34f800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.222+0000 7f350cc00700 1 -- 172.21.15.1:0/2895728308 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f800cc90 con 0x7f35101034b0 2024-04-26T02:02:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.222+0000 7f350cc00700 1 -- 172.21.15.1:0/2895728308 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34f800ce30 con 0x7f35101034b0 2024-04-26T02:02:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.222+0000 7f350cc00700 1 -- 172.21.15.1:0/2895728308 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8010430 con 0x7f35101034b0 2024-04-26T02:02:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.222+0000 7f3516600700 1 -- 172.21.15.1:0/2895728308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35101a0740 con 0x7f35101034b0 2024-04-26T02:02:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.222+0000 7f3516600700 1 -- 172.21.15.1:0/2895728308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35101a7fc0 con 0x7f35101034b0 2024-04-26T02:02:03.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.226+0000 7f350cc00700 1 -- 172.21.15.1:0/2895728308 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f34f8004000 con 0x7f35101034b0 2024-04-26T02:02:03.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.227+0000 7f3521061b80 1 Processor -- start 2024-04-26T02:02:03.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.228+0000 7f3521061b80 1 -- start start 2024-04-26T02:02:03.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.228+0000 7f350cc00700 1 --2- 172.21.15.1:0/2895728308 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f34f40a3db0 0x7f34f40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.228+0000 7f350cc00700 1 -- 172.21.15.1:0/2895728308 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f34f80d0b20 con 0x7f35101034b0 2024-04-26T02:02:03.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.228+0000 7f3521061b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e2c920 0x55a781e2cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.228+0000 7f3521061b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e37bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.228+0000 7f3521061b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a781e380f0 0x55a781e3a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.228+0000 7f3521061b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a781d7ff90 con 0x55a781e2d240 2024-04-26T02:02:03.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.228+0000 7f3521061b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a781d26830 con 0x55a781e380f0 2024-04-26T02:02:03.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.228+0000 7f3521061b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a781d7bf50 con 0x55a781e2c920 2024-04-26T02:02:03.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.228+0000 7f350fe00700 1 --2- 172.21.15.1:0/2895728308 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f34f40a3db0 0x7f34f40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.228+0000 7f350f400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e2c920 0x55a781e2cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.229+0000 7f350f400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e2c920 0x55a781e2cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:47960/0 (socket says 172.21.15.1:47960) 2024-04-26T02:02:03.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.229+0000 7f350f400700 1 -- 172.21.15.1:0/1106682359 learned_addr learned my addr 172.21.15.1:0/1106682359 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:03.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.229+0000 7f350fe00700 1 --2- 172.21.15.1:0/1106682359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a781e380f0 0x55a781e3a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.229+0000 7f3506a00700 1 --2- 172.21.15.1:0/1106682359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e37bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.229+0000 7f350f400700 1 -- 172.21.15.1:0/1106682359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a781e380f0 msgr2=0x55a781e3a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.229+0000 7f350f400700 1 --2- 172.21.15.1:0/1106682359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a781e380f0 0x55a781e3a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.229+0000 7f350f400700 1 -- 172.21.15.1:0/1106682359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 msgr2=0x55a781e37bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.229+0000 7f350f400700 1 --2- 172.21.15.1:0/1106682359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e37bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.229+0000 7f350f400700 1 -- 172.21.15.1:0/1106682359 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a781d1c780 con 0x55a781e2c920 2024-04-26T02:02:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.229+0000 7f350f400700 1 --2- 172.21.15.1:0/1106682359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e2c920 0x55a781e2cd00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f35000098f0 tx=0x7f350000bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2ac4fc632a41549 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.229+0000 7f350fe00700 1 --2- 172.21.15.1:0/2895728308 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f34f40a3db0 0x7f34f40a6270 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3508009ec0 tx=0x7f35080134a0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.230+0000 7f3504c00700 1 -- 172.21.15.1:0/1106682359 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500026070 con 0x55a781e2c920 2024-04-26T02:02:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.230+0000 7f3504c00700 1 -- 172.21.15.1:0/1106682359 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f350000f910 con 0x55a781e2c920 2024-04-26T02:02:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.230+0000 7f3504c00700 1 -- 172.21.15.1:0/1106682359 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500003680 con 0x55a781e2c920 2024-04-26T02:02:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.231+0000 7f3521061b80 1 -- 172.21.15.1:0/1106682359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e2c920 msgr2=0x55a781e2cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.231+0000 7f3521061b80 1 --2- 172.21.15.1:0/1106682359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e2c920 0x55a781e2cd00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f35000098f0 tx=0x7f350000bf60 comp rx=0 tx=0).stop 2024-04-26T02:02:03.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.231+0000 7f3521061b80 1 -- 172.21.15.1:0/1106682359 shutdown_connections 2024-04-26T02:02:03.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.231+0000 7f3521061b80 1 --2- 172.21.15.1:0/1106682359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a781e380f0 0x55a781e3a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.231+0000 7f3521061b80 1 --2- 172.21.15.1:0/1106682359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e2c920 0x55a781e2cd00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.231+0000 7f3521061b80 1 --2- 172.21.15.1:0/1106682359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e37bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.231+0000 7f3521061b80 1 -- 172.21.15.1:0/1106682359 >> 172.21.15.1:0/1106682359 conn(0x55a781d5b6a0 msgr2=0x55a781e28dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:03.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.231+0000 7f3521061b80 1 -- 172.21.15.1:0/1106682359 shutdown_connections 2024-04-26T02:02:03.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.231+0000 7f3521061b80 1 -- 172.21.15.1:0/1106682359 wait complete. 2024-04-26T02:02:03.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.232+0000 7f3521061b80 1 -- start start 2024-04-26T02:02:03.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.232+0000 7f3521061b80 1 Processor -- start 2024-04-26T02:02:03.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.232+0000 7f3521061b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a781e2c920 0x55a781e377d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.232+0000 7f3521061b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e34ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.232+0000 7f3521061b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e380f0 0x55a781e35010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.232+0000 7f3521061b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a781e3abb0 con 0x55a781e2d240 2024-04-26T02:02:03.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.232+0000 7f3521061b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a781d39960 con 0x55a781e2c920 2024-04-26T02:02:03.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.232+0000 7f3521061b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a781e3ad30 con 0x55a781e380f0 2024-04-26T02:02:03.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.232+0000 7f3521061b80 10 client.0 _ll_register_callbacks cb 0x55a781d18820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:03.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.233+0000 7f3506a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e34ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.233+0000 7f3506a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e34ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45588/0 (socket says 172.21.15.1:45588) 2024-04-26T02:02:03.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.233+0000 7f350f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a781e2c920 0x55a781e377d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.233+0000 7f3506a00700 1 -- 172.21.15.1:0/2215511674 learned_addr learned my addr 172.21.15.1:0/2215511674 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:03.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.233+0000 7f350fe00700 1 --2- 172.21.15.1:0/2215511674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e380f0 0x55a781e35010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.233+0000 7f3506a00700 1 -- 172.21.15.1:0/2215511674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e380f0 msgr2=0x55a781e35010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:03.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.233+0000 7f3506a00700 1 --2- 172.21.15.1:0/2215511674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e380f0 0x55a781e35010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.233+0000 7f3506a00700 1 -- 172.21.15.1:0/2215511674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a781e2c920 msgr2=0x55a781e377d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:03.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.233+0000 7f3506a00700 1 --2- 172.21.15.1:0/2215511674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a781e2c920 0x55a781e377d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.233+0000 7f3506a00700 1 -- 172.21.15.1:0/2215511674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a781ccd9d0 con 0x55a781e2d240 2024-04-26T02:02:03.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.234+0000 7f3506a00700 1 --2- 172.21.15.1:0/2215511674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e34ad0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f34f8000c00 tx=0x7f34f809c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429611a38a690ea9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:03.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.234+0000 7f34fd600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T02:02:03.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.234+0000 7f34fd600700 1 -- 172.21.15.1:0/2215511674 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f809ca30 con 0x55a781e2d240 2024-04-26T02:02:03.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.234+0000 7f34fd600700 1 -- 172.21.15.1:0/2215511674 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f34f80042f0 con 0x55a781e2d240 2024-04-26T02:02:03.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.234+0000 7f3521061b80 1 -- 172.21.15.1:0/2215511674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f350000f910 con 0x55a781e2d240 2024-04-26T02:02:03.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.234+0000 7f34fd600700 1 -- 172.21.15.1:0/2215511674 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f34f80044f0 con 0x55a781e2d240 2024-04-26T02:02:03.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.234+0000 7f3521061b80 10 client.16272 fetch_fsmap learned FSMap version 32 2024-04-26T02:02:03.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.234+0000 7f3521061b80 1 -- 172.21.15.1:0/2215511674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a781d16bf0 con 0x55a781e2d240 2024-04-26T02:02:03.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.234+0000 7f34fd600700 1 -- 172.21.15.1:0/2215511674 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f350000f910 con 0x55a781e2d240 2024-04-26T02:02:03.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.235+0000 7f34fd600700 1 -- 172.21.15.1:0/2215511674 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f34f80a5210 con 0x55a781e2d240 2024-04-26T02:02:03.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.235+0000 7f3521061b80 10 client.16272 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:02:03.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.235+0000 7f3521061b80 10 client.16272 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T02:02:03.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.235+0000 7f3521061b80 20 client.16272 populate_metadata read hostname 'smithi001' 2024-04-26T02:02:03.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.235+0000 7f3521061b80 1 --2- 172.21.15.1:0/2215511674 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a781e47840 0x55a781e49c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.235+0000 7f3521061b80 4 client.16272 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:02:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.235+0000 7f3521061b80 1 -- 172.21.15.1:0/2215511674 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a781d8d780 con 0x55a781e47840 2024-04-26T02:02:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.235+0000 7f350f400700 1 --2- 172.21.15.1:0/2215511674 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a781e47840 0x55a781e49c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.236+0000 7f350f400700 1 --2- 172.21.15.1:0/2215511674 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a781e47840 0x55a781e49c30 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ceac1609cd1ff5c server_cookie=69e9cfacc2dd023f in_seq=0 out_seq=0 2024-04-26T02:02:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.236+0000 7f34fd600700 10 client.16272 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:02:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.237+0000 7f34fd600700 1 -- 172.21.15.1:0/2215511674 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a781d8d780 con 0x55a781e47840 2024-04-26T02:02:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.237+0000 7f34fd600700 10 client.16272 handle_command_reply: tid=0 2024-04-26T02:02:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.237+0000 7f3521061b80 1 client.16272 shutdown 2024-04-26T02:02:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.237+0000 7f3521061b80 1 -- 172.21.15.1:0/2215511674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 msgr2=0x55a781e34ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.237+0000 7f3521061b80 1 --2- 172.21.15.1:0/2215511674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e34ad0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f34f8000c00 tx=0x7f34f809c640 comp rx=0 tx=0).stop 2024-04-26T02:02:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.237+0000 7f3521061b80 1 -- 172.21.15.1:0/2215511674 shutdown_connections 2024-04-26T02:02:03.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.237+0000 7f3521061b80 1 --2- 172.21.15.1:0/2215511674 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a781e47840 0x55a781e49c30 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.237+0000 7f3521061b80 1 --2- 172.21.15.1:0/2215511674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e380f0 0x55a781e35010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.237+0000 7f3521061b80 1 --2- 172.21.15.1:0/2215511674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a781e2c920 0x55a781e377d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.237+0000 7f3521061b80 1 --2- 172.21.15.1:0/2215511674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e34ad0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.237+0000 7f3521061b80 1 -- 172.21.15.1:0/2215511674 >> 172.21.15.1:0/2215511674 conn(0x55a781d5b6a0 msgr2=0x55a781e289c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:03.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.237+0000 7f34fd600700 0 client.16272 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:02:03.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.237+0000 7f3521061b80 1 -- 172.21.15.1:0/2215511674 shutdown_connections 2024-04-26T02:02:03.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.237+0000 7f3521061b80 1 -- 172.21.15.1:0/2215511674 wait complete. 2024-04-26T02:02:03.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.237+0000 7f3521061b80 20 client.16272 trim_cache size 0 max 16384 2024-04-26T02:02:03.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.259+0000 7f3521061b80 1 Processor -- start 2024-04-26T02:02:03.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.259+0000 7f3521061b80 1 -- start start 2024-04-26T02:02:03.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.259+0000 7f3521061b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a781e2c920 0x55a781e4f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.259+0000 7f3521061b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e4f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.259+0000 7f3521061b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e380f0 0x55a781e394e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.259+0000 7f3521061b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f35101b1f70 con 0x55a781e2d240 2024-04-26T02:02:03.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.259+0000 7f3521061b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f35101af800 con 0x55a781e2c920 2024-04-26T02:02:03.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.259+0000 7f3521061b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f35101ad670 con 0x55a781e380f0 2024-04-26T02:02:03.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.260+0000 7f3506a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e4f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.260+0000 7f350fe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e380f0 0x55a781e394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.260+0000 7f350fe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e380f0 0x55a781e394e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:47976/0 (socket says 172.21.15.1:47976) 2024-04-26T02:02:03.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.260+0000 7f3506a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e4f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45598/0 (socket says 172.21.15.1:45598) 2024-04-26T02:02:03.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.260+0000 7f350fe00700 1 -- 172.21.15.1:0/4183231309 learned_addr learned my addr 172.21.15.1:0/4183231309 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:03.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.260+0000 7f350f400700 1 --2- 172.21.15.1:0/4183231309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a781e2c920 0x55a781e4f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.260+0000 7f3506a00700 1 -- 172.21.15.1:0/4183231309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e380f0 msgr2=0x55a781e394e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:03.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.260+0000 7f3506a00700 1 --2- 172.21.15.1:0/4183231309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e380f0 0x55a781e394e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.260+0000 7f3506a00700 1 -- 172.21.15.1:0/4183231309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a781e2c920 msgr2=0x55a781e4f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:03.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.260+0000 7f3506a00700 1 --2- 172.21.15.1:0/4183231309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a781e2c920 0x55a781e4f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.260+0000 7f3506a00700 1 -- 172.21.15.1:0/4183231309 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a781e2d620 con 0x55a781e2d240 2024-04-26T02:02:03.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.261+0000 7f3506a00700 1 --2- 172.21.15.1:0/4183231309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e4f9d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f34f8098e90 tx=0x7f34f809af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ba1cfdc6a3c74a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:03.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.261+0000 7f34ffe00700 1 -- 172.21.15.1:0/4183231309 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f809c390 con 0x55a781e2d240 2024-04-26T02:02:03.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.261+0000 7f34ffe00700 1 -- 172.21.15.1:0/4183231309 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34f8067570 con 0x55a781e2d240 2024-04-26T02:02:03.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.261+0000 7f34ffe00700 1 -- 172.21.15.1:0/4183231309 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f80a0de0 con 0x55a781e2d240 2024-04-26T02:02:03.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.262+0000 7f3521061b80 1 -- 172.21.15.1:0/4183231309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 msgr2=0x55a781e4f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:03.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.262+0000 7f3521061b80 1 --2- 172.21.15.1:0/4183231309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e4f9d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f34f8098e90 tx=0x7f34f809af40 comp rx=0 tx=0).stop 2024-04-26T02:02:03.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.262+0000 7f3521061b80 1 -- 172.21.15.1:0/4183231309 shutdown_connections 2024-04-26T02:02:03.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.262+0000 7f3521061b80 1 --2- 172.21.15.1:0/4183231309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e380f0 0x55a781e394e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.262+0000 7f3521061b80 1 --2- 172.21.15.1:0/4183231309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a781e2c920 0x55a781e4f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.262+0000 7f3521061b80 1 --2- 172.21.15.1:0/4183231309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e4f9d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.262+0000 7f3521061b80 1 -- 172.21.15.1:0/4183231309 >> 172.21.15.1:0/4183231309 conn(0x55a781d5b6a0 msgr2=0x55a781e333c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:03.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.262+0000 7f3521061b80 1 -- 172.21.15.1:0/4183231309 shutdown_connections 2024-04-26T02:02:03.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.262+0000 7f3521061b80 1 -- 172.21.15.1:0/4183231309 wait complete. 2024-04-26T02:02:03.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.262+0000 7f3521061b80 1 -- start start 2024-04-26T02:02:03.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.263+0000 7f3521061b80 1 Processor -- start 2024-04-26T02:02:03.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.263+0000 7f3521061b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a781e2c920 0x55a781e401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.263+0000 7f3521061b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e30d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.263+0000 7f3521061b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e380f0 0x55a781e312a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.263+0000 7f3521061b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f35101adcc0 con 0x55a781e2d240 2024-04-26T02:02:03.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.263+0000 7f3521061b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f35101af130 con 0x55a781e2c920 2024-04-26T02:02:03.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.263+0000 7f3521061b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f35101ae390 con 0x55a781e380f0 2024-04-26T02:02:03.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.263+0000 7f3521061b80 10 client.0 _ll_register_callbacks cb 0x55a781d18820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:03.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.264+0000 7f3506a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e30d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.264+0000 7f3506a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e30d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45610/0 (socket says 172.21.15.1:45610) 2024-04-26T02:02:03.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.264+0000 7f350fe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e380f0 0x55a781e312a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.264+0000 7f3506a00700 1 -- 172.21.15.1:0/66302092 learned_addr learned my addr 172.21.15.1:0/66302092 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:03.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.264+0000 7f350f400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a781e2c920 0x55a781e401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.264+0000 7f3506a00700 1 -- 172.21.15.1:0/66302092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e380f0 msgr2=0x55a781e312a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:03.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.264+0000 7f3506a00700 1 --2- 172.21.15.1:0/66302092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e380f0 0x55a781e312a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.264+0000 7f3506a00700 1 -- 172.21.15.1:0/66302092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a781e2c920 msgr2=0x55a781e401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:03.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.264+0000 7f3506a00700 1 --2- 172.21.15.1:0/66302092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a781e2c920 0x55a781e401a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.264+0000 7f3506a00700 1 -- 172.21.15.1:0/66302092 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a781e317e0 con 0x55a781e2d240 2024-04-26T02:02:03.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.264+0000 7f3506a00700 1 --2- 172.21.15.1:0/66302092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e30d60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f34f801b040 tx=0x7f34f809c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b6a7c9b6ac5668 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:03.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.264+0000 7f34fd600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T02:02:03.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.265+0000 7f34fd600700 1 -- 172.21.15.1:0/66302092 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f809c390 con 0x55a781e2d240 2024-04-26T02:02:03.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.265+0000 7f34fd600700 1 -- 172.21.15.1:0/66302092 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f34f80672e0 con 0x55a781e2d240 2024-04-26T02:02:03.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.265+0000 7f3521061b80 1 -- 172.21.15.1:0/66302092 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34f8067570 con 0x55a781e2d240 2024-04-26T02:02:03.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.265+0000 7f34fd600700 1 -- 172.21.15.1:0/66302092 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f34f80bd0b0 con 0x55a781e2d240 2024-04-26T02:02:03.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.265+0000 7f3521061b80 10 client.16278 fetch_fsmap learned FSMap version 32 2024-04-26T02:02:03.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.265+0000 7f3521061b80 1 -- 172.21.15.1:0/66302092 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a781e32b20 con 0x55a781e2d240 2024-04-26T02:02:03.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.265+0000 7f34fd600700 1 -- 172.21.15.1:0/66302092 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34f8067570 con 0x55a781e2d240 2024-04-26T02:02:03.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.265+0000 7f34fd600700 1 -- 172.21.15.1:0/66302092 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f34f8098890 con 0x55a781e2d240 2024-04-26T02:02:03.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.266+0000 7f3521061b80 10 client.16278 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:02:03.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.266+0000 7f3521061b80 10 client.16278 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T02:02:03.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.266+0000 7f3521061b80 20 client.16278 populate_metadata read hostname 'smithi001' 2024-04-26T02:02:03.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.266+0000 7f3521061b80 1 --2- 172.21.15.1:0/66302092 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a781eaef40 0x55a781eb1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:03.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.266+0000 7f3521061b80 4 client.16278 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b515cac0", "release": true, "await": true}] 2024-04-26T02:02:03.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.266+0000 7f3521061b80 1 -- 172.21.15.1:0/66302092 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b515cac0", "release": true, "await": true}) -- 0x55a781d87820 con 0x55a781eaef40 2024-04-26T02:02:03.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.266+0000 7f350f400700 1 --2- 172.21.15.1:0/66302092 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a781eaef40 0x55a781eb1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:03.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.267+0000 7f350f400700 1 --2- 172.21.15.1:0/66302092 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a781eaef40 0x55a781eb1360 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7e771b518194726 server_cookie=4daf666f3623568 in_seq=0 out_seq=0 2024-04-26T02:02:03.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.267+0000 7f34fd600700 10 client.16278 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:02:03.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.455+0000 7f34fd600700 1 -- 172.21.15.1:0/66302092 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55a781d87820 con 0x55a781eaef40 2024-04-26T02:02:03.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.455+0000 7f34fd600700 10 client.16278 handle_command_reply: tid=0 2024-04-26T02:02:03.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.455+0000 7f3521061b80 1 client.16278 shutdown 2024-04-26T02:02:03.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.455+0000 7f3521061b80 1 -- 172.21.15.1:0/66302092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 msgr2=0x55a781e30d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:03.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.455+0000 7f3521061b80 1 --2- 172.21.15.1:0/66302092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e30d60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f34f801b040 tx=0x7f34f809c640 comp rx=0 tx=0).stop 2024-04-26T02:02:03.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.456+0000 7f3521061b80 1 -- 172.21.15.1:0/66302092 shutdown_connections 2024-04-26T02:02:03.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.456+0000 7f3521061b80 1 --2- 172.21.15.1:0/66302092 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a781eaef40 0x55a781eb1360 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.456+0000 7f3521061b80 1 --2- 172.21.15.1:0/66302092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a781e380f0 0x55a781e312a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.456+0000 7f3521061b80 1 --2- 172.21.15.1:0/66302092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a781e2c920 0x55a781e401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.456+0000 7f3521061b80 1 --2- 172.21.15.1:0/66302092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a781e2d240 0x55a781e30d60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.456+0000 7f34fd600700 0 client.16278 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:02:03.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.456+0000 7f3521061b80 1 -- 172.21.15.1:0/66302092 >> 172.21.15.1:0/66302092 conn(0x55a781d5b6a0 msgr2=0x55a781e36300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:03.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.456+0000 7f3521061b80 1 -- 172.21.15.1:0/66302092 shutdown_connections 2024-04-26T02:02:03.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.456+0000 7f3521061b80 1 -- 172.21.15.1:0/66302092 wait complete. 2024-04-26T02:02:03.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.457+0000 7f3521061b80 20 client.16278 trim_cache size 0 max 16384 2024-04-26T02:02:03.456 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T02:02:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 32, 2024-04-26T02:02:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24518, 2024-04-26T02:02:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 32, 2024-04-26T02:02:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-04-26T02:02:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "b515cac0": { 2024-04-26T02:02:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 32, 2024-04-26T02:02:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-04-26T02:02:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T02:02:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-04-26T02:02:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T02:02:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:02:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-04-26T02:02:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 82.1, 2024-04-26T02:02:03.457 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-04-26T02:02:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-04-26T02:02:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-04-26T02:02:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T02:02:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-04-26T02:02:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T02:02:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:02:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:02:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:02:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:02:03.458 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:02:03.458 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T02:02:03.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.458+0000 7f3516600700 1 -- 172.21.15.1:0/2895728308 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f34f40a3db0 msgr2=0x7f34f40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:03.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.458+0000 7f3516600700 1 --2- 172.21.15.1:0/2895728308 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f34f40a3db0 0x7f34f40a6270 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3508009ec0 tx=0x7f35080134a0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.458+0000 7f3516600700 1 -- 172.21.15.1:0/2895728308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35101034b0 msgr2=0x7f351019b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:03.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.458+0000 7f3516600700 1 --2- 172.21.15.1:0/2895728308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35101034b0 0x7f351019b980 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f34f800ec10 tx=0x7f34f800ef20 comp rx=0 tx=0).stop 2024-04-26T02:02:03.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.461+0000 7f3516600700 1 -- 172.21.15.1:0/2895728308 shutdown_connections 2024-04-26T02:02:03.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.461+0000 7f3516600700 1 --2- 172.21.15.1:0/2895728308 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f34f40a3db0 0x7f34f40a6270 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.461+0000 7f3516600700 1 --2- 172.21.15.1:0/2895728308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3510108aa0 0x7f351019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.461+0000 7f3516600700 1 --2- 172.21.15.1:0/2895728308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3510102af0 0x7f351019b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.461+0000 7f3516600700 1 --2- 172.21.15.1:0/2895728308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35101034b0 0x7f351019b980 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:03.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.461+0000 7f3516600700 1 -- 172.21.15.1:0/2895728308 >> 172.21.15.1:0/2895728308 conn(0x7f3510079b90 msgr2=0x7f351010dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:03.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.463+0000 7f3516600700 1 -- 172.21.15.1:0/2895728308 shutdown_connections 2024-04-26T02:02:03.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:02:03.463+0000 7f3516600700 1 -- 172.21.15.1:0/2895728308 wait complete. 2024-04-26T02:02:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:04 smithi001 ceph-mon[27822]: pgmap v324: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-26T02:02:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:04 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:02:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:04 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:04 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:04 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:04 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:04.302 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b515cac0', seconds elapsed: 38.5 2024-04-26T02:02:04.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:04 smithi097 ceph-mon[32853]: pgmap v324: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-26T02:02:04.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:04 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:02:04.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:04 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:04.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:04 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:04.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:04 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:04.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:04 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:04.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:04 smithi173 ceph-mon[32553]: pgmap v324: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-26T02:02:04.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:04 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:02:04.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:04 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:04.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:04 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:04.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:04 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:04.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:04 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:06.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:06 smithi173 ceph-mon[32553]: pgmap v325: 97 pgs: 97 active+clean; 156 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 229 KiB/s wr, 77 op/s 2024-04-26T02:02:06.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:06 smithi173 ceph-mon[32553]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T02:02:06.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:06 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:06.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:06 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:06.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:06 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:06.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:06 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:06.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:06 smithi001 ceph-mon[27822]: pgmap v325: 97 pgs: 97 active+clean; 156 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 229 KiB/s wr, 77 op/s 2024-04-26T02:02:06.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:06 smithi001 ceph-mon[27822]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T02:02:06.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:06 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:06.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:06 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:06.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:06 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:06.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:06 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:06.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:06 smithi097 ceph-mon[32853]: pgmap v325: 97 pgs: 97 active+clean; 156 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 229 KiB/s wr, 77 op/s 2024-04-26T02:02:06.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:06 smithi097 ceph-mon[32853]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T02:02:06.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:06 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:06.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:06 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:06.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:06 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:06.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:06 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:07.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:07 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:07 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:07 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:07 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:07 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:07 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:07 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:07 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:07 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:07 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:07 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:07 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:07 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.569 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:07 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.569 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:07 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.569 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:07 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.569 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:07 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:07.569 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:07 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:07.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:07 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:07 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:07 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:07 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:07 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:07 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:07 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:07 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:07 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:07 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:07 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:07 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:07 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:07 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:07 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:07 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:07 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:07.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:07 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:07.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:07 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:07 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:07 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:07 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:07 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:07 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:07 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:07 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:07 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:07 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:07 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:07 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:07 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:07 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:07 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:07 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:07.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:07 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:07.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:07 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:08 smithi173 ceph-mon[32553]: Adjusting osd_memory_target on smithi173 to 176.2M 2024-04-26T02:02:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:08 smithi173 ceph-mon[32553]: Unable to set osd_memory_target on smithi173 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-26T02:02:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:08 smithi173 ceph-mon[32553]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-26T02:02:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:08 smithi173 ceph-mon[32553]: Unable to set osd_memory_target on smithi097 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-26T02:02:08.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:08 smithi173 ceph-mon[32553]: pgmap v327: 97 pgs: 97 active+clean; 156 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 275 KiB/s wr, 90 op/s 2024-04-26T02:02:08.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:08 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:08 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:08 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:08 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:08 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:08 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:08 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:08 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:08 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:02:08.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:08 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:02:08.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:08 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:08.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:08 smithi001 ceph-mon[27822]: Adjusting osd_memory_target on smithi173 to 176.2M 2024-04-26T02:02:08.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:08 smithi001 ceph-mon[27822]: Unable to set osd_memory_target on smithi173 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-26T02:02:08.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:08 smithi001 ceph-mon[27822]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-26T02:02:08.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:08 smithi001 ceph-mon[27822]: Unable to set osd_memory_target on smithi097 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-26T02:02:08.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:08 smithi001 ceph-mon[27822]: pgmap v327: 97 pgs: 97 active+clean; 156 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 275 KiB/s wr, 90 op/s 2024-04-26T02:02:08.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:08 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:08 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:08 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:08 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:08 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:08 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:08 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:08 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:08 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:02:08.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:08 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:02:08.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:08 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:08.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:08 smithi097 ceph-mon[32853]: Adjusting osd_memory_target on smithi173 to 176.2M 2024-04-26T02:02:08.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:08 smithi097 ceph-mon[32853]: Unable to set osd_memory_target on smithi173 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-26T02:02:08.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:08 smithi097 ceph-mon[32853]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-26T02:02:08.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:08 smithi097 ceph-mon[32853]: Unable to set osd_memory_target on smithi097 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-26T02:02:08.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:08 smithi097 ceph-mon[32853]: pgmap v327: 97 pgs: 97 active+clean; 156 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 275 KiB/s wr, 90 op/s 2024-04-26T02:02:08.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:08 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:08 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:08 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:08 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:08 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:08 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:08 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:08 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:02:08.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:08 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:02:08.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:08 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:02:08.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:08 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:02:09.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.007+0000 7f81d1a00640 1 -- 172.21.15.1:0/2995552095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81cc107270 msgr2=0x7f81cc10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:09.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.007+0000 7f81d1a00640 1 --2- 172.21.15.1:0/2995552095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81cc107270 0x7f81cc10ee40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f81c4009ca0 tx=0x7f81c402a8c0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.007+0000 7f81d1a00640 1 -- 172.21.15.1:0/2995552095 shutdown_connections 2024-04-26T02:02:09.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.007+0000 7f81d1a00640 1 --2- 172.21.15.1:0/2995552095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81cc107270 0x7f81cc10ee40 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.007+0000 7f81d1a00640 1 --2- 172.21.15.1:0/2995552095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81cc102b50 0x7f81cc106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.007+0000 7f81d1a00640 1 --2- 172.21.15.1:0/2995552095 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81cc1021a0 0x7f81cc102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.007+0000 7f81d1a00640 1 -- 172.21.15.1:0/2995552095 >> 172.21.15.1:0/2995552095 conn(0x7f81cc0fda40 msgr2=0x7f81cc0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.007+0000 7f81d1a00640 1 -- 172.21.15.1:0/2995552095 shutdown_connections 2024-04-26T02:02:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.007+0000 7f81d1a00640 1 -- 172.21.15.1:0/2995552095 wait complete. 2024-04-26T02:02:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.008+0000 7f81d1a00640 1 Processor -- start 2024-04-26T02:02:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.008+0000 7f81d1a00640 1 -- start start 2024-04-26T02:02:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.009+0000 7f81d1a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81cc1021a0 0x7f81cc1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.009+0000 7f81d1a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81cc107270 0x7f81cc1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.009+0000 7f81d1a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81cc19a740 0x7f81cc19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.009+0000 7f81d1a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc112210 con 0x7f81cc19a740 2024-04-26T02:02:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.009+0000 7f81d1a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc112090 con 0x7f81cc107270 2024-04-26T02:02:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.009+0000 7f81d1a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc112390 con 0x7f81cc1021a0 2024-04-26T02:02:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.009+0000 7f81c3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81cc107270 0x7f81cc1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.009+0000 7f81c3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81cc107270 0x7f81cc1a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:49226/0 (socket says 172.21.15.1:49226) 2024-04-26T02:02:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.009+0000 7f81c3e00640 1 -- 172.21.15.1:0/3875771259 learned_addr learned my addr 172.21.15.1:0/3875771259 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.009+0000 7f81caa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81cc1021a0 0x7f81cc1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.009+0000 7f81cb400640 1 --2- 172.21.15.1:0/3875771259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81cc19a740 0x7f81cc19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.010+0000 7f81c3e00640 1 -- 172.21.15.1:0/3875771259 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81cc1021a0 msgr2=0x7f81cc1a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.010+0000 7f81c3e00640 1 --2- 172.21.15.1:0/3875771259 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81cc1021a0 0x7f81cc1a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.010+0000 7f81c3e00640 1 -- 172.21.15.1:0/3875771259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81cc19a740 msgr2=0x7f81cc19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.010+0000 7f81c3e00640 1 --2- 172.21.15.1:0/3875771259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81cc19a740 0x7f81cc19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.010+0000 7f81c3e00640 1 -- 172.21.15.1:0/3875771259 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81cc19b340 con 0x7f81cc107270 2024-04-26T02:02:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.010+0000 7f81caa00640 1 --2- 172.21.15.1:0/3875771259 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81cc1021a0 0x7f81cc1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.010+0000 7f81c3e00640 1 --2- 172.21.15.1:0/3875771259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81cc107270 0x7f81cc1a0bc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f81b400e9a0 tx=0x7f81b400ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.010+0000 7f81c3400640 1 -- 172.21.15.1:0/3875771259 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b400cd80 con 0x7f81cc107270 2024-04-26T02:02:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.010+0000 7f81c3400640 1 -- 172.21.15.1:0/3875771259 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81b40035f0 con 0x7f81cc107270 2024-04-26T02:02:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.010+0000 7f81d1a00640 1 -- 172.21.15.1:0/3875771259 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81cc1a7420 con 0x7f81cc107270 2024-04-26T02:02:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.010+0000 7f81c3400640 1 -- 172.21.15.1:0/3875771259 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4010690 con 0x7f81cc107270 2024-04-26T02:02:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.010+0000 7f81d1a00640 1 -- 172.21.15.1:0/3875771259 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81cc1a7820 con 0x7f81cc107270 2024-04-26T02:02:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.014+0000 7f81c3400640 1 -- 172.21.15.1:0/3875771259 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f81b4004000 con 0x7f81cc107270 2024-04-26T02:02:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.014+0000 7f81d4985740 1 Processor -- start 2024-04-26T02:02:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.014+0000 7f81d4985740 1 -- start start 2024-04-26T02:02:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.015+0000 7f81d4985740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5b92760 0x55aee5b92b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.015+0000 7f81d4985740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aee5b93080 0x55aee5ba5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.015+0000 7f81d4985740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5ba6210 0x55aee5ba8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.015+0000 7f81d4985740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55aee5af8900 con 0x55aee5b93080 2024-04-26T02:02:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.015+0000 7f81d4985740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55aee5a9eef0 con 0x55aee5ba6210 2024-04-26T02:02:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.015+0000 7f81d4985740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55aee5af48c0 con 0x55aee5b92760 2024-04-26T02:02:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.015+0000 7f81cb400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5b92760 0x55aee5b92b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.015+0000 7f81cb400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5b92760 0x55aee5b92b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:48030/0 (socket says 172.21.15.1:48030) 2024-04-26T02:02:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.015+0000 7f81cb400640 1 -- 172.21.15.1:0/3035240157 learned_addr learned my addr 172.21.15.1:0/3035240157 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.015+0000 7f81caa00640 1 --2- 172.21.15.1:0/3035240157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aee5b93080 0x55aee5ba5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.015+0000 7f81c3e00640 1 --2- 172.21.15.1:0/3035240157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5ba6210 0x55aee5ba8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.015+0000 7f81cb400640 1 -- 172.21.15.1:0/3035240157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5ba6210 msgr2=0x55aee5ba8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.015+0000 7f81cb400640 1 --2- 172.21.15.1:0/3035240157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5ba6210 0x55aee5ba8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.015+0000 7f81cb400640 1 -- 172.21.15.1:0/3035240157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aee5b93080 msgr2=0x55aee5ba5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.015+0000 7f81cb400640 1 --2- 172.21.15.1:0/3035240157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aee5b93080 0x55aee5ba5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.015+0000 7f81cb400640 1 -- 172.21.15.1:0/3035240157 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aee580b210 con 0x55aee5b92760 2024-04-26T02:02:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.015+0000 7f81c3e00640 1 --2- 172.21.15.1:0/3035240157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5ba6210 0x55aee5ba8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:02:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.015+0000 7f81caa00640 1 --2- 172.21.15.1:0/3035240157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aee5b93080 0x55aee5ba5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:02:09.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.016+0000 7f81cb400640 1 --2- 172.21.15.1:0/3035240157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5b92760 0x55aee5b92b40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f81c4009ca0 tx=0x7f81c40093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dd26ce4ec8d5b7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:09.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.016+0000 7f81c3400640 1 --2- 172.21.15.1:0/3875771259 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f81b00a3d10 0x7f81b00a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:09.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.016+0000 7f81bbe00640 1 -- 172.21.15.1:0/3035240157 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4031cf0 con 0x55aee5b92760 2024-04-26T02:02:09.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.016+0000 7f81bbe00640 1 -- 172.21.15.1:0/3035240157 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c4036030 con 0x55aee5b92760 2024-04-26T02:02:09.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.016+0000 7f81bbe00640 1 -- 172.21.15.1:0/3035240157 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c40042a0 con 0x55aee5b92760 2024-04-26T02:02:09.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.016+0000 7f81caa00640 1 --2- 172.21.15.1:0/3875771259 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f81b00a3d10 0x7f81b00a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:09.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.016+0000 7f81c3400640 1 -- 172.21.15.1:0/3875771259 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8615+0+0 (secure 0 0 0) 0x7f81b40d0a80 con 0x7f81cc107270 2024-04-26T02:02:09.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.017+0000 7f81d4985740 1 -- 172.21.15.1:0/3035240157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5b92760 msgr2=0x55aee5b92b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:09.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.017+0000 7f81d4985740 1 --2- 172.21.15.1:0/3035240157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5b92760 0x55aee5b92b40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f81c4009ca0 tx=0x7f81c40093e0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.017+0000 7f81d4985740 1 -- 172.21.15.1:0/3035240157 shutdown_connections 2024-04-26T02:02:09.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.017+0000 7f81caa00640 1 --2- 172.21.15.1:0/3875771259 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f81b00a3d10 0x7f81b00a61d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f81bc012640 tx=0x7f81bc005f90 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:09.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.017+0000 7f81d4985740 1 --2- 172.21.15.1:0/3035240157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5ba6210 0x55aee5ba8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.017+0000 7f81d4985740 1 --2- 172.21.15.1:0/3035240157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aee5b93080 0x55aee5ba5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.017+0000 7f81d4985740 1 --2- 172.21.15.1:0/3035240157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5b92760 0x55aee5b92b40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.017+0000 7f81d4985740 1 -- 172.21.15.1:0/3035240157 >> 172.21.15.1:0/3035240157 conn(0x55aee5b8ed30 msgr2=0x55aee5b8f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.017+0000 7f81d4985740 1 -- 172.21.15.1:0/3035240157 shutdown_connections 2024-04-26T02:02:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.017+0000 7f81d4985740 1 -- 172.21.15.1:0/3035240157 wait complete. 2024-04-26T02:02:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.018+0000 7f81d4985740 1 -- start start 2024-04-26T02:02:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.018+0000 7f81d4985740 1 Processor -- start 2024-04-26T02:02:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.018+0000 7f81d4985740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5b92760 0x55aee5ba31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:09.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.018+0000 7f81d4985740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aee5b93080 0x55aee5ba3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.019+0000 7f81d4985740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5ba6210 0x55aee5ba3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.019+0000 7f81d4985740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55aee5ba8c00 con 0x55aee5b93080 2024-04-26T02:02:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.019+0000 7f81d4985740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55aee5ab2310 con 0x55aee5b92760 2024-04-26T02:02:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.019+0000 7f81d4985740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55aee5ba8d80 con 0x55aee5ba6210 2024-04-26T02:02:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.019+0000 7f81d4985740 10 client.0 _ll_register_callbacks cb 0x55aee5a80cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.019+0000 7f81caa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5b92760 0x55aee5ba31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.019+0000 7f81caa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5b92760 0x55aee5ba31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:49254/0 (socket says 172.21.15.1:49254) 2024-04-26T02:02:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.019+0000 7f81caa00640 1 -- 172.21.15.1:0/3082086585 learned_addr learned my addr 172.21.15.1:0/3082086585 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.019+0000 7f81c3e00640 1 --2- 172.21.15.1:0/3082086585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aee5b93080 0x55aee5ba3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.019+0000 7f81cb400640 1 --2- 172.21.15.1:0/3082086585 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5ba6210 0x55aee5ba3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.019+0000 7f81caa00640 1 -- 172.21.15.1:0/3082086585 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5ba6210 msgr2=0x55aee5ba3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.019+0000 7f81caa00640 1 --2- 172.21.15.1:0/3082086585 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5ba6210 0x55aee5ba3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.019+0000 7f81caa00640 1 -- 172.21.15.1:0/3082086585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aee5b93080 msgr2=0x55aee5ba3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:09.021 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.019+0000 7f81caa00640 1 --2- 172.21.15.1:0/3082086585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aee5b93080 0x55aee5ba3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.019+0000 7f81caa00640 1 -- 172.21.15.1:0/3082086585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aee58d7620 con 0x55aee5b92760 2024-04-26T02:02:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.020+0000 7f81caa00640 1 --2- 172.21.15.1:0/3082086585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5b92760 0x55aee5ba31d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0058b0 tx=0x7f81bc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=393564c6dc01b49 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.020+0000 7f81b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:02:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.020+0000 7f81b8c00640 1 -- 172.21.15.1:0/3082086585 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc00f850 con 0x55aee5b92760 2024-04-26T02:02:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.020+0000 7f81b8c00640 1 -- 172.21.15.1:0/3082086585 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8615+0+0 (secure 0 0 0) 0x7f81bc00f9f0 con 0x55aee5b92760 2024-04-26T02:02:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.020+0000 7f81d4985740 1 -- 172.21.15.1:0/3082086585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81c4036030 con 0x55aee5b92760 2024-04-26T02:02:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.020+0000 7f81b8c00640 1 -- 172.21.15.1:0/3082086585 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f81bc00fbf0 con 0x55aee5b92760 2024-04-26T02:02:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.020+0000 7f81d4985740 10 client.26459 fetch_fsmap learned FSMap version 32 2024-04-26T02:02:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.020+0000 7f81d4985740 1 -- 172.21.15.1:0/3082086585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55aee5bb4f70 con 0x55aee5b92760 2024-04-26T02:02:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.021+0000 7f81b8c00640 1 -- 172.21.15.1:0/3082086585 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c4036030 con 0x55aee5b92760 2024-04-26T02:02:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.021+0000 7f81b8c00640 1 -- 172.21.15.1:0/3082086585 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f81bc027020 con 0x55aee5b92760 2024-04-26T02:02:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.021+0000 7f81d4985740 10 client.26459 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:02:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.021+0000 7f81d4985740 10 client.26459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:02:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.021+0000 7f81d4985740 20 client.26459 populate_metadata read hostname 'smithi001' 2024-04-26T02:02:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.021+0000 7f81d4985740 1 --2- 172.21.15.1:0/3082086585 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aee5bb6430 0x55aee5bb8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.021+0000 7f81d4985740 4 client.26459 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:02:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.021+0000 7f81d4985740 1 -- 172.21.15.1:0/3082086585 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aee5b0c8b0 con 0x55aee5bb6430 2024-04-26T02:02:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.021+0000 7f81c3e00640 1 --2- 172.21.15.1:0/3082086585 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aee5bb6430 0x55aee5bb8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.022+0000 7f81c3e00640 1 --2- 172.21.15.1:0/3082086585 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aee5bb6430 0x55aee5bb8860 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49fb73878ecf0e27 server_cookie=b2d0181eac8158ec in_seq=0 out_seq=0 2024-04-26T02:02:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.022+0000 7f81b8c00640 10 client.26459 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:02:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.022+0000 7f81b8c00640 1 -- 172.21.15.1:0/3082086585 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aee5b0c8b0 con 0x55aee5bb6430 2024-04-26T02:02:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.022+0000 7f81b8c00640 10 client.26459 handle_command_reply: tid=0 2024-04-26T02:02:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.022+0000 7f81d4985740 1 client.26459 shutdown 2024-04-26T02:02:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.023+0000 7f81d4985740 1 -- 172.21.15.1:0/3082086585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5b92760 msgr2=0x55aee5ba31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.023+0000 7f81d4985740 1 --2- 172.21.15.1:0/3082086585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5b92760 0x55aee5ba31d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0058b0 tx=0x7f81bc004730 comp rx=0 tx=0).stop 2024-04-26T02:02:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.023+0000 7f81d4985740 1 -- 172.21.15.1:0/3082086585 shutdown_connections 2024-04-26T02:02:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.023+0000 7f81d4985740 1 --2- 172.21.15.1:0/3082086585 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aee5bb6430 0x55aee5bb8860 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.023+0000 7f81d4985740 1 --2- 172.21.15.1:0/3082086585 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5ba6210 0x55aee5ba3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.023+0000 7f81d4985740 1 --2- 172.21.15.1:0/3082086585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aee5b93080 0x55aee5ba3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.023+0000 7f81d4985740 1 --2- 172.21.15.1:0/3082086585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5b92760 0x55aee5ba31d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.023+0000 7f81d4985740 1 -- 172.21.15.1:0/3082086585 >> 172.21.15.1:0/3082086585 conn(0x55aee5b8ed30 msgr2=0x55aee5b94ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.023+0000 7f81b8c00640 0 client.26459 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:02:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.023+0000 7f81d4985740 1 -- 172.21.15.1:0/3082086585 shutdown_connections 2024-04-26T02:02:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.023+0000 7f81d4985740 1 -- 172.21.15.1:0/3082086585 wait complete. 2024-04-26T02:02:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.023+0000 7f81d4985740 20 client.26459 trim_cache size 0 max 16384 2024-04-26T02:02:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.040+0000 7f81d4985740 1 Processor -- start 2024-04-26T02:02:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.040+0000 7f81d4985740 1 -- start start 2024-04-26T02:02:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.041+0000 7f81d4985740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5b92760 0x55aee5bb1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.041+0000 7f81d4985740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5b93080 0x55aee5b8dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.041+0000 7f81d4985740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aee5ba6210 0x55aee5bbe170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.041+0000 7f81d4985740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc1ae3f0 con 0x55aee5ba6210 2024-04-26T02:02:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.041+0000 7f81d4985740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc1abcc0 con 0x55aee5b93080 2024-04-26T02:02:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.041+0000 7f81d4985740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc1a9b90 con 0x55aee5b92760 2024-04-26T02:02:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.041+0000 7f81caa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5b92760 0x55aee5bb1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.041+0000 7f81cb400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aee5ba6210 0x55aee5bbe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.041+0000 7f81caa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5b92760 0x55aee5bb1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:48048/0 (socket says 172.21.15.1:48048) 2024-04-26T02:02:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.041+0000 7f81caa00640 1 -- 172.21.15.1:0/1723736614 learned_addr learned my addr 172.21.15.1:0/1723736614 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.041+0000 7f81c3e00640 1 --2- 172.21.15.1:0/1723736614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5b93080 0x55aee5b8dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.042+0000 7f81caa00640 1 -- 172.21.15.1:0/1723736614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5b93080 msgr2=0x55aee5b8dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.042+0000 7f81caa00640 1 --2- 172.21.15.1:0/1723736614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5b93080 0x55aee5b8dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.042+0000 7f81caa00640 1 -- 172.21.15.1:0/1723736614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aee5ba6210 msgr2=0x55aee5bbe170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.042+0000 7f81caa00640 1 --2- 172.21.15.1:0/1723736614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aee5ba6210 0x55aee5bbe170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.042+0000 7f81caa00640 1 -- 172.21.15.1:0/1723736614 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aee5b92eb0 con 0x55aee5b92760 2024-04-26T02:02:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.042+0000 7f81caa00640 1 --2- 172.21.15.1:0/1723736614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5b92760 0x55aee5bb1b00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0093e0 tx=0x7f81bc0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc83b61d64468b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.042+0000 7f81bb400640 1 -- 172.21.15.1:0/1723736614 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc00f850 con 0x55aee5b92760 2024-04-26T02:02:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.042+0000 7f81bb400640 1 -- 172.21.15.1:0/1723736614 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81bc00f9f0 con 0x55aee5b92760 2024-04-26T02:02:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.042+0000 7f81bb400640 1 -- 172.21.15.1:0/1723736614 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc01f500 con 0x55aee5b92760 2024-04-26T02:02:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.043+0000 7f81d4985740 1 -- 172.21.15.1:0/1723736614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5b92760 msgr2=0x55aee5bb1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.043+0000 7f81d4985740 1 --2- 172.21.15.1:0/1723736614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5b92760 0x55aee5bb1b00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0093e0 tx=0x7f81bc0057f0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.043+0000 7f81d4985740 1 -- 172.21.15.1:0/1723736614 shutdown_connections 2024-04-26T02:02:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.043+0000 7f81d4985740 1 --2- 172.21.15.1:0/1723736614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aee5ba6210 0x55aee5bbe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.043+0000 7f81d4985740 1 --2- 172.21.15.1:0/1723736614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5b93080 0x55aee5b8dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.043+0000 7f81d4985740 1 --2- 172.21.15.1:0/1723736614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5b92760 0x55aee5bb1b00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.043+0000 7f81d4985740 1 -- 172.21.15.1:0/1723736614 >> 172.21.15.1:0/1723736614 conn(0x55aee5b8ed30 msgr2=0x55aee5b8d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.043+0000 7f81d4985740 1 -- 172.21.15.1:0/1723736614 shutdown_connections 2024-04-26T02:02:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.043+0000 7f81d4985740 1 -- 172.21.15.1:0/1723736614 wait complete. 2024-04-26T02:02:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.044+0000 7f81d4985740 1 -- start start 2024-04-26T02:02:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.044+0000 7f81d4985740 1 Processor -- start 2024-04-26T02:02:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.045+0000 7f81d4985740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aee5b92760 0x55aee5babac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.045+0000 7f81d4985740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5b93080 0x55aee5bac000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.045+0000 7f81d4985740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5ba6210 0x55aee5bac540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.045+0000 7f81d4985740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc1aa1e0 con 0x55aee5b92760 2024-04-26T02:02:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.045+0000 7f81d4985740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc1ab5f0 con 0x55aee5ba6210 2024-04-26T02:02:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.045+0000 7f81d4985740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc1aa850 con 0x55aee5b93080 2024-04-26T02:02:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.045+0000 7f81d4985740 10 client.0 _ll_register_callbacks cb 0x7f81bc0044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.045+0000 7f81cb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5ba6210 0x55aee5bac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.045+0000 7f81cb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5ba6210 0x55aee5bac540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:49264/0 (socket says 172.21.15.1:49264) 2024-04-26T02:02:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.045+0000 7f81cb400640 1 -- 172.21.15.1:0/2816657523 learned_addr learned my addr 172.21.15.1:0/2816657523 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.045+0000 7f81caa00640 1 --2- 172.21.15.1:0/2816657523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aee5b92760 0x55aee5babac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.046+0000 7f81cb400640 1 -- 172.21.15.1:0/2816657523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5b93080 msgr2=0x55aee5bac000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.046+0000 7f81cb400640 1 --2- 172.21.15.1:0/2816657523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5b93080 0x55aee5bac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.046+0000 7f81cb400640 1 -- 172.21.15.1:0/2816657523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aee5b92760 msgr2=0x55aee5babac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.046+0000 7f81cb400640 1 --2- 172.21.15.1:0/2816657523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aee5b92760 0x55aee5babac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.046+0000 7f81cb400640 1 -- 172.21.15.1:0/2816657523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aee5c1ba20 con 0x55aee5ba6210 2024-04-26T02:02:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.046+0000 7f81cb400640 1 --2- 172.21.15.1:0/2816657523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5ba6210 0x55aee5bac540 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f81c4000f80 tx=0x7f81c402a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18a70b5c9b1b832 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.046+0000 7f81b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:02:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.046+0000 7f81b8c00640 1 -- 172.21.15.1:0/2816657523 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4055070 con 0x55aee5ba6210 2024-04-26T02:02:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.046+0000 7f81b8c00640 1 -- 172.21.15.1:0/2816657523 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8615+0+0 (secure 0 0 0) 0x7f81c4031650 con 0x55aee5ba6210 2024-04-26T02:02:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.046+0000 7f81d4985740 1 -- 172.21.15.1:0/2816657523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81bc00f9f0 con 0x55aee5ba6210 2024-04-26T02:02:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.047+0000 7f81b8c00640 1 -- 172.21.15.1:0/2816657523 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f81c404a040 con 0x55aee5ba6210 2024-04-26T02:02:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.047+0000 7f81d4985740 10 client.26465 fetch_fsmap learned FSMap version 32 2024-04-26T02:02:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.047+0000 7f81d4985740 1 -- 172.21.15.1:0/2816657523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55aee5c1cd10 con 0x55aee5ba6210 2024-04-26T02:02:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.047+0000 7f81b8c00640 1 -- 172.21.15.1:0/2816657523 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81bc00f9f0 con 0x55aee5ba6210 2024-04-26T02:02:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.047+0000 7f81b8c00640 1 -- 172.21.15.1:0/2816657523 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f81c4043070 con 0x55aee5ba6210 2024-04-26T02:02:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.047+0000 7f81d4985740 10 client.26465 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:02:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.047+0000 7f81d4985740 10 client.26465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:02:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.047+0000 7f81d4985740 20 client.26465 populate_metadata read hostname 'smithi001' 2024-04-26T02:02:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.047+0000 7f81d4985740 1 --2- 172.21.15.1:0/2816657523 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aee5c1d230 0x55aee5c1f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.047+0000 7f81d4985740 4 client.26465 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:02:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.047+0000 7f81d4985740 1 -- 172.21.15.1:0/2816657523 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aee5b06320 con 0x55aee5c1d230 2024-04-26T02:02:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.048+0000 7f81caa00640 1 --2- 172.21.15.1:0/2816657523 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aee5c1d230 0x55aee5c1f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.048+0000 7f81caa00640 1 --2- 172.21.15.1:0/2816657523 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aee5c1d230 0x55aee5c1f670 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a67afddb0f7cce9 server_cookie=da61cc4babd36277 in_seq=0 out_seq=0 2024-04-26T02:02:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.048+0000 7f81b8c00640 10 client.26465 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:02:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.064+0000 7f81b8c00640 1 -- 172.21.15.1:0/2816657523 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aee5b06320 con 0x55aee5c1d230 2024-04-26T02:02:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.064+0000 7f81b8c00640 10 client.26465 handle_command_reply: tid=0 2024-04-26T02:02:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.064+0000 7f81d4985740 1 client.26465 shutdown 2024-04-26T02:02:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.064+0000 7f81d4985740 1 -- 172.21.15.1:0/2816657523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5ba6210 msgr2=0x55aee5bac540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.064+0000 7f81d4985740 1 --2- 172.21.15.1:0/2816657523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5ba6210 0x55aee5bac540 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f81c4000f80 tx=0x7f81c402a8c0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.064+0000 7f81d4985740 1 -- 172.21.15.1:0/2816657523 shutdown_connections 2024-04-26T02:02:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.064+0000 7f81d4985740 1 --2- 172.21.15.1:0/2816657523 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55aee5c1d230 0x55aee5c1f670 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.064+0000 7f81d4985740 1 --2- 172.21.15.1:0/2816657523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55aee5ba6210 0x55aee5bac540 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.064+0000 7f81d4985740 1 --2- 172.21.15.1:0/2816657523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aee5b93080 0x55aee5bac000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.064+0000 7f81d4985740 1 --2- 172.21.15.1:0/2816657523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aee5b92760 0x55aee5babac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.064+0000 7f81b8c00640 0 client.26465 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:02:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.064+0000 7f81d4985740 1 -- 172.21.15.1:0/2816657523 >> 172.21.15.1:0/2816657523 conn(0x55aee5b8ed30 msgr2=0x55aee5b8de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.064+0000 7f81d4985740 1 -- 172.21.15.1:0/2816657523 shutdown_connections 2024-04-26T02:02:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.064+0000 7f81d4985740 1 -- 172.21.15.1:0/2816657523 wait complete. 2024-04-26T02:02:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.065+0000 7f81d4985740 20 client.26465 trim_cache size 0 max 16384 2024-04-26T02:02:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T02:02:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T02:02:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T02:02:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T02:02:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.065+0000 7f81d1a00640 1 -- 172.21.15.1:0/3875771259 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f81b00a3d10 msgr2=0x7f81b00a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.065+0000 7f81d1a00640 1 --2- 172.21.15.1:0/3875771259 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f81b00a3d10 0x7f81b00a61d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f81bc012640 tx=0x7f81bc005f90 comp rx=0 tx=0).stop 2024-04-26T02:02:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.065+0000 7f81d1a00640 1 -- 172.21.15.1:0/3875771259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81cc107270 msgr2=0x7f81cc1a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.065+0000 7f81d1a00640 1 --2- 172.21.15.1:0/3875771259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81cc107270 0x7f81cc1a0bc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f81b400e9a0 tx=0x7f81b400ee60 comp rx=0 tx=0).stop 2024-04-26T02:02:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.066+0000 7f81d1a00640 1 -- 172.21.15.1:0/3875771259 shutdown_connections 2024-04-26T02:02:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.066+0000 7f81d1a00640 1 --2- 172.21.15.1:0/3875771259 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f81b00a3d10 0x7f81b00a61d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.066+0000 7f81d1a00640 1 --2- 172.21.15.1:0/3875771259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81cc19a740 0x7f81cc19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.066+0000 7f81d1a00640 1 --2- 172.21.15.1:0/3875771259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81cc107270 0x7f81cc1a0bc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.066+0000 7f81d1a00640 1 --2- 172.21.15.1:0/3875771259 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81cc1021a0 0x7f81cc1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.066+0000 7f81d1a00640 1 -- 172.21.15.1:0/3875771259 >> 172.21.15.1:0/3875771259 conn(0x7f81cc0fda40 msgr2=0x7f81cc0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.066+0000 7f81d1a00640 1 -- 172.21.15.1:0/3875771259 shutdown_connections 2024-04-26T02:02:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:09.066+0000 7f81d1a00640 1 -- 172.21.15.1:0/3875771259 wait complete. 2024-04-26T02:02:09.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:09 smithi173 ceph-mon[32553]: Adjusting osd_memory_target on smithi001 to 177.5M 2024-04-26T02:02:09.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:09 smithi173 ceph-mon[32553]: Unable to set osd_memory_target on smithi001 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-26T02:02:09.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:09 smithi001 ceph-mon[27822]: Adjusting osd_memory_target on smithi001 to 177.5M 2024-04-26T02:02:09.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:09 smithi001 ceph-mon[27822]: Unable to set osd_memory_target on smithi001 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-26T02:02:09.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:09 smithi097 ceph-mon[32853]: Adjusting osd_memory_target on smithi001 to 177.5M 2024-04-26T02:02:09.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:09 smithi097 ceph-mon[32853]: Unable to set osd_memory_target on smithi001 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-26T02:02:10.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:10 smithi173 ceph-mon[32553]: pgmap v328: 97 pgs: 97 active+clean; 156 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 275 KiB/s wr, 90 op/s 2024-04-26T02:02:10.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:10 smithi001 ceph-mon[27822]: pgmap v328: 97 pgs: 97 active+clean; 156 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 275 KiB/s wr, 90 op/s 2024-04-26T02:02:10.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:10 smithi097 ceph-mon[32853]: pgmap v328: 97 pgs: 97 active+clean; 156 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 275 KiB/s wr, 90 op/s 2024-04-26T02:02:12.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:12 smithi173 ceph-mon[32553]: pgmap v329: 97 pgs: 97 active+clean; 156 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 936 KiB/s wr, 310 op/s 2024-04-26T02:02:12.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:12 smithi001 ceph-mon[27822]: pgmap v329: 97 pgs: 97 active+clean; 156 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 936 KiB/s wr, 310 op/s 2024-04-26T02:02:12.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:12 smithi097 ceph-mon[32853]: pgmap v329: 97 pgs: 97 active+clean; 156 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 936 KiB/s wr, 310 op/s 2024-04-26T02:02:12.804 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0x63ef9ee 2024-04-26T02:02:14.553 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:14 smithi097 ceph-mon[32853]: pgmap v330: 97 pgs: 97 active+clean; 156 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 936 KiB/s wr, 310 op/s 2024-04-26T02:02:14.553 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:14 smithi097 ceph-mon[32853]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T02:02:14.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:14 smithi173 ceph-mon[32553]: pgmap v330: 97 pgs: 97 active+clean; 156 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 936 KiB/s wr, 310 op/s 2024-04-26T02:02:14.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:14 smithi173 ceph-mon[32553]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T02:02:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:14 smithi001 ceph-mon[27822]: pgmap v330: 97 pgs: 97 active+clean; 156 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 936 KiB/s wr, 310 op/s 2024-04-26T02:02:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:14 smithi001 ceph-mon[27822]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T02:02:16.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:16 smithi173 ceph-mon[32553]: pgmap v332: 97 pgs: 97 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.3 MiB/s wr, 415 op/s 2024-04-26T02:02:16.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:16 smithi001 ceph-mon[27822]: pgmap v332: 97 pgs: 97 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.3 MiB/s wr, 415 op/s 2024-04-26T02:02:16.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:16 smithi097 ceph-mon[32853]: pgmap v332: 97 pgs: 97 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.3 MiB/s wr, 415 op/s 2024-04-26T02:02:18.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:18 smithi173 ceph-mon[32553]: pgmap v333: 97 pgs: 97 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.2 MiB/s wr, 403 op/s 2024-04-26T02:02:18.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:18 smithi001 ceph-mon[27822]: pgmap v333: 97 pgs: 97 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.2 MiB/s wr, 403 op/s 2024-04-26T02:02:18.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:18 smithi097 ceph-mon[32853]: pgmap v333: 97 pgs: 97 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.2 MiB/s wr, 403 op/s 2024-04-26T02:02:19.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:19 smithi173 ceph-mon[32553]: pgmap v334: 97 pgs: 97 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.2 MiB/s wr, 403 op/s 2024-04-26T02:02:19.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:19 smithi001 ceph-mon[27822]: pgmap v334: 97 pgs: 97 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.2 MiB/s wr, 403 op/s 2024-04-26T02:02:19.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:19 smithi097 ceph-mon[32853]: pgmap v334: 97 pgs: 97 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.2 MiB/s wr, 403 op/s 2024-04-26T02:02:22.237 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.238+0000 7f6898000640 1 -- 172.21.15.1:0/1376794707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6890102b60 msgr2=0x7f6890106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:22.237 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.238+0000 7f6898000640 1 --2- 172.21.15.1:0/1376794707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6890102b60 0x7f6890106ba0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f687c009900 tx=0x7f687c02a410 comp rx=0 tx=0).stop 2024-04-26T02:02:22.237 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.238+0000 7f6898000640 1 -- 172.21.15.1:0/1376794707 shutdown_connections 2024-04-26T02:02:22.237 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.238+0000 7f6898000640 1 --2- 172.21.15.1:0/1376794707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6890107280 0x7f689010ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.237 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.238+0000 7f6898000640 1 --2- 172.21.15.1:0/1376794707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6890102b60 0x7f6890106ba0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.238+0000 7f6898000640 1 --2- 172.21.15.1:0/1376794707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68901021b0 0x7f6890102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.238+0000 7f6898000640 1 -- 172.21.15.1:0/1376794707 >> 172.21.15.1:0/1376794707 conn(0x7f68900fda10 msgr2=0x7f68900ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.238+0000 7f6898000640 1 -- 172.21.15.1:0/1376794707 shutdown_connections 2024-04-26T02:02:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.238+0000 7f6898000640 1 -- 172.21.15.1:0/1376794707 wait complete. 2024-04-26T02:02:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.240+0000 7f6898000640 1 Processor -- start 2024-04-26T02:02:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.240+0000 7f6898000640 1 -- start start 2024-04-26T02:02:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.240+0000 7f6898000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68901021b0 0x7f689019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.240+0000 7f6898000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6890107280 0x7f689019afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.240+0000 7f6898000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f689019f380 0x7f689019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.240+0000 7f6898000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f68901121f0 con 0x7f689019f380 2024-04-26T02:02:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.240+0000 7f6898000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6890112070 con 0x7f68901021b0 2024-04-26T02:02:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.240+0000 7f6898000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6890112370 con 0x7f6890107280 2024-04-26T02:02:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.240+0000 7f6895600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68901021b0 0x7f689019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.241+0000 7f6894c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6890107280 0x7f689019afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.241+0000 7f6895600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68901021b0 0x7f689019aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:41768/0 (socket says 172.21.15.1:41768) 2024-04-26T02:02:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.241+0000 7f6895600640 1 -- 172.21.15.1:0/1289943743 learned_addr learned my addr 172.21.15.1:0/1289943743 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.241+0000 7f6896000640 1 --2- 172.21.15.1:0/1289943743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f689019f380 0x7f689019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.241+0000 7f6895600640 1 -- 172.21.15.1:0/1289943743 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6890107280 msgr2=0x7f689019afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.241+0000 7f6895600640 1 --2- 172.21.15.1:0/1289943743 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6890107280 0x7f689019afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.241+0000 7f6895600640 1 -- 172.21.15.1:0/1289943743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f689019f380 msgr2=0x7f689019f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.241+0000 7f6895600640 1 --2- 172.21.15.1:0/1289943743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f689019f380 0x7f689019f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.241+0000 7f6895600640 1 -- 172.21.15.1:0/1289943743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689019fe70 con 0x7f68901021b0 2024-04-26T02:02:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.241+0000 7f6896000640 1 --2- 172.21.15.1:0/1289943743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f689019f380 0x7f689019f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:02:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.241+0000 7f6894c00640 1 --2- 172.21.15.1:0/1289943743 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6890107280 0x7f689019afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.241+0000 7f6895600640 1 --2- 172.21.15.1:0/1289943743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68901021b0 0x7f689019aa90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f688400ea60 tx=0x7f688400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.242+0000 7f688e000640 1 -- 172.21.15.1:0/1289943743 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688400cde0 con 0x7f68901021b0 2024-04-26T02:02:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.242+0000 7f688e000640 1 -- 172.21.15.1:0/1289943743 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68840035f0 con 0x7f68901021b0 2024-04-26T02:02:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.242+0000 7f688e000640 1 -- 172.21.15.1:0/1289943743 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884010690 con 0x7f68901021b0 2024-04-26T02:02:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.242+0000 7f6898000640 1 -- 172.21.15.1:0/1289943743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68901a0160 con 0x7f68901021b0 2024-04-26T02:02:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.242+0000 7f6898000640 1 -- 172.21.15.1:0/1289943743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68901a7970 con 0x7f68901021b0 2024-04-26T02:02:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.245+0000 7f688e000640 1 -- 172.21.15.1:0/1289943743 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f6884004020 con 0x7f68901021b0 2024-04-26T02:02:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.246+0000 7f689af97740 1 Processor -- start 2024-04-26T02:02:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.246+0000 7f689af97740 1 -- start start 2024-04-26T02:02:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.246+0000 7f689af97740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b33820 0x55b758b33c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.246+0000 7f689af97740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b34140 0x55b758b3e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.246+0000 7f689af97740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b3ea00 0x55b758b40df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.247+0000 7f689af97740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b758a99900 con 0x55b758b34140 2024-04-26T02:02:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.247+0000 7f689af97740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b758a3fef0 con 0x55b758b33820 2024-04-26T02:02:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.247+0000 7f689af97740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b758a958c0 con 0x55b758b3ea00 2024-04-26T02:02:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.247+0000 7f688e000640 1 --2- 172.21.15.1:0/1289943743 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f68780a3d90 0x7f68780a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.247+0000 7f6895600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b34140 0x55b758b3e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.247+0000 7f6894c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b3ea00 0x55b758b40df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.247+0000 7f6896000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b33820 0x55b758b33c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.247+0000 7f6894c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b3ea00 0x55b758b40df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:56064/0 (socket says 172.21.15.1:56064) 2024-04-26T02:02:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.247+0000 7f6895600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b34140 0x55b758b3e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41390/0 (socket says 172.21.15.1:41390) 2024-04-26T02:02:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.247+0000 7f6894c00640 1 -- 172.21.15.1:0/2220404821 learned_addr learned my addr 172.21.15.1:0/2220404821 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.247+0000 7f688e000640 1 -- 172.21.15.1:0/1289943743 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8571+0+0 (secure 0 0 0) 0x7f68840d0b80 con 0x7f68901021b0 2024-04-26T02:02:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.247+0000 7f6896000640 1 --2- 172.21.15.1:0/1289943743 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f68780a3d90 0x7f68780a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.248+0000 7f6896000640 1 -- 172.21.15.1:0/2220404821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b3ea00 msgr2=0x55b758b40df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.248+0000 7f6896000640 1 --2- 172.21.15.1:0/2220404821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b3ea00 0x55b758b40df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.248+0000 7f6896000640 1 -- 172.21.15.1:0/2220404821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b34140 msgr2=0x55b758b3e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.248+0000 7f6896000640 1 --2- 172.21.15.1:0/2220404821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b34140 0x55b758b3e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.248+0000 7f6896000640 1 -- 172.21.15.1:0/2220404821 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7587ac210 con 0x55b758b33820 2024-04-26T02:02:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.248+0000 7f6896000640 1 --2- 172.21.15.1:0/2220404821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b33820 0x55b758b33c00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6888005fd0 tx=0x7f6888012a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b3fdd98aedf5e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.248+0000 7f6882600640 1 -- 172.21.15.1:0/2220404821 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688801b960 con 0x55b758b33820 2024-04-26T02:02:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.249+0000 7f6882600640 1 -- 172.21.15.1:0/2220404821 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f688800e860 con 0x55b758b33820 2024-04-26T02:02:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.249+0000 7f6882600640 1 -- 172.21.15.1:0/2220404821 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688801a370 con 0x55b758b33820 2024-04-26T02:02:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.249+0000 7f689af97740 1 -- 172.21.15.1:0/2220404821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b33820 msgr2=0x55b758b33c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.249+0000 7f6896000640 1 --2- 172.21.15.1:0/1289943743 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f68780a3d90 0x7f68780a6250 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6888022000 tx=0x7f688801b000 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.249+0000 7f689af97740 1 --2- 172.21.15.1:0/2220404821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b33820 0x55b758b33c00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6888005fd0 tx=0x7f6888012a60 comp rx=0 tx=0).stop 2024-04-26T02:02:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.250+0000 7f689af97740 1 -- 172.21.15.1:0/2220404821 shutdown_connections 2024-04-26T02:02:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.250+0000 7f689af97740 1 --2- 172.21.15.1:0/2220404821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b3ea00 0x55b758b40df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.250+0000 7f689af97740 1 --2- 172.21.15.1:0/2220404821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b34140 0x55b758b3e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.250+0000 7f689af97740 1 --2- 172.21.15.1:0/2220404821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b33820 0x55b758b33c00 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.250+0000 7f689af97740 1 -- 172.21.15.1:0/2220404821 >> 172.21.15.1:0/2220404821 conn(0x55b758b2fd30 msgr2=0x55b758b30110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.250+0000 7f689af97740 1 -- 172.21.15.1:0/2220404821 shutdown_connections 2024-04-26T02:02:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.250+0000 7f689af97740 1 -- 172.21.15.1:0/2220404821 wait complete. 2024-04-26T02:02:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.250+0000 7f689af97740 1 -- start start 2024-04-26T02:02:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.250+0000 7f689af97740 1 Processor -- start 2024-04-26T02:02:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.251+0000 7f689af97740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b33820 0x55b758b3b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.251+0000 7f689af97740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b34140 0x55b758b3bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.251+0000 7f689af97740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b3ea00 0x55b758b3c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.251+0000 7f689af97740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b758b41630 con 0x55b758b33820 2024-04-26T02:02:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.251+0000 7f689af97740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b758a53310 con 0x55b758b3ea00 2024-04-26T02:02:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.251+0000 7f689af97740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b758b417b0 con 0x55b758b34140 2024-04-26T02:02:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.251+0000 7f689af97740 10 client.0 _ll_register_callbacks cb 0x55b758a21cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.251+0000 7f6894c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b34140 0x55b758b3bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.251+0000 7f6895600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b33820 0x55b758b3b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.251+0000 7f6894c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b34140 0x55b758b3bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:56066/0 (socket says 172.21.15.1:56066) 2024-04-26T02:02:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.251+0000 7f6894c00640 1 -- 172.21.15.1:0/3573196843 learned_addr learned my addr 172.21.15.1:0/3573196843 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.251+0000 7f6896000640 1 --2- 172.21.15.1:0/3573196843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b3ea00 0x55b758b3c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.252+0000 7f6896000640 1 -- 172.21.15.1:0/3573196843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b34140 msgr2=0x55b758b3bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.252+0000 7f6896000640 1 --2- 172.21.15.1:0/3573196843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b34140 0x55b758b3bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.252+0000 7f6896000640 1 -- 172.21.15.1:0/3573196843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b33820 msgr2=0x55b758b3b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.252+0000 7f6896000640 1 --2- 172.21.15.1:0/3573196843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b33820 0x55b758b3b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.252+0000 7f6896000640 1 -- 172.21.15.1:0/3573196843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b758878620 con 0x55b758b3ea00 2024-04-26T02:02:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.252+0000 7f6894c00640 1 --2- 172.21.15.1:0/3573196843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b34140 0x55b758b3bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.252+0000 7f6895600640 1 --2- 172.21.15.1:0/3573196843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b33820 0x55b758b3b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.252+0000 7f6896000640 1 --2- 172.21.15.1:0/3573196843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b3ea00 0x55b758b3c470 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6888010fa0 tx=0x7f6888004cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f5db1b9e308e5c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.252+0000 7f6876a00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:02:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.252+0000 7f6876a00640 1 -- 172.21.15.1:0/3573196843 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888009050 con 0x55b758b3ea00 2024-04-26T02:02:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.252+0000 7f6876a00640 1 -- 172.21.15.1:0/3573196843 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8571+0+0 (secure 0 0 0) 0x7f6888009c20 con 0x55b758b3ea00 2024-04-26T02:02:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.252+0000 7f689af97740 1 -- 172.21.15.1:0/3573196843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f688800e860 con 0x55b758b3ea00 2024-04-26T02:02:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.253+0000 7f6876a00640 1 -- 172.21.15.1:0/3573196843 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f688800ec20 con 0x55b758b3ea00 2024-04-26T02:02:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.253+0000 7f689af97740 10 client.26477 fetch_fsmap learned FSMap version 32 2024-04-26T02:02:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.253+0000 7f689af97740 1 -- 172.21.15.1:0/3573196843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b758b4d8f0 con 0x55b758b3ea00 2024-04-26T02:02:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.253+0000 7f6876a00640 1 -- 172.21.15.1:0/3573196843 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f688800e860 con 0x55b758b3ea00 2024-04-26T02:02:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.253+0000 7f6876a00640 1 -- 172.21.15.1:0/3573196843 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f6888038070 con 0x55b758b3ea00 2024-04-26T02:02:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.253+0000 7f689af97740 10 client.26477 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:02:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.253+0000 7f689af97740 10 client.26477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:02:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.253+0000 7f689af97740 20 client.26477 populate_metadata read hostname 'smithi001' 2024-04-26T02:02:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.253+0000 7f689af97740 1 --2- 172.21.15.1:0/3573196843 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55b758b4ec40 0x55b758b51030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.253+0000 7f689af97740 4 client.26477 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:02:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.253+0000 7f689af97740 1 -- 172.21.15.1:0/3573196843 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b758aad8b0 con 0x55b758b4ec40 2024-04-26T02:02:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.254+0000 7f6895600640 1 --2- 172.21.15.1:0/3573196843 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55b758b4ec40 0x55b758b51030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.254+0000 7f6895600640 1 --2- 172.21.15.1:0/3573196843 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55b758b4ec40 0x55b758b51030 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79b2a2ee1704a7df server_cookie=a2916f2ee9f1a2e2 in_seq=0 out_seq=0 2024-04-26T02:02:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.254+0000 7f6876a00640 10 client.26477 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:02:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.255+0000 7f6876a00640 1 -- 172.21.15.1:0/3573196843 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b758aad8b0 con 0x55b758b4ec40 2024-04-26T02:02:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.255+0000 7f6876a00640 10 client.26477 handle_command_reply: tid=0 2024-04-26T02:02:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.255+0000 7f689af97740 1 client.26477 shutdown 2024-04-26T02:02:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.255+0000 7f689af97740 1 -- 172.21.15.1:0/3573196843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b3ea00 msgr2=0x55b758b3c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.255+0000 7f689af97740 1 --2- 172.21.15.1:0/3573196843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b3ea00 0x55b758b3c470 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6888010fa0 tx=0x7f6888004cb0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.255+0000 7f689af97740 1 -- 172.21.15.1:0/3573196843 shutdown_connections 2024-04-26T02:02:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.256+0000 7f689af97740 1 --2- 172.21.15.1:0/3573196843 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55b758b4ec40 0x55b758b51030 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.256+0000 7f689af97740 1 --2- 172.21.15.1:0/3573196843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b3ea00 0x55b758b3c470 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.256+0000 7f6876a00640 0 client.26477 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:02:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.256+0000 7f689af97740 1 --2- 172.21.15.1:0/3573196843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b34140 0x55b758b3bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.256+0000 7f689af97740 1 --2- 172.21.15.1:0/3573196843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b33820 0x55b758b3b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.256+0000 7f689af97740 1 -- 172.21.15.1:0/3573196843 >> 172.21.15.1:0/3573196843 conn(0x55b758b2fd30 msgr2=0x55b758b35ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.256+0000 7f689af97740 1 -- 172.21.15.1:0/3573196843 shutdown_connections 2024-04-26T02:02:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.256+0000 7f689af97740 1 -- 172.21.15.1:0/3573196843 wait complete. 2024-04-26T02:02:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.256+0000 7f689af97740 20 client.26477 trim_cache size 0 max 16384 2024-04-26T02:02:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.276+0000 7f689af97740 1 Processor -- start 2024-04-26T02:02:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.276+0000 7f689af97740 1 -- start start 2024-04-26T02:02:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.277+0000 7f689af97740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b33820 0x55b758b2f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.277+0000 7f689af97740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b34140 0x55b758b361e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.277+0000 7f689af97740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b3ea00 0x55b758b3c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.277+0000 7f689af97740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f68901b27a0 con 0x55b758b3ea00 2024-04-26T02:02:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.277+0000 7f689af97740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f68901b0070 con 0x55b758b34140 2024-04-26T02:02:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.277+0000 7f689af97740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f68901adee0 con 0x55b758b33820 2024-04-26T02:02:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.277+0000 7f6894c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b34140 0x55b758b361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.277+0000 7f6894c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b34140 0x55b758b361e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:41794/0 (socket says 172.21.15.1:41794) 2024-04-26T02:02:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.277+0000 7f6894c00640 1 -- 172.21.15.1:0/3193760714 learned_addr learned my addr 172.21.15.1:0/3193760714 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.277+0000 7f6895600640 1 --2- 172.21.15.1:0/3193760714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b33820 0x55b758b2f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.277+0000 7f6896000640 1 --2- 172.21.15.1:0/3193760714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b3ea00 0x55b758b3c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.277+0000 7f6894c00640 1 -- 172.21.15.1:0/3193760714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b33820 msgr2=0x55b758b2f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.277+0000 7f6894c00640 1 --2- 172.21.15.1:0/3193760714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b33820 0x55b758b2f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.277+0000 7f6894c00640 1 -- 172.21.15.1:0/3193760714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b3ea00 msgr2=0x55b758b3c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.277+0000 7f6894c00640 1 --2- 172.21.15.1:0/3193760714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b3ea00 0x55b758b3c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.278+0000 7f6894c00640 1 -- 172.21.15.1:0/3193760714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b758b513a0 con 0x55b758b34140 2024-04-26T02:02:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.278+0000 7f6895600640 1 --2- 172.21.15.1:0/3193760714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b33820 0x55b758b2f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.278+0000 7f6896000640 1 --2- 172.21.15.1:0/3193760714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b3ea00 0x55b758b3c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.278+0000 7f6894c00640 1 --2- 172.21.15.1:0/3193760714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b34140 0x55b758b361e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f689006b650 tx=0x7f687c004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f51f3c8857052687 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.278+0000 7f6881c00640 1 -- 172.21.15.1:0/3193760714 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c035040 con 0x55b758b34140 2024-04-26T02:02:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.278+0000 7f6881c00640 1 -- 172.21.15.1:0/3193760714 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f687c005620 con 0x55b758b34140 2024-04-26T02:02:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.278+0000 7f6881c00640 1 -- 172.21.15.1:0/3193760714 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c003850 con 0x55b758b34140 2024-04-26T02:02:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.279+0000 7f689af97740 1 -- 172.21.15.1:0/3193760714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b34140 msgr2=0x55b758b361e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.279+0000 7f689af97740 1 --2- 172.21.15.1:0/3193760714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b34140 0x55b758b361e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f689006b650 tx=0x7f687c004610 comp rx=0 tx=0).stop 2024-04-26T02:02:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.279+0000 7f689af97740 1 -- 172.21.15.1:0/3193760714 shutdown_connections 2024-04-26T02:02:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.279+0000 7f689af97740 1 --2- 172.21.15.1:0/3193760714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b3ea00 0x55b758b3c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.279+0000 7f689af97740 1 --2- 172.21.15.1:0/3193760714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b34140 0x55b758b361e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.279+0000 7f689af97740 1 --2- 172.21.15.1:0/3193760714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b33820 0x55b758b2f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.279+0000 7f689af97740 1 -- 172.21.15.1:0/3193760714 >> 172.21.15.1:0/3193760714 conn(0x55b758b2fd30 msgr2=0x55b758b3b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.279+0000 7f689af97740 1 -- 172.21.15.1:0/3193760714 shutdown_connections 2024-04-26T02:02:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.279+0000 7f689af97740 1 -- 172.21.15.1:0/3193760714 wait complete. 2024-04-26T02:02:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.280+0000 7f689af97740 1 -- start start 2024-04-26T02:02:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.280+0000 7f689af97740 1 Processor -- start 2024-04-26T02:02:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.280+0000 7f689af97740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b33820 0x55b758b43cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.280+0000 7f689af97740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b34140 0x55b758b44200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.280+0000 7f689af97740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b3ea00 0x55b758bad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.280+0000 7f689af97740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f68901ae530 con 0x55b758b34140 2024-04-26T02:02:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.280+0000 7f689af97740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f68901af9a0 con 0x55b758b3ea00 2024-04-26T02:02:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.280+0000 7f689af97740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f68901aec00 con 0x55b758b33820 2024-04-26T02:02:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.281+0000 7f689af97740 10 client.0 _ll_register_callbacks cb 0x7f6888012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.281+0000 7f6895600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b33820 0x55b758b43cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.281+0000 7f6896000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b3ea00 0x55b758bad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.281+0000 7f6894c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b34140 0x55b758b44200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.281+0000 7f6895600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b33820 0x55b758b43cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:56084/0 (socket says 172.21.15.1:56084) 2024-04-26T02:02:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.281+0000 7f6895600640 1 -- 172.21.15.1:0/105212531 learned_addr learned my addr 172.21.15.1:0/105212531 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.281+0000 7f6896000640 1 -- 172.21.15.1:0/105212531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b33820 msgr2=0x55b758b43cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.281+0000 7f6896000640 1 --2- 172.21.15.1:0/105212531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b33820 0x55b758b43cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.281+0000 7f6896000640 1 -- 172.21.15.1:0/105212531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b34140 msgr2=0x55b758b44200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.281+0000 7f6896000640 1 --2- 172.21.15.1:0/105212531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b34140 0x55b758b44200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.281+0000 7f6896000640 1 -- 172.21.15.1:0/105212531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b758bad700 con 0x55b758b3ea00 2024-04-26T02:02:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.281+0000 7f6895600640 1 --2- 172.21.15.1:0/105212531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b33820 0x55b758b43cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.281+0000 7f6896000640 1 --2- 172.21.15.1:0/105212531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b3ea00 0x55b758bad0c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6888012520 tx=0x7f6888010520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c84c4c78f1553d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.282+0000 7f6876a00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:02:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.282+0000 7f6876a00640 1 -- 172.21.15.1:0/105212531 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888009b20 con 0x55b758b3ea00 2024-04-26T02:02:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.283+0000 7f689af97740 1 -- 172.21.15.1:0/105212531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f687c005620 con 0x55b758b3ea00 2024-04-26T02:02:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.283+0000 7f6876a00640 1 -- 172.21.15.1:0/105212531 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8571+0+0 (secure 0 0 0) 0x7f688801b070 con 0x55b758b3ea00 2024-04-26T02:02:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.283+0000 7f6876a00640 1 -- 172.21.15.1:0/105212531 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f688801b270 con 0x55b758b3ea00 2024-04-26T02:02:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.283+0000 7f689af97740 10 client.26483 fetch_fsmap learned FSMap version 32 2024-04-26T02:02:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.283+0000 7f689af97740 1 -- 172.21.15.1:0/105212531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b758bb4ac0 con 0x55b758b3ea00 2024-04-26T02:02:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.283+0000 7f6876a00640 1 -- 172.21.15.1:0/105212531 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f687c005620 con 0x55b758b3ea00 2024-04-26T02:02:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.283+0000 7f6876a00640 1 -- 172.21.15.1:0/105212531 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f6888038070 con 0x55b758b3ea00 2024-04-26T02:02:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.283+0000 7f689af97740 10 client.26483 fetch_fsmap finished waiting for FSMap version 32 2024-04-26T02:02:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.283+0000 7f689af97740 10 client.26483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:02:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.284+0000 7f689af97740 20 client.26483 populate_metadata read hostname 'smithi001' 2024-04-26T02:02:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.284+0000 7f689af97740 1 --2- 172.21.15.1:0/105212531 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55b758bb5120 0x55b758bb7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:22.292 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.284+0000 7f689af97740 4 client.26483 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:02:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.284+0000 7f689af97740 1 -- 172.21.15.1:0/105212531 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b758aa7320 con 0x55b758bb5120 2024-04-26T02:02:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.284+0000 7f6895600640 1 --2- 172.21.15.1:0/105212531 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55b758bb5120 0x55b758bb7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.284+0000 7f6895600640 1 --2- 172.21.15.1:0/105212531 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55b758bb5120 0x55b758bb7530 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad9f87f6d2b371af server_cookie=ebed0aa1fef9bcf7 in_seq=0 out_seq=0 2024-04-26T02:02:22.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.284+0000 7f6876a00640 10 client.26483 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:02:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.302+0000 7f6876a00640 1 -- 172.21.15.1:0/105212531 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b758aa7320 con 0x55b758bb5120 2024-04-26T02:02:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.302+0000 7f6876a00640 10 client.26483 handle_command_reply: tid=0 2024-04-26T02:02:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.302+0000 7f689af97740 1 client.26483 shutdown 2024-04-26T02:02:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.303+0000 7f689af97740 1 -- 172.21.15.1:0/105212531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b3ea00 msgr2=0x55b758bad0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.303+0000 7f689af97740 1 --2- 172.21.15.1:0/105212531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b3ea00 0x55b758bad0c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6888012520 tx=0x7f6888010520 comp rx=0 tx=0).stop 2024-04-26T02:02:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.303+0000 7f689af97740 1 -- 172.21.15.1:0/105212531 shutdown_connections 2024-04-26T02:02:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.303+0000 7f689af97740 1 --2- 172.21.15.1:0/105212531 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55b758bb5120 0x55b758bb7530 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.303+0000 7f689af97740 1 --2- 172.21.15.1:0/105212531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b758b3ea00 0x55b758bad0c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.303+0000 7f689af97740 1 --2- 172.21.15.1:0/105212531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b758b34140 0x55b758b44200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.303+0000 7f6876a00640 0 client.26483 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:02:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.303+0000 7f689af97740 1 --2- 172.21.15.1:0/105212531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b758b33820 0x55b758b43cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.303+0000 7f689af97740 1 -- 172.21.15.1:0/105212531 >> 172.21.15.1:0/105212531 conn(0x55b758b2fd30 msgr2=0x55b758b3c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.303+0000 7f689af97740 1 -- 172.21.15.1:0/105212531 shutdown_connections 2024-04-26T02:02:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.303+0000 7f689af97740 1 -- 172.21.15.1:0/105212531 wait complete. 2024-04-26T02:02:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.303+0000 7f689af97740 20 client.26483 trim_cache size 0 max 16384 2024-04-26T02:02:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T02:02:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T02:02:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T02:02:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T02:02:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.304+0000 7f6898000640 1 -- 172.21.15.1:0/1289943743 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f68780a3d90 msgr2=0x7f68780a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.304+0000 7f6898000640 1 --2- 172.21.15.1:0/1289943743 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f68780a3d90 0x7f68780a6250 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6888022000 tx=0x7f688801b000 comp rx=0 tx=0).stop 2024-04-26T02:02:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.304+0000 7f6898000640 1 -- 172.21.15.1:0/1289943743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68901021b0 msgr2=0x7f689019aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.304+0000 7f6898000640 1 --2- 172.21.15.1:0/1289943743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68901021b0 0x7f689019aa90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f688400ea60 tx=0x7f688400ef20 comp rx=0 tx=0).stop 2024-04-26T02:02:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.304+0000 7f6898000640 1 -- 172.21.15.1:0/1289943743 shutdown_connections 2024-04-26T02:02:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.304+0000 7f6898000640 1 --2- 172.21.15.1:0/1289943743 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f68780a3d90 0x7f68780a6250 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.304+0000 7f6898000640 1 --2- 172.21.15.1:0/1289943743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f689019f380 0x7f689019f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.304+0000 7f6898000640 1 --2- 172.21.15.1:0/1289943743 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6890107280 0x7f689019afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.304+0000 7f6898000640 1 --2- 172.21.15.1:0/1289943743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68901021b0 0x7f689019aa90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.304+0000 7f6898000640 1 -- 172.21.15.1:0/1289943743 >> 172.21.15.1:0/1289943743 conn(0x7f68900fda10 msgr2=0x7f68900ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.305+0000 7f6898000640 1 -- 172.21.15.1:0/1289943743 shutdown_connections 2024-04-26T02:02:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:22.305+0000 7f6898000640 1 -- 172.21.15.1:0/1289943743 wait complete. 2024-04-26T02:02:22.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:22 smithi173 ceph-mon[32553]: pgmap v335: 97 pgs: 97 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.1 MiB/s wr, 371 op/s 2024-04-26T02:02:22.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:22 smithi001 ceph-mon[27822]: pgmap v335: 97 pgs: 97 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.1 MiB/s wr, 371 op/s 2024-04-26T02:02:22.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:22 smithi097 ceph-mon[32853]: pgmap v335: 97 pgs: 97 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.1 MiB/s wr, 371 op/s 2024-04-26T02:02:23.027 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 02:02:22 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[67264]: ignoring --setuser ceph since I am not root 2024-04-26T02:02:23.027 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 02:02:22 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[67264]: ignoring --setgroup ceph since I am not root 2024-04-26T02:02:23.027 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 02:02:22 smithi173 ceph-mds[67268]: -- 172.21.15.173:0/3345287839 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56108d5f7a40 con 0x56108d5cb000 2024-04-26T02:02:23.027 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 02:02:22 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[67264]: starting mds.f at 2024-04-26T02:02:23.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:23 smithi173 ceph-mon[32553]: daemon mds.f restarted 2024-04-26T02:02:23.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:23 smithi001 ceph-mon[27822]: daemon mds.f restarted 2024-04-26T02:02:23.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:23 smithi097 ceph-mon[32853]: daemon mds.f restarted 2024-04-26T02:02:24.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:24 smithi173 ceph-mon[32553]: pgmap v336: 97 pgs: 97 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.1 MiB/s wr, 371 op/s 2024-04-26T02:02:24.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:24 smithi173 ceph-mon[32553]: osdmap e105: 12 total, 12 up, 12 in 2024-04-26T02:02:24.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:24 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T02:02:24.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:24 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.173:6834/334804458,v1:172.21.15.173:6835/334804458] up:boot 2024-04-26T02:02:24.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:24 smithi173 ceph-mon[32553]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T02:02:24.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:24 smithi173 ceph-mon[32553]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T02:02:24.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:24 smithi001 ceph-mon[27822]: pgmap v336: 97 pgs: 97 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.1 MiB/s wr, 371 op/s 2024-04-26T02:02:24.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:24 smithi001 ceph-mon[27822]: osdmap e105: 12 total, 12 up, 12 in 2024-04-26T02:02:24.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:24 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T02:02:24.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:24 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.173:6834/334804458,v1:172.21.15.173:6835/334804458] up:boot 2024-04-26T02:02:24.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:24 smithi001 ceph-mon[27822]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T02:02:24.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:24 smithi001 ceph-mon[27822]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T02:02:24.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:24 smithi097 ceph-mon[32853]: pgmap v336: 97 pgs: 97 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.1 MiB/s wr, 371 op/s 2024-04-26T02:02:24.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:24 smithi097 ceph-mon[32853]: osdmap e105: 12 total, 12 up, 12 in 2024-04-26T02:02:24.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:24 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T02:02:24.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:24 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.173:6834/334804458,v1:172.21.15.173:6835/334804458] up:boot 2024-04-26T02:02:24.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:24 smithi097 ceph-mon[32853]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T02:02:24.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:24 smithi097 ceph-mon[32853]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T02:02:26.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:26 smithi097 ceph-mon[32853]: pgmap v338: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.1 MiB/s wr, 366 op/s 2024-04-26T02:02:26.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:26 smithi173 ceph-mon[32553]: pgmap v338: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.1 MiB/s wr, 366 op/s 2024-04-26T02:02:26.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:26 smithi001 ceph-mon[27822]: pgmap v338: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.1 MiB/s wr, 366 op/s 2024-04-26T02:02:28.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:28 smithi173 ceph-mon[32553]: pgmap v339: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.1 MiB/s wr, 366 op/s 2024-04-26T02:02:28.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:28 smithi001 ceph-mon[27822]: pgmap v339: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.1 MiB/s wr, 366 op/s 2024-04-26T02:02:28.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:28 smithi097 ceph-mon[32853]: pgmap v339: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.1 MiB/s wr, 366 op/s 2024-04-26T02:02:30.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:30 smithi173 ceph-mon[32553]: pgmap v340: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.1 MiB/s wr, 366 op/s 2024-04-26T02:02:30.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:30 smithi001 ceph-mon[27822]: pgmap v340: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.1 MiB/s wr, 366 op/s 2024-04-26T02:02:30.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:30 smithi097 ceph-mon[32853]: pgmap v340: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.1 MiB/s wr, 366 op/s 2024-04-26T02:02:32.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:32 smithi173 ceph-mon[32553]: pgmap v341: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.1 MiB/s wr, 374 op/s 2024-04-26T02:02:32.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:32 smithi001 ceph-mon[27822]: pgmap v341: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.1 MiB/s wr, 374 op/s 2024-04-26T02:02:32.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:32 smithi097 ceph-mon[32853]: pgmap v341: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.1 MiB/s wr, 374 op/s 2024-04-26T02:02:34.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:34 smithi173 ceph-mon[32553]: pgmap v342: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.1 MiB/s wr, 374 op/s 2024-04-26T02:02:34.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:34 smithi001 ceph-mon[27822]: pgmap v342: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.1 MiB/s wr, 374 op/s 2024-04-26T02:02:34.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:34 smithi097 ceph-mon[32853]: pgmap v342: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.1 MiB/s wr, 374 op/s 2024-04-26T02:02:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.457+0000 7fcdf0a00640 1 -- 172.21.15.1:0/165226501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdec109490 msgr2=0x7fcdec111060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.457+0000 7fcdf0a00640 1 --2- 172.21.15.1:0/165226501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdec109490 0x7fcdec111060 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009ca0 tx=0x7fcddc02a8b0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.457+0000 7fcdf0a00640 1 -- 172.21.15.1:0/165226501 shutdown_connections 2024-04-26T02:02:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.457+0000 7fcdf0a00640 1 --2- 172.21.15.1:0/165226501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdec109490 0x7fcdec111060 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.457+0000 7fcdf0a00640 1 --2- 172.21.15.1:0/165226501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdec104d70 0x7fcdec108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.457+0000 7fcdf0a00640 1 --2- 172.21.15.1:0/165226501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdec1043c0 0x7fcdec1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.457+0000 7fcdf0a00640 1 -- 172.21.15.1:0/165226501 >> 172.21.15.1:0/165226501 conn(0x7fcdec0ffc20 msgr2=0x7fcdec102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.457+0000 7fcdf0a00640 1 -- 172.21.15.1:0/165226501 shutdown_connections 2024-04-26T02:02:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.457+0000 7fcdf0a00640 1 -- 172.21.15.1:0/165226501 wait complete. 2024-04-26T02:02:35.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.459+0000 7fcdf0a00640 1 Processor -- start 2024-04-26T02:02:35.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.459+0000 7fcdf0a00640 1 -- start start 2024-04-26T02:02:35.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.459+0000 7fcdf0a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdec1043c0 0x7fcdec1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:35.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.459+0000 7fcdf0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdec109490 0x7fcdec1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:35.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.459+0000 7fcdf0a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdec19a790 0x7fcdec19ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:35.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.459+0000 7fcdf0a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec114450 con 0x7fcdec19a790 2024-04-26T02:02:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.459+0000 7fcdf0a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec1142d0 con 0x7fcdec109490 2024-04-26T02:02:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.459+0000 7fcdf0a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec1145d0 con 0x7fcdec1043c0 2024-04-26T02:02:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.460+0000 7fcde9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdec109490 0x7fcdec1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.460+0000 7fcde9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdec109490 0x7fcdec1a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:33802/0 (socket says 172.21.15.1:33802) 2024-04-26T02:02:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.460+0000 7fcde9400640 1 -- 172.21.15.1:0/4176401449 learned_addr learned my addr 172.21.15.1:0/4176401449 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.460+0000 7fcde9e00640 1 --2- 172.21.15.1:0/4176401449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdec1043c0 0x7fcdec1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.461+0000 7fcde9400640 1 -- 172.21.15.1:0/4176401449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdec1043c0 msgr2=0x7fcdec1a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.461+0000 7fcde9400640 1 --2- 172.21.15.1:0/4176401449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdec1043c0 0x7fcdec1a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.461+0000 7fcde9400640 1 -- 172.21.15.1:0/4176401449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdec19a790 msgr2=0x7fcdec19ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T02:02:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.461+0000 7fcdea800640 1 --2- 172.21.15.1:0/4176401449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdec19a790 0x7fcdec19ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.461+0000 7fcde9400640 1 --2- 172.21.15.1:0/4176401449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdec19a790 0x7fcdec19ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.461+0000 7fcde9400640 1 -- 172.21.15.1:0/4176401449 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec19b390 con 0x7fcdec109490 2024-04-26T02:02:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.461+0000 7fcdea800640 1 --2- 172.21.15.1:0/4176401449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdec19a790 0x7fcdec19ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:02:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.462+0000 7fcde9400640 1 --2- 172.21.15.1:0/4176401449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdec109490 0x7fcdec1a0bc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400e9f0 tx=0x7fcdd400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.462+0000 7fcde2a00640 1 -- 172.21.15.1:0/4176401449 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd400ccc0 con 0x7fcdec109490 2024-04-26T02:02:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.462+0000 7fcde2a00640 1 -- 172.21.15.1:0/4176401449 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdd40054d0 con 0x7fcdec109490 2024-04-26T02:02:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.462+0000 7fcdf0a00640 1 -- 172.21.15.1:0/4176401449 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdec1a7420 con 0x7fcdec109490 2024-04-26T02:02:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.462+0000 7fcde2a00640 1 -- 172.21.15.1:0/4176401449 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4010430 con 0x7fcdec109490 2024-04-26T02:02:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.462+0000 7fcdf0a00640 1 -- 172.21.15.1:0/4176401449 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdec1a7820 con 0x7fcdec109490 2024-04-26T02:02:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.466+0000 7fcdf383a740 1 Processor -- start 2024-04-26T02:02:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.467+0000 7fcde2a00640 1 -- 172.21.15.1:0/4176401449 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fcdd4005020 con 0x7fcdec109490 2024-04-26T02:02:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.467+0000 7fcdf383a740 1 -- start start 2024-04-26T02:02:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.467+0000 7fcdf383a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a390e1e820 0x55a390e1ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.467+0000 7fcdf383a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e1f140 0x55a390e294c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.467+0000 7fcdf383a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e29a00 0x55a390e2bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.467+0000 7fcdf383a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a390d84900 con 0x55a390e1e820 2024-04-26T02:02:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.467+0000 7fcdf383a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a390d2aef0 con 0x55a390e29a00 2024-04-26T02:02:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.467+0000 7fcdf383a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a390d808c0 con 0x55a390e1f140 2024-04-26T02:02:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.467+0000 7fcde9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e1f140 0x55a390e294c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.467+0000 7fcde9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e29a00 0x55a390e2bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.467+0000 7fcde9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e1f140 0x55a390e294c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:49904/0 (socket says 172.21.15.1:49904) 2024-04-26T02:02:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.467+0000 7fcde9e00640 1 -- 172.21.15.1:0/925096069 learned_addr learned my addr 172.21.15.1:0/925096069 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.467+0000 7fcdea800640 1 --2- 172.21.15.1:0/925096069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a390e1e820 0x55a390e1ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.468+0000 7fcde9e00640 1 -- 172.21.15.1:0/925096069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e29a00 msgr2=0x55a390e2bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.468+0000 7fcde9e00640 1 --2- 172.21.15.1:0/925096069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e29a00 0x55a390e2bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.468+0000 7fcde9e00640 1 -- 172.21.15.1:0/925096069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a390e1e820 msgr2=0x55a390e1ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.468+0000 7fcde9e00640 1 --2- 172.21.15.1:0/925096069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a390e1e820 0x55a390e1ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.468+0000 7fcde9e00640 1 -- 172.21.15.1:0/925096069 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a390a97210 con 0x55a390e1f140 2024-04-26T02:02:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.468+0000 7fcde9400640 1 --2- 172.21.15.1:0/925096069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e29a00 0x55a390e2bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.468+0000 7fcde9e00640 1 --2- 172.21.15.1:0/925096069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e1f140 0x55a390e294c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009cf0 tx=0x7fcde4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=460a84bfa0f9ac4c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.469+0000 7fcddb000640 1 -- 172.21.15.1:0/925096069 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4004230 con 0x55a390e1f140 2024-04-26T02:02:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.469+0000 7fcddb000640 1 -- 172.21.15.1:0/925096069 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcde40043d0 con 0x55a390e1f140 2024-04-26T02:02:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.469+0000 7fcddb000640 1 -- 172.21.15.1:0/925096069 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4019880 con 0x55a390e1f140 2024-04-26T02:02:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.469+0000 7fcde2a00640 1 --2- 172.21.15.1:0/4176401449 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fcdd00a3d90 0x7fcdd00a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.470+0000 7fcde2a00640 1 -- 172.21.15.1:0/4176401449 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8527+0+0 (secure 0 0 0) 0x7fcdd40d0a70 con 0x7fcdec109490 2024-04-26T02:02:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.470+0000 7fcdf383a740 1 -- 172.21.15.1:0/925096069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e1f140 msgr2=0x55a390e294c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.470+0000 7fcdf383a740 1 --2- 172.21.15.1:0/925096069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e1f140 0x55a390e294c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009cf0 tx=0x7fcde4011810 comp rx=0 tx=0).stop 2024-04-26T02:02:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.470+0000 7fcdea800640 1 --2- 172.21.15.1:0/4176401449 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fcdd00a3d90 0x7fcdd00a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.470+0000 7fcdf383a740 1 -- 172.21.15.1:0/925096069 shutdown_connections 2024-04-26T02:02:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.470+0000 7fcdf383a740 1 --2- 172.21.15.1:0/925096069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e29a00 0x55a390e2bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.470+0000 7fcdf383a740 1 --2- 172.21.15.1:0/925096069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e1f140 0x55a390e294c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.470+0000 7fcdf383a740 1 --2- 172.21.15.1:0/925096069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a390e1e820 0x55a390e1ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.470+0000 7fcdf383a740 1 -- 172.21.15.1:0/925096069 >> 172.21.15.1:0/925096069 conn(0x55a390e1ad30 msgr2=0x55a390e1b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.470+0000 7fcdf383a740 1 -- 172.21.15.1:0/925096069 shutdown_connections 2024-04-26T02:02:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.470+0000 7fcdf383a740 1 -- 172.21.15.1:0/925096069 wait complete. 2024-04-26T02:02:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.471+0000 7fcdf383a740 1 -- start start 2024-04-26T02:02:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.471+0000 7fcdea800640 1 --2- 172.21.15.1:0/4176401449 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fcdd00a3d90 0x7fcdd00a6250 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcddc006010 tx=0x7fcddc004a60 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.471+0000 7fcdf383a740 1 Processor -- start 2024-04-26T02:02:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.472+0000 7fcdf383a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e1e820 0x55a390e268c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.472+0000 7fcdf383a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a390e1f140 0x55a390e26e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.472+0000 7fcdf383a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e29a00 0x55a390e27340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.472+0000 7fcdf383a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a390e2c630 con 0x55a390e1f140 2024-04-26T02:02:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.472+0000 7fcdf383a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a390d3e310 con 0x55a390e1e820 2024-04-26T02:02:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.472+0000 7fcdf383a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a390e2c7b0 con 0x55a390e29a00 2024-04-26T02:02:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.472+0000 7fcdf383a740 10 client.0 _ll_register_callbacks cb 0x55a390d0ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.472+0000 7fcde9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e1e820 0x55a390e268c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.472+0000 7fcde9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e1e820 0x55a390e268c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:33820/0 (socket says 172.21.15.1:33820) 2024-04-26T02:02:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.472+0000 7fcde9e00640 1 -- 172.21.15.1:0/2494405057 learned_addr learned my addr 172.21.15.1:0/2494405057 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.472+0000 7fcde9400640 1 --2- 172.21.15.1:0/2494405057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a390e1f140 0x55a390e26e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.472+0000 7fcdea800640 1 --2- 172.21.15.1:0/2494405057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e29a00 0x55a390e27340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.472+0000 7fcde9e00640 1 -- 172.21.15.1:0/2494405057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e29a00 msgr2=0x55a390e27340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.472+0000 7fcde9e00640 1 --2- 172.21.15.1:0/2494405057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e29a00 0x55a390e27340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.472+0000 7fcde9e00640 1 -- 172.21.15.1:0/2494405057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a390e1f140 msgr2=0x55a390e26e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.472+0000 7fcde9e00640 1 --2- 172.21.15.1:0/2494405057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a390e1f140 0x55a390e26e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.473+0000 7fcde9e00640 1 -- 172.21.15.1:0/2494405057 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a390b63620 con 0x55a390e1e820 2024-04-26T02:02:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.473+0000 7fcde9e00640 1 --2- 172.21.15.1:0/2494405057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e1e820 0x55a390e268c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcde4003820 tx=0x7fcde40096f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3238b6b989e9472 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.473+0000 7fcdcf400640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:02:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.473+0000 7fcdcf400640 1 -- 172.21.15.1:0/2494405057 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4027070 con 0x55a390e1e820 2024-04-26T02:02:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.473+0000 7fcdcf400640 1 -- 172.21.15.1:0/2494405057 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8527+0+0 (secure 0 0 0) 0x7fcde4005240 con 0x55a390e1e820 2024-04-26T02:02:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.473+0000 7fcdf383a740 1 -- 172.21.15.1:0/2494405057 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcde40043d0 con 0x55a390e1e820 2024-04-26T02:02:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.473+0000 7fcdcf400640 1 -- 172.21.15.1:0/2494405057 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcde4033050 con 0x55a390e1e820 2024-04-26T02:02:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.473+0000 7fcdf383a740 10 client.26498 fetch_fsmap learned FSMap version 34 2024-04-26T02:02:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.474+0000 7fcdf383a740 1 -- 172.21.15.1:0/2494405057 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a390e38830 con 0x55a390e1e820 2024-04-26T02:02:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.474+0000 7fcdcf400640 1 -- 172.21.15.1:0/2494405057 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcde40043d0 con 0x55a390e1e820 2024-04-26T02:02:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.474+0000 7fcdcf400640 1 -- 172.21.15.1:0/2494405057 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7fcde4004b70 con 0x55a390e1e820 2024-04-26T02:02:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.474+0000 7fcdf383a740 10 client.26498 fetch_fsmap finished waiting for FSMap version 34 2024-04-26T02:02:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.474+0000 7fcdf383a740 10 client.26498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:02:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.474+0000 7fcdf383a740 20 client.26498 populate_metadata read hostname 'smithi001' 2024-04-26T02:02:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.474+0000 7fcdf383a740 1 --2- 172.21.15.1:0/2494405057 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a390e39c40 0x55a390e3c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.474+0000 7fcdf383a740 4 client.26498 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:02:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.474+0000 7fcdf383a740 1 -- 172.21.15.1:0/2494405057 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a390d988b0 con 0x55a390e39c40 2024-04-26T02:02:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.475+0000 7fcde9400640 1 --2- 172.21.15.1:0/2494405057 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a390e39c40 0x55a390e3c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.475+0000 7fcde9400640 1 --2- 172.21.15.1:0/2494405057 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a390e39c40 0x55a390e3c030 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fabb648f5ea4534e server_cookie=5f17657375aed606 in_seq=0 out_seq=0 2024-04-26T02:02:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.475+0000 7fcdcf400640 10 client.26498 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:02:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.476+0000 7fcdcf400640 1 -- 172.21.15.1:0/2494405057 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a390d988b0 con 0x55a390e39c40 2024-04-26T02:02:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.476+0000 7fcdcf400640 10 client.26498 handle_command_reply: tid=0 2024-04-26T02:02:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.476+0000 7fcdf383a740 1 client.26498 shutdown 2024-04-26T02:02:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.476+0000 7fcdf383a740 1 -- 172.21.15.1:0/2494405057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e1e820 msgr2=0x55a390e268c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.476+0000 7fcdf383a740 1 --2- 172.21.15.1:0/2494405057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e1e820 0x55a390e268c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcde4003820 tx=0x7fcde40096f0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.476+0000 7fcdf383a740 1 -- 172.21.15.1:0/2494405057 shutdown_connections 2024-04-26T02:02:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.476+0000 7fcdf383a740 1 --2- 172.21.15.1:0/2494405057 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a390e39c40 0x55a390e3c030 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.476+0000 7fcdf383a740 1 --2- 172.21.15.1:0/2494405057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e29a00 0x55a390e27340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.476+0000 7fcdf383a740 1 --2- 172.21.15.1:0/2494405057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a390e1f140 0x55a390e26e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.476+0000 7fcdf383a740 1 --2- 172.21.15.1:0/2494405057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e1e820 0x55a390e268c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.476+0000 7fcdcf400640 0 client.26498 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:02:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.476+0000 7fcdf383a740 1 -- 172.21.15.1:0/2494405057 >> 172.21.15.1:0/2494405057 conn(0x55a390e1ad30 msgr2=0x55a390e20ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.477+0000 7fcdf383a740 1 -- 172.21.15.1:0/2494405057 shutdown_connections 2024-04-26T02:02:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.477+0000 7fcdf383a740 1 -- 172.21.15.1:0/2494405057 wait complete. 2024-04-26T02:02:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.477+0000 7fcdf383a740 20 client.26498 trim_cache size 0 max 16384 2024-04-26T02:02:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.494+0000 7fcdf383a740 1 Processor -- start 2024-04-26T02:02:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.494+0000 7fcdf383a740 1 -- start start 2024-04-26T02:02:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.495+0000 7fcdf383a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e1e820 0x55a390e35520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.495+0000 7fcdf383a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e1f140 0x55a390e41860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.495+0000 7fcdf383a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a390e29a00 0x55a390e41da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.495+0000 7fcdf383a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8011090 con 0x55a390e29a00 2024-04-26T02:02:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.495+0000 7fcdf383a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc800e960 con 0x55a390e1f140 2024-04-26T02:02:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.495+0000 7fcdf383a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc800c7b0 con 0x55a390e1e820 2024-04-26T02:02:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.495+0000 7fcdea800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a390e29a00 0x55a390e41da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.495+0000 7fcde9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e1f140 0x55a390e41860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.495+0000 7fcde9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e1e820 0x55a390e35520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.495+0000 7fcde9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e1f140 0x55a390e41860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:33832/0 (socket says 172.21.15.1:33832) 2024-04-26T02:02:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.495+0000 7fcde9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e1e820 0x55a390e35520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:49918/0 (socket says 172.21.15.1:49918) 2024-04-26T02:02:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.495+0000 7fcde9400640 1 -- 172.21.15.1:0/1052627704 learned_addr learned my addr 172.21.15.1:0/1052627704 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.495+0000 7fcdea800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a390e29a00 0x55a390e41da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47226/0 (socket says 172.21.15.1:47226) 2024-04-26T02:02:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.495+0000 7fcde9400640 1 -- 172.21.15.1:0/1052627704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e1e820 msgr2=0x55a390e35520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.495+0000 7fcde9400640 1 --2- 172.21.15.1:0/1052627704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e1e820 0x55a390e35520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.495+0000 7fcde9400640 1 -- 172.21.15.1:0/1052627704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a390e29a00 msgr2=0x55a390e41da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.495+0000 7fcde9400640 1 --2- 172.21.15.1:0/1052627704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a390e29a00 0x55a390e41da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.496+0000 7fcde9400640 1 -- 172.21.15.1:0/1052627704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a390e1a860 con 0x55a390e1f140 2024-04-26T02:02:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.496+0000 7fcde9400640 1 --2- 172.21.15.1:0/1052627704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e1f140 0x55a390e41860 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400c6a0 tx=0x7fcdd40bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9471e1bbaf70d828 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.496+0000 7fcdda600640 1 -- 172.21.15.1:0/1052627704 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd40d0e60 con 0x55a390e1f140 2024-04-26T02:02:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.496+0000 7fcdda600640 1 -- 172.21.15.1:0/1052627704 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdd4005310 con 0x55a390e1f140 2024-04-26T02:02:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.496+0000 7fcdda600640 1 -- 172.21.15.1:0/1052627704 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd40a5620 con 0x55a390e1f140 2024-04-26T02:02:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.497+0000 7fcdf383a740 1 -- 172.21.15.1:0/1052627704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e1f140 msgr2=0x55a390e41860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.497+0000 7fcdf383a740 1 --2- 172.21.15.1:0/1052627704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e1f140 0x55a390e41860 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400c6a0 tx=0x7fcdd40bd040 comp rx=0 tx=0).stop 2024-04-26T02:02:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.497+0000 7fcdf383a740 1 -- 172.21.15.1:0/1052627704 shutdown_connections 2024-04-26T02:02:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.497+0000 7fcdf383a740 1 --2- 172.21.15.1:0/1052627704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a390e29a00 0x55a390e41da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.497+0000 7fcdf383a740 1 --2- 172.21.15.1:0/1052627704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e1f140 0x55a390e41860 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.497+0000 7fcdf383a740 1 --2- 172.21.15.1:0/1052627704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e1e820 0x55a390e35520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.497+0000 7fcdf383a740 1 -- 172.21.15.1:0/1052627704 >> 172.21.15.1:0/1052627704 conn(0x55a390e1ad30 msgr2=0x55a390e1b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:35.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.497+0000 7fcdf383a740 1 -- 172.21.15.1:0/1052627704 shutdown_connections 2024-04-26T02:02:35.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.497+0000 7fcdf383a740 1 -- 172.21.15.1:0/1052627704 wait complete. 2024-04-26T02:02:35.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.498+0000 7fcdf383a740 1 -- start start 2024-04-26T02:02:35.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.498+0000 7fcdf383a740 1 Processor -- start 2024-04-26T02:02:35.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.498+0000 7fcdf383a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e1e820 0x55a390e98450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:35.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.498+0000 7fcdf383a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a390e1f140 0x55a390e98990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:35.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.498+0000 7fcdf383a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e29a00 0x55a390e2ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:35.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.498+0000 7fcdf383a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc800ce20 con 0x55a390e1f140 2024-04-26T02:02:35.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.498+0000 7fcdf383a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc800e290 con 0x55a390e29a00 2024-04-26T02:02:35.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.498+0000 7fcdf383a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc800d4f0 con 0x55a390e1e820 2024-04-26T02:02:35.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.498+0000 7fcdf383a740 10 client.0 _ll_register_callbacks cb 0x7fcde4009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.499+0000 7fcde9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e1e820 0x55a390e98450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.499+0000 7fcde9400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a390e1f140 0x55a390e98990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.499+0000 7fcde9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e1e820 0x55a390e98450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:49934/0 (socket says 172.21.15.1:49934) 2024-04-26T02:02:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.499+0000 7fcde9e00640 1 -- 172.21.15.1:0/2197208596 learned_addr learned my addr 172.21.15.1:0/2197208596 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.499+0000 7fcdea800640 1 --2- 172.21.15.1:0/2197208596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e29a00 0x55a390e2ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.499+0000 7fcde9e00640 1 -- 172.21.15.1:0/2197208596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e29a00 msgr2=0x55a390e2ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.499+0000 7fcde9e00640 1 --2- 172.21.15.1:0/2197208596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e29a00 0x55a390e2ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.499+0000 7fcde9e00640 1 -- 172.21.15.1:0/2197208596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a390e1f140 msgr2=0x55a390e98990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.499+0000 7fcde9e00640 1 --2- 172.21.15.1:0/2197208596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a390e1f140 0x55a390e98990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.499+0000 7fcde9e00640 1 -- 172.21.15.1:0/2197208596 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a390e2f500 con 0x55a390e1e820 2024-04-26T02:02:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.499+0000 7fcde9e00640 1 --2- 172.21.15.1:0/2197208596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e1e820 0x55a390e98450 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009c40 tx=0x7fcde4005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba8c298e25cabb3c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.499+0000 7fcdcf400640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T02:02:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.499+0000 7fcdcf400640 1 -- 172.21.15.1:0/2197208596 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4027040 con 0x55a390e1e820 2024-04-26T02:02:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.500+0000 7fcdf383a740 1 -- 172.21.15.1:0/2197208596 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a390d92320 con 0x55a390e1e820 2024-04-26T02:02:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.500+0000 7fcdcf400640 1 -- 172.21.15.1:0/2197208596 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8527+0+0 (secure 0 0 0) 0x7fcde4004fc0 con 0x55a390e1e820 2024-04-26T02:02:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.500+0000 7fcdcf400640 1 -- 172.21.15.1:0/2197208596 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcde4045050 con 0x55a390e1e820 2024-04-26T02:02:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.500+0000 7fcdf383a740 10 client.26515 fetch_fsmap learned FSMap version 34 2024-04-26T02:02:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.500+0000 7fcdf383a740 1 -- 172.21.15.1:0/2197208596 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a390e9fc00 con 0x55a390e1e820 2024-04-26T02:02:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.500+0000 7fcdcf400640 1 -- 172.21.15.1:0/2197208596 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a390d92320 con 0x55a390e1e820 2024-04-26T02:02:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.501+0000 7fcdcf400640 1 -- 172.21.15.1:0/2197208596 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7fcde4005200 con 0x55a390e1e820 2024-04-26T02:02:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.501+0000 7fcdf383a740 10 client.26515 fetch_fsmap finished waiting for FSMap version 34 2024-04-26T02:02:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.501+0000 7fcdf383a740 10 client.26515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:02:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.501+0000 7fcdf383a740 20 client.26515 populate_metadata read hostname 'smithi001' 2024-04-26T02:02:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.501+0000 7fcdf383a740 1 --2- 172.21.15.1:0/2197208596 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a390ea0340 0x55a390ea2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.501+0000 7fcdf383a740 4 client.26515 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:02:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.501+0000 7fcdf383a740 1 -- 172.21.15.1:0/2197208596 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a390d8c190 con 0x55a390ea0340 2024-04-26T02:02:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.502+0000 7fcde9400640 1 --2- 172.21.15.1:0/2197208596 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a390ea0340 0x55a390ea2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:35.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.502+0000 7fcde9400640 1 --2- 172.21.15.1:0/2197208596 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a390ea0340 0x55a390ea2760 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0334d07ef42c25d server_cookie=31d8728cab2fb88a in_seq=0 out_seq=0 2024-04-26T02:02:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.502+0000 7fcdcf400640 10 client.26515 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:02:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.517+0000 7fcdcf400640 1 -- 172.21.15.1:0/2197208596 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a390d8c190 con 0x55a390ea0340 2024-04-26T02:02:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.517+0000 7fcdcf400640 10 client.26515 handle_command_reply: tid=0 2024-04-26T02:02:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.517+0000 7fcdf383a740 1 client.26515 shutdown 2024-04-26T02:02:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.518+0000 7fcdf383a740 1 -- 172.21.15.1:0/2197208596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e1e820 msgr2=0x55a390e98450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.518+0000 7fcdf383a740 1 --2- 172.21.15.1:0/2197208596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e1e820 0x55a390e98450 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009c40 tx=0x7fcde4005dc0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.518+0000 7fcdf383a740 1 -- 172.21.15.1:0/2197208596 shutdown_connections 2024-04-26T02:02:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.518+0000 7fcdf383a740 1 --2- 172.21.15.1:0/2197208596 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55a390ea0340 0x55a390ea2760 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.518+0000 7fcdf383a740 1 --2- 172.21.15.1:0/2197208596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a390e29a00 0x55a390e2ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.518+0000 7fcdf383a740 1 --2- 172.21.15.1:0/2197208596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a390e1f140 0x55a390e98990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.518+0000 7fcdf383a740 1 --2- 172.21.15.1:0/2197208596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a390e1e820 0x55a390e98450 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.518+0000 7fcdcf400640 0 client.26515 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:02:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.518+0000 7fcdf383a740 1 -- 172.21.15.1:0/2197208596 >> 172.21.15.1:0/2197208596 conn(0x55a390e1ad30 msgr2=0x55a390e26860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.518+0000 7fcdf383a740 1 -- 172.21.15.1:0/2197208596 shutdown_connections 2024-04-26T02:02:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.518+0000 7fcdf383a740 1 -- 172.21.15.1:0/2197208596 wait complete. 2024-04-26T02:02:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.518+0000 7fcdf383a740 20 client.26515 trim_cache size 0 max 16384 2024-04-26T02:02:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T02:02:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T02:02:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T02:02:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T02:02:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.519+0000 7fcdf0a00640 1 -- 172.21.15.1:0/4176401449 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fcdd00a3d90 msgr2=0x7fcdd00a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.519+0000 7fcdf0a00640 1 --2- 172.21.15.1:0/4176401449 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fcdd00a3d90 0x7fcdd00a6250 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcddc006010 tx=0x7fcddc004a60 comp rx=0 tx=0).stop 2024-04-26T02:02:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.519+0000 7fcdf0a00640 1 -- 172.21.15.1:0/4176401449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdec109490 msgr2=0x7fcdec1a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.519+0000 7fcdf0a00640 1 --2- 172.21.15.1:0/4176401449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdec109490 0x7fcdec1a0bc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400e9f0 tx=0x7fcdd400eeb0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.519+0000 7fcdf0a00640 1 -- 172.21.15.1:0/4176401449 shutdown_connections 2024-04-26T02:02:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.519+0000 7fcdf0a00640 1 --2- 172.21.15.1:0/4176401449 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fcdd00a3d90 0x7fcdd00a6250 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.519+0000 7fcdf0a00640 1 --2- 172.21.15.1:0/4176401449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcdec19a790 0x7fcdec19ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.519+0000 7fcdf0a00640 1 --2- 172.21.15.1:0/4176401449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdec109490 0x7fcdec1a0bc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.519+0000 7fcdf0a00640 1 --2- 172.21.15.1:0/4176401449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdec1043c0 0x7fcdec1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.519+0000 7fcdf0a00640 1 -- 172.21.15.1:0/4176401449 >> 172.21.15.1:0/4176401449 conn(0x7fcdec0ffc20 msgr2=0x7fcdec101fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.520+0000 7fcdf0a00640 1 -- 172.21.15.1:0/4176401449 shutdown_connections 2024-04-26T02:02:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:35.520+0000 7fcdf0a00640 1 -- 172.21.15.1:0/4176401449 wait complete. 2024-04-26T02:02:36.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:36 smithi173 ceph-mon[32553]: pgmap v343: 97 pgs: 97 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 462 op/s 2024-04-26T02:02:36.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:36 smithi001 ceph-mon[27822]: pgmap v343: 97 pgs: 97 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 462 op/s 2024-04-26T02:02:36.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:36 smithi097 ceph-mon[32853]: pgmap v343: 97 pgs: 97 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 462 op/s 2024-04-26T02:02:38.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:38 smithi173 ceph-mon[32553]: pgmap v344: 97 pgs: 97 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1009 KiB/s wr, 316 op/s 2024-04-26T02:02:38.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:38 smithi001 ceph-mon[27822]: pgmap v344: 97 pgs: 97 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1009 KiB/s wr, 316 op/s 2024-04-26T02:02:38.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:38 smithi097 ceph-mon[32853]: pgmap v344: 97 pgs: 97 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1009 KiB/s wr, 316 op/s 2024-04-26T02:02:40.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:40 smithi001 ceph-mon[27822]: pgmap v345: 97 pgs: 97 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1009 KiB/s wr, 316 op/s 2024-04-26T02:02:40.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:40 smithi097 ceph-mon[32853]: pgmap v345: 97 pgs: 97 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1009 KiB/s wr, 316 op/s 2024-04-26T02:02:40.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:40 smithi173 ceph-mon[32553]: pgmap v345: 97 pgs: 97 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1009 KiB/s wr, 316 op/s 2024-04-26T02:02:42.369 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:42 smithi173 ceph-mon[32553]: pgmap v346: 97 pgs: 97 active+clean; 155 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 464 op/s 2024-04-26T02:02:42.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:42 smithi001 ceph-mon[27822]: pgmap v346: 97 pgs: 97 active+clean; 155 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 464 op/s 2024-04-26T02:02:42.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:42 smithi097 ceph-mon[32853]: pgmap v346: 97 pgs: 97 active+clean; 155 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 464 op/s 2024-04-26T02:02:44.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:44 smithi001 ceph-mon[27822]: pgmap v347: 97 pgs: 97 active+clean; 155 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 914 KiB/s wr, 295 op/s 2024-04-26T02:02:44.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:44 smithi097 ceph-mon[32853]: pgmap v347: 97 pgs: 97 active+clean; 155 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 914 KiB/s wr, 295 op/s 2024-04-26T02:02:44.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:44 smithi173 ceph-mon[32553]: pgmap v347: 97 pgs: 97 active+clean; 155 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 914 KiB/s wr, 295 op/s 2024-04-26T02:02:46.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:46 smithi001 ceph-mon[27822]: pgmap v348: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 435 op/s 2024-04-26T02:02:46.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:46 smithi097 ceph-mon[32853]: pgmap v348: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 435 op/s 2024-04-26T02:02:46.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:46 smithi173 ceph-mon[32553]: pgmap v348: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 435 op/s 2024-04-26T02:02:48.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:48 smithi001 ceph-mon[27822]: pgmap v349: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 886 KiB/s wr, 287 op/s 2024-04-26T02:02:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:48 smithi097 ceph-mon[32853]: pgmap v349: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 886 KiB/s wr, 287 op/s 2024-04-26T02:02:48.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:48 smithi173 ceph-mon[32553]: pgmap v349: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 886 KiB/s wr, 287 op/s 2024-04-26T02:02:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.675+0000 7fe79be00640 1 -- 172.21.15.1:0/2761580699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe79c0742f0 msgr2=0x7fe79c0746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.675+0000 7fe79be00640 1 --2- 172.21.15.1:0/2761580699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe79c0742f0 0x7fe79c0746d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009900 tx=0x7fe78c02a480 comp rx=0 tx=0).stop 2024-04-26T02:02:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.676+0000 7fe79be00640 1 -- 172.21.15.1:0/2761580699 shutdown_connections 2024-04-26T02:02:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.676+0000 7fe79be00640 1 --2- 172.21.15.1:0/2761580699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe79c112d80 0x7fe79c115170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.676+0000 7fe79be00640 1 --2- 172.21.15.1:0/2761580699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe79c074c10 0x7fe79c112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.676+0000 7fe79be00640 1 --2- 172.21.15.1:0/2761580699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe79c0742f0 0x7fe79c0746d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.676+0000 7fe79be00640 1 -- 172.21.15.1:0/2761580699 >> 172.21.15.1:0/2761580699 conn(0x7fe79c0fd860 msgr2=0x7fe79c0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.676+0000 7fe79be00640 1 -- 172.21.15.1:0/2761580699 shutdown_connections 2024-04-26T02:02:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.676+0000 7fe79be00640 1 -- 172.21.15.1:0/2761580699 wait complete. 2024-04-26T02:02:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.677+0000 7fe79be00640 1 Processor -- start 2024-04-26T02:02:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.677+0000 7fe79be00640 1 -- start start 2024-04-26T02:02:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.678+0000 7fe79be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe79c0742f0 0x7fe79c1a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.678+0000 7fe79be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe79c074c10 0x7fe79c1a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.678+0000 7fe79be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe79c112d80 0x7fe79c19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.678+0000 7fe79be00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c118600 con 0x7fe79c074c10 2024-04-26T02:02:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.678+0000 7fe79be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c118480 con 0x7fe79c112d80 2024-04-26T02:02:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.678+0000 7fe79be00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c118780 con 0x7fe79c0742f0 2024-04-26T02:02:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.678+0000 7fe79aa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe79c0742f0 0x7fe79c1a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.678+0000 7fe79aa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe79c0742f0 0x7fe79c1a47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:35134/0 (socket says 172.21.15.1:35134) 2024-04-26T02:02:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.678+0000 7fe79aa00640 1 -- 172.21.15.1:0/2699398893 learned_addr learned my addr 172.21.15.1:0/2699398893 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.678+0000 7fe793e00640 1 --2- 172.21.15.1:0/2699398893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe79c074c10 0x7fe79c1a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.678+0000 7fe79b400640 1 --2- 172.21.15.1:0/2699398893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe79c112d80 0x7fe79c19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.679+0000 7fe79aa00640 1 -- 172.21.15.1:0/2699398893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe79c112d80 msgr2=0x7fe79c19e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.679+0000 7fe79aa00640 1 --2- 172.21.15.1:0/2699398893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe79c112d80 0x7fe79c19e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.679+0000 7fe79aa00640 1 -- 172.21.15.1:0/2699398893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe79c074c10 msgr2=0x7fe79c1a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.679+0000 7fe79aa00640 1 --2- 172.21.15.1:0/2699398893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe79c074c10 0x7fe79c1a4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.679+0000 7fe79aa00640 1 -- 172.21.15.1:0/2699398893 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c19f100 con 0x7fe79c0742f0 2024-04-26T02:02:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.679+0000 7fe793e00640 1 --2- 172.21.15.1:0/2699398893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe79c074c10 0x7fe79c1a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:02:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.679+0000 7fe79aa00640 1 --2- 172.21.15.1:0/2699398893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe79c0742f0 0x7fe79c1a47d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe78c005940 tx=0x7fe78c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.679+0000 7fe7a0a00640 1 -- 172.21.15.1:0/2699398893 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c03f030 con 0x7fe79c0742f0 2024-04-26T02:02:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.679+0000 7fe79be00640 1 -- 172.21.15.1:0/2699398893 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79c19f390 con 0x7fe79c0742f0 2024-04-26T02:02:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.679+0000 7fe79be00640 1 -- 172.21.15.1:0/2699398893 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79c1ab690 con 0x7fe79c0742f0 2024-04-26T02:02:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.679+0000 7fe7a0a00640 1 -- 172.21.15.1:0/2699398893 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe78c003aa0 con 0x7fe79c0742f0 2024-04-26T02:02:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.679+0000 7fe7a0a00640 1 -- 172.21.15.1:0/2699398893 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c049060 con 0x7fe79c0742f0 2024-04-26T02:02:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.683+0000 7fe7a4c31740 1 Processor -- start 2024-04-26T02:02:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.683+0000 7fe7a4c31740 1 -- start start 2024-04-26T02:02:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.684+0000 7fe7a4c31740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50679b0 0x555bb5067d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.684+0000 7fe7a4c31740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bb50682d0 0x555bb5072650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.684+0000 7fe7a4c31740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bb5072b90 0x555bb5074f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.684+0000 7fe7a4c31740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555bb4fcd900 con 0x555bb50682d0 2024-04-26T02:02:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.684+0000 7fe7a4c31740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555bb4f73ef0 con 0x555bb50679b0 2024-04-26T02:02:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.684+0000 7fe7a4c31740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555bb4fc98c0 con 0x555bb5072b90 2024-04-26T02:02:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.684+0000 7fe79b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50679b0 0x555bb5067d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.684+0000 7fe79b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50679b0 0x555bb5067d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:49782/0 (socket says 172.21.15.1:49782) 2024-04-26T02:02:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.684+0000 7fe79b400640 1 -- 172.21.15.1:0/4031859129 learned_addr learned my addr 172.21.15.1:0/4031859129 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.684+0000 7fe7a0a00640 1 -- 172.21.15.1:0/2699398893 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fe78c042030 con 0x7fe79c0742f0 2024-04-26T02:02:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.684+0000 7fe793e00640 1 --2- 172.21.15.1:0/4031859129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bb5072b90 0x555bb5074f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.685+0000 7fe79b400640 1 -- 172.21.15.1:0/4031859129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bb5072b90 msgr2=0x555bb5074f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.685+0000 7fe79b400640 1 --2- 172.21.15.1:0/4031859129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bb5072b90 0x555bb5074f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.685+0000 7fe79b400640 1 -- 172.21.15.1:0/4031859129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bb50682d0 msgr2=0x555bb5072650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.685+0000 7fe79aa00640 1 --2- 172.21.15.1:0/4031859129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bb50682d0 0x555bb5072650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.685+0000 7fe79b400640 1 --2- 172.21.15.1:0/4031859129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bb50682d0 0x555bb5072650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.685+0000 7fe79b400640 1 -- 172.21.15.1:0/4031859129 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bb4ce0210 con 0x555bb50679b0 2024-04-26T02:02:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.685+0000 7fe79aa00640 1 --2- 172.21.15.1:0/4031859129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bb50682d0 0x555bb5072650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:02:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.685+0000 7fe79b400640 1 --2- 172.21.15.1:0/4031859129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50679b0 0x555bb5067d90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe794009ef0 tx=0x7fe79400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4582ee74180cb99 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.685+0000 7fe78be00640 1 -- 172.21.15.1:0/4031859129 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794015070 con 0x555bb50679b0 2024-04-26T02:02:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.685+0000 7fe78be00640 1 -- 172.21.15.1:0/4031859129 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7940118f0 con 0x555bb50679b0 2024-04-26T02:02:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.685+0000 7fe78be00640 1 -- 172.21.15.1:0/4031859129 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7940108f0 con 0x555bb50679b0 2024-04-26T02:02:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.686+0000 7fe7a0a00640 1 --2- 172.21.15.1:0/2699398893 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe7800a3d80 0x7fe7800a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.686+0000 7fe7a0a00640 1 -- 172.21.15.1:0/2699398893 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8527+0+0 (secure 0 0 0) 0x7fe78c0f0b90 con 0x7fe79c0742f0 2024-04-26T02:02:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.686+0000 7fe7a4c31740 1 -- 172.21.15.1:0/4031859129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50679b0 msgr2=0x555bb5067d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.686+0000 7fe7a4c31740 1 --2- 172.21.15.1:0/4031859129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50679b0 0x555bb5067d90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe794009ef0 tx=0x7fe79400f730 comp rx=0 tx=0).stop 2024-04-26T02:02:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.686+0000 7fe79aa00640 1 --2- 172.21.15.1:0/2699398893 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe7800a3d80 0x7fe7800a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.686+0000 7fe7a4c31740 1 -- 172.21.15.1:0/4031859129 shutdown_connections 2024-04-26T02:02:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.686+0000 7fe7a4c31740 1 --2- 172.21.15.1:0/4031859129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bb5072b90 0x555bb5074f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.686+0000 7fe7a4c31740 1 --2- 172.21.15.1:0/4031859129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bb50682d0 0x555bb5072650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.686+0000 7fe7a4c31740 1 --2- 172.21.15.1:0/4031859129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50679b0 0x555bb5067d90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.686+0000 7fe7a4c31740 1 -- 172.21.15.1:0/4031859129 >> 172.21.15.1:0/4031859129 conn(0x555bb5063ec0 msgr2=0x555bb50642a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.687+0000 7fe7a4c31740 1 -- 172.21.15.1:0/4031859129 shutdown_connections 2024-04-26T02:02:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.687+0000 7fe7a4c31740 1 -- 172.21.15.1:0/4031859129 wait complete. 2024-04-26T02:02:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.687+0000 7fe79aa00640 1 --2- 172.21.15.1:0/2699398893 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe7800a3d80 0x7fe7800a6240 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x555bb5068f50 tx=0x7fe78c02a480 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.687+0000 7fe7a4c31740 1 -- start start 2024-04-26T02:02:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.688+0000 7fe7a4c31740 1 Processor -- start 2024-04-26T02:02:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.688+0000 7fe7a4c31740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bb50679b0 0x555bb506fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.688+0000 7fe7a4c31740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50682d0 0x555bb5070070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.688+0000 7fe7a4c31740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bb5072b90 0x555bb50705b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.688+0000 7fe7a4c31740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555bb50757c0 con 0x555bb5072b90 2024-04-26T02:02:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.688+0000 7fe7a4c31740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555bb4f87310 con 0x555bb50682d0 2024-04-26T02:02:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.688+0000 7fe7a4c31740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555bb5075940 con 0x555bb50679b0 2024-04-26T02:02:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.688+0000 7fe7a4c31740 10 client.0 _ll_register_callbacks cb 0x555bb4f55cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.688+0000 7fe793e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50682d0 0x555bb5070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.688+0000 7fe793e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50682d0 0x555bb5070070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:49792/0 (socket says 172.21.15.1:49792) 2024-04-26T02:02:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.688+0000 7fe793e00640 1 -- 172.21.15.1:0/4083166343 learned_addr learned my addr 172.21.15.1:0/4083166343 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.688+0000 7fe79b400640 1 --2- 172.21.15.1:0/4083166343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bb50679b0 0x555bb506fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.689+0000 7fe79b400640 1 --2- 172.21.15.1:0/4083166343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bb5072b90 0x555bb50705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.689+0000 7fe793e00640 1 -- 172.21.15.1:0/4083166343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bb50679b0 msgr2=0x555bb506fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.689+0000 7fe793e00640 1 --2- 172.21.15.1:0/4083166343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bb50679b0 0x555bb506fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.689+0000 7fe793e00640 1 -- 172.21.15.1:0/4083166343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bb5072b90 msgr2=0x555bb50705b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.689+0000 7fe793e00640 1 --2- 172.21.15.1:0/4083166343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bb5072b90 0x555bb50705b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.689+0000 7fe793e00640 1 -- 172.21.15.1:0/4083166343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bb4dac620 con 0x555bb50682d0 2024-04-26T02:02:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.689+0000 7fe793e00640 1 --2- 172.21.15.1:0/4083166343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50682d0 0x555bb5070070 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe784012700 tx=0x7fe784012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3983792ee704d2e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.689+0000 7fe788c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:02:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.689+0000 7fe788c00640 1 -- 172.21.15.1:0/4083166343 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784004320 con 0x555bb50682d0 2024-04-26T02:02:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.689+0000 7fe788c00640 1 -- 172.21.15.1:0/4083166343 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8527+0+0 (secure 0 0 0) 0x7fe7840119a0 con 0x555bb50682d0 2024-04-26T02:02:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.689+0000 7fe7a4c31740 1 -- 172.21.15.1:0/4083166343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7940118f0 con 0x555bb50682d0 2024-04-26T02:02:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.690+0000 7fe788c00640 1 -- 172.21.15.1:0/4083166343 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe7840044c0 con 0x555bb50682d0 2024-04-26T02:02:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.690+0000 7fe7a4c31740 10 client.26516 fetch_fsmap learned FSMap version 34 2024-04-26T02:02:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.690+0000 7fe7a4c31740 1 -- 172.21.15.1:0/4083166343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555bb5081a50 con 0x555bb50682d0 2024-04-26T02:02:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.690+0000 7fe788c00640 1 -- 172.21.15.1:0/4083166343 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7940118f0 con 0x555bb50682d0 2024-04-26T02:02:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.690+0000 7fe788c00640 1 -- 172.21.15.1:0/4083166343 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7fe784025480 con 0x555bb50682d0 2024-04-26T02:02:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.690+0000 7fe7a4c31740 10 client.26516 fetch_fsmap finished waiting for FSMap version 34 2024-04-26T02:02:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.690+0000 7fe7a4c31740 10 client.26516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:02:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.691+0000 7fe7a4c31740 20 client.26516 populate_metadata read hostname 'smithi001' 2024-04-26T02:02:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.691+0000 7fe7a4c31740 1 --2- 172.21.15.1:0/4083166343 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x555bb5082f60 0x555bb5085370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.691+0000 7fe7a4c31740 4 client.26516 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:02:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.691+0000 7fe7a4c31740 1 -- 172.21.15.1:0/4083166343 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bb4fe18b0 con 0x555bb5082f60 2024-04-26T02:02:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.691+0000 7fe79b400640 1 --2- 172.21.15.1:0/4083166343 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x555bb5082f60 0x555bb5085370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.691+0000 7fe79b400640 1 --2- 172.21.15.1:0/4083166343 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x555bb5082f60 0x555bb5085370 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e36c6c2238ddc8 server_cookie=4640889cd490e165 in_seq=0 out_seq=0 2024-04-26T02:02:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.691+0000 7fe788c00640 10 client.26516 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:02:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.692+0000 7fe788c00640 1 -- 172.21.15.1:0/4083166343 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555bb4fe18b0 con 0x555bb5082f60 2024-04-26T02:02:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.692+0000 7fe788c00640 10 client.26516 handle_command_reply: tid=0 2024-04-26T02:02:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.692+0000 7fe7a4c31740 1 client.26516 shutdown 2024-04-26T02:02:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.692+0000 7fe7a4c31740 1 -- 172.21.15.1:0/4083166343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50682d0 msgr2=0x555bb5070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.692+0000 7fe7a4c31740 1 --2- 172.21.15.1:0/4083166343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50682d0 0x555bb5070070 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe784012700 tx=0x7fe784012bc0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.692+0000 7fe7a4c31740 1 -- 172.21.15.1:0/4083166343 shutdown_connections 2024-04-26T02:02:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.692+0000 7fe7a4c31740 1 --2- 172.21.15.1:0/4083166343 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x555bb5082f60 0x555bb5085370 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.692+0000 7fe7a4c31740 1 --2- 172.21.15.1:0/4083166343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bb5072b90 0x555bb50705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.692+0000 7fe7a4c31740 1 --2- 172.21.15.1:0/4083166343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50682d0 0x555bb5070070 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.692+0000 7fe7a4c31740 1 --2- 172.21.15.1:0/4083166343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bb50679b0 0x555bb506fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.692+0000 7fe788c00640 0 client.26516 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:02:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.692+0000 7fe7a4c31740 1 -- 172.21.15.1:0/4083166343 >> 172.21.15.1:0/4083166343 conn(0x555bb5063ec0 msgr2=0x555bb5069c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.693+0000 7fe7a4c31740 1 -- 172.21.15.1:0/4083166343 shutdown_connections 2024-04-26T02:02:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.693+0000 7fe7a4c31740 1 -- 172.21.15.1:0/4083166343 wait complete. 2024-04-26T02:02:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.693+0000 7fe7a4c31740 20 client.26516 trim_cache size 0 max 16384 2024-04-26T02:02:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.710+0000 7fe7a4c31740 1 Processor -- start 2024-04-26T02:02:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.710+0000 7fe7a4c31740 1 -- start start 2024-04-26T02:02:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.711+0000 7fe7a4c31740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bb50679b0 0x555bb507e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.711+0000 7fe7a4c31740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50682d0 0x555bb508ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.711+0000 7fe7a4c31740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bb5072b90 0x555bb508b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.711+0000 7fe7a4c31740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1b68d0 con 0x555bb5072b90 2024-04-26T02:02:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.711+0000 7fe7a4c31740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1b41a0 con 0x555bb50682d0 2024-04-26T02:02:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.711+0000 7fe7a4c31740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1b1ff0 con 0x555bb50679b0 2024-04-26T02:02:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.711+0000 7fe79b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bb50679b0 0x555bb507e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.711+0000 7fe793e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50682d0 0x555bb508ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.711+0000 7fe793e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50682d0 0x555bb508ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:49794/0 (socket says 172.21.15.1:49794) 2024-04-26T02:02:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.711+0000 7fe79b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bb50679b0 0x555bb507e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:35148/0 (socket says 172.21.15.1:35148) 2024-04-26T02:02:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.711+0000 7fe793e00640 1 -- 172.21.15.1:0/4008862381 learned_addr learned my addr 172.21.15.1:0/4008862381 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.711+0000 7fe79b400640 1 --2- 172.21.15.1:0/4008862381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bb5072b90 0x555bb508b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.712+0000 7fe793e00640 1 -- 172.21.15.1:0/4008862381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bb50679b0 msgr2=0x555bb507e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.712+0000 7fe793e00640 1 --2- 172.21.15.1:0/4008862381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bb50679b0 0x555bb507e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.712+0000 7fe793e00640 1 -- 172.21.15.1:0/4008862381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bb5072b90 msgr2=0x555bb508b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.712+0000 7fe793e00640 1 --2- 172.21.15.1:0/4008862381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bb5072b90 0x555bb508b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.712+0000 7fe793e00640 1 -- 172.21.15.1:0/4008862381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bb5068c60 con 0x555bb50682d0 2024-04-26T02:02:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.712+0000 7fe793e00640 1 --2- 172.21.15.1:0/4008862381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50682d0 0x555bb508ac40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe784004020 tx=0x7fe784011890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a16479cb118119 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.712+0000 7fe78b400640 1 -- 172.21.15.1:0/4008862381 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78402b070 con 0x555bb50682d0 2024-04-26T02:02:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.712+0000 7fe78b400640 1 -- 172.21.15.1:0/4008862381 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe784009680 con 0x555bb50682d0 2024-04-26T02:02:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.712+0000 7fe78b400640 1 -- 172.21.15.1:0/4008862381 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7840052a0 con 0x555bb50682d0 2024-04-26T02:02:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.713+0000 7fe7a4c31740 1 -- 172.21.15.1:0/4008862381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50682d0 msgr2=0x555bb508ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.713+0000 7fe7a4c31740 1 --2- 172.21.15.1:0/4008862381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50682d0 0x555bb508ac40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe784004020 tx=0x7fe784011890 comp rx=0 tx=0).stop 2024-04-26T02:02:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.713+0000 7fe7a4c31740 1 -- 172.21.15.1:0/4008862381 shutdown_connections 2024-04-26T02:02:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.713+0000 7fe7a4c31740 1 --2- 172.21.15.1:0/4008862381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bb5072b90 0x555bb508b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.713+0000 7fe7a4c31740 1 --2- 172.21.15.1:0/4008862381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50682d0 0x555bb508ac40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.713+0000 7fe7a4c31740 1 --2- 172.21.15.1:0/4008862381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bb50679b0 0x555bb507e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.713+0000 7fe7a4c31740 1 -- 172.21.15.1:0/4008862381 >> 172.21.15.1:0/4008862381 conn(0x555bb5063ec0 msgr2=0x555bb5069c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.713+0000 7fe7a4c31740 1 -- 172.21.15.1:0/4008862381 shutdown_connections 2024-04-26T02:02:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.713+0000 7fe7a4c31740 1 -- 172.21.15.1:0/4008862381 wait complete. 2024-04-26T02:02:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.714+0000 7fe7a4c31740 1 -- start start 2024-04-26T02:02:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.714+0000 7fe7a4c31740 1 Processor -- start 2024-04-26T02:02:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.714+0000 7fe7a4c31740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bb50679b0 0x555bb507b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.715+0000 7fe7a4c31740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50682d0 0x555bb50784d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.715+0000 7fe7a4c31740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bb5072b90 0x555bb5078a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.715+0000 7fe7a4c31740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1b2660 con 0x555bb5072b90 2024-04-26T02:02:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.715+0000 7fe7a4c31740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1b3ad0 con 0x555bb50682d0 2024-04-26T02:02:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.715+0000 7fe7a4c31740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1b2d30 con 0x555bb50679b0 2024-04-26T02:02:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.715+0000 7fe7a4c31740 10 client.0 _ll_register_callbacks cb 0x7fe784009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.715+0000 7fe793e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50682d0 0x555bb50784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.715+0000 7fe793e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50682d0 0x555bb50784d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:49802/0 (socket says 172.21.15.1:49802) 2024-04-26T02:02:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.715+0000 7fe793e00640 1 -- 172.21.15.1:0/1496738600 learned_addr learned my addr 172.21.15.1:0/1496738600 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:02:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.715+0000 7fe79b400640 1 --2- 172.21.15.1:0/1496738600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bb50679b0 0x555bb507b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.715+0000 7fe79b400640 1 --2- 172.21.15.1:0/1496738600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bb5072b90 0x555bb5078a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.715+0000 7fe793e00640 1 -- 172.21.15.1:0/1496738600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bb50679b0 msgr2=0x555bb507b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.715+0000 7fe793e00640 1 --2- 172.21.15.1:0/1496738600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bb50679b0 0x555bb507b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.715+0000 7fe793e00640 1 -- 172.21.15.1:0/1496738600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bb5072b90 msgr2=0x555bb5078a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.715+0000 7fe793e00640 1 --2- 172.21.15.1:0/1496738600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bb5072b90 0x555bb5078a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.715+0000 7fe793e00640 1 -- 172.21.15.1:0/1496738600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bb50792a0 con 0x555bb50682d0 2024-04-26T02:02:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.716+0000 7fe793e00640 1 --2- 172.21.15.1:0/1496738600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50682d0 0x555bb50784d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe784006000 tx=0x7fe7840119a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a2e19e128944961 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.716+0000 7fe788c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:02:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.716+0000 7fe788c00640 1 -- 172.21.15.1:0/1496738600 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78402b070 con 0x555bb50682d0 2024-04-26T02:02:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.716+0000 7fe7a4c31740 1 -- 172.21.15.1:0/1496738600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe784009680 con 0x555bb50682d0 2024-04-26T02:02:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.716+0000 7fe788c00640 1 -- 172.21.15.1:0/1496738600 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8527+0+0 (secure 0 0 0) 0x7fe784011e10 con 0x555bb50682d0 2024-04-26T02:02:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.716+0000 7fe788c00640 1 -- 172.21.15.1:0/1496738600 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe784040050 con 0x555bb50682d0 2024-04-26T02:02:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.716+0000 7fe7a4c31740 10 client.26522 fetch_fsmap learned FSMap version 34 2024-04-26T02:02:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.716+0000 7fe7a4c31740 1 -- 172.21.15.1:0/1496738600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555bb50e95a0 con 0x555bb50682d0 2024-04-26T02:02:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.717+0000 7fe788c00640 1 -- 172.21.15.1:0/1496738600 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe784009680 con 0x555bb50682d0 2024-04-26T02:02:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.717+0000 7fe788c00640 1 -- 172.21.15.1:0/1496738600 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7fe7840035f0 con 0x555bb50682d0 2024-04-26T02:02:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.717+0000 7fe7a4c31740 10 client.26522 fetch_fsmap finished waiting for FSMap version 34 2024-04-26T02:02:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.717+0000 7fe7a4c31740 10 client.26522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:02:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.717+0000 7fe7a4c31740 20 client.26522 populate_metadata read hostname 'smithi001' 2024-04-26T02:02:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.717+0000 7fe7a4c31740 1 --2- 172.21.15.1:0/1496738600 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x555bb50e9c30 0x555bb50ec050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.717+0000 7fe7a4c31740 4 client.26522 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:02:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.717+0000 7fe7a4c31740 1 -- 172.21.15.1:0/1496738600 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555bb4fdb320 con 0x555bb50e9c30 2024-04-26T02:02:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.717+0000 7fe79b400640 1 --2- 172.21.15.1:0/1496738600 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x555bb50e9c30 0x555bb50ec050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.718+0000 7fe79b400640 1 --2- 172.21.15.1:0/1496738600 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x555bb50e9c30 0x555bb50ec050 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b0548cf83ca9e71 server_cookie=f6e101eb8109dff2 in_seq=0 out_seq=0 2024-04-26T02:02:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.718+0000 7fe788c00640 10 client.26522 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:02:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.732+0000 7fe788c00640 1 -- 172.21.15.1:0/1496738600 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555bb4fdb320 con 0x555bb50e9c30 2024-04-26T02:02:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.732+0000 7fe788c00640 10 client.26522 handle_command_reply: tid=0 2024-04-26T02:02:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.732+0000 7fe7a4c31740 1 client.26522 shutdown 2024-04-26T02:02:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.732+0000 7fe7a4c31740 1 -- 172.21.15.1:0/1496738600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50682d0 msgr2=0x555bb50784d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.733+0000 7fe7a4c31740 1 --2- 172.21.15.1:0/1496738600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50682d0 0x555bb50784d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe784006000 tx=0x7fe7840119a0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.733+0000 7fe7a4c31740 1 -- 172.21.15.1:0/1496738600 shutdown_connections 2024-04-26T02:02:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.733+0000 7fe7a4c31740 1 --2- 172.21.15.1:0/1496738600 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x555bb50e9c30 0x555bb50ec050 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.733+0000 7fe7a4c31740 1 --2- 172.21.15.1:0/1496738600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bb5072b90 0x555bb5078a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.733+0000 7fe7a4c31740 1 --2- 172.21.15.1:0/1496738600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555bb50682d0 0x555bb50784d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.733+0000 7fe7a4c31740 1 --2- 172.21.15.1:0/1496738600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555bb50679b0 0x555bb507b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.733+0000 7fe788c00640 0 client.26522 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:02:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.733+0000 7fe7a4c31740 1 -- 172.21.15.1:0/1496738600 >> 172.21.15.1:0/1496738600 conn(0x555bb5063ec0 msgr2=0x555bb506faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.733+0000 7fe7a4c31740 1 -- 172.21.15.1:0/1496738600 shutdown_connections 2024-04-26T02:02:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.733+0000 7fe7a4c31740 1 -- 172.21.15.1:0/1496738600 wait complete. 2024-04-26T02:02:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.733+0000 7fe7a4c31740 20 client.26522 trim_cache size 0 max 16384 2024-04-26T02:02:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T02:02:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T02:02:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T02:02:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T02:02:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.734+0000 7fe79be00640 1 -- 172.21.15.1:0/2699398893 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe7800a3d80 msgr2=0x7fe7800a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.734+0000 7fe79be00640 1 --2- 172.21.15.1:0/2699398893 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe7800a3d80 0x7fe7800a6240 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x555bb5068f50 tx=0x7fe78c02a480 comp rx=0 tx=0).stop 2024-04-26T02:02:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.734+0000 7fe79be00640 1 -- 172.21.15.1:0/2699398893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe79c0742f0 msgr2=0x7fe79c1a47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.734+0000 7fe79be00640 1 --2- 172.21.15.1:0/2699398893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe79c0742f0 0x7fe79c1a47d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe78c005940 tx=0x7fe78c004270 comp rx=0 tx=0).stop 2024-04-26T02:02:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.734+0000 7fe79be00640 1 -- 172.21.15.1:0/2699398893 shutdown_connections 2024-04-26T02:02:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.734+0000 7fe79be00640 1 --2- 172.21.15.1:0/2699398893 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe7800a3d80 0x7fe7800a6240 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.734+0000 7fe79be00640 1 --2- 172.21.15.1:0/2699398893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe79c112d80 0x7fe79c19e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.734+0000 7fe79be00640 1 --2- 172.21.15.1:0/2699398893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe79c074c10 0x7fe79c1a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.734+0000 7fe79be00640 1 --2- 172.21.15.1:0/2699398893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe79c0742f0 0x7fe79c1a47d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.734+0000 7fe79be00640 1 -- 172.21.15.1:0/2699398893 >> 172.21.15.1:0/2699398893 conn(0x7fe79c0fd860 msgr2=0x7fe79c0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.734+0000 7fe79be00640 1 -- 172.21.15.1:0/2699398893 shutdown_connections 2024-04-26T02:02:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:02:48.734+0000 7fe79be00640 1 -- 172.21.15.1:0/2699398893 wait complete. 2024-04-26T02:02:50.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:50 smithi001 ceph-mon[27822]: pgmap v350: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 886 KiB/s wr, 287 op/s 2024-04-26T02:02:50.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:50 smithi097 ceph-mon[32853]: pgmap v350: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 886 KiB/s wr, 287 op/s 2024-04-26T02:02:50.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:50 smithi173 ceph-mon[32553]: pgmap v350: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 886 KiB/s wr, 287 op/s 2024-04-26T02:02:52.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:52 smithi001 ceph-mon[27822]: pgmap v351: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.3 MiB/s wr, 435 op/s 2024-04-26T02:02:52.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:52 smithi097 ceph-mon[32853]: pgmap v351: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.3 MiB/s wr, 435 op/s 2024-04-26T02:02:52.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:52 smithi173 ceph-mon[32553]: pgmap v351: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.3 MiB/s wr, 435 op/s 2024-04-26T02:02:54.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:54 smithi001 ceph-mon[27822]: pgmap v352: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 867 KiB/s wr, 287 op/s 2024-04-26T02:02:54.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:54 smithi097 ceph-mon[32853]: pgmap v352: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 867 KiB/s wr, 287 op/s 2024-04-26T02:02:54.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:54 smithi173 ceph-mon[32553]: pgmap v352: 97 pgs: 97 active+clean; 152 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 867 KiB/s wr, 287 op/s 2024-04-26T02:02:56.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:56 smithi001 ceph-mon[27822]: pgmap v353: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.3 MiB/s wr, 421 op/s 2024-04-26T02:02:56.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:56 smithi097 ceph-mon[32853]: pgmap v353: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.3 MiB/s wr, 421 op/s 2024-04-26T02:02:56.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:56 smithi173 ceph-mon[32553]: pgmap v353: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.3 MiB/s wr, 421 op/s 2024-04-26T02:02:58.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:02:58 smithi001 ceph-mon[27822]: pgmap v354: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 833 KiB/s wr, 281 op/s 2024-04-26T02:02:58.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:02:58 smithi097 ceph-mon[32853]: pgmap v354: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 833 KiB/s wr, 281 op/s 2024-04-26T02:02:58.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:02:58 smithi173 ceph-mon[32553]: pgmap v354: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 833 KiB/s wr, 281 op/s 2024-04-26T02:03:00.381 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:00 smithi001 ceph-mon[27822]: pgmap v355: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 833 KiB/s wr, 281 op/s 2024-04-26T02:03:00.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:00 smithi097 ceph-mon[32853]: pgmap v355: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 833 KiB/s wr, 281 op/s 2024-04-26T02:03:00.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:00 smithi173 ceph-mon[32553]: pgmap v355: 97 pgs: 97 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 833 KiB/s wr, 281 op/s 2024-04-26T02:03:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.900+0000 7fa7ace00640 1 -- 172.21.15.1:0/3106451603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a810e8f0 msgr2=0x7fa7a8110ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.900+0000 7fa7ace00640 1 --2- 172.21.15.1:0/3106451603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a810e8f0 0x7fa7a8110ce0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009be0 tx=0x7fa7a002a6c0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.901+0000 7fa7ace00640 1 -- 172.21.15.1:0/3106451603 shutdown_connections 2024-04-26T02:03:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.901+0000 7fa7ace00640 1 --2- 172.21.15.1:0/3106451603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a810e8f0 0x7fa7a8110ce0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.901+0000 7fa7ace00640 1 --2- 172.21.15.1:0/3106451603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7a8101660 0x7fa7a810e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.901+0000 7fa7ace00640 1 --2- 172.21.15.1:0/3106451603 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7a8100d40 0x7fa7a8101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.901+0000 7fa7ace00640 1 -- 172.21.15.1:0/3106451603 >> 172.21.15.1:0/3106451603 conn(0x7fa7a8077f70 msgr2=0x7fa7a8078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.901+0000 7fa7ace00640 1 -- 172.21.15.1:0/3106451603 shutdown_connections 2024-04-26T02:03:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.901+0000 7fa7ace00640 1 -- 172.21.15.1:0/3106451603 wait complete. 2024-04-26T02:03:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.902+0000 7fa7ace00640 1 Processor -- start 2024-04-26T02:03:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.902+0000 7fa7ace00640 1 -- start start 2024-04-26T02:03:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.902+0000 7fa7ace00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a8100d40 0x7fa7a81a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.903+0000 7fa7ace00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7a8101660 0x7fa7a81a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.903+0000 7fa7ace00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7a810e8f0 0x7fa7a819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.903+0000 7fa7ace00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81143d0 con 0x7fa7a810e8f0 2024-04-26T02:03:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.903+0000 7fa7ace00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a8114250 con 0x7fa7a8100d40 2024-04-26T02:03:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.903+0000 7fa7ace00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a8114550 con 0x7fa7a8101660 2024-04-26T02:03:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.903+0000 7fa7a5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a8100d40 0x7fa7a81a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.903+0000 7fa7a5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a8100d40 0x7fa7a81a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:56310/0 (socket says 172.21.15.1:56310) 2024-04-26T02:03:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.903+0000 7fa7a5e00640 1 -- 172.21.15.1:0/768629464 learned_addr learned my addr 172.21.15.1:0/768629464 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.903+0000 7fa7a5400640 1 --2- 172.21.15.1:0/768629464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7a8101660 0x7fa7a81a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.903+0000 7fa7a6800640 1 --2- 172.21.15.1:0/768629464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7a810e8f0 0x7fa7a819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.903+0000 7fa7a5e00640 1 -- 172.21.15.1:0/768629464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7a8101660 msgr2=0x7fa7a81a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.903+0000 7fa7a5e00640 1 --2- 172.21.15.1:0/768629464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7a8101660 0x7fa7a81a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.903+0000 7fa7a5e00640 1 -- 172.21.15.1:0/768629464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7a810e8f0 msgr2=0x7fa7a819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.903+0000 7fa7a5e00640 1 --2- 172.21.15.1:0/768629464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7a810e8f0 0x7fa7a819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.903+0000 7fa7a5e00640 1 -- 172.21.15.1:0/768629464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a819b010 con 0x7fa7a8100d40 2024-04-26T02:03:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.903+0000 7fa7a6800640 1 --2- 172.21.15.1:0/768629464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7a810e8f0 0x7fa7a819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:03:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.904+0000 7fa7a5e00640 1 --2- 172.21.15.1:0/768629464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a8100d40 0x7fa7a81a0690 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa79800e9c0 tx=0x7fa79800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.904+0000 7fa79ea00640 1 -- 172.21.15.1:0/768629464 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79800ccc0 con 0x7fa7a8100d40 2024-04-26T02:03:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.904+0000 7fa79ea00640 1 -- 172.21.15.1:0/768629464 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7980054d0 con 0x7fa7a8100d40 2024-04-26T02:03:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.904+0000 7fa79ea00640 1 -- 172.21.15.1:0/768629464 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa798010430 con 0x7fa7a8100d40 2024-04-26T02:03:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.904+0000 7fa7ace00640 1 -- 172.21.15.1:0/768629464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a819b300 con 0x7fa7a8100d40 2024-04-26T02:03:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.904+0000 7fa7ace00640 1 -- 172.21.15.1:0/768629464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a81a7610 con 0x7fa7a8100d40 2024-04-26T02:03:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.908+0000 7fa7afc1e740 1 Processor -- start 2024-04-26T02:03:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.908+0000 7fa79ea00640 1 -- 172.21.15.1:0/768629464 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fa798004f80 con 0x7fa7a8100d40 2024-04-26T02:03:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.908+0000 7fa7afc1e740 1 -- start start 2024-04-26T02:03:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.909+0000 7fa7afc1e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b02109b820 0x55b02109bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.909+0000 7fa7afc1e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b02109c140 0x55b0210a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.909+0000 7fa7afc1e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b0210a6a00 0x55b0210a8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.909+0000 7fa7afc1e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b021001900 con 0x55b0210a6a00 2024-04-26T02:03:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.909+0000 7fa7afc1e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b020fa7ef0 con 0x55b02109b820 2024-04-26T02:03:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.909+0000 7fa7afc1e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b020ffd8c0 con 0x55b02109c140 2024-04-26T02:03:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.909+0000 7fa7a6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b02109b820 0x55b02109bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.909+0000 7fa7a6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b02109b820 0x55b02109bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:56312/0 (socket says 172.21.15.1:56312) 2024-04-26T02:03:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.909+0000 7fa7a6800640 1 -- 172.21.15.1:0/2997962777 learned_addr learned my addr 172.21.15.1:0/2997962777 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.909+0000 7fa7a5e00640 1 --2- 172.21.15.1:0/2997962777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b02109c140 0x55b0210a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.909+0000 7fa7a5400640 1 --2- 172.21.15.1:0/2997962777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b0210a6a00 0x55b0210a8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.909+0000 7fa7a6800640 1 -- 172.21.15.1:0/2997962777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b02109c140 msgr2=0x55b0210a64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.909+0000 7fa7a6800640 1 --2- 172.21.15.1:0/2997962777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b02109c140 0x55b0210a64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.909+0000 7fa7a6800640 1 -- 172.21.15.1:0/2997962777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b0210a6a00 msgr2=0x55b0210a8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.909+0000 7fa7a6800640 1 --2- 172.21.15.1:0/2997962777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b0210a6a00 0x55b0210a8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.909+0000 7fa7a6800640 1 -- 172.21.15.1:0/2997962777 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b020d14210 con 0x55b02109b820 2024-04-26T02:03:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.910+0000 7fa7a6800640 1 --2- 172.21.15.1:0/2997962777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b02109b820 0x55b02109bc00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa7a002abe0 tx=0x7fa7a0030430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=400086c0c51cbd64 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.910+0000 7fa793400640 1 -- 172.21.15.1:0/2997962777 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0040030 con 0x55b02109b820 2024-04-26T02:03:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.910+0000 7fa793400640 1 -- 172.21.15.1:0/2997962777 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7a0003e20 con 0x55b02109b820 2024-04-26T02:03:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.910+0000 7fa793400640 1 -- 172.21.15.1:0/2997962777 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0036040 con 0x55b02109b820 2024-04-26T02:03:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.910+0000 7fa79ea00640 1 --2- 172.21.15.1:0/768629464 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fa78c0a3d90 0x7fa78c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.911+0000 7fa79ea00640 1 -- 172.21.15.1:0/768629464 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8527+0+0 (secure 0 0 0) 0x7fa7980d0d30 con 0x7fa7a8100d40 2024-04-26T02:03:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.911+0000 7fa7a5e00640 1 --2- 172.21.15.1:0/768629464 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fa78c0a3d90 0x7fa78c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.911+0000 7fa7afc1e740 1 -- 172.21.15.1:0/2997962777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b02109b820 msgr2=0x55b02109bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.911+0000 7fa7afc1e740 1 --2- 172.21.15.1:0/2997962777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b02109b820 0x55b02109bc00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa7a002abe0 tx=0x7fa7a0030430 comp rx=0 tx=0).stop 2024-04-26T02:03:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.912+0000 7fa7afc1e740 1 -- 172.21.15.1:0/2997962777 shutdown_connections 2024-04-26T02:03:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.912+0000 7fa7afc1e740 1 --2- 172.21.15.1:0/2997962777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b0210a6a00 0x55b0210a8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.912+0000 7fa7afc1e740 1 --2- 172.21.15.1:0/2997962777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b02109c140 0x55b0210a64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.912+0000 7fa7afc1e740 1 --2- 172.21.15.1:0/2997962777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b02109b820 0x55b02109bc00 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.912+0000 7fa7afc1e740 1 -- 172.21.15.1:0/2997962777 >> 172.21.15.1:0/2997962777 conn(0x55b021097d30 msgr2=0x55b021098110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.912+0000 7fa7a5e00640 1 --2- 172.21.15.1:0/768629464 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fa78c0a3d90 0x7fa78c0a6250 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55b02109cdc0 tx=0x7fa7980d47e0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.912+0000 7fa7afc1e740 1 -- 172.21.15.1:0/2997962777 shutdown_connections 2024-04-26T02:03:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.912+0000 7fa7afc1e740 1 -- 172.21.15.1:0/2997962777 wait complete. 2024-04-26T02:03:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.912+0000 7fa7afc1e740 1 -- start start 2024-04-26T02:03:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.912+0000 7fa7afc1e740 1 Processor -- start 2024-04-26T02:03:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.913+0000 7fa7afc1e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b02109b820 0x55b0210a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.913+0000 7fa7afc1e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b02109c140 0x55b0210a3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.913+0000 7fa7afc1e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b0210a6a00 0x55b0210a4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.913+0000 7fa7afc1e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b0210a95d0 con 0x55b02109c140 2024-04-26T02:03:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.913+0000 7fa7afc1e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b020fbb310 con 0x55b0210a6a00 2024-04-26T02:03:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.913+0000 7fa7afc1e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b0210a9750 con 0x55b02109b820 2024-04-26T02:03:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.913+0000 7fa7afc1e740 10 client.0 _ll_register_callbacks cb 0x55b020f89cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.913+0000 7fa7a6800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b02109b820 0x55b0210a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.913+0000 7fa7a6800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b02109b820 0x55b0210a39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:47534/0 (socket says 172.21.15.1:47534) 2024-04-26T02:03:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.913+0000 7fa7a6800640 1 -- 172.21.15.1:0/4244582367 learned_addr learned my addr 172.21.15.1:0/4244582367 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.913+0000 7fa7a6800640 1 --2- 172.21.15.1:0/4244582367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b0210a6a00 0x55b0210a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.913+0000 7fa7a5400640 1 --2- 172.21.15.1:0/4244582367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b02109c140 0x55b0210a3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.914+0000 7fa7a6800640 1 -- 172.21.15.1:0/4244582367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b0210a6a00 msgr2=0x55b0210a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.914+0000 7fa7a6800640 1 --2- 172.21.15.1:0/4244582367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b0210a6a00 0x55b0210a4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.914+0000 7fa7a6800640 1 -- 172.21.15.1:0/4244582367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b02109c140 msgr2=0x55b0210a3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.914+0000 7fa7a6800640 1 --2- 172.21.15.1:0/4244582367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b02109c140 0x55b0210a3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.914+0000 7fa7a6800640 1 -- 172.21.15.1:0/4244582367 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b020de0620 con 0x55b02109b820 2024-04-26T02:03:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.914+0000 7fa7a6800640 1 --2- 172.21.15.1:0/4244582367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b02109b820 0x55b0210a39c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0005f00 tx=0x7fa7a0009280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc9ad897c19c392f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.914+0000 7fa787e00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T02:03:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.914+0000 7fa787e00640 1 -- 172.21.15.1:0/4244582367 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0036070 con 0x55b02109b820 2024-04-26T02:03:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.914+0000 7fa787e00640 1 -- 172.21.15.1:0/4244582367 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8527+0+0 (secure 0 0 0) 0x7fa7a0030a30 con 0x55b02109b820 2024-04-26T02:03:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.914+0000 7fa7afc1e740 1 -- 172.21.15.1:0/4244582367 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7a0003e20 con 0x55b02109b820 2024-04-26T02:03:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.915+0000 7fa787e00640 1 -- 172.21.15.1:0/4244582367 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa7a0030c30 con 0x55b02109b820 2024-04-26T02:03:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.915+0000 7fa7afc1e740 10 client.26545 fetch_fsmap learned FSMap version 34 2024-04-26T02:03:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.915+0000 7fa7afc1e740 1 -- 172.21.15.1:0/4244582367 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b0210a95d0 con 0x55b02109b820 2024-04-26T02:03:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.915+0000 7fa787e00640 1 -- 172.21.15.1:0/4244582367 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7a0003e20 con 0x55b02109b820 2024-04-26T02:03:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.915+0000 7fa787e00640 1 -- 172.21.15.1:0/4244582367 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7fa7a0031440 con 0x55b02109b820 2024-04-26T02:03:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.915+0000 7fa7afc1e740 10 client.26545 fetch_fsmap finished waiting for FSMap version 34 2024-04-26T02:03:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.915+0000 7fa7afc1e740 10 client.26545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:03:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.915+0000 7fa7afc1e740 20 client.26545 populate_metadata read hostname 'smithi001' 2024-04-26T02:03:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.915+0000 7fa7afc1e740 1 --2- 172.21.15.1:0/4244582367 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55b0210b6a20 0x55b0210b8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.915+0000 7fa7afc1e740 4 client.26545 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:03:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.915+0000 7fa7afc1e740 1 -- 172.21.15.1:0/4244582367 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0210158b0 con 0x55b0210b6a20 2024-04-26T02:03:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.916+0000 7fa7a5400640 1 --2- 172.21.15.1:0/4244582367 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55b0210b6a20 0x55b0210b8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.917+0000 7fa7a5400640 1 --2- 172.21.15.1:0/4244582367 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55b0210b6a20 0x55b0210b8e10 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85d11aed2f3b6450 server_cookie=583f95b90c005398 in_seq=0 out_seq=0 2024-04-26T02:03:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.917+0000 7fa787e00640 10 client.26545 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.917+0000 7fa787e00640 1 -- 172.21.15.1:0/4244582367 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b0210158b0 con 0x55b0210b6a20 2024-04-26T02:03:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.918+0000 7fa787e00640 10 client.26545 handle_command_reply: tid=0 2024-04-26T02:03:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.918+0000 7fa7afc1e740 1 client.26545 shutdown 2024-04-26T02:03:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.918+0000 7fa7afc1e740 1 -- 172.21.15.1:0/4244582367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b02109b820 msgr2=0x55b0210a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:01.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.918+0000 7fa7afc1e740 1 --2- 172.21.15.1:0/4244582367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b02109b820 0x55b0210a39c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0005f00 tx=0x7fa7a0009280 comp rx=0 tx=0).stop 2024-04-26T02:03:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.918+0000 7fa7afc1e740 1 -- 172.21.15.1:0/4244582367 shutdown_connections 2024-04-26T02:03:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.918+0000 7fa7afc1e740 1 --2- 172.21.15.1:0/4244582367 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55b0210b6a20 0x55b0210b8e10 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.918+0000 7fa7afc1e740 1 --2- 172.21.15.1:0/4244582367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b0210a6a00 0x55b0210a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.918+0000 7fa7afc1e740 1 --2- 172.21.15.1:0/4244582367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b02109c140 0x55b0210a3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.918+0000 7fa787e00640 0 client.26545 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.918+0000 7fa7afc1e740 1 --2- 172.21.15.1:0/4244582367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b02109b820 0x55b0210a39c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.918+0000 7fa7afc1e740 1 -- 172.21.15.1:0/4244582367 >> 172.21.15.1:0/4244582367 conn(0x55b021097d30 msgr2=0x55b02109da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.918+0000 7fa7afc1e740 1 -- 172.21.15.1:0/4244582367 shutdown_connections 2024-04-26T02:03:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.918+0000 7fa7afc1e740 1 -- 172.21.15.1:0/4244582367 wait complete. 2024-04-26T02:03:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.918+0000 7fa7afc1e740 20 client.26545 trim_cache size 0 max 16384 2024-04-26T02:03:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.935+0000 7fa7afc1e740 1 Processor -- start 2024-04-26T02:03:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.936+0000 7fa7afc1e740 1 -- start start 2024-04-26T02:03:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.936+0000 7fa7afc1e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b02109b820 0x55b0210b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.936+0000 7fa7afc1e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b02109c140 0x55b0210be570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.936+0000 7fa7afc1e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b0210a6a00 0x55b0210beab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.936+0000 7fa7afc1e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81b2780 con 0x55b0210a6a00 2024-04-26T02:03:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.936+0000 7fa7afc1e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81b0050 con 0x55b02109c140 2024-04-26T02:03:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.936+0000 7fa7afc1e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81adea0 con 0x55b02109b820 2024-04-26T02:03:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.936+0000 7fa7a6800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b02109b820 0x55b0210b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.936+0000 7fa7a6800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b02109b820 0x55b0210b2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:47546/0 (socket says 172.21.15.1:47546) 2024-04-26T02:03:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.936+0000 7fa7a6800640 1 -- 172.21.15.1:0/1430894000 learned_addr learned my addr 172.21.15.1:0/1430894000 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.936+0000 7fa7a6800640 1 --2- 172.21.15.1:0/1430894000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b0210a6a00 0x55b0210beab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.936+0000 7fa7a5400640 1 --2- 172.21.15.1:0/1430894000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b02109c140 0x55b0210be570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.937+0000 7fa7a5400640 1 -- 172.21.15.1:0/1430894000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b02109b820 msgr2=0x55b0210b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.937+0000 7fa7a5400640 1 --2- 172.21.15.1:0/1430894000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b02109b820 0x55b0210b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.937+0000 7fa7a5400640 1 -- 172.21.15.1:0/1430894000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b0210a6a00 msgr2=0x55b0210beab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.937+0000 7fa7a5400640 1 --2- 172.21.15.1:0/1430894000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b0210a6a00 0x55b0210beab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.937+0000 7fa7a5400640 1 -- 172.21.15.1:0/1430894000 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0210a1f30 con 0x55b02109c140 2024-04-26T02:03:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.937+0000 7fa7a6800640 1 --2- 172.21.15.1:0/1430894000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b02109b820 0x55b0210b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.937+0000 7fa7a5400640 1 --2- 172.21.15.1:0/1430894000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b02109c140 0x55b0210be570 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55b0210a3220 tx=0x7fa794005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e659fc2ca225ecb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.937+0000 7fa792a00640 1 -- 172.21.15.1:0/1430894000 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa794004050 con 0x55b02109c140 2024-04-26T02:03:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.937+0000 7fa792a00640 1 -- 172.21.15.1:0/1430894000 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7940041f0 con 0x55b02109c140 2024-04-26T02:03:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.937+0000 7fa792a00640 1 -- 172.21.15.1:0/1430894000 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79401f3f0 con 0x55b02109c140 2024-04-26T02:03:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.938+0000 7fa7afc1e740 1 -- 172.21.15.1:0/1430894000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b02109c140 msgr2=0x55b0210be570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.938+0000 7fa7afc1e740 1 --2- 172.21.15.1:0/1430894000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b02109c140 0x55b0210be570 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55b0210a3220 tx=0x7fa794005f90 comp rx=0 tx=0).stop 2024-04-26T02:03:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.938+0000 7fa7afc1e740 1 -- 172.21.15.1:0/1430894000 shutdown_connections 2024-04-26T02:03:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.938+0000 7fa7afc1e740 1 --2- 172.21.15.1:0/1430894000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b0210a6a00 0x55b0210beab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.938+0000 7fa7afc1e740 1 --2- 172.21.15.1:0/1430894000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b02109c140 0x55b0210be570 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.938+0000 7fa7afc1e740 1 --2- 172.21.15.1:0/1430894000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b02109b820 0x55b0210b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.938+0000 7fa7afc1e740 1 -- 172.21.15.1:0/1430894000 >> 172.21.15.1:0/1430894000 conn(0x55b021097d30 msgr2=0x55b021098110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.938+0000 7fa7afc1e740 1 -- 172.21.15.1:0/1430894000 shutdown_connections 2024-04-26T02:03:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.939+0000 7fa7afc1e740 1 -- 172.21.15.1:0/1430894000 wait complete. 2024-04-26T02:03:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.939+0000 7fa7afc1e740 1 -- start start 2024-04-26T02:03:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.939+0000 7fa7afc1e740 1 Processor -- start 2024-04-26T02:03:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.940+0000 7fa7afc1e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b02109b820 0x55b0210ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.940+0000 7fa7afc1e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b02109c140 0x55b0210ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.940+0000 7fa7afc1e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b0210a6a00 0x55b0210ace30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.940+0000 7fa7afc1e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81ae510 con 0x55b02109b820 2024-04-26T02:03:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.940+0000 7fa7afc1e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81af980 con 0x55b02109c140 2024-04-26T02:03:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.940+0000 7fa7afc1e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a81aebe0 con 0x55b0210a6a00 2024-04-26T02:03:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.940+0000 7fa7afc1e740 10 client.0 _ll_register_callbacks cb 0x7fa794005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.940+0000 7fa7a6800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b02109b820 0x55b0210ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.941+0000 7fa7a5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b02109c140 0x55b0210ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.941+0000 7fa7a6800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b02109b820 0x55b0210ac3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46826/0 (socket says 172.21.15.1:46826) 2024-04-26T02:03:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.941+0000 7fa7a6800640 1 -- 172.21.15.1:0/2354698591 learned_addr learned my addr 172.21.15.1:0/2354698591 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.941+0000 7fa7a6800640 1 --2- 172.21.15.1:0/2354698591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b0210a6a00 0x55b0210ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.941+0000 7fa7a5400640 1 -- 172.21.15.1:0/2354698591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b0210a6a00 msgr2=0x55b0210ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.941+0000 7fa7a5400640 1 --2- 172.21.15.1:0/2354698591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b0210a6a00 0x55b0210ace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.941+0000 7fa7a5400640 1 -- 172.21.15.1:0/2354698591 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b02109b820 msgr2=0x55b0210ac3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.941+0000 7fa7a5400640 1 --2- 172.21.15.1:0/2354698591 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b02109b820 0x55b0210ac3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.941+0000 7fa7a5400640 1 -- 172.21.15.1:0/2354698591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b02111bdf0 con 0x55b02109c140 2024-04-26T02:03:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.941+0000 7fa7a6800640 1 --2- 172.21.15.1:0/2354698591 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b02109b820 0x55b0210ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.941+0000 7fa7a5400640 1 --2- 172.21.15.1:0/2354698591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b02109c140 0x55b0210ac8f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55b02109e680 tx=0x7fa794004120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8bbdcefe8606e3d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.941+0000 7fa787e00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:03:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.942+0000 7fa787e00640 1 -- 172.21.15.1:0/2354698591 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa794005230 con 0x55b02109c140 2024-04-26T02:03:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.942+0000 7fa7afc1e740 1 -- 172.21.15.1:0/2354698591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7940041f0 con 0x55b02109c140 2024-04-26T02:03:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.942+0000 7fa787e00640 1 -- 172.21.15.1:0/2354698591 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8527+0+0 (secure 0 0 0) 0x7fa7940053d0 con 0x55b02109c140 2024-04-26T02:03:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.942+0000 7fa787e00640 1 -- 172.21.15.1:0/2354698591 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa7940055d0 con 0x55b02109c140 2024-04-26T02:03:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.942+0000 7fa7afc1e740 10 client.26540 fetch_fsmap learned FSMap version 34 2024-04-26T02:03:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.942+0000 7fa7afc1e740 1 -- 172.21.15.1:0/2354698591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b02111d060 con 0x55b02109c140 2024-04-26T02:03:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.942+0000 7fa787e00640 1 -- 172.21.15.1:0/2354698591 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7940041f0 con 0x55b02109c140 2024-04-26T02:03:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.942+0000 7fa787e00640 1 -- 172.21.15.1:0/2354698591 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7fa79401f440 con 0x55b02109c140 2024-04-26T02:03:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.942+0000 7fa7afc1e740 10 client.26540 fetch_fsmap finished waiting for FSMap version 34 2024-04-26T02:03:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.942+0000 7fa7afc1e740 10 client.26540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:03:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.942+0000 7fa7afc1e740 20 client.26540 populate_metadata read hostname 'smithi001' 2024-04-26T02:03:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.943+0000 7fa7afc1e740 1 --2- 172.21.15.1:0/2354698591 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55b02111d670 0x55b02111fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.943+0000 7fa7afc1e740 4 client.26540 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:03:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.943+0000 7fa7afc1e740 1 -- 172.21.15.1:0/2354698591 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b02100f320 con 0x55b02111d670 2024-04-26T02:03:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.943+0000 7fa7a6800640 1 --2- 172.21.15.1:0/2354698591 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55b02111d670 0x55b02111fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.943+0000 7fa7a6800640 1 --2- 172.21.15.1:0/2354698591 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55b02111d670 0x55b02111fa60 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=466f961ef61490cf server_cookie=b1dae1dd6adc6627 in_seq=0 out_seq=0 2024-04-26T02:03:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.943+0000 7fa787e00640 10 client.26540 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.964+0000 7fa787e00640 1 -- 172.21.15.1:0/2354698591 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b02100f320 con 0x55b02111d670 2024-04-26T02:03:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.964+0000 7fa787e00640 10 client.26540 handle_command_reply: tid=0 2024-04-26T02:03:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.964+0000 7fa7afc1e740 1 client.26540 shutdown 2024-04-26T02:03:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.965+0000 7fa7afc1e740 1 -- 172.21.15.1:0/2354698591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b02109c140 msgr2=0x55b0210ac8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.965+0000 7fa7afc1e740 1 --2- 172.21.15.1:0/2354698591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b02109c140 0x55b0210ac8f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55b02109e680 tx=0x7fa794004120 comp rx=0 tx=0).stop 2024-04-26T02:03:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.965+0000 7fa7afc1e740 1 -- 172.21.15.1:0/2354698591 shutdown_connections 2024-04-26T02:03:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.965+0000 7fa7afc1e740 1 --2- 172.21.15.1:0/2354698591 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55b02111d670 0x55b02111fa60 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.965+0000 7fa7afc1e740 1 --2- 172.21.15.1:0/2354698591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b0210a6a00 0x55b0210ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.965+0000 7fa7afc1e740 1 --2- 172.21.15.1:0/2354698591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b02109c140 0x55b0210ac8f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.965+0000 7fa7afc1e740 1 --2- 172.21.15.1:0/2354698591 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b02109b820 0x55b0210ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.965+0000 7fa787e00640 0 client.26540 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.965+0000 7fa7afc1e740 1 -- 172.21.15.1:0/2354698591 >> 172.21.15.1:0/2354698591 conn(0x55b021097d30 msgr2=0x55b021096600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.965+0000 7fa7afc1e740 1 -- 172.21.15.1:0/2354698591 shutdown_connections 2024-04-26T02:03:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.965+0000 7fa7afc1e740 1 -- 172.21.15.1:0/2354698591 wait complete. 2024-04-26T02:03:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.965+0000 7fa7afc1e740 20 client.26540 trim_cache size 0 max 16384 2024-04-26T02:03:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T02:03:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T02:03:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T02:03:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T02:03:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.966+0000 7fa7ace00640 1 -- 172.21.15.1:0/768629464 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fa78c0a3d90 msgr2=0x7fa78c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.966+0000 7fa7ace00640 1 --2- 172.21.15.1:0/768629464 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fa78c0a3d90 0x7fa78c0a6250 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55b02109cdc0 tx=0x7fa7980d47e0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.966+0000 7fa7ace00640 1 -- 172.21.15.1:0/768629464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a8100d40 msgr2=0x7fa7a81a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.966+0000 7fa7ace00640 1 --2- 172.21.15.1:0/768629464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a8100d40 0x7fa7a81a0690 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa79800e9c0 tx=0x7fa79800ee80 comp rx=0 tx=0).stop 2024-04-26T02:03:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.966+0000 7fa7ace00640 1 -- 172.21.15.1:0/768629464 shutdown_connections 2024-04-26T02:03:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.966+0000 7fa7ace00640 1 --2- 172.21.15.1:0/768629464 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fa78c0a3d90 0x7fa78c0a6250 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.966+0000 7fa7ace00640 1 --2- 172.21.15.1:0/768629464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7a810e8f0 0x7fa7a819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.966+0000 7fa7ace00640 1 --2- 172.21.15.1:0/768629464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7a8101660 0x7fa7a81a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.966+0000 7fa7ace00640 1 --2- 172.21.15.1:0/768629464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7a8100d40 0x7fa7a81a0690 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.966+0000 7fa7ace00640 1 -- 172.21.15.1:0/768629464 >> 172.21.15.1:0/768629464 conn(0x7fa7a8077f70 msgr2=0x7fa7a810ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.966+0000 7fa7ace00640 1 -- 172.21.15.1:0/768629464 shutdown_connections 2024-04-26T02:03:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:01.966+0000 7fa7ace00640 1 -- 172.21.15.1:0/768629464 wait complete. 2024-04-26T02:03:02.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:02 smithi001 ceph-mon[27822]: pgmap v356: 97 pgs: 97 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.3 MiB/s wr, 413 op/s 2024-04-26T02:03:02.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:02 smithi097 ceph-mon[32853]: pgmap v356: 97 pgs: 97 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.3 MiB/s wr, 413 op/s 2024-04-26T02:03:02.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:02 smithi173 ceph-mon[32553]: pgmap v356: 97 pgs: 97 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.3 MiB/s wr, 413 op/s 2024-04-26T02:03:04.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:04 smithi001 ceph-mon[27822]: pgmap v357: 97 pgs: 97 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 903 KiB/s wr, 266 op/s 2024-04-26T02:03:04.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:04 smithi097 ceph-mon[32853]: pgmap v357: 97 pgs: 97 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 903 KiB/s wr, 266 op/s 2024-04-26T02:03:04.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:04 smithi173 ceph-mon[32553]: pgmap v357: 97 pgs: 97 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 903 KiB/s wr, 266 op/s 2024-04-26T02:03:06.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:06 smithi097 ceph-mon[32853]: pgmap v358: 97 pgs: 97 active+clean; 199 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.4 MiB/s wr, 425 op/s 2024-04-26T02:03:06.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:06 smithi097 ceph-mon[32853]: osdmap e106: 12 total, 12 up, 12 in 2024-04-26T02:03:06.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:06 smithi173 ceph-mon[32553]: pgmap v358: 97 pgs: 97 active+clean; 199 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.4 MiB/s wr, 425 op/s 2024-04-26T02:03:06.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:06 smithi173 ceph-mon[32553]: osdmap e106: 12 total, 12 up, 12 in 2024-04-26T02:03:06.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:06 smithi001 ceph-mon[27822]: pgmap v358: 97 pgs: 97 active+clean; 199 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.4 MiB/s wr, 425 op/s 2024-04-26T02:03:06.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:06 smithi001 ceph-mon[27822]: osdmap e106: 12 total, 12 up, 12 in 2024-04-26T02:03:08.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:08 smithi097 ceph-mon[32853]: pgmap v360: 97 pgs: 97 active+clean; 199 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.2 MiB/s wr, 349 op/s 2024-04-26T02:03:08.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:08 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:03:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:08 smithi173 ceph-mon[32553]: pgmap v360: 97 pgs: 97 active+clean; 199 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.2 MiB/s wr, 349 op/s 2024-04-26T02:03:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:08 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:03:08.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:08 smithi001 ceph-mon[27822]: pgmap v360: 97 pgs: 97 active+clean; 199 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.2 MiB/s wr, 349 op/s 2024-04-26T02:03:08.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:08 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:03:10.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:10 smithi097 ceph-mon[32853]: pgmap v361: 97 pgs: 97 active+clean; 199 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.2 MiB/s wr, 349 op/s 2024-04-26T02:03:10.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:10 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:03:10.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:10 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:03:10.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:10 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:03:10.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:10 smithi173 ceph-mon[32553]: pgmap v361: 97 pgs: 97 active+clean; 199 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.2 MiB/s wr, 349 op/s 2024-04-26T02:03:10.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:10 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:03:10.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:10 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:03:10.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:10 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:03:10.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:10 smithi001 ceph-mon[27822]: pgmap v361: 97 pgs: 97 active+clean; 199 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.2 MiB/s wr, 349 op/s 2024-04-26T02:03:10.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:10 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:03:10.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:10 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:03:10.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:10 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:03:12.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:12 smithi097 ceph-mon[32853]: pgmap v362: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.3 MiB/s wr, 406 op/s 2024-04-26T02:03:12.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:12 smithi173 ceph-mon[32553]: pgmap v362: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.3 MiB/s wr, 406 op/s 2024-04-26T02:03:12.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:12 smithi001 ceph-mon[27822]: pgmap v362: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.3 MiB/s wr, 406 op/s 2024-04-26T02:03:14.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:14 smithi097 ceph-mon[32853]: pgmap v363: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.3 MiB/s wr, 406 op/s 2024-04-26T02:03:14.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:14 smithi173 ceph-mon[32553]: pgmap v363: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.3 MiB/s wr, 406 op/s 2024-04-26T02:03:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:14 smithi001 ceph-mon[27822]: pgmap v363: 97 pgs: 97 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.3 MiB/s wr, 406 op/s 2024-04-26T02:03:14.961 INFO:tasks.workunit.client.0.smithi001.stdout:All 50000 operations completed A-OK! 2024-04-26T02:03:14.996 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T02:03:14.997 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T02:03:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.103+0000 7faf3f200640 1 -- 172.21.15.1:0/3419353306 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf40101fe0 msgr2=0x7faf401023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.103+0000 7faf3f200640 1 --2- 172.21.15.1:0/3419353306 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf40101fe0 0x7faf401023c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faf30009900 tx=0x7faf3002a410 comp rx=0 tx=0).stop 2024-04-26T02:03:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.103+0000 7faf3f200640 1 -- 172.21.15.1:0/3419353306 shutdown_connections 2024-04-26T02:03:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.103+0000 7faf3f200640 1 --2- 172.21.15.1:0/3419353306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf401070b0 0x7faf4010ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.103+0000 7faf3f200640 1 --2- 172.21.15.1:0/3419353306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf40102990 0x7faf401069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.103+0000 7faf3f200640 1 --2- 172.21.15.1:0/3419353306 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf40101fe0 0x7faf401023c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.103+0000 7faf3f200640 1 -- 172.21.15.1:0/3419353306 >> 172.21.15.1:0/3419353306 conn(0x7faf400fd860 msgr2=0x7faf400ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.103+0000 7faf3f200640 1 -- 172.21.15.1:0/3419353306 shutdown_connections 2024-04-26T02:03:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.103+0000 7faf3f200640 1 -- 172.21.15.1:0/3419353306 wait complete. 2024-04-26T02:03:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.104+0000 7faf3f200640 1 Processor -- start 2024-04-26T02:03:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.104+0000 7faf3f200640 1 -- start start 2024-04-26T02:03:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.105+0000 7faf3f200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf40101fe0 0x7faf40074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.105+0000 7faf3f200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf401070b0 0x7faf40074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.105+0000 7faf3f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf40074cd0 0x7faf401a6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.105+0000 7faf3f200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faf40112010 con 0x7faf40101fe0 2024-04-26T02:03:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.105+0000 7faf3f200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faf40111e90 con 0x7faf40074cd0 2024-04-26T02:03:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.105+0000 7faf3f200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faf40112190 con 0x7faf401070b0 2024-04-26T02:03:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.105+0000 7faf3de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf40101fe0 0x7faf40074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.105+0000 7faf3de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf40101fe0 0x7faf40074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55968/0 (socket says 172.21.15.1:55968) 2024-04-26T02:03:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.105+0000 7faf3de00640 1 -- 172.21.15.1:0/1933395712 learned_addr learned my addr 172.21.15.1:0/1933395712 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.105+0000 7faf3e800640 1 --2- 172.21.15.1:0/1933395712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf40074cd0 0x7faf401a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.105+0000 7faf3d400640 1 --2- 172.21.15.1:0/1933395712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf401070b0 0x7faf40074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.105+0000 7faf3e800640 1 -- 172.21.15.1:0/1933395712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf401070b0 msgr2=0x7faf40074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.105+0000 7faf3e800640 1 --2- 172.21.15.1:0/1933395712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf401070b0 0x7faf40074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.106+0000 7faf3e800640 1 -- 172.21.15.1:0/1933395712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf40101fe0 msgr2=0x7faf40074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.106+0000 7faf3e800640 1 --2- 172.21.15.1:0/1933395712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf40101fe0 0x7faf40074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.106+0000 7faf3e800640 1 -- 172.21.15.1:0/1933395712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf401a6e60 con 0x7faf40074cd0 2024-04-26T02:03:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.106+0000 7faf3e800640 1 --2- 172.21.15.1:0/1933395712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf40074cd0 0x7faf401a6760 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faf38009fd0 tx=0x7faf3800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.106+0000 7faf36a00640 1 -- 172.21.15.1:0/1933395712 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3800ce20 con 0x7faf40074cd0 2024-04-26T02:03:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.106+0000 7faf36a00640 1 -- 172.21.15.1:0/1933395712 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf38005020 con 0x7faf40074cd0 2024-04-26T02:03:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.106+0000 7faf3f200640 1 -- 172.21.15.1:0/1933395712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf401a7150 con 0x7faf40074cd0 2024-04-26T02:03:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.106+0000 7faf36a00640 1 -- 172.21.15.1:0/1933395712 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38010690 con 0x7faf40074cd0 2024-04-26T02:03:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.106+0000 7faf3f200640 1 -- 172.21.15.1:0/1933395712 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf401a7650 con 0x7faf40074cd0 2024-04-26T02:03:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.109+0000 7faf36a00640 1 -- 172.21.15.1:0/1933395712 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7faf380054d0 con 0x7faf40074cd0 2024-04-26T02:03:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.110+0000 7faf47fad740 1 Processor -- start 2024-04-26T02:03:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.110+0000 7faf47fad740 1 -- start start 2024-04-26T02:03:15.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.110+0000 7faf47fad740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df45809b0 0x563df4580d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.110+0000 7faf47fad740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df45812d0 0x563df458b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.110+0000 7faf47fad740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df458bb90 0x563df458df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.110+0000 7faf47fad740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563df44e6900 con 0x563df45812d0 2024-04-26T02:03:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.110+0000 7faf47fad740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563df448cef0 con 0x563df458bb90 2024-04-26T02:03:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.110+0000 7faf47fad740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563df44e28c0 con 0x563df45809b0 2024-04-26T02:03:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.111+0000 7faf3d400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df45809b0 0x563df4580d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.111+0000 7faf3d400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df45809b0 0x563df4580d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:37556/0 (socket says 172.21.15.1:37556) 2024-04-26T02:03:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.111+0000 7faf3d400640 1 -- 172.21.15.1:0/3212141663 learned_addr learned my addr 172.21.15.1:0/3212141663 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.111+0000 7faf3de00640 1 --2- 172.21.15.1:0/3212141663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df45812d0 0x563df458b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:15.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.111+0000 7faf3d400640 1 --2- 172.21.15.1:0/3212141663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df458bb90 0x563df458df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.111+0000 7faf3d400640 1 -- 172.21.15.1:0/3212141663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df458bb90 msgr2=0x563df458df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.111+0000 7faf3d400640 1 --2- 172.21.15.1:0/3212141663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df458bb90 0x563df458df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.111+0000 7faf3d400640 1 -- 172.21.15.1:0/3212141663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df45812d0 msgr2=0x563df458b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.111+0000 7faf3d400640 1 --2- 172.21.15.1:0/3212141663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df45812d0 0x563df458b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.111+0000 7faf3d400640 1 -- 172.21.15.1:0/3212141663 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563df41f9210 con 0x563df45809b0 2024-04-26T02:03:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.111+0000 7faf3d400640 1 --2- 172.21.15.1:0/3212141663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df45809b0 0x563df4580d90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009fd0 tx=0x7faf2c005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49d50c39dcd88919 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.112+0000 7faf36a00640 1 --2- 172.21.15.1:0/1933395712 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7faf240a3d30 0x7faf240a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.112+0000 7faf2b400640 1 -- 172.21.15.1:0/3212141663 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c004030 con 0x563df45809b0 2024-04-26T02:03:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.112+0000 7faf2b400640 1 -- 172.21.15.1:0/3212141663 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf2c00fc40 con 0x563df45809b0 2024-04-26T02:03:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.112+0000 7faf2b400640 1 -- 172.21.15.1:0/3212141663 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c003830 con 0x563df45809b0 2024-04-26T02:03:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.112+0000 7faf36a00640 1 -- 172.21.15.1:0/1933395712 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7faf380d0ed0 con 0x7faf40074cd0 2024-04-26T02:03:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.112+0000 7faf3de00640 1 --2- 172.21.15.1:0/1933395712 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7faf240a3d30 0x7faf240a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.113+0000 7faf47fad740 1 -- 172.21.15.1:0/3212141663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df45809b0 msgr2=0x563df4580d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.113+0000 7faf47fad740 1 --2- 172.21.15.1:0/3212141663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df45809b0 0x563df4580d90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009fd0 tx=0x7faf2c005850 comp rx=0 tx=0).stop 2024-04-26T02:03:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.113+0000 7faf3de00640 1 --2- 172.21.15.1:0/1933395712 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7faf240a3d30 0x7faf240a61f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faf30031440 tx=0x7faf300058d0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.113+0000 7faf47fad740 1 -- 172.21.15.1:0/3212141663 shutdown_connections 2024-04-26T02:03:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.113+0000 7faf47fad740 1 --2- 172.21.15.1:0/3212141663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df458bb90 0x563df458df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.113+0000 7faf47fad740 1 --2- 172.21.15.1:0/3212141663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df45812d0 0x563df458b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.113+0000 7faf47fad740 1 --2- 172.21.15.1:0/3212141663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df45809b0 0x563df4580d90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.113+0000 7faf47fad740 1 -- 172.21.15.1:0/3212141663 >> 172.21.15.1:0/3212141663 conn(0x563df457cec0 msgr2=0x563df457d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.113+0000 7faf47fad740 1 -- 172.21.15.1:0/3212141663 shutdown_connections 2024-04-26T02:03:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.113+0000 7faf47fad740 1 -- 172.21.15.1:0/3212141663 wait complete. 2024-04-26T02:03:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.114+0000 7faf47fad740 1 -- start start 2024-04-26T02:03:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.115+0000 7faf47fad740 1 Processor -- start 2024-04-26T02:03:15.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.115+0000 7faf47fad740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df45809b0 0x563df4588ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.115+0000 7faf47fad740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df45812d0 0x563df4589010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.115+0000 7faf47fad740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df458bb90 0x563df4589550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.115+0000 7faf47fad740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563df458e7c0 con 0x563df45812d0 2024-04-26T02:03:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.115+0000 7faf47fad740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563df44a0310 con 0x563df458bb90 2024-04-26T02:03:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.115+0000 7faf47fad740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563df458e940 con 0x563df45809b0 2024-04-26T02:03:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.115+0000 7faf47fad740 10 client.0 _ll_register_callbacks cb 0x563df446ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.116+0000 7faf3de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df45809b0 0x563df4588ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.116+0000 7faf3de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df45809b0 0x563df4588ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:37564/0 (socket says 172.21.15.1:37564) 2024-04-26T02:03:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.116+0000 7faf3de00640 1 -- 172.21.15.1:0/4255308637 learned_addr learned my addr 172.21.15.1:0/4255308637 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.116+0000 7faf3e800640 1 --2- 172.21.15.1:0/4255308637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df458bb90 0x563df4589550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.116+0000 7faf3d400640 1 --2- 172.21.15.1:0/4255308637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df45812d0 0x563df4589010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:15.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.116+0000 7faf3de00640 1 -- 172.21.15.1:0/4255308637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df458bb90 msgr2=0x563df4589550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.116+0000 7faf3de00640 1 --2- 172.21.15.1:0/4255308637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df458bb90 0x563df4589550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.116+0000 7faf3de00640 1 -- 172.21.15.1:0/4255308637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df45812d0 msgr2=0x563df4589010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.116+0000 7faf3de00640 1 --2- 172.21.15.1:0/4255308637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df45812d0 0x563df4589010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.116+0000 7faf3de00640 1 -- 172.21.15.1:0/4255308637 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563df42c5620 con 0x563df45809b0 2024-04-26T02:03:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.116+0000 7faf3e800640 1 --2- 172.21.15.1:0/4255308637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df458bb90 0x563df4589550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.116+0000 7faf3de00640 1 --2- 172.21.15.1:0/4255308637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df45809b0 0x563df4588ad0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faf30031720 tx=0x7faf30009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16663609c7331585 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.116+0000 7faf1fe00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T02:03:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.117+0000 7faf1fe00640 1 -- 172.21.15.1:0/4255308637 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3004c030 con 0x563df45809b0 2024-04-26T02:03:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.117+0000 7faf1fe00640 1 -- 172.21.15.1:0/4255308637 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7faf30004250 con 0x563df45809b0 2024-04-26T02:03:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.117+0000 7faf47fad740 1 -- 172.21.15.1:0/4255308637 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf2c00fc40 con 0x563df45809b0 2024-04-26T02:03:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.117+0000 7faf1fe00640 1 -- 172.21.15.1:0/4255308637 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7faf30035040 con 0x563df45809b0 2024-04-26T02:03:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.117+0000 7faf47fad740 10 client.26563 fetch_fsmap learned FSMap version 34 2024-04-26T02:03:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.117+0000 7faf47fad740 1 -- 172.21.15.1:0/4255308637 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563df459a9d0 con 0x563df45809b0 2024-04-26T02:03:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.118+0000 7faf1fe00640 1 -- 172.21.15.1:0/4255308637 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf2c00fc40 con 0x563df45809b0 2024-04-26T02:03:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.118+0000 7faf1fe00640 1 -- 172.21.15.1:0/4255308637 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7faf30048440 con 0x563df45809b0 2024-04-26T02:03:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.118+0000 7faf47fad740 10 client.26563 fetch_fsmap finished waiting for FSMap version 34 2024-04-26T02:03:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.118+0000 7faf47fad740 10 client.26563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:03:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.118+0000 7faf47fad740 20 client.26563 populate_metadata read hostname 'smithi001' 2024-04-26T02:03:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.118+0000 7faf47fad740 1 --2- 172.21.15.1:0/4255308637 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563df459bee0 0x563df459e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.118+0000 7faf47fad740 4 client.26563 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:03:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.118+0000 7faf47fad740 1 -- 172.21.15.1:0/4255308637 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563df44fa8b0 con 0x563df459bee0 2024-04-26T02:03:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.119+0000 7faf3d400640 1 --2- 172.21.15.1:0/4255308637 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563df459bee0 0x563df459e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.119+0000 7faf3d400640 1 --2- 172.21.15.1:0/4255308637 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563df459bee0 0x563df459e2f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a9b80a7a3cb40a3 server_cookie=20e2daa9177854e2 in_seq=0 out_seq=0 2024-04-26T02:03:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.119+0000 7faf1fe00640 10 client.26563 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.120+0000 7faf1fe00640 1 -- 172.21.15.1:0/4255308637 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563df44fa8b0 con 0x563df459bee0 2024-04-26T02:03:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.120+0000 7faf1fe00640 10 client.26563 handle_command_reply: tid=0 2024-04-26T02:03:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.120+0000 7faf47fad740 1 client.26563 shutdown 2024-04-26T02:03:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.120+0000 7faf47fad740 1 -- 172.21.15.1:0/4255308637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df45809b0 msgr2=0x563df4588ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.120+0000 7faf47fad740 1 --2- 172.21.15.1:0/4255308637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df45809b0 0x563df4588ad0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faf30031720 tx=0x7faf30009260 comp rx=0 tx=0).stop 2024-04-26T02:03:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.120+0000 7faf47fad740 1 -- 172.21.15.1:0/4255308637 shutdown_connections 2024-04-26T02:03:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.120+0000 7faf47fad740 1 --2- 172.21.15.1:0/4255308637 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563df459bee0 0x563df459e2f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.120+0000 7faf47fad740 1 --2- 172.21.15.1:0/4255308637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df458bb90 0x563df4589550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.120+0000 7faf47fad740 1 --2- 172.21.15.1:0/4255308637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df45812d0 0x563df4589010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.120+0000 7faf47fad740 1 --2- 172.21.15.1:0/4255308637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df45809b0 0x563df4588ad0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.120+0000 7faf1fe00640 0 client.26563 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.120+0000 7faf47fad740 1 -- 172.21.15.1:0/4255308637 >> 172.21.15.1:0/4255308637 conn(0x563df457cec0 msgr2=0x563df4582c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.120+0000 7faf47fad740 1 -- 172.21.15.1:0/4255308637 shutdown_connections 2024-04-26T02:03:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.121+0000 7faf47fad740 1 -- 172.21.15.1:0/4255308637 wait complete. 2024-04-26T02:03:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.121+0000 7faf47fad740 20 client.26563 trim_cache size 0 max 16384 2024-04-26T02:03:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.143+0000 7faf47fad740 1 Processor -- start 2024-04-26T02:03:15.142 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.143+0000 7faf47fad740 1 -- start start 2024-04-26T02:03:15.143 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.143+0000 7faf47fad740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df45809b0 0x563df4597680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.143+0000 7faf47fad740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df45812d0 0x563df45a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.144+0000 7faf47fad740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df458bb90 0x563df45a4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.144+0000 7faf47fad740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faf401b2590 con 0x563df45809b0 2024-04-26T02:03:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.144+0000 7faf47fad740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faf401afe60 con 0x563df45812d0 2024-04-26T02:03:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.144+0000 7faf47fad740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faf401adcd0 con 0x563df458bb90 2024-04-26T02:03:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.144+0000 7faf3d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df45812d0 0x563df45a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.144+0000 7faf3d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df45812d0 0x563df45a3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:59100/0 (socket says 172.21.15.1:59100) 2024-04-26T02:03:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.144+0000 7faf3d400640 1 -- 172.21.15.1:0/3028929995 learned_addr learned my addr 172.21.15.1:0/3028929995 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.144+0000 7faf3e800640 1 --2- 172.21.15.1:0/3028929995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df458bb90 0x563df45a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.144+0000 7faf3de00640 1 --2- 172.21.15.1:0/3028929995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df45809b0 0x563df4597680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.144+0000 7faf3d400640 1 -- 172.21.15.1:0/3028929995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df458bb90 msgr2=0x563df45a4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.144+0000 7faf3d400640 1 --2- 172.21.15.1:0/3028929995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df458bb90 0x563df45a4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.144+0000 7faf3d400640 1 -- 172.21.15.1:0/3028929995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df45809b0 msgr2=0x563df4597680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.144+0000 7faf3d400640 1 --2- 172.21.15.1:0/3028929995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df45809b0 0x563df4597680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.144+0000 7faf3d400640 1 -- 172.21.15.1:0/3028929995 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563df4588360 con 0x563df45812d0 2024-04-26T02:03:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.144+0000 7faf3e800640 1 --2- 172.21.15.1:0/3028929995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df458bb90 0x563df45a4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.145+0000 7faf3d400640 1 --2- 172.21.15.1:0/3028929995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df45812d0 0x563df45a3c00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009fd0 tx=0x7faf2c00f9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b6993ec53392269 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.145+0000 7faf2aa00640 1 -- 172.21.15.1:0/3028929995 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c00fa40 con 0x563df45812d0 2024-04-26T02:03:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.145+0000 7faf2aa00640 1 -- 172.21.15.1:0/3028929995 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf2c005bf0 con 0x563df45812d0 2024-04-26T02:03:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.145+0000 7faf2aa00640 1 -- 172.21.15.1:0/3028929995 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c005380 con 0x563df45812d0 2024-04-26T02:03:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.146+0000 7faf47fad740 1 -- 172.21.15.1:0/3028929995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df45812d0 msgr2=0x563df45a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.146+0000 7faf47fad740 1 --2- 172.21.15.1:0/3028929995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df45812d0 0x563df45a3c00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009fd0 tx=0x7faf2c00f9d0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.146+0000 7faf47fad740 1 -- 172.21.15.1:0/3028929995 shutdown_connections 2024-04-26T02:03:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.146+0000 7faf47fad740 1 --2- 172.21.15.1:0/3028929995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df458bb90 0x563df45a4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.146+0000 7faf47fad740 1 --2- 172.21.15.1:0/3028929995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df45812d0 0x563df45a3c00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.146+0000 7faf47fad740 1 --2- 172.21.15.1:0/3028929995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df45809b0 0x563df4597680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.146+0000 7faf47fad740 1 -- 172.21.15.1:0/3028929995 >> 172.21.15.1:0/3028929995 conn(0x563df457cec0 msgr2=0x563df457d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.146+0000 7faf47fad740 1 -- 172.21.15.1:0/3028929995 shutdown_connections 2024-04-26T02:03:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.146+0000 7faf47fad740 1 -- 172.21.15.1:0/3028929995 wait complete. 2024-04-26T02:03:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.147+0000 7faf47fad740 1 -- start start 2024-04-26T02:03:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.147+0000 7faf47fad740 1 Processor -- start 2024-04-26T02:03:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.147+0000 7faf47fad740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df45809b0 0x563df4593f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.147+0000 7faf47fad740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df45812d0 0x563df45912e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.147+0000 7faf47fad740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df458bb90 0x563df4591820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.147+0000 7faf47fad740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faf401ae320 con 0x563df45809b0 2024-04-26T02:03:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.147+0000 7faf47fad740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faf401af790 con 0x563df45812d0 2024-04-26T02:03:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.147+0000 7faf47fad740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faf401ae9f0 con 0x563df458bb90 2024-04-26T02:03:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.147+0000 7faf47fad740 10 client.0 _ll_register_callbacks cb 0x7faf30032e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.148+0000 7faf3d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df45812d0 0x563df45912e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.148+0000 7faf3e800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df458bb90 0x563df4591820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.148+0000 7faf3d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df45812d0 0x563df45912e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:59110/0 (socket says 172.21.15.1:59110) 2024-04-26T02:03:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.148+0000 7faf3d400640 1 -- 172.21.15.1:0/2576338009 learned_addr learned my addr 172.21.15.1:0/2576338009 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.148+0000 7faf3de00640 1 --2- 172.21.15.1:0/2576338009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df45809b0 0x563df4593f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.148+0000 7faf3d400640 1 -- 172.21.15.1:0/2576338009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df458bb90 msgr2=0x563df4591820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.148+0000 7faf3d400640 1 --2- 172.21.15.1:0/2576338009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df458bb90 0x563df4591820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.148+0000 7faf3d400640 1 -- 172.21.15.1:0/2576338009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df45809b0 msgr2=0x563df4593f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.148+0000 7faf3d400640 1 --2- 172.21.15.1:0/2576338009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df45809b0 0x563df4593f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.148+0000 7faf3d400640 1 -- 172.21.15.1:0/2576338009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563df4592050 con 0x563df45812d0 2024-04-26T02:03:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.148+0000 7faf3de00640 1 --2- 172.21.15.1:0/2576338009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df45809b0 0x563df4593f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:03:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.148+0000 7faf3e800640 1 --2- 172.21.15.1:0/2576338009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df458bb90 0x563df4591820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.148+0000 7faf3d400640 1 --2- 172.21.15.1:0/2576338009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df45812d0 0x563df45912e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00fe20 tx=0x7faf2c009d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a89c83145453d9ec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.148+0000 7faf1fe00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:03:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.149+0000 7faf1fe00640 1 -- 172.21.15.1:0/2576338009 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c00fa40 con 0x563df45812d0 2024-04-26T02:03:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.149+0000 7faf47fad740 1 -- 172.21.15.1:0/2576338009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563df44f4320 con 0x563df45812d0 2024-04-26T02:03:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.149+0000 7faf1fe00640 1 -- 172.21.15.1:0/2576338009 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7faf2c005100 con 0x563df45812d0 2024-04-26T02:03:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.149+0000 7faf1fe00640 1 -- 172.21.15.1:0/2576338009 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7faf2c03b050 con 0x563df45812d0 2024-04-26T02:03:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.149+0000 7faf47fad740 10 client.26558 fetch_fsmap learned FSMap version 34 2024-04-26T02:03:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.149+0000 7faf47fad740 1 -- 172.21.15.1:0/2576338009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563df4602460 con 0x563df45812d0 2024-04-26T02:03:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.149+0000 7faf1fe00640 1 -- 172.21.15.1:0/2576338009 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563df44f4320 con 0x563df45812d0 2024-04-26T02:03:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.149+0000 7faf1fe00640 1 -- 172.21.15.1:0/2576338009 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7faf2c0048e0 con 0x563df45812d0 2024-04-26T02:03:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.149+0000 7faf47fad740 10 client.26558 fetch_fsmap finished waiting for FSMap version 34 2024-04-26T02:03:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.150+0000 7faf47fad740 10 client.26558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:03:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.150+0000 7faf47fad740 20 client.26558 populate_metadata read hostname 'smithi001' 2024-04-26T02:03:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.150+0000 7faf47fad740 1 --2- 172.21.15.1:0/2576338009 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563df4602920 0x563df4604d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.150+0000 7faf47fad740 4 client.26558 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:03:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.150+0000 7faf47fad740 1 -- 172.21.15.1:0/2576338009 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563df44ee190 con 0x563df4602920 2024-04-26T02:03:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.150+0000 7faf3de00640 1 --2- 172.21.15.1:0/2576338009 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563df4602920 0x563df4604d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.150+0000 7faf3de00640 1 --2- 172.21.15.1:0/2576338009 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563df4602920 0x563df4604d10 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b905b81468dc7e67 server_cookie=3394e6d59b2ab26c in_seq=0 out_seq=0 2024-04-26T02:03:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.150+0000 7faf1fe00640 10 client.26558 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.305+0000 7faf1fe00640 1 -- 172.21.15.1:0/2576338009 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563df44ee190 con 0x563df4602920 2024-04-26T02:03:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.305+0000 7faf1fe00640 10 client.26558 handle_command_reply: tid=0 2024-04-26T02:03:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.305+0000 7faf47fad740 1 client.26558 shutdown 2024-04-26T02:03:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.306+0000 7faf47fad740 1 -- 172.21.15.1:0/2576338009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df45812d0 msgr2=0x563df45912e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.306+0000 7faf47fad740 1 --2- 172.21.15.1:0/2576338009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df45812d0 0x563df45912e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00fe20 tx=0x7faf2c009d90 comp rx=0 tx=0).stop 2024-04-26T02:03:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.306+0000 7faf47fad740 1 -- 172.21.15.1:0/2576338009 shutdown_connections 2024-04-26T02:03:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.306+0000 7faf47fad740 1 --2- 172.21.15.1:0/2576338009 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x563df4602920 0x563df4604d10 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.306+0000 7faf47fad740 1 --2- 172.21.15.1:0/2576338009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563df458bb90 0x563df4591820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.306+0000 7faf47fad740 1 --2- 172.21.15.1:0/2576338009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563df45812d0 0x563df45912e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.306+0000 7faf47fad740 1 --2- 172.21.15.1:0/2576338009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df45809b0 0x563df4593f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.306+0000 7faf1fe00640 0 client.26558 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.306+0000 7faf47fad740 1 -- 172.21.15.1:0/2576338009 >> 172.21.15.1:0/2576338009 conn(0x563df457cec0 msgr2=0x563df45823b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.307+0000 7faf47fad740 1 -- 172.21.15.1:0/2576338009 shutdown_connections 2024-04-26T02:03:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.307+0000 7faf47fad740 1 -- 172.21.15.1:0/2576338009 wait complete. 2024-04-26T02:03:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.307+0000 7faf47fad740 20 client.26558 trim_cache size 0 max 16384 2024-04-26T02:03:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T02:03:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T02:03:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T02:03:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T02:03:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.308+0000 7faf3f200640 1 -- 172.21.15.1:0/1933395712 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7faf240a3d30 msgr2=0x7faf240a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.308+0000 7faf3f200640 1 --2- 172.21.15.1:0/1933395712 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7faf240a3d30 0x7faf240a61f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faf30031440 tx=0x7faf300058d0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.309+0000 7faf3f200640 1 -- 172.21.15.1:0/1933395712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf40074cd0 msgr2=0x7faf401a6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.309+0000 7faf3f200640 1 --2- 172.21.15.1:0/1933395712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf40074cd0 0x7faf401a6760 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faf38009fd0 tx=0x7faf3800ef10 comp rx=0 tx=0).stop 2024-04-26T02:03:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.309+0000 7faf3f200640 1 -- 172.21.15.1:0/1933395712 shutdown_connections 2024-04-26T02:03:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.309+0000 7faf3f200640 1 --2- 172.21.15.1:0/1933395712 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7faf240a3d30 0x7faf240a61f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.309+0000 7faf3f200640 1 --2- 172.21.15.1:0/1933395712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf40074cd0 0x7faf401a6760 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.309+0000 7faf3f200640 1 --2- 172.21.15.1:0/1933395712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf401070b0 0x7faf40074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.309+0000 7faf3f200640 1 --2- 172.21.15.1:0/1933395712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf40101fe0 0x7faf40074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.309+0000 7faf3f200640 1 -- 172.21.15.1:0/1933395712 >> 172.21.15.1:0/1933395712 conn(0x7faf400fd860 msgr2=0x7faf400fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.309+0000 7faf3f200640 1 -- 172.21.15.1:0/1933395712 shutdown_connections 2024-04-26T02:03:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:15.309+0000 7faf3f200640 1 -- 172.21.15.1:0/1933395712 wait complete. 2024-04-26T02:03:15.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:15 smithi001 ceph-mon[27822]: osdmap e107: 12 total, 12 up, 12 in 2024-04-26T02:03:15.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:15 smithi173 ceph-mon[32553]: osdmap e107: 12 total, 12 up, 12 in 2024-04-26T02:03:15.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:15 smithi097 ceph-mon[32853]: osdmap e107: 12 total, 12 up, 12 in 2024-04-26T02:03:16.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:16 smithi173 ceph-mon[32553]: pgmap v365: 97 pgs: 97 active+clean; 182 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.2 MiB/s wr, 402 op/s 2024-04-26T02:03:16.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:16 smithi001 ceph-mon[27822]: pgmap v365: 97 pgs: 97 active+clean; 182 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.2 MiB/s wr, 402 op/s 2024-04-26T02:03:16.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:16 smithi097 ceph-mon[32853]: pgmap v365: 97 pgs: 97 active+clean; 182 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.2 MiB/s wr, 402 op/s 2024-04-26T02:03:18.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:18 smithi173 ceph-mon[32553]: pgmap v366: 97 pgs: 97 active+clean; 182 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.2 MiB/s wr, 396 op/s 2024-04-26T02:03:18.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:18 smithi001 ceph-mon[27822]: pgmap v366: 97 pgs: 97 active+clean; 182 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.2 MiB/s wr, 396 op/s 2024-04-26T02:03:18.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:18 smithi097 ceph-mon[32853]: pgmap v366: 97 pgs: 97 active+clean; 182 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.2 MiB/s wr, 396 op/s 2024-04-26T02:03:20.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:20 smithi173 ceph-mon[32553]: pgmap v367: 97 pgs: 97 active+clean; 182 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.2 MiB/s wr, 396 op/s 2024-04-26T02:03:20.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:20 smithi001 ceph-mon[27822]: pgmap v367: 97 pgs: 97 active+clean; 182 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.2 MiB/s wr, 396 op/s 2024-04-26T02:03:20.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:20 smithi097 ceph-mon[32853]: pgmap v367: 97 pgs: 97 active+clean; 182 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.2 MiB/s wr, 396 op/s 2024-04-26T02:03:22.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:22 smithi173 ceph-mon[32553]: pgmap v368: 97 pgs: 97 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 268 op/s 2024-04-26T02:03:22.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:22 smithi001 ceph-mon[27822]: pgmap v368: 97 pgs: 97 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 268 op/s 2024-04-26T02:03:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:22 smithi097 ceph-mon[32853]: pgmap v368: 97 pgs: 97 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 268 op/s 2024-04-26T02:03:23.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:23 smithi001 ceph-mon[27822]: pgmap v369: 97 pgs: 97 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 268 op/s 2024-04-26T02:03:23.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:23 smithi097 ceph-mon[32853]: pgmap v369: 97 pgs: 97 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 268 op/s 2024-04-26T02:03:23.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:23 smithi173 ceph-mon[32553]: pgmap v369: 97 pgs: 97 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 268 op/s 2024-04-26T02:03:26.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:26 smithi173 ceph-mon[32553]: pgmap v370: 97 pgs: 97 active+clean; 186 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 397 op/s 2024-04-26T02:03:26.334 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:26 smithi001 ceph-mon[27822]: pgmap v370: 97 pgs: 97 active+clean; 186 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 397 op/s 2024-04-26T02:03:26.389 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-26T02:03:26.389 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.25 2024-04-26T02:03:26.390 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:03:26.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:26 smithi097 ceph-mon[32853]: pgmap v370: 97 pgs: 97 active+clean; 186 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 397 op/s 2024-04-26T02:03:26.935 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:03:28.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:28 smithi173 ceph-mon[32553]: pgmap v371: 97 pgs: 97 active+clean; 186 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 203 op/s 2024-04-26T02:03:28.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:28 smithi173 ceph-mon[32553]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T02:03:28.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:28 smithi001 ceph-mon[27822]: pgmap v371: 97 pgs: 97 active+clean; 186 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 203 op/s 2024-04-26T02:03:28.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:28 smithi001 ceph-mon[27822]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T02:03:28.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:28 smithi097 ceph-mon[32853]: pgmap v371: 97 pgs: 97 active+clean; 186 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 203 op/s 2024-04-26T02:03:28.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:28 smithi097 ceph-mon[32853]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T02:03:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.498+0000 7f6093200640 1 -- 172.21.15.1:0/3194369412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60941041d0 msgr2=0x7f60941045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.498+0000 7f6093200640 1 --2- 172.21.15.1:0/3194369412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60941041d0 0x7f60941045b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6084009900 tx=0x7f608402a3f0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.498+0000 7f6093200640 1 -- 172.21.15.1:0/3194369412 shutdown_connections 2024-04-26T02:03:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.499+0000 7f6093200640 1 --2- 172.21.15.1:0/3194369412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60941092a0 0x7f6094110e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.499+0000 7f6093200640 1 --2- 172.21.15.1:0/3194369412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6094104b80 0x7f6094108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.499+0000 7f6093200640 1 --2- 172.21.15.1:0/3194369412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60941041d0 0x7f60941045b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.499+0000 7f6093200640 1 -- 172.21.15.1:0/3194369412 >> 172.21.15.1:0/3194369412 conn(0x7f60940ffa30 msgr2=0x7f6094101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.499+0000 7f6093200640 1 -- 172.21.15.1:0/3194369412 shutdown_connections 2024-04-26T02:03:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.499+0000 7f6093200640 1 -- 172.21.15.1:0/3194369412 wait complete. 2024-04-26T02:03:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.500+0000 7f6093200640 1 Processor -- start 2024-04-26T02:03:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.500+0000 7f6093200640 1 -- start start 2024-04-26T02:03:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.500+0000 7f6093200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60941041d0 0x7f60941a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.500+0000 7f6093200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60941092a0 0x7f60941a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.500+0000 7f6093200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f609419a5d0 0x7f609419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.500+0000 7f6093200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6094114250 con 0x7f60941041d0 2024-04-26T02:03:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.500+0000 7f6093200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f60941140d0 con 0x7f609419a5d0 2024-04-26T02:03:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.500+0000 7f6093200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f60941143d0 con 0x7f60941092a0 2024-04-26T02:03:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.501+0000 7f608be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60941092a0 0x7f60941a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.501+0000 7f608be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60941092a0 0x7f60941a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:59828/0 (socket says 172.21.15.1:59828) 2024-04-26T02:03:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.501+0000 7f608be00640 1 -- 172.21.15.1:0/3740481767 learned_addr learned my addr 172.21.15.1:0/3740481767 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.501+0000 7f6091e00640 1 --2- 172.21.15.1:0/3740481767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60941041d0 0x7f60941a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.501+0000 7f6092800640 1 --2- 172.21.15.1:0/3740481767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f609419a5d0 0x7f609419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.501+0000 7f608be00640 1 -- 172.21.15.1:0/3740481767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f609419a5d0 msgr2=0x7f609419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.501+0000 7f608be00640 1 --2- 172.21.15.1:0/3740481767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f609419a5d0 0x7f609419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.501+0000 7f608be00640 1 -- 172.21.15.1:0/3740481767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60941041d0 msgr2=0x7f60941a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.501+0000 7f608be00640 1 --2- 172.21.15.1:0/3740481767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60941041d0 0x7f60941a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.501+0000 7f608be00640 1 -- 172.21.15.1:0/3740481767 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609419b0f0 con 0x7f60941092a0 2024-04-26T02:03:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.502+0000 7f608be00640 1 --2- 172.21.15.1:0/3740481767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60941092a0 0x7f60941a0960 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f607c00e9f0 tx=0x7f607c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.502+0000 7f608aa00640 1 -- 172.21.15.1:0/3740481767 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c00ccc0 con 0x7f60941092a0 2024-04-26T02:03:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.502+0000 7f608aa00640 1 -- 172.21.15.1:0/3740481767 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f607c0054d0 con 0x7f60941092a0 2024-04-26T02:03:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.502+0000 7f608aa00640 1 -- 172.21.15.1:0/3740481767 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c010430 con 0x7f60941092a0 2024-04-26T02:03:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.502+0000 7f6093200640 1 -- 172.21.15.1:0/3740481767 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f609419b3e0 con 0x7f60941092a0 2024-04-26T02:03:28.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.502+0000 7f6093200640 1 -- 172.21.15.1:0/3740481767 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60941a75e0 con 0x7f60941092a0 2024-04-26T02:03:28.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.505+0000 7f609bc52740 1 Processor -- start 2024-04-26T02:03:28.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.505+0000 7f608aa00640 1 -- 172.21.15.1:0/3740481767 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f607c0046c0 con 0x7f60941092a0 2024-04-26T02:03:28.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.505+0000 7f609bc52740 1 -- start start 2024-04-26T02:03:28.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.506+0000 7f609bc52740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557e6d6859b0 0x557e6d685d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:28.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.506+0000 7f609bc52740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d6862d0 0x557e6d690650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:28.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.506+0000 7f609bc52740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d690b90 0x557e6d692f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:28.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.506+0000 7f609bc52740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557e6d5eb900 con 0x557e6d690b90 2024-04-26T02:03:28.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.506+0000 7f609bc52740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557e6d591ef0 con 0x557e6d6862d0 2024-04-26T02:03:28.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.506+0000 7f609bc52740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557e6d5e78c0 con 0x557e6d6859b0 2024-04-26T02:03:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.506+0000 7f6091e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d6862d0 0x557e6d690650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.506+0000 7f6092800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557e6d6859b0 0x557e6d685d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.506+0000 7f6092800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557e6d6859b0 0x557e6d685d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:59838/0 (socket says 172.21.15.1:59838) 2024-04-26T02:03:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.506+0000 7f6091e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d6862d0 0x557e6d690650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:40414/0 (socket says 172.21.15.1:40414) 2024-04-26T02:03:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.506+0000 7f6092800640 1 -- 172.21.15.1:0/597556871 learned_addr learned my addr 172.21.15.1:0/597556871 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.506+0000 7f608be00640 1 --2- 172.21.15.1:0/597556871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d690b90 0x557e6d692f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.507+0000 7f6091e00640 1 -- 172.21.15.1:0/597556871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557e6d6859b0 msgr2=0x557e6d685d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.507+0000 7f6091e00640 1 --2- 172.21.15.1:0/597556871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557e6d6859b0 0x557e6d685d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.507+0000 7f6091e00640 1 -- 172.21.15.1:0/597556871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d690b90 msgr2=0x557e6d692f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.507+0000 7f6091e00640 1 --2- 172.21.15.1:0/597556871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d690b90 0x557e6d692f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.507+0000 7f6091e00640 1 -- 172.21.15.1:0/597556871 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e6d2fe210 con 0x557e6d6862d0 2024-04-26T02:03:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.507+0000 7f608be00640 1 --2- 172.21.15.1:0/597556871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d690b90 0x557e6d692f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:03:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.507+0000 7f6091e00640 1 --2- 172.21.15.1:0/597556871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d6862d0 0x557e6d690650 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f608402a9e0 tx=0x7f6084005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65ec951d86b2e3e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.507+0000 7f608aa00640 1 --2- 172.21.15.1:0/3740481767 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f60780a3d80 0x7f60780a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.508+0000 7f6083400640 1 -- 172.21.15.1:0/597556871 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6084035040 con 0x557e6d6862d0 2024-04-26T02:03:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.508+0000 7f608aa00640 1 -- 172.21.15.1:0/3740481767 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f607c0d0ba0 con 0x7f60941092a0 2024-04-26T02:03:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.508+0000 7f6083400640 1 -- 172.21.15.1:0/597556871 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60840054f0 con 0x557e6d6862d0 2024-04-26T02:03:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.508+0000 7f6083400640 1 -- 172.21.15.1:0/597556871 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60840035f0 con 0x557e6d6862d0 2024-04-26T02:03:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.508+0000 7f6092800640 1 --2- 172.21.15.1:0/3740481767 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f60780a3d80 0x7f60780a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.509+0000 7f6092800640 1 --2- 172.21.15.1:0/3740481767 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f60780a3d80 0x7f60780a6240 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f608c011630 tx=0x7f608c009500 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.509+0000 7f609bc52740 1 -- 172.21.15.1:0/597556871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d6862d0 msgr2=0x557e6d690650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.509+0000 7f609bc52740 1 --2- 172.21.15.1:0/597556871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d6862d0 0x557e6d690650 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f608402a9e0 tx=0x7f6084005c00 comp rx=0 tx=0).stop 2024-04-26T02:03:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.509+0000 7f609bc52740 1 -- 172.21.15.1:0/597556871 shutdown_connections 2024-04-26T02:03:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.509+0000 7f609bc52740 1 --2- 172.21.15.1:0/597556871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d690b90 0x557e6d692f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.509+0000 7f609bc52740 1 --2- 172.21.15.1:0/597556871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d6862d0 0x557e6d690650 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.509+0000 7f609bc52740 1 --2- 172.21.15.1:0/597556871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557e6d6859b0 0x557e6d685d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.509+0000 7f609bc52740 1 -- 172.21.15.1:0/597556871 >> 172.21.15.1:0/597556871 conn(0x557e6d681ec0 msgr2=0x557e6d6822a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.509+0000 7f609bc52740 1 -- 172.21.15.1:0/597556871 shutdown_connections 2024-04-26T02:03:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.509+0000 7f609bc52740 1 -- 172.21.15.1:0/597556871 wait complete. 2024-04-26T02:03:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.509+0000 7f609bc52740 1 -- start start 2024-04-26T02:03:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.510+0000 7f609bc52740 1 Processor -- start 2024-04-26T02:03:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.510+0000 7f609bc52740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d6859b0 0x557e6d68dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.510+0000 7f609bc52740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d6862d0 0x557e6d68e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.510+0000 7f609bc52740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557e6d690b90 0x557e6d68e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.510+0000 7f609bc52740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557e6d693760 con 0x557e6d6859b0 2024-04-26T02:03:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.510+0000 7f609bc52740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557e6d5a5310 con 0x557e6d6862d0 2024-04-26T02:03:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.510+0000 7f609bc52740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x557e6d6938e0 con 0x557e6d690b90 2024-04-26T02:03:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.510+0000 7f609bc52740 10 client.0 _ll_register_callbacks cb 0x557e6d573cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.510+0000 7f608be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d6862d0 0x557e6d68e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.510+0000 7f6092800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557e6d690b90 0x557e6d68e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.510+0000 7f608be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d6862d0 0x557e6d68e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:40420/0 (socket says 172.21.15.1:40420) 2024-04-26T02:03:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.510+0000 7f608be00640 1 -- 172.21.15.1:0/614621824 learned_addr learned my addr 172.21.15.1:0/614621824 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.510+0000 7f6091e00640 1 --2- 172.21.15.1:0/614621824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d6859b0 0x557e6d68dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.511+0000 7f608be00640 1 -- 172.21.15.1:0/614621824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557e6d690b90 msgr2=0x557e6d68e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.511+0000 7f608be00640 1 --2- 172.21.15.1:0/614621824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557e6d690b90 0x557e6d68e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.511+0000 7f608be00640 1 -- 172.21.15.1:0/614621824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d6859b0 msgr2=0x557e6d68dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.511+0000 7f608be00640 1 --2- 172.21.15.1:0/614621824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d6859b0 0x557e6d68dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.511+0000 7f608be00640 1 -- 172.21.15.1:0/614621824 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e6d3ca620 con 0x557e6d6862d0 2024-04-26T02:03:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.511+0000 7f608be00640 1 --2- 172.21.15.1:0/614621824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d6862d0 0x557e6d68e150 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f607c0ce2c0 tx=0x7f607c0d4430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aed0ce55620f12d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.511+0000 7f6077e00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:03:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.511+0000 7f6077e00640 1 -- 172.21.15.1:0/614621824 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c0ce6e0 con 0x557e6d6862d0 2024-04-26T02:03:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.511+0000 7f6077e00640 1 -- 172.21.15.1:0/614621824 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f607c0a5290 con 0x557e6d6862d0 2024-04-26T02:03:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.511+0000 7f609bc52740 1 -- 172.21.15.1:0/614621824 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60840054f0 con 0x557e6d6862d0 2024-04-26T02:03:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.511+0000 7f6077e00640 1 -- 172.21.15.1:0/614621824 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f607c09d1b0 con 0x557e6d6862d0 2024-04-26T02:03:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.511+0000 7f609bc52740 10 client.26570 fetch_fsmap learned FSMap version 35 2024-04-26T02:03:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.511+0000 7f609bc52740 1 -- 172.21.15.1:0/614621824 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557e6d69fb60 con 0x557e6d6862d0 2024-04-26T02:03:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.512+0000 7f6077e00640 1 -- 172.21.15.1:0/614621824 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60840054f0 con 0x557e6d6862d0 2024-04-26T02:03:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.512+0000 7f6077e00640 1 -- 172.21.15.1:0/614621824 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 35) ==== 8342+0+0 (secure 0 0 0) 0x7f607c005000 con 0x557e6d6862d0 2024-04-26T02:03:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.512+0000 7f609bc52740 10 client.26570 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:03:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.512+0000 7f609bc52740 10 client.26570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:03:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.512+0000 7f609bc52740 20 client.26570 populate_metadata read hostname 'smithi001' 2024-04-26T02:03:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.512+0000 7f609bc52740 1 --2- 172.21.15.1:0/614621824 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x557e6d6a1020 0x557e6d6a3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.512+0000 7f609bc52740 4 client.26570 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:03:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.512+0000 7f609bc52740 1 -- 172.21.15.1:0/614621824 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e6d5ff8b0 con 0x557e6d6a1020 2024-04-26T02:03:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.512+0000 7f6091e00640 1 --2- 172.21.15.1:0/614621824 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x557e6d6a1020 0x557e6d6a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.513+0000 7f6091e00640 1 --2- 172.21.15.1:0/614621824 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x557e6d6a1020 0x557e6d6a3410 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8409f999459dc435 server_cookie=de9efb7634a5c899 in_seq=0 out_seq=0 2024-04-26T02:03:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.513+0000 7f6077e00640 10 client.26570 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.517+0000 7f6077e00640 1 -- 172.21.15.1:0/614621824 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e6d5ff8b0 con 0x557e6d6a1020 2024-04-26T02:03:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.517+0000 7f6077e00640 10 client.26570 handle_command_reply: tid=0 2024-04-26T02:03:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.517+0000 7f609bc52740 1 client.26570 shutdown 2024-04-26T02:03:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.517+0000 7f609bc52740 1 -- 172.21.15.1:0/614621824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d6862d0 msgr2=0x557e6d68e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.517+0000 7f609bc52740 1 --2- 172.21.15.1:0/614621824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d6862d0 0x557e6d68e150 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f607c0ce2c0 tx=0x7f607c0d4430 comp rx=0 tx=0).stop 2024-04-26T02:03:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.517+0000 7f609bc52740 1 -- 172.21.15.1:0/614621824 shutdown_connections 2024-04-26T02:03:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.517+0000 7f609bc52740 1 --2- 172.21.15.1:0/614621824 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x557e6d6a1020 0x557e6d6a3410 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.517+0000 7f6077e00640 0 client.26570 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.517+0000 7f609bc52740 1 --2- 172.21.15.1:0/614621824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557e6d690b90 0x557e6d68e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.517+0000 7f609bc52740 1 --2- 172.21.15.1:0/614621824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d6862d0 0x557e6d68e150 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.517+0000 7f609bc52740 1 --2- 172.21.15.1:0/614621824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d6859b0 0x557e6d68dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.517+0000 7f609bc52740 1 -- 172.21.15.1:0/614621824 >> 172.21.15.1:0/614621824 conn(0x557e6d681ec0 msgr2=0x557e6d687bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.517+0000 7f609bc52740 1 -- 172.21.15.1:0/614621824 shutdown_connections 2024-04-26T02:03:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.517+0000 7f609bc52740 1 -- 172.21.15.1:0/614621824 wait complete. 2024-04-26T02:03:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.517+0000 7f609bc52740 20 client.26570 trim_cache size 0 max 16384 2024-04-26T02:03:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.536+0000 7f609bc52740 1 Processor -- start 2024-04-26T02:03:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.536+0000 7f609bc52740 1 -- start start 2024-04-26T02:03:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.537+0000 7f609bc52740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557e6d6859b0 0x557e6d6817a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.537+0000 7f609bc52740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d6862d0 0x557e6d688370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.537+0000 7f609bc52740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d690b90 0x557e6d6a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.537+0000 7f609bc52740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f60941b25f0 con 0x557e6d690b90 2024-04-26T02:03:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.537+0000 7f609bc52740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f60941afe90 con 0x557e6d6862d0 2024-04-26T02:03:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.537+0000 7f609bc52740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f60941add00 con 0x557e6d6859b0 2024-04-26T02:03:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.537+0000 7f608be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d6862d0 0x557e6d688370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.537+0000 7f6092800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d690b90 0x557e6d6a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.537+0000 7f608be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d6862d0 0x557e6d688370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:40434/0 (socket says 172.21.15.1:40434) 2024-04-26T02:03:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.537+0000 7f608be00640 1 -- 172.21.15.1:0/2233298099 learned_addr learned my addr 172.21.15.1:0/2233298099 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.537+0000 7f6092800640 1 --2- 172.21.15.1:0/2233298099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d690b90 0x557e6d6a8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38210/0 (socket says 172.21.15.1:38210) 2024-04-26T02:03:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.537+0000 7f6091e00640 1 --2- 172.21.15.1:0/2233298099 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557e6d6859b0 0x557e6d6817a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.537+0000 7f608be00640 1 -- 172.21.15.1:0/2233298099 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557e6d6859b0 msgr2=0x557e6d6817a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.537+0000 7f608be00640 1 --2- 172.21.15.1:0/2233298099 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557e6d6859b0 0x557e6d6817a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.537+0000 7f608be00640 1 -- 172.21.15.1:0/2233298099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d690b90 msgr2=0x557e6d6a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.537+0000 7f608be00640 1 --2- 172.21.15.1:0/2233298099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d690b90 0x557e6d6a8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.538+0000 7f608be00640 1 -- 172.21.15.1:0/2233298099 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e6d68c160 con 0x557e6d6862d0 2024-04-26T02:03:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.538+0000 7f608be00640 1 --2- 172.21.15.1:0/2233298099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d6862d0 0x557e6d688370 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f607c09ca40 tx=0x7f607c005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dba230877bbacf72 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.538+0000 7f6082a00640 1 -- 172.21.15.1:0/2233298099 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c09c410 con 0x557e6d6862d0 2024-04-26T02:03:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.538+0000 7f6082a00640 1 -- 172.21.15.1:0/2233298099 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f607c0d5040 con 0x557e6d6862d0 2024-04-26T02:03:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.538+0000 7f6082a00640 1 -- 172.21.15.1:0/2233298099 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c0a5720 con 0x557e6d6862d0 2024-04-26T02:03:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.539+0000 7f609bc52740 1 -- 172.21.15.1:0/2233298099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d6862d0 msgr2=0x557e6d688370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.539+0000 7f609bc52740 1 --2- 172.21.15.1:0/2233298099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d6862d0 0x557e6d688370 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f607c09ca40 tx=0x7f607c005b40 comp rx=0 tx=0).stop 2024-04-26T02:03:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.539+0000 7f609bc52740 1 -- 172.21.15.1:0/2233298099 shutdown_connections 2024-04-26T02:03:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.539+0000 7f609bc52740 1 --2- 172.21.15.1:0/2233298099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d690b90 0x557e6d6a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.539+0000 7f609bc52740 1 --2- 172.21.15.1:0/2233298099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d6862d0 0x557e6d688370 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.539+0000 7f609bc52740 1 --2- 172.21.15.1:0/2233298099 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557e6d6859b0 0x557e6d6817a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.539+0000 7f609bc52740 1 -- 172.21.15.1:0/2233298099 >> 172.21.15.1:0/2233298099 conn(0x557e6d681ec0 msgr2=0x557e6d68e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.539+0000 7f609bc52740 1 -- 172.21.15.1:0/2233298099 shutdown_connections 2024-04-26T02:03:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.539+0000 7f609bc52740 1 -- 172.21.15.1:0/2233298099 wait complete. 2024-04-26T02:03:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.539+0000 7f609bc52740 1 -- start start 2024-04-26T02:03:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.539+0000 7f609bc52740 1 Processor -- start 2024-04-26T02:03:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.540+0000 7f609bc52740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557e6d6859b0 0x557e6d698c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.540+0000 7f609bc52740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d6862d0 0x557e6d6960c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.540+0000 7f609bc52740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d690b90 0x557e6d696600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.540+0000 7f609bc52740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f60941ae350 con 0x557e6d6862d0 2024-04-26T02:03:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.540+0000 7f609bc52740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f60941af7c0 con 0x557e6d690b90 2024-04-26T02:03:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.540+0000 7f609bc52740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f60941aea20 con 0x557e6d6859b0 2024-04-26T02:03:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.540+0000 7f609bc52740 10 client.0 _ll_register_callbacks cb 0x7f607c098d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.540+0000 7f608be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d6862d0 0x557e6d6960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.540+0000 7f6092800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d690b90 0x557e6d696600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.540+0000 7f608be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d6862d0 0x557e6d6960c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38218/0 (socket says 172.21.15.1:38218) 2024-04-26T02:03:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.540+0000 7f608be00640 1 -- 172.21.15.1:0/2283500707 learned_addr learned my addr 172.21.15.1:0/2283500707 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.540+0000 7f6091e00640 1 --2- 172.21.15.1:0/2283500707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557e6d6859b0 0x557e6d698c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.541+0000 7f608be00640 1 -- 172.21.15.1:0/2283500707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557e6d6859b0 msgr2=0x557e6d698c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.541+0000 7f608be00640 1 --2- 172.21.15.1:0/2283500707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557e6d6859b0 0x557e6d698c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.541+0000 7f608be00640 1 -- 172.21.15.1:0/2283500707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d690b90 msgr2=0x557e6d696600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.541+0000 7f608be00640 1 --2- 172.21.15.1:0/2283500707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d690b90 0x557e6d696600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.541+0000 7f608be00640 1 -- 172.21.15.1:0/2283500707 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e6d696bc0 con 0x557e6d6862d0 2024-04-26T02:03:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.541+0000 7f6092800640 1 --2- 172.21.15.1:0/2283500707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d690b90 0x557e6d696600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.541+0000 7f608be00640 1 --2- 172.21.15.1:0/2283500707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d6862d0 0x557e6d6960c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f607c005430 tx=0x7f607c09c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546995d3ee322877 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.541+0000 7f6077e00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T02:03:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.541+0000 7f6077e00640 1 -- 172.21.15.1:0/2283500707 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c09c410 con 0x557e6d6862d0 2024-04-26T02:03:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.541+0000 7f6077e00640 1 -- 172.21.15.1:0/2283500707 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f607c0a5490 con 0x557e6d6862d0 2024-04-26T02:03:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.541+0000 7f609bc52740 1 -- 172.21.15.1:0/2283500707 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f607c0d5040 con 0x557e6d6862d0 2024-04-26T02:03:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.542+0000 7f6077e00640 1 -- 172.21.15.1:0/2283500707 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f607c0ce6f0 con 0x557e6d6862d0 2024-04-26T02:03:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.542+0000 7f609bc52740 10 client.16398 fetch_fsmap learned FSMap version 35 2024-04-26T02:03:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.542+0000 7f609bc52740 1 -- 172.21.15.1:0/2283500707 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557e6d707340 con 0x557e6d6862d0 2024-04-26T02:03:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.542+0000 7f6077e00640 1 -- 172.21.15.1:0/2283500707 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f607c0d5040 con 0x557e6d6862d0 2024-04-26T02:03:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.542+0000 7f6077e00640 1 -- 172.21.15.1:0/2283500707 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 35) ==== 8342+0+0 (secure 0 0 0) 0x7f607c0a1ff0 con 0x557e6d6862d0 2024-04-26T02:03:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.542+0000 7f609bc52740 10 client.16398 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:03:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.542+0000 7f609bc52740 10 client.16398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:03:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.542+0000 7f609bc52740 20 client.16398 populate_metadata read hostname 'smithi001' 2024-04-26T02:03:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.542+0000 7f609bc52740 1 --2- 172.21.15.1:0/2283500707 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x557e6d6a1020 0x557e6d709830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.542+0000 7f609bc52740 4 client.16398 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:03:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.542+0000 7f609bc52740 1 -- 172.21.15.1:0/2283500707 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e6d5f9320 con 0x557e6d6a1020 2024-04-26T02:03:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.543+0000 7f6091e00640 1 --2- 172.21.15.1:0/2283500707 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x557e6d6a1020 0x557e6d709830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.543+0000 7f6091e00640 1 --2- 172.21.15.1:0/2283500707 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x557e6d6a1020 0x557e6d709830 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7faf1ed805697f4d server_cookie=c3395631c5b1978c in_seq=0 out_seq=0 2024-04-26T02:03:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:28.543+0000 7f6077e00640 10 client.16398 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:29.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.577+0000 7fb94be00700 1 -- 172.21.15.1:0/2691818343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb94406b070 msgr2=0x7fb94410efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:29.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.577+0000 7fb94be00700 1 --2- 172.21.15.1:0/2691818343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb94406b070 0x7fb94410efb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb930009990 tx=0x7fb930009ca0 comp rx=0 tx=0).stop 2024-04-26T02:03:29.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.577+0000 7fb94be00700 1 -- 172.21.15.1:0/2691818343 shutdown_connections 2024-04-26T02:03:29.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.577+0000 7fb94be00700 1 --2- 172.21.15.1:0/2691818343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb94410f4f0 0x7fb9441118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:29.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.577+0000 7fb94be00700 1 --2- 172.21.15.1:0/2691818343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb94406a750 0x7fb94406ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:29.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.577+0000 7fb94be00700 1 --2- 172.21.15.1:0/2691818343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb94406b070 0x7fb94410efb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:29.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.577+0000 7fb94be00700 1 -- 172.21.15.1:0/2691818343 >> 172.21.15.1:0/2691818343 conn(0x7fb9440788f0 msgr2=0x7fb944078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:29.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.577+0000 7fb94be00700 1 -- 172.21.15.1:0/2691818343 shutdown_connections 2024-04-26T02:03:29.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.577+0000 7fb94be00700 1 -- 172.21.15.1:0/2691818343 wait complete. 2024-04-26T02:03:29.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.579+0000 7fb94be00700 1 Processor -- start 2024-04-26T02:03:29.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.579+0000 7fb94be00700 1 -- start start 2024-04-26T02:03:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.579+0000 7fb94be00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb94406a750 0x7fb94419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.579+0000 7fb94be00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb94406b070 0x7fb94419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.579+0000 7fb94be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb94410f4f0 0x7fb94419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.579+0000 7fb94be00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb944114e70 con 0x7fb94410f4f0 2024-04-26T02:03:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.579+0000 7fb94be00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb944114cf0 con 0x7fb94406b070 2024-04-26T02:03:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.579+0000 7fb94be00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb944114ff0 con 0x7fb94406a750 2024-04-26T02:03:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.580+0000 7fb948a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb94406b070 0x7fb94419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.580+0000 7fb948a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb94406b070 0x7fb94419b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:40456/0 (socket says 172.21.15.1:40456) 2024-04-26T02:03:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.580+0000 7fb948a00700 1 -- 172.21.15.1:0/3858947037 learned_addr learned my addr 172.21.15.1:0/3858947037 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.580+0000 7fb949400700 1 --2- 172.21.15.1:0/3858947037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb94406a750 0x7fb94419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.580+0000 7fb949e00700 1 --2- 172.21.15.1:0/3858947037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb94410f4f0 0x7fb94419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.580+0000 7fb948a00700 1 -- 172.21.15.1:0/3858947037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb94406a750 msgr2=0x7fb94419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.580+0000 7fb948a00700 1 --2- 172.21.15.1:0/3858947037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb94406a750 0x7fb94419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.580+0000 7fb948a00700 1 -- 172.21.15.1:0/3858947037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb94410f4f0 msgr2=0x7fb94419fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.580+0000 7fb948a00700 1 --2- 172.21.15.1:0/3858947037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb94410f4f0 0x7fb94419fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.580+0000 7fb948a00700 1 -- 172.21.15.1:0/3858947037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9441a03d0 con 0x7fb94406b070 2024-04-26T02:03:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.580+0000 7fb949e00700 1 --2- 172.21.15.1:0/3858947037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb94410f4f0 0x7fb94419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:03:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.581+0000 7fb948a00700 1 --2- 172.21.15.1:0/3858947037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb94406b070 0x7fb94419b900 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb930005910 tx=0x7fb93000b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.581+0000 7fb942000700 1 -- 172.21.15.1:0/3858947037 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb930027070 con 0x7fb94406b070 2024-04-26T02:03:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.581+0000 7fb942000700 1 -- 172.21.15.1:0/3858947037 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb930022470 con 0x7fb94406b070 2024-04-26T02:03:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.581+0000 7fb942000700 1 -- 172.21.15.1:0/3858947037 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb930017650 con 0x7fb94406b070 2024-04-26T02:03:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.581+0000 7fb94be00700 1 -- 172.21.15.1:0/3858947037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9441a0660 con 0x7fb94406b070 2024-04-26T02:03:29.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.581+0000 7fb94be00700 1 -- 172.21.15.1:0/3858947037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9441a7f40 con 0x7fb94406b070 2024-04-26T02:03:29.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.582+0000 7fb94be00700 1 -- 172.21.15.1:0/3858947037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb944101f60 con 0x7fb94406b070 2024-04-26T02:03:29.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.584+0000 7fb942000700 1 -- 172.21.15.1:0/3858947037 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fb93002a070 con 0x7fb94406b070 2024-04-26T02:03:29.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.586+0000 7fb942000700 1 --2- 172.21.15.1:0/3858947037 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fb92c0a3e30 0x7fb92c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:29.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.586+0000 7fb942000700 1 -- 172.21.15.1:0/3858947037 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7fb9300d1720 con 0x7fb94406b070 2024-04-26T02:03:29.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.586+0000 7fb949400700 1 --2- 172.21.15.1:0/3858947037 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fb92c0a3e30 0x7fb92c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:29.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.587+0000 7fb949400700 1 --2- 172.21.15.1:0/3858947037 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fb92c0a3e30 0x7fb92c0a62f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb938009c60 tx=0x7fb938009260 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.590+0000 7fb942000700 1 -- 172.21.15.1:0/3858947037 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb93005eff0 con 0x7fb94406b070 2024-04-26T02:03:29.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.882+0000 7fb94be00700 1 -- 172.21.15.1:0/3858947037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb94419c720 con 0x7fb94406b070 2024-04-26T02:03:29.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.883+0000 7fb942000700 1 -- 172.21.15.1:0/3858947037 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11839 (secure 0 0 0) 0x7fb930013070 con 0x7fb94406b070 2024-04-26T02:03:29.882 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T02:03:29.883 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T02:03:26.883065+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26497":{"gid":26497,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/334804458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":334804458},{"type":"v1","addr":"172.21.15.173:6835","nonce":334804458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24499,24514,24518]},"id":1}]} 2024-04-26T02:03:29.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.887+0000 7fb94be00700 1 -- 172.21.15.1:0/3858947037 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fb92c0a3e30 msgr2=0x7fb92c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:29.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.887+0000 7fb94be00700 1 --2- 172.21.15.1:0/3858947037 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fb92c0a3e30 0x7fb92c0a62f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb938009c60 tx=0x7fb938009260 comp rx=0 tx=0).stop 2024-04-26T02:03:29.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.887+0000 7fb94be00700 1 -- 172.21.15.1:0/3858947037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb94406b070 msgr2=0x7fb94419b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:29.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.887+0000 7fb94be00700 1 --2- 172.21.15.1:0/3858947037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb94406b070 0x7fb94419b900 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb930005910 tx=0x7fb93000b670 comp rx=0 tx=0).stop 2024-04-26T02:03:29.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.887+0000 7fb94be00700 1 -- 172.21.15.1:0/3858947037 shutdown_connections 2024-04-26T02:03:29.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.887+0000 7fb94be00700 1 --2- 172.21.15.1:0/3858947037 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fb92c0a3e30 0x7fb92c0a62f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:29.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.887+0000 7fb94be00700 1 --2- 172.21.15.1:0/3858947037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb94410f4f0 0x7fb94419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:29.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.887+0000 7fb94be00700 1 --2- 172.21.15.1:0/3858947037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb94406a750 0x7fb94419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:29.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.887+0000 7fb94be00700 1 --2- 172.21.15.1:0/3858947037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb94406b070 0x7fb94419b900 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:29.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.887+0000 7fb94be00700 1 -- 172.21.15.1:0/3858947037 >> 172.21.15.1:0/3858947037 conn(0x7fb9440788f0 msgr2=0x7fb944110a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:29.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.887+0000 7fb94be00700 1 -- 172.21.15.1:0/3858947037 shutdown_connections 2024-04-26T02:03:29.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:29.887+0000 7fb94be00700 1 -- 172.21.15.1:0/3858947037 wait complete. 2024-04-26T02:03:29.887 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 35 2024-04-26T02:03:30.219 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:30 smithi001 ceph-mon[27822]: pgmap v372: 97 pgs: 97 active+clean; 186 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 203 op/s 2024-04-26T02:03:30.219 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:30 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3858947037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:03:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.337+0000 7f6077e00640 1 -- 172.21.15.1:0/2283500707 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557e6d5f9320 con 0x557e6d6a1020 2024-04-26T02:03:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.337+0000 7f6077e00640 10 client.16398 handle_command_reply: tid=0 2024-04-26T02:03:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.337+0000 7f609bc52740 1 client.16398 shutdown 2024-04-26T02:03:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.337+0000 7f609bc52740 1 -- 172.21.15.1:0/2283500707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d6862d0 msgr2=0x557e6d6960c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.337+0000 7f609bc52740 1 --2- 172.21.15.1:0/2283500707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d6862d0 0x557e6d6960c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f607c005430 tx=0x7f607c09c720 comp rx=0 tx=0).stop 2024-04-26T02:03:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.337+0000 7f609bc52740 1 -- 172.21.15.1:0/2283500707 shutdown_connections 2024-04-26T02:03:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.337+0000 7f609bc52740 1 --2- 172.21.15.1:0/2283500707 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x557e6d6a1020 0x557e6d709830 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.337+0000 7f609bc52740 1 --2- 172.21.15.1:0/2283500707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e6d690b90 0x557e6d696600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.337+0000 7f609bc52740 1 --2- 172.21.15.1:0/2283500707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e6d6862d0 0x557e6d6960c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.337+0000 7f609bc52740 1 --2- 172.21.15.1:0/2283500707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557e6d6859b0 0x557e6d698c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.337+0000 7f609bc52740 1 -- 172.21.15.1:0/2283500707 >> 172.21.15.1:0/2283500707 conn(0x557e6d681ec0 msgr2=0x557e6d68f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.338+0000 7f6077e00640 0 client.16398 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.338+0000 7f609bc52740 1 -- 172.21.15.1:0/2283500707 shutdown_connections 2024-04-26T02:03:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.338+0000 7f609bc52740 1 -- 172.21.15.1:0/2283500707 wait complete. 2024-04-26T02:03:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.338+0000 7f609bc52740 20 client.16398 trim_cache size 0 max 16384 2024-04-26T02:03:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T02:03:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T02:03:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T02:03:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T02:03:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.339+0000 7f6093200640 1 -- 172.21.15.1:0/3740481767 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f60780a3d80 msgr2=0x7f60780a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.339+0000 7f6093200640 1 --2- 172.21.15.1:0/3740481767 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f60780a3d80 0x7f60780a6240 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f608c011630 tx=0x7f608c009500 comp rx=0 tx=0).stop 2024-04-26T02:03:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.339+0000 7f6093200640 1 -- 172.21.15.1:0/3740481767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60941092a0 msgr2=0x7f60941a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.339+0000 7f6093200640 1 --2- 172.21.15.1:0/3740481767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60941092a0 0x7f60941a0960 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f607c00e9f0 tx=0x7f607c00eeb0 comp rx=0 tx=0).stop 2024-04-26T02:03:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.339+0000 7f6093200640 1 -- 172.21.15.1:0/3740481767 shutdown_connections 2024-04-26T02:03:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.339+0000 7f6093200640 1 --2- 172.21.15.1:0/3740481767 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f60780a3d80 0x7f60780a6240 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.339+0000 7f6093200640 1 --2- 172.21.15.1:0/3740481767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f609419a5d0 0x7f609419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.340+0000 7f6093200640 1 --2- 172.21.15.1:0/3740481767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60941092a0 0x7f60941a0960 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.340+0000 7f6093200640 1 --2- 172.21.15.1:0/3740481767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60941041d0 0x7f60941a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.340+0000 7f6093200640 1 -- 172.21.15.1:0/3740481767 >> 172.21.15.1:0/3740481767 conn(0x7f60940ffa30 msgr2=0x7f6094100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.340+0000 7f6093200640 1 -- 172.21.15.1:0/3740481767 shutdown_connections 2024-04-26T02:03:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:30.340+0000 7f6093200640 1 -- 172.21.15.1:0/3740481767 wait complete. 2024-04-26T02:03:30.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:30 smithi097 ceph-mon[32853]: pgmap v372: 97 pgs: 97 active+clean; 186 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 203 op/s 2024-04-26T02:03:30.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:30 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3858947037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:03:30.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:30 smithi173 ceph-mon[32553]: pgmap v372: 97 pgs: 97 active+clean; 186 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 203 op/s 2024-04-26T02:03:30.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:30 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3858947037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:03:30.768 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24518 quiesce db / --timeout 60 --expiration 79.25 --await-for 42.0' 2024-04-26T02:03:30.769 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24518 quiesce db / --timeout 60 --expiration 79.25 --await-for 42.0 2024-04-26T02:03:31.252 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:03:32.330 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:32 smithi001 ceph-mon[27822]: pgmap v373: 97 pgs: 97 active+clean; 195 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 338 op/s 2024-04-26T02:03:32.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:32 smithi097 ceph-mon[32853]: pgmap v373: 97 pgs: 97 active+clean; 195 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 338 op/s 2024-04-26T02:03:32.498 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-04-26T02:03:32.498 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-26T02:03:32.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:32 smithi173 ceph-mon[32553]: pgmap v373: 97 pgs: 97 active+clean; 195 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 338 op/s 2024-04-26T02:03:33.724 DEBUG:teuthology.parallel:result is None 2024-04-26T02:03:33.725 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T02:03:33.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.727+0000 7f2d55a00700 1 -- 172.21.15.1:0/80287520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d50075590 msgr2=0x7f2d5010ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.727+0000 7f2d55a00700 1 --2- 172.21.15.1:0/80287520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d50075590 0x7f2d5010ef60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009990 tx=0x7f2d3c009ca0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.727+0000 7f2d55a00700 1 -- 172.21.15.1:0/80287520 shutdown_connections 2024-04-26T02:03:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.727+0000 7f2d55a00700 1 --2- 172.21.15.1:0/80287520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d5010f4a0 0x7f2d50111890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.727+0000 7f2d55a00700 1 --2- 172.21.15.1:0/80287520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d50074c70 0x7f2d50075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.727+0000 7f2d55a00700 1 --2- 172.21.15.1:0/80287520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d50075590 0x7f2d5010ef60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.727+0000 7f2d55a00700 1 -- 172.21.15.1:0/80287520 >> 172.21.15.1:0/80287520 conn(0x7f2d50079b10 msgr2=0x7f2d50079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.727+0000 7f2d55a00700 1 -- 172.21.15.1:0/80287520 shutdown_connections 2024-04-26T02:03:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.728+0000 7f2d55a00700 1 -- 172.21.15.1:0/80287520 wait complete. 2024-04-26T02:03:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.728+0000 7f2d55a00700 1 Processor -- start 2024-04-26T02:03:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.729+0000 7f2d55a00700 1 -- start start 2024-04-26T02:03:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.729+0000 7f2d55a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d50074c70 0x7f2d5019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.729+0000 7f2d55a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d50075590 0x7f2d5019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.729+0000 7f2d55a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d5010f4a0 0x7f2d5019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:33.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.729+0000 7f2d55a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2d50114e00 con 0x7f2d5010f4a0 2024-04-26T02:03:33.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.729+0000 7f2d55a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2d50114c80 con 0x7f2d50075590 2024-04-26T02:03:33.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.729+0000 7f2d55a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2d50114f80 con 0x7f2d50074c70 2024-04-26T02:03:33.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.729+0000 7f2d4ea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d50074c70 0x7f2d5019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:33.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.729+0000 7f2d4ea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d50074c70 0x7f2d5019b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:48602/0 (socket says 172.21.15.1:48602) 2024-04-26T02:03:33.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.729+0000 7f2d4ea00700 1 -- 172.21.15.1:0/1373989171 learned_addr learned my addr 172.21.15.1:0/1373989171 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:33.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.729+0000 7f2d4f400700 1 --2- 172.21.15.1:0/1373989171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d5010f4a0 0x7f2d5019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:33.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.729+0000 7f2d4e000700 1 --2- 172.21.15.1:0/1373989171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d50075590 0x7f2d5019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:33.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.730+0000 7f2d4ea00700 1 -- 172.21.15.1:0/1373989171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d50075590 msgr2=0x7f2d5019b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:33.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.730+0000 7f2d4ea00700 1 --2- 172.21.15.1:0/1373989171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d50075590 0x7f2d5019b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.730+0000 7f2d4ea00700 1 -- 172.21.15.1:0/1373989171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d5010f4a0 msgr2=0x7f2d5019fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:33.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.730+0000 7f2d4ea00700 1 --2- 172.21.15.1:0/1373989171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d5010f4a0 0x7f2d5019fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.730+0000 7f2d4ea00700 1 -- 172.21.15.1:0/1373989171 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d501a02e0 con 0x7f2d50074c70 2024-04-26T02:03:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.730+0000 7f2d4ea00700 1 --2- 172.21.15.1:0/1373989171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d50074c70 0x7f2d5019b2d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000e9a0 tx=0x7f2d4000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.730+0000 7f2d47400700 1 -- 172.21.15.1:0/1373989171 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4000ca10 con 0x7f2d50074c70 2024-04-26T02:03:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.731+0000 7f2d47400700 1 -- 172.21.15.1:0/1373989171 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d4000cbb0 con 0x7f2d50074c70 2024-04-26T02:03:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.731+0000 7f2d55a00700 1 -- 172.21.15.1:0/1373989171 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d501a05d0 con 0x7f2d50074c70 2024-04-26T02:03:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.731+0000 7f2d47400700 1 -- 172.21.15.1:0/1373989171 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40018680 con 0x7f2d50074c70 2024-04-26T02:03:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.731+0000 7f2d55a00700 1 -- 172.21.15.1:0/1373989171 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d501a7e50 con 0x7f2d50074c70 2024-04-26T02:03:33.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.734+0000 7f2d47400700 1 -- 172.21.15.1:0/1373989171 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f2d40010430 con 0x7f2d50074c70 2024-04-26T02:03:33.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.735+0000 7f2d60397b80 1 Processor -- start 2024-04-26T02:03:33.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.736+0000 7f2d60397b80 1 -- start start 2024-04-26T02:03:33.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.736+0000 7f2d47400700 1 --2- 172.21.15.1:0/1373989171 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2d340a3e50 0x7f2d340a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:33.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.736+0000 7f2d47400700 1 -- 172.21.15.1:0/1373989171 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f2d400d0a20 con 0x7f2d50074c70 2024-04-26T02:03:33.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.736+0000 7f2d60397b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08069b0 0x5654e0806d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:33.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.736+0000 7f2d60397b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e08072d0 0x5654e0811c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:33.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.736+0000 7f2d60397b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e0812180 0x5654e0814570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:33.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.736+0000 7f2d60397b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5654e075a020 con 0x5654e0812180 2024-04-26T02:03:33.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.736+0000 7f2d60397b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5654e07008c0 con 0x5654e08069b0 2024-04-26T02:03:33.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.736+0000 7f2d60397b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5654e0755fe0 con 0x5654e08072d0 2024-04-26T02:03:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.736+0000 7f2d4f400700 1 --2- 172.21.15.1:0/1373989171 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2d340a3e50 0x7f2d340a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.736+0000 7f2d4e000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e08072d0 0x5654e0811c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.736+0000 7f2d4e000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e08072d0 0x5654e0811c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:48606/0 (socket says 172.21.15.1:48606) 2024-04-26T02:03:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.736+0000 7f2d4e000700 1 -- 172.21.15.1:0/385248495 learned_addr learned my addr 172.21.15.1:0/385248495 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.737+0000 7f2d4f400700 1 --2- 172.21.15.1:0/385248495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e0812180 0x5654e0814570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.737+0000 7f2d4ea00700 1 --2- 172.21.15.1:0/385248495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08069b0 0x5654e0806d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.737+0000 7f2d4ea00700 1 -- 172.21.15.1:0/385248495 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e08072d0 msgr2=0x5654e0811c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.737+0000 7f2d4ea00700 1 --2- 172.21.15.1:0/385248495 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e08072d0 0x5654e0811c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.737+0000 7f2d4ea00700 1 -- 172.21.15.1:0/385248495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e0812180 msgr2=0x5654e0814570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.737+0000 7f2d4ea00700 1 --2- 172.21.15.1:0/385248495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e0812180 0x5654e0814570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.737+0000 7f2d4ea00700 1 -- 172.21.15.1:0/385248495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654e06f6810 con 0x5654e08069b0 2024-04-26T02:03:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.737+0000 7f2d4f400700 1 --2- 172.21.15.1:0/1373989171 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2d340a3e50 0x7f2d340a6310 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800f870 tx=0x7f2d4800fc30 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:33.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.737+0000 7f2d4ea00700 1 --2- 172.21.15.1:0/385248495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08069b0 0x5654e0806d90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2d400ce0e0 tx=0x7f2d4009c5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2587b75d4c1041ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:33.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.738+0000 7f2d3be00700 1 -- 172.21.15.1:0/385248495 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d400a0ce0 con 0x5654e08069b0 2024-04-26T02:03:33.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.738+0000 7f2d3be00700 1 -- 172.21.15.1:0/385248495 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d400a55a0 con 0x5654e08069b0 2024-04-26T02:03:33.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.738+0000 7f2d3be00700 1 -- 172.21.15.1:0/385248495 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d400a5110 con 0x5654e08069b0 2024-04-26T02:03:33.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.738+0000 7f2d60397b80 1 -- 172.21.15.1:0/385248495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08069b0 msgr2=0x5654e0806d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:33.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.738+0000 7f2d60397b80 1 --2- 172.21.15.1:0/385248495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08069b0 0x5654e0806d90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2d400ce0e0 tx=0x7f2d4009c5a0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.739+0000 7f2d60397b80 1 -- 172.21.15.1:0/385248495 shutdown_connections 2024-04-26T02:03:33.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.739+0000 7f2d60397b80 1 --2- 172.21.15.1:0/385248495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e0812180 0x5654e0814570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.739+0000 7f2d60397b80 1 --2- 172.21.15.1:0/385248495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08069b0 0x5654e0806d90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.739+0000 7f2d60397b80 1 --2- 172.21.15.1:0/385248495 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e08072d0 0x5654e0811c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.739+0000 7f2d60397b80 1 -- 172.21.15.1:0/385248495 >> 172.21.15.1:0/385248495 conn(0x5654e0735730 msgr2=0x5654e0802e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:33.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.739+0000 7f2d60397b80 1 -- 172.21.15.1:0/385248495 shutdown_connections 2024-04-26T02:03:33.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.739+0000 7f2d60397b80 1 -- 172.21.15.1:0/385248495 wait complete. 2024-04-26T02:03:33.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.739+0000 7f2d60397b80 1 -- start start 2024-04-26T02:03:33.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.740+0000 7f2d60397b80 1 Processor -- start 2024-04-26T02:03:33.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.740+0000 7f2d60397b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e08069b0 0x5654e0811960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:33.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.740+0000 7f2d60397b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08072d0 0x5654e080ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:33.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.740+0000 7f2d60397b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e0812180 0x5654e080f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:33.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.740+0000 7f2d60397b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5654e0814c40 con 0x5654e0812180 2024-04-26T02:03:33.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.740+0000 7f2d60397b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5654e07139f0 con 0x5654e08072d0 2024-04-26T02:03:33.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.740+0000 7f2d60397b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5654e0814dc0 con 0x5654e08069b0 2024-04-26T02:03:33.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.740+0000 7f2d60397b80 10 client.0 _ll_register_callbacks cb 0x5654e06f28b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:33.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.741+0000 7f2d4f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e0812180 0x5654e080f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:33.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.741+0000 7f2d4ea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e08069b0 0x5654e0811960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:33.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.741+0000 7f2d4ea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e08069b0 0x5654e0811960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:48610/0 (socket says 172.21.15.1:48610) 2024-04-26T02:03:33.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.741+0000 7f2d4f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e0812180 0x5654e080f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49086/0 (socket says 172.21.15.1:49086) 2024-04-26T02:03:33.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.741+0000 7f2d4e000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08072d0 0x5654e080ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:33.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.741+0000 7f2d4f400700 1 -- 172.21.15.1:0/1312439225 learned_addr learned my addr 172.21.15.1:0/1312439225 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:33.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.741+0000 7f2d4f400700 1 -- 172.21.15.1:0/1312439225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e08069b0 msgr2=0x5654e0811960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:33.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.741+0000 7f2d4f400700 1 --2- 172.21.15.1:0/1312439225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e08069b0 0x5654e0811960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.741+0000 7f2d4f400700 1 -- 172.21.15.1:0/1312439225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08072d0 msgr2=0x5654e080ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:33.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.741+0000 7f2d4f400700 1 --2- 172.21.15.1:0/1312439225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08072d0 0x5654e080ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.741+0000 7f2d4f400700 1 -- 172.21.15.1:0/1312439225 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654e06a7a60 con 0x5654e0812180 2024-04-26T02:03:33.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.741+0000 7f2d4e000700 1 --2- 172.21.15.1:0/1312439225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08072d0 0x5654e080ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:33.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.742+0000 7f2d4f400700 1 --2- 172.21.15.1:0/1312439225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e0812180 0x5654e080f1a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009960 tx=0x7f2d48004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12af9bc6c37a3158 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:33.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.742+0000 7f2d38c00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T02:03:33.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.742+0000 7f2d38c00700 1 -- 172.21.15.1:0/1312439225 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48028070 con 0x5654e0812180 2024-04-26T02:03:33.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.742+0000 7f2d38c00700 1 -- 172.21.15.1:0/1312439225 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f2d48008920 con 0x5654e0812180 2024-04-26T02:03:33.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.742+0000 7f2d60397b80 1 -- 172.21.15.1:0/1312439225 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d400a55a0 con 0x5654e0812180 2024-04-26T02:03:33.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.743+0000 7f2d38c00700 1 -- 172.21.15.1:0/1312439225 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2d48032050 con 0x5654e0812180 2024-04-26T02:03:33.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.743+0000 7f2d60397b80 10 client.16416 fetch_fsmap learned FSMap version 35 2024-04-26T02:03:33.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.743+0000 7f2d60397b80 1 -- 172.21.15.1:0/1312439225 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5654e06f0c80 con 0x5654e0812180 2024-04-26T02:03:33.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.743+0000 7f2d38c00700 1 -- 172.21.15.1:0/1312439225 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d400a55a0 con 0x5654e0812180 2024-04-26T02:03:33.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.744+0000 7f2d38c00700 1 -- 172.21.15.1:0/1312439225 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 35) ==== 8342+0+0 (secure 0 0 0) 0x7f2d48024c50 con 0x5654e0812180 2024-04-26T02:03:33.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.744+0000 7f2d60397b80 10 client.16416 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:03:33.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.744+0000 7f2d60397b80 10 client.16416 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T02:03:33.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.744+0000 7f2d60397b80 20 client.16416 populate_metadata read hostname 'smithi001' 2024-04-26T02:03:33.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.744+0000 7f2d60397b80 1 --2- 172.21.15.1:0/1312439225 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5654e0821a10 0x5654e0823e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:33.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.744+0000 7f2d60397b80 4 client.16416 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:03:33.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.744+0000 7f2d60397b80 1 -- 172.21.15.1:0/1312439225 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654e0767810 con 0x5654e0821a10 2024-04-26T02:03:33.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.744+0000 7f2d4ea00700 1 --2- 172.21.15.1:0/1312439225 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5654e0821a10 0x5654e0823e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:33.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.745+0000 7f2d4ea00700 1 --2- 172.21.15.1:0/1312439225 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5654e0821a10 0x5654e0823e00 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e61051e5daa3daf6 server_cookie=a5ff5670a50d457b in_seq=0 out_seq=0 2024-04-26T02:03:33.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.745+0000 7f2d38c00700 10 client.16416 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:33.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.745+0000 7f2d38c00700 1 -- 172.21.15.1:0/1312439225 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5654e0767810 con 0x5654e0821a10 2024-04-26T02:03:33.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.745+0000 7f2d38c00700 10 client.16416 handle_command_reply: tid=0 2024-04-26T02:03:33.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.746+0000 7f2d60397b80 1 client.16416 shutdown 2024-04-26T02:03:33.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.746+0000 7f2d60397b80 1 -- 172.21.15.1:0/1312439225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e0812180 msgr2=0x5654e080f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:33.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.746+0000 7f2d60397b80 1 --2- 172.21.15.1:0/1312439225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e0812180 0x5654e080f1a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009960 tx=0x7f2d48004b40 comp rx=0 tx=0).stop 2024-04-26T02:03:33.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.746+0000 7f2d60397b80 1 -- 172.21.15.1:0/1312439225 shutdown_connections 2024-04-26T02:03:33.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.746+0000 7f2d60397b80 1 --2- 172.21.15.1:0/1312439225 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5654e0821a10 0x5654e0823e00 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.746+0000 7f2d60397b80 1 --2- 172.21.15.1:0/1312439225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e0812180 0x5654e080f1a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.746+0000 7f2d60397b80 1 --2- 172.21.15.1:0/1312439225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e08069b0 0x5654e0811960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.746+0000 7f2d38c00700 0 client.16416 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:33.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.746+0000 7f2d60397b80 1 --2- 172.21.15.1:0/1312439225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08072d0 0x5654e080ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.746+0000 7f2d60397b80 1 -- 172.21.15.1:0/1312439225 >> 172.21.15.1:0/1312439225 conn(0x5654e0735730 msgr2=0x5654e0802a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:33.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.746+0000 7f2d60397b80 1 -- 172.21.15.1:0/1312439225 shutdown_connections 2024-04-26T02:03:33.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.746+0000 7f2d60397b80 1 -- 172.21.15.1:0/1312439225 wait complete. 2024-04-26T02:03:33.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.746+0000 7f2d60397b80 20 client.16416 trim_cache size 0 max 16384 2024-04-26T02:03:33.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.767+0000 7f2d60397b80 1 Processor -- start 2024-04-26T02:03:33.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.767+0000 7f2d60397b80 1 -- start start 2024-04-26T02:03:33.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.768+0000 7f2d60397b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e0735730 0x5654e080f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:33.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.768+0000 7f2d60397b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08069b0 0x5654e080a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:33.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.768+0000 7f2d60397b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e08072d0 0x5654e081db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:33.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.768+0000 7f2d60397b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2d501b1eb0 con 0x5654e08072d0 2024-04-26T02:03:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.768+0000 7f2d60397b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2d501af710 con 0x5654e08069b0 2024-04-26T02:03:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.768+0000 7f2d60397b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2d501ad580 con 0x5654e0735730 2024-04-26T02:03:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.768+0000 7f2d4f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e08072d0 0x5654e081db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.768+0000 7f2d4f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e08072d0 0x5654e081db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49100/0 (socket says 172.21.15.1:49100) 2024-04-26T02:03:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.768+0000 7f2d4f400700 1 -- 172.21.15.1:0/4153003124 learned_addr learned my addr 172.21.15.1:0/4153003124 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.768+0000 7f2d4ea00700 1 --2- 172.21.15.1:0/4153003124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e0735730 0x5654e080f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.768+0000 7f2d4e000700 1 --2- 172.21.15.1:0/4153003124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08069b0 0x5654e080a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.769+0000 7f2d4ea00700 1 -- 172.21.15.1:0/4153003124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08069b0 msgr2=0x5654e080a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.769+0000 7f2d4ea00700 1 --2- 172.21.15.1:0/4153003124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08069b0 0x5654e080a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.769+0000 7f2d4ea00700 1 -- 172.21.15.1:0/4153003124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e08072d0 msgr2=0x5654e081db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.769+0000 7f2d4ea00700 1 --2- 172.21.15.1:0/4153003124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e08072d0 0x5654e081db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.769+0000 7f2d4ea00700 1 -- 172.21.15.1:0/4153003124 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654e0824430 con 0x5654e0735730 2024-04-26T02:03:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.769+0000 7f2d4f400700 1 --2- 172.21.15.1:0/4153003124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e08072d0 0x5654e081db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:33.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.769+0000 7f2d4ea00700 1 --2- 172.21.15.1:0/4153003124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e0735730 0x5654e080f2d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2d400ce110 tx=0x7f2d40010c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7464017f51ae6cfd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:33.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.769+0000 7f2d3b400700 1 -- 172.21.15.1:0/4153003124 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4009ac30 con 0x5654e0735730 2024-04-26T02:03:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.769+0000 7f2d3b400700 1 -- 172.21.15.1:0/4153003124 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d400870b0 con 0x5654e0735730 2024-04-26T02:03:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.769+0000 7f2d3b400700 1 -- 172.21.15.1:0/4153003124 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d400a5110 con 0x5654e0735730 2024-04-26T02:03:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.770+0000 7f2d60397b80 1 -- 172.21.15.1:0/4153003124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e0735730 msgr2=0x5654e080f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.770+0000 7f2d60397b80 1 --2- 172.21.15.1:0/4153003124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e0735730 0x5654e080f2d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2d400ce110 tx=0x7f2d40010c10 comp rx=0 tx=0).stop 2024-04-26T02:03:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.770+0000 7f2d60397b80 1 -- 172.21.15.1:0/4153003124 shutdown_connections 2024-04-26T02:03:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.770+0000 7f2d60397b80 1 --2- 172.21.15.1:0/4153003124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e08072d0 0x5654e081db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.770+0000 7f2d60397b80 1 --2- 172.21.15.1:0/4153003124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e0735730 0x5654e080f2d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.770+0000 7f2d60397b80 1 --2- 172.21.15.1:0/4153003124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08069b0 0x5654e080a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.770+0000 7f2d60397b80 1 -- 172.21.15.1:0/4153003124 >> 172.21.15.1:0/4153003124 conn(0x5654e08030c0 msgr2=0x5654e0809bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.770+0000 7f2d60397b80 1 -- 172.21.15.1:0/4153003124 shutdown_connections 2024-04-26T02:03:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.770+0000 7f2d60397b80 1 -- 172.21.15.1:0/4153003124 wait complete. 2024-04-26T02:03:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.771+0000 7f2d60397b80 1 -- start start 2024-04-26T02:03:33.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.771+0000 7f2d60397b80 1 Processor -- start 2024-04-26T02:03:33.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.772+0000 7f2d60397b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e0735730 0x5654e0816ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:33.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.772+0000 7f2d60397b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e08069b0 0x5654e08173e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:33.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.772+0000 7f2d60397b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08072d0 0x5654e0887f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:33.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.772+0000 7f2d60397b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2d501adbd0 con 0x5654e08069b0 2024-04-26T02:03:33.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.772+0000 7f2d60397b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2d501af040 con 0x5654e08072d0 2024-04-26T02:03:33.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.772+0000 7f2d60397b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2d501ae2a0 con 0x5654e0735730 2024-04-26T02:03:33.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.772+0000 7f2d60397b80 10 client.0 _ll_register_callbacks cb 0x5654e06f28b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:33.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.772+0000 7f2d4e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e08069b0 0x5654e08173e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:33.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.772+0000 7f2d4e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e08069b0 0x5654e08173e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49102/0 (socket says 172.21.15.1:49102) 2024-04-26T02:03:33.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.772+0000 7f2d4e000700 1 -- 172.21.15.1:0/1746575623 learned_addr learned my addr 172.21.15.1:0/1746575623 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:33.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.772+0000 7f2d4ea00700 1 --2- 172.21.15.1:0/1746575623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e0735730 0x5654e0816ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:33.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.772+0000 7f2d4f400700 1 --2- 172.21.15.1:0/1746575623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08072d0 0x5654e0887f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:33.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.773+0000 7f2d4f400700 1 -- 172.21.15.1:0/1746575623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e0735730 msgr2=0x5654e0816ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:33.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.773+0000 7f2d4f400700 1 --2- 172.21.15.1:0/1746575623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e0735730 0x5654e0816ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.773+0000 7f2d4f400700 1 -- 172.21.15.1:0/1746575623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e08069b0 msgr2=0x5654e08173e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:33.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.773+0000 7f2d4f400700 1 --2- 172.21.15.1:0/1746575623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e08069b0 0x5654e08173e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.773+0000 7f2d4f400700 1 -- 172.21.15.1:0/1746575623 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654e0888440 con 0x5654e08072d0 2024-04-26T02:03:33.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.773+0000 7f2d4f400700 1 --2- 172.21.15.1:0/1746575623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08072d0 0x5654e0887f00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2d48004b10 tx=0x7f2d480060f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=919f62a4278ed052 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:33.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.773+0000 7f2d38c00700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:03:33.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.773+0000 7f2d38c00700 1 -- 172.21.15.1:0/1746575623 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48028040 con 0x5654e08072d0 2024-04-26T02:03:33.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.773+0000 7f2d60397b80 1 -- 172.21.15.1:0/1746575623 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d400870b0 con 0x5654e08072d0 2024-04-26T02:03:33.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.773+0000 7f2d38c00700 1 -- 172.21.15.1:0/1746575623 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f2d48004db0 con 0x5654e08072d0 2024-04-26T02:03:33.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.774+0000 7f2d38c00700 1 -- 172.21.15.1:0/1746575623 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2d4802d730 con 0x5654e08072d0 2024-04-26T02:03:33.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.774+0000 7f2d60397b80 10 client.26600 fetch_fsmap learned FSMap version 35 2024-04-26T02:03:33.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.774+0000 7f2d60397b80 1 -- 172.21.15.1:0/1746575623 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5654e08895d0 con 0x5654e08072d0 2024-04-26T02:03:33.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.774+0000 7f2d38c00700 1 -- 172.21.15.1:0/1746575623 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d400870b0 con 0x5654e08072d0 2024-04-26T02:03:33.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.774+0000 7f2d38c00700 1 -- 172.21.15.1:0/1746575623 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 35) ==== 8342+0+0 (secure 0 0 0) 0x7f2d48046070 con 0x5654e08072d0 2024-04-26T02:03:33.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.774+0000 7f2d60397b80 10 client.26600 fetch_fsmap finished waiting for FSMap version 35 2024-04-26T02:03:33.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.774+0000 7f2d60397b80 10 client.26600 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T02:03:33.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.774+0000 7f2d60397b80 20 client.26600 populate_metadata read hostname 'smithi001' 2024-04-26T02:03:33.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.774+0000 7f2d60397b80 1 --2- 172.21.15.1:0/1746575623 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5654e0812180 0x5654e088bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:33.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.774+0000 7f2d60397b80 4 client.26600 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.25, "await_for": 42.0}] 2024-04-26T02:03:33.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.774+0000 7f2d60397b80 1 -- 172.21.15.1:0/1746575623 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.25, "await_for": 42.0}) -- 0x5654e07618b0 con 0x5654e0812180 2024-04-26T02:03:33.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.775+0000 7f2d4ea00700 1 --2- 172.21.15.1:0/1746575623 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5654e0812180 0x5654e088bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:33.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.775+0000 7f2d38c00700 10 client.26600 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:33.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.775+0000 7f2d4ea00700 1 --2- 172.21.15.1:0/1746575623 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5654e0812180 0x5654e088bdc0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1f1fd7d44ae4917 server_cookie=3e3bc2cd11f63b0a in_seq=0 out_seq=0 2024-04-26T02:03:33.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.798+0000 7f2d38c00700 1 -- 172.21.15.1:0/1746575623 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5654e07618b0 con 0x5654e0812180 2024-04-26T02:03:33.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.798+0000 7f2d38c00700 10 client.26600 handle_command_reply: tid=0 2024-04-26T02:03:33.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.798+0000 7f2d60397b80 1 client.26600 shutdown 2024-04-26T02:03:33.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.799+0000 7f2d60397b80 1 -- 172.21.15.1:0/1746575623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08072d0 msgr2=0x5654e0887f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:33.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.799+0000 7f2d60397b80 1 --2- 172.21.15.1:0/1746575623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08072d0 0x5654e0887f00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2d48004b10 tx=0x7f2d480060f0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.799+0000 7f2d60397b80 1 -- 172.21.15.1:0/1746575623 shutdown_connections 2024-04-26T02:03:33.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.799+0000 7f2d60397b80 1 --2- 172.21.15.1:0/1746575623 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5654e0812180 0x5654e088bdc0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.799+0000 7f2d60397b80 1 --2- 172.21.15.1:0/1746575623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654e08072d0 0x5654e0887f00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.799+0000 7f2d60397b80 1 --2- 172.21.15.1:0/1746575623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654e0735730 0x5654e0816ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.799+0000 7f2d38c00700 0 client.26600 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:33.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.799+0000 7f2d60397b80 1 --2- 172.21.15.1:0/1746575623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654e08069b0 0x5654e08173e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.799+0000 7f2d60397b80 1 -- 172.21.15.1:0/1746575623 >> 172.21.15.1:0/1746575623 conn(0x5654e08030c0 msgr2=0x5654e080f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:33.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.799+0000 7f2d60397b80 1 -- 172.21.15.1:0/1746575623 shutdown_connections 2024-04-26T02:03:33.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.799+0000 7f2d60397b80 1 -- 172.21.15.1:0/1746575623 wait complete. 2024-04-26T02:03:33.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.799+0000 7f2d60397b80 20 client.26600 trim_cache size 0 max 16384 2024-04-26T02:03:33.800 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T02:03:33.800 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 35, 2024-04-26T02:03:33.800 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24518, 2024-04-26T02:03:33.800 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 34, 2024-04-26T02:03:33.800 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-04-26T02:03:33.800 INFO:teuthology.orchestra.run.smithi001.stdout: "7ed9c2aa": { 2024-04-26T02:03:33.800 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 34, 2024-04-26T02:03:33.800 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-04-26T02:03:33.801 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T02:03:33.801 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-04-26T02:03:33.801 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T02:03:33.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:03:33.801 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-04-26T02:03:33.801 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 79.2, 2024-04-26T02:03:33.801 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-04-26T02:03:33.801 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-04-26T02:03:33.801 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-04-26T02:03:33.801 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T02:03:33.801 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-04-26T02:03:33.801 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T02:03:33.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:03:33.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:03:33.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:03:33.802 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:03:33.802 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:03:33.802 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T02:03:33.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.800+0000 7f2d55a00700 1 -- 172.21.15.1:0/1373989171 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2d340a3e50 msgr2=0x7f2d340a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:33.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.800+0000 7f2d55a00700 1 --2- 172.21.15.1:0/1373989171 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2d340a3e50 0x7f2d340a6310 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800f870 tx=0x7f2d4800fc30 comp rx=0 tx=0).stop 2024-04-26T02:03:33.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.800+0000 7f2d55a00700 1 -- 172.21.15.1:0/1373989171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d50074c70 msgr2=0x7f2d5019b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:33.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.800+0000 7f2d55a00700 1 --2- 172.21.15.1:0/1373989171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d50074c70 0x7f2d5019b2d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000e9a0 tx=0x7f2d4000ecb0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.800+0000 7f2d55a00700 1 -- 172.21.15.1:0/1373989171 shutdown_connections 2024-04-26T02:03:33.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.800+0000 7f2d55a00700 1 --2- 172.21.15.1:0/1373989171 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f2d340a3e50 0x7f2d340a6310 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.800+0000 7f2d55a00700 1 --2- 172.21.15.1:0/1373989171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d5010f4a0 0x7f2d5019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.800+0000 7f2d55a00700 1 --2- 172.21.15.1:0/1373989171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d50074c70 0x7f2d5019b2d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.800+0000 7f2d55a00700 1 --2- 172.21.15.1:0/1373989171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d50075590 0x7f2d5019b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:33.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.800+0000 7f2d55a00700 1 -- 172.21.15.1:0/1373989171 >> 172.21.15.1:0/1373989171 conn(0x7f2d50079b10 msgr2=0x7f2d500ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:33.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.800+0000 7f2d55a00700 1 -- 172.21.15.1:0/1373989171 shutdown_connections 2024-04-26T02:03:33.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:33.800+0000 7f2d55a00700 1 -- 172.21.15.1:0/1373989171 wait complete. 2024-04-26T02:03:34.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:34 smithi001 ceph-mon[27822]: pgmap v374: 97 pgs: 97 active+clean; 195 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 266 op/s 2024-04-26T02:03:34.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:34 smithi097 ceph-mon[32853]: pgmap v374: 97 pgs: 97 active+clean; 195 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 266 op/s 2024-04-26T02:03:34.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:34 smithi173 ceph-mon[32553]: pgmap v374: 97 pgs: 97 active+clean; 195 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 266 op/s 2024-04-26T02:03:34.673 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7ed9c2aa', quiesce took 8.3 seconds. Will release after: 19.25 2024-04-26T02:03:36.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:36 smithi001 ceph-mon[27822]: pgmap v375: 97 pgs: 97 active+clean; 187 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 409 op/s 2024-04-26T02:03:36.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:36 smithi097 ceph-mon[32853]: pgmap v375: 97 pgs: 97 active+clean; 187 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 409 op/s 2024-04-26T02:03:36.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:36 smithi173 ceph-mon[32553]: pgmap v375: 97 pgs: 97 active+clean; 187 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 409 op/s 2024-04-26T02:03:38.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:38 smithi001 ceph-mon[27822]: pgmap v376: 97 pgs: 97 active+clean; 187 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 279 op/s 2024-04-26T02:03:38.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:38 smithi001 ceph-mon[27822]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T02:03:38.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:38 smithi097 ceph-mon[32853]: pgmap v376: 97 pgs: 97 active+clean; 187 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 279 op/s 2024-04-26T02:03:38.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:38 smithi097 ceph-mon[32853]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T02:03:38.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:38 smithi173 ceph-mon[32553]: pgmap v376: 97 pgs: 97 active+clean; 187 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 279 op/s 2024-04-26T02:03:38.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:38 smithi173 ceph-mon[32553]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T02:03:40.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:40 smithi001 ceph-mon[27822]: pgmap v377: 97 pgs: 97 active+clean; 187 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 279 op/s 2024-04-26T02:03:40.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:40 smithi097 ceph-mon[32853]: pgmap v377: 97 pgs: 97 active+clean; 187 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 279 op/s 2024-04-26T02:03:40.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:40 smithi173 ceph-mon[32553]: pgmap v377: 97 pgs: 97 active+clean; 187 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 279 op/s 2024-04-26T02:03:42.370 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:42 smithi173 ceph-mon[32553]: pgmap v378: 97 pgs: 97 active+clean; 171 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 295 op/s 2024-04-26T02:03:42.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:42 smithi001 ceph-mon[27822]: pgmap v378: 97 pgs: 97 active+clean; 171 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 295 op/s 2024-04-26T02:03:42.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:42 smithi097 ceph-mon[32853]: pgmap v378: 97 pgs: 97 active+clean; 171 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 295 op/s 2024-04-26T02:03:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.486+0000 7f7007200640 1 -- 172.21.15.1:0/360109369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70081092c0 msgr2=0x7f7008110e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.486+0000 7f7007200640 1 --2- 172.21.15.1:0/360109369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70081092c0 0x7f7008110e90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7000009ca0 tx=0x7f700002a8c0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.486+0000 7f7007200640 1 -- 172.21.15.1:0/360109369 shutdown_connections 2024-04-26T02:03:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.486+0000 7f7007200640 1 --2- 172.21.15.1:0/360109369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70081092c0 0x7f7008110e90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.486+0000 7f7007200640 1 --2- 172.21.15.1:0/360109369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7008104ba0 0x7f7008108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.486+0000 7f7007200640 1 --2- 172.21.15.1:0/360109369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70081041f0 0x7f70081045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.486+0000 7f7007200640 1 -- 172.21.15.1:0/360109369 >> 172.21.15.1:0/360109369 conn(0x7f70080ffa50 msgr2=0x7f7008101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.486+0000 7f7007200640 1 -- 172.21.15.1:0/360109369 shutdown_connections 2024-04-26T02:03:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.486+0000 7f7007200640 1 -- 172.21.15.1:0/360109369 wait complete. 2024-04-26T02:03:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.487+0000 7f7007200640 1 Processor -- start 2024-04-26T02:03:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.488+0000 7f7007200640 1 -- start start 2024-04-26T02:03:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.488+0000 7f7007200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70081041f0 0x7f70081a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.488+0000 7f7007200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70081092c0 0x7f70081a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.488+0000 7f7007200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f700819a5c0 0x7f700819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.488+0000 7f7007200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7008114260 con 0x7f70081041f0 2024-04-26T02:03:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.488+0000 7f7007200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f70081140e0 con 0x7f700819a5c0 2024-04-26T02:03:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.488+0000 7f7007200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f70081143e0 con 0x7f70081092c0 2024-04-26T02:03:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.488+0000 7f7005e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70081041f0 0x7f70081a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.489+0000 7f7005e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70081041f0 0x7f70081a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53536/0 (socket says 172.21.15.1:53536) 2024-04-26T02:03:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.489+0000 7f7005e00640 1 -- 172.21.15.1:0/2743377014 learned_addr learned my addr 172.21.15.1:0/2743377014 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.489+0000 7f7005400640 1 --2- 172.21.15.1:0/2743377014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70081092c0 0x7f70081a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.489+0000 7f7006800640 1 --2- 172.21.15.1:0/2743377014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f700819a5c0 0x7f700819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.489+0000 7f7005e00640 1 -- 172.21.15.1:0/2743377014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70081092c0 msgr2=0x7f70081a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.489+0000 7f7005e00640 1 --2- 172.21.15.1:0/2743377014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70081092c0 0x7f70081a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.489+0000 7f7005e00640 1 -- 172.21.15.1:0/2743377014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f700819a5c0 msgr2=0x7f700819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.489+0000 7f7005e00640 1 --2- 172.21.15.1:0/2743377014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f700819a5c0 0x7f700819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.489+0000 7f7005e00640 1 -- 172.21.15.1:0/2743377014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700819b0e0 con 0x7f70081041f0 2024-04-26T02:03:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.489+0000 7f7005400640 1 --2- 172.21.15.1:0/2743377014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70081092c0 0x7f70081a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.489+0000 7f7005e00640 1 --2- 172.21.15.1:0/2743377014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70081041f0 0x7f70081a0410 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800e9f0 tx=0x7f6ff800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.490+0000 7f6ffea00640 1 -- 172.21.15.1:0/2743377014 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff800cd80 con 0x7f70081041f0 2024-04-26T02:03:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.490+0000 7f6ffea00640 1 -- 172.21.15.1:0/2743377014 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff80108e0 con 0x7f70081041f0 2024-04-26T02:03:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.490+0000 7f6ffea00640 1 -- 172.21.15.1:0/2743377014 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8018980 con 0x7f70081041f0 2024-04-26T02:03:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.490+0000 7f7007200640 1 -- 172.21.15.1:0/2743377014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f700819b3d0 con 0x7f70081041f0 2024-04-26T02:03:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.490+0000 7f7007200640 1 -- 172.21.15.1:0/2743377014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70081a7620 con 0x7f70081041f0 2024-04-26T02:03:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.493+0000 7f700fa21740 1 Processor -- start 2024-04-26T02:03:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.493+0000 7f700fa21740 1 -- start start 2024-04-26T02:03:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.493+0000 7f6ffea00640 1 -- 172.21.15.1:0/2743377014 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f6ff8010430 con 0x7f70081041f0 2024-04-26T02:03:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.493+0000 7f700fa21740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dcd9b0 0x5602c3dcdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.494+0000 7f700fa21740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dce2d0 0x5602c3dd8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.494+0000 7f700fa21740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dd8b90 0x5602c3ddaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.494+0000 7f700fa21740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5602c3d33900 con 0x5602c3dd8b90 2024-04-26T02:03:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.494+0000 7f700fa21740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5602c3cd9ef0 con 0x5602c3dcd9b0 2024-04-26T02:03:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.494+0000 7f700fa21740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5602c3d2f8c0 con 0x5602c3dce2d0 2024-04-26T02:03:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.494+0000 7f7006800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dcd9b0 0x5602c3dcdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.494+0000 7f7006800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dcd9b0 0x5602c3dcdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:41820/0 (socket says 172.21.15.1:41820) 2024-04-26T02:03:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.494+0000 7f7006800640 1 -- 172.21.15.1:0/1311513190 learned_addr learned my addr 172.21.15.1:0/1311513190 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.494+0000 7f7005400640 1 --2- 172.21.15.1:0/1311513190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dd8b90 0x5602c3ddaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.494+0000 7f7005e00640 1 --2- 172.21.15.1:0/1311513190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dce2d0 0x5602c3dd8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.494+0000 7f7006800640 1 -- 172.21.15.1:0/1311513190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dce2d0 msgr2=0x5602c3dd8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.494+0000 7f7006800640 1 --2- 172.21.15.1:0/1311513190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dce2d0 0x5602c3dd8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.494+0000 7f7006800640 1 -- 172.21.15.1:0/1311513190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dd8b90 msgr2=0x5602c3ddaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.494+0000 7f7006800640 1 --2- 172.21.15.1:0/1311513190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dd8b90 0x5602c3ddaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.494+0000 7f7006800640 1 -- 172.21.15.1:0/1311513190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602c3a46210 con 0x5602c3dcd9b0 2024-04-26T02:03:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.494+0000 7f7005400640 1 --2- 172.21.15.1:0/1311513190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dd8b90 0x5602c3ddaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:03:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.494+0000 7f7005e00640 1 --2- 172.21.15.1:0/1311513190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dce2d0 0x5602c3dd8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:03:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.495+0000 7f7006800640 1 --2- 172.21.15.1:0/1311513190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dcd9b0 0x5602c3dcdd90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7000009ca0 tx=0x7f70000093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=821b6895a2730cb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.495+0000 7f6ffea00640 1 --2- 172.21.15.1:0/2743377014 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6fec0a3d00 0x7f6fec0a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.495+0000 7f6ff3000640 1 -- 172.21.15.1:0/1311513190 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000031cf0 con 0x5602c3dcd9b0 2024-04-26T02:03:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.495+0000 7f6ff3000640 1 -- 172.21.15.1:0/1311513190 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7000036030 con 0x5602c3dcd9b0 2024-04-26T02:03:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.495+0000 7f6ff3000640 1 -- 172.21.15.1:0/1311513190 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70000042a0 con 0x5602c3dcd9b0 2024-04-26T02:03:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.495+0000 7f6ffea00640 1 -- 172.21.15.1:0/2743377014 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f6ff80d0b70 con 0x7f70081041f0 2024-04-26T02:03:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.495+0000 7f7005e00640 1 --2- 172.21.15.1:0/2743377014 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6fec0a3d00 0x7f6fec0a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.496+0000 7f700fa21740 1 -- 172.21.15.1:0/1311513190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dcd9b0 msgr2=0x5602c3dcdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.496+0000 7f700fa21740 1 --2- 172.21.15.1:0/1311513190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dcd9b0 0x5602c3dcdd90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7000009ca0 tx=0x7f70000093e0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.496+0000 7f7005e00640 1 --2- 172.21.15.1:0/2743377014 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6fec0a3d00 0x7f6fec0a61c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x5602c3dcef50 tx=0x7f6ff8000bc0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.496+0000 7f700fa21740 1 -- 172.21.15.1:0/1311513190 shutdown_connections 2024-04-26T02:03:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.496+0000 7f700fa21740 1 --2- 172.21.15.1:0/1311513190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dd8b90 0x5602c3ddaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.496+0000 7f700fa21740 1 --2- 172.21.15.1:0/1311513190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dce2d0 0x5602c3dd8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.496+0000 7f700fa21740 1 --2- 172.21.15.1:0/1311513190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dcd9b0 0x5602c3dcdd90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.496+0000 7f700fa21740 1 -- 172.21.15.1:0/1311513190 >> 172.21.15.1:0/1311513190 conn(0x5602c3dc9ec0 msgr2=0x5602c3dca2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.496+0000 7f700fa21740 1 -- 172.21.15.1:0/1311513190 shutdown_connections 2024-04-26T02:03:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.496+0000 7f700fa21740 1 -- 172.21.15.1:0/1311513190 wait complete. 2024-04-26T02:03:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.497+0000 7f700fa21740 1 -- start start 2024-04-26T02:03:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.497+0000 7f700fa21740 1 Processor -- start 2024-04-26T02:03:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.497+0000 7f700fa21740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dcd9b0 0x5602c3dd5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.497+0000 7f700fa21740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dce2d0 0x5602c3dd6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.497+0000 7f700fa21740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dd8b90 0x5602c3dd65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.497+0000 7f700fa21740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5602c3ddb5e0 con 0x5602c3dd8b90 2024-04-26T02:03:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.497+0000 7f700fa21740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5602c3ced310 con 0x5602c3dcd9b0 2024-04-26T02:03:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.497+0000 7f700fa21740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5602c3ddb760 con 0x5602c3dce2d0 2024-04-26T02:03:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.497+0000 7f700fa21740 10 client.0 _ll_register_callbacks cb 0x5602c3cbbcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.498+0000 7f7006800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dcd9b0 0x5602c3dd5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.498+0000 7f7006800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dcd9b0 0x5602c3dd5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:41834/0 (socket says 172.21.15.1:41834) 2024-04-26T02:03:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.498+0000 7f7006800640 1 -- 172.21.15.1:0/4255829811 learned_addr learned my addr 172.21.15.1:0/4255829811 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.498+0000 7f7005400640 1 --2- 172.21.15.1:0/4255829811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dce2d0 0x5602c3dd6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.498+0000 7f7006800640 1 --2- 172.21.15.1:0/4255829811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dd8b90 0x5602c3dd65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.498+0000 7f7005400640 1 -- 172.21.15.1:0/4255829811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dcd9b0 msgr2=0x5602c3dd5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.498+0000 7f7005400640 1 --2- 172.21.15.1:0/4255829811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dcd9b0 0x5602c3dd5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.498+0000 7f7005400640 1 -- 172.21.15.1:0/4255829811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dd8b90 msgr2=0x5602c3dd65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.498+0000 7f7005400640 1 --2- 172.21.15.1:0/4255829811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dd8b90 0x5602c3dd65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.498+0000 7f7005400640 1 -- 172.21.15.1:0/4255829811 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602c3b12620 con 0x5602c3dce2d0 2024-04-26T02:03:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.498+0000 7f7006800640 1 --2- 172.21.15.1:0/4255829811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dd8b90 0x5602c3dd65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.499+0000 7f7005400640 1 --2- 172.21.15.1:0/4255829811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dce2d0 0x5602c3dd6060 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f70080fff70 tx=0x7f6ff4005870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df076b31c40307b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.499+0000 7f6fe7400640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T02:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.499+0000 7f6fe7400640 1 -- 172.21.15.1:0/4255829811 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff400fa90 con 0x5602c3dce2d0 2024-04-26T02:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.499+0000 7f700fa21740 1 -- 172.21.15.1:0/4255829811 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7000036030 con 0x5602c3dce2d0 2024-04-26T02:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.499+0000 7f6fe7400640 1 -- 172.21.15.1:0/4255829811 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f6ff400fc30 con 0x5602c3dce2d0 2024-04-26T02:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.499+0000 7f6fe7400640 1 -- 172.21.15.1:0/4255829811 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6ff400fe30 con 0x5602c3dce2d0 2024-04-26T02:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.499+0000 7f700fa21740 10 client.26623 fetch_fsmap learned FSMap version 36 2024-04-26T02:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.499+0000 7f700fa21740 1 -- 172.21.15.1:0/4255829811 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5602c3de7930 con 0x5602c3dce2d0 2024-04-26T02:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.500+0000 7f6fe7400640 1 -- 172.21.15.1:0/4255829811 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7000036030 con 0x5602c3dce2d0 2024-04-26T02:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.500+0000 7f6fe7400640 1 -- 172.21.15.1:0/4255829811 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f6ff40104c0 con 0x5602c3dce2d0 2024-04-26T02:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.500+0000 7f700fa21740 10 client.26623 fetch_fsmap finished waiting for FSMap version 36 2024-04-26T02:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.500+0000 7f700fa21740 10 client.26623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.500+0000 7f700fa21740 20 client.26623 populate_metadata read hostname 'smithi001' 2024-04-26T02:03:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.500+0000 7f700fa21740 1 --2- 172.21.15.1:0/4255829811 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5602c3de8df0 0x5602c3deb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.500+0000 7f700fa21740 4 client.26623 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:03:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.500+0000 7f700fa21740 1 -- 172.21.15.1:0/4255829811 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602c3d478b0 con 0x5602c3de8df0 2024-04-26T02:03:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.500+0000 7f7006800640 1 --2- 172.21.15.1:0/4255829811 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5602c3de8df0 0x5602c3deb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.501+0000 7f7006800640 1 --2- 172.21.15.1:0/4255829811 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5602c3de8df0 0x5602c3deb200 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=488ef81222463c3a server_cookie=c82e7e99a83c82b2 in_seq=0 out_seq=0 2024-04-26T02:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.501+0000 7f6fe7400640 10 client.26623 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.502+0000 7f6fe7400640 1 -- 172.21.15.1:0/4255829811 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5602c3d478b0 con 0x5602c3de8df0 2024-04-26T02:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.502+0000 7f6fe7400640 10 client.26623 handle_command_reply: tid=0 2024-04-26T02:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.502+0000 7f700fa21740 1 client.26623 shutdown 2024-04-26T02:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.502+0000 7f700fa21740 1 -- 172.21.15.1:0/4255829811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dce2d0 msgr2=0x5602c3dd6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.502+0000 7f700fa21740 1 --2- 172.21.15.1:0/4255829811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dce2d0 0x5602c3dd6060 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f70080fff70 tx=0x7f6ff4005870 comp rx=0 tx=0).stop 2024-04-26T02:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.502+0000 7f700fa21740 1 -- 172.21.15.1:0/4255829811 shutdown_connections 2024-04-26T02:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.502+0000 7f700fa21740 1 --2- 172.21.15.1:0/4255829811 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5602c3de8df0 0x5602c3deb200 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.502+0000 7f700fa21740 1 --2- 172.21.15.1:0/4255829811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dd8b90 0x5602c3dd65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.502+0000 7f700fa21740 1 --2- 172.21.15.1:0/4255829811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dce2d0 0x5602c3dd6060 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.502+0000 7f700fa21740 1 --2- 172.21.15.1:0/4255829811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dcd9b0 0x5602c3dd5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.502+0000 7f700fa21740 1 -- 172.21.15.1:0/4255829811 >> 172.21.15.1:0/4255829811 conn(0x5602c3dc9ec0 msgr2=0x5602c3dcfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.502+0000 7f6fe7400640 0 client.26623 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.502+0000 7f700fa21740 1 -- 172.21.15.1:0/4255829811 shutdown_connections 2024-04-26T02:03:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.502+0000 7f700fa21740 1 -- 172.21.15.1:0/4255829811 wait complete. 2024-04-26T02:03:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.502+0000 7f700fa21740 20 client.26623 trim_cache size 0 max 16384 2024-04-26T02:03:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.519+0000 7f700fa21740 1 Processor -- start 2024-04-26T02:03:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.519+0000 7f700fa21740 1 -- start start 2024-04-26T02:03:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.519+0000 7f700fa21740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dcd9b0 0x5602c3de44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.519+0000 7f700fa21740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dce2d0 0x5602c3dc8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.519+0000 7f700fa21740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dd8b90 0x5602c3df0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.519+0000 7f700fa21740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f70081b2650 con 0x5602c3dcd9b0 2024-04-26T02:03:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.519+0000 7f700fa21740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f70081afef0 con 0x5602c3dce2d0 2024-04-26T02:03:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.519+0000 7f700fa21740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f70081add60 con 0x5602c3dd8b90 2024-04-26T02:03:43.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.520+0000 7f7006800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dcd9b0 0x5602c3de44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.520+0000 7f7006800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dcd9b0 0x5602c3de44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53564/0 (socket says 172.21.15.1:53564) 2024-04-26T02:03:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.520+0000 7f7006800640 1 -- 172.21.15.1:0/1624157133 learned_addr learned my addr 172.21.15.1:0/1624157133 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.520+0000 7f7005400640 1 --2- 172.21.15.1:0/1624157133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dce2d0 0x5602c3dc8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.520+0000 7f7006800640 1 --2- 172.21.15.1:0/1624157133 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dd8b90 0x5602c3df0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.520+0000 7f7006800640 1 -- 172.21.15.1:0/1624157133 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dd8b90 msgr2=0x5602c3df0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.520+0000 7f7006800640 1 --2- 172.21.15.1:0/1624157133 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dd8b90 0x5602c3df0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.520+0000 7f7006800640 1 -- 172.21.15.1:0/1624157133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dce2d0 msgr2=0x5602c3dc8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.520+0000 7f7006800640 1 --2- 172.21.15.1:0/1624157133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dce2d0 0x5602c3dc8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.520+0000 7f7006800640 1 -- 172.21.15.1:0/1624157133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602c3deb880 con 0x5602c3dcd9b0 2024-04-26T02:03:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.521+0000 7f7006800640 1 --2- 172.21.15.1:0/1624157133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dcd9b0 0x5602c3de44a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f700002f540 tx=0x7f7000005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b8bbf2e396146a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.521+0000 7f6ff2600640 1 -- 172.21.15.1:0/1624157133 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000031b70 con 0x5602c3dcd9b0 2024-04-26T02:03:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.521+0000 7f6ff2600640 1 -- 172.21.15.1:0/1624157133 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7000036070 con 0x5602c3dcd9b0 2024-04-26T02:03:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.521+0000 7f6ff2600640 1 -- 172.21.15.1:0/1624157133 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000004430 con 0x5602c3dcd9b0 2024-04-26T02:03:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.522+0000 7f700fa21740 1 -- 172.21.15.1:0/1624157133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dcd9b0 msgr2=0x5602c3de44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.522+0000 7f700fa21740 1 --2- 172.21.15.1:0/1624157133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dcd9b0 0x5602c3de44a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f700002f540 tx=0x7f7000005df0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.522+0000 7f700fa21740 1 -- 172.21.15.1:0/1624157133 shutdown_connections 2024-04-26T02:03:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.522+0000 7f700fa21740 1 --2- 172.21.15.1:0/1624157133 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dd8b90 0x5602c3df0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.522+0000 7f700fa21740 1 --2- 172.21.15.1:0/1624157133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dce2d0 0x5602c3dc8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.522+0000 7f700fa21740 1 --2- 172.21.15.1:0/1624157133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dcd9b0 0x5602c3de44a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.522+0000 7f700fa21740 1 -- 172.21.15.1:0/1624157133 >> 172.21.15.1:0/1624157133 conn(0x5602c3dc9ec0 msgr2=0x5602c3dc8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.522+0000 7f700fa21740 1 -- 172.21.15.1:0/1624157133 shutdown_connections 2024-04-26T02:03:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.522+0000 7f700fa21740 1 -- 172.21.15.1:0/1624157133 wait complete. 2024-04-26T02:03:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.522+0000 7f700fa21740 1 -- start start 2024-04-26T02:03:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.522+0000 7f700fa21740 1 Processor -- start 2024-04-26T02:03:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.523+0000 7f700fa21740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dcd9b0 0x5602c3ddeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.523+0000 7f700fa21740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dce2d0 0x5602c3ddf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.523+0000 7f700fa21740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dd8b90 0x5602c3e4e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.523+0000 7f700fa21740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f70081ae3b0 con 0x5602c3dcd9b0 2024-04-26T02:03:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.523+0000 7f700fa21740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f70081af820 con 0x5602c3dd8b90 2024-04-26T02:03:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.523+0000 7f700fa21740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f70081aea80 con 0x5602c3dce2d0 2024-04-26T02:03:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.523+0000 7f700fa21740 10 client.0 _ll_register_callbacks cb 0x7f6ff40127a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.523+0000 7f7006800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dcd9b0 0x5602c3ddeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.523+0000 7f7006800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dcd9b0 0x5602c3ddeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53568/0 (socket says 172.21.15.1:53568) 2024-04-26T02:03:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.523+0000 7f7006800640 1 -- 172.21.15.1:0/2571293685 learned_addr learned my addr 172.21.15.1:0/2571293685 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.523+0000 7f7005400640 1 --2- 172.21.15.1:0/2571293685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dce2d0 0x5602c3ddf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.523+0000 7f7006800640 1 --2- 172.21.15.1:0/2571293685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dd8b90 0x5602c3e4e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.524+0000 7f7006800640 1 -- 172.21.15.1:0/2571293685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dce2d0 msgr2=0x5602c3ddf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.524+0000 7f7006800640 1 --2- 172.21.15.1:0/2571293685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dce2d0 0x5602c3ddf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.524+0000 7f7006800640 1 -- 172.21.15.1:0/2571293685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dd8b90 msgr2=0x5602c3e4e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.524+0000 7f7006800640 1 --2- 172.21.15.1:0/2571293685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dd8b90 0x5602c3e4e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.524+0000 7f7006800640 1 -- 172.21.15.1:0/2571293685 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602c3e4e8d0 con 0x5602c3dcd9b0 2024-04-26T02:03:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.524+0000 7f7006800640 1 --2- 172.21.15.1:0/2571293685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dcd9b0 0x5602c3ddeac0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f700005a010 tx=0x7f700002b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97624aaa0ea0f9f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.524+0000 7f6fe7400640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T02:03:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.524+0000 7f6fe7400640 1 -- 172.21.15.1:0/2571293685 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000031af0 con 0x5602c3dcd9b0 2024-04-26T02:03:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.524+0000 7f6fe7400640 1 -- 172.21.15.1:0/2571293685 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f7000031c90 con 0x5602c3dcd9b0 2024-04-26T02:03:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.524+0000 7f700fa21740 1 -- 172.21.15.1:0/2571293685 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7000036070 con 0x5602c3dcd9b0 2024-04-26T02:03:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.525+0000 7f6fe7400640 1 -- 172.21.15.1:0/2571293685 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7000031450 con 0x5602c3dcd9b0 2024-04-26T02:03:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.525+0000 7f700fa21740 10 client.16440 fetch_fsmap learned FSMap version 36 2024-04-26T02:03:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.525+0000 7f700fa21740 1 -- 172.21.15.1:0/2571293685 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5602c3e4fb80 con 0x5602c3dcd9b0 2024-04-26T02:03:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.525+0000 7f6fe7400640 1 -- 172.21.15.1:0/2571293685 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7000036070 con 0x5602c3dcd9b0 2024-04-26T02:03:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.525+0000 7f6fe7400640 1 -- 172.21.15.1:0/2571293685 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f7000062030 con 0x5602c3dcd9b0 2024-04-26T02:03:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.525+0000 7f700fa21740 10 client.16440 fetch_fsmap finished waiting for FSMap version 36 2024-04-26T02:03:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.525+0000 7f700fa21740 10 client.16440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:03:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.525+0000 7f700fa21740 20 client.16440 populate_metadata read hostname 'smithi001' 2024-04-26T02:03:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.525+0000 7f700fa21740 1 --2- 172.21.15.1:0/2571293685 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5602c3e4feb0 0x5602c3e522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.525+0000 7f700fa21740 4 client.16440 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:03:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.525+0000 7f700fa21740 1 -- 172.21.15.1:0/2571293685 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5602c3d41320 con 0x5602c3e4feb0 2024-04-26T02:03:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.526+0000 7f7005400640 1 --2- 172.21.15.1:0/2571293685 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5602c3e4feb0 0x5602c3e522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.526+0000 7f7005400640 1 --2- 172.21.15.1:0/2571293685 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5602c3e4feb0 0x5602c3e522d0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1255dedd3dde18fd server_cookie=770b7219c4b40bdf in_seq=0 out_seq=0 2024-04-26T02:03:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.526+0000 7f6fe7400640 10 client.16440 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.607+0000 7f6fe7400640 1 -- 172.21.15.1:0/2571293685 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5602c3d41320 con 0x5602c3e4feb0 2024-04-26T02:03:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.607+0000 7f6fe7400640 10 client.16440 handle_command_reply: tid=0 2024-04-26T02:03:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.607+0000 7f700fa21740 1 client.16440 shutdown 2024-04-26T02:03:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.607+0000 7f700fa21740 1 -- 172.21.15.1:0/2571293685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dcd9b0 msgr2=0x5602c3ddeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.607+0000 7f700fa21740 1 --2- 172.21.15.1:0/2571293685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dcd9b0 0x5602c3ddeac0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f700005a010 tx=0x7f700002b040 comp rx=0 tx=0).stop 2024-04-26T02:03:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.607+0000 7f700fa21740 1 -- 172.21.15.1:0/2571293685 shutdown_connections 2024-04-26T02:03:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.607+0000 7f700fa21740 1 --2- 172.21.15.1:0/2571293685 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5602c3e4feb0 0x5602c3e522d0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.607+0000 7f700fa21740 1 --2- 172.21.15.1:0/2571293685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5602c3dd8b90 0x5602c3e4e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.607+0000 7f700fa21740 1 --2- 172.21.15.1:0/2571293685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602c3dce2d0 0x5602c3ddf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.607+0000 7f700fa21740 1 --2- 172.21.15.1:0/2571293685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5602c3dcd9b0 0x5602c3ddeac0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.607+0000 7f6fe7400640 0 client.16440 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.607+0000 7f700fa21740 1 -- 172.21.15.1:0/2571293685 >> 172.21.15.1:0/2571293685 conn(0x5602c3dc9ec0 msgr2=0x5602c3dcfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.608+0000 7f700fa21740 1 -- 172.21.15.1:0/2571293685 shutdown_connections 2024-04-26T02:03:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.608+0000 7f700fa21740 1 -- 172.21.15.1:0/2571293685 wait complete. 2024-04-26T02:03:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.608+0000 7f700fa21740 20 client.16440 trim_cache size 0 max 16384 2024-04-26T02:03:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T02:03:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T02:03:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T02:03:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T02:03:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.609+0000 7f7007200640 1 -- 172.21.15.1:0/2743377014 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6fec0a3d00 msgr2=0x7f6fec0a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.609+0000 7f7007200640 1 --2- 172.21.15.1:0/2743377014 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6fec0a3d00 0x7f6fec0a61c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x5602c3dcef50 tx=0x7f6ff8000bc0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.609+0000 7f7007200640 1 -- 172.21.15.1:0/2743377014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70081041f0 msgr2=0x7f70081a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.609+0000 7f7007200640 1 --2- 172.21.15.1:0/2743377014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70081041f0 0x7f70081a0410 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800e9f0 tx=0x7f6ff800eeb0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.609+0000 7f7007200640 1 -- 172.21.15.1:0/2743377014 shutdown_connections 2024-04-26T02:03:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.609+0000 7f7007200640 1 --2- 172.21.15.1:0/2743377014 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6fec0a3d00 0x7f6fec0a61c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.609+0000 7f7007200640 1 --2- 172.21.15.1:0/2743377014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f700819a5c0 0x7f700819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.609+0000 7f7007200640 1 --2- 172.21.15.1:0/2743377014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70081092c0 0x7f70081a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.609+0000 7f7007200640 1 --2- 172.21.15.1:0/2743377014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70081041f0 0x7f70081a0410 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.609+0000 7f7007200640 1 -- 172.21.15.1:0/2743377014 >> 172.21.15.1:0/2743377014 conn(0x7f70080ffa50 msgr2=0x7f7008101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.609+0000 7f7007200640 1 -- 172.21.15.1:0/2743377014 shutdown_connections 2024-04-26T02:03:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:43.609+0000 7f7007200640 1 -- 172.21.15.1:0/2743377014 wait complete. 2024-04-26T02:03:44.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:44 smithi001 ceph-mon[27822]: pgmap v379: 97 pgs: 97 active+clean; 171 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 754 KiB/s wr, 160 op/s 2024-04-26T02:03:44.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:44 smithi097 ceph-mon[32853]: pgmap v379: 97 pgs: 97 active+clean; 171 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 754 KiB/s wr, 160 op/s 2024-04-26T02:03:44.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:44 smithi173 ceph-mon[32553]: pgmap v379: 97 pgs: 97 active+clean; 171 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 754 KiB/s wr, 160 op/s 2024-04-26T02:03:46.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:46 smithi001 ceph-mon[27822]: pgmap v380: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 816 KiB/s wr, 190 op/s 2024-04-26T02:03:46.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:46 smithi097 ceph-mon[32853]: pgmap v380: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 816 KiB/s wr, 190 op/s 2024-04-26T02:03:46.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:46 smithi173 ceph-mon[32553]: pgmap v380: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 816 KiB/s wr, 190 op/s 2024-04-26T02:03:48.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:48 smithi001 ceph-mon[27822]: pgmap v381: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 70 KiB/s wr, 47 op/s 2024-04-26T02:03:48.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:48 smithi097 ceph-mon[32853]: pgmap v381: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 70 KiB/s wr, 47 op/s 2024-04-26T02:03:48.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:48 smithi173 ceph-mon[32553]: pgmap v381: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 70 KiB/s wr, 47 op/s 2024-04-26T02:03:50.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:50 smithi001 ceph-mon[27822]: pgmap v382: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 70 KiB/s wr, 47 op/s 2024-04-26T02:03:50.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:50 smithi097 ceph-mon[32853]: pgmap v382: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 70 KiB/s wr, 47 op/s 2024-04-26T02:03:50.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:50 smithi173 ceph-mon[32553]: pgmap v382: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 70 KiB/s wr, 47 op/s 2024-04-26T02:03:52.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:52 smithi097 ceph-mon[32853]: pgmap v383: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 70 KiB/s wr, 50 op/s 2024-04-26T02:03:52.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:52 smithi173 ceph-mon[32553]: pgmap v383: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 70 KiB/s wr, 50 op/s 2024-04-26T02:03:52.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:52 smithi001 ceph-mon[27822]: pgmap v383: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 70 KiB/s wr, 50 op/s 2024-04-26T02:03:53.925 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:03:54.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:54 smithi001 ceph-mon[27822]: pgmap v384: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-26T02:03:54.413 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:03:54.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:54 smithi097 ceph-mon[32853]: pgmap v384: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-26T02:03:54.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:54 smithi173 ceph-mon[32553]: pgmap v384: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-26T02:03:56.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:56 smithi001 ceph-mon[27822]: pgmap v385: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 64 KiB/s wr, 36 op/s 2024-04-26T02:03:56.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:56 smithi097 ceph-mon[32853]: pgmap v385: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 64 KiB/s wr, 36 op/s 2024-04-26T02:03:56.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:56 smithi173 ceph-mon[32553]: pgmap v385: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 64 KiB/s wr, 36 op/s 2024-04-26T02:03:56.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.621+0000 7f913ac00700 1 -- 172.21.15.1:0/3314370404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9134108a30 msgr2=0x7f9134108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:56.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.621+0000 7f913ac00700 1 --2- 172.21.15.1:0/3314370404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9134108a30 0x7f9134108e10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9124009990 tx=0x7f9124009ca0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.621+0000 7f913ac00700 1 -- 172.21.15.1:0/3314370404 shutdown_connections 2024-04-26T02:03:56.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.621+0000 7f913ac00700 1 --2- 172.21.15.1:0/3314370404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f913406b0f0 0x7f913410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.621+0000 7f913ac00700 1 --2- 172.21.15.1:0/3314370404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9134108a30 0x7f9134108e10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.621+0000 7f913ac00700 1 --2- 172.21.15.1:0/3314370404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f913406a750 0x7f913406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.621+0000 7f913ac00700 1 -- 172.21.15.1:0/3314370404 >> 172.21.15.1:0/3314370404 conn(0x7f91340788f0 msgr2=0x7f9134078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:56.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.621+0000 7f913ac00700 1 -- 172.21.15.1:0/3314370404 shutdown_connections 2024-04-26T02:03:56.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.621+0000 7f913ac00700 1 -- 172.21.15.1:0/3314370404 wait complete. 2024-04-26T02:03:56.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.622+0000 7f913ac00700 1 Processor -- start 2024-04-26T02:03:56.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.623+0000 7f913ac00700 1 -- start start 2024-04-26T02:03:56.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.623+0000 7f913ac00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f913406a750 0x7f913419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.623+0000 7f913ac00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f913406b0f0 0x7f913419b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.623+0000 7f913ac00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9134108a30 0x7f913419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.623+0000 7f913ac00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9134112c20 con 0x7f913406a750 2024-04-26T02:03:56.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.623+0000 7f913ac00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9134112aa0 con 0x7f9134108a30 2024-04-26T02:03:56.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.623+0000 7f913ac00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9134112da0 con 0x7f913406b0f0 2024-04-26T02:03:56.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.624+0000 7f9133400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f913406b0f0 0x7f913419b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.624+0000 7f9133e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f913406a750 0x7f913419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.624+0000 7f9133400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f913406b0f0 0x7f913419b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:38728/0 (socket says 172.21.15.1:38728) 2024-04-26T02:03:56.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.624+0000 7f9133400700 1 -- 172.21.15.1:0/3696434569 learned_addr learned my addr 172.21.15.1:0/3696434569 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:56.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.624+0000 7f9138c00700 1 --2- 172.21.15.1:0/3696434569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9134108a30 0x7f913419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.624+0000 7f9133400700 1 -- 172.21.15.1:0/3696434569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9134108a30 msgr2=0x7f913419fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:56.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.624+0000 7f9133400700 1 --2- 172.21.15.1:0/3696434569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9134108a30 0x7f913419fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.624+0000 7f9133400700 1 -- 172.21.15.1:0/3696434569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f913406a750 msgr2=0x7f913419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:56.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.625+0000 7f9133400700 1 --2- 172.21.15.1:0/3696434569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f913406a750 0x7f913419b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.625+0000 7f9133400700 1 -- 172.21.15.1:0/3696434569 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91341a0360 con 0x7f913406b0f0 2024-04-26T02:03:56.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.625+0000 7f9133e00700 1 --2- 172.21.15.1:0/3696434569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f913406a750 0x7f913419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:56.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.625+0000 7f9133400700 1 --2- 172.21.15.1:0/3696434569 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f913406b0f0 0x7f913419b890 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f912000e9a0 tx=0x7f912000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:56.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.625+0000 7f9138c00700 1 --2- 172.21.15.1:0/3696434569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9134108a30 0x7f913419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:56.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.625+0000 7f9130c00700 1 -- 172.21.15.1:0/3696434569 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912000ca10 con 0x7f913406b0f0 2024-04-26T02:03:56.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.625+0000 7f9130c00700 1 -- 172.21.15.1:0/3696434569 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f912000cbb0 con 0x7f913406b0f0 2024-04-26T02:03:56.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.625+0000 7f9130c00700 1 -- 172.21.15.1:0/3696434569 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9120018680 con 0x7f913406b0f0 2024-04-26T02:03:56.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.625+0000 7f913ac00700 1 -- 172.21.15.1:0/3696434569 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91341a0650 con 0x7f913406b0f0 2024-04-26T02:03:56.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.626+0000 7f913ac00700 1 -- 172.21.15.1:0/3696434569 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91341a7ed0 con 0x7f913406b0f0 2024-04-26T02:03:56.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.627+0000 7f913ac00700 1 -- 172.21.15.1:0/3696434569 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9134069730 con 0x7f913406b0f0 2024-04-26T02:03:56.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.629+0000 7f9130c00700 1 -- 172.21.15.1:0/3696434569 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f9120010430 con 0x7f913406b0f0 2024-04-26T02:03:56.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.630+0000 7f9130c00700 1 --2- 172.21.15.1:0/3696434569 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f91180a3e30 0x7f91180a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.630+0000 7f9130c00700 1 -- 172.21.15.1:0/3696434569 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f91200d0b20 con 0x7f913406b0f0 2024-04-26T02:03:56.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.631+0000 7f9133e00700 1 --2- 172.21.15.1:0/3696434569 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f91180a3e30 0x7f91180a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.631+0000 7f9133e00700 1 --2- 172.21.15.1:0/3696434569 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f91180a3e30 0x7f91180a62f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9124006010 tx=0x7f912400b260 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:56.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.635+0000 7f9130c00700 1 -- 172.21.15.1:0/3696434569 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9120053ff0 con 0x7f913406b0f0 2024-04-26T02:03:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.761+0000 7feb3a400640 1 -- 172.21.15.1:0/595625218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb341043c0 msgr2=0x7feb341047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.761+0000 7feb3a400640 1 --2- 172.21.15.1:0/595625218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb341043c0 0x7feb341047a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7feb24009900 tx=0x7feb2402a3f0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.762+0000 7feb3a400640 1 -- 172.21.15.1:0/595625218 shutdown_connections 2024-04-26T02:03:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.762+0000 7feb3a400640 1 --2- 172.21.15.1:0/595625218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb34109490 0x7feb34111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.762+0000 7feb3a400640 1 --2- 172.21.15.1:0/595625218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb34104d70 0x7feb34108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.762+0000 7feb3a400640 1 --2- 172.21.15.1:0/595625218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb341043c0 0x7feb341047a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.762+0000 7feb3a400640 1 -- 172.21.15.1:0/595625218 >> 172.21.15.1:0/595625218 conn(0x7feb340ffc40 msgr2=0x7feb34102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.762+0000 7feb3a400640 1 -- 172.21.15.1:0/595625218 shutdown_connections 2024-04-26T02:03:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.762+0000 7feb3a400640 1 -- 172.21.15.1:0/595625218 wait complete. 2024-04-26T02:03:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.763+0000 7feb3a400640 1 Processor -- start 2024-04-26T02:03:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.763+0000 7feb3a400640 1 -- start start 2024-04-26T02:03:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.764+0000 7feb3a400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb341043c0 0x7feb341a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.764+0000 7feb3a400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb34109490 0x7feb341a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.764+0000 7feb3a400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb3419a790 0x7feb3419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.764+0000 7feb3a400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feb341143d0 con 0x7feb34109490 2024-04-26T02:03:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.764+0000 7feb3a400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feb34114250 con 0x7feb341043c0 2024-04-26T02:03:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.764+0000 7feb3a400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feb34114550 con 0x7feb3419a790 2024-04-26T02:03:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.764+0000 7feb32a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb34109490 0x7feb341a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.764+0000 7feb33400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb341043c0 0x7feb341a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.764+0000 7feb33e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb3419a790 0x7feb3419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.764+0000 7feb32a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb34109490 0x7feb341a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49618/0 (socket says 172.21.15.1:49618) 2024-04-26T02:03:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.764+0000 7feb32a00640 1 -- 172.21.15.1:0/425896867 learned_addr learned my addr 172.21.15.1:0/425896867 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.765+0000 7feb33e00640 1 -- 172.21.15.1:0/425896867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb341043c0 msgr2=0x7feb341a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.765+0000 7feb33e00640 1 --2- 172.21.15.1:0/425896867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb341043c0 0x7feb341a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.765+0000 7feb33e00640 1 -- 172.21.15.1:0/425896867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb34109490 msgr2=0x7feb341a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.765+0000 7feb33e00640 1 --2- 172.21.15.1:0/425896867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb34109490 0x7feb341a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.765+0000 7feb33e00640 1 -- 172.21.15.1:0/425896867 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3419b320 con 0x7feb3419a790 2024-04-26T02:03:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.765+0000 7feb33e00640 1 --2- 172.21.15.1:0/425896867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb3419a790 0x7feb3419ac40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009fd0 tx=0x7feb2c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.765+0000 7feb2be00640 1 -- 172.21.15.1:0/425896867 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c00ce20 con 0x7feb3419a790 2024-04-26T02:03:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.765+0000 7feb2be00640 1 -- 172.21.15.1:0/425896867 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb2c005020 con 0x7feb3419a790 2024-04-26T02:03:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.765+0000 7feb2be00640 1 -- 172.21.15.1:0/425896867 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c010690 con 0x7feb3419a790 2024-04-26T02:03:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.765+0000 7feb3a400640 1 -- 172.21.15.1:0/425896867 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb3419b610 con 0x7feb3419a790 2024-04-26T02:03:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.766+0000 7feb3a400640 1 -- 172.21.15.1:0/425896867 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb341a77a0 con 0x7feb3419a790 2024-04-26T02:03:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.769+0000 7feb2be00640 1 -- 172.21.15.1:0/425896867 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7feb2c0054d0 con 0x7feb3419a790 2024-04-26T02:03:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.770+0000 7feb3d179740 1 Processor -- start 2024-04-26T02:03:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.771+0000 7feb2be00640 1 --2- 172.21.15.1:0/425896867 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7feb180a3d10 0x7feb180a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.771+0000 7feb3d179740 1 -- start start 2024-04-26T02:03:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.771+0000 7feb2be00640 1 -- 172.21.15.1:0/425896867 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7feb2c0d0f10 con 0x7feb3419a790 2024-04-26T02:03:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.771+0000 7feb32a00640 1 --2- 172.21.15.1:0/425896867 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7feb180a3d10 0x7feb180a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.771+0000 7feb3d179740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d30d820 0x56237d30dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.771+0000 7feb3d179740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30e140 0x56237d3184c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.771+0000 7feb3d179740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d318a00 0x56237d31adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.771+0000 7feb3d179740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56237d273900 con 0x56237d318a00 2024-04-26T02:03:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.771+0000 7feb3d179740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56237d219ef0 con 0x56237d30e140 2024-04-26T02:03:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.771+0000 7feb3d179740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56237d26f8c0 con 0x56237d30d820 2024-04-26T02:03:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.772+0000 7feb33400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d30d820 0x56237d30dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.772+0000 7feb33400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d30d820 0x56237d30dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:38764/0 (socket says 172.21.15.1:38764) 2024-04-26T02:03:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.772+0000 7feb33400640 1 -- 172.21.15.1:0/1602034190 learned_addr learned my addr 172.21.15.1:0/1602034190 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.772+0000 7feb33e00640 1 --2- 172.21.15.1:0/1602034190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d318a00 0x56237d31adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.772+0000 7feb32a00640 1 --2- 172.21.15.1:0/1602034190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30e140 0x56237d3184c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.772+0000 7feb33400640 1 -- 172.21.15.1:0/1602034190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30e140 msgr2=0x56237d3184c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.772+0000 7feb33400640 1 --2- 172.21.15.1:0/1602034190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30e140 0x56237d3184c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.772+0000 7feb33400640 1 -- 172.21.15.1:0/1602034190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d318a00 msgr2=0x56237d31adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.772+0000 7feb33400640 1 --2- 172.21.15.1:0/1602034190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d318a00 0x56237d31adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.772+0000 7feb33400640 1 -- 172.21.15.1:0/1602034190 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56237cf86210 con 0x56237d30d820 2024-04-26T02:03:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.772+0000 7feb32a00640 1 --2- 172.21.15.1:0/425896867 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7feb180a3d10 0x7feb180a61d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7feb200095f0 tx=0x7feb20005f90 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.773+0000 7feb33400640 1 --2- 172.21.15.1:0/1602034190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d30d820 0x56237d30dc00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7feb24009900 tx=0x7feb24032b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c44a252f2e79266b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.773+0000 7feb28c00640 1 -- 172.21.15.1:0/1602034190 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24040070 con 0x56237d30d820 2024-04-26T02:03:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.773+0000 7feb28c00640 1 -- 172.21.15.1:0/1602034190 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb24004030 con 0x56237d30d820 2024-04-26T02:03:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.773+0000 7feb28c00640 1 -- 172.21.15.1:0/1602034190 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24035040 con 0x56237d30d820 2024-04-26T02:03:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.774+0000 7feb3d179740 1 -- 172.21.15.1:0/1602034190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d30d820 msgr2=0x56237d30dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.774+0000 7feb3d179740 1 --2- 172.21.15.1:0/1602034190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d30d820 0x56237d30dc00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7feb24009900 tx=0x7feb24032b20 comp rx=0 tx=0).stop 2024-04-26T02:03:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.774+0000 7feb3d179740 1 -- 172.21.15.1:0/1602034190 shutdown_connections 2024-04-26T02:03:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.774+0000 7feb3d179740 1 --2- 172.21.15.1:0/1602034190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d318a00 0x56237d31adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.774+0000 7feb3d179740 1 --2- 172.21.15.1:0/1602034190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30e140 0x56237d3184c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.774+0000 7feb3d179740 1 --2- 172.21.15.1:0/1602034190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d30d820 0x56237d30dc00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.774+0000 7feb3d179740 1 -- 172.21.15.1:0/1602034190 >> 172.21.15.1:0/1602034190 conn(0x56237d309d30 msgr2=0x56237d30a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.774+0000 7feb3d179740 1 -- 172.21.15.1:0/1602034190 shutdown_connections 2024-04-26T02:03:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.774+0000 7feb3d179740 1 -- 172.21.15.1:0/1602034190 wait complete. 2024-04-26T02:03:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.774+0000 7feb3d179740 1 -- start start 2024-04-26T02:03:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.775+0000 7feb3d179740 1 Processor -- start 2024-04-26T02:03:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.775+0000 7feb3d179740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30d820 0x56237d3159f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.775+0000 7feb3d179740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d30e140 0x56237d315f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.775+0000 7feb3d179740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d318a00 0x56237d316470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.775+0000 7feb3d179740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56237d31b5d0 con 0x56237d30e140 2024-04-26T02:03:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.775+0000 7feb3d179740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56237d22d310 con 0x56237d30d820 2024-04-26T02:03:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.775+0000 7feb3d179740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56237d31b750 con 0x56237d318a00 2024-04-26T02:03:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.775+0000 7feb3d179740 10 client.0 _ll_register_callbacks cb 0x56237d1fbcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.775+0000 7feb32a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d30e140 0x56237d315f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.775+0000 7feb33400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30d820 0x56237d3159f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.776+0000 7feb33400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30d820 0x56237d3159f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:34896/0 (socket says 172.21.15.1:34896) 2024-04-26T02:03:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.776+0000 7feb33400640 1 -- 172.21.15.1:0/2223919287 learned_addr learned my addr 172.21.15.1:0/2223919287 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.776+0000 7feb33e00640 1 --2- 172.21.15.1:0/2223919287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d318a00 0x56237d316470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.776+0000 7feb33400640 1 -- 172.21.15.1:0/2223919287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d318a00 msgr2=0x56237d316470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.776+0000 7feb33400640 1 --2- 172.21.15.1:0/2223919287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d318a00 0x56237d316470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.776+0000 7feb33400640 1 -- 172.21.15.1:0/2223919287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d30e140 msgr2=0x56237d315f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.776+0000 7feb33400640 1 --2- 172.21.15.1:0/2223919287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d30e140 0x56237d315f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.776+0000 7feb33400640 1 -- 172.21.15.1:0/2223919287 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56237d052620 con 0x56237d30d820 2024-04-26T02:03:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.776+0000 7feb33400640 1 --2- 172.21.15.1:0/2223919287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30d820 0x56237d3159f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7feb24032ec0 tx=0x7feb24032ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37ce13bd15ac282c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.776+0000 7feb1d600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:03:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.777+0000 7feb1d600640 1 -- 172.21.15.1:0/2223919287 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24040070 con 0x56237d30d820 2024-04-26T02:03:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.777+0000 7feb3d179740 1 -- 172.21.15.1:0/2223919287 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb24004030 con 0x56237d30d820 2024-04-26T02:03:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.777+0000 7feb1d600640 1 -- 172.21.15.1:0/2223919287 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7feb24005230 con 0x56237d30d820 2024-04-26T02:03:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.777+0000 7feb1d600640 1 -- 172.21.15.1:0/2223919287 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7feb24056050 con 0x56237d30d820 2024-04-26T02:03:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.777+0000 7feb3d179740 10 client.26633 fetch_fsmap learned FSMap version 36 2024-04-26T02:03:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.777+0000 7feb3d179740 1 -- 172.21.15.1:0/2223919287 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56237d327890 con 0x56237d30d820 2024-04-26T02:03:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.777+0000 7feb1d600640 1 -- 172.21.15.1:0/2223919287 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb24004030 con 0x56237d30d820 2024-04-26T02:03:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.778+0000 7feb1d600640 1 -- 172.21.15.1:0/2223919287 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7feb24004b90 con 0x56237d30d820 2024-04-26T02:03:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.778+0000 7feb3d179740 10 client.26633 fetch_fsmap finished waiting for FSMap version 36 2024-04-26T02:03:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.778+0000 7feb3d179740 10 client.26633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:03:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.778+0000 7feb3d179740 20 client.26633 populate_metadata read hostname 'smithi001' 2024-04-26T02:03:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.778+0000 7feb3d179740 1 --2- 172.21.15.1:0/2223919287 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56237d328be0 0x56237d32afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.778+0000 7feb3d179740 4 client.26633 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:03:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.778+0000 7feb3d179740 1 -- 172.21.15.1:0/2223919287 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56237d2878b0 con 0x56237d328be0 2024-04-26T02:03:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.778+0000 7feb32a00640 1 --2- 172.21.15.1:0/2223919287 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56237d328be0 0x56237d32afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.779+0000 7feb32a00640 1 --2- 172.21.15.1:0/2223919287 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56237d328be0 0x56237d32afd0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6b22732199146c1 server_cookie=b5b1c8d77f2d1c5b in_seq=0 out_seq=0 2024-04-26T02:03:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.779+0000 7feb1d600640 10 client.26633 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.780+0000 7feb1d600640 1 -- 172.21.15.1:0/2223919287 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56237d2878b0 con 0x56237d328be0 2024-04-26T02:03:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.780+0000 7feb1d600640 10 client.26633 handle_command_reply: tid=0 2024-04-26T02:03:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.780+0000 7feb3d179740 1 client.26633 shutdown 2024-04-26T02:03:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.780+0000 7feb3d179740 1 -- 172.21.15.1:0/2223919287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30d820 msgr2=0x56237d3159f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.780+0000 7feb3d179740 1 --2- 172.21.15.1:0/2223919287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30d820 0x56237d3159f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7feb24032ec0 tx=0x7feb24032ef0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.780+0000 7feb3d179740 1 -- 172.21.15.1:0/2223919287 shutdown_connections 2024-04-26T02:03:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.780+0000 7feb3d179740 1 --2- 172.21.15.1:0/2223919287 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56237d328be0 0x56237d32afd0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.780+0000 7feb3d179740 1 --2- 172.21.15.1:0/2223919287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d318a00 0x56237d316470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.780+0000 7feb3d179740 1 --2- 172.21.15.1:0/2223919287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d30e140 0x56237d315f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.780+0000 7feb3d179740 1 --2- 172.21.15.1:0/2223919287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30d820 0x56237d3159f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.780+0000 7feb1d600640 0 client.26633 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.780+0000 7feb3d179740 1 -- 172.21.15.1:0/2223919287 >> 172.21.15.1:0/2223919287 conn(0x56237d309d30 msgr2=0x56237d30fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.781+0000 7feb3d179740 1 -- 172.21.15.1:0/2223919287 shutdown_connections 2024-04-26T02:03:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.781+0000 7feb3d179740 1 -- 172.21.15.1:0/2223919287 wait complete. 2024-04-26T02:03:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.781+0000 7feb3d179740 20 client.26633 trim_cache size 0 max 16384 2024-04-26T02:03:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.799+0000 7feb3d179740 1 Processor -- start 2024-04-26T02:03:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.799+0000 7feb3d179740 1 -- start start 2024-04-26T02:03:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.800+0000 7feb3d179740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30d820 0x56237d324490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.800+0000 7feb3d179740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d30e140 0x56237d330600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.800+0000 7feb3d179740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d318a00 0x56237d330b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.800+0000 7feb3d179740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feb341b27b0 con 0x56237d30e140 2024-04-26T02:03:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.800+0000 7feb3d179740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feb341b0050 con 0x56237d30d820 2024-04-26T02:03:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.800+0000 7feb3d179740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feb341adec0 con 0x56237d318a00 2024-04-26T02:03:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.800+0000 7feb32a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d30e140 0x56237d330600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.800+0000 7feb33e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d318a00 0x56237d330b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.800+0000 7feb33e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d318a00 0x56237d330b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:38774/0 (socket says 172.21.15.1:38774) 2024-04-26T02:03:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.800+0000 7feb33400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30d820 0x56237d324490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.800+0000 7feb33e00640 1 -- 172.21.15.1:0/118763883 learned_addr learned my addr 172.21.15.1:0/118763883 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.800+0000 7feb32a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d30e140 0x56237d330600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49650/0 (socket says 172.21.15.1:49650) 2024-04-26T02:03:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.801+0000 7feb33e00640 1 -- 172.21.15.1:0/118763883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30d820 msgr2=0x56237d324490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.801+0000 7feb33e00640 1 --2- 172.21.15.1:0/118763883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30d820 0x56237d324490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.801+0000 7feb33e00640 1 -- 172.21.15.1:0/118763883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d30e140 msgr2=0x56237d330600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.801+0000 7feb33e00640 1 --2- 172.21.15.1:0/118763883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d30e140 0x56237d330600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.801+0000 7feb33e00640 1 -- 172.21.15.1:0/118763883 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56237d313f40 con 0x56237d318a00 2024-04-26T02:03:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.801+0000 7feb33e00640 1 --2- 172.21.15.1:0/118763883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d318a00 0x56237d330b40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0ce570 tx=0x7feb2c0bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bfb4f4290364bda server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.801+0000 7feb1fe00640 1 -- 172.21.15.1:0/118763883 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0ba020 con 0x56237d318a00 2024-04-26T02:03:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.801+0000 7feb1fe00640 1 -- 172.21.15.1:0/118763883 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb2c0a5a90 con 0x56237d318a00 2024-04-26T02:03:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.801+0000 7feb1fe00640 1 -- 172.21.15.1:0/118763883 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0ce850 con 0x56237d318a00 2024-04-26T02:03:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.802+0000 7feb3d179740 1 -- 172.21.15.1:0/118763883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d318a00 msgr2=0x56237d330b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.802+0000 7feb3d179740 1 --2- 172.21.15.1:0/118763883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d318a00 0x56237d330b40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0ce570 tx=0x7feb2c0bd040 comp rx=0 tx=0).stop 2024-04-26T02:03:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.802+0000 7feb3d179740 1 -- 172.21.15.1:0/118763883 shutdown_connections 2024-04-26T02:03:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.802+0000 7feb3d179740 1 --2- 172.21.15.1:0/118763883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d318a00 0x56237d330b40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.802+0000 7feb3d179740 1 --2- 172.21.15.1:0/118763883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d30e140 0x56237d330600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.802+0000 7feb3d179740 1 --2- 172.21.15.1:0/118763883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30d820 0x56237d324490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.802+0000 7feb3d179740 1 -- 172.21.15.1:0/118763883 >> 172.21.15.1:0/118763883 conn(0x56237d309d30 msgr2=0x56237d304dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.802+0000 7feb3d179740 1 -- 172.21.15.1:0/118763883 shutdown_connections 2024-04-26T02:03:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.802+0000 7feb3d179740 1 -- 172.21.15.1:0/118763883 wait complete. 2024-04-26T02:03:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.803+0000 7feb3d179740 1 -- start start 2024-04-26T02:03:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.803+0000 7feb3d179740 1 Processor -- start 2024-04-26T02:03:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.803+0000 7feb3d179740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30d820 0x56237d31e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.803+0000 7feb3d179740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d30e140 0x56237d31eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.803+0000 7feb3d179740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d318a00 0x56237d31f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.803+0000 7feb3d179740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feb341ae510 con 0x56237d318a00 2024-04-26T02:03:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.803+0000 7feb3d179740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feb341af980 con 0x56237d30d820 2024-04-26T02:03:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.803+0000 7feb3d179740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feb341aebe0 con 0x56237d30e140 2024-04-26T02:03:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.803+0000 7feb3d179740 10 client.0 _ll_register_callbacks cb 0x7feb24009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:03:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.804+0000 7feb33e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d318a00 0x56237d31f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.804+0000 7feb33400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30d820 0x56237d31e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.804+0000 7feb32a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d30e140 0x56237d31eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.804+0000 7feb33400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30d820 0x56237d31e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:34908/0 (socket says 172.21.15.1:34908) 2024-04-26T02:03:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.804+0000 7feb33400640 1 -- 172.21.15.1:0/2698268200 learned_addr learned my addr 172.21.15.1:0/2698268200 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:03:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.804+0000 7feb33400640 1 -- 172.21.15.1:0/2698268200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d30e140 msgr2=0x56237d31eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.804+0000 7feb33400640 1 --2- 172.21.15.1:0/2698268200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d30e140 0x56237d31eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.804+0000 7feb33400640 1 -- 172.21.15.1:0/2698268200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d318a00 msgr2=0x56237d31f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:03:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.804+0000 7feb33400640 1 --2- 172.21.15.1:0/2698268200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d318a00 0x56237d31f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.804+0000 7feb33400640 1 -- 172.21.15.1:0/2698268200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56237d38e080 con 0x56237d30d820 2024-04-26T02:03:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.804+0000 7feb33e00640 1 --2- 172.21.15.1:0/2698268200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d318a00 0x56237d31f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.804+0000 7feb32a00640 1 --2- 172.21.15.1:0/2698268200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d30e140 0x56237d31eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:03:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.805+0000 7feb33400640 1 --2- 172.21.15.1:0/2698268200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30d820 0x56237d31e5a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7feb24004fe0 tx=0x7feb24032f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27d187d13b11e18e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:03:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.805+0000 7feb1d600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:03:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.805+0000 7feb1d600640 1 -- 172.21.15.1:0/2698268200 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24040050 con 0x56237d30d820 2024-04-26T02:03:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.805+0000 7feb1d600640 1 -- 172.21.15.1:0/2698268200 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7feb24005100 con 0x56237d30d820 2024-04-26T02:03:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.805+0000 7feb3d179740 1 -- 172.21.15.1:0/2698268200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb2c0a5a90 con 0x56237d30d820 2024-04-26T02:03:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.805+0000 7feb1d600640 1 -- 172.21.15.1:0/2698268200 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7feb24068050 con 0x56237d30d820 2024-04-26T02:03:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.805+0000 7feb3d179740 10 client.26639 fetch_fsmap learned FSMap version 36 2024-04-26T02:03:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.805+0000 7feb3d179740 1 -- 172.21.15.1:0/2698268200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56237d38f350 con 0x56237d30d820 2024-04-26T02:03:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.806+0000 7feb1d600640 1 -- 172.21.15.1:0/2698268200 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb2c0a5a90 con 0x56237d30d820 2024-04-26T02:03:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.806+0000 7feb1d600640 1 -- 172.21.15.1:0/2698268200 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7feb2403c440 con 0x56237d30d820 2024-04-26T02:03:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.806+0000 7feb3d179740 10 client.26639 fetch_fsmap finished waiting for FSMap version 36 2024-04-26T02:03:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.806+0000 7feb3d179740 10 client.26639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:03:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.806+0000 7feb3d179740 20 client.26639 populate_metadata read hostname 'smithi001' 2024-04-26T02:03:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.806+0000 7feb3d179740 1 --2- 172.21.15.1:0/2698268200 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56237d38fa00 0x56237d391df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:03:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.806+0000 7feb3d179740 4 client.26639 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:03:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.806+0000 7feb3d179740 1 -- 172.21.15.1:0/2698268200 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56237d281320 con 0x56237d38fa00 2024-04-26T02:03:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.806+0000 7feb32a00640 1 --2- 172.21.15.1:0/2698268200 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56237d38fa00 0x56237d391df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:03:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.807+0000 7feb32a00640 1 --2- 172.21.15.1:0/2698268200 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56237d38fa00 0x56237d391df0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a4652c714b125b8 server_cookie=24d94c3c58d36670 in_seq=0 out_seq=0 2024-04-26T02:03:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.807+0000 7feb1d600640 10 client.26639 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.820+0000 7feb1d600640 1 -- 172.21.15.1:0/2698268200 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56237d281320 con 0x56237d38fa00 2024-04-26T02:03:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.820+0000 7feb1d600640 10 client.26639 handle_command_reply: tid=0 2024-04-26T02:03:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.820+0000 7feb3d179740 1 client.26639 shutdown 2024-04-26T02:03:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.820+0000 7feb3d179740 1 -- 172.21.15.1:0/2698268200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30d820 msgr2=0x56237d31e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.820+0000 7feb3d179740 1 --2- 172.21.15.1:0/2698268200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30d820 0x56237d31e5a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7feb24004fe0 tx=0x7feb24032f60 comp rx=0 tx=0).stop 2024-04-26T02:03:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.820+0000 7feb3d179740 1 -- 172.21.15.1:0/2698268200 shutdown_connections 2024-04-26T02:03:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.820+0000 7feb3d179740 1 --2- 172.21.15.1:0/2698268200 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x56237d38fa00 0x56237d391df0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.820+0000 7feb3d179740 1 --2- 172.21.15.1:0/2698268200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56237d318a00 0x56237d31f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.820+0000 7feb3d179740 1 --2- 172.21.15.1:0/2698268200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56237d30e140 0x56237d31eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.820+0000 7feb3d179740 1 --2- 172.21.15.1:0/2698268200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56237d30d820 0x56237d31e5a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.820+0000 7feb1d600640 0 client.26639 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:03:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.820+0000 7feb3d179740 1 -- 172.21.15.1:0/2698268200 >> 172.21.15.1:0/2698268200 conn(0x56237d309d30 msgr2=0x56237d315990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.821+0000 7feb3d179740 1 -- 172.21.15.1:0/2698268200 shutdown_connections 2024-04-26T02:03:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.821+0000 7feb3d179740 1 -- 172.21.15.1:0/2698268200 wait complete. 2024-04-26T02:03:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.821+0000 7feb3d179740 20 client.26639 trim_cache size 0 max 16384 2024-04-26T02:03:56.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T02:03:56.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T02:03:56.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T02:03:56.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T02:03:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.822+0000 7feb3a400640 1 -- 172.21.15.1:0/425896867 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7feb180a3d10 msgr2=0x7feb180a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.822+0000 7feb3a400640 1 --2- 172.21.15.1:0/425896867 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7feb180a3d10 0x7feb180a61d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7feb200095f0 tx=0x7feb20005f90 comp rx=0 tx=0).stop 2024-04-26T02:03:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.822+0000 7feb3a400640 1 -- 172.21.15.1:0/425896867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb3419a790 msgr2=0x7feb3419ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.822+0000 7feb3a400640 1 --2- 172.21.15.1:0/425896867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb3419a790 0x7feb3419ac40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009fd0 tx=0x7feb2c00ef10 comp rx=0 tx=0).stop 2024-04-26T02:03:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.822+0000 7feb3a400640 1 -- 172.21.15.1:0/425896867 shutdown_connections 2024-04-26T02:03:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.822+0000 7feb3a400640 1 --2- 172.21.15.1:0/425896867 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7feb180a3d10 0x7feb180a61d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.822+0000 7feb3a400640 1 --2- 172.21.15.1:0/425896867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb3419a790 0x7feb3419ac40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.822+0000 7feb3a400640 1 --2- 172.21.15.1:0/425896867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb34109490 0x7feb341a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.822+0000 7feb3a400640 1 --2- 172.21.15.1:0/425896867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb341043c0 0x7feb341a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.822+0000 7feb3a400640 1 -- 172.21.15.1:0/425896867 >> 172.21.15.1:0/425896867 conn(0x7feb340ffc40 msgr2=0x7feb34100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.822+0000 7feb3a400640 1 -- 172.21.15.1:0/425896867 shutdown_connections 2024-04-26T02:03:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:03:56.822+0000 7feb3a400640 1 -- 172.21.15.1:0/425896867 wait complete. 2024-04-26T02:03:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.960+0000 7f913ac00700 1 -- 172.21.15.1:0/3696434569 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f913402df80 con 0x7f913406b0f0 2024-04-26T02:03:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.961+0000 7f9130c00700 1 -- 172.21.15.1:0/3696434569 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11840 (secure 0 0 0) 0x7f912009d130 con 0x7f913406b0f0 2024-04-26T02:03:56.960 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T02:03:56.960 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T02:03:36.329802+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26497":{"gid":26497,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/334804458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":334804458},{"type":"v1","addr":"172.21.15.173:6835","nonce":334804458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24518,24499,24514]},"id":1}]} 2024-04-26T02:03:56.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.964+0000 7f913ac00700 1 -- 172.21.15.1:0/3696434569 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f91180a3e30 msgr2=0x7f91180a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:56.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.964+0000 7f913ac00700 1 --2- 172.21.15.1:0/3696434569 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f91180a3e30 0x7f91180a62f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9124006010 tx=0x7f912400b260 comp rx=0 tx=0).stop 2024-04-26T02:03:56.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.964+0000 7f913ac00700 1 -- 172.21.15.1:0/3696434569 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f913406b0f0 msgr2=0x7f913419b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:03:56.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.964+0000 7f913ac00700 1 --2- 172.21.15.1:0/3696434569 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f913406b0f0 0x7f913419b890 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f912000e9a0 tx=0x7f912000ecb0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.964+0000 7f913ac00700 1 -- 172.21.15.1:0/3696434569 shutdown_connections 2024-04-26T02:03:56.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.964+0000 7f913ac00700 1 --2- 172.21.15.1:0/3696434569 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f91180a3e30 0x7f91180a62f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.964+0000 7f913ac00700 1 --2- 172.21.15.1:0/3696434569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9134108a30 0x7f913419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.964+0000 7f913ac00700 1 --2- 172.21.15.1:0/3696434569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f913406a750 0x7f913419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.964+0000 7f913ac00700 1 --2- 172.21.15.1:0/3696434569 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f913406b0f0 0x7f913419b890 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:03:56.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.964+0000 7f913ac00700 1 -- 172.21.15.1:0/3696434569 >> 172.21.15.1:0/3696434569 conn(0x7f91340788f0 msgr2=0x7f913410de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:03:56.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.964+0000 7f913ac00700 1 -- 172.21.15.1:0/3696434569 shutdown_connections 2024-04-26T02:03:56.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:03:56.964+0000 7f913ac00700 1 -- 172.21.15.1:0/3696434569 wait complete. 2024-04-26T02:03:56.965 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 36 2024-04-26T02:03:57.308 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:57 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3696434569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:03:57.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:57 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3696434569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:03:57.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:57 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3696434569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:03:57.734 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24518 quiesce db --set-id 7ed9c2aa --release --await' 2024-04-26T02:03:57.734 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24518 quiesce db --set-id 7ed9c2aa --release --await 2024-04-26T02:03:58.173 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:03:58.320 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:03:58 smithi001 ceph-mon[27822]: pgmap v386: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-26T02:03:58.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:03:58 smithi097 ceph-mon[32853]: pgmap v386: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-26T02:03:58.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:03:58 smithi173 ceph-mon[32553]: pgmap v386: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-26T02:04:00.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.465+0000 7fd46ee00700 1 -- 172.21.15.1:0/1746042474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4681092d0 msgr2=0x7fd4681096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:00.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.465+0000 7fd46ee00700 1 --2- 172.21.15.1:0/1746042474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4681092d0 0x7fd4681096b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd458009990 tx=0x7fd458009ca0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.465+0000 7fd46ee00700 1 -- 172.21.15.1:0/1746042474 shutdown_connections 2024-04-26T02:04:00.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.465+0000 7fd46ee00700 1 --2- 172.21.15.1:0/1746042474 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd46810e3b0 0x7fd468115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.465+0000 7fd46ee00700 1 --2- 172.21.15.1:0/1746042474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4681092d0 0x7fd4681096b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.465+0000 7fd46ee00700 1 --2- 172.21.15.1:0/1746042474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd468109c80 0x7fd46810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.465+0000 7fd46ee00700 1 -- 172.21.15.1:0/1746042474 >> 172.21.15.1:0/1746042474 conn(0x7fd468076210 msgr2=0x7fd468107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:00.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.465+0000 7fd46ee00700 1 -- 172.21.15.1:0/1746042474 shutdown_connections 2024-04-26T02:04:00.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.465+0000 7fd46ee00700 1 -- 172.21.15.1:0/1746042474 wait complete. 2024-04-26T02:04:00.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:00 smithi001 ceph-mon[27822]: pgmap v387: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-26T02:04:00.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.466+0000 7fd46ee00700 1 Processor -- start 2024-04-26T02:04:00.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.466+0000 7fd46ee00700 1 -- start start 2024-04-26T02:04:00.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.467+0000 7fd46ee00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4681092d0 0x7fd46819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:00.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.467+0000 7fd46ee00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd468109c80 0x7fd46819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:00.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.467+0000 7fd46ee00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd46810e3b0 0x7fd4681a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:00.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.467+0000 7fd46ee00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd468119240 con 0x7fd468109c80 2024-04-26T02:04:00.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.467+0000 7fd46ee00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd4681190c0 con 0x7fd4681092d0 2024-04-26T02:04:00.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.467+0000 7fd46ee00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd4681193c0 con 0x7fd46810e3b0 2024-04-26T02:04:00.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.467+0000 7fd467400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd468109c80 0x7fd46819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:00.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.467+0000 7fd467e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4681092d0 0x7fd46819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:00.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.467+0000 7fd467400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd468109c80 0x7fd46819fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49678/0 (socket says 172.21.15.1:49678) 2024-04-26T02:04:00.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.467+0000 7fd467400700 1 -- 172.21.15.1:0/3863114984 learned_addr learned my addr 172.21.15.1:0/3863114984 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:00.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.467+0000 7fd46ce00700 1 --2- 172.21.15.1:0/3863114984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd46810e3b0 0x7fd4681a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.468+0000 7fd467e00700 1 -- 172.21.15.1:0/3863114984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd46810e3b0 msgr2=0x7fd4681a3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.468+0000 7fd467e00700 1 --2- 172.21.15.1:0/3863114984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd46810e3b0 0x7fd4681a3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.468+0000 7fd467e00700 1 -- 172.21.15.1:0/3863114984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd468109c80 msgr2=0x7fd46819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.468+0000 7fd467e00700 1 --2- 172.21.15.1:0/3863114984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd468109c80 0x7fd46819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.468+0000 7fd467e00700 1 -- 172.21.15.1:0/3863114984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4681a4720 con 0x7fd4681092d0 2024-04-26T02:04:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.468+0000 7fd467e00700 1 --2- 172.21.15.1:0/3863114984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4681092d0 0x7fd46819f710 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd4580059d0 tx=0x7fd4580047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.469+0000 7fd464c00700 1 -- 172.21.15.1:0/3863114984 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd458027070 con 0x7fd4681092d0 2024-04-26T02:04:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.469+0000 7fd464c00700 1 -- 172.21.15.1:0/3863114984 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd45800b610 con 0x7fd4681092d0 2024-04-26T02:04:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.469+0000 7fd46ee00700 1 -- 172.21.15.1:0/3863114984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4681a49b0 con 0x7fd4681092d0 2024-04-26T02:04:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.469+0000 7fd464c00700 1 -- 172.21.15.1:0/3863114984 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4580173f0 con 0x7fd4681092d0 2024-04-26T02:04:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.469+0000 7fd46ee00700 1 -- 172.21.15.1:0/3863114984 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4681ac290 con 0x7fd4681092d0 2024-04-26T02:04:00.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.472+0000 7fd464c00700 1 -- 172.21.15.1:0/3863114984 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fd45802a070 con 0x7fd4681092d0 2024-04-26T02:04:00.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.474+0000 7fd464c00700 1 --2- 172.21.15.1:0/3863114984 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd44c0a3e30 0x7fd44c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:00.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.474+0000 7fd464c00700 1 -- 172.21.15.1:0/3863114984 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7fd4580d1720 con 0x7fd4681092d0 2024-04-26T02:04:00.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.474+0000 7fd467400700 1 --2- 172.21.15.1:0/3863114984 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd44c0a3e30 0x7fd44c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:00.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.475+0000 7fd479789b80 1 Processor -- start 2024-04-26T02:04:00.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.475+0000 7fd479789b80 1 -- start start 2024-04-26T02:04:00.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.475+0000 7fd467400700 1 --2- 172.21.15.1:0/3863114984 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd44c0a3e30 0x7fd44c0a62f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd468078270 tx=0x7fd454009260 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:00.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.475+0000 7fd479789b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3b920 0x564117e3bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:00.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.475+0000 7fd479789b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e3c240 0x564117e46bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:00.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.475+0000 7fd479789b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e470f0 0x564117e494e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:00.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.475+0000 7fd479789b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564117d8ef90 con 0x564117e3b920 2024-04-26T02:04:00.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.475+0000 7fd479789b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564117d35830 con 0x564117e3c240 2024-04-26T02:04:00.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.475+0000 7fd479789b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564117d8af50 con 0x564117e470f0 2024-04-26T02:04:00.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.476+0000 7fd467e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3b920 0x564117e3bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:00.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.476+0000 7fd467e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3b920 0x564117e3bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49690/0 (socket says 172.21.15.1:49690) 2024-04-26T02:04:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.476+0000 7fd467e00700 1 -- 172.21.15.1:0/1674927222 learned_addr learned my addr 172.21.15.1:0/1674927222 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.476+0000 7fd46ce00700 1 --2- 172.21.15.1:0/1674927222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e470f0 0x564117e494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.476+0000 7fd467400700 1 --2- 172.21.15.1:0/1674927222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e3c240 0x564117e46bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.476+0000 7fd467e00700 1 -- 172.21.15.1:0/1674927222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e470f0 msgr2=0x564117e494e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.476+0000 7fd467e00700 1 --2- 172.21.15.1:0/1674927222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e470f0 0x564117e494e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.476+0000 7fd467e00700 1 -- 172.21.15.1:0/1674927222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e3c240 msgr2=0x564117e46bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.476+0000 7fd467e00700 1 --2- 172.21.15.1:0/1674927222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e3c240 0x564117e46bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.476+0000 7fd467e00700 1 -- 172.21.15.1:0/1674927222 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564117d2b780 con 0x564117e3b920 2024-04-26T02:04:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.477+0000 7fd467e00700 1 --2- 172.21.15.1:0/1674927222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3b920 0x564117e3bd00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd45809a1a0 tx=0x7fd4580229c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0889f9851650d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.477+0000 7fd45d600700 1 -- 172.21.15.1:0/1674927222 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4580a40c0 con 0x564117e3b920 2024-04-26T02:04:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.477+0000 7fd45d600700 1 -- 172.21.15.1:0/1674927222 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4580a1b80 con 0x564117e3b920 2024-04-26T02:04:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.477+0000 7fd45d600700 1 -- 172.21.15.1:0/1674927222 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4580a5f20 con 0x564117e3b920 2024-04-26T02:04:00.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.478+0000 7fd479789b80 1 -- 172.21.15.1:0/1674927222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3b920 msgr2=0x564117e3bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:00.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.478+0000 7fd479789b80 1 --2- 172.21.15.1:0/1674927222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3b920 0x564117e3bd00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd45809a1a0 tx=0x7fd4580229c0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.478+0000 7fd479789b80 1 -- 172.21.15.1:0/1674927222 shutdown_connections 2024-04-26T02:04:00.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.478+0000 7fd479789b80 1 --2- 172.21.15.1:0/1674927222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e470f0 0x564117e494e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.478+0000 7fd479789b80 1 --2- 172.21.15.1:0/1674927222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3b920 0x564117e3bd00 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.478+0000 7fd479789b80 1 --2- 172.21.15.1:0/1674927222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e3c240 0x564117e46bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.478+0000 7fd479789b80 1 -- 172.21.15.1:0/1674927222 >> 172.21.15.1:0/1674927222 conn(0x564117d6a6a0 msgr2=0x564117e37dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:00.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.478+0000 7fd479789b80 1 -- 172.21.15.1:0/1674927222 shutdown_connections 2024-04-26T02:04:00.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.478+0000 7fd479789b80 1 -- 172.21.15.1:0/1674927222 wait complete. 2024-04-26T02:04:00.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.479+0000 7fd479789b80 1 -- start start 2024-04-26T02:04:00.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.479+0000 7fd479789b80 1 Processor -- start 2024-04-26T02:04:00.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.479+0000 7fd479789b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3b920 0x564117e46800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:00.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.479+0000 7fd479789b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e3c240 0x564117e43ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:00.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.479+0000 7fd479789b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e470f0 0x564117e440e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:00.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.479+0000 7fd479789b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564117e49a20 con 0x564117e3b920 2024-04-26T02:04:00.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.479+0000 7fd479789b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564117d48960 con 0x564117e3c240 2024-04-26T02:04:00.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.479+0000 7fd479789b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564117e49ba0 con 0x564117e470f0 2024-04-26T02:04:00.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.479+0000 7fd479789b80 10 client.0 _ll_register_callbacks cb 0x564117d27820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:04:00.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.480+0000 7fd467400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e3c240 0x564117e43ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:00.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.480+0000 7fd467e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3b920 0x564117e46800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:00.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.480+0000 7fd467400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e3c240 0x564117e43ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:34948/0 (socket says 172.21.15.1:34948) 2024-04-26T02:04:00.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.480+0000 7fd467400700 1 -- 172.21.15.1:0/4076916344 learned_addr learned my addr 172.21.15.1:0/4076916344 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:00.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.480+0000 7fd46ce00700 1 --2- 172.21.15.1:0/4076916344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e470f0 0x564117e440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.480+0000 7fd467e00700 1 -- 172.21.15.1:0/4076916344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e470f0 msgr2=0x564117e440e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.480+0000 7fd467e00700 1 --2- 172.21.15.1:0/4076916344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e470f0 0x564117e440e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.480+0000 7fd467e00700 1 -- 172.21.15.1:0/4076916344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e3c240 msgr2=0x564117e43ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.480+0000 7fd467e00700 1 --2- 172.21.15.1:0/4076916344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e3c240 0x564117e43ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.480+0000 7fd467e00700 1 -- 172.21.15.1:0/4076916344 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564117cdc9d0 con 0x564117e3b920 2024-04-26T02:04:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.481+0000 7fd467e00700 1 --2- 172.21.15.1:0/4076916344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3b920 0x564117e46800 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd458005f50 tx=0x7fd458009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f675bcf97dd72ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.481+0000 7fd452000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T02:04:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.481+0000 7fd452000700 1 -- 172.21.15.1:0/4076916344 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4580a63b0 con 0x564117e3b920 2024-04-26T02:04:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.481+0000 7fd452000700 1 -- 172.21.15.1:0/4076916344 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7fd45809da30 con 0x564117e3b920 2024-04-26T02:04:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.481+0000 7fd479789b80 1 -- 172.21.15.1:0/4076916344 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4580a1b80 con 0x564117e3b920 2024-04-26T02:04:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.481+0000 7fd452000700 1 -- 172.21.15.1:0/4076916344 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd45809dc30 con 0x564117e3b920 2024-04-26T02:04:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.481+0000 7fd479789b80 10 client.16476 fetch_fsmap learned FSMap version 36 2024-04-26T02:04:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.481+0000 7fd479789b80 1 -- 172.21.15.1:0/4076916344 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564117d25bf0 con 0x564117e3b920 2024-04-26T02:04:00.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.481+0000 7fd452000700 1 -- 172.21.15.1:0/4076916344 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4580a1b80 con 0x564117e3b920 2024-04-26T02:04:00.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.482+0000 7fd452000700 1 -- 172.21.15.1:0/4076916344 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fd45809eb00 con 0x564117e3b920 2024-04-26T02:04:00.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.482+0000 7fd479789b80 10 client.16476 fetch_fsmap finished waiting for FSMap version 36 2024-04-26T02:04:00.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.482+0000 7fd479789b80 10 client.16476 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T02:04:00.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.482+0000 7fd479789b80 20 client.16476 populate_metadata read hostname 'smithi001' 2024-04-26T02:04:00.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.482+0000 7fd479789b80 1 --2- 172.21.15.1:0/4076916344 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564117e56520 0x564117e58910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:00.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.482+0000 7fd479789b80 4 client.16476 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:04:00.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.482+0000 7fd479789b80 1 -- 172.21.15.1:0/4076916344 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564117d9c780 con 0x564117e56520 2024-04-26T02:04:00.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.482+0000 7fd467400700 1 --2- 172.21.15.1:0/4076916344 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564117e56520 0x564117e58910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:00.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.483+0000 7fd467400700 1 --2- 172.21.15.1:0/4076916344 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564117e56520 0x564117e58910 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab6d34666f645629 server_cookie=2b27127c77c1bd0d in_seq=0 out_seq=0 2024-04-26T02:04:00.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.483+0000 7fd452000700 10 client.16476 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:04:00.488 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:00 smithi097 ceph-mon[32853]: pgmap v387: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-26T02:04:00.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.484+0000 7fd452000700 1 -- 172.21.15.1:0/4076916344 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564117d9c780 con 0x564117e56520 2024-04-26T02:04:00.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.484+0000 7fd452000700 10 client.16476 handle_command_reply: tid=0 2024-04-26T02:04:00.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.484+0000 7fd479789b80 1 client.16476 shutdown 2024-04-26T02:04:00.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.484+0000 7fd479789b80 1 -- 172.21.15.1:0/4076916344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3b920 msgr2=0x564117e46800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:00.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.484+0000 7fd479789b80 1 --2- 172.21.15.1:0/4076916344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3b920 0x564117e46800 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd458005f50 tx=0x7fd458009ca0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.484+0000 7fd479789b80 1 -- 172.21.15.1:0/4076916344 shutdown_connections 2024-04-26T02:04:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.484+0000 7fd479789b80 1 --2- 172.21.15.1:0/4076916344 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564117e56520 0x564117e58910 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.484+0000 7fd479789b80 1 --2- 172.21.15.1:0/4076916344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e470f0 0x564117e440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.484+0000 7fd479789b80 1 --2- 172.21.15.1:0/4076916344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3b920 0x564117e46800 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.484+0000 7fd479789b80 1 --2- 172.21.15.1:0/4076916344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e3c240 0x564117e43ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.484+0000 7fd452000700 0 client.16476 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:04:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.484+0000 7fd479789b80 1 -- 172.21.15.1:0/4076916344 >> 172.21.15.1:0/4076916344 conn(0x564117d6a6a0 msgr2=0x564117e379c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.485+0000 7fd479789b80 1 -- 172.21.15.1:0/4076916344 shutdown_connections 2024-04-26T02:04:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.485+0000 7fd479789b80 1 -- 172.21.15.1:0/4076916344 wait complete. 2024-04-26T02:04:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.485+0000 7fd479789b80 20 client.16476 trim_cache size 0 max 16384 2024-04-26T02:04:00.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.506+0000 7fd479789b80 1 Processor -- start 2024-04-26T02:04:00.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.506+0000 7fd479789b80 1 -- start start 2024-04-26T02:04:00.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.507+0000 7fd479789b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3b920 0x564117e3f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:00.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.507+0000 7fd479789b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e3c240 0x564117e42d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:00.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.507+0000 7fd479789b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e470f0 0x564117e528e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:00.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.507+0000 7fd479789b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd4681b6240 con 0x564117e3b920 2024-04-26T02:04:00.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.507+0000 7fd479789b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd4681b3aa0 con 0x564117e470f0 2024-04-26T02:04:00.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.507+0000 7fd479789b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd4681b1910 con 0x564117e3c240 2024-04-26T02:04:00.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.507+0000 7fd467e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3b920 0x564117e3f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:00.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.507+0000 7fd467400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e3c240 0x564117e42d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:00.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.507+0000 7fd467400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e3c240 0x564117e42d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:38838/0 (socket says 172.21.15.1:38838) 2024-04-26T02:04:00.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.507+0000 7fd467400700 1 -- 172.21.15.1:0/866511699 learned_addr learned my addr 172.21.15.1:0/866511699 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:00.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.507+0000 7fd46ce00700 1 --2- 172.21.15.1:0/866511699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e470f0 0x564117e528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:00.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.508+0000 7fd467400700 1 -- 172.21.15.1:0/866511699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e470f0 msgr2=0x564117e528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.508+0000 7fd467400700 1 --2- 172.21.15.1:0/866511699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e470f0 0x564117e528e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.508+0000 7fd467400700 1 -- 172.21.15.1:0/866511699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3b920 msgr2=0x564117e3f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.508+0000 7fd467400700 1 --2- 172.21.15.1:0/866511699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3b920 0x564117e3f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.508+0000 7fd467400700 1 -- 172.21.15.1:0/866511699 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564117e44480 con 0x564117e3c240 2024-04-26T02:04:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.508+0000 7fd467400700 1 --2- 172.21.15.1:0/866511699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e3c240 0x564117e42d60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd454005950 tx=0x7fd454014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31fbb5ca40c9e29b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.508+0000 7fd45cc00700 1 -- 172.21.15.1:0/866511699 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd454003cc0 con 0x564117e3c240 2024-04-26T02:04:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.508+0000 7fd45cc00700 1 -- 172.21.15.1:0/866511699 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd454030070 con 0x564117e3c240 2024-04-26T02:04:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.509+0000 7fd45cc00700 1 -- 172.21.15.1:0/866511699 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd454029a30 con 0x564117e3c240 2024-04-26T02:04:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.509+0000 7fd479789b80 1 -- 172.21.15.1:0/866511699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e3c240 msgr2=0x564117e42d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:00.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.509+0000 7fd479789b80 1 --2- 172.21.15.1:0/866511699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e3c240 0x564117e42d60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd454005950 tx=0x7fd454014000 comp rx=0 tx=0).stop 2024-04-26T02:04:00.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.509+0000 7fd479789b80 1 -- 172.21.15.1:0/866511699 shutdown_connections 2024-04-26T02:04:00.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.509+0000 7fd479789b80 1 --2- 172.21.15.1:0/866511699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e470f0 0x564117e528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.509+0000 7fd479789b80 1 --2- 172.21.15.1:0/866511699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3b920 0x564117e3f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.509+0000 7fd479789b80 1 --2- 172.21.15.1:0/866511699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e3c240 0x564117e42d60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.509+0000 7fd479789b80 1 -- 172.21.15.1:0/866511699 >> 172.21.15.1:0/866511699 conn(0x564117d6a6a0 msgr2=0x564117e3d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:00.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.509+0000 7fd479789b80 1 -- 172.21.15.1:0/866511699 shutdown_connections 2024-04-26T02:04:00.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.510+0000 7fd479789b80 1 -- 172.21.15.1:0/866511699 wait complete. 2024-04-26T02:04:00.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.510+0000 7fd479789b80 1 -- start start 2024-04-26T02:04:00.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.510+0000 7fd479789b80 1 Processor -- start 2024-04-26T02:04:00.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.511+0000 7fd479789b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e3b920 0x564117e4edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:00.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.511+0000 7fd479789b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3c240 0x564117e4f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:00.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.511+0000 7fd479789b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e470f0 0x564117e4bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:00.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.511+0000 7fd479789b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd4681b1f60 con 0x564117e3c240 2024-04-26T02:04:00.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.511+0000 7fd479789b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd4681b33d0 con 0x564117e3b920 2024-04-26T02:04:00.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.511+0000 7fd479789b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd4681b2630 con 0x564117e470f0 2024-04-26T02:04:00.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.511+0000 7fd479789b80 10 client.0 _ll_register_callbacks cb 0x564117d27820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:04:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.511+0000 7fd467e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e3b920 0x564117e4edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.511+0000 7fd467400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3c240 0x564117e4f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.511+0000 7fd467e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e3b920 0x564117e4edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:34964/0 (socket says 172.21.15.1:34964) 2024-04-26T02:04:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.511+0000 7fd467e00700 1 -- 172.21.15.1:0/596512344 learned_addr learned my addr 172.21.15.1:0/596512344 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.511+0000 7fd46ce00700 1 --2- 172.21.15.1:0/596512344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e470f0 0x564117e4bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.511+0000 7fd467e00700 1 -- 172.21.15.1:0/596512344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e470f0 msgr2=0x564117e4bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.511+0000 7fd467e00700 1 --2- 172.21.15.1:0/596512344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e470f0 0x564117e4bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.511+0000 7fd467e00700 1 -- 172.21.15.1:0/596512344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3c240 msgr2=0x564117e4f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.511+0000 7fd467e00700 1 --2- 172.21.15.1:0/596512344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3c240 0x564117e4f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.511+0000 7fd467e00700 1 -- 172.21.15.1:0/596512344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564117e4c600 con 0x564117e3b920 2024-04-26T02:04:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.512+0000 7fd46ce00700 1 --2- 172.21.15.1:0/596512344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e470f0 0x564117e4bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:04:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.512+0000 7fd467e00700 1 --2- 172.21.15.1:0/596512344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e3b920 0x564117e4edf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd458022970 tx=0x7fd458005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cd7941ca71d0d37 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.512+0000 7fd452000700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:04:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.512+0000 7fd452000700 1 -- 172.21.15.1:0/596512344 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45809d130 con 0x564117e3b920 2024-04-26T02:04:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.512+0000 7fd452000700 1 -- 172.21.15.1:0/596512344 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7fd45809cc20 con 0x564117e3b920 2024-04-26T02:04:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.512+0000 7fd479789b80 1 -- 172.21.15.1:0/596512344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd454030070 con 0x564117e3b920 2024-04-26T02:04:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.512+0000 7fd452000700 1 -- 172.21.15.1:0/596512344 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd4580a1b80 con 0x564117e3b920 2024-04-26T02:04:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.512+0000 7fd479789b80 10 client.26657 fetch_fsmap learned FSMap version 36 2024-04-26T02:04:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.512+0000 7fd479789b80 1 -- 172.21.15.1:0/596512344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564117e4c600 con 0x564117e3b920 2024-04-26T02:04:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.512+0000 7fd452000700 1 -- 172.21.15.1:0/596512344 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd454030070 con 0x564117e3b920 2024-04-26T02:04:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.513+0000 7fd452000700 1 -- 172.21.15.1:0/596512344 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7fd4580bc340 con 0x564117e3b920 2024-04-26T02:04:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.513+0000 7fd479789b80 10 client.26657 fetch_fsmap finished waiting for FSMap version 36 2024-04-26T02:04:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.513+0000 7fd479789b80 10 client.26657 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T02:04:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.513+0000 7fd479789b80 20 client.26657 populate_metadata read hostname 'smithi001' 2024-04-26T02:04:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.513+0000 7fd479789b80 1 --2- 172.21.15.1:0/596512344 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564117ebd770 0x564117ebfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.513+0000 7fd479789b80 4 client.26657 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7ed9c2aa", "release": true, "await": true}] 2024-04-26T02:04:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.513+0000 7fd479789b80 1 -- 172.21.15.1:0/596512344 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7ed9c2aa", "release": true, "await": true}) -- 0x564117d96820 con 0x564117ebd770 2024-04-26T02:04:00.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.513+0000 7fd467400700 1 --2- 172.21.15.1:0/596512344 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564117ebd770 0x564117ebfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:00.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.514+0000 7fd467400700 1 --2- 172.21.15.1:0/596512344 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564117ebd770 0x564117ebfb60 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=352c0efa6a673dc0 server_cookie=3cdcb8608a276f1f in_seq=0 out_seq=0 2024-04-26T02:04:00.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.514+0000 7fd452000700 10 client.26657 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:04:00.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.516+0000 7fd452000700 1 -- 172.21.15.1:0/596512344 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x564117d96820 con 0x564117ebd770 2024-04-26T02:04:00.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.516+0000 7fd452000700 10 client.26657 handle_command_reply: tid=0 2024-04-26T02:04:00.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.516+0000 7fd479789b80 1 client.26657 shutdown 2024-04-26T02:04:00.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.516+0000 7fd479789b80 1 -- 172.21.15.1:0/596512344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e3b920 msgr2=0x564117e4edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:00.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.516+0000 7fd479789b80 1 --2- 172.21.15.1:0/596512344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e3b920 0x564117e4edf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd458022970 tx=0x7fd458005ad0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.516+0000 7fd479789b80 1 -- 172.21.15.1:0/596512344 shutdown_connections 2024-04-26T02:04:00.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.516+0000 7fd479789b80 1 --2- 172.21.15.1:0/596512344 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x564117ebd770 0x564117ebfb60 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.516+0000 7fd479789b80 1 --2- 172.21.15.1:0/596512344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564117e470f0 0x564117e4bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.516+0000 7fd479789b80 1 --2- 172.21.15.1:0/596512344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564117e3b920 0x564117e4edf0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.517+0000 7fd452000700 0 client.26657 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:04:00.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.516+0000 7fd479789b80 1 --2- 172.21.15.1:0/596512344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564117e3c240 0x564117e4f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.517+0000 7fd479789b80 1 -- 172.21.15.1:0/596512344 >> 172.21.15.1:0/596512344 conn(0x564117d6a6a0 msgr2=0x564117e42be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:00.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.517+0000 7fd479789b80 1 -- 172.21.15.1:0/596512344 shutdown_connections 2024-04-26T02:04:00.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.517+0000 7fd479789b80 1 -- 172.21.15.1:0/596512344 wait complete. 2024-04-26T02:04:00.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.517+0000 7fd479789b80 20 client.26657 trim_cache size 0 max 16384 2024-04-26T02:04:00.527 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T02:04:00.527 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 36, 2024-04-26T02:04:00.527 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24518, 2024-04-26T02:04:00.527 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 36, 2024-04-26T02:04:00.527 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-04-26T02:04:00.527 INFO:teuthology.orchestra.run.smithi001.stdout: "7ed9c2aa": { 2024-04-26T02:04:00.527 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 36, 2024-04-26T02:04:00.528 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-04-26T02:04:00.528 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T02:04:00.528 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-04-26T02:04:00.528 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T02:04:00.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:00.528 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-04-26T02:04:00.528 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 79.2, 2024-04-26T02:04:00.528 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-04-26T02:04:00.528 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-04-26T02:04:00.528 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-04-26T02:04:00.528 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T02:04:00.528 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-04-26T02:04:00.528 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T02:04:00.528 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:00.528 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:00.529 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:00.529 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:00.529 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:00.529 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T02:04:00.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.518+0000 7fd46ee00700 1 -- 172.21.15.1:0/3863114984 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd44c0a3e30 msgr2=0x7fd44c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:00.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.518+0000 7fd46ee00700 1 --2- 172.21.15.1:0/3863114984 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd44c0a3e30 0x7fd44c0a62f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd468078270 tx=0x7fd454009260 comp rx=0 tx=0).stop 2024-04-26T02:04:00.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.518+0000 7fd46ee00700 1 -- 172.21.15.1:0/3863114984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4681092d0 msgr2=0x7fd46819f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:00.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.518+0000 7fd46ee00700 1 --2- 172.21.15.1:0/3863114984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4681092d0 0x7fd46819f710 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd4580059d0 tx=0x7fd4580047b0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.518+0000 7fd46ee00700 1 -- 172.21.15.1:0/3863114984 shutdown_connections 2024-04-26T02:04:00.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.518+0000 7fd46ee00700 1 --2- 172.21.15.1:0/3863114984 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fd44c0a3e30 0x7fd44c0a62f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.518+0000 7fd46ee00700 1 --2- 172.21.15.1:0/3863114984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd46810e3b0 0x7fd4681a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.518+0000 7fd46ee00700 1 --2- 172.21.15.1:0/3863114984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4681092d0 0x7fd46819f710 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.518+0000 7fd46ee00700 1 --2- 172.21.15.1:0/3863114984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd468109c80 0x7fd46819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:00.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.518+0000 7fd46ee00700 1 -- 172.21.15.1:0/3863114984 >> 172.21.15.1:0/3863114984 conn(0x7fd468076210 msgr2=0x7fd46810cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:00.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.518+0000 7fd46ee00700 1 -- 172.21.15.1:0/3863114984 shutdown_connections 2024-04-26T02:04:00.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:00.518+0000 7fd46ee00700 1 -- 172.21.15.1:0/3863114984 wait complete. 2024-04-26T02:04:00.531 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T02:04:00.531 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-26T02:04:00.566 INFO:tasks.quiescer:joining Quiescers 2024-04-26T02:04:00.566 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-04-26T02:04:00.566 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:04:00.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:00 smithi173 ceph-mon[32553]: pgmap v387: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-26T02:04:01.433 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7ed9c2aa', seconds elapsed: 35.0 2024-04-26T02:04:01.433 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.25 2024-04-26T02:04:01.433 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:04:01.496 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:04:02.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:02 smithi001 ceph-mon[27822]: pgmap v388: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-04-26T02:04:02.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:02 smithi097 ceph-mon[32853]: pgmap v388: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-04-26T02:04:02.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:02 smithi173 ceph-mon[32553]: pgmap v388: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-04-26T02:04:02.825 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:04:04.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.888+0000 7fa2f6600700 1 -- 172.21.15.1:0/3628321115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2f0113c10 msgr2=0x7fa2f0116000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:04.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.888+0000 7fa2f6600700 1 --2- 172.21.15.1:0/3628321115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2f0113c10 0x7fa2f0116000 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800b0a0 tx=0x7fa2e800b3b0 comp rx=0 tx=0).stop 2024-04-26T02:04:04.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.888+0000 7fa2f6600700 1 -- 172.21.15.1:0/3628321115 shutdown_connections 2024-04-26T02:04:04.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.888+0000 7fa2f6600700 1 --2- 172.21.15.1:0/3628321115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2f0113c10 0x7fa2f0116000 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:04.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.888+0000 7fa2f6600700 1 --2- 172.21.15.1:0/3628321115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2f0074c70 0x7fa2f0075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:04.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.888+0000 7fa2f6600700 1 --2- 172.21.15.1:0/3628321115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2f0075590 0x7fa2f01136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:04.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.888+0000 7fa2f6600700 1 -- 172.21.15.1:0/3628321115 >> 172.21.15.1:0/3628321115 conn(0x7fa2f0079ba0 msgr2=0x7fa2f0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:04.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.888+0000 7fa2f6600700 1 -- 172.21.15.1:0/3628321115 shutdown_connections 2024-04-26T02:04:04.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.888+0000 7fa2f6600700 1 -- 172.21.15.1:0/3628321115 wait complete. 2024-04-26T02:04:04.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.889+0000 7fa2f6600700 1 Processor -- start 2024-04-26T02:04:04.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.889+0000 7fa2f6600700 1 -- start start 2024-04-26T02:04:04.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.890+0000 7fa2f6600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2f0074c70 0x7fa2f019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:04.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.890+0000 7fa2f6600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2f0075590 0x7fa2f019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:04.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.890+0000 7fa2f6600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2f0113c10 0x7fa2f01a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:04.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.890+0000 7fa2f6600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f0119220 con 0x7fa2f0075590 2024-04-26T02:04:04.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.890+0000 7fa2f6600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f01190a0 con 0x7fa2f0074c70 2024-04-26T02:04:04.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.890+0000 7fa2f6600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f01193a0 con 0x7fa2f0113c10 2024-04-26T02:04:04.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.890+0000 7fa2eea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2f0075590 0x7fa2f019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:04.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.890+0000 7fa2eea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2f0075590 0x7fa2f019fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51460/0 (socket says 172.21.15.1:51460) 2024-04-26T02:04:04.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.890+0000 7fa2eea00700 1 -- 172.21.15.1:0/1225276829 learned_addr learned my addr 172.21.15.1:0/1225276829 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:04.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.890+0000 7fa2ef400700 1 --2- 172.21.15.1:0/1225276829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2f0074c70 0x7fa2f019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:04.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.890+0000 7fa2efe00700 1 --2- 172.21.15.1:0/1225276829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2f0113c10 0x7fa2f01a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:04.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.891+0000 7fa2eea00700 1 -- 172.21.15.1:0/1225276829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2f0113c10 msgr2=0x7fa2f01a3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:04.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.891+0000 7fa2eea00700 1 --2- 172.21.15.1:0/1225276829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2f0113c10 0x7fa2f01a3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:04.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.891+0000 7fa2eea00700 1 -- 172.21.15.1:0/1225276829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2f0074c70 msgr2=0x7fa2f019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:04.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.891+0000 7fa2eea00700 1 --2- 172.21.15.1:0/1225276829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2f0074c70 0x7fa2f019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:04.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.891+0000 7fa2eea00700 1 -- 172.21.15.1:0/1225276829 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f01a4720 con 0x7fa2f0075590 2024-04-26T02:04:04.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.891+0000 7fa2eea00700 1 --2- 172.21.15.1:0/1225276829 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2f0075590 0x7fa2f019fc50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800eb30 tx=0x7fa2d800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:04.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.891+0000 7fa2e7e00700 1 -- 172.21.15.1:0/1225276829 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d800cbd0 con 0x7fa2f0075590 2024-04-26T02:04:04.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.891+0000 7fa2e7e00700 1 -- 172.21.15.1:0/1225276829 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d80044d0 con 0x7fa2f0075590 2024-04-26T02:04:04.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.891+0000 7fa2e7e00700 1 -- 172.21.15.1:0/1225276829 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8003e30 con 0x7fa2f0075590 2024-04-26T02:04:04.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.891+0000 7fa2f6600700 1 -- 172.21.15.1:0/1225276829 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f01a4a10 con 0x7fa2f0075590 2024-04-26T02:04:04.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.892+0000 7fa2f6600700 1 -- 172.21.15.1:0/1225276829 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f01ac290 con 0x7fa2f0075590 2024-04-26T02:04:04.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.893+0000 7fa2f6600700 1 -- 172.21.15.1:0/1225276829 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2f0069730 con 0x7fa2f0075590 2024-04-26T02:04:04.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.895+0000 7fa2e7e00700 1 -- 172.21.15.1:0/1225276829 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fa2d8004000 con 0x7fa2f0075590 2024-04-26T02:04:04.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.897+0000 7fa2e7e00700 1 --2- 172.21.15.1:0/1225276829 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fa2d40a3e30 0x7fa2d40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:04.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.897+0000 7fa2e7e00700 1 -- 172.21.15.1:0/1225276829 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7fa2d80d0af0 con 0x7fa2f0075590 2024-04-26T02:04:04.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.898+0000 7fa2ef400700 1 --2- 172.21.15.1:0/1225276829 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fa2d40a3e30 0x7fa2d40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:04.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.898+0000 7fa2ef400700 1 --2- 172.21.15.1:0/1225276829 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fa2d40a3e30 0x7fa2d40a62f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009c60 tx=0x7fa2e0009260 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:04.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:03.902+0000 7fa2e7e00700 1 -- 172.21.15.1:0/1225276829 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2d80042f0 con 0x7fa2f0075590 2024-04-26T02:04:04.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:04.187+0000 7fa2f6600700 1 -- 172.21.15.1:0/1225276829 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2f01a0a70 con 0x7fa2f0075590 2024-04-26T02:04:04.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:04.188+0000 7fa2e7e00700 1 -- 172.21.15.1:0/1225276829 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11840 (secure 0 0 0) 0x7fa2f01a0a70 con 0x7fa2f0075590 2024-04-26T02:04:04.187 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T02:04:04.187 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T02:03:36.329802+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26497":{"gid":26497,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/334804458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":334804458},{"type":"v1","addr":"172.21.15.173:6835","nonce":334804458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24518,24499,24514]},"id":1}]} 2024-04-26T02:04:04.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:04.191+0000 7fa2f6600700 1 -- 172.21.15.1:0/1225276829 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fa2d40a3e30 msgr2=0x7fa2d40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:04.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:04.191+0000 7fa2f6600700 1 --2- 172.21.15.1:0/1225276829 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fa2d40a3e30 0x7fa2d40a62f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009c60 tx=0x7fa2e0009260 comp rx=0 tx=0).stop 2024-04-26T02:04:04.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:04.191+0000 7fa2f6600700 1 -- 172.21.15.1:0/1225276829 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2f0075590 msgr2=0x7fa2f019fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:04.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:04.191+0000 7fa2f6600700 1 --2- 172.21.15.1:0/1225276829 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2f0075590 0x7fa2f019fc50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800eb30 tx=0x7fa2d800ee40 comp rx=0 tx=0).stop 2024-04-26T02:04:04.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:04.191+0000 7fa2f6600700 1 -- 172.21.15.1:0/1225276829 shutdown_connections 2024-04-26T02:04:04.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:04.191+0000 7fa2f6600700 1 --2- 172.21.15.1:0/1225276829 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fa2d40a3e30 0x7fa2d40a62f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:04.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:04.191+0000 7fa2f6600700 1 --2- 172.21.15.1:0/1225276829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2f0113c10 0x7fa2f01a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:04.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:04.191+0000 7fa2f6600700 1 --2- 172.21.15.1:0/1225276829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2f0074c70 0x7fa2f019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:04.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:04.191+0000 7fa2f6600700 1 --2- 172.21.15.1:0/1225276829 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2f0075590 0x7fa2f019fc50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:04.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:04.191+0000 7fa2f6600700 1 -- 172.21.15.1:0/1225276829 >> 172.21.15.1:0/1225276829 conn(0x7fa2f0079ba0 msgr2=0x7fa2f01129e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:04.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:04.191+0000 7fa2f6600700 1 -- 172.21.15.1:0/1225276829 shutdown_connections 2024-04-26T02:04:04.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:04.191+0000 7fa2f6600700 1 -- 172.21.15.1:0/1225276829 wait complete. 2024-04-26T02:04:04.192 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 36 2024-04-26T02:04:04.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:04 smithi097 ceph-mon[32853]: pgmap v389: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.5 KiB/s wr, 8 op/s 2024-04-26T02:04:04.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:04 smithi173 ceph-mon[32553]: pgmap v389: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.5 KiB/s wr, 8 op/s 2024-04-26T02:04:04.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:04 smithi001 ceph-mon[27822]: pgmap v389: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.5 KiB/s wr, 8 op/s 2024-04-26T02:04:05.107 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24518 quiesce db --cancel --all' 2024-04-26T02:04:05.107 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24518 quiesce db --cancel --all 2024-04-26T02:04:05.361 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:05 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1225276829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:04:05.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:05 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1225276829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:04:05.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:05 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1225276829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:04:05.599 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:04:06.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.426+0000 7f9ac1e00700 1 -- 172.21.15.1:0/47595151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9abc108f80 msgr2=0x7f9abc109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:06.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.426+0000 7f9ac1e00700 1 --2- 172.21.15.1:0/47595151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9abc108f80 0x7f9abc109400 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009990 tx=0x7f9aa8009ca0 comp rx=0 tx=0).stop 2024-04-26T02:04:06.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.426+0000 7f9ac1e00700 1 -- 172.21.15.1:0/47595151 shutdown_connections 2024-04-26T02:04:06.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.426+0000 7f9ac1e00700 1 --2- 172.21.15.1:0/47595151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9abc109940 0x7f9abc115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:06.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.426+0000 7f9ac1e00700 1 --2- 172.21.15.1:0/47595151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9abc10f030 0x7f9abc10f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:06.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.426+0000 7f9ac1e00700 1 --2- 172.21.15.1:0/47595151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9abc108f80 0x7f9abc109400 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:06.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.426+0000 7f9ac1e00700 1 -- 172.21.15.1:0/47595151 >> 172.21.15.1:0/47595151 conn(0x7f9abc0787d0 msgr2=0x7f9abc107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:06.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.426+0000 7f9ac1e00700 1 -- 172.21.15.1:0/47595151 shutdown_connections 2024-04-26T02:04:06.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.426+0000 7f9ac1e00700 1 -- 172.21.15.1:0/47595151 wait complete. 2024-04-26T02:04:06.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.427+0000 7f9ac1e00700 1 Processor -- start 2024-04-26T02:04:06.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.428+0000 7f9ac1e00700 1 -- start start 2024-04-26T02:04:06.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.428+0000 7f9ac1e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9abc108f80 0x7f9abc19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:06.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.428+0000 7f9ac1e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9abc109940 0x7f9abc19fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:06.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.428+0000 7f9ac1e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9abc10f030 0x7f9abc1a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:06.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.428+0000 7f9ac1e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1190f0 con 0x7f9abc109940 2024-04-26T02:04:06.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.428+0000 7f9ac1e00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc118f70 con 0x7f9abc10f030 2024-04-26T02:04:06.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.428+0000 7f9ac1e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc119270 con 0x7f9abc108f80 2024-04-26T02:04:06.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.428+0000 7f9abbe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9abc109940 0x7f9abc19fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:06.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.429+0000 7f9ac0a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9abc108f80 0x7f9abc19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:06.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.429+0000 7f9ac0a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9abc108f80 0x7f9abc19f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:58706/0 (socket says 172.21.15.1:58706) 2024-04-26T02:04:06.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.429+0000 7f9ac0a00700 1 -- 172.21.15.1:0/1301006498 learned_addr learned my addr 172.21.15.1:0/1301006498 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:06.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.429+0000 7f9ac1400700 1 --2- 172.21.15.1:0/1301006498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9abc10f030 0x7f9abc1a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:06.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.429+0000 7f9ac0a00700 1 -- 172.21.15.1:0/1301006498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9abc10f030 msgr2=0x7f9abc1a3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:06.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.429+0000 7f9ac0a00700 1 --2- 172.21.15.1:0/1301006498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9abc10f030 0x7f9abc1a3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:06.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.429+0000 7f9ac0a00700 1 -- 172.21.15.1:0/1301006498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9abc109940 msgr2=0x7f9abc19fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:06.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.429+0000 7f9ac0a00700 1 --2- 172.21.15.1:0/1301006498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9abc109940 0x7f9abc19fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:06.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.429+0000 7f9ac0a00700 1 -- 172.21.15.1:0/1301006498 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9abc1a4580 con 0x7f9abc108f80 2024-04-26T02:04:06.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.429+0000 7f9ac1400700 1 --2- 172.21.15.1:0/1301006498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9abc10f030 0x7f9abc1a3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:04:06.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.429+0000 7f9abbe00700 1 --2- 172.21.15.1:0/1301006498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9abc109940 0x7f9abc19fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:04:06.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.429+0000 7f9ac0a00700 1 --2- 172.21.15.1:0/1301006498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9abc108f80 0x7f9abc19f5c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000d980 tx=0x7f9ab000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:06.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.430+0000 7f9ab9600700 1 -- 172.21.15.1:0/1301006498 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0016070 con 0x7f9abc108f80 2024-04-26T02:04:06.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.430+0000 7f9ab9600700 1 -- 172.21.15.1:0/1301006498 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ab0004830 con 0x7f9abc108f80 2024-04-26T02:04:06.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.430+0000 7f9ab9600700 1 -- 172.21.15.1:0/1301006498 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0005540 con 0x7f9abc108f80 2024-04-26T02:04:06.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.430+0000 7f9ac1e00700 1 -- 172.21.15.1:0/1301006498 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9abc1a4870 con 0x7f9abc108f80 2024-04-26T02:04:06.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.430+0000 7f9ac1e00700 1 -- 172.21.15.1:0/1301006498 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9abc1ac0f0 con 0x7f9abc108f80 2024-04-26T02:04:06.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.431+0000 7f9ac1e00700 1 -- 172.21.15.1:0/1301006498 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9abc069730 con 0x7f9abc108f80 2024-04-26T02:04:06.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.433+0000 7f9ab9600700 1 -- 172.21.15.1:0/1301006498 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f9ab0004220 con 0x7f9abc108f80 2024-04-26T02:04:06.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:06 smithi001 ceph-mon[27822]: pgmap v390: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-26T02:04:06.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.435+0000 7f9ab9600700 1 --2- 172.21.15.1:0/1301006498 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f9aa40a3db0 0x7f9aa40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:06.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.435+0000 7f9ab9600700 1 -- 172.21.15.1:0/1301006498 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f9ab00d0630 con 0x7f9abc108f80 2024-04-26T02:04:06.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.435+0000 7f9abbe00700 1 --2- 172.21.15.1:0/1301006498 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f9aa40a3db0 0x7f9aa40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:06.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.436+0000 7f9abbe00700 1 --2- 172.21.15.1:0/1301006498 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f9aa40a3db0 0x7f9aa40a6270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800b2e0 tx=0x7f9aa801a040 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:06.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.439+0000 7f9ab9600700 1 -- 172.21.15.1:0/1301006498 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ab001f020 con 0x7f9abc108f80 2024-04-26T02:04:06.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:06 smithi097 ceph-mon[32853]: pgmap v390: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-26T02:04:06.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:06 smithi173 ceph-mon[32553]: pgmap v390: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 28 op/s 2024-04-26T02:04:06.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.724+0000 7f9ac1e00700 1 -- 172.21.15.1:0/1301006498 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9abc02df80 con 0x7f9abc108f80 2024-04-26T02:04:06.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.726+0000 7f9ab9600700 1 -- 172.21.15.1:0/1301006498 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11840 (secure 0 0 0) 0x7f9ab009cc40 con 0x7f9abc108f80 2024-04-26T02:04:06.725 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T02:04:06.725 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T02:03:36.329802+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26497":{"gid":26497,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/334804458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":334804458},{"type":"v1","addr":"172.21.15.173:6835","nonce":334804458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24518,24499,24514]},"id":1}]} 2024-04-26T02:04:06.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.729+0000 7f9ac1e00700 1 -- 172.21.15.1:0/1301006498 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f9aa40a3db0 msgr2=0x7f9aa40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:06.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.729+0000 7f9ac1e00700 1 --2- 172.21.15.1:0/1301006498 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f9aa40a3db0 0x7f9aa40a6270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800b2e0 tx=0x7f9aa801a040 comp rx=0 tx=0).stop 2024-04-26T02:04:06.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.729+0000 7f9ac1e00700 1 -- 172.21.15.1:0/1301006498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9abc108f80 msgr2=0x7f9abc19f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:06.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.729+0000 7f9ac1e00700 1 --2- 172.21.15.1:0/1301006498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9abc108f80 0x7f9abc19f5c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000d980 tx=0x7f9ab000dc90 comp rx=0 tx=0).stop 2024-04-26T02:04:06.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.729+0000 7f9ac1e00700 1 -- 172.21.15.1:0/1301006498 shutdown_connections 2024-04-26T02:04:06.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.729+0000 7f9ac1e00700 1 --2- 172.21.15.1:0/1301006498 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f9aa40a3db0 0x7f9aa40a6270 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:06.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.729+0000 7f9ac1e00700 1 --2- 172.21.15.1:0/1301006498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9abc10f030 0x7f9abc1a3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:06.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.729+0000 7f9ac1e00700 1 --2- 172.21.15.1:0/1301006498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9abc108f80 0x7f9abc19f5c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:06.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.729+0000 7f9ac1e00700 1 --2- 172.21.15.1:0/1301006498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9abc109940 0x7f9abc19fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:06.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.729+0000 7f9ac1e00700 1 -- 172.21.15.1:0/1301006498 >> 172.21.15.1:0/1301006498 conn(0x7f9abc0787d0 msgr2=0x7f9abc079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:06.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.729+0000 7f9ac1e00700 1 -- 172.21.15.1:0/1301006498 shutdown_connections 2024-04-26T02:04:06.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:06.729+0000 7f9ac1e00700 1 -- 172.21.15.1:0/1301006498 wait complete. 2024-04-26T02:04:06.729 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 36 2024-04-26T02:04:07.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:07 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1301006498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:04:07.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:07 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1301006498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:04:07.544 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24518 quiesce db / --timeout 60 --expiration 79.25 --await-for 42.0' 2024-04-26T02:04:07.544 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24518 quiesce db / --timeout 60 --expiration 79.25 --await-for 42.0 2024-04-26T02:04:07.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:07 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1301006498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:04:08.160 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:04:08.311 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:08 smithi001 ceph-mon[27822]: pgmap v391: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 51 KiB/s wr, 25 op/s 2024-04-26T02:04:08.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:08 smithi097 ceph-mon[32853]: pgmap v391: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 51 KiB/s wr, 25 op/s 2024-04-26T02:04:08.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:08 smithi173 ceph-mon[32553]: pgmap v391: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 51 KiB/s wr, 25 op/s 2024-04-26T02:04:09.991 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.992+0000 7f4b4dc00640 1 -- 172.21.15.1:0/3483262402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b481018d0 msgr2=0x7f4b48105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:09.991 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.992+0000 7f4b4dc00640 1 --2- 172.21.15.1:0/3483262402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b481018d0 0x7f4b48105d90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009930 tx=0x7f4b3402a4d0 comp rx=0 tx=0).stop 2024-04-26T02:04:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.992+0000 7f4b4dc00640 1 -- 172.21.15.1:0/3483262402 shutdown_connections 2024-04-26T02:04:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.992+0000 7f4b4dc00640 1 --2- 172.21.15.1:0/3483262402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b481063b0 0x7f4b48106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.992+0000 7f4b4dc00640 1 --2- 172.21.15.1:0/3483262402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b481018d0 0x7f4b48105d90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.992+0000 7f4b4dc00640 1 --2- 172.21.15.1:0/3483262402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b48100fb0 0x7f4b48101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.992+0000 7f4b4dc00640 1 -- 172.21.15.1:0/3483262402 >> 172.21.15.1:0/3483262402 conn(0x7f4b48077f70 msgr2=0x7f4b48078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.992+0000 7f4b4dc00640 1 -- 172.21.15.1:0/3483262402 shutdown_connections 2024-04-26T02:04:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.992+0000 7f4b4dc00640 1 -- 172.21.15.1:0/3483262402 wait complete. 2024-04-26T02:04:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.993+0000 7f4b4dc00640 1 Processor -- start 2024-04-26T02:04:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.993+0000 7f4b4dc00640 1 -- start start 2024-04-26T02:04:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.994+0000 7f4b4dc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b48100fb0 0x7f4b4819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.994+0000 7f4b4dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b481018d0 0x7f4b4819f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.994+0000 7f4b4dc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b481063b0 0x7f4b481a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.994+0000 7f4b4dc00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481187e0 con 0x7f4b481063b0 2024-04-26T02:04:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.994+0000 7f4b4dc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4b48118660 con 0x7f4b481018d0 2024-04-26T02:04:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.994+0000 7f4b4dc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4b48118960 con 0x7f4b48100fb0 2024-04-26T02:04:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.994+0000 7f4b46a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b48100fb0 0x7f4b4819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.994+0000 7f4b46a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b48100fb0 0x7f4b4819eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:58718/0 (socket says 172.21.15.1:58718) 2024-04-26T02:04:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.994+0000 7f4b46a00640 1 -- 172.21.15.1:0/1366820737 learned_addr learned my addr 172.21.15.1:0/1366820737 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.994+0000 7f4b46000640 1 --2- 172.21.15.1:0/1366820737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b481018d0 0x7f4b4819f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.994+0000 7f4b47400640 1 --2- 172.21.15.1:0/1366820737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b481063b0 0x7f4b481a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.995+0000 7f4b46a00640 1 -- 172.21.15.1:0/1366820737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b481018d0 msgr2=0x7f4b4819f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.995+0000 7f4b46a00640 1 --2- 172.21.15.1:0/1366820737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b481018d0 0x7f4b4819f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.995+0000 7f4b46a00640 1 -- 172.21.15.1:0/1366820737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b481063b0 msgr2=0x7f4b481a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.995+0000 7f4b46a00640 1 --2- 172.21.15.1:0/1366820737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b481063b0 0x7f4b481a3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.995+0000 7f4b46a00640 1 -- 172.21.15.1:0/1366820737 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b481a3d90 con 0x7f4b48100fb0 2024-04-26T02:04:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.995+0000 7f4b46000640 1 --2- 172.21.15.1:0/1366820737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b481018d0 0x7f4b4819f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:04:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.995+0000 7f4b46a00640 1 --2- 172.21.15.1:0/1366820737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b48100fb0 0x7f4b4819eda0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800d950 tx=0x7f4b3800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.995+0000 7f4b3f400640 1 -- 172.21.15.1:0/1366820737 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38017070 con 0x7f4b48100fb0 2024-04-26T02:04:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.995+0000 7f4b3f400640 1 -- 172.21.15.1:0/1366820737 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b38009c30 con 0x7f4b48100fb0 2024-04-26T02:04:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.995+0000 7f4b3f400640 1 -- 172.21.15.1:0/1366820737 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3800f620 con 0x7f4b48100fb0 2024-04-26T02:04:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.995+0000 7f4b4dc00640 1 -- 172.21.15.1:0/1366820737 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b481a4080 con 0x7f4b48100fb0 2024-04-26T02:04:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.996+0000 7f4b4dc00640 1 -- 172.21.15.1:0/1366820737 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b481ab910 con 0x7f4b48100fb0 2024-04-26T02:04:09.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:09.999+0000 7f4b3f400640 1 -- 172.21.15.1:0/1366820737 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f4b38010460 con 0x7f4b48100fb0 2024-04-26T02:04:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.000+0000 7f4b3f400640 1 --2- 172.21.15.1:0/1366820737 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f4b2c0a3d90 0x7f4b2c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.000+0000 7f4b50a74740 1 Processor -- start 2024-04-26T02:04:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.000+0000 7f4b3f400640 1 -- 172.21.15.1:0/1366820737 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f4b380d0520 con 0x7f4b48100fb0 2024-04-26T02:04:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.000+0000 7f4b46000640 1 --2- 172.21.15.1:0/1366820737 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f4b2c0a3d90 0x7f4b2c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.001+0000 7f4b50a74740 1 -- start start 2024-04-26T02:04:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.001+0000 7f4b50a74740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeaaed3820 0x55eeaaed3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.001+0000 7f4b50a74740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed4140 0x55eeaaede4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.001+0000 7f4b50a74740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaedea00 0x55eeaaee0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.001+0000 7f4b50a74740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55eeaae39900 con 0x55eeaaed3820 2024-04-26T02:04:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.001+0000 7f4b50a74740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55eeaaddfef0 con 0x55eeaaed4140 2024-04-26T02:04:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.001+0000 7f4b50a74740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55eeaae358c0 con 0x55eeaaedea00 2024-04-26T02:04:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.001+0000 7f4b47400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaedea00 0x55eeaaee0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.002+0000 7f4b47400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaedea00 0x55eeaaee0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:58732/0 (socket says 172.21.15.1:58732) 2024-04-26T02:04:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.002+0000 7f4b47400640 1 -- 172.21.15.1:0/1843076889 learned_addr learned my addr 172.21.15.1:0/1843076889 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.002+0000 7f4b46000640 1 --2- 172.21.15.1:0/1843076889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed4140 0x55eeaaede4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.002+0000 7f4b46a00640 1 --2- 172.21.15.1:0/1843076889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeaaed3820 0x55eeaaed3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.002+0000 7f4b46000640 1 --2- 172.21.15.1:0/1366820737 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f4b2c0a3d90 0x7f4b2c0a6250 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4b34006010 tx=0x7f4b34000bc0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.002+0000 7f4b47400640 1 -- 172.21.15.1:0/1843076889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed4140 msgr2=0x55eeaaede4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.002+0000 7f4b47400640 1 --2- 172.21.15.1:0/1843076889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed4140 0x55eeaaede4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.002+0000 7f4b47400640 1 -- 172.21.15.1:0/1843076889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeaaed3820 msgr2=0x55eeaaed3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.002+0000 7f4b47400640 1 --2- 172.21.15.1:0/1843076889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeaaed3820 0x55eeaaed3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.002+0000 7f4b47400640 1 -- 172.21.15.1:0/1843076889 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeaab4c210 con 0x55eeaaedea00 2024-04-26T02:04:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.002+0000 7f4b46000640 1 --2- 172.21.15.1:0/1843076889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed4140 0x55eeaaede4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:04:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.002+0000 7f4b47400640 1 --2- 172.21.15.1:0/1843076889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaedea00 0x55eeaaee0df0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009f20 tx=0x7f4b4000f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbf73fb3db145407 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.003+0000 7f4b33e00640 1 -- 172.21.15.1:0/1843076889 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40015070 con 0x55eeaaedea00 2024-04-26T02:04:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.003+0000 7f4b33e00640 1 -- 172.21.15.1:0/1843076889 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b4001ab10 con 0x55eeaaedea00 2024-04-26T02:04:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.003+0000 7f4b33e00640 1 -- 172.21.15.1:0/1843076889 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40011c20 con 0x55eeaaedea00 2024-04-26T02:04:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.004+0000 7f4b50a74740 1 -- 172.21.15.1:0/1843076889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaedea00 msgr2=0x55eeaaee0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.004+0000 7f4b50a74740 1 --2- 172.21.15.1:0/1843076889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaedea00 0x55eeaaee0df0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009f20 tx=0x7f4b4000f7f0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.004+0000 7f4b50a74740 1 -- 172.21.15.1:0/1843076889 shutdown_connections 2024-04-26T02:04:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.004+0000 7f4b50a74740 1 --2- 172.21.15.1:0/1843076889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaedea00 0x55eeaaee0df0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.004+0000 7f4b50a74740 1 --2- 172.21.15.1:0/1843076889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed4140 0x55eeaaede4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.004+0000 7f4b50a74740 1 --2- 172.21.15.1:0/1843076889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeaaed3820 0x55eeaaed3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.004+0000 7f4b50a74740 1 -- 172.21.15.1:0/1843076889 >> 172.21.15.1:0/1843076889 conn(0x55eeaaecfd30 msgr2=0x55eeaaed0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.004+0000 7f4b50a74740 1 -- 172.21.15.1:0/1843076889 shutdown_connections 2024-04-26T02:04:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.004+0000 7f4b50a74740 1 -- 172.21.15.1:0/1843076889 wait complete. 2024-04-26T02:04:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.005+0000 7f4b50a74740 1 -- start start 2024-04-26T02:04:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.005+0000 7f4b50a74740 1 Processor -- start 2024-04-26T02:04:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.006+0000 7f4b50a74740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeaaed3820 0x55eeaaedba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.006+0000 7f4b50a74740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed4140 0x55eeaaedbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.006+0000 7f4b50a74740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaedea00 0x55eeaaedc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.006+0000 7f4b50a74740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55eeaaee1630 con 0x55eeaaed3820 2024-04-26T02:04:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.006+0000 7f4b50a74740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55eeaadf3310 con 0x55eeaaed4140 2024-04-26T02:04:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.006+0000 7f4b50a74740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55eeaaee17b0 con 0x55eeaaedea00 2024-04-26T02:04:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.006+0000 7f4b50a74740 10 client.0 _ll_register_callbacks cb 0x55eeaadc1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:04:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.006+0000 7f4b46a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeaaed3820 0x55eeaaedba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.006+0000 7f4b46000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed4140 0x55eeaaedbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.006+0000 7f4b46000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed4140 0x55eeaaedbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:48220/0 (socket says 172.21.15.1:48220) 2024-04-26T02:04:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.006+0000 7f4b46000640 1 -- 172.21.15.1:0/496940501 learned_addr learned my addr 172.21.15.1:0/496940501 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.006+0000 7f4b47400640 1 --2- 172.21.15.1:0/496940501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaedea00 0x55eeaaedc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.007+0000 7f4b46000640 1 -- 172.21.15.1:0/496940501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaedea00 msgr2=0x55eeaaedc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.007+0000 7f4b46000640 1 --2- 172.21.15.1:0/496940501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaedea00 0x55eeaaedc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.007+0000 7f4b46000640 1 -- 172.21.15.1:0/496940501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeaaed3820 msgr2=0x55eeaaedba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.007+0000 7f4b46000640 1 --2- 172.21.15.1:0/496940501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeaaed3820 0x55eeaaedba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.007+0000 7f4b46000640 1 -- 172.21.15.1:0/496940501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeaac18620 con 0x55eeaaed4140 2024-04-26T02:04:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.007+0000 7f4b47400640 1 --2- 172.21.15.1:0/496940501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaedea00 0x55eeaaedc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:04:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.007+0000 7f4b46000640 1 --2- 172.21.15.1:0/496940501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed4140 0x55eeaaedbf60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4b3402aac0 tx=0x7f4b34009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3c2b0b3f6b5e54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.007+0000 7f4b30c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:04:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.007+0000 7f4b30c00640 1 -- 172.21.15.1:0/496940501 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34035050 con 0x55eeaaed4140 2024-04-26T02:04:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.007+0000 7f4b50a74740 1 -- 172.21.15.1:0/496940501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b4001ab10 con 0x55eeaaed4140 2024-04-26T02:04:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.007+0000 7f4b30c00640 1 -- 172.21.15.1:0/496940501 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f4b340037f0 con 0x55eeaaed4140 2024-04-26T02:04:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.008+0000 7f4b30c00640 1 -- 172.21.15.1:0/496940501 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4b34004e20 con 0x55eeaaed4140 2024-04-26T02:04:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.008+0000 7f4b50a74740 10 client.26681 fetch_fsmap learned FSMap version 36 2024-04-26T02:04:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.008+0000 7f4b50a74740 1 -- 172.21.15.1:0/496940501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55eeaaeed920 con 0x55eeaaed4140 2024-04-26T02:04:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.008+0000 7f4b30c00640 1 -- 172.21.15.1:0/496940501 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b4001ab10 con 0x55eeaaed4140 2024-04-26T02:04:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.009+0000 7f4b30c00640 1 -- 172.21.15.1:0/496940501 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f4b340039f0 con 0x55eeaaed4140 2024-04-26T02:04:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.009+0000 7f4b50a74740 10 client.26681 fetch_fsmap finished waiting for FSMap version 36 2024-04-26T02:04:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.009+0000 7f4b50a74740 10 client.26681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:04:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.009+0000 7f4b50a74740 20 client.26681 populate_metadata read hostname 'smithi001' 2024-04-26T02:04:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.009+0000 7f4b50a74740 1 --2- 172.21.15.1:0/496940501 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55eeaaeeede0 0x55eeaaef11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.009+0000 7f4b50a74740 4 client.26681 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:04:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.009+0000 7f4b50a74740 1 -- 172.21.15.1:0/496940501 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eeaae4d8b0 con 0x55eeaaeeede0 2024-04-26T02:04:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.009+0000 7f4b46a00640 1 --2- 172.21.15.1:0/496940501 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55eeaaeeede0 0x55eeaaef11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.010+0000 7f4b46a00640 1 --2- 172.21.15.1:0/496940501 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55eeaaeeede0 0x55eeaaef11d0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0a05a88bcb15b08 server_cookie=6ea92cab0ec4242f in_seq=0 out_seq=0 2024-04-26T02:04:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.010+0000 7f4b30c00640 10 client.26681 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:04:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.010+0000 7f4b30c00640 1 -- 172.21.15.1:0/496940501 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eeaae4d8b0 con 0x55eeaaeeede0 2024-04-26T02:04:10.012 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.010+0000 7f4b30c00640 10 client.26681 handle_command_reply: tid=0 2024-04-26T02:04:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.011+0000 7f4b50a74740 1 client.26681 shutdown 2024-04-26T02:04:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.011+0000 7f4b50a74740 1 -- 172.21.15.1:0/496940501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed4140 msgr2=0x55eeaaedbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.011+0000 7f4b50a74740 1 --2- 172.21.15.1:0/496940501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed4140 0x55eeaaedbf60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4b3402aac0 tx=0x7f4b34009260 comp rx=0 tx=0).stop 2024-04-26T02:04:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.011+0000 7f4b50a74740 1 -- 172.21.15.1:0/496940501 shutdown_connections 2024-04-26T02:04:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.011+0000 7f4b50a74740 1 --2- 172.21.15.1:0/496940501 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55eeaaeeede0 0x55eeaaef11d0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.011+0000 7f4b50a74740 1 --2- 172.21.15.1:0/496940501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaedea00 0x55eeaaedc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.011+0000 7f4b50a74740 1 --2- 172.21.15.1:0/496940501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed4140 0x55eeaaedbf60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.011+0000 7f4b30c00640 0 client.26681 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:04:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.011+0000 7f4b50a74740 1 --2- 172.21.15.1:0/496940501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeaaed3820 0x55eeaaedba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.011+0000 7f4b50a74740 1 -- 172.21.15.1:0/496940501 >> 172.21.15.1:0/496940501 conn(0x55eeaaecfd30 msgr2=0x55eeaaed5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.011+0000 7f4b50a74740 1 -- 172.21.15.1:0/496940501 shutdown_connections 2024-04-26T02:04:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.011+0000 7f4b50a74740 1 -- 172.21.15.1:0/496940501 wait complete. 2024-04-26T02:04:10.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.011+0000 7f4b50a74740 20 client.26681 trim_cache size 0 max 16384 2024-04-26T02:04:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.029+0000 7f4b50a74740 1 Processor -- start 2024-04-26T02:04:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.029+0000 7f4b50a74740 1 -- start start 2024-04-26T02:04:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f4b50a74740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed3820 0x55eeaaeea4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f4b50a74740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaed4140 0x55eeaaed5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f4b50a74740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeaaedea00 0x55eeaaef6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f4b50a74740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481b6c90 con 0x55eeaaedea00 2024-04-26T02:04:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f4b50a74740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481b4530 con 0x55eeaaed3820 2024-04-26T02:04:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f4b50a74740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481b23a0 con 0x55eeaaed4140 2024-04-26T02:04:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f4b46a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed3820 0x55eeaaeea4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f4b46a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed3820 0x55eeaaeea4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:48238/0 (socket says 172.21.15.1:48238) 2024-04-26T02:04:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f4b46a00640 1 -- 172.21.15.1:0/2040142185 learned_addr learned my addr 172.21.15.1:0/2040142185 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f4b46000640 1 --2- 172.21.15.1:0/2040142185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaed4140 0x55eeaaed5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f4b47400640 1 --2- 172.21.15.1:0/2040142185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeaaedea00 0x55eeaaef6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f4b46a00640 1 -- 172.21.15.1:0/2040142185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaed4140 msgr2=0x55eeaaed5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f4b46a00640 1 --2- 172.21.15.1:0/2040142185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaed4140 0x55eeaaed5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f4b46a00640 1 -- 172.21.15.1:0/2040142185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeaaedea00 msgr2=0x55eeaaef6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.031+0000 7f4b46a00640 1 --2- 172.21.15.1:0/2040142185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeaaedea00 0x55eeaaef6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.031+0000 7f4b46a00640 1 -- 172.21.15.1:0/2040142185 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeaaed4ad0 con 0x55eeaaed3820 2024-04-26T02:04:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.031+0000 7f4b47400640 1 --2- 172.21.15.1:0/2040142185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeaaedea00 0x55eeaaef6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:04:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.031+0000 7f4b46a00640 1 --2- 172.21.15.1:0/2040142185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed3820 0x55eeaaeea4f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4b38098e00 tx=0x7f4b38010cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de1a21cf92e0346 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:10.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f6b91600700 1 -- 172.21.15.1:0/83076636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b8c074c70 msgr2=0x7f6b8c075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:10.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f6b91600700 1 --2- 172.21.15.1:0/83076636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b8c074c70 0x7f6b8c075050 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009990 tx=0x7f6b7c009ca0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f6b91600700 1 -- 172.21.15.1:0/83076636 shutdown_connections 2024-04-26T02:04:10.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f6b91600700 1 --2- 172.21.15.1:0/83076636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b8c113c10 0x7f6b8c116000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f6b91600700 1 --2- 172.21.15.1:0/83076636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b8c074c70 0x7f6b8c075050 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f6b91600700 1 --2- 172.21.15.1:0/83076636 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b8c075590 0x7f6b8c1136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f6b91600700 1 -- 172.21.15.1:0/83076636 >> 172.21.15.1:0/83076636 conn(0x7f6b8c079ba0 msgr2=0x7f6b8c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:10.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.030+0000 7f6b91600700 1 -- 172.21.15.1:0/83076636 shutdown_connections 2024-04-26T02:04:10.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.031+0000 7f6b91600700 1 -- 172.21.15.1:0/83076636 wait complete. 2024-04-26T02:04:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.031+0000 7f4b33400640 1 -- 172.21.15.1:0/2040142185 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38020040 con 0x55eeaaed3820 2024-04-26T02:04:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.031+0000 7f4b33400640 1 -- 172.21.15.1:0/2040142185 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b380a07e0 con 0x55eeaaed3820 2024-04-26T02:04:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.031+0000 7f4b33400640 1 -- 172.21.15.1:0/2040142185 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38098180 con 0x55eeaaed3820 2024-04-26T02:04:10.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.032+0000 7f6b91600700 1 Processor -- start 2024-04-26T02:04:10.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.032+0000 7f6b91600700 1 -- start start 2024-04-26T02:04:10.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.032+0000 7f6b91600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b8c074c70 0x7f6b8c1a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.032+0000 7f6b91600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b8c075590 0x7f6b8c1a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.033+0000 7f4b50a74740 1 -- 172.21.15.1:0/2040142185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed3820 msgr2=0x55eeaaeea4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.033+0000 7f4b50a74740 1 --2- 172.21.15.1:0/2040142185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed3820 0x55eeaaeea4f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4b38098e00 tx=0x7f4b38010cb0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.032+0000 7f6b91600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b8c113c10 0x7f6b8c19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.032+0000 7f6b91600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c119260 con 0x7f6b8c074c70 2024-04-26T02:04:10.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.032+0000 7f6b91600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1190e0 con 0x7f6b8c113c10 2024-04-26T02:04:10.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.032+0000 7f6b91600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1193e0 con 0x7f6b8c075590 2024-04-26T02:04:10.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.033+0000 7f6b8b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b8c113c10 0x7f6b8c19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.033+0000 7f6b8aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b8c074c70 0x7f6b8c1a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.033+0000 7f6b8b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b8c113c10 0x7f6b8c19f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:48242/0 (socket says 172.21.15.1:48242) 2024-04-26T02:04:10.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.033+0000 7f6b8b400700 1 -- 172.21.15.1:0/3818285127 learned_addr learned my addr 172.21.15.1:0/3818285127 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:10.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.033+0000 7f6b8a000700 1 --2- 172.21.15.1:0/3818285127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b8c075590 0x7f6b8c1a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.033+0000 7f6b8b400700 1 -- 172.21.15.1:0/3818285127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b8c075590 msgr2=0x7f6b8c1a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:10.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.033+0000 7f6b8b400700 1 --2- 172.21.15.1:0/3818285127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b8c075590 0x7f6b8c1a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.033+0000 7f6b8b400700 1 -- 172.21.15.1:0/3818285127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b8c074c70 msgr2=0x7f6b8c1a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:10.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.033+0000 7f6b8b400700 1 --2- 172.21.15.1:0/3818285127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b8c074c70 0x7f6b8c1a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.033+0000 7f6b8b400700 1 -- 172.21.15.1:0/3818285127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8c19fd40 con 0x7f6b8c113c10 2024-04-26T02:04:10.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.033+0000 7f6b8b400700 1 --2- 172.21.15.1:0/3818285127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b8c113c10 0x7f6b8c19f5e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400ef90 tx=0x7f6b8400c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:10.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.033+0000 7f6b83400700 1 -- 172.21.15.1:0/3818285127 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b840092e0 con 0x7f6b8c113c10 2024-04-26T02:04:10.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.034+0000 7f6b83400700 1 -- 172.21.15.1:0/3818285127 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b84004bb0 con 0x7f6b8c113c10 2024-04-26T02:04:10.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.034+0000 7f6b83400700 1 -- 172.21.15.1:0/3818285127 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84004280 con 0x7f6b8c113c10 2024-04-26T02:04:10.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.034+0000 7f6b91600700 1 -- 172.21.15.1:0/3818285127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8c19ffd0 con 0x7f6b8c113c10 2024-04-26T02:04:10.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.034+0000 7f6b91600700 1 -- 172.21.15.1:0/3818285127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b8c1ac2f0 con 0x7f6b8c113c10 2024-04-26T02:04:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.035+0000 7f4b50a74740 1 -- 172.21.15.1:0/2040142185 shutdown_connections 2024-04-26T02:04:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.035+0000 7f4b50a74740 1 --2- 172.21.15.1:0/2040142185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeaaedea00 0x55eeaaef6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.035+0000 7f4b50a74740 1 --2- 172.21.15.1:0/2040142185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaed4140 0x55eeaaed5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.035+0000 7f4b50a74740 1 --2- 172.21.15.1:0/2040142185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed3820 0x55eeaaeea4f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.035+0000 7f4b50a74740 1 -- 172.21.15.1:0/2040142185 >> 172.21.15.1:0/2040142185 conn(0x55eeaaecfd30 msgr2=0x55eeaaed5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.035+0000 7f4b50a74740 1 -- 172.21.15.1:0/2040142185 shutdown_connections 2024-04-26T02:04:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.035+0000 7f4b50a74740 1 -- 172.21.15.1:0/2040142185 wait complete. 2024-04-26T02:04:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.036+0000 7f4b50a74740 1 -- start start 2024-04-26T02:04:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.036+0000 7f4b50a74740 1 Processor -- start 2024-04-26T02:04:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.037+0000 7f4b50a74740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaed3820 0x55eeaaee6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.037+0000 7f4b50a74740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed4140 0x55eeaaee6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.037+0000 7f4b50a74740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeaaedea00 0x55eeaaee4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.037+0000 7f4b50a74740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481b29f0 con 0x55eeaaedea00 2024-04-26T02:04:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.037+0000 7f4b50a74740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481b3e60 con 0x55eeaaed4140 2024-04-26T02:04:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.037+0000 7f4b50a74740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481b30c0 con 0x55eeaaed3820 2024-04-26T02:04:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.037+0000 7f4b50a74740 10 client.0 _ll_register_callbacks cb 0x7f4b34005400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:04:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.037+0000 7f4b46a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaed3820 0x55eeaaee6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.037+0000 7f4b46000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed4140 0x55eeaaee6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.037+0000 7f4b46a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaed3820 0x55eeaaee6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:58790/0 (socket says 172.21.15.1:58790) 2024-04-26T02:04:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.037+0000 7f4b46000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed4140 0x55eeaaee6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:48252/0 (socket says 172.21.15.1:48252) 2024-04-26T02:04:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.037+0000 7f4b46a00640 1 -- 172.21.15.1:0/3621783254 learned_addr learned my addr 172.21.15.1:0/3621783254 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.037+0000 7f4b47400640 1 --2- 172.21.15.1:0/3621783254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeaaedea00 0x55eeaaee4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.038+0000 7f4b46000640 1 -- 172.21.15.1:0/3621783254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaed3820 msgr2=0x55eeaaee6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.038+0000 7f4b46000640 1 --2- 172.21.15.1:0/3621783254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaed3820 0x55eeaaee6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.038+0000 7f4b46000640 1 -- 172.21.15.1:0/3621783254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeaaedea00 msgr2=0x55eeaaee4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.038+0000 7f4b46000640 1 --2- 172.21.15.1:0/3621783254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeaaedea00 0x55eeaaee4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.038+0000 7f4b46000640 1 -- 172.21.15.1:0/3621783254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeaaee52d0 con 0x55eeaaed4140 2024-04-26T02:04:10.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.037+0000 7f6b83400700 1 -- 172.21.15.1:0/3818285127 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f6b8401b030 con 0x7f6b8c113c10 2024-04-26T02:04:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.038+0000 7f4b46000640 1 --2- 172.21.15.1:0/3621783254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed4140 0x55eeaaee6b70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4b34031440 tx=0x7f4b34003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa725a41494015d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.038+0000 7f4b30c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:04:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.038+0000 7f4b30c00640 1 -- 172.21.15.1:0/3621783254 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3405d070 con 0x55eeaaed4140 2024-04-26T02:04:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.038+0000 7f4b30c00640 1 -- 172.21.15.1:0/3621783254 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f4b34032a30 con 0x55eeaaed4140 2024-04-26T02:04:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.039+0000 7f4b50a74740 1 -- 172.21.15.1:0/3621783254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b380a07e0 con 0x55eeaaed4140 2024-04-26T02:04:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.039+0000 7f4b30c00640 1 -- 172.21.15.1:0/3621783254 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4b34035040 con 0x55eeaaed4140 2024-04-26T02:04:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.039+0000 7f4b50a74740 10 client.26693 fetch_fsmap learned FSMap version 36 2024-04-26T02:04:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.039+0000 7f4b50a74740 1 -- 172.21.15.1:0/3621783254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55eeaaf55ca0 con 0x55eeaaed4140 2024-04-26T02:04:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.039+0000 7f4b30c00640 1 -- 172.21.15.1:0/3621783254 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b380a07e0 con 0x55eeaaed4140 2024-04-26T02:04:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.039+0000 7f4b30c00640 1 -- 172.21.15.1:0/3621783254 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f4b34069b60 con 0x55eeaaed4140 2024-04-26T02:04:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.039+0000 7f4b50a74740 10 client.26693 fetch_fsmap finished waiting for FSMap version 36 2024-04-26T02:04:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.039+0000 7f4b50a74740 10 client.26693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T02:04:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.039+0000 7f4b50a74740 20 client.26693 populate_metadata read hostname 'smithi001' 2024-04-26T02:04:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.039+0000 7f4b50a74740 1 --2- 172.21.15.1:0/3621783254 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55eeaaf56470 0x55eeaaf58890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.039+0000 7f4b50a74740 4 client.26693 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:04:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.039+0000 7f4b50a74740 1 -- 172.21.15.1:0/3621783254 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eeaae47320 con 0x55eeaaf56470 2024-04-26T02:04:10.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.039+0000 7f6b83400700 1 --2- 172.21.15.1:0/3818285127 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6b700a3f80 0x7f6b700a6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.039+0000 7f6b83400700 1 -- 172.21.15.1:0/3818285127 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f6b840d03b0 con 0x7f6b8c113c10 2024-04-26T02:04:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.040+0000 7f4b46a00640 1 --2- 172.21.15.1:0/3621783254 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55eeaaf56470 0x55eeaaf58890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.040+0000 7f4b46a00640 1 --2- 172.21.15.1:0/3621783254 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55eeaaf56470 0x55eeaaf58890 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91c5c72158141541 server_cookie=b0f9de0b04dd164 in_seq=0 out_seq=0 2024-04-26T02:04:10.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.040+0000 7f6b8aa00700 1 --2- 172.21.15.1:0/3818285127 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6b700a3f80 0x7f6b700a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.041+0000 7f4b30c00640 10 client.26693 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:04:10.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.041+0000 7f6b8aa00700 1 --2- 172.21.15.1:0/3818285127 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6b700a3f80 0x7f6b700a6440 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00b2e0 tx=0x7f6b7c01a040 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:10.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.041+0000 7f6b9c02ab80 1 Processor -- start 2024-04-26T02:04:10.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.042+0000 7f6b9c02ab80 1 -- start start 2024-04-26T02:04:10.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.042+0000 7f6b9c02ab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61c1890 0x5636b61c1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.042+0000 7f6b9c02ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c21b0 0x5636b61ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.042+0000 7f6b9c02ab80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61cd060 0x5636b61cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.042+0000 7f6b9c02ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5636b6114f00 con 0x5636b61c21b0 2024-04-26T02:04:10.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.042+0000 7f6b9c02ab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5636b60bb7a0 con 0x5636b61c1890 2024-04-26T02:04:10.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.042+0000 7f6b9c02ab80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5636b6110ec0 con 0x5636b61cd060 2024-04-26T02:04:10.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.043+0000 7f6b8b400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61cd060 0x5636b61cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.043+0000 7f6b8b400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61cd060 0x5636b61cf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:58804/0 (socket says 172.21.15.1:58804) 2024-04-26T02:04:10.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.043+0000 7f6b8b400700 1 -- 172.21.15.1:0/2591368364 learned_addr learned my addr 172.21.15.1:0/2591368364 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:10.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.043+0000 7f6b8a000700 1 --2- 172.21.15.1:0/2591368364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c21b0 0x5636b61ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.043+0000 7f6b8aa00700 1 --2- 172.21.15.1:0/2591368364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61c1890 0x5636b61c1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.044+0000 7f6b8b400700 1 -- 172.21.15.1:0/2591368364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61c1890 msgr2=0x5636b61c1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:10.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.044+0000 7f6b8b400700 1 --2- 172.21.15.1:0/2591368364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61c1890 0x5636b61c1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.044+0000 7f6b8b400700 1 -- 172.21.15.1:0/2591368364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c21b0 msgr2=0x5636b61ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:10.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.044+0000 7f6b8b400700 1 --2- 172.21.15.1:0/2591368364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c21b0 0x5636b61ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.044+0000 7f6b8b400700 1 -- 172.21.15.1:0/2591368364 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636b60b16f0 con 0x5636b61cd060 2024-04-26T02:04:10.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.044+0000 7f6b8b400700 1 --2- 172.21.15.1:0/2591368364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61cd060 0x5636b61cf450 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6b840987b0 tx=0x7f6b8409be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20360585f4718d6d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:10.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.045+0000 7f6b77e00700 1 -- 172.21.15.1:0/2591368364 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8409ca50 con 0x5636b61cd060 2024-04-26T02:04:10.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.045+0000 7f6b77e00700 1 -- 172.21.15.1:0/2591368364 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b8409c350 con 0x5636b61cd060 2024-04-26T02:04:10.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.045+0000 7f6b77e00700 1 -- 172.21.15.1:0/2591368364 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b840a4d00 con 0x5636b61cd060 2024-04-26T02:04:10.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.047+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/2591368364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61cd060 msgr2=0x5636b61cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:10.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.047+0000 7f6b9c02ab80 1 --2- 172.21.15.1:0/2591368364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61cd060 0x5636b61cf450 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6b840987b0 tx=0x7f6b8409be90 comp rx=0 tx=0).stop 2024-04-26T02:04:10.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.047+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/2591368364 shutdown_connections 2024-04-26T02:04:10.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.048+0000 7f6b9c02ab80 1 --2- 172.21.15.1:0/2591368364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61cd060 0x5636b61cf450 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.048+0000 7f6b9c02ab80 1 --2- 172.21.15.1:0/2591368364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61c1890 0x5636b61c1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.048+0000 7f6b9c02ab80 1 --2- 172.21.15.1:0/2591368364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c21b0 0x5636b61ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.048+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/2591368364 >> 172.21.15.1:0/2591368364 conn(0x5636b60f0610 msgr2=0x5636b61bdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:10.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.049+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/2591368364 shutdown_connections 2024-04-26T02:04:10.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.049+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/2591368364 wait complete. 2024-04-26T02:04:10.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.050+0000 7f6b9c02ab80 1 -- start start 2024-04-26T02:04:10.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.050+0000 7f6b9c02ab80 1 Processor -- start 2024-04-26T02:04:10.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.051+0000 7f6b9c02ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c1890 0x5636b61cc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.051+0000 7f6b9c02ab80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61c21b0 0x5636b61c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.051+0000 7f6b9c02ab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61cd060 0x5636b61ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.051+0000 7f6b9c02ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5636b61cfb80 con 0x5636b61c1890 2024-04-26T02:04:10.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.051+0000 7f6b9c02ab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5636b60ce8d0 con 0x5636b61cd060 2024-04-26T02:04:10.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.051+0000 7f6b9c02ab80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5636b61cfd00 con 0x5636b61c21b0 2024-04-26T02:04:10.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.051+0000 7f6b9c02ab80 10 client.0 _ll_register_callbacks cb 0x5636b60ad790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:04:10.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.051+0000 7f6b8aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c1890 0x5636b61cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.051+0000 7f6b8b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61cd060 0x5636b61ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.051+0000 7f6b8b400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61cd060 0x5636b61ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:48264/0 (socket says 172.21.15.1:48264) 2024-04-26T02:04:10.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.051+0000 7f6b8b400700 1 -- 172.21.15.1:0/2731934874 learned_addr learned my addr 172.21.15.1:0/2731934874 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:10.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.052+0000 7f6b8b400700 1 -- 172.21.15.1:0/2731934874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61c21b0 msgr2=0x5636b61c9b50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T02:04:10.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.052+0000 7f6b8a000700 1 --2- 172.21.15.1:0/2731934874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61c21b0 0x5636b61c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.052+0000 7f6b8b400700 1 --2- 172.21.15.1:0/2731934874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61c21b0 0x5636b61c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.052+0000 7f6b8b400700 1 -- 172.21.15.1:0/2731934874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c1890 msgr2=0x5636b61cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:10.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.052+0000 7f6b8b400700 1 --2- 172.21.15.1:0/2731934874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c1890 0x5636b61cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.052+0000 7f6b8b400700 1 -- 172.21.15.1:0/2731934874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636b6062940 con 0x5636b61cd060 2024-04-26T02:04:10.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.052+0000 7f6b8aa00700 1 --2- 172.21.15.1:0/2731934874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c1890 0x5636b61cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:04:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.053+0000 7f4b30c00640 1 -- 172.21.15.1:0/3621783254 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eeaae47320 con 0x55eeaaf56470 2024-04-26T02:04:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.053+0000 7f4b30c00640 10 client.26693 handle_command_reply: tid=0 2024-04-26T02:04:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.053+0000 7f4b50a74740 1 client.26693 shutdown 2024-04-26T02:04:10.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.052+0000 7f6b8a000700 1 --2- 172.21.15.1:0/2731934874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61c21b0 0x5636b61c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:04:10.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.052+0000 7f6b8b400700 1 --2- 172.21.15.1:0/2731934874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61cd060 0x5636b61ca090 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400c850 tx=0x7f6b840a4d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=491d64ddad1dd21f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:10.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.052+0000 7f6b74c00700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-26T02:04:10.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.053+0000 7f6b74c00700 1 -- 172.21.15.1:0/2731934874 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b840a4f30 con 0x5636b61cd060 2024-04-26T02:04:10.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.053+0000 7f6b74c00700 1 -- 172.21.15.1:0/2731934874 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f6b840bb200 con 0x5636b61cd060 2024-04-26T02:04:10.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.053+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/2731934874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b8409c350 con 0x5636b61cd060 2024-04-26T02:04:10.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.053+0000 7f6b74c00700 1 -- 172.21.15.1:0/2731934874 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6b8409ca50 con 0x5636b61cd060 2024-04-26T02:04:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.053+0000 7f4b50a74740 1 -- 172.21.15.1:0/3621783254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed4140 msgr2=0x55eeaaee6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.053+0000 7f4b50a74740 1 --2- 172.21.15.1:0/3621783254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed4140 0x55eeaaee6b70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4b34031440 tx=0x7f4b34003820 comp rx=0 tx=0).stop 2024-04-26T02:04:10.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.053+0000 7f6b9c02ab80 10 client.26699 fetch_fsmap learned FSMap version 36 2024-04-26T02:04:10.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.053+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/2731934874 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5636b60abb60 con 0x5636b61cd060 2024-04-26T02:04:10.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.053+0000 7f6b74c00700 1 -- 172.21.15.1:0/2731934874 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b8409c350 con 0x5636b61cd060 2024-04-26T02:04:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.053+0000 7f4b50a74740 1 -- 172.21.15.1:0/3621783254 shutdown_connections 2024-04-26T02:04:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.053+0000 7f4b50a74740 1 --2- 172.21.15.1:0/3621783254 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x55eeaaf56470 0x55eeaaf58890 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.053+0000 7f4b50a74740 1 --2- 172.21.15.1:0/3621783254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeaaedea00 0x55eeaaee4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.053+0000 7f4b50a74740 1 --2- 172.21.15.1:0/3621783254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eeaaed4140 0x55eeaaee6b70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.053+0000 7f4b50a74740 1 --2- 172.21.15.1:0/3621783254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eeaaed3820 0x55eeaaee6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.053+0000 7f4b50a74740 1 -- 172.21.15.1:0/3621783254 >> 172.21.15.1:0/3621783254 conn(0x55eeaaecfd30 msgr2=0x55eeaaedc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.054+0000 7f4b30c00640 0 client.26693 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:04:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.054+0000 7f4b50a74740 1 -- 172.21.15.1:0/3621783254 shutdown_connections 2024-04-26T02:04:10.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.054+0000 7f6b74c00700 1 -- 172.21.15.1:0/2731934874 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f6b840a0170 con 0x5636b61cd060 2024-04-26T02:04:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.054+0000 7f4b50a74740 1 -- 172.21.15.1:0/3621783254 wait complete. 2024-04-26T02:04:10.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.054+0000 7f6b9c02ab80 10 client.26699 fetch_fsmap finished waiting for FSMap version 36 2024-04-26T02:04:10.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.054+0000 7f6b9c02ab80 10 client.26699 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T02:04:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.054+0000 7f4b50a74740 20 client.26693 trim_cache size 0 max 16384 2024-04-26T02:04:10.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.054+0000 7f6b9c02ab80 20 client.26699 populate_metadata read hostname 'smithi001' 2024-04-26T02:04:10.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.054+0000 7f6b9c02ab80 1 --2- 172.21.15.1:0/2731934874 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5636b61dc7b0 0x5636b61deba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.054+0000 7f6b9c02ab80 4 client.26699 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:04:10.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.054+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/2731934874 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636b61226f0 con 0x5636b61dc7b0 2024-04-26T02:04:10.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.055+0000 7f6b8aa00700 1 --2- 172.21.15.1:0/2731934874 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5636b61dc7b0 0x5636b61deba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-04-26T02:04:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-04-26T02:04:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-04-26T02:04:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-04-26T02:04:10.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.055+0000 7f6b8aa00700 1 --2- 172.21.15.1:0/2731934874 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5636b61dc7b0 0x5636b61deba0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aed0759d348f45a server_cookie=c9bf9af0ddaf7c19 in_seq=0 out_seq=0 2024-04-26T02:04:10.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.055+0000 7f6b74c00700 10 client.26699 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:04:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.055+0000 7f4b4dc00640 1 -- 172.21.15.1:0/1366820737 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f4b2c0a3d90 msgr2=0x7f4b2c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.055+0000 7f4b4dc00640 1 --2- 172.21.15.1:0/1366820737 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f4b2c0a3d90 0x7f4b2c0a6250 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4b34006010 tx=0x7f4b34000bc0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.055+0000 7f4b4dc00640 1 -- 172.21.15.1:0/1366820737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b48100fb0 msgr2=0x7f4b4819eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.055+0000 7f4b4dc00640 1 --2- 172.21.15.1:0/1366820737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b48100fb0 0x7f4b4819eda0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800d950 tx=0x7f4b3800de10 comp rx=0 tx=0).stop 2024-04-26T02:04:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.056+0000 7f4b4dc00640 1 -- 172.21.15.1:0/1366820737 shutdown_connections 2024-04-26T02:04:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.056+0000 7f4b4dc00640 1 --2- 172.21.15.1:0/1366820737 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f4b2c0a3d90 0x7f4b2c0a6250 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.056+0000 7f4b4dc00640 1 --2- 172.21.15.1:0/1366820737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b481063b0 0x7f4b481a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.056+0000 7f4b4dc00640 1 --2- 172.21.15.1:0/1366820737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b481018d0 0x7f4b4819f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.056+0000 7f4b4dc00640 1 --2- 172.21.15.1:0/1366820737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b48100fb0 0x7f4b4819eda0 secure :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800d950 tx=0x7f4b3800de10 comp rx=0 tx=0).stop 2024-04-26T02:04:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.056+0000 7f4b4dc00640 1 -- 172.21.15.1:0/1366820737 >> 172.21.15.1:0/1366820737 conn(0x7f4b48077f70 msgr2=0x7f4b48103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:10.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.056+0000 7f6b74c00700 1 -- 172.21.15.1:0/2731934874 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5636b61226f0 con 0x5636b61dc7b0 2024-04-26T02:04:10.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.056+0000 7f6b74c00700 10 client.26699 handle_command_reply: tid=0 2024-04-26T02:04:10.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.056+0000 7f6b9c02ab80 1 client.26699 shutdown 2024-04-26T02:04:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.056+0000 7f4b4dc00640 1 -- 172.21.15.1:0/1366820737 shutdown_connections 2024-04-26T02:04:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-04-26T02:04:10.056+0000 7f4b4dc00640 1 -- 172.21.15.1:0/1366820737 wait complete. 2024-04-26T02:04:10.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.057+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/2731934874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61cd060 msgr2=0x5636b61ca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:10.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.057+0000 7f6b9c02ab80 1 --2- 172.21.15.1:0/2731934874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61cd060 0x5636b61ca090 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400c850 tx=0x7f6b840a4d80 comp rx=0 tx=0).stop 2024-04-26T02:04:10.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.057+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/2731934874 shutdown_connections 2024-04-26T02:04:10.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.057+0000 7f6b9c02ab80 1 --2- 172.21.15.1:0/2731934874 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5636b61dc7b0 0x5636b61deba0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.057+0000 7f6b74c00700 0 client.26699 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:04:10.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.057+0000 7f6b9c02ab80 1 --2- 172.21.15.1:0/2731934874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61cd060 0x5636b61ca090 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.057+0000 7f6b9c02ab80 1 --2- 172.21.15.1:0/2731934874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c1890 0x5636b61cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.057+0000 7f6b9c02ab80 1 --2- 172.21.15.1:0/2731934874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61c21b0 0x5636b61c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.057+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/2731934874 >> 172.21.15.1:0/2731934874 conn(0x5636b60f0610 msgr2=0x5636b61bd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:10.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.058+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/2731934874 shutdown_connections 2024-04-26T02:04:10.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.058+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/2731934874 wait complete. 2024-04-26T02:04:10.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.058+0000 7f6b9c02ab80 20 client.26699 trim_cache size 0 max 16384 2024-04-26T02:04:10.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.091+0000 7f6b9c02ab80 1 Processor -- start 2024-04-26T02:04:10.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.091+0000 7f6b9c02ab80 1 -- start start 2024-04-26T02:04:10.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.091+0000 7f6b9c02ab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61c1890 0x5636b61c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.091+0000 7f6b9c02ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c21b0 0x5636b61c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.091+0000 7f6b9c02ab80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61cd060 0x5636b61c7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.091+0000 7f6b9c02ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1b6440 con 0x5636b61c21b0 2024-04-26T02:04:10.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.091+0000 7f6b9c02ab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1b3ca0 con 0x5636b61c1890 2024-04-26T02:04:10.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.091+0000 7f6b9c02ab80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1b1b10 con 0x5636b61cd060 2024-04-26T02:04:10.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.091+0000 7f6b8aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61c1890 0x5636b61c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.091+0000 7f6b8aa00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61c1890 0x5636b61c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:48274/0 (socket says 172.21.15.1:48274) 2024-04-26T02:04:10.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.091+0000 7f6b8b400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61cd060 0x5636b61c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.091+0000 7f6b8aa00700 1 -- 172.21.15.1:0/1171255049 learned_addr learned my addr 172.21.15.1:0/1171255049 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:10.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.092+0000 7f6b8a000700 1 --2- 172.21.15.1:0/1171255049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c21b0 0x5636b61c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.092+0000 7f6b8aa00700 1 -- 172.21.15.1:0/1171255049 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61cd060 msgr2=0x5636b61c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:10.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.092+0000 7f6b8aa00700 1 --2- 172.21.15.1:0/1171255049 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61cd060 0x5636b61c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.092+0000 7f6b8aa00700 1 -- 172.21.15.1:0/1171255049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c21b0 msgr2=0x5636b61c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:10.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.092+0000 7f6b8aa00700 1 --2- 172.21.15.1:0/1171255049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c21b0 0x5636b61c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.092+0000 7f6b8aa00700 1 -- 172.21.15.1:0/1171255049 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636b61cc490 con 0x5636b61c1890 2024-04-26T02:04:10.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.092+0000 7f6b8b400700 1 --2- 172.21.15.1:0/1171255049 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61cd060 0x5636b61c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:04:10.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.092+0000 7f6b8a000700 1 --2- 172.21.15.1:0/1171255049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c21b0 0x5636b61c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:04:10.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.092+0000 7f6b8aa00700 1 --2- 172.21.15.1:0/1171255049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61c1890 0x5636b61c8a20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00bc30 tx=0x7f6b7c01e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4687f415f5351f4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:10.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.093+0000 7f6b77400700 1 -- 172.21.15.1:0/1171255049 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c00faa0 con 0x5636b61c1890 2024-04-26T02:04:10.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.093+0000 7f6b77400700 1 -- 172.21.15.1:0/1171255049 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b7c00fc40 con 0x5636b61c1890 2024-04-26T02:04:10.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.093+0000 7f6b77400700 1 -- 172.21.15.1:0/1171255049 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c02f650 con 0x5636b61c1890 2024-04-26T02:04:10.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.093+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/1171255049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61c1890 msgr2=0x5636b61c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:10.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.093+0000 7f6b9c02ab80 1 --2- 172.21.15.1:0/1171255049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61c1890 0x5636b61c8a20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00bc30 tx=0x7f6b7c01e000 comp rx=0 tx=0).stop 2024-04-26T02:04:10.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.093+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/1171255049 shutdown_connections 2024-04-26T02:04:10.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.093+0000 7f6b9c02ab80 1 --2- 172.21.15.1:0/1171255049 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61cd060 0x5636b61c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.093+0000 7f6b9c02ab80 1 --2- 172.21.15.1:0/1171255049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61c1890 0x5636b61c8a20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.093+0000 7f6b9c02ab80 1 --2- 172.21.15.1:0/1171255049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c21b0 0x5636b61c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.094+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/1171255049 >> 172.21.15.1:0/1171255049 conn(0x5636b60f0610 msgr2=0x5636b61c8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:10.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.094+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/1171255049 shutdown_connections 2024-04-26T02:04:10.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.094+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/1171255049 wait complete. 2024-04-26T02:04:10.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.094+0000 7f6b9c02ab80 1 -- start start 2024-04-26T02:04:10.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.094+0000 7f6b9c02ab80 1 Processor -- start 2024-04-26T02:04:10.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.095+0000 7f6b9c02ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c1890 0x5636b61e0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.095+0000 7f6b9c02ab80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61c21b0 0x5636b61e0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.095+0000 7f6b9c02ab80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61cd060 0x5636b61e10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.095+0000 7f6b9c02ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1b2160 con 0x5636b61c1890 2024-04-26T02:04:10.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.095+0000 7f6b9c02ab80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1b35d0 con 0x5636b61cd060 2024-04-26T02:04:10.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.095+0000 7f6b9c02ab80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c1b2830 con 0x5636b61c21b0 2024-04-26T02:04:10.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.095+0000 7f6b9c02ab80 10 client.0 _ll_register_callbacks cb 0x5636b60ad790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:04:10.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.095+0000 7f6b8aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c1890 0x5636b61e0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.095+0000 7f6b8aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c1890 0x5636b61e0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51590/0 (socket says 172.21.15.1:51590) 2024-04-26T02:04:10.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.095+0000 7f6b8a000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61c21b0 0x5636b61e0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.095+0000 7f6b8aa00700 1 -- 172.21.15.1:0/2969899408 learned_addr learned my addr 172.21.15.1:0/2969899408 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:10.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.096+0000 7f6b8b400700 1 --2- 172.21.15.1:0/2969899408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61cd060 0x5636b61e10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.096+0000 7f6b8a000700 1 -- 172.21.15.1:0/2969899408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61cd060 msgr2=0x5636b61e10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:10.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.096+0000 7f6b8a000700 1 --2- 172.21.15.1:0/2969899408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61cd060 0x5636b61e10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.096+0000 7f6b8a000700 1 -- 172.21.15.1:0/2969899408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c1890 msgr2=0x5636b61e0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:10.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.096+0000 7f6b8a000700 1 --2- 172.21.15.1:0/2969899408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c1890 0x5636b61e0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.096+0000 7f6b8a000700 1 -- 172.21.15.1:0/2969899408 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636b62429f0 con 0x5636b61c21b0 2024-04-26T02:04:10.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.096+0000 7f6b8a000700 1 --2- 172.21.15.1:0/2969899408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61c21b0 0x5636b61e0bb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5636b61cadb0 tx=0x7f6b780126d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0ab557f5e4c171 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:10.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.096+0000 7f6b74c00700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T02:04:10.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.097+0000 7f6b74c00700 1 -- 172.21.15.1:0/2969899408 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b78020060 con 0x5636b61c21b0 2024-04-26T02:04:10.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.097+0000 7f6b74c00700 1 -- 172.21.15.1:0/2969899408 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f6b78010650 con 0x5636b61c21b0 2024-04-26T02:04:10.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.097+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/2969899408 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b7c00fc40 con 0x5636b61c21b0 2024-04-26T02:04:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.097+0000 7f6b74c00700 1 -- 172.21.15.1:0/2969899408 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6b78010850 con 0x5636b61c21b0 2024-04-26T02:04:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.097+0000 7f6b9c02ab80 10 client.26716 fetch_fsmap learned FSMap version 36 2024-04-26T02:04:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.097+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/2969899408 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5636b6243da0 con 0x5636b61c21b0 2024-04-26T02:04:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.097+0000 7f6b74c00700 1 -- 172.21.15.1:0/2969899408 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b7c00fc40 con 0x5636b61c21b0 2024-04-26T02:04:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.098+0000 7f6b74c00700 1 -- 172.21.15.1:0/2969899408 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f6b7800fbd0 con 0x5636b61c21b0 2024-04-26T02:04:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.098+0000 7f6b9c02ab80 10 client.26716 fetch_fsmap finished waiting for FSMap version 36 2024-04-26T02:04:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.098+0000 7f6b9c02ab80 10 client.26716 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T02:04:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.098+0000 7f6b9c02ab80 20 client.26716 populate_metadata read hostname 'smithi001' 2024-04-26T02:04:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.098+0000 7f6b9c02ab80 1 --2- 172.21.15.1:0/2969899408 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5636b62444e0 0x5636b6246900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.098+0000 7f6b9c02ab80 4 client.26716 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-26T02:04:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.098+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/2969899408 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x5636b611c790 con 0x5636b62444e0 2024-04-26T02:04:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.098+0000 7f6b8aa00700 1 --2- 172.21.15.1:0/2969899408 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5636b62444e0 0x5636b6246900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.099+0000 7f6b74c00700 10 client.26716 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:04:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.099+0000 7f6b8aa00700 1 --2- 172.21.15.1:0/2969899408 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5636b62444e0 0x5636b6246900 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bd2948bcce2f2a9 server_cookie=9dd2d71a0306e8f2 in_seq=0 out_seq=0 2024-04-26T02:04:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.099+0000 7f6b74c00700 1 -- 172.21.15.1:0/2969899408 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x5636b611c790 con 0x5636b62444e0 2024-04-26T02:04:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.099+0000 7f6b74c00700 10 client.26716 handle_command_reply: tid=0 2024-04-26T02:04:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.099+0000 7f6b9c02ab80 1 client.26716 shutdown 2024-04-26T02:04:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.099+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/2969899408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61c21b0 msgr2=0x5636b61e0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.099+0000 7f6b9c02ab80 1 --2- 172.21.15.1:0/2969899408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61c21b0 0x5636b61e0bb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5636b61cadb0 tx=0x7f6b780126d0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.099+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/2969899408 shutdown_connections 2024-04-26T02:04:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.099+0000 7f6b9c02ab80 1 --2- 172.21.15.1:0/2969899408 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5636b62444e0 0x5636b6246900 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.099+0000 7f6b9c02ab80 1 --2- 172.21.15.1:0/2969899408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5636b61cd060 0x5636b61e10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.099+0000 7f6b9c02ab80 1 --2- 172.21.15.1:0/2969899408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636b61c1890 0x5636b61e0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.099+0000 7f6b9c02ab80 1 --2- 172.21.15.1:0/2969899408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5636b61c21b0 0x5636b61e0bb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.100+0000 7f6b74c00700 0 client.26716 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:04:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.100+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/2969899408 >> 172.21.15.1:0/2969899408 conn(0x5636b60f0610 msgr2=0x5636b61c8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.100+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/2969899408 shutdown_connections 2024-04-26T02:04:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.100+0000 7f6b9c02ab80 1 -- 172.21.15.1:0/2969899408 wait complete. 2024-04-26T02:04:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.100+0000 7f6b9c02ab80 20 client.26716 trim_cache size 0 max 16384 2024-04-26T02:04:10.101 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T02:04:10.101 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 36, 2024-04-26T02:04:10.101 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24518, 2024-04-26T02:04:10.101 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 36, 2024-04-26T02:04:10.101 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": {} 2024-04-26T02:04:10.101 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T02:04:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.101+0000 7f6b91600700 1 -- 172.21.15.1:0/3818285127 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6b700a3f80 msgr2=0x7f6b700a6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.101+0000 7f6b91600700 1 --2- 172.21.15.1:0/3818285127 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6b700a3f80 0x7f6b700a6440 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00b2e0 tx=0x7f6b7c01a040 comp rx=0 tx=0).stop 2024-04-26T02:04:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.101+0000 7f6b91600700 1 -- 172.21.15.1:0/3818285127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b8c113c10 msgr2=0x7f6b8c19f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.101+0000 7f6b91600700 1 --2- 172.21.15.1:0/3818285127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b8c113c10 0x7f6b8c19f5e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400ef90 tx=0x7f6b8400c450 comp rx=0 tx=0).stop 2024-04-26T02:04:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.101+0000 7f6b91600700 1 -- 172.21.15.1:0/3818285127 shutdown_connections 2024-04-26T02:04:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.101+0000 7f6b91600700 1 --2- 172.21.15.1:0/3818285127 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f6b700a3f80 0x7f6b700a6440 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.101+0000 7f6b91600700 1 --2- 172.21.15.1:0/3818285127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b8c113c10 0x7f6b8c19f5e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.101+0000 7f6b91600700 1 --2- 172.21.15.1:0/3818285127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b8c074c70 0x7f6b8c1a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.101+0000 7f6b91600700 1 --2- 172.21.15.1:0/3818285127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b8c075590 0x7f6b8c1a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.101+0000 7f6b91600700 1 -- 172.21.15.1:0/3818285127 >> 172.21.15.1:0/3818285127 conn(0x7f6b8c079ba0 msgr2=0x7f6b8c112a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.101+0000 7f6b91600700 1 -- 172.21.15.1:0/3818285127 shutdown_connections 2024-04-26T02:04:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.101+0000 7f6b91600700 1 -- 172.21.15.1:0/3818285127 wait complete. 2024-04-26T02:04:10.301 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:10 smithi001 ceph-mon[27822]: pgmap v392: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 51 KiB/s wr, 25 op/s 2024-04-26T02:04:10.301 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:10 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:04:10.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:10 smithi097 ceph-mon[32853]: pgmap v392: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 51 KiB/s wr, 25 op/s 2024-04-26T02:04:10.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:10 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:04:10.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:10 smithi173 ceph-mon[32553]: pgmap v392: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 51 KiB/s wr, 25 op/s 2024-04-26T02:04:10.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:10 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:04:10.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.908+0000 7f0db2a00700 1 -- 172.21.15.1:0/3655118057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0dac111290 msgr2=0x7f0dac109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:10.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.908+0000 7f0db2a00700 1 --2- 172.21.15.1:0/3655118057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0dac111290 0x7f0dac109a40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0099c0 tx=0x7f0d9c009cd0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.908+0000 7f0db2a00700 1 -- 172.21.15.1:0/3655118057 shutdown_connections 2024-04-26T02:04:10.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.908+0000 7f0db2a00700 1 --2- 172.21.15.1:0/3655118057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0dac11c220 0x7f0dac11e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.908+0000 7f0db2a00700 1 --2- 172.21.15.1:0/3655118057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0dac111290 0x7f0dac109a40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.908+0000 7f0db2a00700 1 --2- 172.21.15.1:0/3655118057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dac109f80 0x7f0dac10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.908+0000 7f0db2a00700 1 -- 172.21.15.1:0/3655118057 >> 172.21.15.1:0/3655118057 conn(0x7f0dac06dbb0 msgr2=0x7f0dac06dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:10.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.909+0000 7f0db2a00700 1 -- 172.21.15.1:0/3655118057 shutdown_connections 2024-04-26T02:04:10.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.909+0000 7f0db2a00700 1 -- 172.21.15.1:0/3655118057 wait complete. 2024-04-26T02:04:10.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.911+0000 7f0db2a00700 1 Processor -- start 2024-04-26T02:04:10.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.912+0000 7f0db2a00700 1 -- start start 2024-04-26T02:04:10.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.914+0000 7f0db2a00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dac109f80 0x7f0dac1a7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.914+0000 7f0db2a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0dac111290 0x7f0dac1a83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.914+0000 7f0db2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0dac11c220 0x7f0dac1ac730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.915+0000 7f0db2a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac121ad0 con 0x7f0dac11c220 2024-04-26T02:04:10.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.915+0000 7f0db2a00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac121950 con 0x7f0dac109f80 2024-04-26T02:04:10.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.915+0000 7f0db2a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac121c50 con 0x7f0dac111290 2024-04-26T02:04:10.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.915+0000 7f0dabe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dac109f80 0x7f0dac1a7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.915+0000 7f0dabe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dac109f80 0x7f0dac1a7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:48306/0 (socket says 172.21.15.1:48306) 2024-04-26T02:04:10.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.915+0000 7f0dab400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0dac111290 0x7f0dac1a83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.916+0000 7f0dab400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0dac111290 0x7f0dac1a83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:58848/0 (socket says 172.21.15.1:58848) 2024-04-26T02:04:10.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.916+0000 7f0dab400700 1 -- 172.21.15.1:0/1736692112 learned_addr learned my addr 172.21.15.1:0/1736692112 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:10.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.916+0000 7f0db0a00700 1 --2- 172.21.15.1:0/1736692112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0dac11c220 0x7f0dac1ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.916+0000 7f0dab400700 1 -- 172.21.15.1:0/1736692112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dac109f80 msgr2=0x7f0dac1a7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:10.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.916+0000 7f0dab400700 1 --2- 172.21.15.1:0/1736692112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dac109f80 0x7f0dac1a7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.916+0000 7f0dab400700 1 -- 172.21.15.1:0/1736692112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0dac11c220 msgr2=0x7f0dac1ac730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:10.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.916+0000 7f0dab400700 1 --2- 172.21.15.1:0/1736692112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0dac11c220 0x7f0dac1ac730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.916+0000 7f0dab400700 1 -- 172.21.15.1:0/1736692112 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dac1ace70 con 0x7f0dac111290 2024-04-26T02:04:10.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.917+0000 7f0db0a00700 1 --2- 172.21.15.1:0/1736692112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0dac11c220 0x7f0dac1ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:04:10.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.917+0000 7f0dab400700 1 --2- 172.21.15.1:0/1736692112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0dac111290 0x7f0dac1a83a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800d980 tx=0x7f0d9800dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:10.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.917+0000 7f0da8c00700 1 -- 172.21.15.1:0/1736692112 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d98016070 con 0x7f0dac111290 2024-04-26T02:04:10.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.917+0000 7f0da8c00700 1 -- 172.21.15.1:0/1736692112 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d980044d0 con 0x7f0dac111290 2024-04-26T02:04:10.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.917+0000 7f0da8c00700 1 -- 172.21.15.1:0/1736692112 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d98004d90 con 0x7f0dac111290 2024-04-26T02:04:10.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.918+0000 7f0db2a00700 1 -- 172.21.15.1:0/1736692112 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dac1ad160 con 0x7f0dac111290 2024-04-26T02:04:10.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.918+0000 7f0db2a00700 1 -- 172.21.15.1:0/1736692112 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dac1b49e0 con 0x7f0dac111290 2024-04-26T02:04:10.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.930+0000 7f0da8c00700 1 -- 172.21.15.1:0/1736692112 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f0d98004020 con 0x7f0dac111290 2024-04-26T02:04:10.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.933+0000 7f0da8c00700 1 --2- 172.21.15.1:0/1736692112 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0d900a3f20 0x7f0d900a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.934+0000 7f0da8c00700 1 -- 172.21.15.1:0/1736692112 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8615+0+0 (secure 0 0 0) 0x7f0d980d0650 con 0x7f0dac111290 2024-04-26T02:04:10.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.934+0000 7f0dabe00700 1 --2- 172.21.15.1:0/1736692112 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0d900a3f20 0x7f0d900a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.935+0000 7f0dbd3bbb80 1 Processor -- start 2024-04-26T02:04:10.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.936+0000 7f0dabe00700 1 --2- 172.21.15.1:0/1736692112 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0d900a3f20 0x7f0d900a63e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c006010 tx=0x7f0d9c0036e0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:10.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.937+0000 7f0dbd3bbb80 1 -- start start 2024-04-26T02:04:10.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.938+0000 7f0dbd3bbb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a3f9b0 0x5651d9a3fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.938+0000 7f0dbd3bbb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a402d0 0x5651d9a4ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.938+0000 7f0dabe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a3f9b0 0x5651d9a3fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.939+0000 7f0dbd3bbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a4b180 0x5651d9a4d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.939+0000 7f0dbd3bbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5651d9993020 con 0x5651d9a4b180 2024-04-26T02:04:10.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.939+0000 7f0dbd3bbb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5651d99398c0 con 0x5651d9a3f9b0 2024-04-26T02:04:10.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.939+0000 7f0dbd3bbb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5651d998efe0 con 0x5651d9a402d0 2024-04-26T02:04:10.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.939+0000 7f0dabe00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a3f9b0 0x5651d9a3fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:48316/0 (socket says 172.21.15.1:48316) 2024-04-26T02:04:10.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.939+0000 7f0dabe00700 1 -- 172.21.15.1:0/1766994951 learned_addr learned my addr 172.21.15.1:0/1766994951 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:10.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.940+0000 7f0dabe00700 1 -- 172.21.15.1:0/1766994951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a402d0 msgr2=0x5651d9a4ac40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T02:04:10.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.940+0000 7f0dabe00700 1 --2- 172.21.15.1:0/1766994951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a402d0 0x5651d9a4ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.940+0000 7f0dabe00700 1 -- 172.21.15.1:0/1766994951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a4b180 msgr2=0x5651d9a4d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:10.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.940+0000 7f0db0a00700 1 --2- 172.21.15.1:0/1766994951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a4b180 0x5651d9a4d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.942+0000 7f0dabe00700 1 --2- 172.21.15.1:0/1766994951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a4b180 0x5651d9a4d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.942+0000 7f0dabe00700 1 -- 172.21.15.1:0/1766994951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651d992f810 con 0x5651d9a3f9b0 2024-04-26T02:04:10.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.942+0000 7f0db0a00700 1 --2- 172.21.15.1:0/1766994951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a4b180 0x5651d9a4d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:04:10.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.943+0000 7f0dabe00700 1 --2- 172.21.15.1:0/1766994951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a3f9b0 0x5651d9a3fd90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c015000 tx=0x7f0d9c01a070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56874e52c0327c59 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:10.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.944+0000 7f0da1600700 1 -- 172.21.15.1:0/1766994951 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c0039c0 con 0x5651d9a3f9b0 2024-04-26T02:04:10.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.945+0000 7f0da1600700 1 -- 172.21.15.1:0/1766994951 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d9c00bc40 con 0x5651d9a3f9b0 2024-04-26T02:04:10.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.945+0000 7f0da1600700 1 -- 172.21.15.1:0/1766994951 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c0106d0 con 0x5651d9a3f9b0 2024-04-26T02:04:10.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.946+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/1766994951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a3f9b0 msgr2=0x5651d9a3fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:10.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.946+0000 7f0dbd3bbb80 1 --2- 172.21.15.1:0/1766994951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a3f9b0 0x5651d9a3fd90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c015000 tx=0x7f0d9c01a070 comp rx=0 tx=0).stop 2024-04-26T02:04:10.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.946+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/1766994951 shutdown_connections 2024-04-26T02:04:10.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.946+0000 7f0dbd3bbb80 1 --2- 172.21.15.1:0/1766994951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a4b180 0x5651d9a4d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.946+0000 7f0dbd3bbb80 1 --2- 172.21.15.1:0/1766994951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a3f9b0 0x5651d9a3fd90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.946+0000 7f0dbd3bbb80 1 --2- 172.21.15.1:0/1766994951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a402d0 0x5651d9a4ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.946+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/1766994951 >> 172.21.15.1:0/1766994951 conn(0x5651d996e730 msgr2=0x5651d9a3be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:10.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.946+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/1766994951 shutdown_connections 2024-04-26T02:04:10.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.947+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/1766994951 wait complete. 2024-04-26T02:04:10.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.948+0000 7f0dbd3bbb80 1 -- start start 2024-04-26T02:04:10.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.948+0000 7f0dbd3bbb80 1 Processor -- start 2024-04-26T02:04:10.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.950+0000 7f0dbd3bbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a3f9b0 0x5651d9a4a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.952+0000 7f0dbd3bbb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a402d0 0x5651d9a47b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.952+0000 7f0dbd3bbb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a4b180 0x5651d9a480c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.952+0000 7f0dbd3bbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5651d9a4dc40 con 0x5651d9a3f9b0 2024-04-26T02:04:10.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.952+0000 7f0dab400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a402d0 0x5651d9a47b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.952+0000 7f0dbd3bbb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5651d994c9f0 con 0x5651d9a4b180 2024-04-26T02:04:10.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.952+0000 7f0dab400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a402d0 0x5651d9a47b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:58876/0 (socket says 172.21.15.1:58876) 2024-04-26T02:04:10.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.952+0000 7f0dbd3bbb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5651d9a4ddc0 con 0x5651d9a402d0 2024-04-26T02:04:10.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.952+0000 7f0dab400700 1 -- 172.21.15.1:0/2739512909 learned_addr learned my addr 172.21.15.1:0/2739512909 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:10.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.952+0000 7f0dbd3bbb80 10 client.0 _ll_register_callbacks cb 0x5651d992b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:04:10.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.952+0000 7f0db0a00700 1 --2- 172.21.15.1:0/2739512909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a4b180 0x5651d9a480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.953+0000 7f0dab400700 1 -- 172.21.15.1:0/2739512909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a4b180 msgr2=0x5651d9a480c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:10.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.953+0000 7f0dab400700 1 --2- 172.21.15.1:0/2739512909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a4b180 0x5651d9a480c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.953+0000 7f0dab400700 1 -- 172.21.15.1:0/2739512909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a3f9b0 msgr2=0x5651d9a4a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:10.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.953+0000 7f0dab400700 1 --2- 172.21.15.1:0/2739512909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a3f9b0 0x5651d9a4a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.953+0000 7f0dab400700 1 -- 172.21.15.1:0/2739512909 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651d98e0a60 con 0x5651d9a402d0 2024-04-26T02:04:10.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.953+0000 7f0db0a00700 1 --2- 172.21.15.1:0/2739512909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a4b180 0x5651d9a480c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:04:10.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.953+0000 7f0dab400700 1 --2- 172.21.15.1:0/2739512909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a402d0 0x5651d9a47b80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0d98098f60 tx=0x7f0d9809c0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de7d24ccda04fef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:10.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.953+0000 7f0d96000700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-26T02:04:10.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.954+0000 7f0d96000700 1 -- 172.21.15.1:0/2739512909 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9800f5c0 con 0x5651d9a402d0 2024-04-26T02:04:10.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.954+0000 7f0d96000700 1 -- 172.21.15.1:0/2739512909 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8615+0+0 (secure 0 0 0) 0x7f0d9809c7f0 con 0x5651d9a402d0 2024-04-26T02:04:10.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.954+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/2739512909 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d9c00bc40 con 0x5651d9a402d0 2024-04-26T02:04:10.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.954+0000 7f0d96000700 1 -- 172.21.15.1:0/2739512909 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0d98004310 con 0x5651d9a402d0 2024-04-26T02:04:10.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.954+0000 7f0dbd3bbb80 10 client.26725 fetch_fsmap learned FSMap version 36 2024-04-26T02:04:10.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.955+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/2739512909 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5651d9929c80 con 0x5651d9a402d0 2024-04-26T02:04:10.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.956+0000 7f0d96000700 1 -- 172.21.15.1:0/2739512909 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d9c00bc40 con 0x5651d9a402d0 2024-04-26T02:04:10.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.957+0000 7f0d96000700 1 -- 172.21.15.1:0/2739512909 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f0d980a4d70 con 0x5651d9a402d0 2024-04-26T02:04:10.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.958+0000 7f0dbd3bbb80 10 client.26725 fetch_fsmap finished waiting for FSMap version 36 2024-04-26T02:04:10.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.958+0000 7f0dbd3bbb80 10 client.26725 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T02:04:10.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.958+0000 7f0dbd3bbb80 20 client.26725 populate_metadata read hostname 'smithi001' 2024-04-26T02:04:10.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.958+0000 7f0dbd3bbb80 1 --2- 172.21.15.1:0/2739512909 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5651d9a5a900 0x5651d9a5ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:10.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.958+0000 7f0dbd3bbb80 4 client.26725 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:04:10.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.958+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/2739512909 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651d99a0810 con 0x5651d9a5a900 2024-04-26T02:04:10.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.958+0000 7f0dabe00700 1 --2- 172.21.15.1:0/2739512909 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5651d9a5a900 0x5651d9a5ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:10.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.959+0000 7f0dabe00700 1 --2- 172.21.15.1:0/2739512909 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5651d9a5a900 0x5651d9a5ccf0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a13079d66678114 server_cookie=7d9743e33578d355 in_seq=0 out_seq=0 2024-04-26T02:04:10.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.959+0000 7f0d96000700 10 client.26725 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:04:10.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.960+0000 7f0d96000700 1 -- 172.21.15.1:0/2739512909 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651d99a0810 con 0x5651d9a5a900 2024-04-26T02:04:10.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.960+0000 7f0d96000700 10 client.26725 handle_command_reply: tid=0 2024-04-26T02:04:10.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.960+0000 7f0dbd3bbb80 1 client.26725 shutdown 2024-04-26T02:04:10.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.961+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/2739512909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a402d0 msgr2=0x5651d9a47b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:10.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.961+0000 7f0dbd3bbb80 1 --2- 172.21.15.1:0/2739512909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a402d0 0x5651d9a47b80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0d98098f60 tx=0x7f0d9809c0d0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.961+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/2739512909 shutdown_connections 2024-04-26T02:04:10.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.961+0000 7f0dbd3bbb80 1 --2- 172.21.15.1:0/2739512909 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5651d9a5a900 0x5651d9a5ccf0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.961+0000 7f0dbd3bbb80 1 --2- 172.21.15.1:0/2739512909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a4b180 0x5651d9a480c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.961+0000 7f0dbd3bbb80 1 --2- 172.21.15.1:0/2739512909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a3f9b0 0x5651d9a4a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.961+0000 7f0dbd3bbb80 1 --2- 172.21.15.1:0/2739512909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a402d0 0x5651d9a47b80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:10.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.961+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/2739512909 >> 172.21.15.1:0/2739512909 conn(0x5651d996e730 msgr2=0x5651d9a3ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:10.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.962+0000 7f0d96000700 0 client.26725 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:04:10.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.962+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/2739512909 shutdown_connections 2024-04-26T02:04:10.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.962+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/2739512909 wait complete. 2024-04-26T02:04:10.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:10.962+0000 7f0dbd3bbb80 20 client.26725 trim_cache size 0 max 16384 2024-04-26T02:04:11.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.005+0000 7f0dbd3bbb80 1 Processor -- start 2024-04-26T02:04:11.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.005+0000 7f0dbd3bbb80 1 -- start start 2024-04-26T02:04:11.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.005+0000 7f0dbd3bbb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a3f9b0 0x5651d9a624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:11.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.005+0000 7f0dbd3bbb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a4b180 0x5651d9a629f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:11.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.005+0000 7f0dbd3bbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a5cfe0 0x5651d9a53090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:11.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.005+0000 7f0dbd3bbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1beb80 con 0x5651d9a5cfe0 2024-04-26T02:04:11.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.005+0000 7f0dbd3bbb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1bc3e0 con 0x5651d9a4b180 2024-04-26T02:04:11.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.005+0000 7f0dbd3bbb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1ba250 con 0x5651d9a3f9b0 2024-04-26T02:04:11.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.006+0000 7f0dab400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a4b180 0x5651d9a629f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:11.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.006+0000 7f0db0a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a5cfe0 0x5651d9a53090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:11.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.006+0000 7f0dabe00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a3f9b0 0x5651d9a624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:11.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.006+0000 7f0dab400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a4b180 0x5651d9a629f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:48336/0 (socket says 172.21.15.1:48336) 2024-04-26T02:04:11.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.007+0000 7f0dab400700 1 -- 172.21.15.1:0/4141031182 learned_addr learned my addr 172.21.15.1:0/4141031182 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:11.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.007+0000 7f0db0a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a5cfe0 0x5651d9a53090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51626/0 (socket says 172.21.15.1:51626) 2024-04-26T02:04:11.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.007+0000 7f0db0a00700 1 -- 172.21.15.1:0/4141031182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a3f9b0 msgr2=0x5651d9a624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:11.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.007+0000 7f0db0a00700 1 --2- 172.21.15.1:0/4141031182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a3f9b0 0x5651d9a624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:11.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.007+0000 7f0db0a00700 1 -- 172.21.15.1:0/4141031182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a4b180 msgr2=0x5651d9a629f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:11.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.007+0000 7f0db0a00700 1 --2- 172.21.15.1:0/4141031182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a4b180 0x5651d9a629f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:11.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.007+0000 7f0db0a00700 1 -- 172.21.15.1:0/4141031182 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651d9a46110 con 0x5651d9a5cfe0 2024-04-26T02:04:11.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.008+0000 7f0db0a00700 1 --2- 172.21.15.1:0/4141031182 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a5cfe0 0x5651d9a53090 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0da40087e0 tx=0x7f0da400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa748633065eb6c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:11.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.009+0000 7f0da0c00700 1 -- 172.21.15.1:0/4141031182 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4016610 con 0x5651d9a5cfe0 2024-04-26T02:04:11.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.010+0000 7f0da0c00700 1 -- 172.21.15.1:0/4141031182 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0da401b070 con 0x5651d9a5cfe0 2024-04-26T02:04:11.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.010+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/4141031182 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a5cfe0 msgr2=0x5651d9a53090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:11.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.011+0000 7f0da0c00700 1 -- 172.21.15.1:0/4141031182 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4016610 con 0x5651d9a5cfe0 2024-04-26T02:04:11.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.011+0000 7f0dbd3bbb80 1 --2- 172.21.15.1:0/4141031182 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a5cfe0 0x5651d9a53090 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0da40087e0 tx=0x7f0da400b040 comp rx=0 tx=0).stop 2024-04-26T02:04:11.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.011+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/4141031182 shutdown_connections 2024-04-26T02:04:11.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.012+0000 7f0dbd3bbb80 1 --2- 172.21.15.1:0/4141031182 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a5cfe0 0x5651d9a53090 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:11.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.012+0000 7f0dbd3bbb80 1 --2- 172.21.15.1:0/4141031182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a3f9b0 0x5651d9a624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:11.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.012+0000 7f0dbd3bbb80 1 --2- 172.21.15.1:0/4141031182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a4b180 0x5651d9a629f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:11.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.012+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/4141031182 >> 172.21.15.1:0/4141031182 conn(0x5651d996e730 msgr2=0x5651d9a46490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:11.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.012+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/4141031182 shutdown_connections 2024-04-26T02:04:11.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.013+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/4141031182 wait complete. 2024-04-26T02:04:11.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.014+0000 7f0dbd3bbb80 1 -- start start 2024-04-26T02:04:11.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.014+0000 7f0dbd3bbb80 1 Processor -- start 2024-04-26T02:04:11.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.015+0000 7f0dbd3bbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a3f9b0 0x5651d9abc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:11.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.016+0000 7f0dbd3bbb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a4b180 0x5651d9abacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:11.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.016+0000 7f0dbd3bbb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a5cfe0 0x5651d9abb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:11.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.017+0000 7f0db0a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a5cfe0 0x5651d9abb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:11.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.017+0000 7f0dbd3bbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1ba8a0 con 0x5651d9a3f9b0 2024-04-26T02:04:11.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.017+0000 7f0dbd3bbb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1bbd10 con 0x5651d9a4b180 2024-04-26T02:04:11.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.017+0000 7f0db0a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a5cfe0 0x5651d9abb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:58896/0 (socket says 172.21.15.1:58896) 2024-04-26T02:04:11.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.017+0000 7f0db0a00700 1 -- 172.21.15.1:0/2588903268 learned_addr learned my addr 172.21.15.1:0/2588903268 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:11.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.017+0000 7f0dab400700 1 --2- 172.21.15.1:0/2588903268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a4b180 0x5651d9abacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:11.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.018+0000 7f0dabe00700 1 --2- 172.21.15.1:0/2588903268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a3f9b0 0x5651d9abc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:11.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.018+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/2588903268 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac1baf70 con 0x5651d9a5cfe0 2024-04-26T02:04:11.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.019+0000 7f0dbd3bbb80 10 client.0 _ll_register_callbacks cb 0x5651d992b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:04:11.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.019+0000 7f0dabe00700 1 -- 172.21.15.1:0/2588903268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a5cfe0 msgr2=0x5651d9abb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:11.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.019+0000 7f0dabe00700 1 --2- 172.21.15.1:0/2588903268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a5cfe0 0x5651d9abb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:11.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.019+0000 7f0dabe00700 1 -- 172.21.15.1:0/2588903268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a4b180 msgr2=0x5651d9abacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:04:11.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.020+0000 7f0dabe00700 1 --2- 172.21.15.1:0/2588903268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a4b180 0x5651d9abacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:11.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.020+0000 7f0dabe00700 1 -- 172.21.15.1:0/2588903268 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651d9abba70 con 0x5651d9a3f9b0 2024-04-26T02:04:11.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.020+0000 7f0dab400700 1 --2- 172.21.15.1:0/2588903268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a4b180 0x5651d9abacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:04:11.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.021+0000 7f0dabe00700 1 --2- 172.21.15.1:0/2588903268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a3f9b0 0x5651d9abc5c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x5651d9a5d450 tx=0x7f0d9c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64faf4a99bc1329 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:11.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.021+0000 7f0d96000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-04-26T02:04:11.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.022+0000 7f0d96000700 1 -- 172.21.15.1:0/2588903268 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c00b790 con 0x5651d9a3f9b0 2024-04-26T02:04:11.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.022+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/2588903268 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0da401b070 con 0x5651d9a3f9b0 2024-04-26T02:04:11.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.023+0000 7f0d96000700 1 -- 172.21.15.1:0/2588903268 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8615+0+0 (secure 0 0 0) 0x7f0d9c00b930 con 0x5651d9a3f9b0 2024-04-26T02:04:11.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.024+0000 7f0d96000700 1 -- 172.21.15.1:0/2588903268 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0d9c00bb30 con 0x5651d9a3f9b0 2024-04-26T02:04:11.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.024+0000 7f0dbd3bbb80 10 client.16533 fetch_fsmap learned FSMap version 36 2024-04-26T02:04:11.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.024+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/2588903268 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5651d9a530b0 con 0x5651d9a3f9b0 2024-04-26T02:04:11.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.025+0000 7f0d96000700 1 -- 172.21.15.1:0/2588903268 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0da401b070 con 0x5651d9a3f9b0 2024-04-26T02:04:11.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.025+0000 7f0d96000700 1 -- 172.21.15.1:0/2588903268 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 36) ==== 8346+0+0 (secure 0 0 0) 0x7f0d9c045560 con 0x5651d9a3f9b0 2024-04-26T02:04:11.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.026+0000 7f0dbd3bbb80 10 client.16533 fetch_fsmap finished waiting for FSMap version 36 2024-04-26T02:04:11.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.026+0000 7f0dbd3bbb80 10 client.16533 resolve_mds: validated gid 24518 aka daemon mds.b 2024-04-26T02:04:11.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.026+0000 7f0dbd3bbb80 20 client.16533 populate_metadata read hostname 'smithi001' 2024-04-26T02:04:11.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.026+0000 7f0dbd3bbb80 1 --2- 172.21.15.1:0/2588903268 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5651d9a533f0 0x5651d9ad5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:11.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.026+0000 7f0dbd3bbb80 4 client.16533 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.25, "await_for": 42.0}] 2024-04-26T02:04:11.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.026+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/2588903268 --> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.25, "await_for": 42.0}) -- 0x5651d999a8b0 con 0x5651d9a533f0 2024-04-26T02:04:11.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.027+0000 7f0dab400700 1 --2- 172.21.15.1:0/2588903268 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5651d9a533f0 0x5651d9ad5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:11.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.028+0000 7f0dab400700 1 --2- 172.21.15.1:0/2588903268 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5651d9a533f0 0x5651d9ad5d50 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=795a1c58b08f8173 server_cookie=24908d408a14d47 in_seq=0 out_seq=0 2024-04-26T02:04:11.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.030+0000 7f0d96000700 10 client.16533 ms_handle_connect on v2:172.21.15.97:6838/1315554162 2024-04-26T02:04:11.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.037+0000 7f0d96000700 1 -- 172.21.15.1:0/2588903268 <== mds.0 v2:172.21.15.97:6838/1315554162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5651d999a8b0 con 0x5651d9a533f0 2024-04-26T02:04:11.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.037+0000 7f0d96000700 10 client.16533 handle_command_reply: tid=0 2024-04-26T02:04:11.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.037+0000 7f0dbd3bbb80 1 client.16533 shutdown 2024-04-26T02:04:11.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.037+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/2588903268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a3f9b0 msgr2=0x5651d9abc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:11.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.037+0000 7f0dbd3bbb80 1 --2- 172.21.15.1:0/2588903268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a3f9b0 0x5651d9abc5c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x5651d9a5d450 tx=0x7f0d9c009310 comp rx=0 tx=0).stop 2024-04-26T02:04:11.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.038+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/2588903268 shutdown_connections 2024-04-26T02:04:11.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.038+0000 7f0dbd3bbb80 1 --2- 172.21.15.1:0/2588903268 >> [v2:172.21.15.97:6838/1315554162,v1:172.21.15.97:6839/1315554162] conn(0x5651d9a533f0 0x5651d9ad5d50 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:11.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.038+0000 7f0dbd3bbb80 1 --2- 172.21.15.1:0/2588903268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651d9a5cfe0 0x5651d9abb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:11.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.038+0000 7f0dbd3bbb80 1 --2- 172.21.15.1:0/2588903268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5651d9a3f9b0 0x5651d9abc5c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:11.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.038+0000 7f0dbd3bbb80 1 --2- 172.21.15.1:0/2588903268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5651d9a4b180 0x5651d9abacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:11.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.038+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/2588903268 >> 172.21.15.1:0/2588903268 conn(0x5651d996e730 msgr2=0x5651d9a493b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:11.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.038+0000 7f0d96000700 0 client.16533 ms_handle_reset on v2:172.21.15.97:6838/1315554162 2024-04-26T02:04:11.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.039+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/2588903268 shutdown_connections 2024-04-26T02:04:11.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.039+0000 7f0dbd3bbb80 1 -- 172.21.15.1:0/2588903268 wait complete. 2024-04-26T02:04:11.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.039+0000 7f0dbd3bbb80 20 client.16533 trim_cache size 0 max 16384 2024-04-26T02:04:11.039 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T02:04:11.039 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 36, 2024-04-26T02:04:11.039 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24518, 2024-04-26T02:04:11.039 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 38, 2024-04-26T02:04:11.039 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-04-26T02:04:11.039 INFO:teuthology.orchestra.run.smithi001.stdout: "5c84345f": { 2024-04-26T02:04:11.039 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 38, 2024-04-26T02:04:11.039 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-04-26T02:04:11.039 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T02:04:11.039 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-04-26T02:04:11.039 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T02:04:11.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:11.040 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-04-26T02:04:11.040 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 79.2, 2024-04-26T02:04:11.040 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-04-26T02:04:11.040 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-04-26T02:04:11.040 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-04-26T02:04:11.040 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-04-26T02:04:11.040 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-04-26T02:04:11.040 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-04-26T02:04:11.040 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:11.040 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:11.040 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:11.040 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:11.040 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:11.041 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T02:04:11.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.040+0000 7f0db2a00700 1 -- 172.21.15.1:0/1736692112 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0d900a3f20 msgr2=0x7f0d900a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:11.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.040+0000 7f0db2a00700 1 --2- 172.21.15.1:0/1736692112 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0d900a3f20 0x7f0d900a63e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c006010 tx=0x7f0d9c0036e0 comp rx=0 tx=0).stop 2024-04-26T02:04:11.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.040+0000 7f0db2a00700 1 -- 172.21.15.1:0/1736692112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0dac111290 msgr2=0x7f0dac1a83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:11.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.040+0000 7f0db2a00700 1 --2- 172.21.15.1:0/1736692112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0dac111290 0x7f0dac1a83a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800d980 tx=0x7f0d9800dd40 comp rx=0 tx=0).stop 2024-04-26T02:04:11.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.041+0000 7f0db2a00700 1 -- 172.21.15.1:0/1736692112 shutdown_connections 2024-04-26T02:04:11.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.041+0000 7f0db2a00700 1 --2- 172.21.15.1:0/1736692112 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f0d900a3f20 0x7f0d900a63e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:11.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.041+0000 7f0db2a00700 1 --2- 172.21.15.1:0/1736692112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0dac11c220 0x7f0dac1ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:11.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.041+0000 7f0db2a00700 1 --2- 172.21.15.1:0/1736692112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dac109f80 0x7f0dac1a7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:11.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.041+0000 7f0db2a00700 1 --2- 172.21.15.1:0/1736692112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0dac111290 0x7f0dac1a83a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:11.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.041+0000 7f0db2a00700 1 -- 172.21.15.1:0/1736692112 >> 172.21.15.1:0/1736692112 conn(0x7f0dac06dbb0 msgr2=0x7f0dac11a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:11.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.041+0000 7f0db2a00700 1 -- 172.21.15.1:0/1736692112 shutdown_connections 2024-04-26T02:04:11.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:11.041+0000 7f0db2a00700 1 -- 172.21.15.1:0/1736692112 wait complete. 2024-04-26T02:04:11.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:11 smithi097 ceph-mon[32853]: osdmap e108: 12 total, 12 up, 12 in 2024-04-26T02:04:11.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:11 smithi173 ceph-mon[32553]: osdmap e108: 12 total, 12 up, 12 in 2024-04-26T02:04:11.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:11 smithi001 ceph-mon[27822]: osdmap e108: 12 total, 12 up, 12 in 2024-04-26T02:04:12.107 INFO:tasks.quiescer:done joining Quiescers 2024-04-26T02:04:12.108 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-26T02:04:12.118 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi001.front.sepia.ceph.com 2024-04-26T02:04:12.136 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-26T02:04:12.146 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-26T02:04:12.146 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T02:04:12.147 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-04-26T02:04:12.147 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-26T02:04:12.264 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-26T02:04:12.332 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:12 smithi001 ceph-mon[27822]: pgmap v394: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-04-26T02:04:12.332 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:12 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:04:12.333 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:12 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:04:12.333 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:12 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:04:12.355 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-26T02:04:12.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:12 smithi097 ceph-mon[32853]: pgmap v394: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-04-26T02:04:12.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:12 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:04:12.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:12 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:04:12.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:12 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:04:12.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.561+0000 7fb83fe00640 1 -- 172.21.15.1:0/220662823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8401041f0 msgr2=0x7fb8401045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:12.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.561+0000 7fb83fe00640 1 --2- 172.21.15.1:0/220662823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8401041f0 0x7fb8401045d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb830009900 tx=0x7fb83002a3f0 comp rx=0 tx=0).stop 2024-04-26T02:04:12.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.562+0000 7fb83fe00640 1 -- 172.21.15.1:0/220662823 shutdown_connections 2024-04-26T02:04:12.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.562+0000 7fb83fe00640 1 --2- 172.21.15.1:0/220662823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8401092c0 0x7fb840110e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:12.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.562+0000 7fb83fe00640 1 --2- 172.21.15.1:0/220662823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb840104ba0 0x7fb840108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:12.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.562+0000 7fb83fe00640 1 --2- 172.21.15.1:0/220662823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8401041f0 0x7fb8401045d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:12.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.562+0000 7fb83fe00640 1 -- 172.21.15.1:0/220662823 >> 172.21.15.1:0/220662823 conn(0x7fb8400ffa50 msgr2=0x7fb840101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:12.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.562+0000 7fb83fe00640 1 -- 172.21.15.1:0/220662823 shutdown_connections 2024-04-26T02:04:12.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.562+0000 7fb83fe00640 1 -- 172.21.15.1:0/220662823 wait complete. 2024-04-26T02:04:12.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.563+0000 7fb83fe00640 1 Processor -- start 2024-04-26T02:04:12.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.563+0000 7fb83fe00640 1 -- start start 2024-04-26T02:04:12.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.563+0000 7fb83fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8401041f0 0x7fb8401a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:12.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.563+0000 7fb83fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8401092c0 0x7fb8401a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:12.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.563+0000 7fb83fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb84019a650 0x7fb84019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:12.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.563+0000 7fb83fe00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb840114250 con 0x7fb8401041f0 2024-04-26T02:04:12.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.563+0000 7fb83fe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401140d0 con 0x7fb8401092c0 2024-04-26T02:04:12.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.563+0000 7fb83fe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401143d0 con 0x7fb84019a650 2024-04-26T02:04:12.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.563+0000 7fb83ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8401041f0 0x7fb8401a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:12.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.563+0000 7fb83ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8401041f0 0x7fb8401a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51644/0 (socket says 172.21.15.1:51644) 2024-04-26T02:04:12.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.563+0000 7fb83ea00640 1 -- 172.21.15.1:0/2057654288 learned_addr learned my addr 172.21.15.1:0/2057654288 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:12.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.564+0000 7fb83e000640 1 --2- 172.21.15.1:0/2057654288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8401092c0 0x7fb8401a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:12.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.564+0000 7fb83f400640 1 --2- 172.21.15.1:0/2057654288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb84019a650 0x7fb84019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:12.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.564+0000 7fb83ea00640 1 -- 172.21.15.1:0/2057654288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb84019a650 msgr2=0x7fb84019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:12.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.564+0000 7fb83ea00640 1 --2- 172.21.15.1:0/2057654288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb84019a650 0x7fb84019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:12.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.564+0000 7fb83ea00640 1 -- 172.21.15.1:0/2057654288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8401092c0 msgr2=0x7fb8401a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:12.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.564+0000 7fb83ea00640 1 --2- 172.21.15.1:0/2057654288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8401092c0 0x7fb8401a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:12.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.564+0000 7fb83ea00640 1 -- 172.21.15.1:0/2057654288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84019b170 con 0x7fb8401041f0 2024-04-26T02:04:12.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.564+0000 7fb83ea00640 1 --2- 172.21.15.1:0/2057654288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8401041f0 0x7fb8401a0450 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb830005ab0 tx=0x7fb830003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:12.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.565+0000 7fb837400640 1 -- 172.21.15.1:0/2057654288 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83003f070 con 0x7fb8401041f0 2024-04-26T02:04:12.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.565+0000 7fb83fe00640 1 -- 172.21.15.1:0/2057654288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84019b400 con 0x7fb8401041f0 2024-04-26T02:04:12.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.565+0000 7fb837400640 1 -- 172.21.15.1:0/2057654288 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb830031d60 con 0x7fb8401041f0 2024-04-26T02:04:12.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.565+0000 7fb837400640 1 -- 172.21.15.1:0/2057654288 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830035030 con 0x7fb8401041f0 2024-04-26T02:04:12.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.565+0000 7fb83fe00640 1 -- 172.21.15.1:0/2057654288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8401a7600 con 0x7fb8401041f0 2024-04-26T02:04:12.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.566+0000 7fb83fe00640 1 -- 172.21.15.1:0/2057654288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8400687f0 con 0x7fb8401041f0 2024-04-26T02:04:12.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.568+0000 7fb837400640 1 -- 172.21.15.1:0/2057654288 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fb830042070 con 0x7fb8401041f0 2024-04-26T02:04:12.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:12 smithi173 ceph-mon[32553]: pgmap v394: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-04-26T02:04:12.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:12 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:04:12.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:12 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:04:12.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:12 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:04:12.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.570+0000 7fb837400640 1 --2- 172.21.15.1:0/2057654288 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fb8240a3d80 0x7fb8240a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:12.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.570+0000 7fb837400640 1 -- 172.21.15.1:0/2057654288 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8615+0+0 (secure 0 0 0) 0x7fb8300f0c40 con 0x7fb8401041f0 2024-04-26T02:04:12.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.570+0000 7fb83e000640 1 --2- 172.21.15.1:0/2057654288 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fb8240a3d80 0x7fb8240a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:12.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.571+0000 7fb83e000640 1 --2- 172.21.15.1:0/2057654288 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fb8240a3d80 0x7fb8240a6240 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb8400fff70 tx=0x7fb82c00b410 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:12.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.575+0000 7fb837400640 1 -- 172.21.15.1:0/2057654288 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb8300bd220 con 0x7fb8401041f0 2024-04-26T02:04:12.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.795+0000 7fb83fe00640 1 -- 172.21.15.1:0/2057654288 --> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fb84010c560 con 0x7fb8240a3d80 2024-04-26T02:04:12.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.809+0000 7fb837400640 1 -- 172.21.15.1:0/2057654288 <== mgr.25087 v2:172.21.15.1:6800/275906565 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fb82c00bc40 con 0x7fb8240a3d80 2024-04-26T02:04:12.808 INFO:teuthology.orchestra.run.smithi001.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-26T00:00:00", "created": "2024-04-26T01:51:40", "first": "2024-04-26T01:52:00", "last": "2024-04-26T02:04:00", "last_pruned": "2024-04-26T02:04:01", "created_count": 13, "pruned_count": 7, "active": true} 2024-04-26T02:04:12.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.811+0000 7fb83fe00640 1 -- 172.21.15.1:0/2057654288 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fb8240a3d80 msgr2=0x7fb8240a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:12.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.812+0000 7fb83fe00640 1 --2- 172.21.15.1:0/2057654288 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fb8240a3d80 0x7fb8240a6240 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb8400fff70 tx=0x7fb82c00b410 comp rx=0 tx=0).stop 2024-04-26T02:04:12.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.812+0000 7fb83fe00640 1 -- 172.21.15.1:0/2057654288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8401041f0 msgr2=0x7fb8401a0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:12.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.812+0000 7fb83fe00640 1 --2- 172.21.15.1:0/2057654288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8401041f0 0x7fb8401a0450 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb830005ab0 tx=0x7fb830003c80 comp rx=0 tx=0).stop 2024-04-26T02:04:12.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.812+0000 7fb83fe00640 1 -- 172.21.15.1:0/2057654288 shutdown_connections 2024-04-26T02:04:12.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.812+0000 7fb83fe00640 1 --2- 172.21.15.1:0/2057654288 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fb8240a3d80 0x7fb8240a6240 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:12.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.812+0000 7fb83fe00640 1 --2- 172.21.15.1:0/2057654288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb84019a650 0x7fb84019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:12.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.812+0000 7fb83fe00640 1 --2- 172.21.15.1:0/2057654288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8401092c0 0x7fb8401a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:12.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.812+0000 7fb83fe00640 1 --2- 172.21.15.1:0/2057654288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8401041f0 0x7fb8401a0450 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:12.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.812+0000 7fb83fe00640 1 -- 172.21.15.1:0/2057654288 >> 172.21.15.1:0/2057654288 conn(0x7fb8400ffa50 msgr2=0x7fb840100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:12.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.812+0000 7fb83fe00640 1 -- 172.21.15.1:0/2057654288 shutdown_connections 2024-04-26T02:04:12.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:12.812+0000 7fb83fe00640 1 -- 172.21.15.1:0/2057654288 wait complete. 2024-04-26T02:04:12.830 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-26T02:04:13.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.037+0000 7f92bcc00640 1 -- 172.21.15.1:0/523416787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b810e470 msgr2=0x7f92b810e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:13.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.037+0000 7f92bcc00640 1 --2- 172.21.15.1:0/523416787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b810e470 0x7f92b810e850 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009860 tx=0x7f92a802a3d0 comp rx=0 tx=0).stop 2024-04-26T02:04:13.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.037+0000 7f92bcc00640 1 -- 172.21.15.1:0/523416787 shutdown_connections 2024-04-26T02:04:13.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.037+0000 7f92bcc00640 1 --2- 172.21.15.1:0/523416787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b8108d50 0x7f92b8115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:13.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.037+0000 7f92bcc00640 1 --2- 172.21.15.1:0/523416787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92b81083b0 0x7f92b8108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:13.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.037+0000 7f92bcc00640 1 --2- 172.21.15.1:0/523416787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b810e470 0x7f92b810e850 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:13.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.037+0000 7f92bcc00640 1 -- 172.21.15.1:0/523416787 >> 172.21.15.1:0/523416787 conn(0x7f92b8077b00 msgr2=0x7f92b8079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:13.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.037+0000 7f92bcc00640 1 -- 172.21.15.1:0/523416787 shutdown_connections 2024-04-26T02:04:13.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.038+0000 7f92bcc00640 1 -- 172.21.15.1:0/523416787 wait complete. 2024-04-26T02:04:13.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.039+0000 7f92bcc00640 1 Processor -- start 2024-04-26T02:04:13.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.039+0000 7f92bcc00640 1 -- start start 2024-04-26T02:04:13.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.039+0000 7f92bcc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b81083b0 0x7f92b81a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:13.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.039+0000 7f92bcc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b8108d50 0x7f92b81a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:13.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.039+0000 7f92bcc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92b810e470 0x7f92b819e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:13.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.039+0000 7f92bcc00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f92b8118630 con 0x7f92b810e470 2024-04-26T02:04:13.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.039+0000 7f92bcc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f92b81184b0 con 0x7f92b8108d50 2024-04-26T02:04:13.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.039+0000 7f92bcc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f92b81187b0 con 0x7f92b81083b0 2024-04-26T02:04:13.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.040+0000 7f92b7400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b81083b0 0x7f92b81a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:13.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.040+0000 7f92b7400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b81083b0 0x7f92b81a47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:46800/0 (socket says 172.21.15.1:46800) 2024-04-26T02:04:13.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.040+0000 7f92b7400640 1 -- 172.21.15.1:0/1490754865 learned_addr learned my addr 172.21.15.1:0/1490754865 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:13.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.040+0000 7f92b7e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92b810e470 0x7f92b819e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:13.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.040+0000 7f92b6a00640 1 --2- 172.21.15.1:0/1490754865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b8108d50 0x7f92b81a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:13.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.040+0000 7f92b7400640 1 -- 172.21.15.1:0/1490754865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b8108d50 msgr2=0x7f92b81a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:13.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.040+0000 7f92b7400640 1 --2- 172.21.15.1:0/1490754865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b8108d50 0x7f92b81a4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:13.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.040+0000 7f92b7400640 1 -- 172.21.15.1:0/1490754865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92b810e470 msgr2=0x7f92b819e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:13.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.040+0000 7f92b7400640 1 --2- 172.21.15.1:0/1490754865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92b810e470 0x7f92b819e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:13.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.040+0000 7f92b7400640 1 -- 172.21.15.1:0/1490754865 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b819f150 con 0x7f92b81083b0 2024-04-26T02:04:13.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.040+0000 7f92b6a00640 1 --2- 172.21.15.1:0/1490754865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b8108d50 0x7f92b81a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:04:13.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.041+0000 7f92b7400640 1 --2- 172.21.15.1:0/1490754865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b81083b0 0x7f92b81a47d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f92a8005940 tx=0x7f92a8030f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:13.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.041+0000 7f92afe00640 1 -- 172.21.15.1:0/1490754865 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8040070 con 0x7f92b81083b0 2024-04-26T02:04:13.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.041+0000 7f92afe00640 1 -- 172.21.15.1:0/1490754865 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92a80055f0 con 0x7f92b81083b0 2024-04-26T02:04:13.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.041+0000 7f92bcc00640 1 -- 172.21.15.1:0/1490754865 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b819f3e0 con 0x7f92b81083b0 2024-04-26T02:04:13.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.041+0000 7f92afe00640 1 -- 172.21.15.1:0/1490754865 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a80046c0 con 0x7f92b81083b0 2024-04-26T02:04:13.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.041+0000 7f92bcc00640 1 -- 172.21.15.1:0/1490754865 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b81ab690 con 0x7f92b81083b0 2024-04-26T02:04:13.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.042+0000 7f92bcc00640 1 -- 172.21.15.1:0/1490754865 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b80687f0 con 0x7f92b81083b0 2024-04-26T02:04:13.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.044+0000 7f92afe00640 1 -- 172.21.15.1:0/1490754865 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f92a8043070 con 0x7f92b81083b0 2024-04-26T02:04:13.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.046+0000 7f92afe00640 1 --2- 172.21.15.1:0/1490754865 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f929c0a3d00 0x7f929c0a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:13.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.046+0000 7f92afe00640 1 -- 172.21.15.1:0/1490754865 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8615+0+0 (secure 0 0 0) 0x7f92a80f13a0 con 0x7f92b81083b0 2024-04-26T02:04:13.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.046+0000 7f92b6a00640 1 --2- 172.21.15.1:0/1490754865 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f929c0a3d00 0x7f929c0a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:13.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.047+0000 7f92b6a00640 1 --2- 172.21.15.1:0/1490754865 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f929c0a3d00 0x7f929c0a61c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f92b8078db0 tx=0x7f92a00095c0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:13.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.051+0000 7f92afe00640 1 -- 172.21.15.1:0/1490754865 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f92a80bd330 con 0x7f92b81083b0 2024-04-26T02:04:13.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.282+0000 7f92bcc00640 1 -- 172.21.15.1:0/1490754865 --> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f92b81108d0 con 0x7f929c0a3d00 2024-04-26T02:04:13.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.297+0000 7f92afe00640 1 -- 172.21.15.1:0/1490754865 <== mgr.25087 v2:172.21.15.1:6800/275906565 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f92a000eda0 con 0x7f929c0a3d00 2024-04-26T02:04:13.296 INFO:teuthology.orchestra.run.smithi001.stdout:/ 1m 6m3h 2024-04-26T02:04:13.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.299+0000 7f92bcc00640 1 -- 172.21.15.1:0/1490754865 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f929c0a3d00 msgr2=0x7f929c0a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:13.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.299+0000 7f92bcc00640 1 --2- 172.21.15.1:0/1490754865 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f929c0a3d00 0x7f929c0a61c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f92b8078db0 tx=0x7f92a00095c0 comp rx=0 tx=0).stop 2024-04-26T02:04:13.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.299+0000 7f92bcc00640 1 -- 172.21.15.1:0/1490754865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b81083b0 msgr2=0x7f92b81a47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.299+0000 7f92bcc00640 1 --2- 172.21.15.1:0/1490754865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b81083b0 0x7f92b81a47d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f92a8005940 tx=0x7f92a8030f20 comp rx=0 tx=0).stop 2024-04-26T02:04:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.299+0000 7f92bcc00640 1 -- 172.21.15.1:0/1490754865 shutdown_connections 2024-04-26T02:04:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.299+0000 7f92bcc00640 1 --2- 172.21.15.1:0/1490754865 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f929c0a3d00 0x7f929c0a61c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.299+0000 7f92bcc00640 1 --2- 172.21.15.1:0/1490754865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92b810e470 0x7f92b819e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.299+0000 7f92bcc00640 1 --2- 172.21.15.1:0/1490754865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b8108d50 0x7f92b81a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.299+0000 7f92bcc00640 1 --2- 172.21.15.1:0/1490754865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b81083b0 0x7f92b81a47d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.299+0000 7f92bcc00640 1 -- 172.21.15.1:0/1490754865 >> 172.21.15.1:0/1490754865 conn(0x7f92b8077b00 msgr2=0x7f92b8079470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.300+0000 7f92bcc00640 1 -- 172.21.15.1:0/1490754865 shutdown_connections 2024-04-26T02:04:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.300+0000 7f92bcc00640 1 -- 172.21.15.1:0/1490754865 wait complete. 2024-04-26T02:04:13.317 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-26T02:04:13.328 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T02:04:13.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.542+0000 7f11c9000640 1 -- 172.21.15.1:0/4122105808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11c41016f0 msgr2=0x7f11c4105bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:13.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.542+0000 7f11c9000640 1 --2- 172.21.15.1:0/4122105808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11c41016f0 0x7f11c4105bb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009900 tx=0x7f11ac02a470 comp rx=0 tx=0).stop 2024-04-26T02:04:13.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.542+0000 7f11c9000640 1 -- 172.21.15.1:0/4122105808 shutdown_connections 2024-04-26T02:04:13.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.542+0000 7f11c9000640 1 --2- 172.21.15.1:0/4122105808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11c41061d0 0x7f11c4106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:13.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.542+0000 7f11c9000640 1 --2- 172.21.15.1:0/4122105808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11c41016f0 0x7f11c4105bb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:13.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.542+0000 7f11c9000640 1 --2- 172.21.15.1:0/4122105808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11c4100dd0 0x7f11c41011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:13.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.542+0000 7f11c9000640 1 -- 172.21.15.1:0/4122105808 >> 172.21.15.1:0/4122105808 conn(0x7f11c40fc7e0 msgr2=0x7f11c40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:13.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.542+0000 7f11c9000640 1 -- 172.21.15.1:0/4122105808 shutdown_connections 2024-04-26T02:04:13.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.543+0000 7f11c9000640 1 -- 172.21.15.1:0/4122105808 wait complete. 2024-04-26T02:04:13.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.543+0000 7f11c9000640 1 Processor -- start 2024-04-26T02:04:13.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.544+0000 7f11c9000640 1 -- start start 2024-04-26T02:04:13.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.544+0000 7f11c9000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11c4100dd0 0x7f11c41a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:13.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.544+0000 7f11c9000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11c41016f0 0x7f11c41a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:13.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.544+0000 7f11c9000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11c41061d0 0x7f11c419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:13.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.544+0000 7f11c9000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f11c4118680 con 0x7f11c41016f0 2024-04-26T02:04:13.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.544+0000 7f11c9000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f11c4118500 con 0x7f11c41061d0 2024-04-26T02:04:13.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.544+0000 7f11c9000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f11c4118800 con 0x7f11c4100dd0 2024-04-26T02:04:13.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.544+0000 7f11c2a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11c41016f0 0x7f11c41a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:13.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.544+0000 7f11c2a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11c41016f0 0x7f11c41a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43742/0 (socket says 172.21.15.1:43742) 2024-04-26T02:04:13.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.544+0000 7f11c2a00640 1 -- 172.21.15.1:0/4003337373 learned_addr learned my addr 172.21.15.1:0/4003337373 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:13.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.544+0000 7f11c3400640 1 --2- 172.21.15.1:0/4003337373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11c4100dd0 0x7f11c41a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:13.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.545+0000 7f11c3e00640 1 --2- 172.21.15.1:0/4003337373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11c41061d0 0x7f11c419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:13.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.545+0000 7f11c2a00640 1 -- 172.21.15.1:0/4003337373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11c4100dd0 msgr2=0x7f11c41a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:13.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.545+0000 7f11c2a00640 1 --2- 172.21.15.1:0/4003337373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11c4100dd0 0x7f11c41a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:13.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.545+0000 7f11c2a00640 1 -- 172.21.15.1:0/4003337373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11c41061d0 msgr2=0x7f11c419eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:13.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.545+0000 7f11c2a00640 1 --2- 172.21.15.1:0/4003337373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11c41061d0 0x7f11c419eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:13.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.545+0000 7f11c2a00640 1 -- 172.21.15.1:0/4003337373 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c419f260 con 0x7f11c41016f0 2024-04-26T02:04:13.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.545+0000 7f11c2a00640 1 --2- 172.21.15.1:0/4003337373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11c41016f0 0x7f11c41a4e70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f11ac005940 tx=0x7f11ac0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:13.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.545+0000 7f11bbe00640 1 -- 172.21.15.1:0/4003337373 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac03f030 con 0x7f11c41016f0 2024-04-26T02:04:13.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.546+0000 7f11bbe00640 1 -- 172.21.15.1:0/4003337373 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11ac0044c0 con 0x7f11c41016f0 2024-04-26T02:04:13.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.546+0000 7f11c9000640 1 -- 172.21.15.1:0/4003337373 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c419f4f0 con 0x7f11c41016f0 2024-04-26T02:04:13.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.546+0000 7f11bbe00640 1 -- 172.21.15.1:0/4003337373 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac035070 con 0x7f11c41016f0 2024-04-26T02:04:13.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.546+0000 7f11c9000640 1 -- 172.21.15.1:0/4003337373 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c41ab7f0 con 0x7f11c41016f0 2024-04-26T02:04:13.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.547+0000 7f11c9000640 1 -- 172.21.15.1:0/4003337373 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11c404fd50 con 0x7f11c41016f0 2024-04-26T02:04:13.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.549+0000 7f11bbe00640 1 -- 172.21.15.1:0/4003337373 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f11ac042030 con 0x7f11c41016f0 2024-04-26T02:04:13.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.550+0000 7f11bbe00640 1 --2- 172.21.15.1:0/4003337373 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f11a80a3d10 0x7f11a80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:13.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.550+0000 7f11bbe00640 1 -- 172.21.15.1:0/4003337373 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8615+0+0 (secure 0 0 0) 0x7f11ac0f0c10 con 0x7f11c41016f0 2024-04-26T02:04:13.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.551+0000 7f11c3400640 1 --2- 172.21.15.1:0/4003337373 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f11a80a3d10 0x7f11a80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:13.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.551+0000 7f11c3400640 1 --2- 172.21.15.1:0/4003337373 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f11a80a3d10 0x7f11a80a61d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f11b4005fd0 tx=0x7f11b4009500 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:13.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.555+0000 7f11bbe00640 1 -- 172.21.15.1:0/4003337373 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f11ac0bcba0 con 0x7f11c41016f0 2024-04-26T02:04:13.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.762+0000 7f11c9000640 1 -- 172.21.15.1:0/4003337373 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11c41ab9c0 con 0x7f11c41016f0 2024-04-26T02:04:13.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.766+0000 7f11bbe00640 1 -- 172.21.15.1:0/4003337373 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+23177 (secure 0 0 0) 0x7f11ac0c0ea0 con 0x7f11c41016f0 2024-04-26T02:04:13.766 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T02:04:13.766 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":108,"fsid":"77cfcc80-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:38:03.531182+0000","modified":"2024-04-26T02:04:10.201502+0000","last_up_change":"2024-04-26T01:46:16.151275+0000","last_in_change":"2024-04-26T01:45:56.514228+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:42:04.805488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:47:21.957421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:47:25.860260+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb10ae12-09ee-4d6b-bc78-8c6016613b81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6803","nonce":3176633965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6805","nonce":3176633965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6809","nonce":3176633965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6807","nonce":3176633965}]},"public_addr":"172.21.15.1:6803/3176633965","cluster_addr":"172.21.15.1:6805/3176633965","heartbeat_back_addr":"172.21.15.1:6809/3176633965","heartbeat_front_addr":"172.21.15.1:6807/3176633965","state":["exists","up"]},{"osd":1,"uuid":"acd12888-6ff6-40bc-99e7-e5c395e85b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6801","nonce":3755022621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6803","nonce":3755022621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6807","nonce":3755022621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6805","nonce":3755022621}]},"public_addr":"172.21.15.97:6801/3755022621","cluster_addr":"172.21.15.97:6803/3755022621","heartbeat_back_addr":"172.21.15.97:6807/3755022621","heartbeat_front_addr":"172.21.15.97:6805/3755022621","state":["exists","up"]},{"osd":2,"uuid":"79971508-e256-4145-8111-a75666e886fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6801","nonce":734861771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6803","nonce":734861771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6807","nonce":734861771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6805","nonce":734861771}]},"public_addr":"172.21.15.173:6801/734861771","cluster_addr":"172.21.15.173:6803/734861771","heartbeat_back_addr":"172.21.15.173:6807/734861771","heartbeat_front_addr":"172.21.15.173:6805/734861771","state":["exists","up"]},{"osd":3,"uuid":"a8ccc4a9-0f31-446a-8bcb-4691532397c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6811","nonce":3349961961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6813","nonce":3349961961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6817","nonce":3349961961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6815","nonce":3349961961}]},"public_addr":"172.21.15.1:6811/3349961961","cluster_addr":"172.21.15.1:6813/3349961961","heartbeat_back_addr":"172.21.15.1:6817/3349961961","heartbeat_front_addr":"172.21.15.1:6815/3349961961","state":["exists","up"]},{"osd":4,"uuid":"5b724648-4df7-40fc-89d5-55f170cdd806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6809","nonce":2494639397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6811","nonce":2494639397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6815","nonce":2494639397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6813","nonce":2494639397}]},"public_addr":"172.21.15.97:6809/2494639397","cluster_addr":"172.21.15.97:6811/2494639397","heartbeat_back_addr":"172.21.15.97:6815/2494639397","heartbeat_front_addr":"172.21.15.97:6813/2494639397","state":["exists","up"]},{"osd":5,"uuid":"a5a24a9c-3007-4b87-8562-40c98c41ef8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6809","nonce":1026507957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6811","nonce":1026507957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6815","nonce":1026507957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6813","nonce":1026507957}]},"public_addr":"172.21.15.173:6809/1026507957","cluster_addr":"172.21.15.173:6811/1026507957","heartbeat_back_addr":"172.21.15.173:6815/1026507957","heartbeat_front_addr":"172.21.15.173:6813/1026507957","state":["exists","up"]},{"osd":6,"uuid":"97cd2e88-a56c-4b1a-8104-65b23a77986a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6819","nonce":444030587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6821","nonce":444030587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6825","nonce":444030587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6823","nonce":444030587}]},"public_addr":"172.21.15.1:6819/444030587","cluster_addr":"172.21.15.1:6821/444030587","heartbeat_back_addr":"172.21.15.1:6825/444030587","heartbeat_front_addr":"172.21.15.1:6823/444030587","state":["exists","up"]},{"osd":7,"uuid":"afc0c2dd-e112-4358-b3aa-944366d5d383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6817","nonce":2600022531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6819","nonce":2600022531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6823","nonce":2600022531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6821","nonce":2600022531}]},"public_addr":"172.21.15.97:6817/2600022531","cluster_addr":"172.21.15.97:6819/2600022531","heartbeat_back_addr":"172.21.15.97:6823/2600022531","heartbeat_front_addr":"172.21.15.97:6821/2600022531","state":["exists","up"]},{"osd":8,"uuid":"41fa612b-793e-44da-af8b-44d17a9de303","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6817","nonce":1334205527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6819","nonce":1334205527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6823","nonce":1334205527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6821","nonce":1334205527}]},"public_addr":"172.21.15.173:6817/1334205527","cluster_addr":"172.21.15.173:6819/1334205527","heartbeat_back_addr":"172.21.15.173:6823/1334205527","heartbeat_front_addr":"172.21.15.173:6821/1334205527","state":["exists","up"]},{"osd":9,"uuid":"bbbb163b-f8c4-4be5-8f80-82445e8d407f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6827","nonce":2973486839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6829","nonce":2973486839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6833","nonce":2973486839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6831","nonce":2973486839}]},"public_addr":"172.21.15.1:6827/2973486839","cluster_addr":"172.21.15.1:6829/2973486839","heartbeat_back_addr":"172.21.15.1:6833/2973486839","heartbeat_front_addr":"172.21.15.1:6831/2973486839","state":["exists","up"]},{"osd":10,"uuid":"1b447b34-0600-4727-826d-b3d68adfb18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6825","nonce":1693425268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6827","nonce":1693425268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6831","nonce":1693425268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6829","nonce":1693425268}]},"public_addr":"172.21.15.97:6825/1693425268","cluster_addr":"172.21.15.97:6827/1693425268","heartbeat_back_addr":"172.21.15.97:6831/1693425268","heartbeat_front_addr":"172.21.15.97:6829/1693425268","state":["exists","up"]},{"osd":11,"uuid":"f439aae1-f50a-4176-889d-36fa5131541a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6825","nonce":1141524960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6827","nonce":1141524960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6831","nonce":1141524960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6829","nonce":1141524960}]},"public_addr":"172.21.15.173:6825/1141524960","cluster_addr":"172.21.15.173:6827/1141524960","heartbeat_back_addr":"172.21.15.173:6831/1141524960","heartbeat_front_addr":"172.21.15.173:6829/1141524960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:12.955643+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:38.846652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:02.556062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:34.900933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:01.170878+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:24.842573+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:58.304440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:24.868689+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:48.772003+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:22.611681+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:49.470779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:14.163984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1607430337":"2024-04-27T01:51:38.660496+0000","172.21.15.1:0/25191068":"2024-04-27T01:51:38.660496+0000","172.21.15.1:6801/2140737121":"2024-04-27T01:51:38.660496+0000","172.21.15.1:0/304803917":"2024-04-27T01:51:38.660496+0000","172.21.15.1:6801/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6800/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/1000222947":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6800/2140737121":"2024-04-27T01:51:38.660496+0000","172.21.15.1:0/457958597":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6801/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:6800/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/941157221":"2024-04-27T01:51:38.660496+0000","172.21.15.1:0/2715798962":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/1923058728":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/2489038777":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/2869502448":"2024-04-27T01:38:45.974031+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T02:04:13.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.769+0000 7f11c9000640 1 -- 172.21.15.1:0/4003337373 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f11a80a3d10 msgr2=0x7f11a80a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:13.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.769+0000 7f11c9000640 1 --2- 172.21.15.1:0/4003337373 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f11a80a3d10 0x7f11a80a61d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f11b4005fd0 tx=0x7f11b4009500 comp rx=0 tx=0).stop 2024-04-26T02:04:13.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.769+0000 7f11c9000640 1 -- 172.21.15.1:0/4003337373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11c41016f0 msgr2=0x7f11c41a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:13.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.769+0000 7f11c9000640 1 --2- 172.21.15.1:0/4003337373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11c41016f0 0x7f11c41a4e70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f11ac005940 tx=0x7f11ac0040c0 comp rx=0 tx=0).stop 2024-04-26T02:04:13.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.770+0000 7f11c9000640 1 -- 172.21.15.1:0/4003337373 shutdown_connections 2024-04-26T02:04:13.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.770+0000 7f11c9000640 1 --2- 172.21.15.1:0/4003337373 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f11a80a3d10 0x7f11a80a61d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:13.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.770+0000 7f11c9000640 1 --2- 172.21.15.1:0/4003337373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11c41061d0 0x7f11c419eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:13.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.770+0000 7f11c9000640 1 --2- 172.21.15.1:0/4003337373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11c41016f0 0x7f11c41a4e70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:13.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.770+0000 7f11c9000640 1 --2- 172.21.15.1:0/4003337373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11c4100dd0 0x7f11c41a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:13.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.770+0000 7f11c9000640 1 -- 172.21.15.1:0/4003337373 >> 172.21.15.1:0/4003337373 conn(0x7f11c40fc7e0 msgr2=0x7f11c4103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:13.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.770+0000 7f11c9000640 1 -- 172.21.15.1:0/4003337373 shutdown_connections 2024-04-26T02:04:13.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:13.770+0000 7f11c9000640 1 -- 172.21.15.1:0/4003337373 wait complete. 2024-04-26T02:04:13.790 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T01:42:04.805488+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T01:47:21.957421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T01:47:25.860260+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T02:04:13.791 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T02:04:14.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.011+0000 7f6419e00640 1 -- 172.21.15.1:0/499854926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f641410e3d0 msgr2=0x7f641410e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:14.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.012+0000 7f6419e00640 1 --2- 172.21.15.1:0/499854926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f641410e3d0 0x7f641410e7b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6408009900 tx=0x7f640802a3f0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.012+0000 7f6419e00640 1 -- 172.21.15.1:0/499854926 shutdown_connections 2024-04-26T02:04:14.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.012+0000 7f6419e00640 1 --2- 172.21.15.1:0/499854926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6414108e30 0x7f6414115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.012+0000 7f6419e00640 1 --2- 172.21.15.1:0/499854926 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6414108490 0x7f64141088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.012+0000 7f6419e00640 1 --2- 172.21.15.1:0/499854926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f641410e3d0 0x7f641410e7b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.012+0000 7f6419e00640 1 -- 172.21.15.1:0/499854926 >> 172.21.15.1:0/499854926 conn(0x7f6414077d60 msgr2=0x7f6414106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:14.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.012+0000 7f6419e00640 1 -- 172.21.15.1:0/499854926 shutdown_connections 2024-04-26T02:04:14.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.012+0000 7f6419e00640 1 -- 172.21.15.1:0/499854926 wait complete. 2024-04-26T02:04:14.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.013+0000 7f6419e00640 1 Processor -- start 2024-04-26T02:04:14.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.013+0000 7f6419e00640 1 -- start start 2024-04-26T02:04:14.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.013+0000 7f6419e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6414108490 0x7f64141a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:14.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.014+0000 7f6419e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6414108e30 0x7f64141a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:14.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.014+0000 7f6419e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f641410e3d0 0x7f641419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:14.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.014+0000 7f6419e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f64141185b0 con 0x7f6414108490 2024-04-26T02:04:14.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.014+0000 7f6419e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6414118430 con 0x7f6414108e30 2024-04-26T02:04:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.014+0000 7f6419e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6414118730 con 0x7f641410e3d0 2024-04-26T02:04:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.014+0000 7f6418a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6414108490 0x7f64141a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.014+0000 7f6418a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6414108490 0x7f64141a47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43760/0 (socket says 172.21.15.1:43760) 2024-04-26T02:04:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.014+0000 7f6418a00640 1 -- 172.21.15.1:0/2360827451 learned_addr learned my addr 172.21.15.1:0/2360827451 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.014+0000 7f6413e00640 1 --2- 172.21.15.1:0/2360827451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6414108e30 0x7f64141a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.014+0000 7f6419400640 1 --2- 172.21.15.1:0/2360827451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f641410e3d0 0x7f641419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.014+0000 7f6418a00640 1 -- 172.21.15.1:0/2360827451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f641410e3d0 msgr2=0x7f641419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.014+0000 7f6418a00640 1 --2- 172.21.15.1:0/2360827451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f641410e3d0 0x7f641419e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.014+0000 7f6418a00640 1 -- 172.21.15.1:0/2360827451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6414108e30 msgr2=0x7f64141a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.014+0000 7f6418a00640 1 --2- 172.21.15.1:0/2360827451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6414108e30 0x7f64141a4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.014+0000 7f6418a00640 1 -- 172.21.15.1:0/2360827451 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641419f0d0 con 0x7f6414108490 2024-04-26T02:04:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.014+0000 7f6413e00640 1 --2- 172.21.15.1:0/2360827451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6414108e30 0x7f64141a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:04:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.015+0000 7f6418a00640 1 --2- 172.21.15.1:0/2360827451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6414108490 0x7f64141a47a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6408006010 tx=0x7f64080040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:14.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.015+0000 7f6411600640 1 -- 172.21.15.1:0/2360827451 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640803f030 con 0x7f6414108490 2024-04-26T02:04:14.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.015+0000 7f6411600640 1 -- 172.21.15.1:0/2360827451 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64080044c0 con 0x7f6414108490 2024-04-26T02:04:14.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.015+0000 7f6419e00640 1 -- 172.21.15.1:0/2360827451 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f641419f360 con 0x7f6414108490 2024-04-26T02:04:14.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.015+0000 7f6411600640 1 -- 172.21.15.1:0/2360827451 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408035070 con 0x7f6414108490 2024-04-26T02:04:14.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.015+0000 7f6419e00640 1 -- 172.21.15.1:0/2360827451 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64141ab660 con 0x7f6414108490 2024-04-26T02:04:14.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.016+0000 7f6419e00640 1 -- 172.21.15.1:0/2360827451 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f641404fce0 con 0x7f6414108490 2024-04-26T02:04:14.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.018+0000 7f6411600640 1 -- 172.21.15.1:0/2360827451 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f6408042030 con 0x7f6414108490 2024-04-26T02:04:14.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.021+0000 7f6411600640 1 --2- 172.21.15.1:0/2360827451 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f63fc0a3d20 0x7f63fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:14.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.021+0000 7f6411600640 1 -- 172.21.15.1:0/2360827451 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8615+0+0 (secure 0 0 0) 0x7f64080f0b20 con 0x7f6414108490 2024-04-26T02:04:14.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.021+0000 7f6413e00640 1 --2- 172.21.15.1:0/2360827451 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f63fc0a3d20 0x7f63fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:14.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.022+0000 7f6413e00640 1 --2- 172.21.15.1:0/2360827451 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f63fc0a3d20 0x7f63fc0a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f641410db80 tx=0x7f6400005f90 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:14.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.026+0000 7f6411600640 1 -- 172.21.15.1:0/2360827451 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f64080bcab0 con 0x7f6414108490 2024-04-26T02:04:14.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.242+0000 7f6419e00640 1 -- 172.21.15.1:0/2360827451 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6414110850 con 0x7f6414108490 2024-04-26T02:04:14.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.243+0000 7f6411600640 1 -- 172.21.15.1:0/2360827451 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) ==== 93+0+10 (secure 0 0 0) 0x7f64080c0db0 con 0x7f6414108490 2024-04-26T02:04:14.242 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-04-26T02:04:14.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.246+0000 7f6419e00640 1 -- 172.21.15.1:0/2360827451 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f63fc0a3d20 msgr2=0x7f63fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:14.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.246+0000 7f6419e00640 1 --2- 172.21.15.1:0/2360827451 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f63fc0a3d20 0x7f63fc0a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f641410db80 tx=0x7f6400005f90 comp rx=0 tx=0).stop 2024-04-26T02:04:14.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.246+0000 7f6419e00640 1 -- 172.21.15.1:0/2360827451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6414108490 msgr2=0x7f64141a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:14.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.246+0000 7f6419e00640 1 --2- 172.21.15.1:0/2360827451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6414108490 0x7f64141a47a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6408006010 tx=0x7f64080040c0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.246+0000 7f6419e00640 1 -- 172.21.15.1:0/2360827451 shutdown_connections 2024-04-26T02:04:14.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.246+0000 7f6419e00640 1 --2- 172.21.15.1:0/2360827451 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f63fc0a3d20 0x7f63fc0a61e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.246+0000 7f6419e00640 1 --2- 172.21.15.1:0/2360827451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f641410e3d0 0x7f641419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.246+0000 7f6419e00640 1 --2- 172.21.15.1:0/2360827451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6414108e30 0x7f64141a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.246+0000 7f6419e00640 1 --2- 172.21.15.1:0/2360827451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6414108490 0x7f64141a47a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.246+0000 7f6419e00640 1 -- 172.21.15.1:0/2360827451 >> 172.21.15.1:0/2360827451 conn(0x7f6414077d60 msgr2=0x7f6414078f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:14.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.246+0000 7f6419e00640 1 -- 172.21.15.1:0/2360827451 shutdown_connections 2024-04-26T02:04:14.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.247+0000 7f6419e00640 1 -- 172.21.15.1:0/2360827451 wait complete. 2024-04-26T02:04:14.266 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T02:04:14.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.475+0000 7f72cfe00640 1 -- 172.21.15.1:0/1525346477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72d0074c40 msgr2=0x7f72d010e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:14.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.475+0000 7f72cfe00640 1 --2- 172.21.15.1:0/1525346477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72d0074c40 0x7f72d010e0b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f72bc009900 tx=0x7f72bc02a480 comp rx=0 tx=0).stop 2024-04-26T02:04:14.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.475+0000 7f72cfe00640 1 -- 172.21.15.1:0/1525346477 shutdown_connections 2024-04-26T02:04:14.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.475+0000 7f72cfe00640 1 --2- 172.21.15.1:0/1525346477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72d010e5f0 0x7f72d0110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.475+0000 7f72cfe00640 1 --2- 172.21.15.1:0/1525346477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72d0074c40 0x7f72d010e0b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.475+0000 7f72cfe00640 1 --2- 172.21.15.1:0/1525346477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d0074320 0x7f72d0074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.475+0000 7f72cfe00640 1 -- 172.21.15.1:0/1525346477 >> 172.21.15.1:0/1525346477 conn(0x7f72d00fd8a0 msgr2=0x7f72d00ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:14.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.475+0000 7f72cfe00640 1 -- 172.21.15.1:0/1525346477 shutdown_connections 2024-04-26T02:04:14.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.475+0000 7f72cfe00640 1 -- 172.21.15.1:0/1525346477 wait complete. 2024-04-26T02:04:14.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.476+0000 7f72cfe00640 1 Processor -- start 2024-04-26T02:04:14.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.476+0000 7f72cfe00640 1 -- start start 2024-04-26T02:04:14.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.477+0000 7f72cfe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72d0074320 0x7f72d01a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:14.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.477+0000 7f72cfe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72d0074c40 0x7f72d01a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:14.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.477+0000 7f72cfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d010e5f0 0x7f72d019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:14.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.477+0000 7f72cfe00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f72d0114200 con 0x7f72d0074c40 2024-04-26T02:04:14.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.477+0000 7f72cfe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f72d0114080 con 0x7f72d010e5f0 2024-04-26T02:04:14.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.477+0000 7f72cfe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f72d0114380 con 0x7f72d0074320 2024-04-26T02:04:14.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.477+0000 7f72ce000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72d0074c40 0x7f72d01a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:14.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.477+0000 7f72ce000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72d0074c40 0x7f72d01a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43790/0 (socket says 172.21.15.1:43790) 2024-04-26T02:04:14.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.477+0000 7f72ce000640 1 -- 172.21.15.1:0/3653799968 learned_addr learned my addr 172.21.15.1:0/3653799968 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:14.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.477+0000 7f72cea00640 1 --2- 172.21.15.1:0/3653799968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72d0074320 0x7f72d01a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:14.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.477+0000 7f72cf400640 1 --2- 172.21.15.1:0/3653799968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d010e5f0 0x7f72d019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:14.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.478+0000 7f72ce000640 1 -- 172.21.15.1:0/3653799968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72d0074320 msgr2=0x7f72d01a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:14.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.478+0000 7f72ce000640 1 --2- 172.21.15.1:0/3653799968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72d0074320 0x7f72d01a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.478+0000 7f72ce000640 1 -- 172.21.15.1:0/3653799968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d010e5f0 msgr2=0x7f72d019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:14.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.478+0000 7f72ce000640 1 --2- 172.21.15.1:0/3653799968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d010e5f0 0x7f72d019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.478+0000 7f72ce000640 1 -- 172.21.15.1:0/3653799968 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d019ade0 con 0x7f72d0074c40 2024-04-26T02:04:14.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.478+0000 7f72ce000640 1 --2- 172.21.15.1:0/3653799968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72d0074c40 0x7f72d01a09a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f72bc005940 tx=0x7f72bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:14.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.478+0000 7f72d4a00640 1 -- 172.21.15.1:0/3653799968 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc03f030 con 0x7f72d0074c40 2024-04-26T02:04:14.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.478+0000 7f72d4a00640 1 -- 172.21.15.1:0/3653799968 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72bc003aa0 con 0x7f72d0074c40 2024-04-26T02:04:14.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.478+0000 7f72cfe00640 1 -- 172.21.15.1:0/3653799968 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d019b070 con 0x7f72d0074c40 2024-04-26T02:04:14.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.479+0000 7f72d4a00640 1 -- 172.21.15.1:0/3653799968 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc035070 con 0x7f72d0074c40 2024-04-26T02:04:14.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.479+0000 7f72cfe00640 1 -- 172.21.15.1:0/3653799968 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d01a7320 con 0x7f72d0074c40 2024-04-26T02:04:14.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.480+0000 7f72cfe00640 1 -- 172.21.15.1:0/3653799968 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72d004fce0 con 0x7f72d0074c40 2024-04-26T02:04:14.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.482+0000 7f72d4a00640 1 -- 172.21.15.1:0/3653799968 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f72bc042030 con 0x7f72d0074c40 2024-04-26T02:04:14.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.483+0000 7f72d4a00640 1 --2- 172.21.15.1:0/3653799968 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f72b40a3da0 0x7f72b40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:14.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.484+0000 7f72d4a00640 1 -- 172.21.15.1:0/3653799968 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8615+0+0 (secure 0 0 0) 0x7f72bc0f0c60 con 0x7f72d0074c40 2024-04-26T02:04:14.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.484+0000 7f72cea00640 1 --2- 172.21.15.1:0/3653799968 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f72b40a3da0 0x7f72b40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:14.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.484+0000 7f72cea00640 1 --2- 172.21.15.1:0/3653799968 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f72b40a3da0 0x7f72b40a6260 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f72c0005fd0 tx=0x7f72c0009500 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:14.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:14 smithi097 ceph-mon[32853]: from='client.16539 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T02:04:14.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:14 smithi097 ceph-mon[32853]: pgmap v395: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-04-26T02:04:14.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:14 smithi097 ceph-mon[32853]: from='client.26743 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T02:04:14.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:14 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/4003337373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T02:04:14.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.488+0000 7f72d4a00640 1 -- 172.21.15.1:0/3653799968 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f72bc0bcbf0 con 0x7f72d0074c40 2024-04-26T02:04:14.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:14 smithi173 ceph-mon[32553]: from='client.16539 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T02:04:14.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:14 smithi173 ceph-mon[32553]: pgmap v395: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-04-26T02:04:14.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:14 smithi173 ceph-mon[32553]: from='client.26743 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T02:04:14.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:14 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/4003337373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T02:04:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:14 smithi001 ceph-mon[27822]: from='client.16539 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T02:04:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:14 smithi001 ceph-mon[27822]: pgmap v395: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-04-26T02:04:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:14 smithi001 ceph-mon[27822]: from='client.26743 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T02:04:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:14 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/4003337373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T02:04:14.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.698+0000 7f72cfe00640 1 -- 172.21.15.1:0/3653799968 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f72d0109770 con 0x7f72d0074c40 2024-04-26T02:04:14.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.699+0000 7f72d4a00640 1 -- 172.21.15.1:0/3653799968 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) ==== 104+0+11 (secure 0 0 0) 0x7f72bc0c0ef0 con 0x7f72d0074c40 2024-04-26T02:04:14.698 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-04-26T02:04:14.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.703+0000 7f72cfe00640 1 -- 172.21.15.1:0/3653799968 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f72b40a3da0 msgr2=0x7f72b40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:14.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.703+0000 7f72cfe00640 1 --2- 172.21.15.1:0/3653799968 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f72b40a3da0 0x7f72b40a6260 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f72c0005fd0 tx=0x7f72c0009500 comp rx=0 tx=0).stop 2024-04-26T02:04:14.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.703+0000 7f72cfe00640 1 -- 172.21.15.1:0/3653799968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72d0074c40 msgr2=0x7f72d01a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:14.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.703+0000 7f72cfe00640 1 --2- 172.21.15.1:0/3653799968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72d0074c40 0x7f72d01a09a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f72bc005940 tx=0x7f72bc004270 comp rx=0 tx=0).stop 2024-04-26T02:04:14.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.704+0000 7f72cfe00640 1 -- 172.21.15.1:0/3653799968 shutdown_connections 2024-04-26T02:04:14.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.704+0000 7f72cfe00640 1 --2- 172.21.15.1:0/3653799968 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f72b40a3da0 0x7f72b40a6260 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.704+0000 7f72cfe00640 1 --2- 172.21.15.1:0/3653799968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d010e5f0 0x7f72d019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.704+0000 7f72cfe00640 1 --2- 172.21.15.1:0/3653799968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72d0074c40 0x7f72d01a09a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.704+0000 7f72cfe00640 1 --2- 172.21.15.1:0/3653799968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72d0074320 0x7f72d01a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.704+0000 7f72cfe00640 1 -- 172.21.15.1:0/3653799968 >> 172.21.15.1:0/3653799968 conn(0x7f72d00fd8a0 msgr2=0x7f72d00fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:14.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.704+0000 7f72cfe00640 1 -- 172.21.15.1:0/3653799968 shutdown_connections 2024-04-26T02:04:14.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.704+0000 7f72cfe00640 1 -- 172.21.15.1:0/3653799968 wait complete. 2024-04-26T02:04:14.724 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T02:04:14.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.952+0000 7f533be00640 1 -- 172.21.15.1:0/1824127820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f533c108490 msgr2=0x7f533c1088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:14.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.952+0000 7f533be00640 1 --2- 172.21.15.1:0/1824127820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f533c108490 0x7f533c1088f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5328009930 tx=0x7f532802a440 comp rx=0 tx=0).stop 2024-04-26T02:04:14.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.952+0000 7f533be00640 1 -- 172.21.15.1:0/1824127820 shutdown_connections 2024-04-26T02:04:14.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.952+0000 7f533be00640 1 --2- 172.21.15.1:0/1824127820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f533c108e30 0x7f533c115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.952+0000 7f533be00640 1 --2- 172.21.15.1:0/1824127820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f533c108490 0x7f533c1088f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.952+0000 7f533be00640 1 --2- 172.21.15.1:0/1824127820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f533c10e3d0 0x7f533c10e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.952+0000 7f533be00640 1 -- 172.21.15.1:0/1824127820 >> 172.21.15.1:0/1824127820 conn(0x7f533c077d60 msgr2=0x7f533c106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:14.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.952+0000 7f533be00640 1 -- 172.21.15.1:0/1824127820 shutdown_connections 2024-04-26T02:04:14.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.952+0000 7f533be00640 1 -- 172.21.15.1:0/1824127820 wait complete. 2024-04-26T02:04:14.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.953+0000 7f533be00640 1 Processor -- start 2024-04-26T02:04:14.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.953+0000 7f533be00640 1 -- start start 2024-04-26T02:04:14.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.954+0000 7f533be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f533c108490 0x7f533c1a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:14.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.954+0000 7f533be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f533c108e30 0x7f533c1a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.954+0000 7f533be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f533c10e3d0 0x7f533c19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.954+0000 7f533be00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f533c1185e0 con 0x7f533c10e3d0 2024-04-26T02:04:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.954+0000 7f533be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f533c118460 con 0x7f533c108490 2024-04-26T02:04:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.954+0000 7f533be00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f533c118760 con 0x7f533c108e30 2024-04-26T02:04:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.954+0000 7f533b400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f533c10e3d0 0x7f533c19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.954+0000 7f533a000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f533c108e30 0x7f533c1a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.954+0000 7f533b400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f533c10e3d0 0x7f533c19e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43816/0 (socket says 172.21.15.1:43816) 2024-04-26T02:04:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.954+0000 7f533b400640 1 -- 172.21.15.1:0/3387760829 learned_addr learned my addr 172.21.15.1:0/3387760829 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.954+0000 7f533aa00640 1 --2- 172.21.15.1:0/3387760829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f533c108490 0x7f533c1a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.955+0000 7f533a000640 1 -- 172.21.15.1:0/3387760829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f533c108490 msgr2=0x7f533c1a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.955+0000 7f533a000640 1 --2- 172.21.15.1:0/3387760829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f533c108490 0x7f533c1a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.955+0000 7f533a000640 1 -- 172.21.15.1:0/3387760829 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f533c10e3d0 msgr2=0x7f533c19e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.955+0000 7f533a000640 1 --2- 172.21.15.1:0/3387760829 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f533c10e3d0 0x7f533c19e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:14.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.955+0000 7f533a000640 1 -- 172.21.15.1:0/3387760829 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533c19f120 con 0x7f533c108e30 2024-04-26T02:04:14.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.955+0000 7f533a000640 1 --2- 172.21.15.1:0/3387760829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f533c108e30 0x7f533c1a4d30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f532802a960 tx=0x7f5328003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:14.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.955+0000 7f5333400640 1 -- 172.21.15.1:0/3387760829 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532803f070 con 0x7f533c108e30 2024-04-26T02:04:14.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.956+0000 7f533be00640 1 -- 172.21.15.1:0/3387760829 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f533c19f3b0 con 0x7f533c108e30 2024-04-26T02:04:14.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.956+0000 7f5333400640 1 -- 172.21.15.1:0/3387760829 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5328031b60 con 0x7f533c108e30 2024-04-26T02:04:14.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.956+0000 7f5333400640 1 -- 172.21.15.1:0/3387760829 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328035030 con 0x7f533c108e30 2024-04-26T02:04:14.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.956+0000 7f533be00640 1 -- 172.21.15.1:0/3387760829 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f533c1ab6b0 con 0x7f533c108e30 2024-04-26T02:04:14.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.957+0000 7f533be00640 1 -- 172.21.15.1:0/3387760829 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f533c04fce0 con 0x7f533c108e30 2024-04-26T02:04:14.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.959+0000 7f5333400640 1 -- 172.21.15.1:0/3387760829 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f5328042070 con 0x7f533c108e30 2024-04-26T02:04:14.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.960+0000 7f5333400640 1 --2- 172.21.15.1:0/3387760829 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f53200a3d20 0x7f53200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:14.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.961+0000 7f5333400640 1 -- 172.21.15.1:0/3387760829 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8615+0+0 (secure 0 0 0) 0x7f53280f1370 con 0x7f533c108e30 2024-04-26T02:04:14.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.961+0000 7f533aa00640 1 --2- 172.21.15.1:0/3387760829 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f53200a3d20 0x7f53200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:14.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.962+0000 7f533aa00640 1 --2- 172.21.15.1:0/3387760829 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f53200a3d20 0x7f53200a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f533c19fae0 tx=0x7f532c005f90 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:14.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:14.963+0000 7f5333400640 1 -- 172.21.15.1:0/3387760829 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f53280bd300 con 0x7f533c108e30 2024-04-26T02:04:15.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.173+0000 7f533be00640 1 -- 172.21.15.1:0/3387760829 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f533c1108b0 con 0x7f533c108e30 2024-04-26T02:04:15.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.174+0000 7f5333400640 1 -- 172.21.15.1:0/3387760829 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) ==== 100+0+11 (secure 0 0 0) 0x7f53280c1600 con 0x7f533c108e30 2024-04-26T02:04:15.173 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-04-26T02:04:15.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.176+0000 7f533be00640 1 -- 172.21.15.1:0/3387760829 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f53200a3d20 msgr2=0x7f53200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:15.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.176+0000 7f533be00640 1 --2- 172.21.15.1:0/3387760829 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f53200a3d20 0x7f53200a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f533c19fae0 tx=0x7f532c005f90 comp rx=0 tx=0).stop 2024-04-26T02:04:15.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.176+0000 7f533be00640 1 -- 172.21.15.1:0/3387760829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f533c108e30 msgr2=0x7f533c1a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:15.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.176+0000 7f533be00640 1 --2- 172.21.15.1:0/3387760829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f533c108e30 0x7f533c1a4d30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f532802a960 tx=0x7f5328003c10 comp rx=0 tx=0).stop 2024-04-26T02:04:15.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.176+0000 7f533be00640 1 -- 172.21.15.1:0/3387760829 shutdown_connections 2024-04-26T02:04:15.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.176+0000 7f533be00640 1 --2- 172.21.15.1:0/3387760829 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f53200a3d20 0x7f53200a61e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.176+0000 7f533be00640 1 --2- 172.21.15.1:0/3387760829 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f533c10e3d0 0x7f533c19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.176+0000 7f533be00640 1 --2- 172.21.15.1:0/3387760829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f533c108e30 0x7f533c1a4d30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.176+0000 7f533be00640 1 --2- 172.21.15.1:0/3387760829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f533c108490 0x7f533c1a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.176+0000 7f533be00640 1 -- 172.21.15.1:0/3387760829 >> 172.21.15.1:0/3387760829 conn(0x7f533c077d60 msgr2=0x7f533c078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:15.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.177+0000 7f533be00640 1 -- 172.21.15.1:0/3387760829 shutdown_connections 2024-04-26T02:04:15.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.177+0000 7f533be00640 1 -- 172.21.15.1:0/3387760829 wait complete. 2024-04-26T02:04:15.197 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-26T02:04:15.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.399+0000 7f8790e00640 1 -- 172.21.15.1:0/145541657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f878c104d60 msgr2=0x7f878c108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:15.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.399+0000 7f8790e00640 1 --2- 172.21.15.1:0/145541657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f878c104d60 0x7f878c108da0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8774009860 tx=0x7f877402a340 comp rx=0 tx=0).stop 2024-04-26T02:04:15.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.400+0000 7f8790e00640 1 -- 172.21.15.1:0/145541657 shutdown_connections 2024-04-26T02:04:15.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.400+0000 7f8790e00640 1 --2- 172.21.15.1:0/145541657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f878c109480 0x7f878c111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.400+0000 7f8790e00640 1 --2- 172.21.15.1:0/145541657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f878c104d60 0x7f878c108da0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.400+0000 7f8790e00640 1 --2- 172.21.15.1:0/145541657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f878c1043b0 0x7f878c104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.400+0000 7f8790e00640 1 -- 172.21.15.1:0/145541657 >> 172.21.15.1:0/145541657 conn(0x7f878c0ffc10 msgr2=0x7f878c102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:15.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.400+0000 7f8790e00640 1 -- 172.21.15.1:0/145541657 shutdown_connections 2024-04-26T02:04:15.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.400+0000 7f8790e00640 1 -- 172.21.15.1:0/145541657 wait complete. 2024-04-26T02:04:15.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.401+0000 7f8790e00640 1 Processor -- start 2024-04-26T02:04:15.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.401+0000 7f8790e00640 1 -- start start 2024-04-26T02:04:15.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.401+0000 7f8790e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f878c1043b0 0x7f878c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.401+0000 7f8790e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f878c109480 0x7f878c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.401+0000 7f8790e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f878c19a7c0 0x7f878c19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.401+0000 7f8790e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1143c0 con 0x7f878c19a7c0 2024-04-26T02:04:15.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.401+0000 7f8790e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f878c114240 con 0x7f878c109480 2024-04-26T02:04:15.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.401+0000 7f8790e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f878c114540 con 0x7f878c1043b0 2024-04-26T02:04:15.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.402+0000 7f878a800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f878c19a7c0 0x7f878c19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:15.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.402+0000 7f8789e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f878c1043b0 0x7f878c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:15.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.402+0000 7f878a800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f878c19a7c0 0x7f878c19ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43820/0 (socket says 172.21.15.1:43820) 2024-04-26T02:04:15.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.402+0000 7f878a800640 1 -- 172.21.15.1:0/4025841844 learned_addr learned my addr 172.21.15.1:0/4025841844 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:15.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.402+0000 7f8783e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f878c109480 0x7f878c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:15.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.402+0000 7f8789e00640 1 -- 172.21.15.1:0/4025841844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f878c109480 msgr2=0x7f878c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:15.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.402+0000 7f8789e00640 1 --2- 172.21.15.1:0/4025841844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f878c109480 0x7f878c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.402+0000 7f8789e00640 1 -- 172.21.15.1:0/4025841844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f878c19a7c0 msgr2=0x7f878c19ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:15.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.402+0000 7f8789e00640 1 --2- 172.21.15.1:0/4025841844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f878c19a7c0 0x7f878c19ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:15.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.402+0000 7f8789e00640 1 -- 172.21.15.1:0/4025841844 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878c19b3c0 con 0x7f878c1043b0 2024-04-26T02:04:15.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.402+0000 7f878a800640 1 --2- 172.21.15.1:0/4025841844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f878c19a7c0 0x7f878c19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:04:15.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.403+0000 7f8789e00640 1 --2- 172.21.15.1:0/4025841844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f878c1043b0 0x7f878c1a06b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f877c005fd0 tx=0x7f877c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:15.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.403+0000 7f8782a00640 1 -- 172.21.15.1:0/4025841844 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c00cd00 con 0x7f878c1043b0 2024-04-26T02:04:15.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.403+0000 7f8782a00640 1 -- 172.21.15.1:0/4025841844 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f877c0035f0 con 0x7f878c1043b0 2024-04-26T02:04:15.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.403+0000 7f8782a00640 1 -- 172.21.15.1:0/4025841844 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c010670 con 0x7f878c1043b0 2024-04-26T02:04:15.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.403+0000 7f8790e00640 1 -- 172.21.15.1:0/4025841844 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878c1a7450 con 0x7f878c1043b0 2024-04-26T02:04:15.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.403+0000 7f8790e00640 1 -- 172.21.15.1:0/4025841844 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f878c1a7800 con 0x7f878c1043b0 2024-04-26T02:04:15.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.404+0000 7f8790e00640 1 -- 172.21.15.1:0/4025841844 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f878c105880 con 0x7f878c1043b0 2024-04-26T02:04:15.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.406+0000 7f8782a00640 1 -- 172.21.15.1:0/4025841844 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f877c0037a0 con 0x7f878c1043b0 2024-04-26T02:04:15.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.408+0000 7f8782a00640 1 --2- 172.21.15.1:0/4025841844 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f87700a3d20 0x7f87700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:15.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.408+0000 7f8782a00640 1 -- 172.21.15.1:0/4025841844 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8615+0+0 (secure 0 0 0) 0x7f877c0d0a90 con 0x7f878c1043b0 2024-04-26T02:04:15.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.408+0000 7f8783e00640 1 --2- 172.21.15.1:0/4025841844 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f87700a3d20 0x7f87700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:15.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.409+0000 7f8783e00640 1 --2- 172.21.15.1:0/4025841844 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f87700a3d20 0x7f87700a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f87740303f0 tx=0x7f8774030340 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:15.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.412+0000 7f8782a00640 1 -- 172.21.15.1:0/4025841844 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f877c009510 con 0x7f878c1043b0 2024-04-26T02:04:15.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:15 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/2360827451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T02:04:15.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:15 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3653799968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T02:04:15.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:15 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3387760829' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T02:04:15.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:15 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2360827451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T02:04:15.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:15 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3653799968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T02:04:15.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:15 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3387760829' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T02:04:15.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.675+0000 7f8790e00640 1 -- 172.21.15.1:0/4025841844 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f878c10c710 con 0x7f878c1043b0 2024-04-26T02:04:15.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.685+0000 7f8782a00640 1 -- 172.21.15.1:0/4025841844 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746050 (secure 0 0 0) 0x7f877c09d070 con 0x7f878c1043b0 2024-04-26T02:04:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T02:04:15.692 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T02:04:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 26, 2024-04-26T02:04:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.692 INFO:teuthology.orchestra.run.smithi001.stdout: "active_gid": 25087, 2024-04-26T02:04:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "active_name": "x", 2024-04-26T02:04:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "active_addrs": { 2024-04-26T02:04:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-04-26T02:04:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-04-26T02:04:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2024-04-26T02:04:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 275906565 2024-04-26T02:04:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-04-26T02:04:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2024-04-26T02:04:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 275906565 2024-04-26T02:04:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.693 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-04-26T02:04:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: "active_addr": "172.21.15.1:6801/275906565", 2024-04-26T02:04:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: "active_change": "2024-04-26T01:51:38.660965+0000", 2024-04-26T02:04:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: "active_mgr_features": 4540701547738038271, 2024-04-26T02:04:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: "available": true, 2024-04-26T02:04:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: "standbys": [ 2024-04-26T02:04:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: "gid": 25055, 2024-04-26T02:04:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "y", 2024-04-26T02:04:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_features": 4540701547738038271, 2024-04-26T02:04:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: "available_modules": [ 2024-04-26T02:04:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.694 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alerts", 2024-04-26T02:04:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-04-26T02:04:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-04-26T02:04:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-04-26T02:04:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T02:04:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.695 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.696 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.697 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.698 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_destination": { 2024-04-26T02:04:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_destination", 2024-04-26T02:04:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email address to send alerts to", 2024-04-26T02:04:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.701 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_from_name": { 2024-04-26T02:04:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_from_name", 2024-04-26T02:04:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "Ceph", 2024-04-26T02:04:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email From: name", 2024-04-26T02:04:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.702 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_host": { 2024-04-26T02:04:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_host", 2024-04-26T02:04:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP server", 2024-04-26T02:04:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.703 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_password": { 2024-04-26T02:04:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_password", 2024-04-26T02:04:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Password to authenticate with", 2024-04-26T02:04:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_port": { 2024-04-26T02:04:15.704 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_port", 2024-04-26T02:04:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "465", 2024-04-26T02:04:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP port", 2024-04-26T02:04:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_sender": { 2024-04-26T02:04:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_sender", 2024-04-26T02:04:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.705 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP envelope sender", 2024-04-26T02:04:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_ssl": { 2024-04-26T02:04:15.706 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_ssl", 2024-04-26T02:04:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T02:04:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.707 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_user": { 2024-04-26T02:04:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_user", 2024-04-26T02:04:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "User to authenticate as", 2024-04-26T02:04:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.708 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "balancer", 2024-04-26T02:04:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "active": { 2024-04-26T02:04:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "active", 2024-04-26T02:04:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.709 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T02:04:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_time": { 2024-04-26T02:04:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_time", 2024-04-26T02:04:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0000", 2024-04-26T02:04:15.710 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T02:04:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T02:04:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_weekday": { 2024-04-26T02:04:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_weekday", 2024-04-26T02:04:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-04-26T02:04:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-04-26T02:04:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-04-26T02:04:15.711 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-04-26T02:04:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T02:04:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T02:04:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_max_iterations": { 2024-04-26T02:04:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_max_iterations", 2024-04-26T02:04:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-04-26T02:04:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "25", 2024-04-26T02:04:15.712 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-04-26T02:04:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "250", 2024-04-26T02:04:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T02:04:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_metrics": { 2024-04-26T02:04:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_metrics", 2024-04-26T02:04:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T02:04:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.713 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T02:04:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T02:04:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_step": { 2024-04-26T02:04:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_step", 2024-04-26T02:04:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-04-26T02:04:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.5", 2024-04-26T02:04:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0.001", 2024-04-26T02:04:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "0.999", 2024-04-26T02:04:15.714 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "aggressiveness of optimization", 2024-04-26T02:04:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T02:04:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "end_time": { 2024-04-26T02:04:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_time", 2024-04-26T02:04:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2359", 2024-04-26T02:04:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T02:04:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T02:04:15.715 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "end_weekday": { 2024-04-26T02:04:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_weekday", 2024-04-26T02:04:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-04-26T02:04:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-04-26T02:04:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-04-26T02:04:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-04-26T02:04:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T02:04:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T02:04:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.716 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.717 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.718 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.719 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.720 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: "min_score": { 2024-04-26T02:04:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_score", 2024-04-26T02:04:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-04-26T02:04:15.721 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-04-26T02:04:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T02:04:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-04-26T02:04:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-04-26T02:04:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "upmap", 2024-04-26T02:04:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-compat", 2024-04-26T02:04:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "none", 2024-04-26T02:04:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "read", 2024-04-26T02:04:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap", 2024-04-26T02:04:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap-read" 2024-04-26T02:04:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Balancer mode", 2024-04-26T02:04:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_ids": { 2024-04-26T02:04:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_ids", 2024-04-26T02:04:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T02:04:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.724 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-04-26T02:04:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-04-26T02:04:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-04-26T02:04:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T02:04:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_deviation": { 2024-04-26T02:04:15.725 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_deviation", 2024-04-26T02:04:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-04-26T02:04:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-04-26T02:04:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T02:04:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T02:04:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_optimizations": { 2024-04-26T02:04:15.726 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_optimizations", 2024-04-26T02:04:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-04-26T02:04:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T02:04:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.727 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm", 2024-04-26T02:04:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_down_multiplier": { 2024-04-26T02:04:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_down_multiplier", 2024-04-26T02:04:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-04-26T02:04:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-04-26T02:04:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T02:04:15.728 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_refresh_rate": { 2024-04-26T02:04:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_refresh_rate", 2024-04-26T02:04:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "20", 2024-04-26T02:04:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T02:04:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.729 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_starting_port": { 2024-04-26T02:04:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_starting_port", 2024-04-26T02:04:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4721", 2024-04-26T02:04:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T02:04:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.730 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_ptrace": { 2024-04-26T02:04:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_ptrace", 2024-04-26T02:04:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T02:04:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T02:04:15.731 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_interval": { 2024-04-26T02:04:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_interval", 2024-04-26T02:04:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-04-26T02:04:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T02:04:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.732 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_memory_target_ratio": { 2024-04-26T02:04:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T02:04:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-04-26T02:04:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.7", 2024-04-26T02:04:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T02:04:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.733 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm_log_destination": { 2024-04-26T02:04:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm_log_destination", 2024-04-26T02:04:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: "file", 2024-04-26T02:04:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: "file,syslog", 2024-04-26T02:04:15.734 INFO:teuthology.orchestra.run.smithi001.stdout: "syslog" 2024-04-26T02:04:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T02:04:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "cgroups_split": { 2024-04-26T02:04:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cgroups_split", 2024-04-26T02:04:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.735 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T02:04:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: "config_checks_enabled": { 2024-04-26T02:04:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_checks_enabled", 2024-04-26T02:04:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.736 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.737 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T02:04:15.737 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.737 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.737 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.737 INFO:teuthology.orchestra.run.smithi001.stdout: "config_dashboard": { 2024-04-26T02:04:15.737 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_dashboard", 2024-04-26T02:04:15.737 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.737 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.737 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.737 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.738 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.738 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.738 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.738 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T02:04:15.738 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.738 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.738 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.738 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_alertmanager": { 2024-04-26T02:04:15.738 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_alertmanager", 2024-04-26T02:04:15.738 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.738 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.738 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.738 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T02:04:15.739 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.739 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.739 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.739 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-04-26T02:04:15.739 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.739 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.739 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.739 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_base": { 2024-04-26T02:04:15.739 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_base", 2024-04-26T02:04:15.739 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.739 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.739 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.739 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T02:04:15.739 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.740 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.740 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.740 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Container image name, without the tag", 2024-04-26T02:04:15.740 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.740 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.740 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.740 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_elasticsearch": { 2024-04-26T02:04:15.740 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_elasticsearch", 2024-04-26T02:04:15.740 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.740 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.740 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.740 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T02:04:15.740 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.741 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.741 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.741 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "elasticsearch container image", 2024-04-26T02:04:15.741 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.741 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.741 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.741 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_grafana": { 2024-04-26T02:04:15.741 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_grafana", 2024-04-26T02:04:15.741 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.741 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.741 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.741 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T02:04:15.741 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.741 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.741 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-04-26T02:04:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_haproxy": { 2024-04-26T02:04:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_haproxy", 2024-04-26T02:04:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.742 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T02:04:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "HAproxy container image", 2024-04-26T02:04:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_agent": { 2024-04-26T02:04:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_agent", 2024-04-26T02:04:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.743 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T02:04:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger agent container image", 2024-04-26T02:04:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_collector": { 2024-04-26T02:04:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_collector", 2024-04-26T02:04:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.744 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T02:04:15.745 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.745 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.745 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.745 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger collector container image", 2024-04-26T02:04:15.745 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.745 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.745 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.745 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_query": { 2024-04-26T02:04:15.745 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_query", 2024-04-26T02:04:15.745 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T02:04:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger query container image", 2024-04-26T02:04:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_keepalived": { 2024-04-26T02:04:15.746 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_keepalived", 2024-04-26T02:04:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T02:04:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Keepalived container image", 2024-04-26T02:04:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_loki": { 2024-04-26T02:04:15.747 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_loki", 2024-04-26T02:04:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T02:04:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Loki container image", 2024-04-26T02:04:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_node_exporter": { 2024-04-26T02:04:15.748 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_node_exporter", 2024-04-26T02:04:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T02:04:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-04-26T02:04:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_nvmeof": { 2024-04-26T02:04:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_nvmeof", 2024-04-26T02:04:15.749 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T02:04:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Nvme-of container image", 2024-04-26T02:04:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_prometheus": { 2024-04-26T02:04:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_prometheus", 2024-04-26T02:04:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.750 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T02:04:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-04-26T02:04:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_promtail": { 2024-04-26T02:04:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_promtail", 2024-04-26T02:04:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.751 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T02:04:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Promtail container image", 2024-04-26T02:04:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_samba": { 2024-04-26T02:04:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_samba", 2024-04-26T02:04:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T02:04:15.752 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Samba/SMB container image", 2024-04-26T02:04:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_snmp_gateway": { 2024-04-26T02:04:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_snmp_gateway", 2024-04-26T02:04:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T02:04:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.753 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SNMP Gateway container image", 2024-04-26T02:04:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "container_init": { 2024-04-26T02:04:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_init", 2024-04-26T02:04:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.754 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T02:04:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: "daemon_cache_timeout": { 2024-04-26T02:04:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "daemon_cache_timeout", 2024-04-26T02:04:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-04-26T02:04:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.755 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T02:04:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: "default_cephadm_command_timeout": { 2024-04-26T02:04:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T02:04:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "900", 2024-04-26T02:04:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.756 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T02:04:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "default_registry": { 2024-04-26T02:04:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_registry", 2024-04-26T02:04:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io", 2024-04-26T02:04:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T02:04:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: "device_cache_timeout": { 2024-04-26T02:04:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_cache_timeout", 2024-04-26T02:04:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1800", 2024-04-26T02:04:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.758 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache device inventory", 2024-04-26T02:04:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: "device_enhanced_scan": { 2024-04-26T02:04:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_enhanced_scan", 2024-04-26T02:04:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T02:04:15.759 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: "facts_cache_timeout": { 2024-04-26T02:04:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "facts_cache_timeout", 2024-04-26T02:04:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-04-26T02:04:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache host facts data", 2024-04-26T02:04:15.760 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: "grafana_dashboards_path": { 2024-04-26T02:04:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "grafana_dashboards_path", 2024-04-26T02:04:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T02:04:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T02:04:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.761 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "host_check_interval": { 2024-04-26T02:04:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "host_check_interval", 2024-04-26T02:04:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-04-26T02:04:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to perform a host check", 2024-04-26T02:04:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.762 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "hw_monitoring": { 2024-04-26T02:04:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hw_monitoring", 2024-04-26T02:04:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T02:04:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.763 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: "inventory_list_all": { 2024-04-26T02:04:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "inventory_list_all", 2024-04-26T02:04:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T02:04:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.764 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.765 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.766 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.766 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.766 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.766 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.766 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.766 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.766 INFO:teuthology.orchestra.run.smithi001.stdout: "log_refresh_metadata": { 2024-04-26T02:04:15.766 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_refresh_metadata", 2024-04-26T02:04:15.766 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.766 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.766 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T02:04:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.767 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T02:04:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.768 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.769 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.770 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T02:04:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T02:04:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T02:04:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.771 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T02:04:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T02:04:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "*", 2024-04-26T02:04:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T02:04:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.772 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "max_count_per_host": { 2024-04-26T02:04:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_count_per_host", 2024-04-26T02:04:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of daemons per service per host", 2024-04-26T02:04:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.773 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd_draining_count": { 2024-04-26T02:04:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_osd_draining_count", 2024-04-26T02:04:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T02:04:15.774 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "migration_current": { 2024-04-26T02:04:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "migration_current", 2024-04-26T02:04:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "internal - do not modify", 2024-04-26T02:04:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.775 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-04-26T02:04:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-04-26T02:04:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "root", 2024-04-26T02:04:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm-package", 2024-04-26T02:04:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: "root" 2024-04-26T02:04:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T02:04:15.776 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "oob_default_addr": { 2024-04-26T02:04:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "oob_default_addr", 2024-04-26T02:04:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "169.254.1.1", 2024-04-26T02:04:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T02:04:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.777 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.778 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.778 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_alerts_path": { 2024-04-26T02:04:15.778 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_alerts_path", 2024-04-26T02:04:15.778 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.778 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.778 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.778 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T02:04:15.778 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.778 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.778 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.778 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T02:04:15.778 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.778 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_insecure": { 2024-04-26T02:04:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_insecure", 2024-04-26T02:04:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T02:04:15.779 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_password": { 2024-04-26T02:04:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_password", 2024-04-26T02:04:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T02:04:15.780 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_url": { 2024-04-26T02:04:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_url", 2024-04-26T02:04:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.781 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T02:04:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_username": { 2024-04-26T02:04:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_username", 2024-04-26T02:04:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.782 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T02:04:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "secure_monitoring_stack": { 2024-04-26T02:04:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secure_monitoring_stack", 2024-04-26T02:04:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.783 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T02:04:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: "service_discovery_port": { 2024-04-26T02:04:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "service_discovery_port", 2024-04-26T02:04:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8765", 2024-04-26T02:04:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "cephadm service discovery port", 2024-04-26T02:04:15.784 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: "ssh_config_file": { 2024-04-26T02:04:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssh_config_file", 2024-04-26T02:04:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T02:04:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.785 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.786 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.786 INFO:teuthology.orchestra.run.smithi001.stdout: "use_agent": { 2024-04-26T02:04:15.786 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_agent", 2024-04-26T02:04:15.786 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.786 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.786 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.786 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.786 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.786 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.786 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.786 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T02:04:15.786 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.786 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: "use_repo_digest": { 2024-04-26T02:04:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_repo_digest", 2024-04-26T02:04:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T02:04:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.787 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_failed_host_check": { 2024-04-26T02:04:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_failed_host_check", 2024-04-26T02:04:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T02:04:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.788 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_daemons": { 2024-04-26T02:04:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_daemons", 2024-04-26T02:04:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T02:04:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_hosts": { 2024-04-26T02:04:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_hosts", 2024-04-26T02:04:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T02:04:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.790 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crash", 2024-04-26T02:04:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.791 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.793 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:15.793 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:15.793 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.793 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.793 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.793 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.793 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.793 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.793 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.793 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.793 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.793 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.793 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.793 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:15.794 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:15.794 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.794 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.794 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.794 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:15.794 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.794 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.794 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.794 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.794 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.794 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.794 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.794 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.795 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.795 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.795 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.795 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.795 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.795 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.795 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:15.795 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:15.795 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.795 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.795 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.795 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.795 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.795 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.795 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.796 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.796 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.796 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.796 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.796 INFO:teuthology.orchestra.run.smithi001.stdout: "retain_interval": { 2024-04-26T02:04:15.796 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retain_interval", 2024-04-26T02:04:15.796 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:15.796 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.796 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.796 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "31536000", 2024-04-26T02:04:15.796 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.796 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.797 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.797 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T02:04:15.797 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.797 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.797 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.797 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_recent_interval": { 2024-04-26T02:04:15.797 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_recent_interval", 2024-04-26T02:04:15.797 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:15.797 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.797 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.797 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1209600", 2024-04-26T02:04:15.797 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.797 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.797 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.798 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T02:04:15.798 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.798 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.798 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.798 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.798 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.798 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dashboard", 2024-04-26T02:04:15.798 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:15.798 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:15.798 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.798 INFO:teuthology.orchestra.run.smithi001.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T02:04:15.799 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T02:04:15.799 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.799 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.799 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.799 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:15.799 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.799 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.799 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.799 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.799 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.799 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.799 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.799 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T02:04:15.799 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T02:04:15.800 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.800 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.800 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.800 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.800 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.800 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.800 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.800 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.800 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.800 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.800 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.800 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T02:04:15.800 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T02:04:15.801 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.801 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.801 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.801 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.801 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.801 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.801 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.801 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.801 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.801 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.801 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.801 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_ENABLED": { 2024-04-26T02:04:15.801 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T02:04:15.802 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.802 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.802 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.802 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.802 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.802 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.802 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.802 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.802 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.802 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.802 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.802 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T02:04:15.802 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T02:04:15.802 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.803 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.803 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.803 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.803 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.803 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.803 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.803 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.803 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.803 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.803 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.803 INFO:teuthology.orchestra.run.smithi001.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T02:04:15.803 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T02:04:15.803 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.803 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.804 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.804 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.804 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.804 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.804 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.804 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.804 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.804 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.804 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.804 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T02:04:15.804 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T02:04:15.804 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.805 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.805 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.805 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.805 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.805 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.805 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.805 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.805 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.805 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.805 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.805 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T02:04:15.805 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T02:04:15.805 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.805 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.806 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.806 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.806 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.806 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.806 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.806 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.806 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.806 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.806 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.806 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T02:04:15.806 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T02:04:15.806 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.807 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.807 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.807 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.807 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.807 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.807 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.807 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.807 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.807 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.807 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.807 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T02:04:15.807 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T02:04:15.807 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.807 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.808 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.808 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.808 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.808 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.808 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.808 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.808 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.808 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.808 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.808 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T02:04:15.808 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T02:04:15.808 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.808 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.808 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.809 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.809 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.809 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.809 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.809 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.809 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.809 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.809 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.809 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T02:04:15.809 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T02:04:15.809 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.809 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.809 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.810 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.810 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.810 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.810 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.810 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.810 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.810 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.810 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.810 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T02:04:15.810 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T02:04:15.810 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.810 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.810 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.810 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.811 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.811 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.811 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.811 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.811 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.811 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.811 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.811 INFO:teuthology.orchestra.run.smithi001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T02:04:15.811 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T02:04:15.811 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.811 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.811 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.811 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.812 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.812 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.812 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.812 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.812 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.812 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.812 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.812 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T02:04:15.812 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T02:04:15.812 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.812 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.812 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.812 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-04-26T02:04:15.812 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.813 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.813 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.813 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.813 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.813 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.813 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.813 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T02:04:15.813 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T02:04:15.813 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.813 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.813 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.813 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.813 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.813 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.814 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.814 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.814 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.814 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.814 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.814 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_URL": { 2024-04-26T02:04:15.814 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_URL", 2024-04-26T02:04:15.814 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.814 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.814 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.814 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.814 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.814 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.815 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.815 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.815 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.815 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.815 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.815 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T02:04:15.815 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T02:04:15.815 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.815 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.815 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.815 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-04-26T02:04:15.815 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.815 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.815 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.816 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.816 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.816 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.816 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.816 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T02:04:15.816 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T02:04:15.816 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.816 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.816 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.816 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.816 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.816 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.816 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.816 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.817 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.817 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.817 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.817 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T02:04:15.817 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T02:04:15.817 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.817 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.817 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.817 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.817 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.817 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.817 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.817 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.818 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.818 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.818 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.818 INFO:teuthology.orchestra.run.smithi001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T02:04:15.818 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T02:04:15.818 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.818 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.818 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.818 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.818 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.818 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.818 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.819 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.819 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.819 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.819 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.819 INFO:teuthology.orchestra.run.smithi001.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T02:04:15.819 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T02:04:15.819 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.819 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.819 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.819 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.819 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.819 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.819 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.819 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.820 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.820 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.820 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.820 INFO:teuthology.orchestra.run.smithi001.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T02:04:15.820 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T02:04:15.820 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.820 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.820 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.820 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "[]", 2024-04-26T02:04:15.820 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.820 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.820 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.821 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.821 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.821 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.821 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.821 INFO:teuthology.orchestra.run.smithi001.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T02:04:15.821 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T02:04:15.821 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.821 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.821 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.821 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "{}", 2024-04-26T02:04:15.821 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.821 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.821 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.821 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.822 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.822 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.822 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.822 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T02:04:15.822 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T02:04:15.822 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.822 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.822 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.822 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.822 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.822 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.822 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.822 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.823 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.823 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.823 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.823 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T02:04:15.823 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T02:04:15.823 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.823 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.823 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.823 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.823 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.823 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.823 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.823 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.823 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.824 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.824 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.824 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T02:04:15.824 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T02:04:15.824 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.824 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.824 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.824 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.824 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.824 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.824 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.824 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.824 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.824 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.825 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.825 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T02:04:15.825 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T02:04:15.825 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.825 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.825 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.825 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.825 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.825 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.825 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.825 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.825 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.825 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.826 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.826 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T02:04:15.826 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T02:04:15.826 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.826 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.826 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.826 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.826 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.826 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.826 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.826 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.826 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.826 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.826 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.827 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T02:04:15.827 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T02:04:15.827 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.827 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.827 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.827 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.827 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.827 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.827 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.827 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.827 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.827 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.827 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.828 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T02:04:15.828 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T02:04:15.828 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.828 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.828 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.828 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.828 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.828 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.828 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.828 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.828 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.828 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.828 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.829 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T02:04:15.829 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T02:04:15.829 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.829 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.829 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.829 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.829 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.829 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.829 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.829 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.829 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.829 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.829 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.829 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T02:04:15.830 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T02:04:15.830 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.830 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.830 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.830 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.830 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.830 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.830 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.830 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.830 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.830 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.830 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.830 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T02:04:15.831 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T02:04:15.831 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.831 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.831 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.831 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.831 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.831 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.831 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.831 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.831 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.831 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.831 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.831 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T02:04:15.832 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T02:04:15.832 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.832 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.832 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.832 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T02:04:15.832 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.832 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.832 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.832 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.832 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.832 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.832 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.833 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T02:04:15.833 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T02:04:15.833 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.833 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.833 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.833 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:15.833 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.833 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.833 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.833 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.833 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.833 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.833 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.834 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T02:04:15.834 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T02:04:15.834 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.834 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.834 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.834 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8", 2024-04-26T02:04:15.834 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.834 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.834 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.834 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.834 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.834 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.834 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.835 INFO:teuthology.orchestra.run.smithi001.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T02:04:15.835 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T02:04:15.835 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.835 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.835 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.835 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "45", 2024-04-26T02:04:15.835 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.835 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.835 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.835 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.835 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.835 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.835 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.836 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T02:04:15.836 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T02:04:15.836 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.836 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.836 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.836 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.836 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.836 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.836 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.836 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.836 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.836 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.836 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.837 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T02:04:15.837 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T02:04:15.837 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.837 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.837 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.837 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-04-26T02:04:15.837 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.837 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.837 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.837 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.837 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.837 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.837 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.838 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T02:04:15.838 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T02:04:15.838 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.838 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.838 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.838 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.838 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.838 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.838 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.838 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.838 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.838 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.838 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.839 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T02:04:15.839 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T02:04:15.839 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.839 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.839 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.839 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.839 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.839 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.839 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.839 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.839 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.839 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.839 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.840 INFO:teuthology.orchestra.run.smithi001.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T02:04:15.840 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T02:04:15.840 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.840 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.840 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.840 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.840 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.840 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.840 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.840 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.840 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.840 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.840 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.841 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T02:04:15.841 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T02:04:15.841 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.841 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.841 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.841 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-04-26T02:04:15.841 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.841 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.841 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.841 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.841 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.841 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.841 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.842 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T02:04:15.842 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T02:04:15.842 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.842 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.842 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.842 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:15.842 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.842 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.842 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.842 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.842 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.842 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.842 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.843 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T02:04:15.843 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T02:04:15.843 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.843 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.843 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.843 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-04-26T02:04:15.843 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.843 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.843 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.843 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.843 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.843 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.843 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.844 INFO:teuthology.orchestra.run.smithi001.stdout: "cross_origin_url": { 2024-04-26T02:04:15.844 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cross_origin_url", 2024-04-26T02:04:15.844 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.844 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.844 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.844 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.844 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.844 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.844 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.844 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.844 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.844 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.844 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.845 INFO:teuthology.orchestra.run.smithi001.stdout: "crt_file": { 2024-04-26T02:04:15.845 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crt_file", 2024-04-26T02:04:15.845 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.845 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.845 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.845 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.845 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.845 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.845 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.845 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.845 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.845 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.845 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.846 INFO:teuthology.orchestra.run.smithi001.stdout: "debug": { 2024-04-26T02:04:15.846 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "debug", 2024-04-26T02:04:15.846 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.846 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.846 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.846 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.846 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.846 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.846 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.846 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable/disable debug options", 2024-04-26T02:04:15.846 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.846 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.846 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.847 INFO:teuthology.orchestra.run.smithi001.stdout: "jwt_token_ttl": { 2024-04-26T02:04:15.847 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "jwt_token_ttl", 2024-04-26T02:04:15.847 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.847 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.847 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.847 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "28800", 2024-04-26T02:04:15.847 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.847 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.847 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.847 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.847 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.847 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.847 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.848 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-04-26T02:04:15.848 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-04-26T02:04:15.848 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.848 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.848 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.848 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.848 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.848 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.848 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.848 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.848 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.848 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.848 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.849 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.849 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.849 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.849 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.849 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.849 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.849 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.849 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.849 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.849 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.849 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.849 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.849 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.850 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.850 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.850 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.850 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.850 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.850 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.850 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.850 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:15.850 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:15.850 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.850 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.850 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.850 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.851 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.852 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.852 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.852 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.852 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.852 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.852 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.852 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.852 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.852 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.852 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.852 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.852 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.852 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:15.852 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:15.852 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.853 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.853 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.853 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.853 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.853 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.853 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.853 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.853 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.853 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.853 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.853 INFO:teuthology.orchestra.run.smithi001.stdout: "motd": { 2024-04-26T02:04:15.853 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "motd", 2024-04-26T02:04:15.853 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.853 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.853 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.854 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.854 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.854 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.854 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.854 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "The message of the day", 2024-04-26T02:04:15.854 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.854 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.854 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.854 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect_resolve_ip_addr": { 2024-04-26T02:04:15.854 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T02:04:15.854 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.854 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.854 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.854 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.855 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.855 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.855 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.855 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.855 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.855 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.855 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.855 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-04-26T02:04:15.855 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-04-26T02:04:15.855 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.855 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.855 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.855 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-04-26T02:04:15.855 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.855 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.856 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.856 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.856 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.856 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.856 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.856 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-04-26T02:04:15.856 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-04-26T02:04:15.856 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.856 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.856 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.856 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8080", 2024-04-26T02:04:15.856 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.856 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.856 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.856 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.857 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.857 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.857 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.857 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-04-26T02:04:15.857 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-04-26T02:04:15.857 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.857 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.857 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.857 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.857 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.857 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.857 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.857 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.857 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.857 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl_server_port": { 2024-04-26T02:04:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl_server_port", 2024-04-26T02:04:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8443", 2024-04-26T02:04:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.858 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.859 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-04-26T02:04:15.859 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-04-26T02:04:15.859 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.859 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.859 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.859 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "redirect", 2024-04-26T02:04:15.859 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.859 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.859 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.859 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.859 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect" 2024-04-26T02:04:15.859 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.859 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.860 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.860 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.860 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.860 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-04-26T02:04:15.860 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-04-26T02:04:15.860 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.860 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.860 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.860 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-04-26T02:04:15.860 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-04-26T02:04:15.860 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-04-26T02:04:15.860 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.860 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.861 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.861 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.861 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.861 INFO:teuthology.orchestra.run.smithi001.stdout: "url_prefix": { 2024-04-26T02:04:15.861 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url_prefix", 2024-04-26T02:04:15.861 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.861 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.861 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.861 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.861 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.861 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.861 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.861 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.862 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.862 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.862 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.862 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.862 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-04-26T02:04:15.862 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:15.862 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:15.862 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.865 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_monitoring": { 2024-04-26T02:04:15.866 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_monitoring", 2024-04-26T02:04:15.866 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.866 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.866 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.866 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.866 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.866 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.866 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.866 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "monitor device health metrics", 2024-04-26T02:04:15.866 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.866 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.866 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.867 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.867 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.867 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.867 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.867 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.867 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.867 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.867 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.867 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.867 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.867 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.867 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.867 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.868 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.868 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.868 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.868 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.868 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.868 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.868 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.868 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:15.868 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:15.868 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.868 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.868 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.868 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.869 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.869 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.869 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.869 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.869 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.869 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.869 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.869 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:15.869 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:15.869 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.869 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.869 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.870 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:15.870 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.870 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.870 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.870 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.870 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.870 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.870 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.870 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.870 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.870 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.870 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.870 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.871 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.871 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.871 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:15.871 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:15.871 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.871 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.871 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.871 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.871 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.871 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.871 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.871 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.871 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.872 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.872 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.872 INFO:teuthology.orchestra.run.smithi001.stdout: "mark_out_threshold": { 2024-04-26T02:04:15.872 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mark_out_threshold", 2024-04-26T02:04:15.872 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:15.872 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.872 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.872 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2419200", 2024-04-26T02:04:15.872 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.872 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.872 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.872 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T02:04:15.872 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.873 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.873 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.873 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": { 2024-04-26T02:04:15.873 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_name", 2024-04-26T02:04:15.873 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.873 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.873 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.873 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "device_health_metrics", 2024-04-26T02:04:15.873 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.873 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.873 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.873 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T02:04:15.873 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.874 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.874 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.874 INFO:teuthology.orchestra.run.smithi001.stdout: "retention_period": { 2024-04-26T02:04:15.874 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retention_period", 2024-04-26T02:04:15.874 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:15.874 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.874 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.874 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15552000", 2024-04-26T02:04:15.874 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.874 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.874 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.874 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain device health metrics", 2024-04-26T02:04:15.874 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.875 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.875 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.875 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_frequency": { 2024-04-26T02:04:15.875 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_frequency", 2024-04-26T02:04:15.875 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:15.875 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.875 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.875 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-04-26T02:04:15.875 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.875 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.875 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.875 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T02:04:15.876 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.876 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.876 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.876 INFO:teuthology.orchestra.run.smithi001.stdout: "self_heal": { 2024-04-26T02:04:15.876 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "self_heal", 2024-04-26T02:04:15.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.876 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.876 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.876 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.876 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.876 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.876 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.876 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T02:04:15.877 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.877 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.877 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.877 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-04-26T02:04:15.877 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-04-26T02:04:15.877 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:15.877 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.877 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.877 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-04-26T02:04:15.877 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.877 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.877 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.877 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T02:04:15.878 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.878 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.878 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.878 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_threshold": { 2024-04-26T02:04:15.878 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_threshold", 2024-04-26T02:04:15.878 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:15.878 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.878 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.878 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7257600", 2024-04-26T02:04:15.878 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.878 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.878 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.878 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T02:04:15.879 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.879 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.879 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.879 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.879 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.879 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "diskprediction_local", 2024-04-26T02:04:15.879 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:15.879 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:15.879 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.879 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.879 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.879 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.880 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.880 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.880 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.880 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.880 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.880 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.880 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.880 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.880 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.880 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.880 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.880 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.880 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.880 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.881 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.881 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.881 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.881 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:15.881 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:15.881 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.881 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.881 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.881 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.881 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.881 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.881 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.881 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.882 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.882 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.882 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.882 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:15.882 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:15.882 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.882 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.882 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.882 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:15.882 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.882 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.882 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.882 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.883 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.883 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.883 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.883 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.883 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.883 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.883 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.883 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.883 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.883 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.883 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:15.883 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:15.884 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.884 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.884 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.884 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.884 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.884 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.884 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.884 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.884 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.884 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.884 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.884 INFO:teuthology.orchestra.run.smithi001.stdout: "predict_interval": { 2024-04-26T02:04:15.884 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predict_interval", 2024-04-26T02:04:15.885 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.885 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.885 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.885 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-04-26T02:04:15.885 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.885 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.885 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.885 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.885 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.885 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.885 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.885 INFO:teuthology.orchestra.run.smithi001.stdout: "predictor_model": { 2024-04-26T02:04:15.885 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predictor_model", 2024-04-26T02:04:15.886 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.886 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.886 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.886 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "prophetstor", 2024-04-26T02:04:15.886 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.886 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.886 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.886 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.886 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.886 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.886 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.886 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-04-26T02:04:15.887 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-04-26T02:04:15.887 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.887 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.887 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.887 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-04-26T02:04:15.887 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.887 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.887 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.887 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.887 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.887 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.887 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.887 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.887 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.888 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.888 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "influx", 2024-04-26T02:04:15.888 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": false, 2024-04-26T02:04:15.888 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "influxdb python module not found", 2024-04-26T02:04:15.888 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.888 INFO:teuthology.orchestra.run.smithi001.stdout: "batch_size": { 2024-04-26T02:04:15.888 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "batch_size", 2024-04-26T02:04:15.888 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.888 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.888 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.888 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5000", 2024-04-26T02:04:15.888 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.888 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.889 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.889 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T02:04:15.889 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.889 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.889 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.889 INFO:teuthology.orchestra.run.smithi001.stdout: "database": { 2024-04-26T02:04:15.889 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "database", 2024-04-26T02:04:15.889 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.889 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.889 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.889 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "ceph", 2024-04-26T02:04:15.889 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.889 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.889 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.890 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T02:04:15.890 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.890 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.890 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.890 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": { 2024-04-26T02:04:15.890 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hostname", 2024-04-26T02:04:15.890 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.890 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.890 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.890 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.890 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.890 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.890 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.891 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server hostname", 2024-04-26T02:04:15.891 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.891 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.891 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.891 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-04-26T02:04:15.891 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-04-26T02:04:15.891 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:15.891 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.891 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.891 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "30", 2024-04-26T02:04:15.891 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "5", 2024-04-26T02:04:15.891 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.891 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.891 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T02:04:15.892 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.892 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.892 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.892 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.892 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.892 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.892 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.892 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.892 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.892 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.892 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.892 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.892 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.892 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.892 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.894 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.894 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.894 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.894 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.894 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.894 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.894 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:15.894 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:15.894 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.894 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.894 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.894 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:15.894 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.894 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.895 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.896 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.896 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.896 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.896 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.896 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.896 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.896 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.896 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.896 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.896 INFO:teuthology.orchestra.run.smithi001.stdout: "password": { 2024-04-26T02:04:15.896 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "password", 2024-04-26T02:04:15.896 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.896 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.896 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.896 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.897 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.897 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.897 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.897 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "password of InfluxDB server user", 2024-04-26T02:04:15.897 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.897 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.897 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.897 INFO:teuthology.orchestra.run.smithi001.stdout: "port": { 2024-04-26T02:04:15.897 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "port", 2024-04-26T02:04:15.897 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.897 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.897 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.897 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8086", 2024-04-26T02:04:15.897 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.898 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.898 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.898 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server port", 2024-04-26T02:04:15.898 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.898 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.898 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.898 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-04-26T02:04:15.898 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-04-26T02:04:15.898 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.898 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.898 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.898 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "false", 2024-04-26T02:04:15.899 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.899 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.899 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.899 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T02:04:15.899 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.899 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.899 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.899 INFO:teuthology.orchestra.run.smithi001.stdout: "threads": { 2024-04-26T02:04:15.899 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threads", 2024-04-26T02:04:15.899 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.899 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.899 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.899 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-04-26T02:04:15.900 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-04-26T02:04:15.900 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "32", 2024-04-26T02:04:15.900 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.900 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T02:04:15.900 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.900 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.900 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.900 INFO:teuthology.orchestra.run.smithi001.stdout: "username": { 2024-04-26T02:04:15.900 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "username", 2024-04-26T02:04:15.900 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.900 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.900 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.900 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.900 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.901 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.901 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.901 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "username of InfluxDB server user", 2024-04-26T02:04:15.901 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.901 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.901 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.901 INFO:teuthology.orchestra.run.smithi001.stdout: "verify_ssl": { 2024-04-26T02:04:15.901 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "verify_ssl", 2024-04-26T02:04:15.901 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.901 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.901 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.901 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "true", 2024-04-26T02:04:15.901 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.902 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.902 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.902 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T02:04:15.902 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.902 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.902 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.902 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.902 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.902 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "insights", 2024-04-26T02:04:15.902 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:15.902 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:15.903 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.903 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.903 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.903 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.903 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.903 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.903 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.903 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.903 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.903 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.903 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.903 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.903 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.903 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.903 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.904 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.905 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.906 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "iostat", 2024-04-26T02:04:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.908 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:15.909 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:15.910 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.911 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.912 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.913 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.913 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.913 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.913 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.913 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.913 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.913 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "k8sevents", 2024-04-26T02:04:15.913 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:15.913 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:15.913 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.913 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_event_retention_days": { 2024-04-26T02:04:15.913 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ceph_event_retention_days", 2024-04-26T02:04:15.913 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.914 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.914 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.914 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7", 2024-04-26T02:04:15.914 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.914 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.914 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.914 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T02:04:15.914 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.914 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.914 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.914 INFO:teuthology.orchestra.run.smithi001.stdout: "config_check_secs": { 2024-04-26T02:04:15.914 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_check_secs", 2024-04-26T02:04:15.915 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.915 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.915 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.915 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:15.915 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "10", 2024-04-26T02:04:15.915 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.915 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.915 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T02:04:15.915 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.915 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.915 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.915 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.915 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.916 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.916 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.916 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.916 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.916 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.916 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.916 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.916 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.916 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.916 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.916 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.916 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.916 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.917 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.917 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.917 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.917 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.917 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.917 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:15.917 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:15.917 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.917 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.917 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.917 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.917 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.918 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.918 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.918 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.918 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.918 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.918 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.918 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:15.918 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:15.918 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.918 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.918 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.918 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:15.918 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.918 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.919 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.919 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.919 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.919 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.919 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.919 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.919 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.919 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.919 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.919 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.919 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.919 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.919 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:15.920 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:15.920 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.920 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.920 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.920 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.920 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.920 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.920 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.920 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.920 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.920 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.920 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.920 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.920 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.921 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "localpool", 2024-04-26T02:04:15.921 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:15.921 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:15.921 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.921 INFO:teuthology.orchestra.run.smithi001.stdout: "failure_domain": { 2024-04-26T02:04:15.921 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "failure_domain", 2024-04-26T02:04:15.921 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.921 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.921 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.921 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "host", 2024-04-26T02:04:15.921 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.921 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.921 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.922 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "failure domain for any created local pool", 2024-04-26T02:04:15.922 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T02:04:15.922 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.922 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.922 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.922 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.922 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.922 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.922 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.922 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.922 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.922 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.922 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.922 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.923 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.923 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.923 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.923 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.923 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.923 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.923 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.923 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.923 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.923 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.923 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:15.923 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:15.923 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.923 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.924 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.924 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.924 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.924 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.924 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.924 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.924 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.924 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.924 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.924 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:15.924 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:15.924 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.924 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.924 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.925 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:15.925 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.925 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.925 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.925 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.925 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.925 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.925 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.925 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.925 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.925 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.925 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.925 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.925 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.926 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.926 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:15.926 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:15.926 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.926 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.926 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.926 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.926 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.926 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.926 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.926 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.926 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.926 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.927 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.927 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": { 2024-04-26T02:04:15.927 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_size", 2024-04-26T02:04:15.927 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.927 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.927 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.927 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.927 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.927 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.927 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.927 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default min_size for any created local pool", 2024-04-26T02:04:15.927 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T02:04:15.927 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.928 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.928 INFO:teuthology.orchestra.run.smithi001.stdout: "num_rep": { 2024-04-26T02:04:15.928 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "num_rep", 2024-04-26T02:04:15.928 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.928 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.928 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.928 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3", 2024-04-26T02:04:15.928 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.928 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.928 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.928 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default replica count for any created local pool", 2024-04-26T02:04:15.928 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.928 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.928 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.929 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": { 2024-04-26T02:04:15.929 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_num", 2024-04-26T02:04:15.929 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.929 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.929 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.929 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "128", 2024-04-26T02:04:15.929 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.929 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.929 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.929 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T02:04:15.929 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.929 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.930 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.930 INFO:teuthology.orchestra.run.smithi001.stdout: "prefix": { 2024-04-26T02:04:15.930 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prefix", 2024-04-26T02:04:15.930 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.930 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.930 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.930 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.930 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.930 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.930 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.930 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name prefix for any created local pool", 2024-04-26T02:04:15.930 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.930 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.930 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.931 INFO:teuthology.orchestra.run.smithi001.stdout: "subtree": { 2024-04-26T02:04:15.931 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "subtree", 2024-04-26T02:04:15.931 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.931 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.931 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.931 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "rack", 2024-04-26T02:04:15.931 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.931 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.931 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.931 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T02:04:15.931 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T02:04:15.931 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.931 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.931 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.932 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.932 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mds_autoscaler", 2024-04-26T02:04:15.932 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:15.932 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:15.932 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.932 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.932 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.932 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.932 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.932 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.932 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.932 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.932 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.932 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.933 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.933 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.933 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.933 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.933 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.933 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.933 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.933 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.933 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.933 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.933 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.933 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:15.933 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:15.933 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.934 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.934 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.934 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.934 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.934 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.934 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.934 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.934 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.934 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.934 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.934 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:15.934 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:15.934 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.934 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.935 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.935 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:15.935 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.935 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.935 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.935 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.935 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.935 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.935 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.935 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.935 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.935 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.935 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.935 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.936 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.937 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.937 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.937 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirroring", 2024-04-26T02:04:15.937 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:15.937 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:15.937 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.937 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.937 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.937 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.937 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.937 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.937 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.937 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.937 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.938 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.938 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.938 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.938 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.938 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.938 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.938 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.938 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.938 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.938 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.938 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.938 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.938 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:15.939 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:15.939 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.939 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.939 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.939 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.939 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.939 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.939 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.939 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.939 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.939 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.939 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.939 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:15.940 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:15.940 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.940 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.940 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.940 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:15.940 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.940 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.940 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.940 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.940 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.940 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.940 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.940 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.940 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.940 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.941 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.941 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.941 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.941 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.941 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:15.941 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:15.941 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.941 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.941 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.941 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.941 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.941 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.942 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.942 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.942 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.942 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.942 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.942 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.942 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.942 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "nfs", 2024-04-26T02:04:15.942 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:15.942 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:15.942 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.942 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.943 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.943 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.943 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.943 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.943 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.943 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.943 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.943 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.943 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.943 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.943 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.943 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.943 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.943 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.944 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.944 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.944 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.944 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.944 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.944 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:15.944 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:15.944 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.944 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.944 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.944 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.944 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.944 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.945 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.945 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.945 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.945 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.945 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.945 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:15.945 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:15.945 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.945 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.945 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.945 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:15.945 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.945 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.946 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.946 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.946 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.946 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.946 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.946 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.946 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.946 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.946 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.946 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.946 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.946 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.947 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:15.947 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:15.947 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.947 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.947 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.947 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.947 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.947 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.947 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.947 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.947 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.947 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.947 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.947 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.948 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.948 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-04-26T02:04:15.948 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:15.948 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:15.948 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.948 INFO:teuthology.orchestra.run.smithi001.stdout: "fail_fs": { 2024-04-26T02:04:15.948 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "fail_fs", 2024-04-26T02:04:15.948 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.948 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.948 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.948 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.949 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.949 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.949 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.949 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T02:04:15.949 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.949 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.949 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.949 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.949 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.949 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.949 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.949 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.950 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.950 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.950 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.950 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.950 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.950 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.950 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.950 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.950 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.950 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.950 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.950 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.950 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.950 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.951 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.951 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:15.951 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:15.951 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.951 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.951 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.951 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.951 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.951 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.951 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.951 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.951 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.951 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.952 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.952 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:15.952 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:15.952 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.952 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.952 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.952 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:15.952 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.952 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.952 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.952 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.952 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.953 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.954 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.954 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.954 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.954 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.954 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.954 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.954 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.954 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator": { 2024-04-26T02:04:15.954 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-04-26T02:04:15.954 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.954 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.954 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.955 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.955 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.955 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.955 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.955 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-04-26T02:04:15.955 INFO:teuthology.orchestra.run.smithi001.stdout: "rook", 2024-04-26T02:04:15.955 INFO:teuthology.orchestra.run.smithi001.stdout: "test_orchestrator" 2024-04-26T02:04:15.955 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.955 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Orchestrator backend", 2024-04-26T02:04:15.955 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.955 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.955 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.955 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.956 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.956 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_perf_query", 2024-04-26T02:04:15.956 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:15.956 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:15.956 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.956 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.956 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.956 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.956 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.956 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.956 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.957 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.957 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.957 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.957 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.957 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.957 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.957 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.957 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.957 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.957 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.957 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.957 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.957 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.957 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.958 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:15.958 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:15.958 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.958 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.958 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.958 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.958 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.958 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.958 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.958 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.958 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.958 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.959 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.959 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:15.959 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:15.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.959 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.959 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.959 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:15.959 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.959 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.959 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.959 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.959 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.959 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.959 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.960 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.960 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.960 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.960 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.960 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.960 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.960 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.960 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:15.960 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:15.960 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.960 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.960 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.960 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_support", 2024-04-26T02:04:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.961 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.962 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.962 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.962 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.962 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.962 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.962 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.962 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.962 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.962 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.962 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.962 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.962 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.962 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.963 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.964 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:15.965 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.966 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.966 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.966 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.966 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.966 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.966 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.966 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.966 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.966 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.966 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.966 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.966 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.967 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_autoscaler", 2024-04-26T02:04:15.967 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:15.967 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:15.967 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.967 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.967 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.967 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.967 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.967 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.967 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.967 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.967 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.968 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:15.969 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.970 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.970 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.970 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:15.970 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.970 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.970 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.970 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.970 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.970 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.970 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.970 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.970 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.970 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.970 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.972 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.972 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-04-26T02:04:15.972 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-04-26T02:04:15.972 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:15.972 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.972 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.972 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-04-26T02:04:15.972 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.972 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.972 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.972 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.972 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.972 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: "threshold": { 2024-04-26T02:04:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threshold", 2024-04-26T02:04:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-04-26T02:04:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-04-26T02:04:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1.0", 2024-04-26T02:04:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "scaling threshold", 2024-04-26T02:04:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T02:04:15.973 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.974 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.974 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.974 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.974 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.974 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "progress", 2024-04-26T02:04:15.974 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:15.974 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:15.974 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.974 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_pg_recovery_event": { 2024-04-26T02:04:15.974 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_pg_recovery_event", 2024-04-26T02:04:15.974 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.974 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.974 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T02:04:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-04-26T02:04:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-04-26T02:04:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.975 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.976 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.977 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.977 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.977 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.977 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.977 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.977 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.977 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.977 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.977 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.977 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.977 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.977 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.977 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.977 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:15.977 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:15.978 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.978 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.978 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.978 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.978 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.978 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.978 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.978 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.978 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.978 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.978 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.978 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:15.978 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:15.978 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.979 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.979 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.979 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:15.979 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.979 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.979 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.979 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.979 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.979 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.979 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.979 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.979 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.979 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.980 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.980 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.980 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.980 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.980 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:15.980 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:15.980 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.980 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.980 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.980 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.980 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.980 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: "max_completed_events": { 2024-04-26T02:04:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_completed_events", 2024-04-26T02:04:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "50", 2024-04-26T02:04:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.981 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.982 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "number of past completed events to remember", 2024-04-26T02:04:15.982 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.982 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.982 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.982 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-04-26T02:04:15.982 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-04-26T02:04:15.982 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:15.982 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.982 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.982 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-04-26T02:04:15.982 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.982 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.983 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.983 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long the module is going to sleep", 2024-04-26T02:04:15.983 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.983 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.983 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.983 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.983 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.983 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.983 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus", 2024-04-26T02:04:15.983 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:15.983 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:15.983 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: "cache": { 2024-04-26T02:04:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cache", 2024-04-26T02:04:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.984 INFO:teuthology.orchestra.run.smithi001.stdout: "exclude_perf_counters": { 2024-04-26T02:04:15.985 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "exclude_perf_counters", 2024-04-26T02:04:15.985 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.985 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.985 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.985 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:15.985 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.985 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.985 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.985 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T02:04:15.985 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T02:04:15.985 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.985 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.985 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.986 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.986 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.986 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.986 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.986 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.986 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.986 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.986 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.986 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.986 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.986 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.986 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.986 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.986 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.987 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.988 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.988 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.988 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.988 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.988 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.988 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:15.988 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:15.988 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.988 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.988 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.988 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:15.988 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.988 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.988 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.989 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.989 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:15.989 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:15.989 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:15.989 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:15.989 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:15.989 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.989 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.989 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.989 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.989 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.989 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:15.989 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:15.989 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:15.990 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.990 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.990 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:15.990 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.990 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.990 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.990 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.990 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.990 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.990 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.990 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools": { 2024-04-26T02:04:15.990 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools", 2024-04-26T02:04:15.990 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T02:04:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T02:04:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.991 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300", 2024-04-26T02:04:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_interval": { 2024-04-26T02:04:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_interval", 2024-04-26T02:04:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-04-26T02:04:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.992 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15.0", 2024-04-26T02:04:15.993 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.993 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.993 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.993 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.993 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.993 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.993 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.993 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-04-26T02:04:15.993 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-04-26T02:04:15.993 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.993 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.993 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.993 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-04-26T02:04:15.994 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.994 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.994 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.994 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T02:04:15.994 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.994 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.994 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.994 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-04-26T02:04:15.994 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-04-26T02:04:15.994 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.994 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.994 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.995 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "9283", 2024-04-26T02:04:15.995 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.995 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.995 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.995 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T02:04:15.995 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.995 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.995 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.995 INFO:teuthology.orchestra.run.smithi001.stdout: "stale_cache_strategy": { 2024-04-26T02:04:15.995 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stale_cache_strategy", 2024-04-26T02:04:15.995 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.995 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.995 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:15.996 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "log", 2024-04-26T02:04:15.996 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.996 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.996 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.996 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.996 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.996 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.996 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.996 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-04-26T02:04:15.996 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-04-26T02:04:15.996 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.996 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.996 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "default", 2024-04-26T02:04:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: "default", 2024-04-26T02:04:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: "error" 2024-04-26T02:04:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-04-26T02:04:15.997 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-04-26T02:04:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-04-26T02:04:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-04-26T02:04:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-04-26T02:04:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:15.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-04-26T02:04:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:15.999 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.000 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.001 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.002 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.003 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_snap_create": { 2024-04-26T02:04:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_snap_create", 2024-04-26T02:04:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.004 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: "mirror_snapshot_schedule": { 2024-04-26T02:04:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T02:04:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.006 INFO:teuthology.orchestra.run.smithi001.stdout: "trash_purge_schedule": { 2024-04-26T02:04:16.006 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "trash_purge_schedule", 2024-04-26T02:04:16.006 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.006 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.006 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.006 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.006 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.006 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.006 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.006 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.006 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "restful", 2024-04-26T02:04:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_auth": { 2024-04-26T02:04:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_auth", 2024-04-26T02:04:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-04-26T02:04:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-04-26T02:04:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.008 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.009 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.010 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.011 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.012 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.013 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-04-26T02:04:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-04-26T02:04:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-04-26T02:04:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-04-26T02:04:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.015 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rgw", 2024-04-26T02:04:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.016 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.017 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.018 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.019 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.020 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rook", 2024-04-26T02:04:16.021 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.022 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.023 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.024 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.025 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.026 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: "storage_class": { 2024-04-26T02:04:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "storage_class", 2024-04-26T02:04:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "local", 2024-04-26T02:04:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T02:04:16.027 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "selftest", 2024-04-26T02:04:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.028 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.029 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.030 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.031 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.032 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: "roption1": { 2024-04-26T02:04:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption1", 2024-04-26T02:04:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.033 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: "roption2": { 2024-04-26T02:04:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption2", 2024-04-26T02:04:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "xyz", 2024-04-26T02:04:16.034 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption1": { 2024-04-26T02:04:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption1", 2024-04-26T02:04:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption2": { 2024-04-26T02:04:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption2", 2024-04-26T02:04:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.036 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption3": { 2024-04-26T02:04:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption3", 2024-04-26T02:04:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-04-26T02:04:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.037 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption4": { 2024-04-26T02:04:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption4", 2024-04-26T02:04:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.038 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption5": { 2024-04-26T02:04:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption5", 2024-04-26T02:04:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.039 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption6": { 2024-04-26T02:04:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption6", 2024-04-26T02:04:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.040 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption7": { 2024-04-26T02:04:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption7", 2024-04-26T02:04:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-04-26T02:04:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "42", 2024-04-26T02:04:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: "testkey": { 2024-04-26T02:04:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testkey", 2024-04-26T02:04:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.042 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: "testlkey": { 2024-04-26T02:04:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testlkey", 2024-04-26T02:04:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.043 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: "testnewline": { 2024-04-26T02:04:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testnewline", 2024-04-26T02:04:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.044 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-04-26T02:04:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_m_granularity": { 2024-04-26T02:04:16.045 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_m_granularity", 2024-04-26T02:04:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T02:04:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: "dump_on_update": { 2024-04-26T02:04:16.046 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dump_on_update", 2024-04-26T02:04:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "dump database to debug log on update", 2024-04-26T02:04:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.047 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.048 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.049 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.050 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.051 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.052 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stats", 2024-04-26T02:04:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.053 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.054 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.055 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.056 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.057 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "status", 2024-04-26T02:04:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.058 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.059 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.061 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.062 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.063 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telegraf", 2024-04-26T02:04:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: "address": { 2024-04-26T02:04:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "address", 2024-04-26T02:04:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T02:04:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.064 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-04-26T02:04:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-04-26T02:04:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15", 2024-04-26T02:04:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.065 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.066 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.067 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.068 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.069 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.070 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telemetry", 2024-04-26T02:04:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_basic": { 2024-04-26T02:04:16.071 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_basic", 2024-04-26T02:04:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T02:04:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_crash": { 2024-04-26T02:04:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_crash", 2024-04-26T02:04:16.072 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T02:04:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_device": { 2024-04-26T02:04:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_device", 2024-04-26T02:04:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.073 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T02:04:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_ident": { 2024-04-26T02:04:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_ident", 2024-04-26T02:04:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.074 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T02:04:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_perf": { 2024-04-26T02:04:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_perf", 2024-04-26T02:04:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.075 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T02:04:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: "contact": { 2024-04-26T02:04:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "contact", 2024-04-26T02:04:16.076 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: "description": { 2024-04-26T02:04:16.077 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "description", 2024-04-26T02:04:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.078 INFO:teuthology.orchestra.run.smithi001.stdout: "device_url": { 2024-04-26T02:04:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_url", 2024-04-26T02:04:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T02:04:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-04-26T02:04:16.079 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-04-26T02:04:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-04-26T02:04:16.080 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-04-26T02:04:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "24", 2024-04-26T02:04:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "8", 2024-04-26T02:04:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: "last_opt_revision": { 2024-04-26T02:04:16.081 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "last_opt_revision", 2024-04-26T02:04:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1", 2024-04-26T02:04:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard": { 2024-04-26T02:04:16.082 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard", 2024-04-26T02:04:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard_description": { 2024-04-26T02:04:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard_description", 2024-04-26T02:04:16.083 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.084 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.085 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.085 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.085 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.085 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.085 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.085 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.085 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.085 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.085 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.085 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.085 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.085 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.085 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.085 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.085 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.086 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.086 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.086 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.086 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.086 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.086 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.086 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.086 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.086 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.086 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.086 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.086 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.086 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.087 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.087 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.087 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.087 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.087 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.087 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.087 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.087 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.087 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.087 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.087 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.087 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.087 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.088 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.088 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.088 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.088 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.088 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.088 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.088 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.088 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.088 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.088 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.088 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.088 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.089 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.089 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.089 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.089 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.089 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.089 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.089 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.089 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.089 INFO:teuthology.orchestra.run.smithi001.stdout: "organization": { 2024-04-26T02:04:16.089 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "organization", 2024-04-26T02:04:16.089 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.089 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.089 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.090 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.090 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.090 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.090 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.090 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.090 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.090 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.090 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.090 INFO:teuthology.orchestra.run.smithi001.stdout: "proxy": { 2024-04-26T02:04:16.090 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "proxy", 2024-04-26T02:04:16.090 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.090 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.090 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.091 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.091 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.091 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.091 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.091 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.091 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.091 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.091 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.091 INFO:teuthology.orchestra.run.smithi001.stdout: "url": { 2024-04-26T02:04:16.091 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url", 2024-04-26T02:04:16.091 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.091 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.091 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.092 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T02:04:16.092 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.092 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.092 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.092 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.092 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.092 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.092 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.092 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.092 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.092 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "test_orchestrator", 2024-04-26T02:04:16.092 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.092 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.093 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.093 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.093 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.093 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.093 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.093 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.093 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.093 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.093 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.093 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.093 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.093 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.093 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.093 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.093 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.094 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.094 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.094 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.094 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.094 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.094 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.094 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.094 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.094 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.094 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.094 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.094 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.094 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.094 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.095 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.095 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.095 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.095 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.095 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.095 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.095 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.095 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.095 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.095 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.095 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.095 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.095 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.096 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.096 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.096 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.096 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.096 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.096 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.096 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.096 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.096 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.096 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.096 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.096 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.096 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.096 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.096 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-04-26T02:04:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.097 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.098 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.098 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.098 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.098 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.098 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.098 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.098 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.098 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.098 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.098 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.098 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.098 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.098 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.098 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.099 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.099 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.099 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.099 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.099 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.099 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.099 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.099 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.099 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.099 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.099 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.099 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.099 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.099 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.100 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.100 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.100 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.100 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.100 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.100 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.100 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.100 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.100 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.100 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.100 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.100 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.100 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.101 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.101 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.101 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.101 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.101 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.101 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.101 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.101 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.101 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.101 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.101 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.101 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.101 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.101 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.102 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.102 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.102 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.102 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.102 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.102 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.102 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.102 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.102 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.102 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.102 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.102 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.102 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_clones": { 2024-04-26T02:04:16.103 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_clones", 2024-04-26T02:04:16.103 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.103 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.103 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.103 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4", 2024-04-26T02:04:16.103 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.103 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.103 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.103 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T02:04:16.103 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.103 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.103 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.103 INFO:teuthology.orchestra.run.smithi001.stdout: "periodic_async_work": { 2024-04-26T02:04:16.103 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "periodic_async_work", 2024-04-26T02:04:16.104 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.104 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.104 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.104 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.104 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.104 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.104 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.104 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Periodically check for async work", 2024-04-26T02:04:16.104 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.104 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.104 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.104 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_delay": { 2024-04-26T02:04:16.104 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_delay", 2024-04-26T02:04:16.104 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.105 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.105 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.105 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-04-26T02:04:16.105 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.105 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.105 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.105 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T02:04:16.105 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.105 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.105 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.105 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_no_wait": { 2024-04-26T02:04:16.105 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T02:04:16.105 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.105 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.106 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.106 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.106 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.106 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.106 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.106 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T02:04:16.106 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.106 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.106 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.106 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.106 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.107 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix", 2024-04-26T02:04:16.107 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.107 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.107 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.107 INFO:teuthology.orchestra.run.smithi001.stdout: "discovery_interval": { 2024-04-26T02:04:16.107 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "discovery_interval", 2024-04-26T02:04:16.107 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-04-26T02:04:16.107 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.107 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.107 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "100", 2024-04-26T02:04:16.107 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.107 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.107 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.107 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.107 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.108 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.108 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.108 INFO:teuthology.orchestra.run.smithi001.stdout: "identifier": { 2024-04-26T02:04:16.108 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "identifier", 2024-04-26T02:04:16.108 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.108 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.108 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.108 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.108 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.108 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.108 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.108 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.108 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.109 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.109 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.109 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-04-26T02:04:16.109 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-04-26T02:04:16.109 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.109 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.109 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.109 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-04-26T02:04:16.109 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.109 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.109 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.109 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.109 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.110 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.110 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.110 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.110 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.110 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.110 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.110 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.110 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.110 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.110 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.110 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.110 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.110 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.110 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.111 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.111 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.111 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.111 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.111 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.111 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.111 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.111 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.111 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.111 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.111 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.111 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.111 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.112 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.112 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.112 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.112 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.112 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.112 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.112 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.112 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.112 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.112 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.112 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.112 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.112 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.113 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.113 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.113 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.113 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.113 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.113 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.113 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.113 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.113 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.113 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.113 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.113 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.113 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.113 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.113 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.114 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.114 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.114 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.114 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.114 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.114 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.114 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.114 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.114 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.114 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.114 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.114 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.114 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.115 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_host": { 2024-04-26T02:04:16.115 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_host", 2024-04-26T02:04:16.115 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.115 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.115 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.115 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.115 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.115 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.115 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.115 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.115 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.115 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.115 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.116 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_port": { 2024-04-26T02:04:16.116 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_port", 2024-04-26T02:04:16.116 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.116 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.116 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.116 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10051", 2024-04-26T02:04:16.116 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.116 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.116 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.116 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.116 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.116 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.117 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.117 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_sender": { 2024-04-26T02:04:16.117 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_sender", 2024-04-26T02:04:16.117 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.117 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.117 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.117 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T02:04:16.117 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.117 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.117 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.117 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.117 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.117 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.117 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.118 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.118 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.118 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.118 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-04-26T02:04:16.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.118 INFO:teuthology.orchestra.run.smithi001.stdout: "gid": 25084, 2024-04-26T02:04:16.118 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "z", 2024-04-26T02:04:16.118 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_features": 4540701547738038271, 2024-04-26T02:04:16.118 INFO:teuthology.orchestra.run.smithi001.stdout: "available_modules": [ 2024-04-26T02:04:16.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.118 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alerts", 2024-04-26T02:04:16.118 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.118 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.119 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.119 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-04-26T02:04:16.119 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-04-26T02:04:16.119 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.119 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.119 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.119 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-04-26T02:04:16.119 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.119 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.119 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.119 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T02:04:16.119 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.119 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.119 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.120 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.120 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.120 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.120 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.120 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.120 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.120 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.120 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.120 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.120 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.120 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.120 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.120 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.120 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.121 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.121 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.121 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.121 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.121 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.121 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.121 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.121 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.121 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.121 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.121 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.121 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.121 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.121 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.122 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.122 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.122 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.122 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.122 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.122 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.122 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.122 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.122 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.122 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.122 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.122 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.122 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.123 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.123 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.123 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.123 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.123 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.123 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.123 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.123 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.123 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.123 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.123 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.123 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.123 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.123 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.124 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.124 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.124 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.124 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.124 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.124 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.124 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.124 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.124 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.124 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.124 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.124 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_destination": { 2024-04-26T02:04:16.124 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_destination", 2024-04-26T02:04:16.124 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.125 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.125 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.125 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.125 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.125 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.125 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.125 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email address to send alerts to", 2024-04-26T02:04:16.125 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.125 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.125 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.125 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_from_name": { 2024-04-26T02:04:16.125 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_from_name", 2024-04-26T02:04:16.125 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.125 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.126 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.126 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "Ceph", 2024-04-26T02:04:16.126 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.126 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.126 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.126 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email From: name", 2024-04-26T02:04:16.126 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.126 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.126 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.126 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_host": { 2024-04-26T02:04:16.126 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_host", 2024-04-26T02:04:16.126 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.126 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.127 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.127 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.127 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.127 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.127 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.127 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP server", 2024-04-26T02:04:16.127 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.127 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.127 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.127 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_password": { 2024-04-26T02:04:16.127 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_password", 2024-04-26T02:04:16.127 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.127 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.127 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.127 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.128 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.128 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.128 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.128 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Password to authenticate with", 2024-04-26T02:04:16.128 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.128 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.128 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.128 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_port": { 2024-04-26T02:04:16.128 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_port", 2024-04-26T02:04:16.128 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.128 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.128 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.128 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "465", 2024-04-26T02:04:16.129 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.129 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.129 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.129 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP port", 2024-04-26T02:04:16.129 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.129 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.129 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.129 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_sender": { 2024-04-26T02:04:16.129 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_sender", 2024-04-26T02:04:16.129 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.129 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.129 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.129 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.129 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.129 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.130 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.130 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP envelope sender", 2024-04-26T02:04:16.130 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.130 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.130 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.130 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_ssl": { 2024-04-26T02:04:16.130 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_ssl", 2024-04-26T02:04:16.130 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.130 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.130 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.130 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.130 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.130 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.131 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.131 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T02:04:16.131 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.131 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.131 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.131 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_user": { 2024-04-26T02:04:16.131 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_user", 2024-04-26T02:04:16.131 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.131 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.131 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.131 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.131 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.131 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.131 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.131 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "User to authenticate as", 2024-04-26T02:04:16.132 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.132 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.132 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.132 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.132 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.132 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "balancer", 2024-04-26T02:04:16.132 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.132 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.132 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.132 INFO:teuthology.orchestra.run.smithi001.stdout: "active": { 2024-04-26T02:04:16.132 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "active", 2024-04-26T02:04:16.132 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.132 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.133 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.133 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.133 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.133 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.133 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.133 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T02:04:16.133 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.133 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.133 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.133 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_time": { 2024-04-26T02:04:16.133 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_time", 2024-04-26T02:04:16.133 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.134 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.134 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.134 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0000", 2024-04-26T02:04:16.134 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.134 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.134 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.134 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T02:04:16.134 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T02:04:16.134 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.134 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.134 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_weekday": { 2024-04-26T02:04:16.134 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_weekday", 2024-04-26T02:04:16.135 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-04-26T02:04:16.135 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.135 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.135 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-04-26T02:04:16.135 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-04-26T02:04:16.135 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-04-26T02:04:16.135 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.135 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T02:04:16.135 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T02:04:16.135 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.135 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.135 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_max_iterations": { 2024-04-26T02:04:16.135 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_max_iterations", 2024-04-26T02:04:16.135 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-04-26T02:04:16.136 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.136 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.136 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "25", 2024-04-26T02:04:16.136 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-04-26T02:04:16.136 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "250", 2024-04-26T02:04:16.136 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.136 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T02:04:16.136 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.136 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.136 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.136 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_metrics": { 2024-04-26T02:04:16.136 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_metrics", 2024-04-26T02:04:16.136 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.137 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.137 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.137 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T02:04:16.137 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.137 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.137 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.137 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T02:04:16.137 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T02:04:16.137 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.137 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.137 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_step": { 2024-04-26T02:04:16.137 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_step", 2024-04-26T02:04:16.137 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-04-26T02:04:16.137 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.138 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.138 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.5", 2024-04-26T02:04:16.138 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0.001", 2024-04-26T02:04:16.138 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "0.999", 2024-04-26T02:04:16.138 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.138 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "aggressiveness of optimization", 2024-04-26T02:04:16.138 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T02:04:16.138 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.138 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.138 INFO:teuthology.orchestra.run.smithi001.stdout: "end_time": { 2024-04-26T02:04:16.138 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_time", 2024-04-26T02:04:16.138 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.138 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.139 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.139 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2359", 2024-04-26T02:04:16.139 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.139 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.139 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.139 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T02:04:16.139 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T02:04:16.139 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.139 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.139 INFO:teuthology.orchestra.run.smithi001.stdout: "end_weekday": { 2024-04-26T02:04:16.139 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_weekday", 2024-04-26T02:04:16.139 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-04-26T02:04:16.139 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.139 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.140 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-04-26T02:04:16.140 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-04-26T02:04:16.140 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-04-26T02:04:16.140 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.140 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T02:04:16.140 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T02:04:16.140 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.140 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.140 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.140 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.140 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.140 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.141 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.141 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.141 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.141 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.141 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.141 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.141 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.141 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.141 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.141 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.141 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.141 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.141 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.141 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.141 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.142 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.142 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.142 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.142 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.142 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.142 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.142 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.142 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.142 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.142 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.142 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.142 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.142 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.143 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.143 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.143 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.143 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.143 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.143 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.143 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.143 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.143 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.143 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.143 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.143 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.143 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.143 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.144 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.144 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.144 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.144 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.144 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.144 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.144 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.144 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.144 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.144 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.144 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.144 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.144 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.144 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.145 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.145 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.145 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.145 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.145 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.145 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.145 INFO:teuthology.orchestra.run.smithi001.stdout: "min_score": { 2024-04-26T02:04:16.145 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_score", 2024-04-26T02:04:16.145 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-04-26T02:04:16.145 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.145 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.145 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-04-26T02:04:16.145 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.146 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.146 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.146 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T02:04:16.146 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.146 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.146 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.146 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-04-26T02:04:16.146 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-04-26T02:04:16.146 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.146 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.146 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.146 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "upmap", 2024-04-26T02:04:16.146 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.147 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.147 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.147 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-compat", 2024-04-26T02:04:16.147 INFO:teuthology.orchestra.run.smithi001.stdout: "none", 2024-04-26T02:04:16.147 INFO:teuthology.orchestra.run.smithi001.stdout: "read", 2024-04-26T02:04:16.147 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap", 2024-04-26T02:04:16.147 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap-read" 2024-04-26T02:04:16.147 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.147 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Balancer mode", 2024-04-26T02:04:16.147 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.147 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.147 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.147 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_ids": { 2024-04-26T02:04:16.148 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_ids", 2024-04-26T02:04:16.148 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.148 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.148 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.148 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.148 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.148 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.148 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.148 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T02:04:16.148 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.148 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.148 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.148 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-04-26T02:04:16.149 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-04-26T02:04:16.149 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.149 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.149 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.149 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-04-26T02:04:16.149 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.149 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.149 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.149 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T02:04:16.149 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.149 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.149 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.149 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_deviation": { 2024-04-26T02:04:16.150 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_deviation", 2024-04-26T02:04:16.150 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.150 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.150 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.150 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-04-26T02:04:16.150 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-04-26T02:04:16.150 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.150 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.150 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T02:04:16.150 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T02:04:16.150 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.150 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.151 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_optimizations": { 2024-04-26T02:04:16.151 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_optimizations", 2024-04-26T02:04:16.151 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-04-26T02:04:16.151 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.151 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.151 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:16.151 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.151 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.151 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.151 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T02:04:16.151 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.151 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.151 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.151 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.151 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.152 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm", 2024-04-26T02:04:16.152 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.152 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.152 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.152 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_down_multiplier": { 2024-04-26T02:04:16.152 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_down_multiplier", 2024-04-26T02:04:16.152 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-04-26T02:04:16.152 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.152 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.152 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-04-26T02:04:16.152 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.152 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.152 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.153 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T02:04:16.153 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.153 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.153 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.153 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_refresh_rate": { 2024-04-26T02:04:16.153 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_refresh_rate", 2024-04-26T02:04:16.153 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.153 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.153 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.153 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "20", 2024-04-26T02:04:16.153 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.153 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.153 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.154 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T02:04:16.154 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.154 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.154 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.154 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_starting_port": { 2024-04-26T02:04:16.154 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_starting_port", 2024-04-26T02:04:16.154 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.154 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.154 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.154 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4721", 2024-04-26T02:04:16.154 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.154 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.154 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.155 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T02:04:16.155 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.155 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.155 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.155 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_ptrace": { 2024-04-26T02:04:16.155 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_ptrace", 2024-04-26T02:04:16.155 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.155 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.155 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.155 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.155 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.155 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.155 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.156 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T02:04:16.156 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T02:04:16.156 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.156 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.156 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_interval": { 2024-04-26T02:04:16.156 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_interval", 2024-04-26T02:04:16.156 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.156 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.156 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.156 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-04-26T02:04:16.156 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.156 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.156 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.157 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T02:04:16.157 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.157 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.157 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.157 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_memory_target_ratio": { 2024-04-26T02:04:16.157 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T02:04:16.157 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-04-26T02:04:16.157 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.157 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.157 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.7", 2024-04-26T02:04:16.157 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.157 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.157 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.158 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T02:04:16.158 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.158 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.158 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.158 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm_log_destination": { 2024-04-26T02:04:16.158 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm_log_destination", 2024-04-26T02:04:16.158 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.158 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.158 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.158 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.158 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.158 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.158 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.159 INFO:teuthology.orchestra.run.smithi001.stdout: "file", 2024-04-26T02:04:16.159 INFO:teuthology.orchestra.run.smithi001.stdout: "file,syslog", 2024-04-26T02:04:16.159 INFO:teuthology.orchestra.run.smithi001.stdout: "syslog" 2024-04-26T02:04:16.159 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.159 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T02:04:16.159 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.159 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.159 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.159 INFO:teuthology.orchestra.run.smithi001.stdout: "cgroups_split": { 2024-04-26T02:04:16.159 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cgroups_split", 2024-04-26T02:04:16.159 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.159 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.159 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.160 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.160 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.160 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.160 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.160 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T02:04:16.160 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.160 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.160 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.160 INFO:teuthology.orchestra.run.smithi001.stdout: "config_checks_enabled": { 2024-04-26T02:04:16.160 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_checks_enabled", 2024-04-26T02:04:16.160 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.160 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.160 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.161 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.161 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.161 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.161 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.161 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T02:04:16.161 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.161 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.161 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.161 INFO:teuthology.orchestra.run.smithi001.stdout: "config_dashboard": { 2024-04-26T02:04:16.161 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_dashboard", 2024-04-26T02:04:16.161 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.161 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.161 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.162 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.162 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.162 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.162 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.162 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T02:04:16.162 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.162 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.162 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.162 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_alertmanager": { 2024-04-26T02:04:16.162 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_alertmanager", 2024-04-26T02:04:16.162 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.162 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.163 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.163 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T02:04:16.163 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.163 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.163 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.163 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-04-26T02:04:16.163 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.163 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.163 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.163 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_base": { 2024-04-26T02:04:16.163 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_base", 2024-04-26T02:04:16.163 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.163 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.164 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.164 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T02:04:16.164 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.164 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.164 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.164 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Container image name, without the tag", 2024-04-26T02:04:16.164 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.164 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.164 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.164 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_elasticsearch": { 2024-04-26T02:04:16.164 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_elasticsearch", 2024-04-26T02:04:16.164 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.164 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.165 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.165 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T02:04:16.165 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.165 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.165 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.165 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "elasticsearch container image", 2024-04-26T02:04:16.165 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.165 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.165 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.165 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_grafana": { 2024-04-26T02:04:16.165 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_grafana", 2024-04-26T02:04:16.165 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.166 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.166 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.166 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T02:04:16.166 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.166 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.166 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.166 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-04-26T02:04:16.166 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.166 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.166 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.166 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_haproxy": { 2024-04-26T02:04:16.166 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_haproxy", 2024-04-26T02:04:16.166 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.167 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.167 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.167 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T02:04:16.167 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.167 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.167 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.167 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "HAproxy container image", 2024-04-26T02:04:16.167 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.167 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.167 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.167 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_agent": { 2024-04-26T02:04:16.167 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_agent", 2024-04-26T02:04:16.167 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.168 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.168 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.168 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T02:04:16.168 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.168 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.168 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.168 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger agent container image", 2024-04-26T02:04:16.168 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.168 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.168 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.168 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_collector": { 2024-04-26T02:04:16.168 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_collector", 2024-04-26T02:04:16.168 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.168 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.169 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.169 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T02:04:16.169 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.169 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.169 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.169 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger collector container image", 2024-04-26T02:04:16.169 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.169 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.169 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.169 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_query": { 2024-04-26T02:04:16.169 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_query", 2024-04-26T02:04:16.169 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.170 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.170 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.170 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T02:04:16.170 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.170 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.170 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.170 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger query container image", 2024-04-26T02:04:16.170 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.170 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.170 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.170 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_keepalived": { 2024-04-26T02:04:16.170 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_keepalived", 2024-04-26T02:04:16.171 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.171 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.171 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.171 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T02:04:16.171 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.171 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.171 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.171 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Keepalived container image", 2024-04-26T02:04:16.171 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.171 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.171 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.171 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_loki": { 2024-04-26T02:04:16.171 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_loki", 2024-04-26T02:04:16.171 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.172 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.172 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.172 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T02:04:16.172 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.172 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.172 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.172 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Loki container image", 2024-04-26T02:04:16.172 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.172 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.172 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.172 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_node_exporter": { 2024-04-26T02:04:16.172 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_node_exporter", 2024-04-26T02:04:16.172 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.172 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.173 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.173 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T02:04:16.173 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.173 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.173 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.173 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-04-26T02:04:16.173 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.173 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.173 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.173 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_nvmeof": { 2024-04-26T02:04:16.173 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_nvmeof", 2024-04-26T02:04:16.173 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.174 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.174 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.174 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T02:04:16.174 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.174 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.174 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.174 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Nvme-of container image", 2024-04-26T02:04:16.174 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.174 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.174 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.174 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_prometheus": { 2024-04-26T02:04:16.174 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_prometheus", 2024-04-26T02:04:16.174 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.175 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.175 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.175 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T02:04:16.175 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.175 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.175 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.175 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-04-26T02:04:16.175 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.175 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.175 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.175 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_promtail": { 2024-04-26T02:04:16.175 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_promtail", 2024-04-26T02:04:16.175 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.176 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.176 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.176 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T02:04:16.176 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.176 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.176 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.176 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Promtail container image", 2024-04-26T02:04:16.176 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.176 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.176 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.176 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_samba": { 2024-04-26T02:04:16.176 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_samba", 2024-04-26T02:04:16.176 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.177 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.177 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.177 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T02:04:16.177 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.177 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.177 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.177 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Samba/SMB container image", 2024-04-26T02:04:16.177 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.177 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.177 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.177 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_snmp_gateway": { 2024-04-26T02:04:16.177 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_snmp_gateway", 2024-04-26T02:04:16.177 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.178 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.178 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.178 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T02:04:16.178 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.178 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.178 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.178 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SNMP Gateway container image", 2024-04-26T02:04:16.178 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.178 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.178 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.178 INFO:teuthology.orchestra.run.smithi001.stdout: "container_init": { 2024-04-26T02:04:16.178 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_init", 2024-04-26T02:04:16.179 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.179 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.179 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.179 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.179 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.179 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.179 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.179 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T02:04:16.179 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.179 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.179 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.179 INFO:teuthology.orchestra.run.smithi001.stdout: "daemon_cache_timeout": { 2024-04-26T02:04:16.179 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "daemon_cache_timeout", 2024-04-26T02:04:16.180 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.180 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.180 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.180 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-04-26T02:04:16.180 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.180 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.180 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.180 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T02:04:16.180 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.180 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.180 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.180 INFO:teuthology.orchestra.run.smithi001.stdout: "default_cephadm_command_timeout": { 2024-04-26T02:04:16.180 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T02:04:16.181 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.181 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.181 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.181 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "900", 2024-04-26T02:04:16.181 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.181 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.181 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.181 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T02:04:16.181 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.181 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.181 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.181 INFO:teuthology.orchestra.run.smithi001.stdout: "default_registry": { 2024-04-26T02:04:16.181 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_registry", 2024-04-26T02:04:16.182 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.182 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.182 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.182 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io", 2024-04-26T02:04:16.182 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.182 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.182 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.182 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T02:04:16.182 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.182 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.182 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.182 INFO:teuthology.orchestra.run.smithi001.stdout: "device_cache_timeout": { 2024-04-26T02:04:16.182 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_cache_timeout", 2024-04-26T02:04:16.183 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.183 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.183 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.183 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1800", 2024-04-26T02:04:16.183 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.183 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.183 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.183 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache device inventory", 2024-04-26T02:04:16.183 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.183 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.183 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.183 INFO:teuthology.orchestra.run.smithi001.stdout: "device_enhanced_scan": { 2024-04-26T02:04:16.183 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_enhanced_scan", 2024-04-26T02:04:16.183 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.184 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.184 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.184 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.184 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.184 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.184 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.184 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T02:04:16.184 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.184 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.184 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.184 INFO:teuthology.orchestra.run.smithi001.stdout: "facts_cache_timeout": { 2024-04-26T02:04:16.184 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "facts_cache_timeout", 2024-04-26T02:04:16.184 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.184 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.185 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.185 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-04-26T02:04:16.185 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.185 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.185 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.185 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache host facts data", 2024-04-26T02:04:16.185 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.185 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.185 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.185 INFO:teuthology.orchestra.run.smithi001.stdout: "grafana_dashboards_path": { 2024-04-26T02:04:16.185 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "grafana_dashboards_path", 2024-04-26T02:04:16.185 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.185 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.185 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.186 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T02:04:16.186 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.186 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.186 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.186 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T02:04:16.186 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.186 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.186 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.186 INFO:teuthology.orchestra.run.smithi001.stdout: "host_check_interval": { 2024-04-26T02:04:16.186 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "host_check_interval", 2024-04-26T02:04:16.186 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.186 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.186 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.187 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-04-26T02:04:16.187 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.187 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.187 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.187 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to perform a host check", 2024-04-26T02:04:16.187 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.187 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.187 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.187 INFO:teuthology.orchestra.run.smithi001.stdout: "hw_monitoring": { 2024-04-26T02:04:16.187 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hw_monitoring", 2024-04-26T02:04:16.187 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.187 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.187 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.188 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.188 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.188 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.188 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.188 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T02:04:16.188 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.188 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.188 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.188 INFO:teuthology.orchestra.run.smithi001.stdout: "inventory_list_all": { 2024-04-26T02:04:16.188 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "inventory_list_all", 2024-04-26T02:04:16.188 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.188 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.188 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.188 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.189 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.189 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.189 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.189 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T02:04:16.189 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.189 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.189 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.189 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.189 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.189 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.189 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.189 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.189 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.190 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.190 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.190 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.190 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.190 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.190 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.190 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.190 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.190 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.190 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.190 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.190 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.190 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.191 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.191 INFO:teuthology.orchestra.run.smithi001.stdout: "log_refresh_metadata": { 2024-04-26T02:04:16.191 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_refresh_metadata", 2024-04-26T02:04:16.191 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.191 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.191 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.191 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.191 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.191 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.191 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.191 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T02:04:16.191 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.191 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.192 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.192 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.192 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.192 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.192 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.192 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.192 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.192 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.192 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.192 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.192 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T02:04:16.192 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.193 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.193 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.193 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.193 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.193 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.193 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.193 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.193 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.193 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.193 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.193 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.193 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.193 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.193 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.194 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.194 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.194 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.194 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.194 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.194 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.194 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.194 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.194 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.194 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.194 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.194 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.194 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.195 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.195 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.195 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.195 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.195 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.195 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.195 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.195 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.195 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T02:04:16.195 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T02:04:16.195 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.195 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.195 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.196 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.196 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.196 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.196 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.196 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T02:04:16.196 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.196 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.196 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.196 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T02:04:16.196 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T02:04:16.196 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.196 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.196 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.196 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "*", 2024-04-26T02:04:16.197 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.197 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.197 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.197 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T02:04:16.197 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.197 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.197 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.197 INFO:teuthology.orchestra.run.smithi001.stdout: "max_count_per_host": { 2024-04-26T02:04:16.197 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_count_per_host", 2024-04-26T02:04:16.197 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.197 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.197 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.197 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:16.197 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.197 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.198 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.198 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of daemons per service per host", 2024-04-26T02:04:16.198 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.198 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.198 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.198 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd_draining_count": { 2024-04-26T02:04:16.198 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_osd_draining_count", 2024-04-26T02:04:16.198 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.198 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.198 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.198 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:16.198 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.198 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.198 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.199 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T02:04:16.199 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.199 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.199 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.199 INFO:teuthology.orchestra.run.smithi001.stdout: "migration_current": { 2024-04-26T02:04:16.199 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "migration_current", 2024-04-26T02:04:16.199 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.199 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.199 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.199 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.199 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.199 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.199 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.199 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "internal - do not modify", 2024-04-26T02:04:16.199 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.200 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.200 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.200 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-04-26T02:04:16.200 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-04-26T02:04:16.200 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.200 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.200 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.200 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "root", 2024-04-26T02:04:16.200 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.200 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.200 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.200 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm-package", 2024-04-26T02:04:16.200 INFO:teuthology.orchestra.run.smithi001.stdout: "root" 2024-04-26T02:04:16.200 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.201 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T02:04:16.201 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.201 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.201 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.201 INFO:teuthology.orchestra.run.smithi001.stdout: "oob_default_addr": { 2024-04-26T02:04:16.201 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "oob_default_addr", 2024-04-26T02:04:16.201 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.201 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.201 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.201 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "169.254.1.1", 2024-04-26T02:04:16.201 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.201 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.201 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.202 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T02:04:16.202 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.202 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.202 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.202 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_alerts_path": { 2024-04-26T02:04:16.202 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_alerts_path", 2024-04-26T02:04:16.202 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.202 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.202 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.202 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T02:04:16.202 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.202 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.202 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.203 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T02:04:16.203 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.203 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.203 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.203 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_insecure": { 2024-04-26T02:04:16.203 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_insecure", 2024-04-26T02:04:16.203 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.203 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.203 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.203 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.203 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.203 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.203 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.204 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T02:04:16.204 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.204 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.204 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.204 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_password": { 2024-04-26T02:04:16.204 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_password", 2024-04-26T02:04:16.204 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.204 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.204 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.204 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.204 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.204 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.204 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.205 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T02:04:16.205 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.205 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.205 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.205 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_url": { 2024-04-26T02:04:16.205 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_url", 2024-04-26T02:04:16.205 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.205 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.205 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.205 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.205 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.205 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.206 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.206 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T02:04:16.206 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.206 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.206 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.206 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_username": { 2024-04-26T02:04:16.206 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_username", 2024-04-26T02:04:16.206 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.206 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.206 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.206 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.206 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.206 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.206 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.207 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T02:04:16.207 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.207 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.207 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.207 INFO:teuthology.orchestra.run.smithi001.stdout: "secure_monitoring_stack": { 2024-04-26T02:04:16.207 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secure_monitoring_stack", 2024-04-26T02:04:16.207 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.207 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.207 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.207 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.207 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.207 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.207 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.208 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T02:04:16.208 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.208 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.208 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.208 INFO:teuthology.orchestra.run.smithi001.stdout: "service_discovery_port": { 2024-04-26T02:04:16.208 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "service_discovery_port", 2024-04-26T02:04:16.208 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.208 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.208 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.208 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8765", 2024-04-26T02:04:16.208 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.208 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.208 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.208 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "cephadm service discovery port", 2024-04-26T02:04:16.209 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.209 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.209 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.209 INFO:teuthology.orchestra.run.smithi001.stdout: "ssh_config_file": { 2024-04-26T02:04:16.209 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssh_config_file", 2024-04-26T02:04:16.209 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.209 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.209 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.209 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.209 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.209 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.209 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.210 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T02:04:16.210 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.210 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.210 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.210 INFO:teuthology.orchestra.run.smithi001.stdout: "use_agent": { 2024-04-26T02:04:16.210 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_agent", 2024-04-26T02:04:16.210 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.210 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.210 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.210 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.210 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.210 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.210 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.211 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T02:04:16.211 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.211 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.211 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.211 INFO:teuthology.orchestra.run.smithi001.stdout: "use_repo_digest": { 2024-04-26T02:04:16.211 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_repo_digest", 2024-04-26T02:04:16.211 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.211 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.211 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.211 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.211 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.211 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.211 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.211 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T02:04:16.212 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.212 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.212 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.212 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_failed_host_check": { 2024-04-26T02:04:16.212 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_failed_host_check", 2024-04-26T02:04:16.212 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.212 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.212 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.212 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.212 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.212 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.212 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.212 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T02:04:16.213 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.213 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.213 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.213 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_daemons": { 2024-04-26T02:04:16.213 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_daemons", 2024-04-26T02:04:16.213 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.213 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.213 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.213 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.213 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.213 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.213 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.213 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T02:04:16.214 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.214 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.214 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.214 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_hosts": { 2024-04-26T02:04:16.214 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_hosts", 2024-04-26T02:04:16.214 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.214 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.214 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.214 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.214 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.214 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.214 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.214 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T02:04:16.215 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.215 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.215 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.215 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.215 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.215 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crash", 2024-04-26T02:04:16.215 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.215 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.215 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.215 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.215 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.215 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.216 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.216 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.216 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.216 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.216 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.216 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.216 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.216 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.216 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.216 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.216 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.216 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.216 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.216 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.217 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.217 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.217 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.217 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.217 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.217 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.217 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.217 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.217 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.217 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.217 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.217 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.217 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.217 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.218 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.218 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.218 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.218 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.218 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.218 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.218 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.218 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.218 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.218 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.218 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.218 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.218 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.219 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.219 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.219 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.219 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.219 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.219 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.219 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.219 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.219 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.219 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.219 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.219 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.219 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.220 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.220 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.220 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.220 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.220 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.220 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.220 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.220 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.220 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.220 INFO:teuthology.orchestra.run.smithi001.stdout: "retain_interval": { 2024-04-26T02:04:16.220 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retain_interval", 2024-04-26T02:04:16.220 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.220 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.220 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.221 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "31536000", 2024-04-26T02:04:16.221 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.221 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.221 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.221 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T02:04:16.221 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.221 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.221 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.221 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_recent_interval": { 2024-04-26T02:04:16.221 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_recent_interval", 2024-04-26T02:04:16.221 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.221 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.221 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.221 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1209600", 2024-04-26T02:04:16.221 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.222 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.222 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.222 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T02:04:16.222 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.222 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.222 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.222 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.222 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.222 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dashboard", 2024-04-26T02:04:16.222 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.222 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.223 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.223 INFO:teuthology.orchestra.run.smithi001.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T02:04:16.223 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T02:04:16.223 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.223 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.223 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.223 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:16.223 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.223 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.223 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.223 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.223 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.223 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.223 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.224 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T02:04:16.224 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T02:04:16.224 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.224 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.224 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.224 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.224 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.224 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.224 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.224 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.224 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.224 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.224 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.224 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T02:04:16.225 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T02:04:16.225 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.225 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.225 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.225 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.225 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.225 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.225 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.225 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.225 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.225 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.225 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.225 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_ENABLED": { 2024-04-26T02:04:16.225 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T02:04:16.226 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.226 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.226 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.226 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.226 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.226 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.226 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.226 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.226 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.226 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.226 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.226 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T02:04:16.226 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T02:04:16.227 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.227 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.227 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.227 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.227 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.227 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.227 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.227 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.227 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.227 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.227 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.227 INFO:teuthology.orchestra.run.smithi001.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T02:04:16.228 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T02:04:16.228 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.228 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.228 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.228 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.228 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.228 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.228 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.228 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.228 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.228 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.228 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.228 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T02:04:16.229 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T02:04:16.229 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.229 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.229 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.229 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.229 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.229 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.229 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.229 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.229 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.229 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.229 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.229 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T02:04:16.229 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T02:04:16.230 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.230 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.230 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.230 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.230 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.230 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.230 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.230 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.230 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.230 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.230 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.230 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T02:04:16.230 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T02:04:16.231 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.231 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.231 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.231 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.231 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.231 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.231 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.231 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.231 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.231 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.231 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.231 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T02:04:16.231 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T02:04:16.232 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.232 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.232 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.232 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.232 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.232 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.232 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.232 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.232 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.232 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.232 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.232 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T02:04:16.232 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T02:04:16.233 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.233 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.233 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.233 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.233 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.233 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.233 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.233 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.233 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.233 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.233 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.233 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T02:04:16.233 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T02:04:16.234 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.234 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.234 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.234 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.234 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.234 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.234 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.234 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.234 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.234 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.234 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.234 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T02:04:16.234 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T02:04:16.235 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.235 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.235 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.235 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.235 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.235 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.235 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.235 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.235 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.235 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.235 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.235 INFO:teuthology.orchestra.run.smithi001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T02:04:16.235 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T02:04:16.236 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.236 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.236 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.236 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.236 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.236 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.236 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.236 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.236 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.236 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.236 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.236 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T02:04:16.236 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T02:04:16.237 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.237 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.237 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.237 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-04-26T02:04:16.237 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.237 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.237 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.237 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.237 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.237 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.237 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.237 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T02:04:16.237 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T02:04:16.237 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.238 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.238 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.238 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.238 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.238 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.238 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.238 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.238 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.238 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.238 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.238 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_URL": { 2024-04-26T02:04:16.238 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_URL", 2024-04-26T02:04:16.238 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.238 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.239 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.239 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.239 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.239 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.239 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.239 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.239 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.239 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.239 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.239 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T02:04:16.239 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T02:04:16.239 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.239 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.240 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.240 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-04-26T02:04:16.240 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.240 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.240 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.240 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.240 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.240 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.240 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.240 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T02:04:16.240 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T02:04:16.240 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.240 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.241 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.241 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.241 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.241 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.241 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.241 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.241 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.241 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.241 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.241 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T02:04:16.241 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T02:04:16.241 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.241 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.242 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.242 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.242 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.242 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.242 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.242 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.242 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.242 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.242 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.242 INFO:teuthology.orchestra.run.smithi001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T02:04:16.242 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T02:04:16.242 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.242 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.243 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.243 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.243 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.243 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.243 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.243 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.243 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.243 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.243 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.243 INFO:teuthology.orchestra.run.smithi001.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T02:04:16.243 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T02:04:16.243 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.243 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.244 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.244 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.244 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.244 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.244 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.244 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.244 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.244 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.244 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.244 INFO:teuthology.orchestra.run.smithi001.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T02:04:16.244 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T02:04:16.244 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.244 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.244 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.245 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "[]", 2024-04-26T02:04:16.245 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.245 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.245 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.245 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.245 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.245 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.245 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.245 INFO:teuthology.orchestra.run.smithi001.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T02:04:16.245 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T02:04:16.245 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.245 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.245 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.245 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "{}", 2024-04-26T02:04:16.246 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.246 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.246 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.246 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.246 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.246 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.246 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.246 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T02:04:16.246 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T02:04:16.246 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.246 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.246 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.246 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.247 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.247 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.247 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.247 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.247 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.247 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.247 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.247 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T02:04:16.247 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T02:04:16.247 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.247 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.247 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.247 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.248 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.248 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.248 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.248 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.248 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.248 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.248 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.248 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T02:04:16.248 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T02:04:16.248 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.248 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.248 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.248 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.248 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.249 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.249 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.249 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.249 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.249 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.249 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.249 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T02:04:16.249 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T02:04:16.249 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.249 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.249 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.249 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.249 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.250 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.250 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.250 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.250 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.250 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.250 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.250 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T02:04:16.250 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T02:04:16.250 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.250 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.250 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.250 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.250 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.251 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.251 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.251 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.251 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.251 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.251 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.251 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T02:04:16.251 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T02:04:16.251 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.251 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.251 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.251 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.251 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.251 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.252 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.252 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.252 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.252 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.252 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.252 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T02:04:16.252 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T02:04:16.252 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.252 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.252 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.252 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.252 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.253 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.253 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.253 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.253 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.253 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.253 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.253 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T02:04:16.253 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T02:04:16.253 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.253 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.253 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.253 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.253 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.254 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.254 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.254 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.254 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.254 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.254 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.254 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T02:04:16.254 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T02:04:16.254 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.254 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.254 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.254 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.254 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.255 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.255 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.255 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.255 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.255 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.255 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.255 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T02:04:16.255 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T02:04:16.255 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.255 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.255 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.255 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.255 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.256 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.256 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.256 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.256 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.256 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.256 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.256 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T02:04:16.256 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T02:04:16.256 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.256 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.256 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.256 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T02:04:16.256 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.257 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.257 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.257 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.257 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.257 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.257 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.257 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T02:04:16.257 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T02:04:16.257 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.257 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.257 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.257 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:16.257 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.257 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.258 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.258 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.258 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.258 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.258 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.258 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T02:04:16.258 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T02:04:16.258 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.258 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.258 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.258 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8", 2024-04-26T02:04:16.258 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.258 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.259 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.259 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.259 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.259 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.259 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.259 INFO:teuthology.orchestra.run.smithi001.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T02:04:16.259 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T02:04:16.259 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.259 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.259 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.259 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "45", 2024-04-26T02:04:16.259 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.260 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.260 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.260 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.260 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.260 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.260 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.260 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T02:04:16.260 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T02:04:16.260 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.260 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.260 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.260 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.260 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.260 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T02:04:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T02:04:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-04-26T02:04:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.261 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T02:04:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T02:04:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.262 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T02:04:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T02:04:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.263 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T02:04:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T02:04:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.264 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T02:04:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T02:04:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-04-26T02:04:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.265 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T02:04:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T02:04:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.266 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T02:04:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T02:04:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-04-26T02:04:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.267 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "cross_origin_url": { 2024-04-26T02:04:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cross_origin_url", 2024-04-26T02:04:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.268 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "crt_file": { 2024-04-26T02:04:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crt_file", 2024-04-26T02:04:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.269 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "debug": { 2024-04-26T02:04:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "debug", 2024-04-26T02:04:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable/disable debug options", 2024-04-26T02:04:16.270 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "jwt_token_ttl": { 2024-04-26T02:04:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "jwt_token_ttl", 2024-04-26T02:04:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "28800", 2024-04-26T02:04:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.271 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-04-26T02:04:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-04-26T02:04:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.272 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.273 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.274 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.275 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.276 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.277 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "motd": { 2024-04-26T02:04:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "motd", 2024-04-26T02:04:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "The message of the day", 2024-04-26T02:04:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.278 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect_resolve_ip_addr": { 2024-04-26T02:04:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T02:04:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.279 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-04-26T02:04:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-04-26T02:04:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-04-26T02:04:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.280 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-04-26T02:04:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-04-26T02:04:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8080", 2024-04-26T02:04:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.281 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-04-26T02:04:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-04-26T02:04:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl_server_port": { 2024-04-26T02:04:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl_server_port", 2024-04-26T02:04:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8443", 2024-04-26T02:04:16.283 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-04-26T02:04:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-04-26T02:04:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "redirect", 2024-04-26T02:04:16.284 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect" 2024-04-26T02:04:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-04-26T02:04:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-04-26T02:04:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.285 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-04-26T02:04:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-04-26T02:04:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-04-26T02:04:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "url_prefix": { 2024-04-26T02:04:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url_prefix", 2024-04-26T02:04:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.286 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.287 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-04-26T02:04:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_monitoring": { 2024-04-26T02:04:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_monitoring", 2024-04-26T02:04:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "monitor device health metrics", 2024-04-26T02:04:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.288 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.289 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.290 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.291 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.292 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "mark_out_threshold": { 2024-04-26T02:04:16.293 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mark_out_threshold", 2024-04-26T02:04:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2419200", 2024-04-26T02:04:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T02:04:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.294 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": { 2024-04-26T02:04:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_name", 2024-04-26T02:04:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "device_health_metrics", 2024-04-26T02:04:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T02:04:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.295 INFO:teuthology.orchestra.run.smithi001.stdout: "retention_period": { 2024-04-26T02:04:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retention_period", 2024-04-26T02:04:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15552000", 2024-04-26T02:04:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain device health metrics", 2024-04-26T02:04:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_frequency": { 2024-04-26T02:04:16.296 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_frequency", 2024-04-26T02:04:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-04-26T02:04:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T02:04:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "self_heal": { 2024-04-26T02:04:16.297 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "self_heal", 2024-04-26T02:04:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T02:04:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-04-26T02:04:16.298 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-04-26T02:04:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-04-26T02:04:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T02:04:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_threshold": { 2024-04-26T02:04:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_threshold", 2024-04-26T02:04:16.299 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7257600", 2024-04-26T02:04:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T02:04:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "diskprediction_local", 2024-04-26T02:04:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.301 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.302 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.303 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.304 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.305 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: "predict_interval": { 2024-04-26T02:04:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predict_interval", 2024-04-26T02:04:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-04-26T02:04:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.306 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "predictor_model": { 2024-04-26T02:04:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predictor_model", 2024-04-26T02:04:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "prophetstor", 2024-04-26T02:04:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.307 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-04-26T02:04:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-04-26T02:04:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-04-26T02:04:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.308 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "influx", 2024-04-26T02:04:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": false, 2024-04-26T02:04:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "influxdb python module not found", 2024-04-26T02:04:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: "batch_size": { 2024-04-26T02:04:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "batch_size", 2024-04-26T02:04:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.309 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5000", 2024-04-26T02:04:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T02:04:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: "database": { 2024-04-26T02:04:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "database", 2024-04-26T02:04:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "ceph", 2024-04-26T02:04:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T02:04:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": { 2024-04-26T02:04:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hostname", 2024-04-26T02:04:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.311 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server hostname", 2024-04-26T02:04:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-04-26T02:04:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-04-26T02:04:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.312 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "30", 2024-04-26T02:04:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "5", 2024-04-26T02:04:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T02:04:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.313 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.315 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.316 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.317 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "password": { 2024-04-26T02:04:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "password", 2024-04-26T02:04:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.318 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "password of InfluxDB server user", 2024-04-26T02:04:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "port": { 2024-04-26T02:04:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "port", 2024-04-26T02:04:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8086", 2024-04-26T02:04:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.319 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server port", 2024-04-26T02:04:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-04-26T02:04:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-04-26T02:04:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "false", 2024-04-26T02:04:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.320 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T02:04:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "threads": { 2024-04-26T02:04:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threads", 2024-04-26T02:04:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-04-26T02:04:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-04-26T02:04:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "32", 2024-04-26T02:04:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T02:04:16.321 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: "username": { 2024-04-26T02:04:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "username", 2024-04-26T02:04:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "username of InfluxDB server user", 2024-04-26T02:04:16.322 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "verify_ssl": { 2024-04-26T02:04:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "verify_ssl", 2024-04-26T02:04:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "true", 2024-04-26T02:04:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T02:04:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.323 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "insights", 2024-04-26T02:04:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.324 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.327 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.328 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "iostat", 2024-04-26T02:04:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.329 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.330 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.331 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.332 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.333 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.334 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "k8sevents", 2024-04-26T02:04:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_event_retention_days": { 2024-04-26T02:04:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ceph_event_retention_days", 2024-04-26T02:04:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7", 2024-04-26T02:04:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.335 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T02:04:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "config_check_secs": { 2024-04-26T02:04:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_check_secs", 2024-04-26T02:04:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "10", 2024-04-26T02:04:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.336 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T02:04:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.337 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.339 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.340 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.341 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "localpool", 2024-04-26T02:04:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: "failure_domain": { 2024-04-26T02:04:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "failure_domain", 2024-04-26T02:04:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "host", 2024-04-26T02:04:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "failure domain for any created local pool", 2024-04-26T02:04:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T02:04:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.344 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.345 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.346 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.347 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.348 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": { 2024-04-26T02:04:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_size", 2024-04-26T02:04:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default min_size for any created local pool", 2024-04-26T02:04:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T02:04:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "num_rep": { 2024-04-26T02:04:16.349 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "num_rep", 2024-04-26T02:04:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3", 2024-04-26T02:04:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default replica count for any created local pool", 2024-04-26T02:04:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": { 2024-04-26T02:04:16.350 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_num", 2024-04-26T02:04:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "128", 2024-04-26T02:04:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T02:04:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "prefix": { 2024-04-26T02:04:16.351 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prefix", 2024-04-26T02:04:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name prefix for any created local pool", 2024-04-26T02:04:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "subtree": { 2024-04-26T02:04:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "subtree", 2024-04-26T02:04:16.352 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "rack", 2024-04-26T02:04:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T02:04:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T02:04:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mds_autoscaler", 2024-04-26T02:04:16.353 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.354 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.355 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.356 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.357 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirroring", 2024-04-26T02:04:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.359 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.360 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.361 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.362 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.363 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "nfs", 2024-04-26T02:04:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.364 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.365 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.366 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.367 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.368 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-04-26T02:04:16.369 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "fail_fs": { 2024-04-26T02:04:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "fail_fs", 2024-04-26T02:04:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T02:04:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.370 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.371 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.372 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.373 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator": { 2024-04-26T02:04:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-04-26T02:04:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-04-26T02:04:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "rook", 2024-04-26T02:04:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "test_orchestrator" 2024-04-26T02:04:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Orchestrator backend", 2024-04-26T02:04:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_perf_query", 2024-04-26T02:04:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.377 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.378 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.379 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.380 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_support", 2024-04-26T02:04:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.383 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.384 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.385 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.387 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_autoscaler", 2024-04-26T02:04:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.388 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.389 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.390 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.391 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.392 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-04-26T02:04:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-04-26T02:04:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-04-26T02:04:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.393 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "threshold": { 2024-04-26T02:04:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threshold", 2024-04-26T02:04:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-04-26T02:04:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-04-26T02:04:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1.0", 2024-04-26T02:04:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "scaling threshold", 2024-04-26T02:04:16.394 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T02:04:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "progress", 2024-04-26T02:04:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_pg_recovery_event": { 2024-04-26T02:04:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_pg_recovery_event", 2024-04-26T02:04:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.395 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T02:04:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-04-26T02:04:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-04-26T02:04:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.396 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.397 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.398 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.399 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.400 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.401 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "max_completed_events": { 2024-04-26T02:04:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_completed_events", 2024-04-26T02:04:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "50", 2024-04-26T02:04:16.402 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "number of past completed events to remember", 2024-04-26T02:04:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-04-26T02:04:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-04-26T02:04:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-04-26T02:04:16.403 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long the module is going to sleep", 2024-04-26T02:04:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus", 2024-04-26T02:04:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.404 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "cache": { 2024-04-26T02:04:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cache", 2024-04-26T02:04:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.405 INFO:teuthology.orchestra.run.smithi001.stdout: "exclude_perf_counters": { 2024-04-26T02:04:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "exclude_perf_counters", 2024-04-26T02:04:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T02:04:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T02:04:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.408 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.410 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools": { 2024-04-26T02:04:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools", 2024-04-26T02:04:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.411 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T02:04:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T02:04:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.412 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300", 2024-04-26T02:04:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_interval": { 2024-04-26T02:04:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_interval", 2024-04-26T02:04:16.413 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-04-26T02:04:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15.0", 2024-04-26T02:04:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-04-26T02:04:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-04-26T02:04:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-04-26T02:04:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T02:04:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-04-26T02:04:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-04-26T02:04:16.415 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "9283", 2024-04-26T02:04:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T02:04:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "stale_cache_strategy": { 2024-04-26T02:04:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stale_cache_strategy", 2024-04-26T02:04:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.416 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "log", 2024-04-26T02:04:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-04-26T02:04:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-04-26T02:04:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.417 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "default", 2024-04-26T02:04:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "default", 2024-04-26T02:04:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "error" 2024-04-26T02:04:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-04-26T02:04:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-04-26T02:04:16.418 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-04-26T02:04:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-04-26T02:04:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-04-26T02:04:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-04-26T02:04:16.419 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.420 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.421 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.422 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.423 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_snap_create": { 2024-04-26T02:04:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_snap_create", 2024-04-26T02:04:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.424 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "mirror_snapshot_schedule": { 2024-04-26T02:04:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T02:04:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.425 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "trash_purge_schedule": { 2024-04-26T02:04:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "trash_purge_schedule", 2024-04-26T02:04:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.426 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "restful", 2024-04-26T02:04:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_auth": { 2024-04-26T02:04:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_auth", 2024-04-26T02:04:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.427 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-04-26T02:04:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-04-26T02:04:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.428 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.429 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.430 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.431 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.432 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-04-26T02:04:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-04-26T02:04:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-04-26T02:04:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-04-26T02:04:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rgw", 2024-04-26T02:04:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.436 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.437 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.438 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.439 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.440 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rook", 2024-04-26T02:04:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.441 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.442 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.443 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.444 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.445 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "storage_class": { 2024-04-26T02:04:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "storage_class", 2024-04-26T02:04:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "local", 2024-04-26T02:04:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.446 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T02:04:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "selftest", 2024-04-26T02:04:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.447 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.448 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.449 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.450 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.451 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "roption1": { 2024-04-26T02:04:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption1", 2024-04-26T02:04:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.452 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: "roption2": { 2024-04-26T02:04:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption2", 2024-04-26T02:04:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.453 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "xyz", 2024-04-26T02:04:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption1": { 2024-04-26T02:04:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption1", 2024-04-26T02:04:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.454 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption2": { 2024-04-26T02:04:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption2", 2024-04-26T02:04:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.455 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption3": { 2024-04-26T02:04:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption3", 2024-04-26T02:04:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-04-26T02:04:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.456 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption4": { 2024-04-26T02:04:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption4", 2024-04-26T02:04:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.457 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption5": { 2024-04-26T02:04:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption5", 2024-04-26T02:04:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.458 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption6": { 2024-04-26T02:04:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption6", 2024-04-26T02:04:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.459 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption7": { 2024-04-26T02:04:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption7", 2024-04-26T02:04:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-04-26T02:04:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "42", 2024-04-26T02:04:16.460 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: "testkey": { 2024-04-26T02:04:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testkey", 2024-04-26T02:04:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.461 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: "testlkey": { 2024-04-26T02:04:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testlkey", 2024-04-26T02:04:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.462 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "testnewline": { 2024-04-26T02:04:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testnewline", 2024-04-26T02:04:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.463 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-04-26T02:04:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_m_granularity": { 2024-04-26T02:04:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_m_granularity", 2024-04-26T02:04:16.464 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T02:04:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: "dump_on_update": { 2024-04-26T02:04:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dump_on_update", 2024-04-26T02:04:16.465 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "dump database to debug log on update", 2024-04-26T02:04:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.466 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.467 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.468 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.469 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.470 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stats", 2024-04-26T02:04:16.471 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.472 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.472 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.472 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.472 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.472 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.472 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.472 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.472 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.472 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.472 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.472 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.472 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.472 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.472 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.472 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.472 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.473 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.473 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.473 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.473 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.473 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.473 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.473 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.473 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.473 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.473 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.473 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.473 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.473 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.473 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.473 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.474 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.474 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.474 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.474 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.474 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.474 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.474 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.474 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.474 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.474 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.474 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.474 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.474 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.475 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.475 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.475 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.475 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.475 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.475 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.475 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.475 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.475 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.475 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.475 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.475 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.475 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.475 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.477 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.477 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.477 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.477 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.477 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.477 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.477 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.477 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.477 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.478 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.478 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.478 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.478 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.478 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "status", 2024-04-26T02:04:16.478 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.478 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.478 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.478 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.478 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.478 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.478 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.478 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.479 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.479 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.480 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:15 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/2360827451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T02:04:16.480 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:15 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3653799968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T02:04:16.480 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:15 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3387760829' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T02:04:16.483 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.483 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.483 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.483 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.483 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.483 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.483 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.483 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.483 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.483 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.484 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.484 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.484 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.484 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.484 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.484 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.484 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.484 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.484 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.484 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.484 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.484 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.484 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.484 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.485 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.485 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.485 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.485 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.485 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.485 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.485 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.485 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.485 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.485 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.485 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.485 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.485 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.485 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.486 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.486 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.486 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.486 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.486 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.486 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.486 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.486 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.486 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.486 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.486 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.486 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.486 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.487 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.487 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.487 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.487 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.487 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.487 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.487 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.487 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.487 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.487 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.487 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.487 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telegraf", 2024-04-26T02:04:16.487 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.487 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.487 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.488 INFO:teuthology.orchestra.run.smithi001.stdout: "address": { 2024-04-26T02:04:16.488 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "address", 2024-04-26T02:04:16.488 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.488 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.488 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.488 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T02:04:16.488 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.488 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.488 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.488 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.488 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.488 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.488 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.488 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-04-26T02:04:16.488 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-04-26T02:04:16.489 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.489 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.489 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.489 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15", 2024-04-26T02:04:16.489 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.489 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.489 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.489 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.489 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.489 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.489 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.489 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.489 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.489 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.490 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.490 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.490 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.490 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.490 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.490 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.490 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.490 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.490 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.490 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.490 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.490 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.490 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.490 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.491 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.491 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.491 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.491 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.491 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.491 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.491 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.491 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.491 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.491 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.491 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.491 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.491 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.491 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.492 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.492 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.492 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.492 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.492 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.492 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.492 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.492 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.492 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.492 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.492 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.492 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.492 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.493 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.493 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.493 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.493 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.493 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.493 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.493 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.493 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.493 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.493 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.493 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.493 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.493 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.493 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.494 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.494 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.494 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.494 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.494 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.494 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.494 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.494 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.494 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.494 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.494 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.494 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telemetry", 2024-04-26T02:04:16.495 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.495 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.495 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.495 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_basic": { 2024-04-26T02:04:16.495 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_basic", 2024-04-26T02:04:16.495 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.495 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.495 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.495 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.495 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.495 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.495 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.495 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T02:04:16.495 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.495 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.496 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.496 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_crash": { 2024-04-26T02:04:16.496 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_crash", 2024-04-26T02:04:16.496 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.496 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.496 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.496 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.496 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.496 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.496 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.496 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T02:04:16.496 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.497 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.497 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.497 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_device": { 2024-04-26T02:04:16.497 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_device", 2024-04-26T02:04:16.497 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.497 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.497 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.497 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.497 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.497 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.497 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.497 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T02:04:16.497 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.498 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.498 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.498 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_ident": { 2024-04-26T02:04:16.498 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_ident", 2024-04-26T02:04:16.498 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.498 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.498 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.498 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.498 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.498 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.498 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.498 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T02:04:16.498 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.499 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.499 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.499 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_perf": { 2024-04-26T02:04:16.499 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_perf", 2024-04-26T02:04:16.499 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.499 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.499 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.499 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.499 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.499 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.499 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.499 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T02:04:16.499 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.500 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.500 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.500 INFO:teuthology.orchestra.run.smithi001.stdout: "contact": { 2024-04-26T02:04:16.500 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "contact", 2024-04-26T02:04:16.500 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.500 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.500 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.500 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.500 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.500 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.500 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.500 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.500 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.501 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.501 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.501 INFO:teuthology.orchestra.run.smithi001.stdout: "description": { 2024-04-26T02:04:16.501 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "description", 2024-04-26T02:04:16.501 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.501 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.501 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.501 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.501 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.501 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.501 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.501 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.501 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.502 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.502 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.502 INFO:teuthology.orchestra.run.smithi001.stdout: "device_url": { 2024-04-26T02:04:16.502 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_url", 2024-04-26T02:04:16.502 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.502 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.502 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.502 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T02:04:16.502 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.502 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.502 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.502 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.502 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.503 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.503 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.503 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-04-26T02:04:16.503 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-04-26T02:04:16.503 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.503 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.503 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.503 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.503 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.503 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.503 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.503 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.503 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.504 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.504 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.504 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-04-26T02:04:16.504 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-04-26T02:04:16.504 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.504 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.504 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.504 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "24", 2024-04-26T02:04:16.504 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "8", 2024-04-26T02:04:16.504 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.504 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.504 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.504 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.504 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.505 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.505 INFO:teuthology.orchestra.run.smithi001.stdout: "last_opt_revision": { 2024-04-26T02:04:16.505 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "last_opt_revision", 2024-04-26T02:04:16.505 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.505 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.505 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.505 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1", 2024-04-26T02:04:16.505 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.505 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.505 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.505 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.505 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.505 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.505 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.506 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard": { 2024-04-26T02:04:16.506 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard", 2024-04-26T02:04:16.506 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.506 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.506 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.506 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.506 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.506 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.506 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.506 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.506 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.506 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.506 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.507 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard_description": { 2024-04-26T02:04:16.507 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard_description", 2024-04-26T02:04:16.507 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.507 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.507 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.507 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.507 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.507 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.507 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.507 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.507 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.507 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.507 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.507 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.508 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.508 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.508 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.508 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.508 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.508 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.508 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.508 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.508 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.508 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.508 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.508 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.508 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.508 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.509 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.509 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.509 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.509 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.509 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.509 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.509 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.509 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.509 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.509 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.509 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.509 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.509 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.509 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.510 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.510 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.510 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.510 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.510 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.510 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.510 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.510 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.510 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.510 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.510 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.510 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.510 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.511 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.511 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.511 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.511 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.511 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.511 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.511 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.511 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.511 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.511 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.511 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.511 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.511 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.512 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.512 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.512 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.512 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.512 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.512 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.512 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.512 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.512 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.512 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.512 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.512 INFO:teuthology.orchestra.run.smithi001.stdout: "organization": { 2024-04-26T02:04:16.512 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "organization", 2024-04-26T02:04:16.512 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.512 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.513 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.513 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.513 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.513 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.513 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.513 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.513 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.513 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.513 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.513 INFO:teuthology.orchestra.run.smithi001.stdout: "proxy": { 2024-04-26T02:04:16.513 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "proxy", 2024-04-26T02:04:16.513 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.513 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.513 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.513 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.514 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.514 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.514 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.514 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.514 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.514 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.514 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.514 INFO:teuthology.orchestra.run.smithi001.stdout: "url": { 2024-04-26T02:04:16.514 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url", 2024-04-26T02:04:16.514 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.514 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.514 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.514 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T02:04:16.514 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.515 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.515 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.515 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.515 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.515 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.515 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.515 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.515 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.515 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.515 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "test_orchestrator", 2024-04-26T02:04:16.515 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.515 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.515 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.515 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.516 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.516 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.516 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.516 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.516 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.516 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.516 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.516 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.516 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.516 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.516 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.516 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.516 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.516 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.517 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.517 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.517 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.517 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.517 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.517 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.517 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.517 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.517 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.517 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.517 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.517 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.517 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.517 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.517 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.518 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.518 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.518 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.518 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.518 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.518 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.518 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.518 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.518 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.518 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.518 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.518 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.518 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.519 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.519 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.519 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.519 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.519 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.519 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.519 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.519 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.519 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.519 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.519 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.519 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.519 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.519 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.519 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.519 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.520 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.520 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.520 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.520 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.520 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.520 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.520 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.520 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.520 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.520 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-04-26T02:04:16.520 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.520 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.520 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.520 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.521 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.521 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.521 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.521 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.521 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.521 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.521 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.521 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.521 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.521 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.521 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.521 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.521 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.521 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.521 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.521 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.522 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.522 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.522 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.522 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.522 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.522 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.522 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.522 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.522 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.522 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.522 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.522 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.523 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.523 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.523 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.523 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.523 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.523 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.523 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.523 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.523 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.523 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.523 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.523 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.523 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.523 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.523 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.523 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.524 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.524 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.524 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.524 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.524 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.524 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.524 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.524 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.524 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.524 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.524 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.524 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.524 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.524 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.524 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.525 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.525 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.525 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.525 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.525 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.525 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.525 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_clones": { 2024-04-26T02:04:16.525 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_clones", 2024-04-26T02:04:16.525 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.525 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.525 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.525 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4", 2024-04-26T02:04:16.525 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.525 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.526 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.526 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T02:04:16.526 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.526 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.526 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.526 INFO:teuthology.orchestra.run.smithi001.stdout: "periodic_async_work": { 2024-04-26T02:04:16.526 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "periodic_async_work", 2024-04-26T02:04:16.526 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.526 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.526 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.526 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.526 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.526 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.526 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.526 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Periodically check for async work", 2024-04-26T02:04:16.526 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.527 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.527 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.527 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_delay": { 2024-04-26T02:04:16.527 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_delay", 2024-04-26T02:04:16.527 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.527 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.527 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.527 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-04-26T02:04:16.527 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.527 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.527 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.527 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T02:04:16.527 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.527 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.528 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.528 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_no_wait": { 2024-04-26T02:04:16.528 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T02:04:16.528 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.528 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.528 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.528 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.528 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.528 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.528 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.528 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T02:04:16.528 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.528 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.528 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.528 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.528 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.529 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.529 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix", 2024-04-26T02:04:16.529 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.529 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.529 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.529 INFO:teuthology.orchestra.run.smithi001.stdout: "discovery_interval": { 2024-04-26T02:04:16.529 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "discovery_interval", 2024-04-26T02:04:16.529 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-04-26T02:04:16.529 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.529 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.529 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "100", 2024-04-26T02:04:16.529 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.529 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.529 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.529 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.530 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.530 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.530 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.530 INFO:teuthology.orchestra.run.smithi001.stdout: "identifier": { 2024-04-26T02:04:16.530 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "identifier", 2024-04-26T02:04:16.530 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.530 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.530 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.530 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.530 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.530 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.530 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.531 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.531 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.531 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.531 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.531 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-04-26T02:04:16.531 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-04-26T02:04:16.531 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.531 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.531 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.531 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-04-26T02:04:16.531 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.531 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.531 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.531 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.531 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.532 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.532 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.532 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.532 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.532 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.532 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.532 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.532 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.532 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.532 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.532 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.532 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.532 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.532 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.532 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.533 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.533 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.533 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.533 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.533 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.533 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.533 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.533 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.533 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.533 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.533 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.533 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.533 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.533 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.533 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.534 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.534 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.534 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.534 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.534 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.534 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.534 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.534 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.534 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.534 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.534 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.534 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.535 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.535 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.535 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.535 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.535 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.535 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.535 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.535 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.535 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.535 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.535 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.535 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.535 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.535 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.535 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.535 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.536 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.536 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.536 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.536 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.536 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.536 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.536 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.536 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.536 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.536 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.536 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_host": { 2024-04-26T02:04:16.536 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_host", 2024-04-26T02:04:16.536 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.536 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.536 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.537 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.537 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.537 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.537 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.537 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.537 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.537 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.537 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.537 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_port": { 2024-04-26T02:04:16.537 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_port", 2024-04-26T02:04:16.537 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.537 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.537 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.538 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10051", 2024-04-26T02:04:16.538 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.538 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.538 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.538 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.538 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.538 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.538 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.538 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_sender": { 2024-04-26T02:04:16.538 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_sender", 2024-04-26T02:04:16.538 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.538 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.539 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.539 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T02:04:16.539 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.539 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.539 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.539 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.539 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.539 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.539 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.539 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.539 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.539 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.539 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-04-26T02:04:16.539 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.540 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.540 INFO:teuthology.orchestra.run.smithi001.stdout: "modules": [ 2024-04-26T02:04:16.540 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-04-26T02:04:16.540 INFO:teuthology.orchestra.run.smithi001.stdout: "dashboard", 2024-04-26T02:04:16.540 INFO:teuthology.orchestra.run.smithi001.stdout: "iostat", 2024-04-26T02:04:16.540 INFO:teuthology.orchestra.run.smithi001.stdout: "nfs", 2024-04-26T02:04:16.540 INFO:teuthology.orchestra.run.smithi001.stdout: "restful", 2024-04-26T02:04:16.540 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_schedule" 2024-04-26T02:04:16.540 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.540 INFO:teuthology.orchestra.run.smithi001.stdout: "available_modules": [ 2024-04-26T02:04:16.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.540 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alerts", 2024-04-26T02:04:16.540 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.541 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.541 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.541 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-04-26T02:04:16.541 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-04-26T02:04:16.541 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.541 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.541 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.541 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-04-26T02:04:16.541 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.541 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.541 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.541 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T02:04:16.541 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.541 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.542 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.542 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.542 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.542 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.542 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.542 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.542 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.542 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.542 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.542 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.542 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.542 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.543 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.543 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.543 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.543 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.543 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.543 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.543 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.543 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.543 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.543 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.543 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.543 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.543 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.543 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.544 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.544 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.544 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.544 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.544 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.544 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.544 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.544 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.544 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.544 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.544 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.544 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.544 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.545 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.545 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.545 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.545 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.545 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.545 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.545 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.545 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.545 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.545 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.545 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.545 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.545 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.545 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.546 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.546 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.546 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.546 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.546 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.546 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.546 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.546 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.546 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.546 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.546 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.546 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.546 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.547 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.547 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_destination": { 2024-04-26T02:04:16.547 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_destination", 2024-04-26T02:04:16.547 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.547 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.547 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.547 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.547 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.547 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.547 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.547 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email address to send alerts to", 2024-04-26T02:04:16.547 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.547 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.547 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.548 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_from_name": { 2024-04-26T02:04:16.548 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_from_name", 2024-04-26T02:04:16.548 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.548 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.548 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.548 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "Ceph", 2024-04-26T02:04:16.548 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.548 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.548 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.548 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email From: name", 2024-04-26T02:04:16.548 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.548 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.548 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.549 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_host": { 2024-04-26T02:04:16.549 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_host", 2024-04-26T02:04:16.549 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.549 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.549 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.549 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.549 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.549 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.549 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.549 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP server", 2024-04-26T02:04:16.549 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.549 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.549 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.549 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_password": { 2024-04-26T02:04:16.550 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_password", 2024-04-26T02:04:16.550 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.550 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.550 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.550 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.550 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.550 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.550 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.550 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Password to authenticate with", 2024-04-26T02:04:16.550 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.550 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.550 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.550 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_port": { 2024-04-26T02:04:16.550 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_port", 2024-04-26T02:04:16.551 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.551 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.551 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.551 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "465", 2024-04-26T02:04:16.551 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.551 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.551 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.551 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP port", 2024-04-26T02:04:16.551 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.551 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.551 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.551 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_sender": { 2024-04-26T02:04:16.551 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_sender", 2024-04-26T02:04:16.552 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.552 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.552 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.552 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.552 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.552 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.552 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.552 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP envelope sender", 2024-04-26T02:04:16.552 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.552 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.552 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.552 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_ssl": { 2024-04-26T02:04:16.552 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_ssl", 2024-04-26T02:04:16.552 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.553 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.553 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.553 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.553 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.553 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.553 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.553 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T02:04:16.553 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.553 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.553 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.553 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_user": { 2024-04-26T02:04:16.553 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_user", 2024-04-26T02:04:16.553 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.554 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.554 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.554 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.554 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.554 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.554 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.554 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "User to authenticate as", 2024-04-26T02:04:16.554 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.554 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.554 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.554 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.554 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.555 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "balancer", 2024-04-26T02:04:16.555 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.555 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.555 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.555 INFO:teuthology.orchestra.run.smithi001.stdout: "active": { 2024-04-26T02:04:16.555 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "active", 2024-04-26T02:04:16.555 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.555 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.555 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.555 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.555 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.555 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.555 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.555 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T02:04:16.556 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.556 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.556 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.556 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_time": { 2024-04-26T02:04:16.556 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_time", 2024-04-26T02:04:16.556 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.556 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.556 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.556 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0000", 2024-04-26T02:04:16.556 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.556 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.556 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.556 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T02:04:16.556 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T02:04:16.557 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.557 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.557 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_weekday": { 2024-04-26T02:04:16.557 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_weekday", 2024-04-26T02:04:16.557 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-04-26T02:04:16.557 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.557 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.557 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-04-26T02:04:16.557 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-04-26T02:04:16.557 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-04-26T02:04:16.557 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.557 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T02:04:16.557 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T02:04:16.557 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.558 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.558 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_max_iterations": { 2024-04-26T02:04:16.558 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_max_iterations", 2024-04-26T02:04:16.558 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-04-26T02:04:16.558 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.558 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.558 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "25", 2024-04-26T02:04:16.558 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-04-26T02:04:16.558 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "250", 2024-04-26T02:04:16.558 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.558 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T02:04:16.558 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.558 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.558 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.559 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_metrics": { 2024-04-26T02:04:16.559 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_metrics", 2024-04-26T02:04:16.559 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.559 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.559 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.559 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T02:04:16.559 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.559 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.559 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.559 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T02:04:16.559 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T02:04:16.559 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.559 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.559 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_step": { 2024-04-26T02:04:16.560 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_step", 2024-04-26T02:04:16.560 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-04-26T02:04:16.560 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.560 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.560 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.5", 2024-04-26T02:04:16.560 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0.001", 2024-04-26T02:04:16.560 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "0.999", 2024-04-26T02:04:16.560 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.560 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "aggressiveness of optimization", 2024-04-26T02:04:16.560 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T02:04:16.560 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.560 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.560 INFO:teuthology.orchestra.run.smithi001.stdout: "end_time": { 2024-04-26T02:04:16.560 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_time", 2024-04-26T02:04:16.561 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.561 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.561 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.561 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2359", 2024-04-26T02:04:16.561 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.561 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.561 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.561 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T02:04:16.561 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T02:04:16.561 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.561 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.561 INFO:teuthology.orchestra.run.smithi001.stdout: "end_weekday": { 2024-04-26T02:04:16.561 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_weekday", 2024-04-26T02:04:16.562 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-04-26T02:04:16.562 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.562 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.562 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-04-26T02:04:16.562 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-04-26T02:04:16.562 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-04-26T02:04:16.562 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.562 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T02:04:16.562 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T02:04:16.562 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.562 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.562 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.562 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.562 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.563 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.563 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.563 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.563 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.563 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.563 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.563 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.563 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.563 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.563 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.563 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.563 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.563 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.563 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.563 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.563 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.563 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.564 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.564 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.564 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.564 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.564 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.564 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.564 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.564 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.564 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.564 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.564 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.564 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.564 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.565 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.565 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.565 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.565 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.565 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.565 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.565 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.565 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.565 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.565 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.565 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.565 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.565 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.565 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.566 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.566 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.566 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.566 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.566 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.566 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.566 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.566 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.566 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.566 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.566 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.566 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.566 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.566 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.567 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.567 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.567 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.567 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.567 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.567 INFO:teuthology.orchestra.run.smithi001.stdout: "min_score": { 2024-04-26T02:04:16.567 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_score", 2024-04-26T02:04:16.567 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-04-26T02:04:16.567 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.567 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.567 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-04-26T02:04:16.567 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.568 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.568 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.568 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T02:04:16.568 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.568 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.568 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.568 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-04-26T02:04:16.568 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-04-26T02:04:16.568 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.568 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.568 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.568 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "upmap", 2024-04-26T02:04:16.568 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.569 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.569 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.569 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-compat", 2024-04-26T02:04:16.569 INFO:teuthology.orchestra.run.smithi001.stdout: "none", 2024-04-26T02:04:16.569 INFO:teuthology.orchestra.run.smithi001.stdout: "read", 2024-04-26T02:04:16.569 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap", 2024-04-26T02:04:16.569 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap-read" 2024-04-26T02:04:16.569 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.569 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Balancer mode", 2024-04-26T02:04:16.569 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.569 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.569 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.569 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_ids": { 2024-04-26T02:04:16.570 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_ids", 2024-04-26T02:04:16.570 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.570 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.570 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.570 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.570 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.570 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.570 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.570 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T02:04:16.570 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.570 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.570 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.570 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-04-26T02:04:16.570 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-04-26T02:04:16.571 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.571 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.571 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.571 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-04-26T02:04:16.571 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.571 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.571 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.571 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T02:04:16.571 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.571 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.571 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.571 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_deviation": { 2024-04-26T02:04:16.571 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_deviation", 2024-04-26T02:04:16.571 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.572 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.572 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.572 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-04-26T02:04:16.572 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-04-26T02:04:16.572 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.572 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.572 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T02:04:16.572 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T02:04:16.572 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.572 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.572 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_optimizations": { 2024-04-26T02:04:16.572 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_optimizations", 2024-04-26T02:04:16.572 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-04-26T02:04:16.572 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.572 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.573 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:16.573 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.573 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.573 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.573 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T02:04:16.573 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.573 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.573 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.573 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.573 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm", 2024-04-26T02:04:16.573 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.573 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.573 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.574 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_down_multiplier": { 2024-04-26T02:04:16.574 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_down_multiplier", 2024-04-26T02:04:16.574 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-04-26T02:04:16.574 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.574 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.574 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-04-26T02:04:16.574 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.574 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.574 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.574 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T02:04:16.574 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.574 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.574 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.574 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_refresh_rate": { 2024-04-26T02:04:16.575 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_refresh_rate", 2024-04-26T02:04:16.575 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.575 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.575 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.575 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "20", 2024-04-26T02:04:16.575 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.575 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.575 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.575 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T02:04:16.575 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.575 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.575 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.575 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_starting_port": { 2024-04-26T02:04:16.575 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_starting_port", 2024-04-26T02:04:16.575 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.576 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.576 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.576 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4721", 2024-04-26T02:04:16.576 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.576 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.576 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.576 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T02:04:16.576 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.576 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.576 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.576 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_ptrace": { 2024-04-26T02:04:16.576 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_ptrace", 2024-04-26T02:04:16.576 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.576 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.576 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.577 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.577 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.577 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.577 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.577 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T02:04:16.577 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T02:04:16.577 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.577 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.577 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_interval": { 2024-04-26T02:04:16.577 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_interval", 2024-04-26T02:04:16.577 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.577 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.577 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.577 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-04-26T02:04:16.577 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.578 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.578 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.578 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T02:04:16.578 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.578 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.578 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.578 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_memory_target_ratio": { 2024-04-26T02:04:16.578 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T02:04:16.578 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-04-26T02:04:16.578 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.578 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.578 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.7", 2024-04-26T02:04:16.579 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.579 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.579 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.579 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T02:04:16.579 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.579 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.579 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.579 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm_log_destination": { 2024-04-26T02:04:16.579 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm_log_destination", 2024-04-26T02:04:16.579 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.579 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.579 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.579 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.580 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.580 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.580 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.580 INFO:teuthology.orchestra.run.smithi001.stdout: "file", 2024-04-26T02:04:16.580 INFO:teuthology.orchestra.run.smithi001.stdout: "file,syslog", 2024-04-26T02:04:16.580 INFO:teuthology.orchestra.run.smithi001.stdout: "syslog" 2024-04-26T02:04:16.580 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.580 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T02:04:16.580 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.580 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.580 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.580 INFO:teuthology.orchestra.run.smithi001.stdout: "cgroups_split": { 2024-04-26T02:04:16.580 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cgroups_split", 2024-04-26T02:04:16.580 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.581 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.581 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.581 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.581 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.581 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.581 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.581 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T02:04:16.581 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.581 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.581 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.581 INFO:teuthology.orchestra.run.smithi001.stdout: "config_checks_enabled": { 2024-04-26T02:04:16.581 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_checks_enabled", 2024-04-26T02:04:16.581 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.581 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.581 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.582 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.582 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.582 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.582 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.582 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T02:04:16.582 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.582 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.582 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.582 INFO:teuthology.orchestra.run.smithi001.stdout: "config_dashboard": { 2024-04-26T02:04:16.582 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_dashboard", 2024-04-26T02:04:16.582 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.582 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.582 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.583 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.583 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.583 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.583 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.583 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T02:04:16.583 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.583 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.583 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.583 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_alertmanager": { 2024-04-26T02:04:16.583 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_alertmanager", 2024-04-26T02:04:16.583 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.583 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.583 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.583 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T02:04:16.584 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.584 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.584 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.584 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-04-26T02:04:16.584 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.584 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.584 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.584 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_base": { 2024-04-26T02:04:16.584 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_base", 2024-04-26T02:04:16.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.584 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.584 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.584 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T02:04:16.584 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Container image name, without the tag", 2024-04-26T02:04:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_elasticsearch": { 2024-04-26T02:04:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_elasticsearch", 2024-04-26T02:04:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T02:04:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.586 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "elasticsearch container image", 2024-04-26T02:04:16.586 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.586 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.586 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.586 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_grafana": { 2024-04-26T02:04:16.586 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_grafana", 2024-04-26T02:04:16.586 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.586 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.586 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.586 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T02:04:16.586 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.586 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.586 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.586 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-04-26T02:04:16.587 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.587 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.587 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.587 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_haproxy": { 2024-04-26T02:04:16.587 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_haproxy", 2024-04-26T02:04:16.587 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.587 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.587 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.587 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T02:04:16.587 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.587 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.587 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.587 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "HAproxy container image", 2024-04-26T02:04:16.587 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.588 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.588 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.588 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_agent": { 2024-04-26T02:04:16.588 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_agent", 2024-04-26T02:04:16.588 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.588 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.588 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.588 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T02:04:16.588 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.588 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.588 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.588 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger agent container image", 2024-04-26T02:04:16.588 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.588 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.589 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.589 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_collector": { 2024-04-26T02:04:16.589 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_collector", 2024-04-26T02:04:16.589 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.589 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.589 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.589 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T02:04:16.589 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.589 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.589 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.589 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger collector container image", 2024-04-26T02:04:16.589 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.589 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.589 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.590 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_query": { 2024-04-26T02:04:16.590 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_query", 2024-04-26T02:04:16.590 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.590 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.590 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.590 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T02:04:16.590 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.590 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.590 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.590 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger query container image", 2024-04-26T02:04:16.590 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.590 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.590 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.591 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_keepalived": { 2024-04-26T02:04:16.591 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_keepalived", 2024-04-26T02:04:16.591 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.591 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.591 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.591 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T02:04:16.591 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.591 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.591 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.591 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Keepalived container image", 2024-04-26T02:04:16.592 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.592 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.592 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.592 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_loki": { 2024-04-26T02:04:16.592 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_loki", 2024-04-26T02:04:16.592 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.592 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.592 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.592 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T02:04:16.592 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.592 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.592 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.592 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Loki container image", 2024-04-26T02:04:16.593 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.593 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.593 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.593 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_node_exporter": { 2024-04-26T02:04:16.593 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_node_exporter", 2024-04-26T02:04:16.593 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.593 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.593 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.593 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T02:04:16.593 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.593 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.593 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.593 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-04-26T02:04:16.593 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.594 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.594 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.594 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_nvmeof": { 2024-04-26T02:04:16.594 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_nvmeof", 2024-04-26T02:04:16.594 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.594 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.594 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.594 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T02:04:16.594 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.594 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.594 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.594 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Nvme-of container image", 2024-04-26T02:04:16.594 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.594 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.595 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.595 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_prometheus": { 2024-04-26T02:04:16.595 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_prometheus", 2024-04-26T02:04:16.595 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.595 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.595 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.595 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T02:04:16.595 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.595 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.595 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.595 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-04-26T02:04:16.595 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.595 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.595 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.596 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_promtail": { 2024-04-26T02:04:16.596 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_promtail", 2024-04-26T02:04:16.596 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.596 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.596 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.596 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T02:04:16.596 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.596 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.596 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.596 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Promtail container image", 2024-04-26T02:04:16.596 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.596 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.596 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.597 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_samba": { 2024-04-26T02:04:16.597 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_samba", 2024-04-26T02:04:16.597 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.597 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.597 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.597 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T02:04:16.597 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.597 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.597 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.597 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Samba/SMB container image", 2024-04-26T02:04:16.597 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.597 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.597 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.597 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_snmp_gateway": { 2024-04-26T02:04:16.598 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_snmp_gateway", 2024-04-26T02:04:16.598 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.598 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.598 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.598 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T02:04:16.598 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.598 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.598 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.598 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SNMP Gateway container image", 2024-04-26T02:04:16.598 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.598 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.598 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.598 INFO:teuthology.orchestra.run.smithi001.stdout: "container_init": { 2024-04-26T02:04:16.598 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_init", 2024-04-26T02:04:16.599 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.599 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.599 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.599 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.599 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.599 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.599 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.599 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T02:04:16.599 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.599 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.599 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.599 INFO:teuthology.orchestra.run.smithi001.stdout: "daemon_cache_timeout": { 2024-04-26T02:04:16.599 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "daemon_cache_timeout", 2024-04-26T02:04:16.599 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.599 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.599 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.600 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-04-26T02:04:16.600 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.600 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.600 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.600 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T02:04:16.600 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.600 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.600 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.600 INFO:teuthology.orchestra.run.smithi001.stdout: "default_cephadm_command_timeout": { 2024-04-26T02:04:16.600 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T02:04:16.600 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.600 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.600 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.600 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "900", 2024-04-26T02:04:16.601 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.601 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.601 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.601 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T02:04:16.601 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.601 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.601 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.601 INFO:teuthology.orchestra.run.smithi001.stdout: "default_registry": { 2024-04-26T02:04:16.601 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_registry", 2024-04-26T02:04:16.601 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.601 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.601 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.601 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io", 2024-04-26T02:04:16.601 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.601 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.602 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.602 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T02:04:16.602 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.602 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.602 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.602 INFO:teuthology.orchestra.run.smithi001.stdout: "device_cache_timeout": { 2024-04-26T02:04:16.602 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_cache_timeout", 2024-04-26T02:04:16.602 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.602 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.602 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.602 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1800", 2024-04-26T02:04:16.602 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.602 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.603 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.603 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache device inventory", 2024-04-26T02:04:16.603 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.603 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.603 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.603 INFO:teuthology.orchestra.run.smithi001.stdout: "device_enhanced_scan": { 2024-04-26T02:04:16.603 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_enhanced_scan", 2024-04-26T02:04:16.603 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.603 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.603 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.603 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.603 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.603 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.603 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.603 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T02:04:16.604 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.604 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.604 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.604 INFO:teuthology.orchestra.run.smithi001.stdout: "facts_cache_timeout": { 2024-04-26T02:04:16.604 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "facts_cache_timeout", 2024-04-26T02:04:16.604 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.604 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.604 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.604 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-04-26T02:04:16.604 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.604 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.604 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.604 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache host facts data", 2024-04-26T02:04:16.605 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.605 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.605 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.605 INFO:teuthology.orchestra.run.smithi001.stdout: "grafana_dashboards_path": { 2024-04-26T02:04:16.605 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "grafana_dashboards_path", 2024-04-26T02:04:16.605 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.605 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.605 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.605 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T02:04:16.605 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.605 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.605 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.605 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T02:04:16.605 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.605 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.606 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.606 INFO:teuthology.orchestra.run.smithi001.stdout: "host_check_interval": { 2024-04-26T02:04:16.606 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "host_check_interval", 2024-04-26T02:04:16.606 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.606 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.606 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.606 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-04-26T02:04:16.606 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.606 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.606 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.606 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to perform a host check", 2024-04-26T02:04:16.606 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.606 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.607 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.607 INFO:teuthology.orchestra.run.smithi001.stdout: "hw_monitoring": { 2024-04-26T02:04:16.607 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hw_monitoring", 2024-04-26T02:04:16.607 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.607 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.607 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.607 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.607 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.607 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.607 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.607 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T02:04:16.607 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.607 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.607 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.608 INFO:teuthology.orchestra.run.smithi001.stdout: "inventory_list_all": { 2024-04-26T02:04:16.608 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "inventory_list_all", 2024-04-26T02:04:16.608 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.608 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.608 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.608 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.608 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.608 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.608 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.608 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T02:04:16.608 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.608 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.608 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.609 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.609 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.609 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.609 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.609 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.609 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.609 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.609 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.609 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.609 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.609 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.609 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.609 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.609 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.609 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.610 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.610 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.610 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.610 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.610 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.610 INFO:teuthology.orchestra.run.smithi001.stdout: "log_refresh_metadata": { 2024-04-26T02:04:16.610 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_refresh_metadata", 2024-04-26T02:04:16.610 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.610 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.610 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.610 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.610 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.610 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.611 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.611 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T02:04:16.611 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.611 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.611 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.611 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.611 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.611 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.611 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.611 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.611 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.611 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.611 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.612 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.612 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T02:04:16.612 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.612 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.612 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.612 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.612 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.612 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.612 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.612 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.612 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.612 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.612 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.612 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.613 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.613 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.613 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.613 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.613 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.613 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.613 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.613 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.613 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.613 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.613 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.613 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.613 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.613 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.613 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.614 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.614 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.614 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.614 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.614 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.614 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.614 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.614 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.614 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.614 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T02:04:16.614 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T02:04:16.614 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.615 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.615 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.615 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.615 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.615 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.615 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.615 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T02:04:16.615 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.615 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.615 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.615 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T02:04:16.615 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T02:04:16.615 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.615 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.616 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.616 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "*", 2024-04-26T02:04:16.616 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.616 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.616 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.616 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T02:04:16.616 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.616 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.616 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.616 INFO:teuthology.orchestra.run.smithi001.stdout: "max_count_per_host": { 2024-04-26T02:04:16.616 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_count_per_host", 2024-04-26T02:04:16.616 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.616 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.617 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.617 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:16.617 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.617 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.617 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.617 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of daemons per service per host", 2024-04-26T02:04:16.617 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.617 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.617 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.617 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd_draining_count": { 2024-04-26T02:04:16.617 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_osd_draining_count", 2024-04-26T02:04:16.617 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.617 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.617 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.617 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:16.618 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.618 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.618 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.618 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T02:04:16.618 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.618 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.618 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.618 INFO:teuthology.orchestra.run.smithi001.stdout: "migration_current": { 2024-04-26T02:04:16.618 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "migration_current", 2024-04-26T02:04:16.618 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.618 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.618 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.618 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.618 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.619 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.619 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.619 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "internal - do not modify", 2024-04-26T02:04:16.619 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.619 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.619 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.619 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-04-26T02:04:16.619 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-04-26T02:04:16.619 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.619 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.619 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.619 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "root", 2024-04-26T02:04:16.619 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.619 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.619 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.619 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm-package", 2024-04-26T02:04:16.620 INFO:teuthology.orchestra.run.smithi001.stdout: "root" 2024-04-26T02:04:16.620 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.620 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T02:04:16.620 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.620 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.620 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.620 INFO:teuthology.orchestra.run.smithi001.stdout: "oob_default_addr": { 2024-04-26T02:04:16.620 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "oob_default_addr", 2024-04-26T02:04:16.620 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.620 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.620 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.620 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "169.254.1.1", 2024-04-26T02:04:16.620 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.620 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.620 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.621 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T02:04:16.621 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.621 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.621 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.621 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_alerts_path": { 2024-04-26T02:04:16.621 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_alerts_path", 2024-04-26T02:04:16.621 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.621 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.621 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.621 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T02:04:16.621 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.621 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.621 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.621 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T02:04:16.621 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.622 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.622 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.622 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_insecure": { 2024-04-26T02:04:16.622 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_insecure", 2024-04-26T02:04:16.622 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.622 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.622 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.622 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.622 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.622 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.622 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.622 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T02:04:16.622 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.623 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.623 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.623 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_password": { 2024-04-26T02:04:16.623 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_password", 2024-04-26T02:04:16.623 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.623 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.623 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.623 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.623 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.623 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.623 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.623 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T02:04:16.623 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.623 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.623 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.624 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_url": { 2024-04-26T02:04:16.624 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_url", 2024-04-26T02:04:16.624 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.624 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.624 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.624 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.624 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.624 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.624 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.624 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T02:04:16.624 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.624 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.624 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.625 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_username": { 2024-04-26T02:04:16.625 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_username", 2024-04-26T02:04:16.625 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.625 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.625 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.625 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.625 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.625 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.625 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.625 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T02:04:16.625 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.625 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.625 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.626 INFO:teuthology.orchestra.run.smithi001.stdout: "secure_monitoring_stack": { 2024-04-26T02:04:16.626 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secure_monitoring_stack", 2024-04-26T02:04:16.626 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.626 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.626 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.626 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.626 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.626 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.626 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.626 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T02:04:16.626 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.626 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.626 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.627 INFO:teuthology.orchestra.run.smithi001.stdout: "service_discovery_port": { 2024-04-26T02:04:16.627 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "service_discovery_port", 2024-04-26T02:04:16.627 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.627 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.627 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.627 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8765", 2024-04-26T02:04:16.627 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.627 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.627 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.627 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "cephadm service discovery port", 2024-04-26T02:04:16.627 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.627 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.627 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.628 INFO:teuthology.orchestra.run.smithi001.stdout: "ssh_config_file": { 2024-04-26T02:04:16.628 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssh_config_file", 2024-04-26T02:04:16.628 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.628 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.628 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.628 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.628 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.628 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.628 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.628 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T02:04:16.628 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.628 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.628 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.628 INFO:teuthology.orchestra.run.smithi001.stdout: "use_agent": { 2024-04-26T02:04:16.629 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_agent", 2024-04-26T02:04:16.629 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.629 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.629 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.629 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.629 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.629 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.629 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.629 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T02:04:16.629 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.629 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.629 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.629 INFO:teuthology.orchestra.run.smithi001.stdout: "use_repo_digest": { 2024-04-26T02:04:16.630 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_repo_digest", 2024-04-26T02:04:16.630 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.630 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.630 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.630 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.630 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.630 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.630 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.630 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T02:04:16.630 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.630 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.630 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.631 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_failed_host_check": { 2024-04-26T02:04:16.631 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_failed_host_check", 2024-04-26T02:04:16.631 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.631 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.631 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.631 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.631 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.631 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.631 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.631 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T02:04:16.631 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.631 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.631 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.631 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_daemons": { 2024-04-26T02:04:16.632 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_daemons", 2024-04-26T02:04:16.632 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.632 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.632 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.632 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.632 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.632 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.632 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.632 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T02:04:16.632 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.632 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.632 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.632 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_hosts": { 2024-04-26T02:04:16.632 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_hosts", 2024-04-26T02:04:16.633 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.633 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.633 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.633 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.633 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.633 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.633 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.633 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T02:04:16.633 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.633 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.633 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.633 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.633 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.633 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.634 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crash", 2024-04-26T02:04:16.634 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.634 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.634 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.634 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.634 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.634 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.634 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.634 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.634 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.634 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.634 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.634 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.635 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.635 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.635 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.635 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.635 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.635 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.635 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.635 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.635 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.635 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.635 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.635 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.635 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.635 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.636 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.636 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.636 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.636 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.636 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.636 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.636 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.636 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.636 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.636 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.636 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.636 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.636 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.636 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.636 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.636 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.637 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.637 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.637 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.637 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.637 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.637 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.637 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.637 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.637 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.637 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.637 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.637 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.637 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.637 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.638 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.638 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.638 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.638 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.638 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.638 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.638 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.638 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.638 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.638 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.638 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.638 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.638 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.639 INFO:teuthology.orchestra.run.smithi001.stdout: "retain_interval": { 2024-04-26T02:04:16.639 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retain_interval", 2024-04-26T02:04:16.639 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.639 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.639 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.639 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "31536000", 2024-04-26T02:04:16.639 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.639 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.639 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.639 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T02:04:16.639 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.639 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.639 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.639 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_recent_interval": { 2024-04-26T02:04:16.640 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_recent_interval", 2024-04-26T02:04:16.640 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.640 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.640 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.640 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1209600", 2024-04-26T02:04:16.640 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.640 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.640 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.640 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T02:04:16.640 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.640 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.640 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.640 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.640 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.641 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.641 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dashboard", 2024-04-26T02:04:16.641 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.641 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.641 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.641 INFO:teuthology.orchestra.run.smithi001.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T02:04:16.641 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T02:04:16.641 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.641 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.641 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.641 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:16.641 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.641 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.642 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.642 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.642 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.642 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.642 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.642 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T02:04:16.642 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T02:04:16.642 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.642 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.642 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.642 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.642 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.642 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.643 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.643 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.643 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.643 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.643 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.643 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T02:04:16.643 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T02:04:16.643 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.643 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.643 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.643 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.643 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.643 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.643 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.644 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.644 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.644 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.644 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.644 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_ENABLED": { 2024-04-26T02:04:16.644 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T02:04:16.644 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.644 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.644 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.644 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.644 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.644 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.644 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.644 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.645 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.645 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.645 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.645 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T02:04:16.645 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T02:04:16.645 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.645 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.645 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.645 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.645 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.645 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.645 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.645 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.645 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.646 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.646 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.646 INFO:teuthology.orchestra.run.smithi001.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T02:04:16.646 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T02:04:16.646 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.646 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.646 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.646 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.646 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.646 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.646 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.646 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.646 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.647 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.647 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.647 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T02:04:16.647 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T02:04:16.647 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.647 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.647 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.647 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.647 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.647 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.647 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.647 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.647 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.647 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.647 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.648 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T02:04:16.648 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T02:04:16.648 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.648 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.648 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.648 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.648 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.648 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.648 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.648 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.648 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.648 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.648 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.649 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T02:04:16.649 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T02:04:16.649 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.649 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.649 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.649 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.649 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.649 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.649 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.649 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.649 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.649 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.649 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.650 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T02:04:16.650 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T02:04:16.650 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.650 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.650 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.650 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.650 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.650 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.650 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.650 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.650 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.650 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.650 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.651 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T02:04:16.651 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T02:04:16.651 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.651 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.651 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.651 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.651 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.651 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.651 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.651 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.651 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.651 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.652 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.652 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T02:04:16.652 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T02:04:16.652 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.652 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.652 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.652 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.652 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.652 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.652 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.652 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.652 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.652 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.652 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.653 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T02:04:16.653 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T02:04:16.653 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.653 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.653 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.653 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.653 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.653 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.653 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.653 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.653 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.653 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.653 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.654 INFO:teuthology.orchestra.run.smithi001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T02:04:16.654 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T02:04:16.654 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.654 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.654 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.654 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.654 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.654 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.654 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.654 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.654 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.654 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.654 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.655 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T02:04:16.655 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T02:04:16.655 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.655 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.655 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.655 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-04-26T02:04:16.655 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.655 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.655 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.655 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.655 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.655 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.655 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.656 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T02:04:16.656 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T02:04:16.656 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.656 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.656 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.656 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.656 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.656 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.656 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.656 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.656 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.656 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.656 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.656 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_URL": { 2024-04-26T02:04:16.656 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_URL", 2024-04-26T02:04:16.657 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.657 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.657 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.657 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.657 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.657 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.657 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.657 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.657 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.657 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.657 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.657 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T02:04:16.657 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T02:04:16.658 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.658 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.658 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.658 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-04-26T02:04:16.658 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.658 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.658 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.658 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.658 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.658 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.658 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.658 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T02:04:16.658 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T02:04:16.658 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.659 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.659 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.659 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.659 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.659 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.659 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.659 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.659 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.659 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.659 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.659 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T02:04:16.659 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T02:04:16.659 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.659 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.660 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.660 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.660 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.660 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.660 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.660 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.660 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.660 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.660 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.660 INFO:teuthology.orchestra.run.smithi001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T02:04:16.660 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T02:04:16.660 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.660 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.660 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.661 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.661 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.661 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.661 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.661 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.661 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.661 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.661 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.661 INFO:teuthology.orchestra.run.smithi001.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T02:04:16.661 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T02:04:16.661 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.661 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.661 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.661 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.662 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.662 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.662 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.662 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.662 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.662 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.662 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.662 INFO:teuthology.orchestra.run.smithi001.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T02:04:16.662 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T02:04:16.662 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.662 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.662 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.662 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "[]", 2024-04-26T02:04:16.662 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.662 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.663 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.663 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.663 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.663 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.663 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.663 INFO:teuthology.orchestra.run.smithi001.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T02:04:16.663 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T02:04:16.663 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.663 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.663 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.663 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "{}", 2024-04-26T02:04:16.663 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.663 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.663 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.664 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.664 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.664 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.664 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.664 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T02:04:16.664 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T02:04:16.664 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.664 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.664 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.664 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.664 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.664 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.664 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.664 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.664 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.665 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.665 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.665 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T02:04:16.665 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T02:04:16.665 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.665 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.665 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.665 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.665 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.665 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.665 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.665 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.665 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.665 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.666 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.666 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T02:04:16.666 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T02:04:16.666 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.666 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.666 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.666 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.666 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.666 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.666 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.666 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.666 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.667 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.667 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.667 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T02:04:16.667 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T02:04:16.667 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.667 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.667 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.667 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.667 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.667 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.667 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.667 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.667 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.667 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.667 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.668 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T02:04:16.668 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T02:04:16.668 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.668 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.668 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.668 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.668 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.668 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.668 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.668 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.668 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.668 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.668 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T02:04:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T02:04:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T02:04:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T02:04:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T02:04:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T02:04:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T02:04:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T02:04:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T02:04:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T02:04:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T02:04:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T02:04:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T02:04:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T02:04:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T02:04:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T02:04:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T02:04:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8", 2024-04-26T02:04:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T02:04:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T02:04:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "45", 2024-04-26T02:04:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T02:04:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T02:04:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T02:04:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T02:04:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-04-26T02:04:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T02:04:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T02:04:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T02:04:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T02:04:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T02:04:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T02:04:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T02:04:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T02:04:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.682 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.682 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-04-26T02:04:16.682 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.682 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.682 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.682 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.682 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.682 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.682 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.682 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T02:04:16.682 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T02:04:16.682 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.682 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.682 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.683 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:16.683 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.683 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.683 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.683 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.683 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.683 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.683 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.683 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T02:04:16.683 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T02:04:16.683 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.683 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.683 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.683 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-04-26T02:04:16.684 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.684 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.684 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.684 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.684 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.684 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.684 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.684 INFO:teuthology.orchestra.run.smithi001.stdout: "cross_origin_url": { 2024-04-26T02:04:16.684 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cross_origin_url", 2024-04-26T02:04:16.684 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.684 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.684 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.685 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.685 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.685 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.685 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.685 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.685 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.685 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.685 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.685 INFO:teuthology.orchestra.run.smithi001.stdout: "crt_file": { 2024-04-26T02:04:16.685 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crt_file", 2024-04-26T02:04:16.685 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.685 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.685 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.685 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.685 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.686 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.686 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.686 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.686 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.686 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.686 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.686 INFO:teuthology.orchestra.run.smithi001.stdout: "debug": { 2024-04-26T02:04:16.686 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "debug", 2024-04-26T02:04:16.686 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.686 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.686 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.686 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.686 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.686 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.687 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.687 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable/disable debug options", 2024-04-26T02:04:16.687 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.687 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.687 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.687 INFO:teuthology.orchestra.run.smithi001.stdout: "jwt_token_ttl": { 2024-04-26T02:04:16.687 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "jwt_token_ttl", 2024-04-26T02:04:16.687 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.687 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.687 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.687 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "28800", 2024-04-26T02:04:16.687 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.687 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.687 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.688 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.688 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.688 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.688 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.688 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-04-26T02:04:16.688 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-04-26T02:04:16.688 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.688 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.688 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.688 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.688 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.688 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.688 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.688 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.688 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.689 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.690 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.691 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.692 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "motd": { 2024-04-26T02:04:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "motd", 2024-04-26T02:04:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.693 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "The message of the day", 2024-04-26T02:04:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect_resolve_ip_addr": { 2024-04-26T02:04:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T02:04:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.694 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-04-26T02:04:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-04-26T02:04:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.695 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-04-26T02:04:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-04-26T02:04:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-04-26T02:04:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.696 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8080", 2024-04-26T02:04:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-04-26T02:04:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-04-26T02:04:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.697 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl_server_port": { 2024-04-26T02:04:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl_server_port", 2024-04-26T02:04:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8443", 2024-04-26T02:04:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.698 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-04-26T02:04:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-04-26T02:04:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "redirect", 2024-04-26T02:04:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.699 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect" 2024-04-26T02:04:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-04-26T02:04:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-04-26T02:04:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-04-26T02:04:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-04-26T02:04:16.700 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-04-26T02:04:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "url_prefix": { 2024-04-26T02:04:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url_prefix", 2024-04-26T02:04:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-04-26T02:04:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.702 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_monitoring": { 2024-04-26T02:04:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_monitoring", 2024-04-26T02:04:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "monitor device health metrics", 2024-04-26T02:04:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.703 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.704 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.705 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.706 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.707 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "mark_out_threshold": { 2024-04-26T02:04:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mark_out_threshold", 2024-04-26T02:04:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.708 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2419200", 2024-04-26T02:04:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T02:04:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": { 2024-04-26T02:04:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_name", 2024-04-26T02:04:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.709 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "device_health_metrics", 2024-04-26T02:04:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T02:04:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "retention_period": { 2024-04-26T02:04:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retention_period", 2024-04-26T02:04:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.710 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15552000", 2024-04-26T02:04:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain device health metrics", 2024-04-26T02:04:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_frequency": { 2024-04-26T02:04:16.711 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_frequency", 2024-04-26T02:04:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-04-26T02:04:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T02:04:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: "self_heal": { 2024-04-26T02:04:16.712 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "self_heal", 2024-04-26T02:04:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T02:04:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-04-26T02:04:16.713 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-04-26T02:04:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-04-26T02:04:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T02:04:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_threshold": { 2024-04-26T02:04:16.714 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_threshold", 2024-04-26T02:04:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7257600", 2024-04-26T02:04:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T02:04:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.715 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "diskprediction_local", 2024-04-26T02:04:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.716 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.717 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.718 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.719 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: "predict_interval": { 2024-04-26T02:04:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predict_interval", 2024-04-26T02:04:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-04-26T02:04:16.720 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "predictor_model": { 2024-04-26T02:04:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predictor_model", 2024-04-26T02:04:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "prophetstor", 2024-04-26T02:04:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.721 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-04-26T02:04:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-04-26T02:04:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-04-26T02:04:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.722 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "influx", 2024-04-26T02:04:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": false, 2024-04-26T02:04:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "influxdb python module not found", 2024-04-26T02:04:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "batch_size": { 2024-04-26T02:04:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "batch_size", 2024-04-26T02:04:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.723 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5000", 2024-04-26T02:04:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T02:04:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "database": { 2024-04-26T02:04:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "database", 2024-04-26T02:04:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.724 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "ceph", 2024-04-26T02:04:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T02:04:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": { 2024-04-26T02:04:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hostname", 2024-04-26T02:04:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.725 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server hostname", 2024-04-26T02:04:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-04-26T02:04:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-04-26T02:04:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "30", 2024-04-26T02:04:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "5", 2024-04-26T02:04:16.726 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T02:04:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.727 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.728 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.729 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.730 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: "password": { 2024-04-26T02:04:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "password", 2024-04-26T02:04:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "password of InfluxDB server user", 2024-04-26T02:04:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.732 INFO:teuthology.orchestra.run.smithi001.stdout: "port": { 2024-04-26T02:04:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "port", 2024-04-26T02:04:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8086", 2024-04-26T02:04:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server port", 2024-04-26T02:04:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-04-26T02:04:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-04-26T02:04:16.733 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "false", 2024-04-26T02:04:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T02:04:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "threads": { 2024-04-26T02:04:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threads", 2024-04-26T02:04:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.734 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-04-26T02:04:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-04-26T02:04:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "32", 2024-04-26T02:04:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T02:04:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "username": { 2024-04-26T02:04:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "username", 2024-04-26T02:04:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.735 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "username of InfluxDB server user", 2024-04-26T02:04:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "verify_ssl": { 2024-04-26T02:04:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "verify_ssl", 2024-04-26T02:04:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "true", 2024-04-26T02:04:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.736 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T02:04:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "insights", 2024-04-26T02:04:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.737 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.738 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "iostat", 2024-04-26T02:04:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "k8sevents", 2024-04-26T02:04:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_event_retention_days": { 2024-04-26T02:04:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ceph_event_retention_days", 2024-04-26T02:04:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7", 2024-04-26T02:04:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T02:04:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "config_check_secs": { 2024-04-26T02:04:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_check_secs", 2024-04-26T02:04:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "10", 2024-04-26T02:04:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T02:04:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.753 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.754 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "localpool", 2024-04-26T02:04:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "failure_domain": { 2024-04-26T02:04:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "failure_domain", 2024-04-26T02:04:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "host", 2024-04-26T02:04:16.755 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "failure domain for any created local pool", 2024-04-26T02:04:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T02:04:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.756 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.757 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.758 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.759 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.760 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": { 2024-04-26T02:04:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_size", 2024-04-26T02:04:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default min_size for any created local pool", 2024-04-26T02:04:16.761 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T02:04:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "num_rep": { 2024-04-26T02:04:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "num_rep", 2024-04-26T02:04:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3", 2024-04-26T02:04:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.762 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default replica count for any created local pool", 2024-04-26T02:04:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": { 2024-04-26T02:04:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_num", 2024-04-26T02:04:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "128", 2024-04-26T02:04:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.763 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T02:04:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "prefix": { 2024-04-26T02:04:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prefix", 2024-04-26T02:04:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.764 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name prefix for any created local pool", 2024-04-26T02:04:16.765 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.765 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.765 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.765 INFO:teuthology.orchestra.run.smithi001.stdout: "subtree": { 2024-04-26T02:04:16.765 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "subtree", 2024-04-26T02:04:16.765 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.765 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.765 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.765 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "rack", 2024-04-26T02:04:16.765 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.765 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.765 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.765 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T02:04:16.765 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T02:04:16.765 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.766 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.766 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.766 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.766 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.766 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mds_autoscaler", 2024-04-26T02:04:16.766 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.766 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.766 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.766 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.766 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.766 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.766 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.766 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.767 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.767 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.767 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.767 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.767 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.767 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.767 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.767 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.767 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.767 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.767 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.767 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.767 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.767 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.767 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.768 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.768 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.768 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.768 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.768 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.768 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.768 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.768 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.768 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.768 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.768 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.768 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.768 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.768 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.769 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.769 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.769 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.769 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.769 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.769 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.769 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.769 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.769 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.769 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.769 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.769 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.769 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.769 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.769 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.770 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.770 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.770 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.770 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.770 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.770 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.770 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.770 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.770 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.770 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.770 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.770 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.770 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.770 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.771 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.771 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.771 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.771 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.771 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.771 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirroring", 2024-04-26T02:04:16.771 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.771 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.771 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.771 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.771 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.771 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.771 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.771 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.772 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.772 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.772 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.772 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.772 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.772 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.772 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.772 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.772 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.772 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.772 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.772 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.772 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.772 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.772 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.773 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.773 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.773 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.773 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.773 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.773 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.773 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.773 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.773 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.773 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.773 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.773 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.773 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.774 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.774 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.774 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.774 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.774 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.774 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.774 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.774 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.774 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.774 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.774 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.774 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.774 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.775 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.775 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.775 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.775 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.775 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.775 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.775 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.775 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.775 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.775 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.775 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.775 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.775 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.775 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.775 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.776 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.776 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.776 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.776 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.776 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.776 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.776 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.776 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "nfs", 2024-04-26T02:04:16.776 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.776 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.776 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.776 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.777 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.777 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.777 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.777 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.777 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.777 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.777 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.777 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.777 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.777 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.777 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.777 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.777 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.777 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.777 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.778 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.778 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.778 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.778 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.778 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.778 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.778 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.778 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.778 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.778 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.778 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.778 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.778 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.779 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.779 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.779 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.779 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.779 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.779 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.779 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.779 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.779 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.779 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.779 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.779 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.779 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.779 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.779 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.780 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.780 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.780 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.780 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.780 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.780 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.780 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.780 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.780 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.780 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.780 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.780 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.780 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.780 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.780 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.781 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.781 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.781 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.781 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.781 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.781 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.781 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.781 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.781 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.781 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.781 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-04-26T02:04:16.781 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.781 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.781 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.781 INFO:teuthology.orchestra.run.smithi001.stdout: "fail_fs": { 2024-04-26T02:04:16.782 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "fail_fs", 2024-04-26T02:04:16.782 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.782 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.782 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.782 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.782 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.782 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.782 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.782 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T02:04:16.782 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.782 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.782 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.782 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.782 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.783 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.783 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.783 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.783 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.783 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.783 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.783 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.783 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.783 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.783 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.783 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.783 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.783 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.783 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.783 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.783 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.783 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.784 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.784 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.784 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.784 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.784 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.784 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.784 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.784 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.784 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.784 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.784 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.784 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.784 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.784 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.785 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.785 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.785 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.785 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.785 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.785 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.785 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.785 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.785 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.785 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.785 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.785 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.785 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.785 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.786 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.786 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.786 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.786 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.786 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.786 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.786 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.786 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.786 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.786 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.786 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.786 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.786 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.787 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.787 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.787 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.787 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.787 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.787 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.787 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator": { 2024-04-26T02:04:16.787 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-04-26T02:04:16.787 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.787 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.787 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.787 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.787 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.787 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.788 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.788 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-04-26T02:04:16.788 INFO:teuthology.orchestra.run.smithi001.stdout: "rook", 2024-04-26T02:04:16.788 INFO:teuthology.orchestra.run.smithi001.stdout: "test_orchestrator" 2024-04-26T02:04:16.788 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.788 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Orchestrator backend", 2024-04-26T02:04:16.788 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.788 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.788 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.788 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.788 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.788 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_perf_query", 2024-04-26T02:04:16.788 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.788 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.789 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.789 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.789 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.789 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.789 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.789 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.789 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.789 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.789 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.789 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.789 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.789 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.789 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.789 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.790 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.790 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.790 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.790 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.790 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.790 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.790 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.790 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.790 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.790 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.790 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.790 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.790 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.791 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.791 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.791 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.791 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.791 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.791 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.791 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.791 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.791 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.791 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.791 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.791 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.791 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.791 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.792 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.792 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.792 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.792 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.792 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.792 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.792 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.792 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.792 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.792 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.792 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.792 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.792 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.792 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.792 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.793 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.793 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.793 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.793 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.793 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.793 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.793 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.793 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.793 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.793 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.793 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.793 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.793 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.794 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_support", 2024-04-26T02:04:16.794 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.794 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.794 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.794 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.794 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.794 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.794 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.794 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.794 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.794 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.794 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.794 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.794 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.794 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.795 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.795 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.795 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.795 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.795 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.795 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.795 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.795 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.795 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.795 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.795 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.795 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.795 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.795 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.796 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.796 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.796 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.796 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.796 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.796 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.796 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.796 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.796 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.796 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.796 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.796 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.796 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.796 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.797 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.797 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.797 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.797 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.797 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.797 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.797 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.797 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.797 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.797 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.797 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.797 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.797 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.797 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.797 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.798 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.798 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.798 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.798 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.798 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.798 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.798 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.798 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.798 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.798 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.798 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.798 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.798 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.798 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.799 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_autoscaler", 2024-04-26T02:04:16.799 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.799 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.799 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.799 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.799 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.799 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.799 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.799 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.799 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.799 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.799 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.799 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.799 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.799 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.800 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.800 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.800 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.800 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.800 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.800 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.800 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.800 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.800 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.800 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.800 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.800 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.800 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.800 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.800 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.801 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.801 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.801 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.801 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.801 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.801 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.801 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.801 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.801 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.801 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.801 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.801 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.801 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.801 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.801 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.802 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.802 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.802 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.802 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.802 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.802 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.802 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.802 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.802 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.802 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.802 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.802 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.802 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-04-26T02:04:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-04-26T02:04:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.803 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.804 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.804 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-04-26T02:04:16.804 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.804 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.804 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.804 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.804 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.804 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.804 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.804 INFO:teuthology.orchestra.run.smithi001.stdout: "threshold": { 2024-04-26T02:04:16.804 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threshold", 2024-04-26T02:04:16.804 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-04-26T02:04:16.804 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.804 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.805 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-04-26T02:04:16.805 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1.0", 2024-04-26T02:04:16.805 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.805 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.805 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "scaling threshold", 2024-04-26T02:04:16.805 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T02:04:16.805 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.805 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.805 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.805 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.805 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "progress", 2024-04-26T02:04:16.806 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.806 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.806 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.806 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_pg_recovery_event": { 2024-04-26T02:04:16.806 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_pg_recovery_event", 2024-04-26T02:04:16.806 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.806 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.806 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.806 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.806 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.806 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.806 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.806 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T02:04:16.806 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.806 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.806 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.807 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-04-26T02:04:16.807 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-04-26T02:04:16.807 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.807 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.807 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.807 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.807 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.807 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.807 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.807 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.807 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.807 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.807 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.807 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.809 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.809 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.809 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.809 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.809 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.809 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.809 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.809 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.809 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.809 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.809 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.809 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.809 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.809 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.809 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.810 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.810 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.810 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.810 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.810 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.810 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.810 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.810 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.810 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.810 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.810 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.810 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.810 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.810 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.811 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.812 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.812 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.812 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.812 INFO:teuthology.orchestra.run.smithi001.stdout: "max_completed_events": { 2024-04-26T02:04:16.812 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_completed_events", 2024-04-26T02:04:16.812 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.812 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.812 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.812 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "50", 2024-04-26T02:04:16.812 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.812 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.812 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.812 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "number of past completed events to remember", 2024-04-26T02:04:16.812 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.812 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-04-26T02:04:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-04-26T02:04:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-04-26T02:04:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long the module is going to sleep", 2024-04-26T02:04:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.813 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.814 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.814 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus", 2024-04-26T02:04:16.814 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.814 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.814 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.814 INFO:teuthology.orchestra.run.smithi001.stdout: "cache": { 2024-04-26T02:04:16.814 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cache", 2024-04-26T02:04:16.814 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.814 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.814 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.814 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.814 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.814 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.814 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.815 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.815 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.815 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.815 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.815 INFO:teuthology.orchestra.run.smithi001.stdout: "exclude_perf_counters": { 2024-04-26T02:04:16.815 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "exclude_perf_counters", 2024-04-26T02:04:16.815 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.815 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.815 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.815 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.815 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.815 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.815 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.815 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T02:04:16.816 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T02:04:16.816 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.816 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.816 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.816 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.816 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.816 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.816 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.816 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.816 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.816 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.816 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.816 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.816 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.817 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.817 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.817 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.817 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.817 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.817 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.817 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.817 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.817 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.817 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.817 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.817 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.817 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.817 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.818 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.820 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools": { 2024-04-26T02:04:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools", 2024-04-26T02:04:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T02:04:16.821 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T02:04:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300", 2024-04-26T02:04:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_interval": { 2024-04-26T02:04:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_interval", 2024-04-26T02:04:16.822 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-04-26T02:04:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15.0", 2024-04-26T02:04:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-04-26T02:04:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-04-26T02:04:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.823 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-04-26T02:04:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T02:04:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-04-26T02:04:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-04-26T02:04:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.824 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "9283", 2024-04-26T02:04:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T02:04:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "stale_cache_strategy": { 2024-04-26T02:04:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stale_cache_strategy", 2024-04-26T02:04:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.825 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "log", 2024-04-26T02:04:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-04-26T02:04:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-04-26T02:04:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.826 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "default", 2024-04-26T02:04:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "default", 2024-04-26T02:04:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "error" 2024-04-26T02:04:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-04-26T02:04:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-04-26T02:04:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.827 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-04-26T02:04:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-04-26T02:04:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-04-26T02:04:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-04-26T02:04:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.828 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.829 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.831 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.832 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_snap_create": { 2024-04-26T02:04:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_snap_create", 2024-04-26T02:04:16.833 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-04-26T02:04:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "mirror_snapshot_schedule": { 2024-04-26T02:04:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T02:04:16.834 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "trash_purge_schedule": { 2024-04-26T02:04:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "trash_purge_schedule", 2024-04-26T02:04:16.835 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.836 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "restful", 2024-04-26T02:04:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_auth": { 2024-04-26T02:04:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_auth", 2024-04-26T02:04:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.837 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-04-26T02:04:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-04-26T02:04:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.838 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.839 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.840 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.841 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.842 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.843 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-04-26T02:04:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-04-26T02:04:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.844 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-04-26T02:04:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-04-26T02:04:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.845 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rgw", 2024-04-26T02:04:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.846 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.847 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.848 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.849 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.850 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rook", 2024-04-26T02:04:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.851 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.852 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.853 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.854 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "storage_class": { 2024-04-26T02:04:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "storage_class", 2024-04-26T02:04:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "local", 2024-04-26T02:04:16.856 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T02:04:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "selftest", 2024-04-26T02:04:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "roption1": { 2024-04-26T02:04:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption1", 2024-04-26T02:04:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "roption2": { 2024-04-26T02:04:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption2", 2024-04-26T02:04:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "xyz", 2024-04-26T02:04:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption1": { 2024-04-26T02:04:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption1", 2024-04-26T02:04:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption2": { 2024-04-26T02:04:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption2", 2024-04-26T02:04:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption3": { 2024-04-26T02:04:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption3", 2024-04-26T02:04:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-04-26T02:04:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption4": { 2024-04-26T02:04:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption4", 2024-04-26T02:04:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption5": { 2024-04-26T02:04:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption5", 2024-04-26T02:04:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption6": { 2024-04-26T02:04:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption6", 2024-04-26T02:04:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption7": { 2024-04-26T02:04:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption7", 2024-04-26T02:04:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-04-26T02:04:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "42", 2024-04-26T02:04:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "testkey": { 2024-04-26T02:04:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testkey", 2024-04-26T02:04:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "testlkey": { 2024-04-26T02:04:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testlkey", 2024-04-26T02:04:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "testnewline": { 2024-04-26T02:04:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testnewline", 2024-04-26T02:04:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-04-26T02:04:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_m_granularity": { 2024-04-26T02:04:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_m_granularity", 2024-04-26T02:04:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.874 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T02:04:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "dump_on_update": { 2024-04-26T02:04:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dump_on_update", 2024-04-26T02:04:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.875 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "dump database to debug log on update", 2024-04-26T02:04:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.876 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.877 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.878 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.879 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.880 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stats", 2024-04-26T02:04:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.881 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.882 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.885 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.886 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.887 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.888 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "status", 2024-04-26T02:04:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.889 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.890 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.891 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.892 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.893 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telegraf", 2024-04-26T02:04:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.894 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "address": { 2024-04-26T02:04:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "address", 2024-04-26T02:04:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T02:04:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.895 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-04-26T02:04:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-04-26T02:04:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15", 2024-04-26T02:04:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.896 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.897 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.898 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.899 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.900 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telemetry", 2024-04-26T02:04:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_basic": { 2024-04-26T02:04:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_basic", 2024-04-26T02:04:16.901 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T02:04:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_crash": { 2024-04-26T02:04:16.902 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_crash", 2024-04-26T02:04:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T02:04:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_device": { 2024-04-26T02:04:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_device", 2024-04-26T02:04:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.903 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T02:04:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_ident": { 2024-04-26T02:04:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_ident", 2024-04-26T02:04:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.904 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T02:04:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_perf": { 2024-04-26T02:04:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_perf", 2024-04-26T02:04:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.905 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T02:04:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "contact": { 2024-04-26T02:04:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "contact", 2024-04-26T02:04:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.906 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "description": { 2024-04-26T02:04:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "description", 2024-04-26T02:04:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.907 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "device_url": { 2024-04-26T02:04:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_url", 2024-04-26T02:04:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T02:04:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.908 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-04-26T02:04:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-04-26T02:04:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.909 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-04-26T02:04:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-04-26T02:04:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "24", 2024-04-26T02:04:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "8", 2024-04-26T02:04:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.910 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "last_opt_revision": { 2024-04-26T02:04:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "last_opt_revision", 2024-04-26T02:04:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1", 2024-04-26T02:04:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.911 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.912 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.912 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.912 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.912 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard": { 2024-04-26T02:04:16.912 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard", 2024-04-26T02:04:16.912 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.912 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.912 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.912 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.912 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.912 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.912 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.912 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.912 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.913 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.913 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.913 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard_description": { 2024-04-26T02:04:16.913 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard_description", 2024-04-26T02:04:16.913 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.913 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.913 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.913 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.913 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.913 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.913 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.913 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.913 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.913 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.913 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.914 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.914 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.914 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.914 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.914 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.914 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.914 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.914 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.914 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.914 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.914 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.914 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.914 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.914 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.915 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.915 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.915 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.915 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.915 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.915 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.915 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.915 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.915 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.915 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.915 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.915 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.915 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.915 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.916 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.916 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.916 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.916 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.916 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.916 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.916 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.916 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.916 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.916 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.916 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.916 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.916 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.916 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.917 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.917 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.917 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.917 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.917 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.917 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.917 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.917 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.917 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.917 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.917 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.917 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.917 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.917 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.918 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.918 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.918 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.918 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.918 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.918 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.918 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.918 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.918 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.918 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.918 INFO:teuthology.orchestra.run.smithi001.stdout: "organization": { 2024-04-26T02:04:16.918 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "organization", 2024-04-26T02:04:16.918 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.919 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.919 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.919 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.919 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.919 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.919 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.919 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.919 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.919 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.919 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.919 INFO:teuthology.orchestra.run.smithi001.stdout: "proxy": { 2024-04-26T02:04:16.919 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "proxy", 2024-04-26T02:04:16.919 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.919 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.920 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.920 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.920 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.920 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.920 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.920 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.920 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.920 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.920 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.920 INFO:teuthology.orchestra.run.smithi001.stdout: "url": { 2024-04-26T02:04:16.920 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url", 2024-04-26T02:04:16.920 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.920 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.921 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.921 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T02:04:16.921 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.921 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.921 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.921 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.921 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.921 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.921 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.921 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.921 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.921 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.921 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "test_orchestrator", 2024-04-26T02:04:16.922 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.922 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.922 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.922 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.922 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.922 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.922 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.922 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.922 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.922 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.922 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.922 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.922 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.923 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.923 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.923 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.923 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.923 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.923 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.923 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.923 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.923 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.923 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.923 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.923 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.923 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.923 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.924 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.924 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.924 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.924 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.924 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.924 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.924 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.924 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.924 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.924 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.924 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.924 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.924 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.924 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.925 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.925 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.925 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.925 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.925 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.925 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.925 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.925 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.925 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.925 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.925 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.925 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.925 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.925 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.925 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.926 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.926 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.926 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.926 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.926 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.926 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.926 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.926 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.926 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.926 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.926 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.926 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.927 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.927 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.927 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.927 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-04-26T02:04:16.927 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.927 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.927 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.927 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.927 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.927 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.927 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.927 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.927 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.927 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.928 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.928 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.928 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.928 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.928 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.928 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.928 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.928 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.928 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.928 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.928 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.928 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.928 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.928 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.929 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.930 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.930 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.930 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.930 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.930 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.930 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.930 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.930 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.930 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.930 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.930 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.930 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.930 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.930 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.931 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.931 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.931 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.931 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.931 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.931 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.931 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.931 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.931 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.931 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.931 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.931 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.931 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.931 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.931 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.932 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.932 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.932 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_clones": { 2024-04-26T02:04:16.932 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_clones", 2024-04-26T02:04:16.932 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.932 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.932 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.932 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4", 2024-04-26T02:04:16.932 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.932 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.932 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.932 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T02:04:16.932 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.932 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.933 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.933 INFO:teuthology.orchestra.run.smithi001.stdout: "periodic_async_work": { 2024-04-26T02:04:16.933 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "periodic_async_work", 2024-04-26T02:04:16.933 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.933 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.933 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.933 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.933 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.933 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.933 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.933 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Periodically check for async work", 2024-04-26T02:04:16.933 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.933 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.933 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.934 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_delay": { 2024-04-26T02:04:16.934 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_delay", 2024-04-26T02:04:16.934 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.934 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.934 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.934 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-04-26T02:04:16.934 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.934 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.934 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.934 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T02:04:16.934 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.934 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.934 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.935 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_no_wait": { 2024-04-26T02:04:16.935 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T02:04:16.935 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.935 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.935 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.935 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-04-26T02:04:16.935 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.935 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.935 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.935 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T02:04:16.935 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.935 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.935 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.935 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.936 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.936 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.936 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix", 2024-04-26T02:04:16.936 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-04-26T02:04:16.936 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-04-26T02:04:16.936 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-04-26T02:04:16.936 INFO:teuthology.orchestra.run.smithi001.stdout: "discovery_interval": { 2024-04-26T02:04:16.936 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "discovery_interval", 2024-04-26T02:04:16.936 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-04-26T02:04:16.936 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.936 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.936 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "100", 2024-04-26T02:04:16.936 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.936 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.936 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.937 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.937 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.937 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.937 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.937 INFO:teuthology.orchestra.run.smithi001.stdout: "identifier": { 2024-04-26T02:04:16.937 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "identifier", 2024-04-26T02:04:16.937 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.937 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.937 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.937 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.937 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.937 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.937 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.937 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.937 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.938 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.938 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.938 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-04-26T02:04:16.938 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-04-26T02:04:16.938 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-04-26T02:04:16.938 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.938 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.938 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-04-26T02:04:16.938 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.938 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.938 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.938 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.938 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.939 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.939 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.939 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-04-26T02:04:16.939 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-04-26T02:04:16.939 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.939 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.939 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.939 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.939 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.939 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.939 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.939 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.939 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.940 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.940 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.940 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.940 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.940 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.940 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.940 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.940 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.940 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.940 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-04-26T02:04:16.940 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-04-26T02:04:16.940 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.940 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.940 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.941 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.941 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.941 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.941 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.941 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.941 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.941 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.941 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.941 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-04-26T02:04:16.941 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-04-26T02:04:16.941 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.941 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.941 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.941 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-04-26T02:04:16.942 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.942 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.942 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-04-26T02:04:16.942 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-04-26T02:04:16.942 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-04-26T02:04:16.942 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-04-26T02:04:16.942 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-04-26T02:04:16.942 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-04-26T02:04:16.942 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-04-26T02:04:16.942 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.942 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.942 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.942 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.942 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.943 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-04-26T02:04:16.943 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-04-26T02:04:16.943 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-04-26T02:04:16.943 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.943 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-04-26T02:04:16.943 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-04-26T02:04:16.943 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.943 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.943 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.943 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.943 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.943 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.943 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.944 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_host": { 2024-04-26T02:04:16.944 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_host", 2024-04-26T02:04:16.944 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.944 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.944 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.944 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-04-26T02:04:16.944 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.944 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.944 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.944 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.944 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.944 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.944 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.945 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_port": { 2024-04-26T02:04:16.945 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_port", 2024-04-26T02:04:16.945 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-04-26T02:04:16.945 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.945 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.945 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10051", 2024-04-26T02:04:16.945 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.945 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.945 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.945 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.945 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.945 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.945 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.946 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_sender": { 2024-04-26T02:04:16.946 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_sender", 2024-04-26T02:04:16.946 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-04-26T02:04:16.946 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-04-26T02:04:16.946 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-04-26T02:04:16.946 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T02:04:16.946 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-04-26T02:04:16.946 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-04-26T02:04:16.946 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-04-26T02:04:16.946 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-04-26T02:04:16.946 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-04-26T02:04:16.946 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-04-26T02:04:16.946 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-04-26T02:04:16.947 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.947 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.947 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.947 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.947 INFO:teuthology.orchestra.run.smithi001.stdout: "services": { 2024-04-26T02:04:16.947 INFO:teuthology.orchestra.run.smithi001.stdout: "dashboard": "https://172.21.15.1:8443/" 2024-04-26T02:04:16.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.947 INFO:teuthology.orchestra.run.smithi001.stdout: "always_on_modules": { 2024-04-26T02:04:16.947 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus": [ 2024-04-26T02:04:16.947 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-04-26T02:04:16.947 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-04-26T02:04:16.947 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-04-26T02:04:16.947 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-04-26T02:04:16.948 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-04-26T02:04:16.948 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-04-26T02:04:16.948 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-04-26T02:04:16.948 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-04-26T02:04:16.948 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-04-26T02:04:16.948 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-04-26T02:04:16.948 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.948 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific": [ 2024-04-26T02:04:16.948 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-04-26T02:04:16.948 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-04-26T02:04:16.948 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-04-26T02:04:16.948 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-04-26T02:04:16.948 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-04-26T02:04:16.948 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-04-26T02:04:16.949 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-04-26T02:04:16.949 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-04-26T02:04:16.949 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-04-26T02:04:16.949 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-04-26T02:04:16.949 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.949 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy": [ 2024-04-26T02:04:16.949 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-04-26T02:04:16.949 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-04-26T02:04:16.949 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-04-26T02:04:16.949 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-04-26T02:04:16.949 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-04-26T02:04:16.949 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-04-26T02:04:16.949 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-04-26T02:04:16.949 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-04-26T02:04:16.949 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-04-26T02:04:16.950 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-04-26T02:04:16.950 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.950 INFO:teuthology.orchestra.run.smithi001.stdout: "reef": [ 2024-04-26T02:04:16.950 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-04-26T02:04:16.950 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-04-26T02:04:16.950 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-04-26T02:04:16.950 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-04-26T02:04:16.950 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-04-26T02:04:16.950 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-04-26T02:04:16.950 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-04-26T02:04:16.950 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-04-26T02:04:16.950 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-04-26T02:04:16.950 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-04-26T02:04:16.950 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-04-26T02:04:16.951 INFO:teuthology.orchestra.run.smithi001.stdout: "squid": [ 2024-04-26T02:04:16.951 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-04-26T02:04:16.951 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-04-26T02:04:16.951 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-04-26T02:04:16.951 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-04-26T02:04:16.951 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-04-26T02:04:16.951 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-04-26T02:04:16.951 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-04-26T02:04:16.951 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-04-26T02:04:16.951 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-04-26T02:04:16.951 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-04-26T02:04:16.951 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-04-26T02:04:16.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.951 INFO:teuthology.orchestra.run.smithi001.stdout: "last_failure_osd_epoch": 93, 2024-04-26T02:04:16.951 INFO:teuthology.orchestra.run.smithi001.stdout: "active_clients": [ 2024-04-26T02:04:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-04-26T02:04:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-04-26T02:04:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-04-26T02:04:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-04-26T02:04:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1108980090 2024-04-26T02:04:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-04-26T02:04:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "libcephsqlite", 2024-04-26T02:04:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-04-26T02:04:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-04-26T02:04:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-04-26T02:04:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1153327574 2024-04-26T02:04:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-04-26T02:04:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-04-26T02:04:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-04-26T02:04:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-04-26T02:04:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-04-26T02:04:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2558271520 2024-04-26T02:04:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-04-26T02:04:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-04-26T02:04:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-04-26T02:04:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-04-26T02:04:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-04-26T02:04:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2183939041 2024-04-26T02:04:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-04-26T02:04:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-04-26T02:04:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-04-26T02:04:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-04-26T02:04:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-04-26T02:04:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1032981508 2024-04-26T02:04:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-04-26T02:04:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-04-26T02:04:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-04-26T02:04:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-04-26T02:04:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-04-26T02:04:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-04-26T02:04:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4229665704 2024-04-26T02:04:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.956 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-04-26T02:04:16.956 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:16.956 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-04-26T02:04:16.956 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T02:04:16.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.699+0000 7f8790e00640 1 -- 172.21.15.1:0/4025841844 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f87700a3d20 msgr2=0x7f87700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:16.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.699+0000 7f8790e00640 1 --2- 172.21.15.1:0/4025841844 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f87700a3d20 0x7f87700a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f87740303f0 tx=0x7f8774030340 comp rx=0 tx=0).stop 2024-04-26T02:04:16.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.699+0000 7f8790e00640 1 -- 172.21.15.1:0/4025841844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f878c1043b0 msgr2=0x7f878c1a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:16.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.699+0000 7f8790e00640 1 --2- 172.21.15.1:0/4025841844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f878c1043b0 0x7f878c1a06b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f877c005fd0 tx=0x7f877c00ee50 comp rx=0 tx=0).stop 2024-04-26T02:04:16.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.699+0000 7f8790e00640 1 -- 172.21.15.1:0/4025841844 shutdown_connections 2024-04-26T02:04:16.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.699+0000 7f8790e00640 1 --2- 172.21.15.1:0/4025841844 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f87700a3d20 0x7f87700a61e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:16.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.699+0000 7f8790e00640 1 --2- 172.21.15.1:0/4025841844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f878c19a7c0 0x7f878c19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:16.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.699+0000 7f8790e00640 1 --2- 172.21.15.1:0/4025841844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f878c109480 0x7f878c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:16.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.699+0000 7f8790e00640 1 --2- 172.21.15.1:0/4025841844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f878c1043b0 0x7f878c1a06b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:16.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.699+0000 7f8790e00640 1 -- 172.21.15.1:0/4025841844 >> 172.21.15.1:0/4025841844 conn(0x7f878c0ffc10 msgr2=0x7f878c0ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:16.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.699+0000 7f8790e00640 1 -- 172.21.15.1:0/4025841844 shutdown_connections 2024-04-26T02:04:16.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:15.699+0000 7f8790e00640 1 -- 172.21.15.1:0/4025841844 wait complete. 2024-04-26T02:04:16.998 DEBUG:tasks.check_counter:Getting stats from a 2024-04-26T02:04:16.998 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-26T02:04:16.999 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:16 smithi001 ceph-mon[27822]: pgmap v396: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-04-26T02:04:16.999 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:16 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/4025841844' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T02:04:16.999 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:16 smithi173 ceph-mon[32553]: pgmap v396: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-04-26T02:04:17.000 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:16 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/4025841844' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T02:04:17.000 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:16 smithi097 ceph-mon[32853]: pgmap v396: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-04-26T02:04:17.000 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:16 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/4025841844' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T02:04:17.456 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:04:18.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:18 smithi173 ceph-mon[32553]: pgmap v397: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-04-26T02:04:18.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:18 smithi001 ceph-mon[27822]: pgmap v397: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-04-26T02:04:18.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:18 smithi097 ceph-mon[32853]: pgmap v397: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-04-26T02:04:19.817 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T02:04:19.817 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:04:19.817 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 4, 2024-04-26T02:04:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 4, 2024-04-26T02:04:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 3040, 2024-04-26T02:04:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 448, 2024-04-26T02:04:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-04-26T02:04:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2024-04-26T02:04:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.004973462, 2024-04-26T02:04:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000052154, 2024-04-26T02:04:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.001296126, 2024-04-26T02:04:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:04:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:19.818 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 3, 2024-04-26T02:04:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.001859016, 2024-04-26T02:04:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000619672 2024-04-26T02:04:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-04-26T02:04:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-04-26T02:04:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-04-26T02:04:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-26T02:04:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T02:04:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:04:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 287, 2024-04-26T02:04:19.819 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 276, 2024-04-26T02:04:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 1608368, 2024-04-26T02:04:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 170304, 2024-04-26T02:04:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2024-04-26T02:04:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2024-04-26T02:04:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.119349168, 2024-04-26T02:04:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.044987772, 2024-04-26T02:04:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.077542336, 2024-04-26T02:04:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.013778897, 2024-04-26T02:04:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 275, 2024-04-26T02:04:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.029399330, 2024-04-26T02:04:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000106906 2024-04-26T02:04:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:19.820 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-04-26T02:04:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-04-26T02:04:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-04-26T02:04:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 1608368, 2024-04-26T02:04:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 170304 2024-04-26T02:04:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:04:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 2, 2024-04-26T02:04:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 211, 2024-04-26T02:04:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 320, 2024-04-26T02:04:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 193344, 2024-04-26T02:04:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 5, 2024-04-26T02:04:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 3, 2024-04-26T02:04:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.043806283, 2024-04-26T02:04:19.821 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.032541949, 2024-04-26T02:04:19.822 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.005236701, 2024-04-26T02:04:19.822 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:04:19.822 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:19.822 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 208, 2024-04-26T02:04:19.822 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.048426832, 2024-04-26T02:04:19.822 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000232821 2024-04-26T02:04:19.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:19.822 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:19.822 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-04-26T02:04:19.822 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-04-26T02:04:19.822 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-04-26T02:04:19.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:19.822 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T02:04:19.822 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 193344 2024-04-26T02:04:19.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-04-26T02:04:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2024-04-26T02:04:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-04-26T02:04:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-04-26T02:04:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:04:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-04-26T02:04:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:04:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:04:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:04:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:04:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:04:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:04:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:04:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:04:19.823 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:04:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:04:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:04:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:04:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:04:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-04-26T02:04:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:04:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-04-26T02:04:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:04:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:04:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:04:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:04:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:04:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:04:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:04:19.824 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-04-26T02:04:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:04:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:04:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:04:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-04-26T02:04:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-04-26T02:04:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-04-26T02:04:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:04:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:04:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:04:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:04:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 149745, 2024-04-26T02:04:19.826 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 31, 2024-04-26T02:04:19.826 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:04:19.826 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-04-26T02:04:19.826 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-04-26T02:04:19.826 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-04-26T02:04:19.826 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:04:19.826 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-04-26T02:04:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:04:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-04-26T02:04:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2024-04-26T02:04:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2024-04-26T02:04:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:04:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:04:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-04-26T02:04:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-04-26T02:04:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2024-04-26T02:04:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2024-04-26T02:04:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-04-26T02:04:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-04-26T02:04:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-04-26T02:04:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-04-26T02:04:19.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:04:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-04-26T02:04:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-04-26T02:04:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-04-26T02:04:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 289, 2024-04-26T02:04:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 1567848, 2024-04-26T02:04:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 289, 2024-04-26T02:04:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-04-26T02:04:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-04-26T02:04:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 289, 2024-04-26T02:04:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 1567848, 2024-04-26T02:04:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-04-26T02:04:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-04-26T02:04:19.828 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-04-26T02:04:19.829 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-04-26T02:04:19.829 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:19.829 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:19.829 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T02:04:20.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:20 smithi173 ceph-mon[32553]: pgmap v398: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-04-26T02:04:20.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:20 smithi173 ceph-mon[32553]: osdmap e109: 12 total, 12 up, 12 in 2024-04-26T02:04:20.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:20 smithi001 ceph-mon[27822]: pgmap v398: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-04-26T02:04:20.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:20 smithi001 ceph-mon[27822]: osdmap e109: 12 total, 12 up, 12 in 2024-04-26T02:04:20.669 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-26T02:04:20.670 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-26T02:04:20.670 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-04-26T02:04:20.670 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-04-26T02:04:20.670 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-26T02:04:20.670 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-26T02:04:20.670 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-26T02:04:20.670 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-04-26T02:04:20.670 DEBUG:tasks.check_counter:Getting stats from d 2024-04-26T02:04:20.670 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-26T02:04:20.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:20 smithi097 ceph-mon[32853]: pgmap v398: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-04-26T02:04:20.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:20 smithi097 ceph-mon[32853]: osdmap e109: 12 total, 12 up, 12 in 2024-04-26T02:04:21.137 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:04:22.334 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:22 smithi001 ceph-mon[27822]: pgmap v400: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2024-04-26T02:04:22.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:22 smithi173 ceph-mon[32553]: pgmap v400: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2024-04-26T02:04:22.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:22 smithi097 ceph-mon[32853]: pgmap v400: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2024-04-26T02:04:23.257 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T02:04:23.257 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:04:23.258 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 4, 2024-04-26T02:04:23.258 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 4, 2024-04-26T02:04:23.258 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 3040, 2024-04-26T02:04:23.258 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 448, 2024-04-26T02:04:23.258 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-04-26T02:04:23.258 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2024-04-26T02:04:23.258 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.004892813, 2024-04-26T02:04:23.258 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000094458, 2024-04-26T02:04:23.258 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.001478568, 2024-04-26T02:04:23.258 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:04:23.258 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:23.258 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 3, 2024-04-26T02:04:23.258 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.001630683, 2024-04-26T02:04:23.258 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000543561 2024-04-26T02:04:23.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:23.259 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:23.259 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-04-26T02:04:23.259 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-04-26T02:04:23.259 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-04-26T02:04:23.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:23.259 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-26T02:04:23.259 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T02:04:23.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:23.259 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:04:23.259 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 2, 2024-04-26T02:04:23.259 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 216, 2024-04-26T02:04:23.259 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 320, 2024-04-26T02:04:23.259 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 194224, 2024-04-26T02:04:23.259 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 5, 2024-04-26T02:04:23.260 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 3, 2024-04-26T02:04:23.260 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.044883673, 2024-04-26T02:04:23.260 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.032869968, 2024-04-26T02:04:23.260 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.005703668, 2024-04-26T02:04:23.260 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:04:23.260 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:23.260 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 213, 2024-04-26T02:04:23.260 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.050530671, 2024-04-26T02:04:23.260 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000237233 2024-04-26T02:04:23.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:23.260 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:23.260 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-04-26T02:04:23.260 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-04-26T02:04:23.260 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-04-26T02:04:23.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:23.261 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T02:04:23.261 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 194224 2024-04-26T02:04:23.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:23.261 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:04:23.261 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 294, 2024-04-26T02:04:23.261 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 283, 2024-04-26T02:04:23.261 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 1610272, 2024-04-26T02:04:23.261 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 174768, 2024-04-26T02:04:23.261 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2024-04-26T02:04:23.261 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2024-04-26T02:04:23.261 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.127919228, 2024-04-26T02:04:23.261 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.044685084, 2024-04-26T02:04:23.261 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.090981186, 2024-04-26T02:04:23.261 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.013700688, 2024-04-26T02:04:23.261 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:23.262 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 282, 2024-04-26T02:04:23.262 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.029304321, 2024-04-26T02:04:23.262 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000103916 2024-04-26T02:04:23.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:23.262 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:23.262 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-04-26T02:04:23.262 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-04-26T02:04:23.262 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-04-26T02:04:23.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:23.262 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 1610272, 2024-04-26T02:04:23.262 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 174768 2024-04-26T02:04:23.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:23.262 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-04-26T02:04:23.262 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2024-04-26T02:04:23.263 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-04-26T02:04:23.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:23.263 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-04-26T02:04:23.263 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:04:23.263 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-04-26T02:04:23.263 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:04:23.263 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:04:23.263 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:04:23.263 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:04:23.263 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:04:23.263 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:04:23.263 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:04:23.263 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:04:23.263 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:04:23.264 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:04:23.264 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:04:23.264 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:04:23.264 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:04:23.264 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-04-26T02:04:23.264 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:04:23.264 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-04-26T02:04:23.264 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:04:23.264 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:04:23.264 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:04:23.264 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:04:23.264 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:04:23.264 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:04:23.264 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:04:23.265 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-04-26T02:04:23.265 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:04:23.265 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:04:23.265 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:04:23.265 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-04-26T02:04:23.265 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-04-26T02:04:23.265 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-04-26T02:04:23.265 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:04:23.265 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:04:23.265 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:04:23.265 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:04:23.265 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 150776, 2024-04-26T02:04:23.265 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 32, 2024-04-26T02:04:23.265 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:04:23.266 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-04-26T02:04:23.266 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-04-26T02:04:23.266 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-04-26T02:04:23.266 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:04:23.266 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-04-26T02:04:23.266 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:04:23.266 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-04-26T02:04:23.266 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2024-04-26T02:04:23.266 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2024-04-26T02:04:23.266 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:04:23.266 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:04:23.266 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-04-26T02:04:23.266 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-04-26T02:04:23.266 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2024-04-26T02:04:23.266 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2024-04-26T02:04:23.267 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-04-26T02:04:23.267 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-04-26T02:04:23.267 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-04-26T02:04:23.267 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-04-26T02:04:23.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:23.267 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:04:23.267 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-04-26T02:04:23.267 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-04-26T02:04:23.267 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-04-26T02:04:23.267 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 296, 2024-04-26T02:04:23.267 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 1568758, 2024-04-26T02:04:23.267 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:23.267 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 296, 2024-04-26T02:04:23.267 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-04-26T02:04:23.267 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-04-26T02:04:23.268 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 296, 2024-04-26T02:04:23.268 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 1568758, 2024-04-26T02:04:23.268 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-04-26T02:04:23.268 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-04-26T02:04:23.268 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-04-26T02:04:23.268 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-04-26T02:04:23.268 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:23.268 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:23.268 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T02:04:24.057 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-26T02:04:24.057 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-26T02:04:24.057 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-26T02:04:24.057 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-26T02:04:24.057 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-26T02:04:24.057 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-26T02:04:24.057 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-26T02:04:24.057 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-26T02:04:24.057 DEBUG:tasks.check_counter:Getting stats from g 2024-04-26T02:04:24.057 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-26T02:04:24.529 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:04:24.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:24 smithi173 ceph-mon[32553]: pgmap v401: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2024-04-26T02:04:24.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:24 smithi001 ceph-mon[27822]: pgmap v401: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2024-04-26T02:04:24.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:24 smithi097 ceph-mon[32853]: pgmap v401: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2024-04-26T02:04:26.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:26 smithi001 ceph-mon[27822]: pgmap v402: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T02:04:26.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:26 smithi097 ceph-mon[32853]: pgmap v402: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T02:04:26.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:26 smithi173 ceph-mon[32553]: pgmap v402: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T02:04:26.828 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T02:04:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:04:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 0, 2024-04-26T02:04:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 2, 2024-04-26T02:04:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 0, 2024-04-26T02:04:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 0, 2024-04-26T02:04:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-04-26T02:04:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2024-04-26T02:04:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.004809321, 2024-04-26T02:04:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T02:04:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.000880773, 2024-04-26T02:04:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:04:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-04-26T02:04:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-04-26T02:04:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-04-26T02:04:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-04-26T02:04:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-04-26T02:04:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-04-26T02:04:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T02:04:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T02:04:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:04:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 6, 2024-04-26T02:04:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 215, 2024-04-26T02:04:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 3360, 2024-04-26T02:04:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 194144, 2024-04-26T02:04:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 5, 2024-04-26T02:04:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 3, 2024-04-26T02:04:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.044062441, 2024-04-26T02:04:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.032792870, 2024-04-26T02:04:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.004714520, 2024-04-26T02:04:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:04:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 213, 2024-04-26T02:04:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.041826749, 2024-04-26T02:04:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000196369 2024-04-26T02:04:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-04-26T02:04:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-04-26T02:04:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-04-26T02:04:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-26T02:04:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 194144 2024-04-26T02:04:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:04:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 290, 2024-04-26T02:04:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 279, 2024-04-26T02:04:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 1609184, 2024-04-26T02:04:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 172208, 2024-04-26T02:04:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2024-04-26T02:04:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2024-04-26T02:04:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.131350451, 2024-04-26T02:04:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.053571902, 2024-04-26T02:04:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.087239014, 2024-04-26T02:04:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.013003716, 2024-04-26T02:04:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 278, 2024-04-26T02:04:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.034850532, 2024-04-26T02:04:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000125361 2024-04-26T02:04:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-04-26T02:04:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-04-26T02:04:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-04-26T02:04:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 1609184, 2024-04-26T02:04:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 172208 2024-04-26T02:04:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-04-26T02:04:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2024-04-26T02:04:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-04-26T02:04:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-04-26T02:04:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:04:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-04-26T02:04:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:04:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:04:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:04:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:04:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:04:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:04:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:04:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:04:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:04:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:04:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:04:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:04:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:04:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-04-26T02:04:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:04:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-04-26T02:04:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:04:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:04:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:04:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:04:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:04:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:04:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:04:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-04-26T02:04:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:04:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:04:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:04:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-04-26T02:04:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-04-26T02:04:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-04-26T02:04:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:04:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:04:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:04:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:04:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 149745, 2024-04-26T02:04:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 31, 2024-04-26T02:04:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:04:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-04-26T02:04:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-04-26T02:04:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-04-26T02:04:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:04:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-04-26T02:04:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:04:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-04-26T02:04:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2024-04-26T02:04:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2024-04-26T02:04:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:04:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:04:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-04-26T02:04:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-04-26T02:04:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2024-04-26T02:04:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2024-04-26T02:04:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-04-26T02:04:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-04-26T02:04:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-04-26T02:04:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-04-26T02:04:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:04:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-04-26T02:04:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-04-26T02:04:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-04-26T02:04:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 292, 2024-04-26T02:04:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 1568238, 2024-04-26T02:04:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 292, 2024-04-26T02:04:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-04-26T02:04:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-04-26T02:04:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 292, 2024-04-26T02:04:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 1568238, 2024-04-26T02:04:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-04-26T02:04:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-04-26T02:04:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-04-26T02:04:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-04-26T02:04:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:26.839 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T02:04:27.716 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-26T02:04:27.716 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-26T02:04:27.716 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-04-26T02:04:27.716 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-04-26T02:04:27.717 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-26T02:04:27.717 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-26T02:04:27.717 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-26T02:04:27.717 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-04-26T02:04:27.717 DEBUG:tasks.check_counter:Getting stats from j 2024-04-26T02:04:27.717 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-26T02:04:28.181 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:04:28.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:28 smithi001 ceph-mon[27822]: pgmap v403: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T02:04:28.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:28 smithi097 ceph-mon[32853]: pgmap v403: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T02:04:28.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:28 smithi173 ceph-mon[32553]: pgmap v403: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T02:04:30.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:30 smithi001 ceph-mon[27822]: pgmap v404: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T02:04:30.445 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-04-26T02:04:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:04:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 0, 2024-04-26T02:04:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 2, 2024-04-26T02:04:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 0, 2024-04-26T02:04:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 0, 2024-04-26T02:04:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-04-26T02:04:30.445 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2024-04-26T02:04:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.003993976, 2024-04-26T02:04:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T02:04:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.000790742, 2024-04-26T02:04:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:04:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-04-26T02:04:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-04-26T02:04:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-04-26T02:04:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-04-26T02:04:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-04-26T02:04:30.446 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-04-26T02:04:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T02:04:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T02:04:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:04:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 2, 2024-04-26T02:04:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 219, 2024-04-26T02:04:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 320, 2024-04-26T02:04:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 194752, 2024-04-26T02:04:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 5, 2024-04-26T02:04:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 3, 2024-04-26T02:04:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.042471392, 2024-04-26T02:04:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.032794015, 2024-04-26T02:04:30.447 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.003147267, 2024-04-26T02:04:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:04:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 216, 2024-04-26T02:04:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.051351246, 2024-04-26T02:04:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000237737 2024-04-26T02:04:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-04-26T02:04:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-04-26T02:04:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-04-26T02:04:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T02:04:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 194752 2024-04-26T02:04:30.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:04:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 301, 2024-04-26T02:04:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 288, 2024-04-26T02:04:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 1614128, 2024-04-26T02:04:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 177152, 2024-04-26T02:04:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2024-04-26T02:04:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2024-04-26T02:04:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.118089901, 2024-04-26T02:04:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.045654644, 2024-04-26T02:04:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.073831064, 2024-04-26T02:04:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.013134540, 2024-04-26T02:04:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 288, 2024-04-26T02:04:30.449 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.030018395, 2024-04-26T02:04:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000104230 2024-04-26T02:04:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-04-26T02:04:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-04-26T02:04:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-04-26T02:04:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 1614128, 2024-04-26T02:04:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 177152 2024-04-26T02:04:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-04-26T02:04:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2024-04-26T02:04:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-04-26T02:04:30.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-04-26T02:04:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:04:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-04-26T02:04:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:04:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:04:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:04:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:04:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:04:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:04:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:04:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:04:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:04:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:04:30.451 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:04:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:04:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:04:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-04-26T02:04:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:04:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-04-26T02:04:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:04:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:04:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:04:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:04:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:04:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:04:30.452 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:04:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-04-26T02:04:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:04:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:04:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:04:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-04-26T02:04:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-04-26T02:04:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-04-26T02:04:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:04:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:04:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:04:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:04:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 150776, 2024-04-26T02:04:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 32, 2024-04-26T02:04:30.453 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:04:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-04-26T02:04:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-04-26T02:04:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-04-26T02:04:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:04:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-04-26T02:04:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:04:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-04-26T02:04:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2024-04-26T02:04:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2024-04-26T02:04:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:04:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:04:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-04-26T02:04:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-04-26T02:04:30.454 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2024-04-26T02:04:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2024-04-26T02:04:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-04-26T02:04:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-04-26T02:04:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-04-26T02:04:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-04-26T02:04:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-04-26T02:04:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:04:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-04-26T02:04:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-04-26T02:04:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-04-26T02:04:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 299, 2024-04-26T02:04:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 1569148, 2024-04-26T02:04:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 299, 2024-04-26T02:04:30.455 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-04-26T02:04:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-04-26T02:04:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 299, 2024-04-26T02:04:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 1569148, 2024-04-26T02:04:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-04-26T02:04:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-04-26T02:04:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-04-26T02:04:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-04-26T02:04:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:30.456 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-04-26T02:04:30.456 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-04-26T02:04:30.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:30 smithi097 ceph-mon[32853]: pgmap v404: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T02:04:30.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:30 smithi173 ceph-mon[32553]: pgmap v404: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-26T02:04:31.200 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-26T02:04:31.200 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-26T02:04:31.200 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-26T02:04:31.200 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-26T02:04:31.200 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-26T02:04:31.200 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-26T02:04:31.200 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-26T02:04:31.201 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-26T02:04:31.201 DEBUG:tasks.check_counter:Getting stats from b 2024-04-26T02:04:31.201 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-26T02:04:31.450 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T02:04:32.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:32 smithi097 ceph-mon[32853]: pgmap v405: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2024-04-26T02:04:32.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:32 smithi001 ceph-mon[27822]: pgmap v405: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2024-04-26T02:04:32.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:32 smithi173 ceph-mon[32553]: pgmap v405: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2024-04-26T02:04:33.300 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-26T02:04:33.300 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:04:33.300 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 33753, 2024-04-26T02:04:33.300 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 35433, 2024-04-26T02:04:33.300 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 8861829, 2024-04-26T02:04:33.300 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 109444181, 2024-04-26T02:04:33.301 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 13, 2024-04-26T02:04:33.301 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 4, 2024-04-26T02:04:33.301 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 2.507455438, 2024-04-26T02:04:33.301 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.910005509, 2024-04-26T02:04:33.301 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.624275696, 2024-04-26T02:04:33.301 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.837364138, 2024-04-26T02:04:33.301 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:33.301 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 35435, 2024-04-26T02:04:33.301 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 3.877777844, 2024-04-26T02:04:33.301 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000109433 2024-04-26T02:04:33.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.301 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:33.301 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 10735, 2024-04-26T02:04:33.301 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.001690083, 2024-04-26T02:04:33.302 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000157 2024-04-26T02:04:33.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.302 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 448352, 2024-04-26T02:04:33.302 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 128128 2024-04-26T02:04:33.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.302 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:04:33.302 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 27883, 2024-04-26T02:04:33.302 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 27861, 2024-04-26T02:04:33.302 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 9592590, 2024-04-26T02:04:33.302 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 64671322, 2024-04-26T02:04:33.302 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 164, 2024-04-26T02:04:33.302 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 8, 2024-04-26T02:04:33.302 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 1.802152219, 2024-04-26T02:04:33.302 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.531542112, 2024-04-26T02:04:33.303 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.533901148, 2024-04-26T02:04:33.303 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.624285715, 2024-04-26T02:04:33.303 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:33.303 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 27859, 2024-04-26T02:04:33.303 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 3.771000665, 2024-04-26T02:04:33.303 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000135360 2024-04-26T02:04:33.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.303 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:33.303 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 437, 2024-04-26T02:04:33.303 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000065295, 2024-04-26T02:04:33.303 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000149 2024-04-26T02:04:33.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.303 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1806256, 2024-04-26T02:04:33.303 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 584240 2024-04-26T02:04:33.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.304 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:04:33.304 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 69480, 2024-04-26T02:04:33.304 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 124333, 2024-04-26T02:04:33.304 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 22064447, 2024-04-26T02:04:33.304 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 91888732, 2024-04-26T02:04:33.304 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 14, 2024-04-26T02:04:33.304 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 7, 2024-04-26T02:04:33.304 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 4.727041970, 2024-04-26T02:04:33.304 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 2.077589538, 2024-04-26T02:04:33.304 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 2.538042357, 2024-04-26T02:04:33.305 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.470345197, 2024-04-26T02:04:33.305 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:33.305 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 124328, 2024-04-26T02:04:33.305 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 5.723537210, 2024-04-26T02:04:33.305 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000046035 2024-04-26T02:04:33.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.305 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:33.305 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 122039, 2024-04-26T02:04:33.305 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.022719821, 2024-04-26T02:04:33.305 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000186 2024-04-26T02:04:33.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.305 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 43120, 2024-04-26T02:04:33.305 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 176512 2024-04-26T02:04:33.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.306 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-26T02:04:33.306 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-04-26T02:04:33.306 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-26T02:04:33.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.306 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-04-26T02:04:33.306 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-26T02:04:33.306 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-26T02:04:33.306 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 91072, 2024-04-26T02:04:33.306 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 30.567259679, 2024-04-26T02:04:33.306 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000335638 2024-04-26T02:04:33.306 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:33.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.306 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-04-26T02:04:33.306 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-26T02:04:33.307 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-26T02:04:33.307 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 8909, 2024-04-26T02:04:33.307 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1.635378464, 2024-04-26T02:04:33.307 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000183564 2024-04-26T02:04:33.307 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:33.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.307 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-04-26T02:04:33.307 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 25476, 2024-04-26T02:04:33.307 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 25474, 2024-04-26T02:04:33.307 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-04-26T02:04:33.307 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 25474, 2024-04-26T02:04:33.307 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 266.423369582, 2024-04-26T02:04:33.307 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.010458638 2024-04-26T02:04:33.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.308 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-04-26T02:04:33.308 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 2, 2024-04-26T02:04:33.308 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-04-26T02:04:33.308 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-04-26T02:04:33.308 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 1432, 2024-04-26T02:04:33.308 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 22, 2024-04-26T02:04:33.308 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-04-26T02:04:33.308 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 10252, 2024-04-26T02:04:33.308 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 7112, 2024-04-26T02:04:33.308 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 3129, 2024-04-26T02:04:33.308 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 11, 2024-04-26T02:04:33.308 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 68, 2024-04-26T02:04:33.308 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 27, 2024-04-26T02:04:33.308 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 2, 2024-04-26T02:04:33.308 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 3, 2024-04-26T02:04:33.309 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 5, 2024-04-26T02:04:33.309 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 31699, 2024-04-26T02:04:33.309 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 30168, 2024-04-26T02:04:33.309 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-04-26T02:04:33.309 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 3, 2024-04-26T02:04:33.309 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-04-26T02:04:33.309 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-04-26T02:04:33.309 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 3, 2024-04-26T02:04:33.309 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-04-26T02:04:33.309 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-04-26T02:04:33.309 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 2, 2024-04-26T02:04:33.309 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 15, 2024-04-26T02:04:33.309 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 2, 2024-04-26T02:04:33.310 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 14, 2024-04-26T02:04:33.310 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-04-26T02:04:33.310 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-04-26T02:04:33.310 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-04-26T02:04:33.310 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 4, 2024-04-26T02:04:33.310 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 234, 2024-04-26T02:04:33.310 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 6, 2024-04-26T02:04:33.310 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T02:04:33.310 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-04-26T02:04:33.310 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T02:04:33.310 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-04-26T02:04:33.310 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-04-26T02:04:33.310 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T02:04:33.310 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T02:04:33.311 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-04-26T02:04:33.311 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 6, 2024-04-26T02:04:33.311 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 13558, 2024-04-26T02:04:33.311 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 19953, 2024-04-26T02:04:33.311 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 7863, 2024-04-26T02:04:33.311 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T02:04:33.311 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T02:04:33.311 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 20194, 2024-04-26T02:04:33.311 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 13355, 2024-04-26T02:04:33.311 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 433, 2024-04-26T02:04:33.311 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 14584 2024-04-26T02:04:33.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.311 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-04-26T02:04:33.311 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 409, 2024-04-26T02:04:33.311 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 9, 2024-04-26T02:04:33.312 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 6, 2024-04-26T02:04:33.312 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 6, 2024-04-26T02:04:33.312 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 519, 2024-04-26T02:04:33.312 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 57, 2024-04-26T02:04:33.312 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-04-26T02:04:33.312 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-04-26T02:04:33.312 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 5162, 2024-04-26T02:04:33.312 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 5102, 2024-04-26T02:04:33.312 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 3, 2024-04-26T02:04:33.312 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-04-26T02:04:33.312 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-04-26T02:04:33.312 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-04-26T02:04:33.312 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-04-26T02:04:33.312 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-04-26T02:04:33.313 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-04-26T02:04:33.313 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 9, 2024-04-26T02:04:33.313 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 32227, 2024-04-26T02:04:33.313 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T02:04:33.313 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 2, 2024-04-26T02:04:33.313 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-04-26T02:04:33.313 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 22, 2024-04-26T02:04:33.313 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-04-26T02:04:33.313 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-04-26T02:04:33.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.313 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-04-26T02:04:33.313 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-04-26T02:04:33.313 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 47270, 2024-04-26T02:04:33.313 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 47267, 2024-04-26T02:04:33.313 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 47267, 2024-04-26T02:04:33.313 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 3, 2024-04-26T02:04:33.314 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-04-26T02:04:33.314 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 15, 2024-04-26T02:04:33.314 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 116, 2024-04-26T02:04:33.314 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 115, 2024-04-26T02:04:33.314 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 115, 2024-04-26T02:04:33.314 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 1, 2024-04-26T02:04:33.314 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 1, 2024-04-26T02:04:33.314 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-04-26T02:04:33.314 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 1, 2024-04-26T02:04:33.314 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 182834806, 2024-04-26T02:04:33.314 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 182840288, 2024-04-26T02:04:33.314 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 0, 2024-04-26T02:04:33.314 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-04-26T02:04:33.315 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 25444, 2024-04-26T02:04:33.315 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 112.149905633, 2024-04-26T02:04:33.315 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.004407715 2024-04-26T02:04:33.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.315 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 0 2024-04-26T02:04:33.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.315 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-04-26T02:04:33.315 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 5564, 2024-04-26T02:04:33.315 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 10682, 2024-04-26T02:04:33.315 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 5118, 2024-04-26T02:04:33.315 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 230, 2024-04-26T02:04:33.315 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 266, 2024-04-26T02:04:33.315 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 36, 2024-04-26T02:04:33.315 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 10252, 2024-04-26T02:04:33.316 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 16029, 2024-04-26T02:04:33.316 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 5777, 2024-04-26T02:04:33.316 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 3, 2024-04-26T02:04:33.316 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 10050, 2024-04-26T02:04:33.316 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 10047, 2024-04-26T02:04:33.316 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 119288, 2024-04-26T02:04:33.316 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 231800 2024-04-26T02:04:33.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.316 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-04-26T02:04:33.316 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 43808, 2024-04-26T02:04:33.316 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 609, 2024-04-26T02:04:33.316 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 25476, 2024-04-26T02:04:33.316 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 43, 2024-04-26T02:04:33.316 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 1223, 2024-04-26T02:04:33.316 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-04-26T02:04:33.317 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 5099, 2024-04-26T02:04:33.317 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 95.408482654, 2024-04-26T02:04:33.317 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.018711214 2024-04-26T02:04:33.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.317 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-04-26T02:04:33.317 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 155, 2024-04-26T02:04:33.317 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 3.851135531, 2024-04-26T02:04:33.317 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.024846035 2024-04-26T02:04:33.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.317 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-04-26T02:04:33.317 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:33.317 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:33.317 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:33.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.317 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-04-26T02:04:33.318 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3, 2024-04-26T02:04:33.318 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.003745240, 2024-04-26T02:04:33.318 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001248413 2024-04-26T02:04:33.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.318 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-04-26T02:04:33.318 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 5281, 2024-04-26T02:04:33.318 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 9.705036223, 2024-04-26T02:04:33.318 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001837726 2024-04-26T02:04:33.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.318 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-04-26T02:04:33.318 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:33.318 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:33.318 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:33.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.319 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-04-26T02:04:33.319 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-26T02:04:33.319 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000165610, 2024-04-26T02:04:33.319 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000165610 2024-04-26T02:04:33.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.319 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-04-26T02:04:33.319 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:33.319 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:33.319 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:33.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.319 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-04-26T02:04:33.319 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:33.319 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:33.319 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:33.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.320 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-04-26T02:04:33.320 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:33.320 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:33.320 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:33.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.320 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-04-26T02:04:33.320 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 13, 2024-04-26T02:04:33.321 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.003141309, 2024-04-26T02:04:33.321 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000241639 2024-04-26T02:04:33.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.321 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-04-26T02:04:33.321 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 73, 2024-04-26T02:04:33.321 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 11.917537548, 2024-04-26T02:04:33.321 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.163253939 2024-04-26T02:04:33.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.321 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-04-26T02:04:33.321 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:33.321 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:33.321 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:33.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.321 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-04-26T02:04:33.322 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 13, 2024-04-26T02:04:33.322 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.108773372, 2024-04-26T02:04:33.322 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.008367182 2024-04-26T02:04:33.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.322 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-04-26T02:04:33.322 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 425, 2024-04-26T02:04:33.322 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.120570791, 2024-04-26T02:04:33.322 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000283695 2024-04-26T02:04:33.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.322 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-04-26T02:04:33.322 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 477, 2024-04-26T02:04:33.322 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2.617477297, 2024-04-26T02:04:33.322 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.005487373 2024-04-26T02:04:33.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.323 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-04-26T02:04:33.323 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 92, 2024-04-26T02:04:33.323 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.223852755, 2024-04-26T02:04:33.323 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.002433182 2024-04-26T02:04:33.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.323 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-04-26T02:04:33.323 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:33.323 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:33.323 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:33.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.323 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-04-26T02:04:33.323 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:33.323 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:33.323 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:33.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.324 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-04-26T02:04:33.324 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 63, 2024-04-26T02:04:33.324 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 29.034094679, 2024-04-26T02:04:33.324 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.460858645 2024-04-26T02:04:33.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.324 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-04-26T02:04:33.324 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 7, 2024-04-26T02:04:33.324 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.041925016, 2024-04-26T02:04:33.324 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.005989288 2024-04-26T02:04:33.324 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.324 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-04-26T02:04:33.324 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:33.324 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:33.325 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:33.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.325 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-04-26T02:04:33.325 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 8710, 2024-04-26T02:04:33.325 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 104.370777916, 2024-04-26T02:04:33.325 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.011982867 2024-04-26T02:04:33.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.325 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-04-26T02:04:33.325 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:33.325 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:33.325 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:33.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.326 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-04-26T02:04:33.326 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 8, 2024-04-26T02:04:33.326 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.403191944, 2024-04-26T02:04:33.326 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.050398993 2024-04-26T02:04:33.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.326 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-04-26T02:04:33.326 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:33.326 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:33.326 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:33.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.326 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-04-26T02:04:33.326 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 8, 2024-04-26T02:04:33.327 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.021966814, 2024-04-26T02:04:33.327 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.002745851 2024-04-26T02:04:33.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.327 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-04-26T02:04:33.327 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 6, 2024-04-26T02:04:33.327 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.004673689, 2024-04-26T02:04:33.327 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000778948 2024-04-26T02:04:33.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.327 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-04-26T02:04:33.327 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 5040, 2024-04-26T02:04:33.327 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 8.586821194, 2024-04-26T02:04:33.327 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001703734 2024-04-26T02:04:33.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.327 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-04-26T02:04:33.328 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-04-26T02:04:33.328 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-04-26T02:04:33.328 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:33.328 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:33.328 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:33.328 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:33.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.328 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-04-26T02:04:33.328 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 2, 2024-04-26T02:04:33.328 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 6, 2024-04-26T02:04:33.328 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 4, 2024-04-26T02:04:33.328 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 2, 2024-04-26T02:04:33.328 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-04-26T02:04:33.328 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 6772, 2024-04-26T02:04:33.328 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 3386, 2024-04-26T02:04:33.329 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 760, 2024-04-26T02:04:33.329 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-04-26T02:04:33.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.329 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-26T02:04:33.329 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 360, 2024-04-26T02:04:33.329 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 360, 2024-04-26T02:04:33.329 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:04:33.329 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:04:33.329 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:04:33.329 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:04:33.329 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:04:33.329 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:04:33.329 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:04:33.329 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:04:33.330 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:04:33.330 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:04:33.330 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:04:33.330 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:04:33.330 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:04:33.331 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-26T02:04:33.331 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:04:33.331 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-26T02:04:33.331 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:04:33.331 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:04:33.331 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:04:33.331 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:04:33.331 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:04:33.331 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:04:33.331 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:04:33.331 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-26T02:04:33.331 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:04:33.331 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:04:33.331 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:04:33.332 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-26T02:04:33.332 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-26T02:04:33.332 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-26T02:04:33.332 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:04:33.332 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:04:33.332 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:04:33.332 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:04:33.332 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 940108, 2024-04-26T02:04:33.332 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 4852, 2024-04-26T02:04:33.332 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:04:33.332 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-04-26T02:04:33.332 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-26T02:04:33.332 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-26T02:04:33.332 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:04:33.332 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-26T02:04:33.333 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:04:33.333 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-26T02:04:33.333 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 8224, 2024-04-26T02:04:33.333 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 171, 2024-04-26T02:04:33.333 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:04:33.333 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:04:33.333 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-26T02:04:33.333 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-26T02:04:33.333 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 20351247, 2024-04-26T02:04:33.333 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 194205, 2024-04-26T02:04:33.333 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-26T02:04:33.333 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-26T02:04:33.333 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-26T02:04:33.333 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-26T02:04:33.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.334 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-04-26T02:04:33.334 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-04-26T02:04:33.334 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-04-26T02:04:33.334 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 74988, 2024-04-26T02:04:33.334 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 193732141, 2024-04-26T02:04:33.334 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 37, 2024-04-26T02:04:33.334 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 74988, 2024-04-26T02:04:33.334 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-04-26T02:04:33.334 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 74988, 2024-04-26T02:04:33.335 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1190.367063042, 2024-04-26T02:04:33.335 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.015874100 2024-04-26T02:04:33.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.335 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-04-26T02:04:33.335 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-04-26T02:04:33.335 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 74988, 2024-04-26T02:04:33.335 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 87975 2024-04-26T02:04:33.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.335 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 74988, 2024-04-26T02:04:33.335 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 66, 2024-04-26T02:04:33.335 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 74922, 2024-04-26T02:04:33.335 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-04-26T02:04:33.335 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-04-26T02:04:33.335 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 1181, 2024-04-26T02:04:33.335 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 5012, 2024-04-26T02:04:33.336 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 66, 2024-04-26T02:04:33.336 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 25510, 2024-04-26T02:04:33.336 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 239, 2024-04-26T02:04:33.336 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-04-26T02:04:33.336 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-04-26T02:04:33.336 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-04-26T02:04:33.336 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 6, 2024-04-26T02:04:33.336 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 5743, 2024-04-26T02:04:33.336 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-04-26T02:04:33.336 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-04-26T02:04:33.336 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-04-26T02:04:33.336 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 0, 2024-04-26T02:04:33.336 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 10025, 2024-04-26T02:04:33.336 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-04-26T02:04:33.336 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-04-26T02:04:33.337 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-04-26T02:04:33.337 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-04-26T02:04:33.337 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-04-26T02:04:33.337 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-04-26T02:04:33.337 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T02:04:33.337 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-04-26T02:04:33.337 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-04-26T02:04:33.337 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 36717, 2024-04-26T02:04:33.337 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-04-26T02:04:33.337 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-04-26T02:04:33.337 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-04-26T02:04:33.337 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-04-26T02:04:33.337 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-04-26T02:04:33.337 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-04-26T02:04:33.338 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-04-26T02:04:33.338 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-04-26T02:04:33.338 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-04-26T02:04:33.338 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-04-26T02:04:33.338 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-04-26T02:04:33.338 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-04-26T02:04:33.338 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-04-26T02:04:33.338 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-04-26T02:04:33.338 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-04-26T02:04:33.338 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-04-26T02:04:33.338 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 109, 2024-04-26T02:04:33.338 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-04-26T02:04:33.338 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 19, 2024-04-26T02:04:33.338 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 12, 2024-04-26T02:04:33.338 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 12, 2024-04-26T02:04:33.339 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-04-26T02:04:33.339 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-04-26T02:04:33.339 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 3175, 2024-04-26T02:04:33.339 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 0, 2024-04-26T02:04:33.339 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 301 2024-04-26T02:04:33.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.339 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-04-26T02:04:33.339 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 1, 2024-04-26T02:04:33.339 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-04-26T02:04:33.339 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-04-26T02:04:33.339 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-04-26T02:04:33.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.339 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-04-26T02:04:33.339 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 32829 2024-04-26T02:04:33.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.340 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-04-26T02:04:33.340 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-04-26T02:04:33.340 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 34, 2024-04-26T02:04:33.340 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-04-26T02:04:33.340 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 10, 2024-04-26T02:04:33.340 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 5131, 2024-04-26T02:04:33.340 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 5102, 2024-04-26T02:04:33.340 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-04-26T02:04:33.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.340 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:04:33.340 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-26T02:04:33.340 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-26T02:04:33.340 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-26T02:04:33.341 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 131112, 2024-04-26T02:04:33.341 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 28635450, 2024-04-26T02:04:33.341 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:33.341 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 131112, 2024-04-26T02:04:33.341 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-26T02:04:33.341 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-26T02:04:33.341 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 131112, 2024-04-26T02:04:33.341 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 28635450, 2024-04-26T02:04:33.341 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-26T02:04:33.341 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:33.341 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:33.341 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:33.341 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:33.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.342 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-04-26T02:04:33.342 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-26T02:04:33.342 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-26T02:04:33.342 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-26T02:04:33.342 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-26T02:04:33.342 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-26T02:04:33.342 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:33.342 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-26T02:04:33.342 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 74988, 2024-04-26T02:04:33.342 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 194466451, 2024-04-26T02:04:33.342 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 74988, 2024-04-26T02:04:33.342 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 194466451, 2024-04-26T02:04:33.342 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-26T02:04:33.342 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:33.343 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:33.346 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:33.346 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:33.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.346 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-04-26T02:04:33.346 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-26T02:04:33.346 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-04-26T02:04:33.346 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-26T02:04:33.346 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-26T02:04:33.346 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-26T02:04:33.346 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:33.346 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-26T02:04:33.346 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 74988, 2024-04-26T02:04:33.347 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 74988, 2024-04-26T02:04:33.347 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 74988, 2024-04-26T02:04:33.347 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 74988, 2024-04-26T02:04:33.347 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-26T02:04:33.347 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:33.347 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:33.347 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:33.347 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:33.347 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.347 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-04-26T02:04:33.347 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-26T02:04:33.347 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-26T02:04:33.347 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-26T02:04:33.347 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 5102, 2024-04-26T02:04:33.348 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 734310, 2024-04-26T02:04:33.348 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:33.348 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 5102, 2024-04-26T02:04:33.348 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-26T02:04:33.348 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-26T02:04:33.348 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 66, 2024-04-26T02:04:33.348 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 734310, 2024-04-26T02:04:33.348 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-26T02:04:33.348 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:33.348 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:33.348 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:33.348 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:33.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:33.348 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x55c105180460": { 2024-04-26T02:04:33.348 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-26T02:04:33.349 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-26T02:04:33.349 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-26T02:04:33.349 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 47270, 2024-04-26T02:04:33.349 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 178645984, 2024-04-26T02:04:33.349 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:33.349 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 47270, 2024-04-26T02:04:33.349 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-26T02:04:33.349 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-26T02:04:33.349 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 25444, 2024-04-26T02:04:33.349 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 178645984, 2024-04-26T02:04:33.349 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-26T02:04:33.349 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:33.349 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:33.349 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:33.349 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:33.350 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:33.350 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-26T02:04:33.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:33 smithi097 ceph-mon[32853]: pgmap v406: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:33.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:33 smithi001 ceph-mon[27822]: pgmap v406: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:33.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:33 smithi173 ceph-mon[32553]: pgmap v406: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:33.969 INFO:tasks.check_counter:Daemon mds.b mds.exported=2 2024-04-26T02:04:33.969 INFO:tasks.check_counter:Daemon mds.b mds.imported=2 2024-04-26T02:04:33.969 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=409 2024-04-26T02:04:33.969 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=9 2024-04-26T02:04:33.969 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-26T02:04:33.969 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=13 2024-04-26T02:04:33.969 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=7 2024-04-26T02:04:33.969 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=22 2024-04-26T02:04:33.969 DEBUG:tasks.check_counter:Getting stats from e 2024-04-26T02:04:33.970 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-26T02:04:34.215 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T02:04:35.951 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-26T02:04:35.951 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:04:35.951 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 1023, 2024-04-26T02:04:35.951 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 1025, 2024-04-26T02:04:35.951 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 336907, 2024-04-26T02:04:35.951 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 325378, 2024-04-26T02:04:35.951 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 7, 2024-04-26T02:04:35.951 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2024-04-26T02:04:35.951 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.084469947, 2024-04-26T02:04:35.951 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.034114823, 2024-04-26T02:04:35.951 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.019216685, 2024-04-26T02:04:35.951 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.021673178, 2024-04-26T02:04:35.952 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:35.952 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1023, 2024-04-26T02:04:35.952 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.040031732, 2024-04-26T02:04:35.952 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000039131 2024-04-26T02:04:35.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.952 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:35.952 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.952 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.952 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.952 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T02:04:35.952 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T02:04:35.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.952 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:04:35.952 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 920, 2024-04-26T02:04:35.953 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 1134, 2024-04-26T02:04:35.953 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 228226, 2024-04-26T02:04:35.953 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 827441, 2024-04-26T02:04:35.953 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 7, 2024-04-26T02:04:35.953 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 5, 2024-04-26T02:04:35.953 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.087904236, 2024-04-26T02:04:35.953 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.030631597, 2024-04-26T02:04:35.953 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.019066846, 2024-04-26T02:04:35.953 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.028933672, 2024-04-26T02:04:35.953 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:35.953 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1132, 2024-04-26T02:04:35.953 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.065920887, 2024-04-26T02:04:35.953 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000058233 2024-04-26T02:04:35.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.954 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:35.954 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.954 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.954 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.954 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-26T02:04:35.954 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 536784 2024-04-26T02:04:35.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.954 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:04:35.954 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 1335, 2024-04-26T02:04:35.954 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 1312, 2024-04-26T02:04:35.954 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 2023808, 2024-04-26T02:04:35.954 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 499194, 2024-04-26T02:04:35.954 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 3, 2024-04-26T02:04:35.954 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2024-04-26T02:04:35.955 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.162426465, 2024-04-26T02:04:35.955 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.050415466, 2024-04-26T02:04:35.955 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.067213461, 2024-04-26T02:04:35.955 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.043089022, 2024-04-26T02:04:35.955 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:35.955 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1311, 2024-04-26T02:04:35.955 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.065229151, 2024-04-26T02:04:35.955 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000049755 2024-04-26T02:04:35.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.955 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:35.955 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.955 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.955 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.955 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1686800, 2024-04-26T02:04:35.956 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 180240 2024-04-26T02:04:35.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.956 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-26T02:04:35.956 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-04-26T02:04:35.956 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-26T02:04:35.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.956 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-04-26T02:04:35.956 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-26T02:04:35.956 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-26T02:04:35.956 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 4956, 2024-04-26T02:04:35.956 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.127319363, 2024-04-26T02:04:35.956 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000025689 2024-04-26T02:04:35.956 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:35.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.957 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-04-26T02:04:35.957 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-26T02:04:35.957 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-26T02:04:35.957 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.957 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.957 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.957 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:35.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.957 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-04-26T02:04:35.957 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 0, 2024-04-26T02:04:35.957 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 0, 2024-04-26T02:04:35.957 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-04-26T02:04:35.957 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.957 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.958 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.958 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-04-26T02:04:35.958 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-04-26T02:04:35.958 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-04-26T02:04:35.958 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-04-26T02:04:35.958 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 0, 2024-04-26T02:04:35.958 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-04-26T02:04:35.958 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-04-26T02:04:35.958 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 15, 2024-04-26T02:04:35.958 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 0, 2024-04-26T02:04:35.958 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 15, 2024-04-26T02:04:35.958 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-04-26T02:04:35.958 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 15, 2024-04-26T02:04:35.959 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-04-26T02:04:35.959 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 0, 2024-04-26T02:04:35.959 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 0, 2024-04-26T02:04:35.959 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 1, 2024-04-26T02:04:35.959 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 0, 2024-04-26T02:04:35.959 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 0, 2024-04-26T02:04:35.959 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-04-26T02:04:35.959 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2024-04-26T02:04:35.959 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-04-26T02:04:35.959 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-04-26T02:04:35.959 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2024-04-26T02:04:35.959 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-04-26T02:04:35.959 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-04-26T02:04:35.959 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-04-26T02:04:35.959 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-04-26T02:04:35.959 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-04-26T02:04:35.960 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-04-26T02:04:35.960 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-04-26T02:04:35.960 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-04-26T02:04:35.960 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-04-26T02:04:35.960 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 0, 2024-04-26T02:04:35.960 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 0, 2024-04-26T02:04:35.960 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-04-26T02:04:35.960 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T02:04:35.960 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-04-26T02:04:35.960 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T02:04:35.960 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-04-26T02:04:35.960 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-04-26T02:04:35.960 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T02:04:35.960 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T02:04:35.960 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-04-26T02:04:35.961 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-04-26T02:04:35.961 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T02:04:35.961 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 0, 2024-04-26T02:04:35.961 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T02:04:35.961 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T02:04:35.961 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T02:04:35.961 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 0, 2024-04-26T02:04:35.961 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 0, 2024-04-26T02:04:35.961 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 0, 2024-04-26T02:04:35.961 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 0 2024-04-26T02:04:35.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.961 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-04-26T02:04:35.961 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-04-26T02:04:35.961 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-04-26T02:04:35.961 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-04-26T02:04:35.961 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 0, 2024-04-26T02:04:35.962 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-04-26T02:04:35.962 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-04-26T02:04:35.962 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-04-26T02:04:35.962 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-04-26T02:04:35.962 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 0, 2024-04-26T02:04:35.962 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 0, 2024-04-26T02:04:35.962 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-04-26T02:04:35.962 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-04-26T02:04:35.962 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-04-26T02:04:35.962 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-04-26T02:04:35.962 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-04-26T02:04:35.962 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-04-26T02:04:35.962 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-04-26T02:04:35.962 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 0, 2024-04-26T02:04:35.962 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 0, 2024-04-26T02:04:35.963 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T02:04:35.963 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-04-26T02:04:35.963 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-04-26T02:04:35.963 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-04-26T02:04:35.963 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-04-26T02:04:35.963 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-04-26T02:04:35.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.963 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-04-26T02:04:35.963 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-04-26T02:04:35.963 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 0, 2024-04-26T02:04:35.963 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-04-26T02:04:35.963 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-04-26T02:04:35.963 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 35, 2024-04-26T02:04:35.963 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-04-26T02:04:35.964 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-04-26T02:04:35.964 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 0, 2024-04-26T02:04:35.964 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-04-26T02:04:35.964 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-04-26T02:04:35.964 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 2, 2024-04-26T02:04:35.964 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 2, 2024-04-26T02:04:35.964 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-04-26T02:04:35.964 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-04-26T02:04:35.964 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-04-26T02:04:35.964 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 4262818, 2024-04-26T02:04:35.964 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 4260213, 2024-04-26T02:04:35.964 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-04-26T02:04:35.964 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.964 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.964 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.965 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 35 2024-04-26T02:04:35.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.965 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-04-26T02:04:35.965 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 10, 2024-04-26T02:04:35.965 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 23, 2024-04-26T02:04:35.965 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 13, 2024-04-26T02:04:35.965 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 6, 2024-04-26T02:04:35.965 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 18, 2024-04-26T02:04:35.965 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 12, 2024-04-26T02:04:35.965 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 15, 2024-04-26T02:04:35.965 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 35, 2024-04-26T02:04:35.965 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 20, 2024-04-26T02:04:35.965 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 0, 2024-04-26T02:04:35.965 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 0, 2024-04-26T02:04:35.966 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 0, 2024-04-26T02:04:35.966 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 51340, 2024-04-26T02:04:35.966 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 215416 2024-04-26T02:04:35.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.966 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-04-26T02:04:35.966 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 0, 2024-04-26T02:04:35.966 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-04-26T02:04:35.966 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 0, 2024-04-26T02:04:35.966 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 0, 2024-04-26T02:04:35.966 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-04-26T02:04:35.966 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-04-26T02:04:35.966 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.966 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.966 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-04-26T02:04:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-04-26T02:04:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-04-26T02:04:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.967 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-04-26T02:04:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-04-26T02:04:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-04-26T02:04:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-04-26T02:04:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.968 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.969 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.969 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-04-26T02:04:35.969 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.969 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.969 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.969 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-04-26T02:04:35.969 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.969 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.969 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.969 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-04-26T02:04:35.969 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.969 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.969 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.970 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-04-26T02:04:35.970 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.970 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.970 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.970 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-04-26T02:04:35.970 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.971 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.971 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.971 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-04-26T02:04:35.971 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.971 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.971 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.971 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-04-26T02:04:35.971 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.971 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.971 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.971 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-04-26T02:04:35.971 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.972 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.972 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.972 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-04-26T02:04:35.972 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.972 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.972 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.972 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-04-26T02:04:35.972 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.972 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.972 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.972 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-04-26T02:04:35.972 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.972 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.973 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.973 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-04-26T02:04:35.973 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.973 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.973 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.973 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-04-26T02:04:35.973 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.973 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.973 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.973 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-04-26T02:04:35.973 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.973 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.974 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.974 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-04-26T02:04:35.974 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.974 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.974 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.974 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-04-26T02:04:35.974 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.974 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.974 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.974 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-04-26T02:04:35.974 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.974 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.974 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.975 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-04-26T02:04:35.975 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.975 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.975 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.975 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-04-26T02:04:35.975 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.975 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.975 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.975 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-04-26T02:04:35.975 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.975 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.975 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.976 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-04-26T02:04:35.976 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.976 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.976 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.976 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-04-26T02:04:35.976 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-04-26T02:04:35.976 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-04-26T02:04:35.976 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.976 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.976 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.976 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:35.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.976 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-04-26T02:04:35.976 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 2, 2024-04-26T02:04:35.977 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 6, 2024-04-26T02:04:35.977 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 4, 2024-04-26T02:04:35.977 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 5, 2024-04-26T02:04:35.977 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-04-26T02:04:35.977 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 0, 2024-04-26T02:04:35.977 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 0, 2024-04-26T02:04:35.977 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 4696, 2024-04-26T02:04:35.977 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-04-26T02:04:35.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.977 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-26T02:04:35.977 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:04:35.977 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-26T02:04:35.977 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:04:35.977 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:04:35.977 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:04:35.977 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:04:35.978 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:04:35.978 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:04:35.978 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:04:35.978 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:04:35.978 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:04:35.978 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:04:35.978 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:04:35.978 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:04:35.978 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:04:35.978 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-26T02:04:35.978 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:04:35.978 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-26T02:04:35.978 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:04:35.978 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:04:35.978 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:04:35.978 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:04:35.979 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:04:35.979 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:04:35.979 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:04:35.979 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-26T02:04:35.979 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:04:35.979 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:04:35.979 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:04:35.979 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-26T02:04:35.979 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-26T02:04:35.979 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-26T02:04:35.979 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:04:35.979 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:04:35.979 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:04:35.979 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:04:35.979 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 190472, 2024-04-26T02:04:35.979 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 61, 2024-04-26T02:04:35.980 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:04:35.980 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-04-26T02:04:35.980 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-26T02:04:35.980 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-26T02:04:35.980 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:04:35.980 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-26T02:04:35.980 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:04:35.980 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-26T02:04:35.980 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 8224, 2024-04-26T02:04:35.980 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 171, 2024-04-26T02:04:35.980 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:04:35.980 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:04:35.980 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-26T02:04:35.980 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-26T02:04:35.980 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 46883, 2024-04-26T02:04:35.980 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 703, 2024-04-26T02:04:35.981 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-26T02:04:35.981 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-26T02:04:35.981 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-26T02:04:35.981 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-26T02:04:35.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.981 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-04-26T02:04:35.981 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-04-26T02:04:35.981 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-04-26T02:04:35.981 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 2940, 2024-04-26T02:04:35.981 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 21, 2024-04-26T02:04:35.981 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 3, 2024-04-26T02:04:35.981 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 2940, 2024-04-26T02:04:35.981 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-04-26T02:04:35.981 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2940, 2024-04-26T02:04:35.981 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2.637956083, 2024-04-26T02:04:35.982 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000897263 2024-04-26T02:04:35.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.982 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-04-26T02:04:35.982 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-04-26T02:04:35.982 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2940, 2024-04-26T02:04:35.982 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2941 2024-04-26T02:04:35.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.982 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 2940, 2024-04-26T02:04:35.982 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 2940, 2024-04-26T02:04:35.982 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 0, 2024-04-26T02:04:35.982 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-04-26T02:04:35.982 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-04-26T02:04:35.982 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 1915, 2024-04-26T02:04:35.982 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 0, 2024-04-26T02:04:35.982 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 1023, 2024-04-26T02:04:35.982 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 0, 2024-04-26T02:04:35.983 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 0, 2024-04-26T02:04:35.983 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-04-26T02:04:35.983 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-04-26T02:04:35.983 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-04-26T02:04:35.983 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-04-26T02:04:35.983 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 0, 2024-04-26T02:04:35.983 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-04-26T02:04:35.983 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-04-26T02:04:35.983 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-04-26T02:04:35.983 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 1, 2024-04-26T02:04:35.983 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 0, 2024-04-26T02:04:35.983 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-04-26T02:04:35.983 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-04-26T02:04:35.983 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-04-26T02:04:35.983 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-04-26T02:04:35.983 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-04-26T02:04:35.984 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-04-26T02:04:35.984 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T02:04:35.984 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-04-26T02:04:35.984 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-04-26T02:04:35.984 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-04-26T02:04:35.984 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-04-26T02:04:35.984 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-04-26T02:04:35.984 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-04-26T02:04:35.984 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-04-26T02:04:35.984 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-04-26T02:04:35.984 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-04-26T02:04:35.984 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-04-26T02:04:35.984 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-04-26T02:04:35.984 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-04-26T02:04:35.984 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-04-26T02:04:35.984 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-04-26T02:04:35.985 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-04-26T02:04:35.985 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-04-26T02:04:35.985 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-04-26T02:04:35.985 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-04-26T02:04:35.985 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-04-26T02:04:35.985 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 109, 2024-04-26T02:04:35.985 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-04-26T02:04:35.985 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 19, 2024-04-26T02:04:35.985 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 5, 2024-04-26T02:04:35.985 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 6, 2024-04-26T02:04:35.985 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 1, 2024-04-26T02:04:35.985 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-04-26T02:04:35.985 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 0, 2024-04-26T02:04:35.985 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 2, 2024-04-26T02:04:35.985 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 0 2024-04-26T02:04:35.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.986 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-04-26T02:04:35.986 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 0, 2024-04-26T02:04:35.986 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-04-26T02:04:35.986 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-04-26T02:04:35.986 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-04-26T02:04:35.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.986 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-04-26T02:04:35.986 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-04-26T02:04:35.986 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.986 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-04-26T02:04:35.986 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-04-26T02:04:35.986 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T02:04:35.986 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-04-26T02:04:35.986 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 0, 2024-04-26T02:04:35.987 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 0, 2024-04-26T02:04:35.987 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 0, 2024-04-26T02:04:35.987 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-04-26T02:04:35.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.987 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:04:35.987 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-26T02:04:35.987 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-26T02:04:35.987 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-26T02:04:35.987 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 3274, 2024-04-26T02:04:35.987 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 2274254, 2024-04-26T02:04:35.987 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:35.987 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 3274, 2024-04-26T02:04:35.987 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-26T02:04:35.987 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-26T02:04:35.987 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 3274, 2024-04-26T02:04:35.987 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 2274254, 2024-04-26T02:04:35.988 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-26T02:04:35.988 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.988 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.988 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.988 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:35.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.988 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-04-26T02:04:35.988 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-26T02:04:35.988 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-26T02:04:35.988 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-26T02:04:35.988 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-26T02:04:35.988 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-26T02:04:35.988 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:35.988 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-26T02:04:35.988 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 2940, 2024-04-26T02:04:35.989 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 68519, 2024-04-26T02:04:35.989 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2940, 2024-04-26T02:04:35.989 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 68519, 2024-04-26T02:04:35.989 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-26T02:04:35.989 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.989 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.989 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.989 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:35.989 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.989 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-04-26T02:04:35.989 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-26T02:04:35.989 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-04-26T02:04:35.989 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-26T02:04:35.989 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-26T02:04:35.989 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-26T02:04:35.990 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:35.990 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-26T02:04:35.990 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 2940, 2024-04-26T02:04:35.990 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 2940, 2024-04-26T02:04:35.990 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2940, 2024-04-26T02:04:35.990 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 2940, 2024-04-26T02:04:35.990 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-26T02:04:35.990 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.990 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.990 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.990 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:35.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.990 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-04-26T02:04:35.990 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-26T02:04:35.991 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-26T02:04:35.991 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-26T02:04:35.991 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-26T02:04:35.991 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-26T02:04:35.991 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:35.991 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-26T02:04:35.991 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-26T02:04:35.991 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-26T02:04:35.991 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-04-26T02:04:35.991 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-04-26T02:04:35.991 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-26T02:04:35.991 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.991 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.991 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.991 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:35.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:35.992 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x55a52cc946e0": { 2024-04-26T02:04:35.992 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-26T02:04:35.992 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-26T02:04:35.992 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-26T02:04:35.992 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-26T02:04:35.992 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-26T02:04:35.992 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:35.992 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-26T02:04:35.992 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-26T02:04:35.992 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-26T02:04:35.992 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-04-26T02:04:35.992 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-04-26T02:04:35.993 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-26T02:04:35.993 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:35.993 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:35.993 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:35.993 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:35.993 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:35.993 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-26T02:04:36.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:36 smithi097 ceph-mon[32853]: pgmap v407: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T02:04:36.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:36 smithi001 ceph-mon[27822]: pgmap v407: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T02:04:36.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:36 smithi173 ceph-mon[32553]: pgmap v407: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T02:04:36.799 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-26T02:04:36.799 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-26T02:04:36.799 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-26T02:04:36.799 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-26T02:04:36.799 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-26T02:04:36.799 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-26T02:04:36.799 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T02:04:36.799 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-04-26T02:04:36.800 DEBUG:tasks.check_counter:Getting stats from h 2024-04-26T02:04:36.800 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-26T02:04:37.037 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T02:04:38.129 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:38 smithi097 ceph-mon[32853]: pgmap v408: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:38.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:38 smithi001 ceph-mon[27822]: pgmap v408: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:38.569 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:38 smithi173 ceph-mon[32553]: pgmap v408: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:38.751 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-26T02:04:38.751 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:04:38.751 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 2816, 2024-04-26T02:04:38.751 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 2816, 2024-04-26T02:04:38.751 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 851487, 2024-04-26T02:04:38.751 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 894671, 2024-04-26T02:04:38.751 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 6, 2024-04-26T02:04:38.751 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2024-04-26T02:04:38.751 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.197175203, 2024-04-26T02:04:38.751 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.064738345, 2024-04-26T02:04:38.751 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.048224680, 2024-04-26T02:04:38.751 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.067948537, 2024-04-26T02:04:38.751 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:38.752 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2815, 2024-04-26T02:04:38.752 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.080705614, 2024-04-26T02:04:38.752 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000028669 2024-04-26T02:04:38.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.752 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:38.752 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.752 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.752 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.752 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-26T02:04:38.752 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T02:04:38.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.752 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:04:38.752 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 5, 2024-04-26T02:04:38.752 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 220, 2024-04-26T02:04:38.752 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 1036, 2024-04-26T02:04:38.753 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 615402, 2024-04-26T02:04:38.753 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 7, 2024-04-26T02:04:38.753 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 4, 2024-04-26T02:04:38.753 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.012528164, 2024-04-26T02:04:38.753 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.008140608, 2024-04-26T02:04:38.753 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.002002131, 2024-04-26T02:04:38.753 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000058166, 2024-04-26T02:04:38.753 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:38.753 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 217, 2024-04-26T02:04:38.753 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.038019630, 2024-04-26T02:04:38.753 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000175205 2024-04-26T02:04:38.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.753 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:38.753 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.754 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.754 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.754 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-26T02:04:38.754 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 614704 2024-04-26T02:04:38.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.754 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:04:38.754 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 332, 2024-04-26T02:04:38.754 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 309, 2024-04-26T02:04:38.754 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 1686796, 2024-04-26T02:04:38.754 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 179918, 2024-04-26T02:04:38.754 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2024-04-26T02:04:38.754 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2024-04-26T02:04:38.755 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.054262096, 2024-04-26T02:04:38.755 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.016297003, 2024-04-26T02:04:38.755 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.038866033, 2024-04-26T02:04:38.755 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.006967894, 2024-04-26T02:04:38.755 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:38.755 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 308, 2024-04-26T02:04:38.755 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.015488145, 2024-04-26T02:04:38.755 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000050286 2024-04-26T02:04:38.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.755 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:38.755 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.755 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.755 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.756 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1686528, 2024-04-26T02:04:38.756 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 179600 2024-04-26T02:04:38.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.756 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-26T02:04:38.756 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-04-26T02:04:38.756 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-26T02:04:38.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.756 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-04-26T02:04:38.756 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-26T02:04:38.756 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-26T02:04:38.757 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 4825, 2024-04-26T02:04:38.757 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.104912622, 2024-04-26T02:04:38.757 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000021743 2024-04-26T02:04:38.757 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:38.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.757 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-04-26T02:04:38.757 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-26T02:04:38.757 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-26T02:04:38.757 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.757 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.757 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.757 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:38.757 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.758 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-04-26T02:04:38.758 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 0, 2024-04-26T02:04:38.758 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 0, 2024-04-26T02:04:38.758 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-04-26T02:04:38.758 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.758 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.758 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.758 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-04-26T02:04:38.758 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-04-26T02:04:38.758 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-04-26T02:04:38.758 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-04-26T02:04:38.758 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 0, 2024-04-26T02:04:38.758 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-04-26T02:04:38.759 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-04-26T02:04:38.759 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 5, 2024-04-26T02:04:38.759 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 0, 2024-04-26T02:04:38.759 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 5, 2024-04-26T02:04:38.759 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-04-26T02:04:38.759 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 5, 2024-04-26T02:04:38.759 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-04-26T02:04:38.759 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 0, 2024-04-26T02:04:38.759 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 0, 2024-04-26T02:04:38.759 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 1, 2024-04-26T02:04:38.759 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 0, 2024-04-26T02:04:38.759 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 0, 2024-04-26T02:04:38.759 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-04-26T02:04:38.759 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2024-04-26T02:04:38.760 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-04-26T02:04:38.760 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-04-26T02:04:38.760 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2024-04-26T02:04:38.760 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-04-26T02:04:38.760 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-04-26T02:04:38.760 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-04-26T02:04:38.760 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-04-26T02:04:38.760 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-04-26T02:04:38.760 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-04-26T02:04:38.760 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-04-26T02:04:38.760 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-04-26T02:04:38.760 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-04-26T02:04:38.760 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 0, 2024-04-26T02:04:38.760 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 0, 2024-04-26T02:04:38.760 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-04-26T02:04:38.761 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T02:04:38.761 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-04-26T02:04:38.761 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T02:04:38.761 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-04-26T02:04:38.761 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-04-26T02:04:38.761 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T02:04:38.761 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T02:04:38.761 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-04-26T02:04:38.761 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-04-26T02:04:38.761 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T02:04:38.761 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 0, 2024-04-26T02:04:38.761 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T02:04:38.761 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T02:04:38.761 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T02:04:38.762 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 0, 2024-04-26T02:04:38.762 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 0, 2024-04-26T02:04:38.762 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 0, 2024-04-26T02:04:38.762 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 0 2024-04-26T02:04:38.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.762 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-04-26T02:04:38.762 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-04-26T02:04:38.762 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-04-26T02:04:38.762 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-04-26T02:04:38.762 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 0, 2024-04-26T02:04:38.762 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-04-26T02:04:38.762 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-04-26T02:04:38.762 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-04-26T02:04:38.762 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-04-26T02:04:38.763 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 0, 2024-04-26T02:04:38.763 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 0, 2024-04-26T02:04:38.763 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-04-26T02:04:38.763 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-04-26T02:04:38.763 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-04-26T02:04:38.763 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-04-26T02:04:38.763 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-04-26T02:04:38.763 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-04-26T02:04:38.763 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-04-26T02:04:38.763 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 0, 2024-04-26T02:04:38.763 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 0, 2024-04-26T02:04:38.763 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T02:04:38.763 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-04-26T02:04:38.763 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-04-26T02:04:38.763 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-04-26T02:04:38.764 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-04-26T02:04:38.764 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-04-26T02:04:38.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.764 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-04-26T02:04:38.764 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-04-26T02:04:38.764 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 0, 2024-04-26T02:04:38.764 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-04-26T02:04:38.764 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-04-26T02:04:38.764 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 33, 2024-04-26T02:04:38.764 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-04-26T02:04:38.764 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-04-26T02:04:38.764 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 0, 2024-04-26T02:04:38.764 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-04-26T02:04:38.764 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-04-26T02:04:38.764 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 2, 2024-04-26T02:04:38.765 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 2, 2024-04-26T02:04:38.765 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-04-26T02:04:38.765 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-04-26T02:04:38.765 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-04-26T02:04:38.765 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 4243528, 2024-04-26T02:04:38.765 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 4240923, 2024-04-26T02:04:38.765 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-04-26T02:04:38.765 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.765 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.765 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.765 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 33 2024-04-26T02:04:38.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.765 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-04-26T02:04:38.765 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 4, 2024-04-26T02:04:38.766 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 23, 2024-04-26T02:04:38.766 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 19, 2024-04-26T02:04:38.766 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 2, 2024-04-26T02:04:38.766 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 14, 2024-04-26T02:04:38.766 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 12, 2024-04-26T02:04:38.766 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 5, 2024-04-26T02:04:38.766 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 25, 2024-04-26T02:04:38.766 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 20, 2024-04-26T02:04:38.766 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 0, 2024-04-26T02:04:38.766 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 0, 2024-04-26T02:04:38.766 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 0, 2024-04-26T02:04:38.766 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 51428, 2024-04-26T02:04:38.766 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 215416 2024-04-26T02:04:38.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.766 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-04-26T02:04:38.767 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 0, 2024-04-26T02:04:38.767 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-04-26T02:04:38.767 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 0, 2024-04-26T02:04:38.767 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 0, 2024-04-26T02:04:38.767 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-04-26T02:04:38.767 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-04-26T02:04:38.767 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.767 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.767 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.767 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-04-26T02:04:38.767 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.767 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.767 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.767 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-04-26T02:04:38.768 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.768 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.768 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.768 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-04-26T02:04:38.768 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.768 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.768 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.768 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-04-26T02:04:38.768 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.768 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.768 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.768 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-04-26T02:04:38.768 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.769 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.769 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.769 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-04-26T02:04:38.769 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.769 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.769 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.769 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-04-26T02:04:38.769 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.769 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.769 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.769 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-04-26T02:04:38.769 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.769 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.769 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.770 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-04-26T02:04:38.770 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.770 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.770 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.770 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-04-26T02:04:38.770 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.770 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.770 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.770 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-04-26T02:04:38.770 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.771 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.771 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.771 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-04-26T02:04:38.771 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.771 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.771 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.771 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-04-26T02:04:38.772 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.772 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.772 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.772 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-04-26T02:04:38.772 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.772 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.772 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.772 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-04-26T02:04:38.772 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.772 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.772 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.772 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-04-26T02:04:38.772 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.773 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.773 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.773 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-04-26T02:04:38.773 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.773 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.773 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.773 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-04-26T02:04:38.773 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.773 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.773 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.773 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-04-26T02:04:38.773 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.773 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.773 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.774 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-04-26T02:04:38.774 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.774 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.774 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.774 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-04-26T02:04:38.774 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.774 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.774 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.774 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-04-26T02:04:38.774 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.774 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.775 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.775 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-04-26T02:04:38.775 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.775 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.775 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.775 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-04-26T02:04:38.775 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.775 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.775 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.775 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-04-26T02:04:38.775 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.775 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.776 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.776 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.778 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-04-26T02:04:38.778 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.778 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.778 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.778 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-04-26T02:04:38.778 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.778 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.778 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.778 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-04-26T02:04:38.778 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.778 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.779 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.779 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-04-26T02:04:38.779 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-04-26T02:04:38.779 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-04-26T02:04:38.779 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.779 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.779 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.779 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:38.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.779 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-04-26T02:04:38.779 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 2, 2024-04-26T02:04:38.779 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 4, 2024-04-26T02:04:38.779 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 2, 2024-04-26T02:04:38.779 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 2, 2024-04-26T02:04:38.779 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-04-26T02:04:38.780 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 0, 2024-04-26T02:04:38.780 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 0, 2024-04-26T02:04:38.780 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 5479, 2024-04-26T02:04:38.780 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-04-26T02:04:38.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.780 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-26T02:04:38.780 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:04:38.780 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-26T02:04:38.780 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:04:38.780 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:04:38.780 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:04:38.780 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:04:38.780 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:04:38.780 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:04:38.780 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:04:38.780 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:04:38.781 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:04:38.781 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:04:38.781 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:04:38.781 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:04:38.781 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:04:38.781 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-26T02:04:38.781 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:04:38.781 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-26T02:04:38.781 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:04:38.781 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:04:38.781 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:04:38.781 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:04:38.781 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:04:38.781 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:04:38.781 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:04:38.781 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-26T02:04:38.782 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:04:38.782 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:04:38.782 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:04:38.782 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-26T02:04:38.782 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-26T02:04:38.782 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-26T02:04:38.782 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:04:38.782 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:04:38.782 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:04:38.782 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:04:38.782 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 182059, 2024-04-26T02:04:38.782 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 48, 2024-04-26T02:04:38.782 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:04:38.782 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-04-26T02:04:38.782 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-26T02:04:38.782 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-26T02:04:38.783 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:04:38.783 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-26T02:04:38.783 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:04:38.783 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-26T02:04:38.783 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 8224, 2024-04-26T02:04:38.783 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 171, 2024-04-26T02:04:38.783 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:04:38.783 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:04:38.783 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-26T02:04:38.783 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-26T02:04:38.783 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 17248, 2024-04-26T02:04:38.783 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 52, 2024-04-26T02:04:38.783 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-26T02:04:38.783 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-26T02:04:38.783 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-26T02:04:38.783 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-26T02:04:38.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.784 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-04-26T02:04:38.784 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-04-26T02:04:38.784 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-04-26T02:04:38.784 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 2815, 2024-04-26T02:04:38.784 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 21, 2024-04-26T02:04:38.784 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 3, 2024-04-26T02:04:38.784 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 2815, 2024-04-26T02:04:38.784 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-04-26T02:04:38.784 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2815, 2024-04-26T02:04:38.784 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1.508974967, 2024-04-26T02:04:38.784 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000536047 2024-04-26T02:04:38.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.784 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-04-26T02:04:38.784 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-04-26T02:04:38.784 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2815, 2024-04-26T02:04:38.785 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2816 2024-04-26T02:04:38.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.785 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 2815, 2024-04-26T02:04:38.785 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 2815, 2024-04-26T02:04:38.785 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 0, 2024-04-26T02:04:38.785 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-04-26T02:04:38.785 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-04-26T02:04:38.785 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 1787, 2024-04-26T02:04:38.785 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 0, 2024-04-26T02:04:38.785 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 1026, 2024-04-26T02:04:38.785 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 0, 2024-04-26T02:04:38.785 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 0, 2024-04-26T02:04:38.785 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-04-26T02:04:38.785 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-04-26T02:04:38.785 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-04-26T02:04:38.785 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-04-26T02:04:38.785 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 0, 2024-04-26T02:04:38.786 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-04-26T02:04:38.786 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-04-26T02:04:38.786 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-04-26T02:04:38.786 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 1, 2024-04-26T02:04:38.786 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 0, 2024-04-26T02:04:38.786 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-04-26T02:04:38.786 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-04-26T02:04:38.786 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-04-26T02:04:38.786 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-04-26T02:04:38.786 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-04-26T02:04:38.786 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-04-26T02:04:38.786 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T02:04:38.786 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-04-26T02:04:38.786 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-04-26T02:04:38.786 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-04-26T02:04:38.787 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-04-26T02:04:38.787 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-04-26T02:04:38.787 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-04-26T02:04:38.787 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-04-26T02:04:38.787 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-04-26T02:04:38.787 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-04-26T02:04:38.787 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-04-26T02:04:38.787 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-04-26T02:04:38.787 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-04-26T02:04:38.787 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-04-26T02:04:38.787 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-04-26T02:04:38.787 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-04-26T02:04:38.787 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-04-26T02:04:38.787 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-04-26T02:04:38.787 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-04-26T02:04:38.787 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-04-26T02:04:38.787 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 109, 2024-04-26T02:04:38.788 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-04-26T02:04:38.788 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 19, 2024-04-26T02:04:38.788 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 4, 2024-04-26T02:04:38.788 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 5, 2024-04-26T02:04:38.788 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 1, 2024-04-26T02:04:38.788 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-04-26T02:04:38.788 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 0, 2024-04-26T02:04:38.788 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 2, 2024-04-26T02:04:38.788 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 0 2024-04-26T02:04:38.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.788 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-04-26T02:04:38.788 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 0, 2024-04-26T02:04:38.788 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-04-26T02:04:38.788 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-04-26T02:04:38.788 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-04-26T02:04:38.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.788 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-04-26T02:04:38.789 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-04-26T02:04:38.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.789 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-04-26T02:04:38.789 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-04-26T02:04:38.789 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T02:04:38.789 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-04-26T02:04:38.789 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 0, 2024-04-26T02:04:38.789 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 0, 2024-04-26T02:04:38.789 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 0, 2024-04-26T02:04:38.789 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-04-26T02:04:38.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.789 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:04:38.789 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-26T02:04:38.789 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-26T02:04:38.789 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-26T02:04:38.789 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 3149, 2024-04-26T02:04:38.790 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 2235891, 2024-04-26T02:04:38.790 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:38.790 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 3149, 2024-04-26T02:04:38.790 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-26T02:04:38.790 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-26T02:04:38.790 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 3149, 2024-04-26T02:04:38.790 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 2235891, 2024-04-26T02:04:38.790 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-26T02:04:38.790 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.790 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.790 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.790 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:38.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.790 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-04-26T02:04:38.790 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-26T02:04:38.791 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-26T02:04:38.791 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-26T02:04:38.791 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-26T02:04:38.791 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-26T02:04:38.791 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:38.791 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-26T02:04:38.791 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 2815, 2024-04-26T02:04:38.791 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 49229, 2024-04-26T02:04:38.791 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2815, 2024-04-26T02:04:38.791 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 49229, 2024-04-26T02:04:38.791 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-26T02:04:38.791 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.791 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.791 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.791 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:38.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.791 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-04-26T02:04:38.792 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-26T02:04:38.792 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-04-26T02:04:38.792 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-26T02:04:38.792 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-26T02:04:38.792 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-26T02:04:38.792 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:38.792 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-26T02:04:38.792 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 2815, 2024-04-26T02:04:38.792 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 2815, 2024-04-26T02:04:38.792 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2815, 2024-04-26T02:04:38.792 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 2815, 2024-04-26T02:04:38.792 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-26T02:04:38.792 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.792 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.792 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.793 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:38.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.793 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-04-26T02:04:38.793 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-26T02:04:38.793 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-26T02:04:38.793 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-26T02:04:38.793 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-26T02:04:38.793 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-26T02:04:38.793 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:38.793 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-26T02:04:38.793 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-26T02:04:38.793 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-26T02:04:38.793 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-04-26T02:04:38.793 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-04-26T02:04:38.793 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-26T02:04:38.793 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.793 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.794 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.794 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:38.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:38.794 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x55ca9ed006e0": { 2024-04-26T02:04:38.794 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-26T02:04:38.794 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-26T02:04:38.794 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-26T02:04:38.794 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-26T02:04:38.794 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-26T02:04:38.794 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:38.794 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-26T02:04:38.794 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-26T02:04:38.794 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-26T02:04:38.794 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-04-26T02:04:38.794 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-04-26T02:04:38.795 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-26T02:04:38.795 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:38.795 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:38.795 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:38.795 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:38.795 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:38.795 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-26T02:04:39.529 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-26T02:04:39.529 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-26T02:04:39.529 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-26T02:04:39.529 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-04-26T02:04:39.529 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-04-26T02:04:39.529 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-26T02:04:39.529 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T02:04:39.529 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-04-26T02:04:39.529 DEBUG:tasks.check_counter:Getting stats from k 2024-04-26T02:04:39.530 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-26T02:04:39.764 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T02:04:40.153 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:40 smithi097 ceph-mon[32853]: pgmap v409: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:40.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:40 smithi001 ceph-mon[27822]: pgmap v409: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:40.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:40 smithi173 ceph-mon[32553]: pgmap v409: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:41.494 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-26T02:04:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:04:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 0, 2024-04-26T02:04:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 2, 2024-04-26T02:04:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 0, 2024-04-26T02:04:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 0, 2024-04-26T02:04:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2024-04-26T02:04:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 0, 2024-04-26T02:04:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.000989211, 2024-04-26T02:04:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T02:04:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.000228210, 2024-04-26T02:04:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:04:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T02:04:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T02:04:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:04:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 2, 2024-04-26T02:04:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 222, 2024-04-26T02:04:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 320, 2024-04-26T02:04:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 195280, 2024-04-26T02:04:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 5, 2024-04-26T02:04:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2024-04-26T02:04:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.012671768, 2024-04-26T02:04:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.008838759, 2024-04-26T02:04:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.001208304, 2024-04-26T02:04:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:04:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 219, 2024-04-26T02:04:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.046991585, 2024-04-26T02:04:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000214573 2024-04-26T02:04:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T02:04:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 195280 2024-04-26T02:04:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:04:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 306, 2024-04-26T02:04:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 293, 2024-04-26T02:04:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 1616112, 2024-04-26T02:04:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 180048, 2024-04-26T02:04:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 2, 2024-04-26T02:04:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 1, 2024-04-26T02:04:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.052796276, 2024-04-26T02:04:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.017084243, 2024-04-26T02:04:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.037016729, 2024-04-26T02:04:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.006844243, 2024-04-26T02:04:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 293, 2024-04-26T02:04:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.014265117, 2024-04-26T02:04:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000048686 2024-04-26T02:04:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1616112, 2024-04-26T02:04:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 180048 2024-04-26T02:04:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-26T02:04:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 0, 2024-04-26T02:04:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-26T02:04:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-26T02:04:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:04:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-26T02:04:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:04:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:04:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:04:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:04:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:04:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:04:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:04:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:04:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:04:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:04:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:04:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:04:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:04:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-26T02:04:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:04:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-26T02:04:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:04:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:04:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:04:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:04:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:04:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:04:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:04:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-26T02:04:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:04:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:04:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:04:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-26T02:04:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-26T02:04:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-26T02:04:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:04:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:04:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:04:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:04:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 148692, 2024-04-26T02:04:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 30, 2024-04-26T02:04:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:04:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-04-26T02:04:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-26T02:04:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-26T02:04:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:04:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-26T02:04:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:04:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-26T02:04:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 0, 2024-04-26T02:04:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 0, 2024-04-26T02:04:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:04:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:04:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-26T02:04:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-26T02:04:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 928, 2024-04-26T02:04:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 2, 2024-04-26T02:04:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-26T02:04:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-26T02:04:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-26T02:04:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-26T02:04:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-26T02:04:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:04:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-26T02:04:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-26T02:04:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-26T02:04:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 304, 2024-04-26T02:04:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 1570435, 2024-04-26T02:04:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 304, 2024-04-26T02:04:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-26T02:04:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-26T02:04:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 304, 2024-04-26T02:04:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 1570435, 2024-04-26T02:04:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-26T02:04:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-26T02:04:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-26T02:04:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-26T02:04:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-26T02:04:41.514 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-26T02:04:42.202 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-26T02:04:42.202 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-26T02:04:42.202 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-04-26T02:04:42.202 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-04-26T02:04:42.202 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-26T02:04:42.202 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-26T02:04:42.202 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-26T02:04:42.202 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-04-26T02:04:42.202 DEBUG:tasks.check_counter:Getting stats from c 2024-04-26T02:04:42.203 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-26T02:04:42.369 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:42 smithi173 ceph-mon[32553]: pgmap v410: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T02:04:42.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:42 smithi001 ceph-mon[27822]: pgmap v410: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T02:04:42.451 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T02:04:42.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:42 smithi097 ceph-mon[32853]: pgmap v410: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T02:04:43.951 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-26T02:04:43.951 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:04:43.952 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 6308, 2024-04-26T02:04:43.952 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 3872, 2024-04-26T02:04:43.952 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 5301768, 2024-04-26T02:04:43.952 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 1148910, 2024-04-26T02:04:43.952 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 11, 2024-04-26T02:04:43.952 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 3, 2024-04-26T02:04:43.952 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.303590104, 2024-04-26T02:04:43.952 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.116762070, 2024-04-26T02:04:43.952 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.243646555, 2024-04-26T02:04:43.952 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.007460004, 2024-04-26T02:04:43.952 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:43.952 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3874, 2024-04-26T02:04:43.952 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.102283092, 2024-04-26T02:04:43.953 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000026402 2024-04-26T02:04:43.953 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.953 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:43.953 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 9866, 2024-04-26T02:04:43.953 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.001813628, 2024-04-26T02:04:43.953 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000183 2024-04-26T02:04:43.953 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.953 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 928, 2024-04-26T02:04:43.953 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 1216 2024-04-26T02:04:43.953 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.953 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:04:43.953 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 413, 2024-04-26T02:04:43.953 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 412, 2024-04-26T02:04:43.954 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 1712006, 2024-04-26T02:04:43.954 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 216553, 2024-04-26T02:04:43.954 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 9, 2024-04-26T02:04:43.954 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 6, 2024-04-26T02:04:43.954 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.066001379, 2024-04-26T02:04:43.954 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.020904721, 2024-04-26T02:04:43.954 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.050240072, 2024-04-26T02:04:43.954 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.007389125, 2024-04-26T02:04:43.954 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:43.954 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 411, 2024-04-26T02:04:43.954 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.032694611, 2024-04-26T02:04:43.954 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000079548 2024-04-26T02:04:43.954 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.955 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:43.955 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 60, 2024-04-26T02:04:43.955 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000012204, 2024-04-26T02:04:43.955 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000203 2024-04-26T02:04:43.955 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.955 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 1704512, 2024-04-26T02:04:43.955 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 197312 2024-04-26T02:04:43.955 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.955 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:04:43.955 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 444, 2024-04-26T02:04:43.955 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 766, 2024-04-26T02:04:43.955 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 151085, 2024-04-26T02:04:43.955 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 735587, 2024-04-26T02:04:43.955 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 21, 2024-04-26T02:04:43.956 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 7, 2024-04-26T02:04:43.956 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.060104720, 2024-04-26T02:04:43.956 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.023931490, 2024-04-26T02:04:43.956 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.040918881, 2024-04-26T02:04:43.956 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.000287242, 2024-04-26T02:04:43.956 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:43.956 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 760, 2024-04-26T02:04:43.956 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.054054874, 2024-04-26T02:04:43.956 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000071124 2024-04-26T02:04:43.956 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.956 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:43.956 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 890, 2024-04-26T02:04:43.956 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000195053, 2024-04-26T02:04:43.956 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000219 2024-04-26T02:04:43.957 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.957 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 6224, 2024-04-26T02:04:43.957 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 541200 2024-04-26T02:04:43.957 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.957 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2024-04-26T02:04:43.957 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 1, 2024-04-26T02:04:43.957 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2024-04-26T02:04:43.957 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.957 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-MDSRank": { 2024-04-26T02:04:43.957 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-26T02:04:43.957 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-26T02:04:43.957 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 119, 2024-04-26T02:04:43.957 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.014506681, 2024-04-26T02:04:43.957 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000121904 2024-04-26T02:04:43.958 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:43.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.958 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-PurgeQueue": { 2024-04-26T02:04:43.958 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-26T02:04:43.958 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-26T02:04:43.958 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 17, 2024-04-26T02:04:43.958 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000431110, 2024-04-26T02:04:43.958 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000025359 2024-04-26T02:04:43.958 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:43.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.958 INFO:teuthology.orchestra.run.smithi173.stdout: "mds": { 2024-04-26T02:04:43.958 INFO:teuthology.orchestra.run.smithi173.stdout: "request": 343, 2024-04-26T02:04:43.958 INFO:teuthology.orchestra.run.smithi173.stdout: "reply": 17, 2024-04-26T02:04:43.959 INFO:teuthology.orchestra.run.smithi173.stdout: "reply_latency": { 2024-04-26T02:04:43.959 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 17, 2024-04-26T02:04:43.959 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.033507891, 2024-04-26T02:04:43.959 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001971052 2024-04-26T02:04:43.959 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.959 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_reply": 0, 2024-04-26T02:04:43.959 INFO:teuthology.orchestra.run.smithi173.stdout: "forward": 326, 2024-04-26T02:04:43.959 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_complete": 0, 2024-04-26T02:04:43.959 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_keys": 0, 2024-04-26T02:04:43.959 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_commit": 11, 2024-04-26T02:04:43.959 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_split": 0, 2024-04-26T02:04:43.959 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_merge": 0, 2024-04-26T02:04:43.959 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes": 31, 2024-04-26T02:04:43.959 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_top": 4, 2024-04-26T02:04:43.959 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_bottom": 11, 2024-04-26T02:04:43.960 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pin_tail": 16, 2024-04-26T02:04:43.960 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pinned": 27, 2024-04-26T02:04:43.960 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_expired": 118, 2024-04-26T02:04:43.960 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_with_caps": 1, 2024-04-26T02:04:43.960 INFO:teuthology.orchestra.run.smithi173.stdout: "caps": 1, 2024-04-26T02:04:43.960 INFO:teuthology.orchestra.run.smithi173.stdout: "subtrees": 3, 2024-04-26T02:04:43.960 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse": 642, 2024-04-26T02:04:43.960 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_hit": 250, 2024-04-26T02:04:43.960 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_forward": 324, 2024-04-26T02:04:43.960 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_discover": 56, 2024-04-26T02:04:43.960 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_dir_fetch": 0, 2024-04-26T02:04:43.960 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_remote_ino": 0, 2024-04-26T02:04:43.960 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_lock": 0, 2024-04-26T02:04:43.960 INFO:teuthology.orchestra.run.smithi173.stdout: "load_cent": 0, 2024-04-26T02:04:43.961 INFO:teuthology.orchestra.run.smithi173.stdout: "q": 0, 2024-04-26T02:04:43.961 INFO:teuthology.orchestra.run.smithi173.stdout: "exported": 1, 2024-04-26T02:04:43.961 INFO:teuthology.orchestra.run.smithi173.stdout: "exported_inodes": 6, 2024-04-26T02:04:43.961 INFO:teuthology.orchestra.run.smithi173.stdout: "imported": 1, 2024-04-26T02:04:43.961 INFO:teuthology.orchestra.run.smithi173.stdout: "imported_inodes": 11, 2024-04-26T02:04:43.961 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_dir_fetch": 0, 2024-04-26T02:04:43.961 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_backtrace_fetch": 0, 2024-04-26T02:04:43.961 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_peer_discover": 0, 2024-04-26T02:04:43.961 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rfiles": 4, 2024-04-26T02:04:43.961 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rbytes": 234, 2024-04-26T02:04:43.961 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rsnaps": 6, 2024-04-26T02:04:43.961 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T02:04:43.961 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_set_tag": 0, 2024-04-26T02:04:43.961 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T02:04:43.962 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_inotable_repaired": 0, 2024-04-26T02:04:43.962 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_inodes": 0, 2024-04-26T02:04:43.962 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T02:04:43.962 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T02:04:43.962 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_file_inodes": 0, 2024-04-26T02:04:43.962 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_inode_file_caps": 0, 2024-04-26T02:04:43.962 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_revoke": 22, 2024-04-26T02:04:43.962 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_grant": 21, 2024-04-26T02:04:43.962 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T02:04:43.962 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T02:04:43.962 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T02:04:43.962 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps": 25, 2024-04-26T02:04:43.962 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps_dirty": 2, 2024-04-26T02:04:43.963 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_cap_release": 2, 2024-04-26T02:04:43.963 INFO:teuthology.orchestra.run.smithi173.stdout: "process_request_cap_release": 15 2024-04-26T02:04:43.963 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.963 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_cache": { 2024-04-26T02:04:43.963 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update": 7, 2024-04-26T02:04:43.963 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update_receipt": 466, 2024-04-26T02:04:43.963 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_try_discover": 260, 2024-04-26T02:04:43.963 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_send_discover": 262, 2024-04-26T02:04:43.963 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_handle_discover": 13, 2024-04-26T02:04:43.963 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays": 4, 2024-04-26T02:04:43.963 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_delayed": 0, 2024-04-26T02:04:43.963 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_enqueuing": 0, 2024-04-26T02:04:43.963 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_created": 6, 2024-04-26T02:04:43.963 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_enqueued": 2, 2024-04-26T02:04:43.964 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_reintegrated": 0, 2024-04-26T02:04:43.964 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_migrated": 0, 2024-04-26T02:04:43.964 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_processing": 0, 2024-04-26T02:04:43.964 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_enqueued": 0, 2024-04-26T02:04:43.964 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_prioritized": 0, 2024-04-26T02:04:43.964 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_started": 0, 2024-04-26T02:04:43.964 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_completed": 0, 2024-04-26T02:04:43.964 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_path": 9, 2024-04-26T02:04:43.964 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_inode": 336, 2024-04-26T02:04:43.964 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T02:04:43.964 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_exportdir": 1, 2024-04-26T02:04:43.964 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_flush": 0, 2024-04-26T02:04:43.964 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragmentdir": 0, 2024-04-26T02:04:43.964 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragstats": 0, 2024-04-26T02:04:43.964 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_inodestats": 0 2024-04-26T02:04:43.965 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.965 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_log": { 2024-04-26T02:04:43.965 INFO:teuthology.orchestra.run.smithi173.stdout: "evlrg": 0, 2024-04-26T02:04:43.965 INFO:teuthology.orchestra.run.smithi173.stdout: "evadd": 35, 2024-04-26T02:04:43.965 INFO:teuthology.orchestra.run.smithi173.stdout: "evex": 0, 2024-04-26T02:04:43.965 INFO:teuthology.orchestra.run.smithi173.stdout: "evtrm": 0, 2024-04-26T02:04:43.965 INFO:teuthology.orchestra.run.smithi173.stdout: "ev": 35, 2024-04-26T02:04:43.965 INFO:teuthology.orchestra.run.smithi173.stdout: "evexg": 0, 2024-04-26T02:04:43.965 INFO:teuthology.orchestra.run.smithi173.stdout: "evexd": 0, 2024-04-26T02:04:43.965 INFO:teuthology.orchestra.run.smithi173.stdout: "segadd": 2, 2024-04-26T02:04:43.965 INFO:teuthology.orchestra.run.smithi173.stdout: "segex": 0, 2024-04-26T02:04:43.965 INFO:teuthology.orchestra.run.smithi173.stdout: "segtrm": 0, 2024-04-26T02:04:43.965 INFO:teuthology.orchestra.run.smithi173.stdout: "seg": 2, 2024-04-26T02:04:43.965 INFO:teuthology.orchestra.run.smithi173.stdout: "segmjr": 2, 2024-04-26T02:04:43.966 INFO:teuthology.orchestra.run.smithi173.stdout: "segexg": 0, 2024-04-26T02:04:43.966 INFO:teuthology.orchestra.run.smithi173.stdout: "segexd": 0, 2024-04-26T02:04:43.966 INFO:teuthology.orchestra.run.smithi173.stdout: "expos": 4194304, 2024-04-26T02:04:43.966 INFO:teuthology.orchestra.run.smithi173.stdout: "wrpos": 4262818, 2024-04-26T02:04:43.966 INFO:teuthology.orchestra.run.smithi173.stdout: "rdpos": 0, 2024-04-26T02:04:43.966 INFO:teuthology.orchestra.run.smithi173.stdout: "jlat": { 2024-04-26T02:04:43.966 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 29, 2024-04-26T02:04:43.966 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.113990073, 2024-04-26T02:04:43.966 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.003930692 2024-04-26T02:04:43.966 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.966 INFO:teuthology.orchestra.run.smithi173.stdout: "replayed": 0 2024-04-26T02:04:43.966 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.966 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_mem": { 2024-04-26T02:04:43.967 INFO:teuthology.orchestra.run.smithi173.stdout: "ino": 29, 2024-04-26T02:04:43.967 INFO:teuthology.orchestra.run.smithi173.stdout: "ino+": 89, 2024-04-26T02:04:43.967 INFO:teuthology.orchestra.run.smithi173.stdout: "ino-": 60, 2024-04-26T02:04:43.967 INFO:teuthology.orchestra.run.smithi173.stdout: "dir": 26, 2024-04-26T02:04:43.967 INFO:teuthology.orchestra.run.smithi173.stdout: "dir+": 238, 2024-04-26T02:04:43.967 INFO:teuthology.orchestra.run.smithi173.stdout: "dir-": 212, 2024-04-26T02:04:43.967 INFO:teuthology.orchestra.run.smithi173.stdout: "dn": 31, 2024-04-26T02:04:43.967 INFO:teuthology.orchestra.run.smithi173.stdout: "dn+": 154, 2024-04-26T02:04:43.967 INFO:teuthology.orchestra.run.smithi173.stdout: "dn-": 123, 2024-04-26T02:04:43.967 INFO:teuthology.orchestra.run.smithi173.stdout: "cap": 1, 2024-04-26T02:04:43.967 INFO:teuthology.orchestra.run.smithi173.stdout: "cap+": 14, 2024-04-26T02:04:43.968 INFO:teuthology.orchestra.run.smithi173.stdout: "cap-": 13, 2024-04-26T02:04:43.968 INFO:teuthology.orchestra.run.smithi173.stdout: "rss": 57808, 2024-04-26T02:04:43.968 INFO:teuthology.orchestra.run.smithi173.stdout: "heap": 223608 2024-04-26T02:04:43.968 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.968 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_server": { 2024-04-26T02:04:43.968 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_client_request": 375, 2024-04-26T02:04:43.968 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_server_request": 4, 2024-04-26T02:04:43.968 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_request": 343, 2024-04-26T02:04:43.968 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_session": 37, 2024-04-26T02:04:43.968 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_peer_request": 313, 2024-04-26T02:04:43.968 INFO:teuthology.orchestra.run.smithi173.stdout: "req_create_latency": { 2024-04-26T02:04:43.968 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.968 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.968 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.969 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getattr_latency": { 2024-04-26T02:04:43.969 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2, 2024-04-26T02:04:43.969 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.001709866, 2024-04-26T02:04:43.969 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000854933 2024-04-26T02:04:43.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.969 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getfilelock_latency": { 2024-04-26T02:04:43.969 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.969 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.969 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.969 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.969 INFO:teuthology.orchestra.run.smithi173.stdout: "req_link_latency": { 2024-04-26T02:04:43.969 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.969 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.970 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.970 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookup_latency": { 2024-04-26T02:04:43.970 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 4, 2024-04-26T02:04:43.970 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.001131598, 2024-04-26T02:04:43.970 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000282899 2024-04-26T02:04:43.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.970 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookuphash_latency": { 2024-04-26T02:04:43.970 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.970 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.970 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.970 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupino_latency": { 2024-04-26T02:04:43.970 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1, 2024-04-26T02:04:43.971 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000240638, 2024-04-26T02:04:43.971 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000240638 2024-04-26T02:04:43.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.971 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupname_latency": { 2024-04-26T02:04:43.971 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.971 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.971 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.971 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupparent_latency": { 2024-04-26T02:04:43.971 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.971 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.971 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.971 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.971 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupsnap_latency": { 2024-04-26T02:04:43.972 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.972 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.972 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.972 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.972 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lssnap_latency": { 2024-04-26T02:04:43.972 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.972 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.972 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.972 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.973 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mkdir_latency": { 2024-04-26T02:04:43.973 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.973 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.973 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.973 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.973 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mknod_latency": { 2024-04-26T02:04:43.973 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.973 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.973 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.973 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.973 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mksnap_latency": { 2024-04-26T02:04:43.974 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.974 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.974 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.974 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.974 INFO:teuthology.orchestra.run.smithi173.stdout: "req_open_latency": { 2024-04-26T02:04:43.974 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.974 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.974 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.974 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.974 INFO:teuthology.orchestra.run.smithi173.stdout: "req_readdir_latency": { 2024-04-26T02:04:43.974 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 4, 2024-04-26T02:04:43.974 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.001765685, 2024-04-26T02:04:43.974 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000441421 2024-04-26T02:04:43.975 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.975 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rename_latency": { 2024-04-26T02:04:43.975 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.975 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.975 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.975 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.975 INFO:teuthology.orchestra.run.smithi173.stdout: "req_renamesnap_latency": { 2024-04-26T02:04:43.975 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.975 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.975 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.975 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.975 INFO:teuthology.orchestra.run.smithi173.stdout: "req_snapdiff_latency": { 2024-04-26T02:04:43.975 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.975 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.976 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.976 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.976 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmdir_latency": { 2024-04-26T02:04:43.976 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 4, 2024-04-26T02:04:43.976 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.026259512, 2024-04-26T02:04:43.976 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.006564878 2024-04-26T02:04:43.976 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.976 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmsnap_latency": { 2024-04-26T02:04:43.976 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.976 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.976 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.976 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.976 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmxattr_latency": { 2024-04-26T02:04:43.976 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.977 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.977 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.977 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.977 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setattr_latency": { 2024-04-26T02:04:43.977 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.977 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.977 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.977 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.977 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setdirlayout_latency": { 2024-04-26T02:04:43.977 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.977 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.977 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.977 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.977 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setfilelock_latency": { 2024-04-26T02:04:43.977 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.978 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.978 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.978 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.978 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setlayout_latency": { 2024-04-26T02:04:43.978 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.978 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.978 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.978 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.978 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setxattr_latency": { 2024-04-26T02:04:43.978 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.978 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.978 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.978 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.979 INFO:teuthology.orchestra.run.smithi173.stdout: "req_symlink_latency": { 2024-04-26T02:04:43.979 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.979 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.979 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.979 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.979 INFO:teuthology.orchestra.run.smithi173.stdout: "req_unlink_latency": { 2024-04-26T02:04:43.979 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2, 2024-04-26T02:04:43.979 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.002400592, 2024-04-26T02:04:43.979 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001200296 2024-04-26T02:04:43.979 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.979 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_revoke_eviction": 0, 2024-04-26T02:04:43.979 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_acquisition_throttle": 0, 2024-04-26T02:04:43.979 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getvxattr_latency": { 2024-04-26T02:04:43.979 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.979 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.980 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.980 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:43.980 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.980 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_sessions": { 2024-04-26T02:04:43.980 INFO:teuthology.orchestra.run.smithi173.stdout: "session_count": 2, 2024-04-26T02:04:43.980 INFO:teuthology.orchestra.run.smithi173.stdout: "session_add": 6, 2024-04-26T02:04:43.980 INFO:teuthology.orchestra.run.smithi173.stdout: "session_remove": 4, 2024-04-26T02:04:43.980 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_open": 2, 2024-04-26T02:04:43.980 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_stale": 0, 2024-04-26T02:04:43.980 INFO:teuthology.orchestra.run.smithi173.stdout: "total_load": 11, 2024-04-26T02:04:43.980 INFO:teuthology.orchestra.run.smithi173.stdout: "average_load": 5, 2024-04-26T02:04:43.980 INFO:teuthology.orchestra.run.smithi173.stdout: "avg_session_uptime": 772, 2024-04-26T02:04:43.980 INFO:teuthology.orchestra.run.smithi173.stdout: "mdthresh_evicted": 0 2024-04-26T02:04:43.980 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.981 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2024-04-26T02:04:43.981 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:04:43.981 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2024-04-26T02:04:43.981 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:04:43.981 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:04:43.981 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:04:43.981 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:04:43.981 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:04:43.981 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:04:43.981 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:04:43.981 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:04:43.981 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:04:43.981 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:04:43.981 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:04:43.981 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:04:43.982 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:04:43.982 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2024-04-26T02:04:43.982 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:04:43.982 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2024-04-26T02:04:43.982 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:04:43.982 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:04:43.982 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:04:43.982 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:04:43.982 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:04:43.982 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:04:43.982 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:04:43.982 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2024-04-26T02:04:43.982 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:04:43.982 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:04:43.983 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:04:43.983 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2024-04-26T02:04:43.983 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2024-04-26T02:04:43.983 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2024-04-26T02:04:43.983 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:04:43.983 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:04:43.983 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:04:43.983 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:04:43.983 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 311509, 2024-04-26T02:04:43.983 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 107, 2024-04-26T02:04:43.983 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:04:43.983 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 0, 2024-04-26T02:04:43.983 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2024-04-26T02:04:43.983 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2024-04-26T02:04:43.984 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:04:43.984 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2024-04-26T02:04:43.984 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:04:43.984 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2024-04-26T02:04:43.984 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 8224, 2024-04-26T02:04:43.984 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 171, 2024-04-26T02:04:43.984 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:04:43.984 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:04:43.984 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2024-04-26T02:04:43.984 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2024-04-26T02:04:43.984 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 129833, 2024-04-26T02:04:43.984 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 1185, 2024-04-26T02:04:43.984 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2024-04-26T02:04:43.984 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2024-04-26T02:04:43.984 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2024-04-26T02:04:43.984 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2024-04-26T02:04:43.985 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.985 INFO:teuthology.orchestra.run.smithi173.stdout: "objecter": { 2024-04-26T02:04:43.985 INFO:teuthology.orchestra.run.smithi173.stdout: "op_active": 0, 2024-04-26T02:04:43.985 INFO:teuthology.orchestra.run.smithi173.stdout: "op_laggy": 0, 2024-04-26T02:04:43.985 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send": 91, 2024-04-26T02:04:43.985 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send_bytes": 82057, 2024-04-26T02:04:43.985 INFO:teuthology.orchestra.run.smithi173.stdout: "op_resend": 33, 2024-04-26T02:04:43.985 INFO:teuthology.orchestra.run.smithi173.stdout: "op_reply": 91, 2024-04-26T02:04:43.985 INFO:teuthology.orchestra.run.smithi173.stdout: "op_latency": { 2024-04-26T02:04:43.985 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 91, 2024-04-26T02:04:43.985 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.691983412, 2024-04-26T02:04:43.985 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.007604213 2024-04-26T02:04:43.985 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.985 INFO:teuthology.orchestra.run.smithi173.stdout: "op_inflight": 0, 2024-04-26T02:04:43.985 INFO:teuthology.orchestra.run.smithi173.stdout: "oplen_avg": { 2024-04-26T02:04:43.986 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 91, 2024-04-26T02:04:43.986 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 124 2024-04-26T02:04:43.986 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.986 INFO:teuthology.orchestra.run.smithi173.stdout: "op": 91, 2024-04-26T02:04:43.986 INFO:teuthology.orchestra.run.smithi173.stdout: "op_r": 1, 2024-04-26T02:04:43.986 INFO:teuthology.orchestra.run.smithi173.stdout: "op_w": 90, 2024-04-26T02:04:43.986 INFO:teuthology.orchestra.run.smithi173.stdout: "op_rmw": 0, 2024-04-26T02:04:43.986 INFO:teuthology.orchestra.run.smithi173.stdout: "op_pg": 0, 2024-04-26T02:04:43.986 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_stat": 0, 2024-04-26T02:04:43.986 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_create": 10, 2024-04-26T02:04:43.986 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_read": 1, 2024-04-26T02:04:43.986 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_write": 30, 2024-04-26T02:04:43.986 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writefull": 14, 2024-04-26T02:04:43.986 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writesame": 0, 2024-04-26T02:04:43.987 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_append": 0, 2024-04-26T02:04:43.987 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_zero": 0, 2024-04-26T02:04:43.987 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_truncate": 0, 2024-04-26T02:04:43.987 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_delete": 14, 2024-04-26T02:04:43.987 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_mapext": 0, 2024-04-26T02:04:43.987 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_sparse_read": 0, 2024-04-26T02:04:43.987 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_clonerange": 0, 2024-04-26T02:04:43.987 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_getxattr": 0, 2024-04-26T02:04:43.987 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_setxattr": 20, 2024-04-26T02:04:43.987 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_cmpxattr": 0, 2024-04-26T02:04:43.987 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_rmxattr": 0, 2024-04-26T02:04:43.987 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_resetxattrs": 0, 2024-04-26T02:04:43.987 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_call": 0, 2024-04-26T02:04:43.987 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_watch": 0, 2024-04-26T02:04:43.987 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_notify": 0, 2024-04-26T02:04:43.988 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T02:04:43.988 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls": 0, 2024-04-26T02:04:43.988 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls_filter": 0, 2024-04-26T02:04:43.988 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_other": 0, 2024-04-26T02:04:43.988 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_active": 0, 2024-04-26T02:04:43.988 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_send": 0, 2024-04-26T02:04:43.988 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_resend": 0, 2024-04-26T02:04:43.988 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_ping": 0, 2024-04-26T02:04:43.988 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_active": 0, 2024-04-26T02:04:43.988 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_send": 0, 2024-04-26T02:04:43.988 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_resend": 0, 2024-04-26T02:04:43.988 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_active": 0, 2024-04-26T02:04:43.988 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_send": 0, 2024-04-26T02:04:43.988 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_resend": 0, 2024-04-26T02:04:43.988 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_active": 0, 2024-04-26T02:04:43.988 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_send": 0, 2024-04-26T02:04:43.989 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_resend": 0, 2024-04-26T02:04:43.989 INFO:teuthology.orchestra.run.smithi173.stdout: "command_active": 0, 2024-04-26T02:04:43.989 INFO:teuthology.orchestra.run.smithi173.stdout: "command_send": 0, 2024-04-26T02:04:43.989 INFO:teuthology.orchestra.run.smithi173.stdout: "command_resend": 0, 2024-04-26T02:04:43.989 INFO:teuthology.orchestra.run.smithi173.stdout: "map_epoch": 109, 2024-04-26T02:04:43.989 INFO:teuthology.orchestra.run.smithi173.stdout: "map_full": 0, 2024-04-26T02:04:43.989 INFO:teuthology.orchestra.run.smithi173.stdout: "map_inc": 19, 2024-04-26T02:04:43.989 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_sessions": 11, 2024-04-26T02:04:43.989 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_open": 15, 2024-04-26T02:04:43.989 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_close": 4, 2024-04-26T02:04:43.989 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_laggy": 0, 2024-04-26T02:04:43.989 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_wr": 29, 2024-04-26T02:04:43.989 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_rd": 0, 2024-04-26T02:04:43.989 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_del": 6 2024-04-26T02:04:43.989 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.990 INFO:teuthology.orchestra.run.smithi173.stdout: "oft": { 2024-04-26T02:04:43.990 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_objs": 1, 2024-04-26T02:04:43.990 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_kv_pairs": 0, 2024-04-26T02:04:43.990 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_updates": 0, 2024-04-26T02:04:43.990 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_removes": 0 2024-04-26T02:04:43.990 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.990 INFO:teuthology.orchestra.run.smithi173.stdout: "osd-slow-ops": { 2024-04-26T02:04:43.990 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_ops_count": 345 2024-04-26T02:04:43.990 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.990 INFO:teuthology.orchestra.run.smithi173.stdout: "purge_queue": { 2024-04-26T02:04:43.990 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops": 0, 2024-04-26T02:04:43.990 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops_high_water": 2, 2024-04-26T02:04:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing": 0, 2024-04-26T02:04:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_high_water": 2, 2024-04-26T02:04:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed_ops": 2, 2024-04-26T02:04:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed": 2, 2024-04-26T02:04:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_item_in_journal": 0 2024-04-26T02:04:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:04:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-26T02:04:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-26T02:04:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-26T02:04:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 7161, 2024-04-26T02:04:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 6497253, 2024-04-26T02:04:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 7161, 2024-04-26T02:04:43.991 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-26T02:04:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-26T02:04:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 7161, 2024-04-26T02:04:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 6497253, 2024-04-26T02:04:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-26T02:04:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_bytes": { 2024-04-26T02:04:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-26T02:04:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-26T02:04:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-26T02:04:43.992 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-26T02:04:43.993 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-26T02:04:43.993 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:43.993 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-26T02:04:43.993 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 91, 2024-04-26T02:04:43.993 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 82355, 2024-04-26T02:04:43.993 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 91, 2024-04-26T02:04:43.993 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 82355, 2024-04-26T02:04:43.993 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-26T02:04:43.993 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.993 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.993 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.993 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:43.993 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.993 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_ops": { 2024-04-26T02:04:43.994 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-26T02:04:43.994 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 1024, 2024-04-26T02:04:43.994 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-26T02:04:43.994 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-26T02:04:43.994 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-26T02:04:43.994 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:43.994 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-26T02:04:43.994 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 91, 2024-04-26T02:04:43.994 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 91, 2024-04-26T02:04:43.994 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 91, 2024-04-26T02:04:43.994 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 91, 2024-04-26T02:04:43.994 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-26T02:04:43.994 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.994 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.995 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.995 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:43.995 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.995 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle": { 2024-04-26T02:04:43.995 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-26T02:04:43.995 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-26T02:04:43.995 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-26T02:04:43.995 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 2, 2024-04-26T02:04:43.995 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 298, 2024-04-26T02:04:43.995 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:43.995 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 2, 2024-04-26T02:04:43.995 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-26T02:04:43.995 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-26T02:04:43.995 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 1, 2024-04-26T02:04:43.995 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 298, 2024-04-26T02:04:43.996 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-26T02:04:43.996 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.996 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.996 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.996 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:43.996 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:43.996 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle-0x563f4e4f0460": { 2024-04-26T02:04:43.996 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-26T02:04:43.996 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-26T02:04:43.996 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-26T02:04:43.996 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 35, 2024-04-26T02:04:43.996 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 68514, 2024-04-26T02:04:43.996 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:43.996 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 35, 2024-04-26T02:04:43.997 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-26T02:04:43.997 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-26T02:04:43.997 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 29, 2024-04-26T02:04:43.997 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 68514, 2024-04-26T02:04:43.997 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-26T02:04:43.997 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:43.997 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:43.997 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:43.997 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:43.997 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:43.997 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-26T02:04:44.286 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:44 smithi097 ceph-mon[32853]: pgmap v411: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:44.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:44 smithi173 ceph-mon[32553]: pgmap v411: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:44.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:44 smithi001 ceph-mon[27822]: pgmap v411: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:44.640 INFO:tasks.check_counter:Daemon mds.c mds.exported=1 2024-04-26T02:04:44.640 INFO:tasks.check_counter:Daemon mds.c mds.imported=1 2024-04-26T02:04:44.640 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=7 2024-04-26T02:04:44.640 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=466 2024-04-26T02:04:44.640 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-26T02:04:44.641 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-26T02:04:44.641 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T02:04:44.641 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-26T02:04:44.641 DEBUG:tasks.check_counter:Getting stats from f 2024-04-26T02:04:44.641 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-26T02:04:44.885 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T02:04:46.207 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:46 smithi173 ceph-mon[32553]: pgmap v412: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T02:04:46.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:46 smithi001 ceph-mon[27822]: pgmap v412: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T02:04:46.429 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-26T02:04:46.429 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:04:46.429 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 160, 2024-04-26T02:04:46.429 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 155, 2024-04-26T02:04:46.429 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 12762558, 2024-04-26T02:04:46.429 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 58919, 2024-04-26T02:04:46.430 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 6, 2024-04-26T02:04:46.430 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 2, 2024-04-26T02:04:46.430 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.020616334, 2024-04-26T02:04:46.430 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.004675811, 2024-04-26T02:04:46.430 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.012131774, 2024-04-26T02:04:46.430 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.003062562, 2024-04-26T02:04:46.430 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:46.430 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 155, 2024-04-26T02:04:46.430 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.007866225, 2024-04-26T02:04:46.430 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000050749 2024-04-26T02:04:46.430 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.430 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:46.430 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.431 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.431 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.431 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 181120, 2024-04-26T02:04:46.431 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 26800 2024-04-26T02:04:46.431 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.431 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:04:46.431 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 257, 2024-04-26T02:04:46.431 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 288, 2024-04-26T02:04:46.431 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 8473915, 2024-04-26T02:04:46.431 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 219645, 2024-04-26T02:04:46.431 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 6, 2024-04-26T02:04:46.431 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 5, 2024-04-26T02:04:46.431 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.033116612, 2024-04-26T02:04:46.432 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.008804756, 2024-04-26T02:04:46.432 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.012999266, 2024-04-26T02:04:46.432 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.008686029, 2024-04-26T02:04:46.432 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:46.432 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 286, 2024-04-26T02:04:46.432 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.014270599, 2024-04-26T02:04:46.432 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000049897 2024-04-26T02:04:46.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.432 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:46.432 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.432 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.432 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.432 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.432 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-26T02:04:46.433 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 138176 2024-04-26T02:04:46.433 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.433 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:04:46.433 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 196, 2024-04-26T02:04:46.433 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 198, 2024-04-26T02:04:46.433 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 23519232, 2024-04-26T02:04:46.433 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 62343, 2024-04-26T02:04:46.433 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 6, 2024-04-26T02:04:46.433 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 5, 2024-04-26T02:04:46.433 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.026879422, 2024-04-26T02:04:46.433 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.005395022, 2024-04-26T02:04:46.433 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.015221267, 2024-04-26T02:04:46.433 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.004735828, 2024-04-26T02:04:46.433 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:46.433 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 196, 2024-04-26T02:04:46.434 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.014561777, 2024-04-26T02:04:46.434 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000074294 2024-04-26T02:04:46.434 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.434 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:46.434 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.434 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.434 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.434 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.434 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T02:04:46.434 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T02:04:46.434 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.434 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2024-04-26T02:04:46.434 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 1, 2024-04-26T02:04:46.435 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2024-04-26T02:04:46.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.435 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-MDSRank": { 2024-04-26T02:04:46.435 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-26T02:04:46.435 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-26T02:04:46.435 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 897, 2024-04-26T02:04:46.435 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.063007797, 2024-04-26T02:04:46.435 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000070242 2024-04-26T02:04:46.435 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:46.435 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.435 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-PurgeQueue": { 2024-04-26T02:04:46.435 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-26T02:04:46.435 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-26T02:04:46.435 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.436 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.436 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.436 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:46.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.436 INFO:teuthology.orchestra.run.smithi173.stdout: "mds": { 2024-04-26T02:04:46.436 INFO:teuthology.orchestra.run.smithi173.stdout: "request": 0, 2024-04-26T02:04:46.436 INFO:teuthology.orchestra.run.smithi173.stdout: "reply": 0, 2024-04-26T02:04:46.436 INFO:teuthology.orchestra.run.smithi173.stdout: "reply_latency": { 2024-04-26T02:04:46.436 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.436 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.436 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.436 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.436 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_reply": 0, 2024-04-26T02:04:46.436 INFO:teuthology.orchestra.run.smithi173.stdout: "forward": 0, 2024-04-26T02:04:46.437 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_complete": 0, 2024-04-26T02:04:46.437 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_keys": 0, 2024-04-26T02:04:46.437 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_commit": 0, 2024-04-26T02:04:46.437 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_split": 0, 2024-04-26T02:04:46.437 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_merge": 0, 2024-04-26T02:04:46.437 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes": 13888, 2024-04-26T02:04:46.437 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_top": 9676, 2024-04-26T02:04:46.437 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_bottom": 4212, 2024-04-26T02:04:46.437 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pin_tail": 0, 2024-04-26T02:04:46.437 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pinned": 64, 2024-04-26T02:04:46.437 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_expired": 0, 2024-04-26T02:04:46.437 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_with_caps": 0, 2024-04-26T02:04:46.437 INFO:teuthology.orchestra.run.smithi173.stdout: "caps": 0, 2024-04-26T02:04:46.437 INFO:teuthology.orchestra.run.smithi173.stdout: "subtrees": 2, 2024-04-26T02:04:46.437 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse": 0, 2024-04-26T02:04:46.438 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_hit": 0, 2024-04-26T02:04:46.438 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_forward": 0, 2024-04-26T02:04:46.438 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_discover": 0, 2024-04-26T02:04:46.438 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_dir_fetch": 0, 2024-04-26T02:04:46.438 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_remote_ino": 0, 2024-04-26T02:04:46.438 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_lock": 0, 2024-04-26T02:04:46.438 INFO:teuthology.orchestra.run.smithi173.stdout: "load_cent": 0, 2024-04-26T02:04:46.438 INFO:teuthology.orchestra.run.smithi173.stdout: "q": 0, 2024-04-26T02:04:46.438 INFO:teuthology.orchestra.run.smithi173.stdout: "exported": 0, 2024-04-26T02:04:46.438 INFO:teuthology.orchestra.run.smithi173.stdout: "exported_inodes": 0, 2024-04-26T02:04:46.438 INFO:teuthology.orchestra.run.smithi173.stdout: "imported": 0, 2024-04-26T02:04:46.438 INFO:teuthology.orchestra.run.smithi173.stdout: "imported_inodes": 0, 2024-04-26T02:04:46.438 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_dir_fetch": 0, 2024-04-26T02:04:46.439 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_backtrace_fetch": 0, 2024-04-26T02:04:46.439 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_peer_discover": 0, 2024-04-26T02:04:46.439 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rfiles": 4, 2024-04-26T02:04:46.439 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rbytes": 234, 2024-04-26T02:04:46.439 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rsnaps": 6, 2024-04-26T02:04:46.439 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T02:04:46.439 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_set_tag": 0, 2024-04-26T02:04:46.439 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T02:04:46.439 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_inotable_repaired": 0, 2024-04-26T02:04:46.439 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_inodes": 0, 2024-04-26T02:04:46.439 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T02:04:46.439 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T02:04:46.439 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_file_inodes": 0, 2024-04-26T02:04:46.439 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_inode_file_caps": 0, 2024-04-26T02:04:46.439 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T02:04:46.440 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_grant": 0, 2024-04-26T02:04:46.440 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T02:04:46.440 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T02:04:46.440 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T02:04:46.440 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps": 0, 2024-04-26T02:04:46.440 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps_dirty": 0, 2024-04-26T02:04:46.440 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_cap_release": 0, 2024-04-26T02:04:46.440 INFO:teuthology.orchestra.run.smithi173.stdout: "process_request_cap_release": 0 2024-04-26T02:04:46.440 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.440 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_cache": { 2024-04-26T02:04:46.440 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update": 0, 2024-04-26T02:04:46.440 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update_receipt": 0, 2024-04-26T02:04:46.440 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_try_discover": 0, 2024-04-26T02:04:46.440 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_send_discover": 0, 2024-04-26T02:04:46.441 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_handle_discover": 0, 2024-04-26T02:04:46.441 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays": 0, 2024-04-26T02:04:46.441 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_delayed": 0, 2024-04-26T02:04:46.441 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_enqueuing": 0, 2024-04-26T02:04:46.441 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_created": 0, 2024-04-26T02:04:46.441 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_enqueued": 0, 2024-04-26T02:04:46.441 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_reintegrated": 0, 2024-04-26T02:04:46.441 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_migrated": 0, 2024-04-26T02:04:46.441 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_processing": 0, 2024-04-26T02:04:46.441 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_enqueued": 0, 2024-04-26T02:04:46.441 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_prioritized": 0, 2024-04-26T02:04:46.441 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_started": 0, 2024-04-26T02:04:46.441 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_completed": 0, 2024-04-26T02:04:46.441 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_path": 0, 2024-04-26T02:04:46.442 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_inode": 0, 2024-04-26T02:04:46.442 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T02:04:46.442 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_exportdir": 0, 2024-04-26T02:04:46.442 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_flush": 0, 2024-04-26T02:04:46.442 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragmentdir": 0, 2024-04-26T02:04:46.442 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragstats": 0, 2024-04-26T02:04:46.442 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_inodestats": 0 2024-04-26T02:04:46.442 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.442 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_log": { 2024-04-26T02:04:46.442 INFO:teuthology.orchestra.run.smithi173.stdout: "evlrg": 0, 2024-04-26T02:04:46.442 INFO:teuthology.orchestra.run.smithi173.stdout: "evadd": 0, 2024-04-26T02:04:46.442 INFO:teuthology.orchestra.run.smithi173.stdout: "evex": 0, 2024-04-26T02:04:46.442 INFO:teuthology.orchestra.run.smithi173.stdout: "evtrm": 0, 2024-04-26T02:04:46.442 INFO:teuthology.orchestra.run.smithi173.stdout: "ev": 11874, 2024-04-26T02:04:46.442 INFO:teuthology.orchestra.run.smithi173.stdout: "evexg": 0, 2024-04-26T02:04:46.443 INFO:teuthology.orchestra.run.smithi173.stdout: "evexd": 0, 2024-04-26T02:04:46.443 INFO:teuthology.orchestra.run.smithi173.stdout: "segadd": 0, 2024-04-26T02:04:46.443 INFO:teuthology.orchestra.run.smithi173.stdout: "segex": 0, 2024-04-26T02:04:46.443 INFO:teuthology.orchestra.run.smithi173.stdout: "segtrm": 0, 2024-04-26T02:04:46.443 INFO:teuthology.orchestra.run.smithi173.stdout: "seg": 2, 2024-04-26T02:04:46.443 INFO:teuthology.orchestra.run.smithi173.stdout: "segmjr": 9, 2024-04-26T02:04:46.443 INFO:teuthology.orchestra.run.smithi173.stdout: "segexg": 0, 2024-04-26T02:04:46.443 INFO:teuthology.orchestra.run.smithi173.stdout: "segexd": 0, 2024-04-26T02:04:46.443 INFO:teuthology.orchestra.run.smithi173.stdout: "expos": 182834806, 2024-04-26T02:04:46.443 INFO:teuthology.orchestra.run.smithi173.stdout: "wrpos": 182840288, 2024-04-26T02:04:46.443 INFO:teuthology.orchestra.run.smithi173.stdout: "rdpos": 182840176, 2024-04-26T02:04:46.443 INFO:teuthology.orchestra.run.smithi173.stdout: "jlat": { 2024-04-26T02:04:46.443 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.443 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.443 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.444 INFO:teuthology.orchestra.run.smithi173.stdout: "replayed": 11874 2024-04-26T02:04:46.444 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.444 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_mem": { 2024-04-26T02:04:46.444 INFO:teuthology.orchestra.run.smithi173.stdout: "ino": 4641, 2024-04-26T02:04:46.444 INFO:teuthology.orchestra.run.smithi173.stdout: "ino+": 9204, 2024-04-26T02:04:46.444 INFO:teuthology.orchestra.run.smithi173.stdout: "ino-": 4563, 2024-04-26T02:04:46.444 INFO:teuthology.orchestra.run.smithi173.stdout: "dir": 220, 2024-04-26T02:04:46.444 INFO:teuthology.orchestra.run.smithi173.stdout: "dir+": 220, 2024-04-26T02:04:46.444 INFO:teuthology.orchestra.run.smithi173.stdout: "dir-": 0, 2024-04-26T02:04:46.444 INFO:teuthology.orchestra.run.smithi173.stdout: "dn": 13888, 2024-04-26T02:04:46.444 INFO:teuthology.orchestra.run.smithi173.stdout: "dn+": 13888, 2024-04-26T02:04:46.444 INFO:teuthology.orchestra.run.smithi173.stdout: "dn-": 0, 2024-04-26T02:04:46.444 INFO:teuthology.orchestra.run.smithi173.stdout: "cap": 0, 2024-04-26T02:04:46.445 INFO:teuthology.orchestra.run.smithi173.stdout: "cap+": 0, 2024-04-26T02:04:46.445 INFO:teuthology.orchestra.run.smithi173.stdout: "cap-": 0, 2024-04-26T02:04:46.445 INFO:teuthology.orchestra.run.smithi173.stdout: "rss": 74596, 2024-04-26T02:04:46.445 INFO:teuthology.orchestra.run.smithi173.stdout: "heap": 215392 2024-04-26T02:04:46.445 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.445 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_server": { 2024-04-26T02:04:46.445 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_client_request": 0, 2024-04-26T02:04:46.445 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_server_request": 0, 2024-04-26T02:04:46.445 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_request": 0, 2024-04-26T02:04:46.445 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_session": 0, 2024-04-26T02:04:46.445 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_peer_request": 0, 2024-04-26T02:04:46.445 INFO:teuthology.orchestra.run.smithi173.stdout: "req_create_latency": { 2024-04-26T02:04:46.445 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.445 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.445 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.446 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getattr_latency": { 2024-04-26T02:04:46.446 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.446 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.446 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.446 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getfilelock_latency": { 2024-04-26T02:04:46.446 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.446 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.446 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.446 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.446 INFO:teuthology.orchestra.run.smithi173.stdout: "req_link_latency": { 2024-04-26T02:04:46.446 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.446 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.447 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.447 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookup_latency": { 2024-04-26T02:04:46.447 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.447 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.447 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.447 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookuphash_latency": { 2024-04-26T02:04:46.447 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.447 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.447 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.447 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.447 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupino_latency": { 2024-04-26T02:04:46.448 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.448 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.448 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.448 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupname_latency": { 2024-04-26T02:04:46.448 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.448 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.448 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.448 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupparent_latency": { 2024-04-26T02:04:46.448 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.448 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.448 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.448 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.449 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupsnap_latency": { 2024-04-26T02:04:46.449 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.449 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.449 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.449 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lssnap_latency": { 2024-04-26T02:04:46.449 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.449 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.449 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.449 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mkdir_latency": { 2024-04-26T02:04:46.449 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.449 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.449 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.449 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.449 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mknod_latency": { 2024-04-26T02:04:46.450 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.450 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.450 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.450 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mksnap_latency": { 2024-04-26T02:04:46.450 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.450 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.450 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.450 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.450 INFO:teuthology.orchestra.run.smithi173.stdout: "req_open_latency": { 2024-04-26T02:04:46.450 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.450 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.450 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.451 INFO:teuthology.orchestra.run.smithi173.stdout: "req_readdir_latency": { 2024-04-26T02:04:46.451 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.451 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.451 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.451 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rename_latency": { 2024-04-26T02:04:46.451 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.451 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.451 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.451 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.451 INFO:teuthology.orchestra.run.smithi173.stdout: "req_renamesnap_latency": { 2024-04-26T02:04:46.451 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.451 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.451 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.452 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.452 INFO:teuthology.orchestra.run.smithi173.stdout: "req_snapdiff_latency": { 2024-04-26T02:04:46.452 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.452 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.452 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.452 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.452 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmdir_latency": { 2024-04-26T02:04:46.452 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.452 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.452 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.452 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.452 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmsnap_latency": { 2024-04-26T02:04:46.452 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.452 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.453 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.453 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmxattr_latency": { 2024-04-26T02:04:46.453 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.453 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.453 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.453 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setattr_latency": { 2024-04-26T02:04:46.453 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.453 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.453 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.453 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.453 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setdirlayout_latency": { 2024-04-26T02:04:46.453 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.453 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.454 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.454 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setfilelock_latency": { 2024-04-26T02:04:46.454 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.454 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.454 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.454 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setlayout_latency": { 2024-04-26T02:04:46.454 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.454 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.454 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.454 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.455 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setxattr_latency": { 2024-04-26T02:04:46.455 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.455 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.455 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.455 INFO:teuthology.orchestra.run.smithi173.stdout: "req_symlink_latency": { 2024-04-26T02:04:46.455 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.455 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.455 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.455 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.455 INFO:teuthology.orchestra.run.smithi173.stdout: "req_unlink_latency": { 2024-04-26T02:04:46.455 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.455 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.456 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.456 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.456 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_revoke_eviction": 0, 2024-04-26T02:04:46.456 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_acquisition_throttle": 0, 2024-04-26T02:04:46.456 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getvxattr_latency": { 2024-04-26T02:04:46.456 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.456 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.456 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.456 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:46.456 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.456 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_sessions": { 2024-04-26T02:04:46.456 INFO:teuthology.orchestra.run.smithi173.stdout: "session_count": 2, 2024-04-26T02:04:46.456 INFO:teuthology.orchestra.run.smithi173.stdout: "session_add": 2, 2024-04-26T02:04:46.457 INFO:teuthology.orchestra.run.smithi173.stdout: "session_remove": 0, 2024-04-26T02:04:46.457 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_open": 0, 2024-04-26T02:04:46.457 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_stale": 0, 2024-04-26T02:04:46.457 INFO:teuthology.orchestra.run.smithi173.stdout: "total_load": 0, 2024-04-26T02:04:46.457 INFO:teuthology.orchestra.run.smithi173.stdout: "average_load": 0, 2024-04-26T02:04:46.457 INFO:teuthology.orchestra.run.smithi173.stdout: "avg_session_uptime": 2348, 2024-04-26T02:04:46.457 INFO:teuthology.orchestra.run.smithi173.stdout: "mdthresh_evicted": 0 2024-04-26T02:04:46.457 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.457 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2024-04-26T02:04:46.457 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:04:46.457 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2024-04-26T02:04:46.457 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:04:46.457 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:04:46.458 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:04:46.458 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:04:46.458 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:04:46.458 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:04:46.458 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:04:46.458 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:04:46.458 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:04:46.458 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:04:46.458 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:04:46.458 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:04:46.458 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:04:46.458 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2024-04-26T02:04:46.458 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:04:46.459 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2024-04-26T02:04:46.459 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:04:46.459 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:04:46.459 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:04:46.459 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:04:46.459 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:04:46.459 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:04:46.459 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:04:46.459 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2024-04-26T02:04:46.459 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:04:46.459 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:04:46.459 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:04:46.459 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2024-04-26T02:04:46.459 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2024-04-26T02:04:46.459 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2024-04-26T02:04:46.459 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:04:46.460 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:04:46.460 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:04:46.460 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:04:46.460 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 395125, 2024-04-26T02:04:46.460 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 4774, 2024-04-26T02:04:46.460 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:04:46.460 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 0, 2024-04-26T02:04:46.460 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2024-04-26T02:04:46.460 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2024-04-26T02:04:46.460 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:04:46.460 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2024-04-26T02:04:46.460 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:04:46.460 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2024-04-26T02:04:46.460 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 8224, 2024-04-26T02:04:46.461 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 171, 2024-04-26T02:04:46.461 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:04:46.461 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:04:46.461 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2024-04-26T02:04:46.461 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2024-04-26T02:04:46.461 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 20717594, 2024-04-26T02:04:46.461 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 169007, 2024-04-26T02:04:46.461 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2024-04-26T02:04:46.461 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2024-04-26T02:04:46.461 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2024-04-26T02:04:46.461 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2024-04-26T02:04:46.461 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.461 INFO:teuthology.orchestra.run.smithi173.stdout: "objecter": { 2024-04-26T02:04:46.461 INFO:teuthology.orchestra.run.smithi173.stdout: "op_active": 0, 2024-04-26T02:04:46.461 INFO:teuthology.orchestra.run.smithi173.stdout: "op_laggy": 0, 2024-04-26T02:04:46.462 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send": 553, 2024-04-26T02:04:46.462 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send_bytes": 26, 2024-04-26T02:04:46.462 INFO:teuthology.orchestra.run.smithi173.stdout: "op_resend": 4, 2024-04-26T02:04:46.462 INFO:teuthology.orchestra.run.smithi173.stdout: "op_reply": 553, 2024-04-26T02:04:46.462 INFO:teuthology.orchestra.run.smithi173.stdout: "op_latency": { 2024-04-26T02:04:46.462 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 553, 2024-04-26T02:04:46.462 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.436989620, 2024-04-26T02:04:46.462 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000790216 2024-04-26T02:04:46.462 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.462 INFO:teuthology.orchestra.run.smithi173.stdout: "op_inflight": 0, 2024-04-26T02:04:46.462 INFO:teuthology.orchestra.run.smithi173.stdout: "oplen_avg": { 2024-04-26T02:04:46.462 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 553, 2024-04-26T02:04:46.462 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 554 2024-04-26T02:04:46.463 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.463 INFO:teuthology.orchestra.run.smithi173.stdout: "op": 553, 2024-04-26T02:04:46.463 INFO:teuthology.orchestra.run.smithi173.stdout: "op_r": 553, 2024-04-26T02:04:46.463 INFO:teuthology.orchestra.run.smithi173.stdout: "op_w": 0, 2024-04-26T02:04:46.463 INFO:teuthology.orchestra.run.smithi173.stdout: "op_rmw": 0, 2024-04-26T02:04:46.463 INFO:teuthology.orchestra.run.smithi173.stdout: "op_pg": 0, 2024-04-26T02:04:46.463 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_stat": 314, 2024-04-26T02:04:46.463 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_create": 0, 2024-04-26T02:04:46.463 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_read": 236, 2024-04-26T02:04:46.463 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_write": 0, 2024-04-26T02:04:46.463 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writefull": 0, 2024-04-26T02:04:46.463 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writesame": 0, 2024-04-26T02:04:46.463 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_append": 0, 2024-04-26T02:04:46.464 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_zero": 0, 2024-04-26T02:04:46.464 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_truncate": 0, 2024-04-26T02:04:46.464 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_delete": 0, 2024-04-26T02:04:46.464 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_mapext": 0, 2024-04-26T02:04:46.464 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_sparse_read": 0, 2024-04-26T02:04:46.464 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_clonerange": 0, 2024-04-26T02:04:46.464 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_getxattr": 2, 2024-04-26T02:04:46.464 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_setxattr": 0, 2024-04-26T02:04:46.464 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_cmpxattr": 0, 2024-04-26T02:04:46.464 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_rmxattr": 0, 2024-04-26T02:04:46.464 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_resetxattrs": 0, 2024-04-26T02:04:46.464 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_call": 0, 2024-04-26T02:04:46.464 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_watch": 0, 2024-04-26T02:04:46.464 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_notify": 0, 2024-04-26T02:04:46.465 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T02:04:46.465 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls": 0, 2024-04-26T02:04:46.465 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls_filter": 0, 2024-04-26T02:04:46.465 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_other": 0, 2024-04-26T02:04:46.465 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_active": 0, 2024-04-26T02:04:46.465 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_send": 0, 2024-04-26T02:04:46.465 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_resend": 0, 2024-04-26T02:04:46.465 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_ping": 0, 2024-04-26T02:04:46.465 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_active": 0, 2024-04-26T02:04:46.465 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_send": 0, 2024-04-26T02:04:46.465 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_resend": 0, 2024-04-26T02:04:46.465 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_active": 0, 2024-04-26T02:04:46.465 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_send": 0, 2024-04-26T02:04:46.465 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_resend": 0, 2024-04-26T02:04:46.466 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_active": 0, 2024-04-26T02:04:46.466 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_send": 0, 2024-04-26T02:04:46.466 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_resend": 0, 2024-04-26T02:04:46.466 INFO:teuthology.orchestra.run.smithi173.stdout: "command_active": 0, 2024-04-26T02:04:46.466 INFO:teuthology.orchestra.run.smithi173.stdout: "command_send": 0, 2024-04-26T02:04:46.466 INFO:teuthology.orchestra.run.smithi173.stdout: "command_resend": 0, 2024-04-26T02:04:46.466 INFO:teuthology.orchestra.run.smithi173.stdout: "map_epoch": 109, 2024-04-26T02:04:46.466 INFO:teuthology.orchestra.run.smithi173.stdout: "map_full": 0, 2024-04-26T02:04:46.466 INFO:teuthology.orchestra.run.smithi173.stdout: "map_inc": 4, 2024-04-26T02:04:46.466 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_sessions": 9, 2024-04-26T02:04:46.466 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_open": 9, 2024-04-26T02:04:46.466 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_close": 0, 2024-04-26T02:04:46.466 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_laggy": 0, 2024-04-26T02:04:46.467 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_wr": 0, 2024-04-26T02:04:46.467 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_rd": 2, 2024-04-26T02:04:46.467 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_del": 0 2024-04-26T02:04:46.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.467 INFO:teuthology.orchestra.run.smithi173.stdout: "oft": { 2024-04-26T02:04:46.467 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_objs": 0, 2024-04-26T02:04:46.467 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_kv_pairs": 0, 2024-04-26T02:04:46.467 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_updates": 0, 2024-04-26T02:04:46.467 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_removes": 0 2024-04-26T02:04:46.467 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.467 INFO:teuthology.orchestra.run.smithi173.stdout: "osd-slow-ops": { 2024-04-26T02:04:46.468 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_ops_count": 0 2024-04-26T02:04:46.468 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.468 INFO:teuthology.orchestra.run.smithi173.stdout: "purge_queue": { 2024-04-26T02:04:46.468 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops": 0, 2024-04-26T02:04:46.468 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T02:04:46.468 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing": 0, 2024-04-26T02:04:46.468 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_high_water": 0, 2024-04-26T02:04:46.468 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed_ops": 0, 2024-04-26T02:04:46.468 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed": 0, 2024-04-26T02:04:46.468 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_item_in_journal": 0 2024-04-26T02:04:46.468 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.468 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:04:46.469 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-26T02:04:46.469 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-26T02:04:46.469 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-26T02:04:46.469 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 609, 2024-04-26T02:04:46.469 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 44695065, 2024-04-26T02:04:46.469 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:46.469 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 609, 2024-04-26T02:04:46.469 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-26T02:04:46.469 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-26T02:04:46.469 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 609, 2024-04-26T02:04:46.469 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 44695065, 2024-04-26T02:04:46.469 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-26T02:04:46.469 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.469 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.470 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.470 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:46.470 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.470 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_bytes": { 2024-04-26T02:04:46.470 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-26T02:04:46.470 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-26T02:04:46.470 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-26T02:04:46.470 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-26T02:04:46.470 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-26T02:04:46.470 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:46.470 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-26T02:04:46.470 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 553, 2024-04-26T02:04:46.470 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 44416122, 2024-04-26T02:04:46.470 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 553, 2024-04-26T02:04:46.471 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 44416122, 2024-04-26T02:04:46.471 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-26T02:04:46.471 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.471 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.471 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.471 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:46.471 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.471 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_ops": { 2024-04-26T02:04:46.471 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-26T02:04:46.471 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 1024, 2024-04-26T02:04:46.471 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-26T02:04:46.471 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-26T02:04:46.471 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-26T02:04:46.472 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:46.472 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-26T02:04:46.472 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 553, 2024-04-26T02:04:46.472 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 553, 2024-04-26T02:04:46.472 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 553, 2024-04-26T02:04:46.472 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 553, 2024-04-26T02:04:46.472 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-26T02:04:46.472 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.472 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.472 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.472 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:46.472 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.472 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle": { 2024-04-26T02:04:46.472 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-26T02:04:46.472 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-26T02:04:46.473 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-26T02:04:46.473 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-26T02:04:46.473 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-26T02:04:46.473 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:46.473 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-26T02:04:46.473 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-26T02:04:46.473 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-26T02:04:46.473 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 0, 2024-04-26T02:04:46.473 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 0, 2024-04-26T02:04:46.473 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-26T02:04:46.473 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.473 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.473 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.473 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:46.473 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:46.474 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle-0x56108e5266e0": { 2024-04-26T02:04:46.474 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-26T02:04:46.474 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-26T02:04:46.474 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-26T02:04:46.474 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-26T02:04:46.474 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-26T02:04:46.474 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:46.474 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-26T02:04:46.474 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-26T02:04:46.474 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-26T02:04:46.474 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 0, 2024-04-26T02:04:46.474 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 0, 2024-04-26T02:04:46.474 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-26T02:04:46.474 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:46.474 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:46.475 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:46.475 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:46.475 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:46.475 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-26T02:04:46.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:46 smithi097 ceph-mon[32853]: pgmap v412: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T02:04:47.195 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-26T02:04:47.195 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-26T02:04:47.195 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-26T02:04:47.195 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-26T02:04:47.195 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-04-26T02:04:47.195 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-26T02:04:47.195 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T02:04:47.195 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-04-26T02:04:47.195 DEBUG:tasks.check_counter:Getting stats from i 2024-04-26T02:04:47.196 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-26T02:04:47.434 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T02:04:48.363 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:48 smithi173 ceph-mon[32553]: pgmap v413: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:48.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:48 smithi001 ceph-mon[27822]: pgmap v413: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:48.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:48 smithi097 ceph-mon[32853]: pgmap v413: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:48.892 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-26T02:04:48.892 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:04:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 6658, 2024-04-26T02:04:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 4173, 2024-04-26T02:04:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 6992994, 2024-04-26T02:04:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 1247075, 2024-04-26T02:04:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 8, 2024-04-26T02:04:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 4, 2024-04-26T02:04:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.363407000, 2024-04-26T02:04:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.138501124, 2024-04-26T02:04:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.285059404, 2024-04-26T02:04:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.013904334, 2024-04-26T02:04:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:48.893 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 4172, 2024-04-26T02:04:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.111524435, 2024-04-26T02:04:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000026731 2024-04-26T02:04:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 9842, 2024-04-26T02:04:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.001815161, 2024-04-26T02:04:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000184 2024-04-26T02:04:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 1705536, 2024-04-26T02:04:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 193008 2024-04-26T02:04:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:04:48.894 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 387, 2024-04-26T02:04:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 654, 2024-04-26T02:04:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 115620, 2024-04-26T02:04:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 684224, 2024-04-26T02:04:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 21, 2024-04-26T02:04:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 6, 2024-04-26T02:04:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.056536738, 2024-04-26T02:04:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.022375869, 2024-04-26T02:04:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.038792703, 2024-04-26T02:04:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.000423933, 2024-04-26T02:04:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 651, 2024-04-26T02:04:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.067038713, 2024-04-26T02:04:48.895 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000102978 2024-04-26T02:04:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 746, 2024-04-26T02:04:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000166673, 2024-04-26T02:04:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000223 2024-04-26T02:04:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 2976, 2024-04-26T02:04:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 545328 2024-04-26T02:04:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:04:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 123, 2024-04-26T02:04:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 94, 2024-04-26T02:04:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 63504, 2024-04-26T02:04:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 56462, 2024-04-26T02:04:48.896 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 9, 2024-04-26T02:04:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 6, 2024-04-26T02:04:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.009927339, 2024-04-26T02:04:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.002752441, 2024-04-26T02:04:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.006584996, 2024-04-26T02:04:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.000987572, 2024-04-26T02:04:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 93, 2024-04-26T02:04:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.018085800, 2024-04-26T02:04:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000194470 2024-04-26T02:04:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 129, 2024-04-26T02:04:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000015770, 2024-04-26T02:04:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000122 2024-04-26T02:04:48.897 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 6416, 2024-04-26T02:04:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 5936 2024-04-26T02:04:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2024-04-26T02:04:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 1, 2024-04-26T02:04:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2024-04-26T02:04:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-MDSRank": { 2024-04-26T02:04:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-26T02:04:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-26T02:04:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 102, 2024-04-26T02:04:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.008201961, 2024-04-26T02:04:48.898 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000080411 2024-04-26T02:04:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-PurgeQueue": { 2024-04-26T02:04:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-26T02:04:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-26T02:04:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 19, 2024-04-26T02:04:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000533716, 2024-04-26T02:04:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000028090 2024-04-26T02:04:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: "mds": { 2024-04-26T02:04:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: "request": 336, 2024-04-26T02:04:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: "reply": 15, 2024-04-26T02:04:48.899 INFO:teuthology.orchestra.run.smithi173.stdout: "reply_latency": { 2024-04-26T02:04:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 15, 2024-04-26T02:04:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.012822103, 2024-04-26T02:04:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000854806 2024-04-26T02:04:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_reply": 0, 2024-04-26T02:04:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "forward": 321, 2024-04-26T02:04:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_complete": 0, 2024-04-26T02:04:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_keys": 0, 2024-04-26T02:04:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_commit": 11, 2024-04-26T02:04:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_split": 0, 2024-04-26T02:04:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_merge": 0, 2024-04-26T02:04:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes": 21, 2024-04-26T02:04:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_top": 4, 2024-04-26T02:04:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_bottom": 7, 2024-04-26T02:04:48.900 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pin_tail": 10, 2024-04-26T02:04:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pinned": 14, 2024-04-26T02:04:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_expired": 126, 2024-04-26T02:04:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_with_caps": 0, 2024-04-26T02:04:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: "caps": 0, 2024-04-26T02:04:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: "subtrees": 4, 2024-04-26T02:04:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse": 643, 2024-04-26T02:04:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_hit": 256, 2024-04-26T02:04:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_forward": 320, 2024-04-26T02:04:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_discover": 58, 2024-04-26T02:04:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_dir_fetch": 0, 2024-04-26T02:04:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_remote_ino": 0, 2024-04-26T02:04:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_lock": 0, 2024-04-26T02:04:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: "load_cent": 0, 2024-04-26T02:04:48.901 INFO:teuthology.orchestra.run.smithi173.stdout: "q": 0, 2024-04-26T02:04:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "exported": 1, 2024-04-26T02:04:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "exported_inodes": 8, 2024-04-26T02:04:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "imported": 1, 2024-04-26T02:04:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "imported_inodes": 4, 2024-04-26T02:04:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_dir_fetch": 0, 2024-04-26T02:04:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_backtrace_fetch": 0, 2024-04-26T02:04:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_peer_discover": 0, 2024-04-26T02:04:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rfiles": 4, 2024-04-26T02:04:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rbytes": 234, 2024-04-26T02:04:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rsnaps": 6, 2024-04-26T02:04:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T02:04:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_set_tag": 0, 2024-04-26T02:04:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T02:04:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_inotable_repaired": 0, 2024-04-26T02:04:48.902 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_inodes": 0, 2024-04-26T02:04:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T02:04:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T02:04:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_file_inodes": 0, 2024-04-26T02:04:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_inode_file_caps": 0, 2024-04-26T02:04:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_revoke": 14, 2024-04-26T02:04:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_grant": 9, 2024-04-26T02:04:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T02:04:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T02:04:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T02:04:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps": 18, 2024-04-26T02:04:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps_dirty": 4, 2024-04-26T02:04:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_cap_release": 3, 2024-04-26T02:04:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "process_request_cap_release": 10 2024-04-26T02:04:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.903 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_cache": { 2024-04-26T02:04:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update": 2, 2024-04-26T02:04:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update_receipt": 475, 2024-04-26T02:04:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_try_discover": 266, 2024-04-26T02:04:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_send_discover": 266, 2024-04-26T02:04:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_handle_discover": 2, 2024-04-26T02:04:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays": 0, 2024-04-26T02:04:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_delayed": 0, 2024-04-26T02:04:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_enqueuing": 0, 2024-04-26T02:04:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_created": 4, 2024-04-26T02:04:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_enqueued": 4, 2024-04-26T02:04:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_reintegrated": 0, 2024-04-26T02:04:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_migrated": 0, 2024-04-26T02:04:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_processing": 0, 2024-04-26T02:04:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_enqueued": 0, 2024-04-26T02:04:48.904 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_prioritized": 0, 2024-04-26T02:04:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_started": 0, 2024-04-26T02:04:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_completed": 0, 2024-04-26T02:04:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_path": 9, 2024-04-26T02:04:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_inode": 321, 2024-04-26T02:04:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T02:04:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_exportdir": 1, 2024-04-26T02:04:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_flush": 0, 2024-04-26T02:04:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragmentdir": 0, 2024-04-26T02:04:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragstats": 0, 2024-04-26T02:04:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_inodestats": 0 2024-04-26T02:04:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_log": { 2024-04-26T02:04:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "evlrg": 0, 2024-04-26T02:04:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "evadd": 33, 2024-04-26T02:04:48.905 INFO:teuthology.orchestra.run.smithi173.stdout: "evex": 0, 2024-04-26T02:04:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "evtrm": 0, 2024-04-26T02:04:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "ev": 33, 2024-04-26T02:04:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "evexg": 0, 2024-04-26T02:04:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "evexd": 0, 2024-04-26T02:04:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "segadd": 2, 2024-04-26T02:04:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "segex": 0, 2024-04-26T02:04:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "segtrm": 0, 2024-04-26T02:04:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "seg": 2, 2024-04-26T02:04:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "segmjr": 2, 2024-04-26T02:04:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "segexg": 0, 2024-04-26T02:04:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "segexd": 0, 2024-04-26T02:04:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "expos": 4194304, 2024-04-26T02:04:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "wrpos": 4243528, 2024-04-26T02:04:48.906 INFO:teuthology.orchestra.run.smithi173.stdout: "rdpos": 0, 2024-04-26T02:04:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: "jlat": { 2024-04-26T02:04:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 20, 2024-04-26T02:04:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.038602415, 2024-04-26T02:04:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001930120 2024-04-26T02:04:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: "replayed": 0 2024-04-26T02:04:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_mem": { 2024-04-26T02:04:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: "ino": 26, 2024-04-26T02:04:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: "ino+": 96, 2024-04-26T02:04:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: "ino-": 70, 2024-04-26T02:04:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: "dir": 25, 2024-04-26T02:04:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: "dir+": 241, 2024-04-26T02:04:48.907 INFO:teuthology.orchestra.run.smithi173.stdout: "dir-": 216, 2024-04-26T02:04:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "dn": 21, 2024-04-26T02:04:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "dn+": 152, 2024-04-26T02:04:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "dn-": 131, 2024-04-26T02:04:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "cap": 0, 2024-04-26T02:04:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "cap+": 14, 2024-04-26T02:04:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "cap-": 14, 2024-04-26T02:04:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "rss": 55392, 2024-04-26T02:04:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "heap": 223608 2024-04-26T02:04:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_server": { 2024-04-26T02:04:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_client_request": 345, 2024-04-26T02:04:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_server_request": 4, 2024-04-26T02:04:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_request": 336, 2024-04-26T02:04:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_session": 28, 2024-04-26T02:04:48.908 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_peer_request": 312, 2024-04-26T02:04:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: "req_create_latency": { 2024-04-26T02:04:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1, 2024-04-26T02:04:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.003204121, 2024-04-26T02:04:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.003204121 2024-04-26T02:04:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getattr_latency": { 2024-04-26T02:04:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2, 2024-04-26T02:04:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000581095, 2024-04-26T02:04:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000290547 2024-04-26T02:04:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getfilelock_latency": { 2024-04-26T02:04:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.909 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: "req_link_latency": { 2024-04-26T02:04:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookup_latency": { 2024-04-26T02:04:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 6, 2024-04-26T02:04:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.001425842, 2024-04-26T02:04:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000237640 2024-04-26T02:04:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookuphash_latency": { 2024-04-26T02:04:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.910 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupino_latency": { 2024-04-26T02:04:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupname_latency": { 2024-04-26T02:04:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupparent_latency": { 2024-04-26T02:04:48.911 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.912 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.912 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.912 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.912 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupsnap_latency": { 2024-04-26T02:04:48.912 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.912 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.912 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.912 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.912 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lssnap_latency": { 2024-04-26T02:04:48.912 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.912 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.912 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.912 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.913 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mkdir_latency": { 2024-04-26T02:04:48.913 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.913 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.913 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.913 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.913 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mknod_latency": { 2024-04-26T02:04:48.913 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.914 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.914 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.914 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.914 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mksnap_latency": { 2024-04-26T02:04:48.914 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.914 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.914 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.914 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.914 INFO:teuthology.orchestra.run.smithi173.stdout: "req_open_latency": { 2024-04-26T02:04:48.914 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.914 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.914 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.914 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.915 INFO:teuthology.orchestra.run.smithi173.stdout: "req_readdir_latency": { 2024-04-26T02:04:48.915 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2, 2024-04-26T02:04:48.915 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000826010, 2024-04-26T02:04:48.915 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000413005 2024-04-26T02:04:48.915 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.915 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rename_latency": { 2024-04-26T02:04:48.915 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.915 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.915 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.915 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.915 INFO:teuthology.orchestra.run.smithi173.stdout: "req_renamesnap_latency": { 2024-04-26T02:04:48.915 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.915 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.915 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: "req_snapdiff_latency": { 2024-04-26T02:04:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmdir_latency": { 2024-04-26T02:04:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmsnap_latency": { 2024-04-26T02:04:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.916 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmxattr_latency": { 2024-04-26T02:04:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setattr_latency": { 2024-04-26T02:04:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setdirlayout_latency": { 2024-04-26T02:04:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.917 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.918 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.918 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.918 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setfilelock_latency": { 2024-04-26T02:04:48.918 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.918 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.918 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.918 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.918 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setlayout_latency": { 2024-04-26T02:04:48.918 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.918 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.918 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.918 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.918 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setxattr_latency": { 2024-04-26T02:04:48.918 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: "req_symlink_latency": { 2024-04-26T02:04:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: "req_unlink_latency": { 2024-04-26T02:04:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 4, 2024-04-26T02:04:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.006785035, 2024-04-26T02:04:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001696258 2024-04-26T02:04:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_revoke_eviction": 0, 2024-04-26T02:04:48.919 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_acquisition_throttle": 0, 2024-04-26T02:04:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getvxattr_latency": { 2024-04-26T02:04:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_sessions": { 2024-04-26T02:04:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: "session_count": 2, 2024-04-26T02:04:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: "session_add": 4, 2024-04-26T02:04:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: "session_remove": 2, 2024-04-26T02:04:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_open": 2, 2024-04-26T02:04:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_stale": 0, 2024-04-26T02:04:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: "total_load": 8, 2024-04-26T02:04:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: "average_load": 4, 2024-04-26T02:04:48.920 INFO:teuthology.orchestra.run.smithi173.stdout: "avg_session_uptime": 726, 2024-04-26T02:04:48.921 INFO:teuthology.orchestra.run.smithi173.stdout: "mdthresh_evicted": 0 2024-04-26T02:04:48.921 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.921 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2024-04-26T02:04:48.921 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:04:48.921 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2024-04-26T02:04:48.921 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:04:48.921 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:04:48.921 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:04:48.921 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:04:48.921 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:04:48.921 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:04:48.921 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:04:48.921 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:04:48.921 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:04:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:04:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:04:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:04:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:04:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2024-04-26T02:04:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:04:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2024-04-26T02:04:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:04:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:04:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:04:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:04:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:04:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:04:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:04:48.922 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2024-04-26T02:04:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:04:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:04:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:04:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2024-04-26T02:04:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2024-04-26T02:04:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2024-04-26T02:04:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:04:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:04:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:04:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:04:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 280311, 2024-04-26T02:04:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 98, 2024-04-26T02:04:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:04:48.923 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 0, 2024-04-26T02:04:48.924 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2024-04-26T02:04:48.924 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2024-04-26T02:04:48.924 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:04:48.924 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2024-04-26T02:04:48.924 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:04:48.924 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2024-04-26T02:04:48.924 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 8224, 2024-04-26T02:04:48.924 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 171, 2024-04-26T02:04:48.924 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:04:48.924 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:04:48.924 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2024-04-26T02:04:48.924 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2024-04-26T02:04:48.924 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 109741, 2024-04-26T02:04:48.924 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 423, 2024-04-26T02:04:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2024-04-26T02:04:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2024-04-26T02:04:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2024-04-26T02:04:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2024-04-26T02:04:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: "objecter": { 2024-04-26T02:04:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: "op_active": 0, 2024-04-26T02:04:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: "op_laggy": 0, 2024-04-26T02:04:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send": 86, 2024-04-26T02:04:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send_bytes": 62887, 2024-04-26T02:04:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: "op_resend": 33, 2024-04-26T02:04:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: "op_reply": 86, 2024-04-26T02:04:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: "op_latency": { 2024-04-26T02:04:48.925 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 86, 2024-04-26T02:04:48.926 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.926977681, 2024-04-26T02:04:48.926 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.010778810 2024-04-26T02:04:48.926 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.926 INFO:teuthology.orchestra.run.smithi173.stdout: "op_inflight": 0, 2024-04-26T02:04:48.926 INFO:teuthology.orchestra.run.smithi173.stdout: "oplen_avg": { 2024-04-26T02:04:48.926 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 86, 2024-04-26T02:04:48.926 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 120 2024-04-26T02:04:48.926 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.926 INFO:teuthology.orchestra.run.smithi173.stdout: "op": 86, 2024-04-26T02:04:48.926 INFO:teuthology.orchestra.run.smithi173.stdout: "op_r": 1, 2024-04-26T02:04:48.926 INFO:teuthology.orchestra.run.smithi173.stdout: "op_w": 85, 2024-04-26T02:04:48.926 INFO:teuthology.orchestra.run.smithi173.stdout: "op_rmw": 0, 2024-04-26T02:04:48.926 INFO:teuthology.orchestra.run.smithi173.stdout: "op_pg": 0, 2024-04-26T02:04:48.926 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_stat": 0, 2024-04-26T02:04:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_create": 10, 2024-04-26T02:04:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_read": 1, 2024-04-26T02:04:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_write": 21, 2024-04-26T02:04:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writefull": 14, 2024-04-26T02:04:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writesame": 0, 2024-04-26T02:04:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_append": 0, 2024-04-26T02:04:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_zero": 0, 2024-04-26T02:04:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_truncate": 0, 2024-04-26T02:04:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_delete": 16, 2024-04-26T02:04:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_mapext": 0, 2024-04-26T02:04:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_sparse_read": 0, 2024-04-26T02:04:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_clonerange": 0, 2024-04-26T02:04:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_getxattr": 0, 2024-04-26T02:04:48.927 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_setxattr": 20, 2024-04-26T02:04:48.928 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_cmpxattr": 0, 2024-04-26T02:04:48.928 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_rmxattr": 0, 2024-04-26T02:04:48.928 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_resetxattrs": 0, 2024-04-26T02:04:48.928 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_call": 0, 2024-04-26T02:04:48.928 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_watch": 0, 2024-04-26T02:04:48.928 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_notify": 0, 2024-04-26T02:04:48.928 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T02:04:48.928 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls": 0, 2024-04-26T02:04:48.928 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls_filter": 0, 2024-04-26T02:04:48.928 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_other": 0, 2024-04-26T02:04:48.928 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_active": 0, 2024-04-26T02:04:48.928 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_send": 0, 2024-04-26T02:04:48.928 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_resend": 0, 2024-04-26T02:04:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_ping": 0, 2024-04-26T02:04:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_active": 0, 2024-04-26T02:04:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_send": 0, 2024-04-26T02:04:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_resend": 0, 2024-04-26T02:04:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_active": 0, 2024-04-26T02:04:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_send": 0, 2024-04-26T02:04:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_resend": 0, 2024-04-26T02:04:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_active": 0, 2024-04-26T02:04:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_send": 0, 2024-04-26T02:04:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_resend": 0, 2024-04-26T02:04:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "command_active": 0, 2024-04-26T02:04:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "command_send": 0, 2024-04-26T02:04:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "command_resend": 0, 2024-04-26T02:04:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "map_epoch": 109, 2024-04-26T02:04:48.929 INFO:teuthology.orchestra.run.smithi173.stdout: "map_full": 0, 2024-04-26T02:04:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: "map_inc": 19, 2024-04-26T02:04:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_sessions": 10, 2024-04-26T02:04:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_open": 14, 2024-04-26T02:04:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_close": 4, 2024-04-26T02:04:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_laggy": 0, 2024-04-26T02:04:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_wr": 31, 2024-04-26T02:04:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_rd": 0, 2024-04-26T02:04:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_del": 7 2024-04-26T02:04:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: "oft": { 2024-04-26T02:04:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_objs": 1, 2024-04-26T02:04:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_kv_pairs": 0, 2024-04-26T02:04:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_updates": 0, 2024-04-26T02:04:48.930 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_removes": 0 2024-04-26T02:04:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: "osd-slow-ops": { 2024-04-26T02:04:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_ops_count": 330 2024-04-26T02:04:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: "purge_queue": { 2024-04-26T02:04:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops": 0, 2024-04-26T02:04:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops_high_water": 4, 2024-04-26T02:04:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing": 0, 2024-04-26T02:04:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_high_water": 4, 2024-04-26T02:04:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed_ops": 4, 2024-04-26T02:04:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed": 4, 2024-04-26T02:04:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_item_in_journal": 0 2024-04-26T02:04:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.931 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:04:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-26T02:04:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-26T02:04:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-26T02:04:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 7164, 2024-04-26T02:04:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 6504320, 2024-04-26T02:04:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 7164, 2024-04-26T02:04:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-26T02:04:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-26T02:04:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 7164, 2024-04-26T02:04:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 6504320, 2024-04-26T02:04:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-26T02:04:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.932 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_bytes": { 2024-04-26T02:04:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-26T02:04:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-26T02:04:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-26T02:04:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-26T02:04:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-26T02:04:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-26T02:04:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 86, 2024-04-26T02:04:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 63483, 2024-04-26T02:04:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 86, 2024-04-26T02:04:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 63483, 2024-04-26T02:04:48.933 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-26T02:04:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_ops": { 2024-04-26T02:04:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-26T02:04:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 1024, 2024-04-26T02:04:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-26T02:04:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-26T02:04:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-26T02:04:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-26T02:04:48.934 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 86, 2024-04-26T02:04:48.935 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 86, 2024-04-26T02:04:48.935 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 86, 2024-04-26T02:04:48.935 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 86, 2024-04-26T02:04:48.935 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-26T02:04:48.935 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.935 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.935 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.935 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:48.935 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.935 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle": { 2024-04-26T02:04:48.935 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-26T02:04:48.935 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-26T02:04:48.935 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-26T02:04:48.935 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 4, 2024-04-26T02:04:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 596, 2024-04-26T02:04:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 4, 2024-04-26T02:04:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-26T02:04:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-26T02:04:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 1, 2024-04-26T02:04:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 596, 2024-04-26T02:04:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-26T02:04:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle-0x5614036a6460": { 2024-04-26T02:04:48.936 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-26T02:04:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-26T02:04:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-26T02:04:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 33, 2024-04-26T02:04:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 49224, 2024-04-26T02:04:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 33, 2024-04-26T02:04:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-26T02:04:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-26T02:04:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 20, 2024-04-26T02:04:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 49224, 2024-04-26T02:04:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-26T02:04:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:48.937 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:48.938 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:48.938 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:48.938 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-26T02:04:49.650 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-04-26T02:04:49.650 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-04-26T02:04:49.650 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=2 2024-04-26T02:04:49.650 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=475 2024-04-26T02:04:49.651 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-26T02:04:49.651 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-26T02:04:49.651 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T02:04:49.651 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-04-26T02:04:49.651 DEBUG:tasks.check_counter:Getting stats from l 2024-04-26T02:04:49.651 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-26T02:04:49.889 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T02:04:50.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:50 smithi173 ceph-mon[32553]: pgmap v414: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:50.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:50 smithi001 ceph-mon[27822]: pgmap v414: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:50.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:50 smithi097 ceph-mon[32853]: pgmap v414: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:51.408 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-26T02:04:51.408 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:04:51.408 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 0, 2024-04-26T02:04:51.408 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 2, 2024-04-26T02:04:51.408 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 0, 2024-04-26T02:04:51.408 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 0, 2024-04-26T02:04:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 4, 2024-04-26T02:04:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 0, 2024-04-26T02:04:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.000954917, 2024-04-26T02:04:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T02:04:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.000128882, 2024-04-26T02:04:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:04:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:51.409 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T02:04:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T02:04:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:04:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 309, 2024-04-26T02:04:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 296, 2024-04-26T02:04:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 1616928, 2024-04-26T02:04:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 181968, 2024-04-26T02:04:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 2, 2024-04-26T02:04:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 1, 2024-04-26T02:04:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.059654648, 2024-04-26T02:04:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.018831066, 2024-04-26T02:04:51.410 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.041798097, 2024-04-26T02:04:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.006961814, 2024-04-26T02:04:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 296, 2024-04-26T02:04:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.014777500, 2024-04-26T02:04:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000049923 2024-04-26T02:04:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 1616928, 2024-04-26T02:04:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 181968 2024-04-26T02:04:51.411 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:04:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 2, 2024-04-26T02:04:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 224, 2024-04-26T02:04:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 320, 2024-04-26T02:04:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 195632, 2024-04-26T02:04:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 5, 2024-04-26T02:04:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 3, 2024-04-26T02:04:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.013793776, 2024-04-26T02:04:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.009611306, 2024-04-26T02:04:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.001312359, 2024-04-26T02:04:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:04:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:04:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 221, 2024-04-26T02:04:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.048913514, 2024-04-26T02:04:51.412 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000221328 2024-04-26T02:04:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:04:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T02:04:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 195632 2024-04-26T02:04:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2024-04-26T02:04:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 0, 2024-04-26T02:04:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2024-04-26T02:04:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2024-04-26T02:04:51.413 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:04:51.414 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2024-04-26T02:04:51.414 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:04:51.414 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:04:51.414 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:04:51.414 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:04:51.414 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:04:51.414 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:04:51.414 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:04:51.414 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:04:51.414 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:04:51.414 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:04:51.414 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:04:51.414 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:04:51.414 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:04:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2024-04-26T02:04:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:04:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2024-04-26T02:04:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:04:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:04:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:04:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:04:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:04:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:04:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:04:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2024-04-26T02:04:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:04:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:04:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:04:51.415 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2024-04-26T02:04:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2024-04-26T02:04:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2024-04-26T02:04:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:04:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:04:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:04:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:04:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 148732, 2024-04-26T02:04:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 31, 2024-04-26T02:04:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:04:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 0, 2024-04-26T02:04:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2024-04-26T02:04:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2024-04-26T02:04:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:04:51.416 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2024-04-26T02:04:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:04:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2024-04-26T02:04:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 0, 2024-04-26T02:04:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 0, 2024-04-26T02:04:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:04:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:04:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2024-04-26T02:04:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2024-04-26T02:04:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 928, 2024-04-26T02:04:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 2, 2024-04-26T02:04:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2024-04-26T02:04:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2024-04-26T02:04:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2024-04-26T02:04:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2024-04-26T02:04:51.417 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-26T02:04:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:04:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-26T02:04:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-26T02:04:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-26T02:04:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 307, 2024-04-26T02:04:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 1570825, 2024-04-26T02:04:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-26T02:04:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 307, 2024-04-26T02:04:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-26T02:04:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-26T02:04:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 307, 2024-04-26T02:04:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 1570825, 2024-04-26T02:04:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-26T02:04:51.418 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-26T02:04:51.419 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-26T02:04:51.419 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-26T02:04:51.419 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:51.419 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-26T02:04:51.419 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-26T02:04:51.998 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-26T02:04:51.998 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-26T02:04:51.999 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-04-26T02:04:51.999 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-04-26T02:04:51.999 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-26T02:04:51.999 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-26T02:04:51.999 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-26T02:04:51.999 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-26T02:04:51.999 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-26T02:04:52.012 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-26T02:04:52.012 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-04-26T02:04:52.012 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T02:04:52.077 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-04-26T02:04:52.077 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T02:04:52.137 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd blocklist ls 2024-04-26T02:04:52.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:52 smithi173 ceph-mon[32553]: pgmap v415: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T02:04:52.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:52 smithi001 ceph-mon[27822]: pgmap v415: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T02:04:52.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:52 smithi097 ceph-mon[32853]: pgmap v415: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T02:04:52.614 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:04:54.249 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:54 smithi001 ceph-mon[27822]: pgmap v416: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:54.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:54 smithi097 ceph-mon[32853]: pgmap v416: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:54.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:54 smithi173 ceph-mon[32553]: pgmap v416: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:54.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.805+0000 7feb8ea00700 1 -- 172.21.15.1:0/2616406987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb88105870 msgr2=0x7feb881098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:54.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.805+0000 7feb8ea00700 1 --2- 172.21.15.1:0/2616406987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb88105870 0x7feb881098c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feb740099c0 tx=0x7feb74009cd0 comp rx=0 tx=0).stop 2024-04-26T02:04:54.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.806+0000 7feb8ea00700 1 -- 172.21.15.1:0/2616406987 shutdown_connections 2024-04-26T02:04:54.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.806+0000 7feb8ea00700 1 --2- 172.21.15.1:0/2616406987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb88109fa0 0x7feb88111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:54.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.806+0000 7feb8ea00700 1 --2- 172.21.15.1:0/2616406987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb88104ec0 0x7feb881052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:54.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.806+0000 7feb8ea00700 1 --2- 172.21.15.1:0/2616406987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb88105870 0x7feb881098c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:54.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.806+0000 7feb8ea00700 1 -- 172.21.15.1:0/2616406987 >> 172.21.15.1:0/2616406987 conn(0x7feb88100750 msgr2=0x7feb88102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:54.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.806+0000 7feb8ea00700 1 -- 172.21.15.1:0/2616406987 shutdown_connections 2024-04-26T02:04:54.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.806+0000 7feb8ea00700 1 -- 172.21.15.1:0/2616406987 wait complete. 2024-04-26T02:04:54.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.807+0000 7feb8ea00700 1 Processor -- start 2024-04-26T02:04:54.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.807+0000 7feb8ea00700 1 -- start start 2024-04-26T02:04:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.807+0000 7feb8ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb88104ec0 0x7feb88076f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.807+0000 7feb8ea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb88105870 0x7feb88077490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.807+0000 7feb8ea00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb88109fa0 0x7feb880779d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.807+0000 7feb8ea00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feb88114e60 con 0x7feb88104ec0 2024-04-26T02:04:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.807+0000 7feb8ea00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feb88114ce0 con 0x7feb88109fa0 2024-04-26T02:04:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.807+0000 7feb8ea00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feb88114fe0 con 0x7feb88105870 2024-04-26T02:04:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.808+0000 7feb87e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb88104ec0 0x7feb88076f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.808+0000 7feb87400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb88105870 0x7feb88077490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.808+0000 7feb8ca00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb88109fa0 0x7feb880779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.808+0000 7feb87400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb88105870 0x7feb88077490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:35790/0 (socket says 172.21.15.1:35790) 2024-04-26T02:04:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.808+0000 7feb87400700 1 -- 172.21.15.1:0/2825628833 learned_addr learned my addr 172.21.15.1:0/2825628833 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.808+0000 7feb8ca00700 1 -- 172.21.15.1:0/2825628833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb88105870 msgr2=0x7feb88077490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.808+0000 7feb8ca00700 1 --2- 172.21.15.1:0/2825628833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb88105870 0x7feb88077490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.808+0000 7feb8ca00700 1 -- 172.21.15.1:0/2825628833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb88104ec0 msgr2=0x7feb88076f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.808+0000 7feb8ca00700 1 --2- 172.21.15.1:0/2825628833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb88104ec0 0x7feb88076f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.808+0000 7feb8ca00700 1 -- 172.21.15.1:0/2825628833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb881aba00 con 0x7feb88109fa0 2024-04-26T02:04:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.809+0000 7feb8ca00700 1 --2- 172.21.15.1:0/2825628833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb88109fa0 0x7feb880779d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7feb80009f60 tx=0x7feb80009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.809+0000 7feb84c00700 1 -- 172.21.15.1:0/2825628833 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb800092e0 con 0x7feb88109fa0 2024-04-26T02:04:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.809+0000 7feb8ea00700 1 -- 172.21.15.1:0/2825628833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb881abc90 con 0x7feb88109fa0 2024-04-26T02:04:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.809+0000 7feb84c00700 1 -- 172.21.15.1:0/2825628833 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb80004bb0 con 0x7feb88109fa0 2024-04-26T02:04:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.809+0000 7feb84c00700 1 -- 172.21.15.1:0/2825628833 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80004020 con 0x7feb88109fa0 2024-04-26T02:04:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.809+0000 7feb8ea00700 1 -- 172.21.15.1:0/2825628833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb881ac220 con 0x7feb88109fa0 2024-04-26T02:04:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.810+0000 7feb8ea00700 1 -- 172.21.15.1:0/2825628833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb881063a0 con 0x7feb88109fa0 2024-04-26T02:04:54.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.812+0000 7feb84c00700 1 -- 172.21.15.1:0/2825628833 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7feb8001b030 con 0x7feb88109fa0 2024-04-26T02:04:54.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.814+0000 7feb84c00700 1 --2- 172.21.15.1:0/2825628833 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7feb6c0a3e30 0x7feb6c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:54.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.814+0000 7feb84c00700 1 -- 172.21.15.1:0/2825628833 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8571+0+0 (secure 0 0 0) 0x7feb800d0150 con 0x7feb88109fa0 2024-04-26T02:04:54.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.814+0000 7feb87e00700 1 --2- 172.21.15.1:0/2825628833 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7feb6c0a3e30 0x7feb6c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:54.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.815+0000 7feb87e00700 1 --2- 172.21.15.1:0/2825628833 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7feb6c0a3e30 0x7feb6c0a62f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7feb78009c60 tx=0x7feb78009260 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:54.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:54.818+0000 7feb84c00700 1 -- 172.21.15.1:0/2825628833 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb80022020 con 0x7feb88109fa0 2024-04-26T02:04:55.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:55.138+0000 7feb8ea00700 1 -- 172.21.15.1:0/2825628833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7feb8802df80 con 0x7feb88109fa0 2024-04-26T02:04:55.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:55.139+0000 7feb84c00700 1 -- 172.21.15.1:0/2825628833 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v109) ==== 81+0+923 (secure 0 0 0) 0x7feb80010040 con 0x7feb88109fa0 2024-04-26T02:04:55.138 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1607430337 2024-04-27T01:51:38.660496+0000 2024-04-26T02:04:55.138 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/25191068 2024-04-27T01:51:38.660496+0000 2024-04-26T02:04:55.138 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/2140737121 2024-04-27T01:51:38.660496+0000 2024-04-26T02:04:55.138 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/304803917 2024-04-27T01:51:38.660496+0000 2024-04-26T02:04:55.138 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/1870580982 2024-04-27T01:39:20.645943+0000 2024-04-26T02:04:55.138 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/1870580982 2024-04-27T01:39:20.645943+0000 2024-04-26T02:04:55.138 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1000222947 2024-04-27T01:39:20.645943+0000 2024-04-26T02:04:55.138 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/2140737121 2024-04-27T01:51:38.660496+0000 2024-04-26T02:04:55.138 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/457958597 2024-04-27T01:39:20.645943+0000 2024-04-26T02:04:55.139 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/810889357 2024-04-27T01:38:45.974031+0000 2024-04-26T02:04:55.139 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/810889357 2024-04-27T01:38:45.974031+0000 2024-04-26T02:04:55.139 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/941157221 2024-04-27T01:51:38.660496+0000 2024-04-26T02:04:55.139 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2715798962 2024-04-27T01:38:45.974031+0000 2024-04-26T02:04:55.139 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1923058728 2024-04-27T01:39:20.645943+0000 2024-04-26T02:04:55.139 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2489038777 2024-04-27T01:38:45.974031+0000 2024-04-26T02:04:55.139 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2869502448 2024-04-27T01:38:45.974031+0000 2024-04-26T02:04:55.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:55.142+0000 7feb8ea00700 1 -- 172.21.15.1:0/2825628833 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7feb6c0a3e30 msgr2=0x7feb6c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:55.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:55.142+0000 7feb8ea00700 1 --2- 172.21.15.1:0/2825628833 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7feb6c0a3e30 0x7feb6c0a62f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7feb78009c60 tx=0x7feb78009260 comp rx=0 tx=0).stop 2024-04-26T02:04:55.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:55.142+0000 7feb8ea00700 1 -- 172.21.15.1:0/2825628833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb88109fa0 msgr2=0x7feb880779d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:55.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:55.142+0000 7feb8ea00700 1 --2- 172.21.15.1:0/2825628833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb88109fa0 0x7feb880779d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7feb80009f60 tx=0x7feb80009f90 comp rx=0 tx=0).stop 2024-04-26T02:04:55.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:55.142+0000 7feb8ea00700 1 -- 172.21.15.1:0/2825628833 shutdown_connections 2024-04-26T02:04:55.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:55.142+0000 7feb8ea00700 1 --2- 172.21.15.1:0/2825628833 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7feb6c0a3e30 0x7feb6c0a62f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:55.142+0000 7feb8ea00700 1 --2- 172.21.15.1:0/2825628833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb88109fa0 0x7feb880779d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:55.142+0000 7feb8ea00700 1 --2- 172.21.15.1:0/2825628833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb88104ec0 0x7feb88076f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:55.142+0000 7feb8ea00700 1 --2- 172.21.15.1:0/2825628833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb88105870 0x7feb88077490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:55.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:55.142+0000 7feb8ea00700 1 -- 172.21.15.1:0/2825628833 >> 172.21.15.1:0/2825628833 conn(0x7feb88100750 msgr2=0x7feb88102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:55.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:55.143+0000 7feb8ea00700 1 -- 172.21.15.1:0/2825628833 shutdown_connections 2024-04-26T02:04:55.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:55.143+0000 7feb8ea00700 1 -- 172.21.15.1:0/2825628833 wait complete. 2024-04-26T02:04:55.143 INFO:teuthology.orchestra.run.smithi001.stderr:listed 16 entries 2024-04-26T02:04:55.974 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T02:04:55.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:04:55.975 DEBUG:teuthology.orchestra.run.smithi001:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T02:04:56.097 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-04-26T02:04:56.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:04:56.098 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T02:04:56.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:04:56.181 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-04-26T02:04:56.222 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T02:04:56.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:04:56.223 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T02:04:56.317 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:56 smithi001 ceph-mon[27822]: pgmap v417: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T02:04:56.317 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:56 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2825628833' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T02:04:56.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:56 smithi173 ceph-mon[32553]: pgmap v417: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T02:04:56.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:56 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/2825628833' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T02:04:56.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:04:56.329 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link delete ceph-brx 2024-04-26T02:04:56.442 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-26T02:04:56.454 INFO:tasks.cephadm:Teardown begin 2024-04-26T02:04:56.454 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T02:04:56.454 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:04:56.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:56 smithi097 ceph-mon[32853]: pgmap v417: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T02:04:56.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:56 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/2825628833' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T02:04:56.879 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:04:58.359 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:04:58 smithi001 ceph-mon[27822]: pgmap v418: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:58.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:04:58 smithi097 ceph-mon[32853]: pgmap v418: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:58.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:04:58 smithi173 ceph-mon[32553]: pgmap v418: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:04:59.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.156+0000 7fca8d400700 1 -- 172.21.15.1:0/3929569673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca88108f80 msgr2=0x7fca88109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:59.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.156+0000 7fca8d400700 1 --2- 172.21.15.1:0/3929569673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca88108f80 0x7fca88109400 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fca740098f0 tx=0x7fca74009c00 comp rx=0 tx=0).stop 2024-04-26T02:04:59.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.156+0000 7fca8d400700 1 -- 172.21.15.1:0/3929569673 shutdown_connections 2024-04-26T02:04:59.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.156+0000 7fca8d400700 1 --2- 172.21.15.1:0/3929569673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca88109940 0x7fca88115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:59.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.156+0000 7fca8d400700 1 --2- 172.21.15.1:0/3929569673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca8810f030 0x7fca8810f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:59.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.156+0000 7fca8d400700 1 --2- 172.21.15.1:0/3929569673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca88108f80 0x7fca88109400 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:59.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.156+0000 7fca8d400700 1 -- 172.21.15.1:0/3929569673 >> 172.21.15.1:0/3929569673 conn(0x7fca880787d0 msgr2=0x7fca88107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:59.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.156+0000 7fca8d400700 1 -- 172.21.15.1:0/3929569673 shutdown_connections 2024-04-26T02:04:59.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.156+0000 7fca8d400700 1 -- 172.21.15.1:0/3929569673 wait complete. 2024-04-26T02:04:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.157+0000 7fca8d400700 1 Processor -- start 2024-04-26T02:04:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.158+0000 7fca8d400700 1 -- start start 2024-04-26T02:04:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.158+0000 7fca8d400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca88108f80 0x7fca8819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.158+0000 7fca8d400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca88109940 0x7fca8819fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.158+0000 7fca8d400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca8810f030 0x7fca881a3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.158+0000 7fca8d400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fca88119170 con 0x7fca8810f030 2024-04-26T02:04:59.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.158+0000 7fca8d400700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fca88118ff0 con 0x7fca88109940 2024-04-26T02:04:59.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.158+0000 7fca8d400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fca881192f0 con 0x7fca88108f80 2024-04-26T02:04:59.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.158+0000 7fca8ca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca8810f030 0x7fca881a3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:59.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.158+0000 7fca87e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca88108f80 0x7fca8819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:59.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.158+0000 7fca87400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca88109940 0x7fca8819fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:59.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.158+0000 7fca87e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca88108f80 0x7fca8819f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:35796/0 (socket says 172.21.15.1:35796) 2024-04-26T02:04:59.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.158+0000 7fca87e00700 1 -- 172.21.15.1:0/3362442089 learned_addr learned my addr 172.21.15.1:0/3362442089 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:04:59.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.159+0000 7fca87400700 1 -- 172.21.15.1:0/3362442089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca88108f80 msgr2=0x7fca8819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:59.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.159+0000 7fca87400700 1 --2- 172.21.15.1:0/3362442089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca88108f80 0x7fca8819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:59.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.159+0000 7fca87400700 1 -- 172.21.15.1:0/3362442089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca8810f030 msgr2=0x7fca881a3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:59.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.159+0000 7fca87400700 1 --2- 172.21.15.1:0/3362442089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca8810f030 0x7fca881a3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:59.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.159+0000 7fca87400700 1 -- 172.21.15.1:0/3362442089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca881a4650 con 0x7fca88109940 2024-04-26T02:04:59.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.159+0000 7fca8ca00700 1 --2- 172.21.15.1:0/3362442089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca8810f030 0x7fca881a3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:04:59.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.159+0000 7fca87400700 1 --2- 172.21.15.1:0/3362442089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca88109940 0x7fca8819fb80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fca7400c010 tx=0x7fca7400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:59.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.159+0000 7fca84c00700 1 -- 172.21.15.1:0/3362442089 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca740050c0 con 0x7fca88109940 2024-04-26T02:04:59.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.160+0000 7fca8d400700 1 -- 172.21.15.1:0/3362442089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca881a48e0 con 0x7fca88109940 2024-04-26T02:04:59.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.160+0000 7fca84c00700 1 -- 172.21.15.1:0/3362442089 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca7400b560 con 0x7fca88109940 2024-04-26T02:04:59.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.160+0000 7fca84c00700 1 -- 172.21.15.1:0/3362442089 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74003a00 con 0x7fca88109940 2024-04-26T02:04:59.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.160+0000 7fca8d400700 1 -- 172.21.15.1:0/3362442089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca881ac1c0 con 0x7fca88109940 2024-04-26T02:04:59.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.161+0000 7fca8d400700 1 -- 172.21.15.1:0/3362442089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca88069730 con 0x7fca88109940 2024-04-26T02:04:59.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.163+0000 7fca84c00700 1 -- 172.21.15.1:0/3362442089 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fca7400b710 con 0x7fca88109940 2024-04-26T02:04:59.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.164+0000 7fca84c00700 1 --2- 172.21.15.1:0/3362442089 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fca6c0a3db0 0x7fca6c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:04:59.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.164+0000 7fca84c00700 1 -- 172.21.15.1:0/3362442089 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8571+0+0 (secure 0 0 0) 0x7fca740d17e0 con 0x7fca88109940 2024-04-26T02:04:59.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.164+0000 7fca87e00700 1 --2- 172.21.15.1:0/3362442089 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fca6c0a3db0 0x7fca6c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:04:59.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.165+0000 7fca87e00700 1 --2- 172.21.15.1:0/3362442089 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fca6c0a3db0 0x7fca6c0a6270 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fca78005950 tx=0x7fca78009310 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:04:59.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.166+0000 7fca84c00700 1 -- 172.21.15.1:0/3362442089 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fca74013030 con 0x7fca88109940 2024-04-26T02:04:59.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.446+0000 7fca8d400700 1 -- 172.21.15.1:0/3362442089 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca8802df80 con 0x7fca88109940 2024-04-26T02:04:59.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.448+0000 7fca84c00700 1 -- 172.21.15.1:0/3362442089 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11840 (secure 0 0 0) 0x7fca74021380 con 0x7fca88109940 2024-04-26T02:04:59.447 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T02:04:59.447 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T02:03:36.329802+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26497":{"gid":26497,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/334804458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":334804458},{"type":"v1","addr":"172.21.15.173:6835","nonce":334804458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24518,24499,24514]},"id":1}]} 2024-04-26T02:04:59.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.450+0000 7fca8d400700 1 -- 172.21.15.1:0/3362442089 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fca6c0a3db0 msgr2=0x7fca6c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:59.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.450+0000 7fca8d400700 1 --2- 172.21.15.1:0/3362442089 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fca6c0a3db0 0x7fca6c0a6270 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fca78005950 tx=0x7fca78009310 comp rx=0 tx=0).stop 2024-04-26T02:04:59.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.450+0000 7fca8d400700 1 -- 172.21.15.1:0/3362442089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca88109940 msgr2=0x7fca8819fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:04:59.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.450+0000 7fca8d400700 1 --2- 172.21.15.1:0/3362442089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca88109940 0x7fca8819fb80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fca7400c010 tx=0x7fca7400fa00 comp rx=0 tx=0).stop 2024-04-26T02:04:59.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.451+0000 7fca8d400700 1 -- 172.21.15.1:0/3362442089 shutdown_connections 2024-04-26T02:04:59.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.451+0000 7fca8d400700 1 --2- 172.21.15.1:0/3362442089 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fca6c0a3db0 0x7fca6c0a6270 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:59.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.451+0000 7fca8d400700 1 --2- 172.21.15.1:0/3362442089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca8810f030 0x7fca881a3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:59.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.451+0000 7fca8d400700 1 --2- 172.21.15.1:0/3362442089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca88108f80 0x7fca8819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:59.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.451+0000 7fca8d400700 1 --2- 172.21.15.1:0/3362442089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca88109940 0x7fca8819fb80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:04:59.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.451+0000 7fca8d400700 1 -- 172.21.15.1:0/3362442089 >> 172.21.15.1:0/3362442089 conn(0x7fca880787d0 msgr2=0x7fca88079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:04:59.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.451+0000 7fca8d400700 1 -- 172.21.15.1:0/3362442089 shutdown_connections 2024-04-26T02:04:59.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:04:59.451+0000 7fca8d400700 1 -- 172.21.15.1:0/3362442089 wait complete. 2024-04-26T02:04:59.451 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 36 2024-04-26T02:05:00.338 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:05:00.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:00 smithi001 ceph-mon[27822]: pgmap v419: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:05:00.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:00 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3362442089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:05:00.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:00 smithi097 ceph-mon[32853]: pgmap v419: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:05:00.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:00 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3362442089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:05:00.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:00 smithi173 ceph-mon[32553]: pgmap v419: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T02:05:00.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:00 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3362442089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:05:00.815 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:05:02.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:02 smithi001 ceph-mon[27822]: pgmap v420: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-26T02:05:02.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:02 smithi097 ceph-mon[32853]: pgmap v420: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-26T02:05:02.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:02 smithi173 ceph-mon[32553]: pgmap v420: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-26T02:05:02.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.988+0000 7fe848200700 1 -- 172.21.15.1:0/1751049079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe840108a30 msgr2=0x7fe840108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:02.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.988+0000 7fe848200700 1 --2- 172.21.15.1:0/1751049079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe840108a30 0x7fe840108e10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe8340098f0 tx=0x7fe834009c00 comp rx=0 tx=0).stop 2024-04-26T02:05:02.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.988+0000 7fe848200700 1 -- 172.21.15.1:0/1751049079 shutdown_connections 2024-04-26T02:05:02.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.988+0000 7fe848200700 1 --2- 172.21.15.1:0/1751049079 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe84006b0f0 0x7fe84010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:02.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.988+0000 7fe848200700 1 --2- 172.21.15.1:0/1751049079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe840108a30 0x7fe840108e10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:02.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.988+0000 7fe848200700 1 --2- 172.21.15.1:0/1751049079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe84006a750 0x7fe84006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:02.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.988+0000 7fe848200700 1 -- 172.21.15.1:0/1751049079 >> 172.21.15.1:0/1751049079 conn(0x7fe8400788f0 msgr2=0x7fe840078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:02.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.988+0000 7fe848200700 1 -- 172.21.15.1:0/1751049079 shutdown_connections 2024-04-26T02:05:02.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.988+0000 7fe848200700 1 -- 172.21.15.1:0/1751049079 wait complete. 2024-04-26T02:05:02.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.989+0000 7fe848200700 1 Processor -- start 2024-04-26T02:05:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.990+0000 7fe848200700 1 -- start start 2024-04-26T02:05:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.990+0000 7fe848200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe84006a750 0x7fe84019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.990+0000 7fe848200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe84006b0f0 0x7fe84019b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.990+0000 7fe848200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe840108a30 0x7fe84019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.990+0000 7fe848200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe840114db0 con 0x7fe84006a750 2024-04-26T02:05:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.990+0000 7fe848200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe840114c30 con 0x7fe840108a30 2024-04-26T02:05:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.990+0000 7fe848200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe840114f30 con 0x7fe84006b0f0 2024-04-26T02:05:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.990+0000 7fe845800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe84006a750 0x7fe84019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.990+0000 7fe845800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe84006a750 0x7fe84019b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39450/0 (socket says 172.21.15.1:39450) 2024-04-26T02:05:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.990+0000 7fe845800700 1 -- 172.21.15.1:0/2160154980 learned_addr learned my addr 172.21.15.1:0/2160154980 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:05:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.990+0000 7fe844e00700 1 --2- 172.21.15.1:0/2160154980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe84006b0f0 0x7fe84019b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.990+0000 7fe846200700 1 --2- 172.21.15.1:0/2160154980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe840108a30 0x7fe84019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:02.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.991+0000 7fe846200700 1 -- 172.21.15.1:0/2160154980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe84006b0f0 msgr2=0x7fe84019b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:02.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.991+0000 7fe846200700 1 --2- 172.21.15.1:0/2160154980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe84006b0f0 0x7fe84019b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:02.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.991+0000 7fe846200700 1 -- 172.21.15.1:0/2160154980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe84006a750 msgr2=0x7fe84019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:02.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.991+0000 7fe846200700 1 --2- 172.21.15.1:0/2160154980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe84006a750 0x7fe84019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:02.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.991+0000 7fe846200700 1 -- 172.21.15.1:0/2160154980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8401a0370 con 0x7fe840108a30 2024-04-26T02:05:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.991+0000 7fe844e00700 1 --2- 172.21.15.1:0/2160154980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe84006b0f0 0x7fe84019b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:05:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.991+0000 7fe846200700 1 --2- 172.21.15.1:0/2160154980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe840108a30 0x7fe84019fc30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe83800efc0 tx=0x7fe83800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.992+0000 7fe83e000700 1 -- 172.21.15.1:0/2160154980 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8380092e0 con 0x7fe840108a30 2024-04-26T02:05:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.992+0000 7fe848200700 1 -- 172.21.15.1:0/2160154980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8401a0600 con 0x7fe840108a30 2024-04-26T02:05:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.992+0000 7fe83e000700 1 -- 172.21.15.1:0/2160154980 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe838004bb0 con 0x7fe840108a30 2024-04-26T02:05:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.992+0000 7fe83e000700 1 -- 172.21.15.1:0/2160154980 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe838004020 con 0x7fe840108a30 2024-04-26T02:05:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.992+0000 7fe848200700 1 -- 172.21.15.1:0/2160154980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8401a7ee0 con 0x7fe840108a30 2024-04-26T02:05:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.993+0000 7fe848200700 1 -- 172.21.15.1:0/2160154980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe840069730 con 0x7fe840108a30 2024-04-26T02:05:02.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.996+0000 7fe83e000700 1 -- 172.21.15.1:0/2160154980 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7fe83801b030 con 0x7fe840108a30 2024-04-26T02:05:02.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.996+0000 7fe83e000700 1 --2- 172.21.15.1:0/2160154980 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe8280a3e30 0x7fe8280a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:02.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.997+0000 7fe83e000700 1 -- 172.21.15.1:0/2160154980 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8571+0+0 (secure 0 0 0) 0x7fe8380d01e0 con 0x7fe840108a30 2024-04-26T02:05:02.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.997+0000 7fe845800700 1 --2- 172.21.15.1:0/2160154980 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe8280a3e30 0x7fe8280a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:02.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:02.998+0000 7fe845800700 1 --2- 172.21.15.1:0/2160154980 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe8280a3e30 0x7fe8280a62f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe834009680 tx=0x7fe834019040 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:03.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:03.002+0000 7fe83e000700 1 -- 172.21.15.1:0/2160154980 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe838022020 con 0x7fe840108a30 2024-04-26T02:05:03.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:03.292+0000 7fe848200700 1 -- 172.21.15.1:0/2160154980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe84002df80 con 0x7fe840108a30 2024-04-26T02:05:03.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:03.293+0000 7fe83e000700 1 -- 172.21.15.1:0/2160154980 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11840 (secure 0 0 0) 0x7fe838010040 con 0x7fe840108a30 2024-04-26T02:05:03.294 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T02:05:03.294 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/1477554934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1477554934},{"type":"v1","addr":"172.21.15.1:6835","nonce":1477554934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14370,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/76325266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":76325266},{"type":"v1","addr":"172.21.15.1:6837","nonce":76325266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/42404130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":42404130},{"type":"v1","addr":"172.21.15.1:6839","nonce":42404130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14406,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3885053557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3885053557},{"type":"v1","addr":"172.21.15.1:6841","nonce":3885053557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/4061367802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":4061367802},{"type":"v1","addr":"172.21.15.97:6833","nonce":4061367802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/188689332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":188689332},{"type":"v1","addr":"172.21.15.173:6833","nonce":188689332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:47:29.999521+0000","modified":"2024-04-26T02:03:36.329802+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24518,"mds_1":24514,"mds_2":24499},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/763675408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":763675408},{"type":"v1","addr":"172.21.15.97:6835","nonce":763675408}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3607142398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3607142398},{"type":"v1","addr":"172.21.15.97:6837","nonce":3607142398}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.173:6837/2502298793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2502298793},{"type":"v1","addr":"172.21.15.173:6837","nonce":2502298793}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.173:6839/373692430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":373692430},{"type":"v1","addr":"172.21.15.173:6839","nonce":373692430}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.97:6839/1315554162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":1315554162},{"type":"v1","addr":"172.21.15.97:6839","nonce":1315554162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26497":{"gid":26497,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/334804458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":334804458},{"type":"v1","addr":"172.21.15.173:6835","nonce":334804458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24518,"qdb_cluster":[24518,24499,24514]},"id":1}]} 2024-04-26T02:05:03.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:03.297+0000 7fe848200700 1 -- 172.21.15.1:0/2160154980 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe8280a3e30 msgr2=0x7fe8280a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:03.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:03.297+0000 7fe848200700 1 --2- 172.21.15.1:0/2160154980 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe8280a3e30 0x7fe8280a62f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe834009680 tx=0x7fe834019040 comp rx=0 tx=0).stop 2024-04-26T02:05:03.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:03.297+0000 7fe848200700 1 -- 172.21.15.1:0/2160154980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe840108a30 msgr2=0x7fe84019fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:03.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:03.297+0000 7fe848200700 1 --2- 172.21.15.1:0/2160154980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe840108a30 0x7fe84019fc30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe83800efc0 tx=0x7fe83800c3a0 comp rx=0 tx=0).stop 2024-04-26T02:05:03.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:03.298+0000 7fe848200700 1 -- 172.21.15.1:0/2160154980 shutdown_connections 2024-04-26T02:05:03.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:03.298+0000 7fe848200700 1 --2- 172.21.15.1:0/2160154980 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7fe8280a3e30 0x7fe8280a62f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:03.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:03.298+0000 7fe848200700 1 --2- 172.21.15.1:0/2160154980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe840108a30 0x7fe84019fc30 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:03.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:03.298+0000 7fe848200700 1 --2- 172.21.15.1:0/2160154980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe84006a750 0x7fe84019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:03.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:03.298+0000 7fe848200700 1 --2- 172.21.15.1:0/2160154980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe84006b0f0 0x7fe84019b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:03.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:03.298+0000 7fe848200700 1 -- 172.21.15.1:0/2160154980 >> 172.21.15.1:0/2160154980 conn(0x7fe8400788f0 msgr2=0x7fe8400ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:03.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:03.298+0000 7fe848200700 1 -- 172.21.15.1:0/2160154980 shutdown_connections 2024-04-26T02:05:03.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:03.298+0000 7fe848200700 1 -- 172.21.15.1:0/2160154980 wait complete. 2024-04-26T02:05:03.298 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 36 2024-04-26T02:05:04.084 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T02:05:04.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:04 smithi001 ceph-mon[27822]: pgmap v421: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-26T02:05:04.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:04 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/2160154980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:05:04.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:04 smithi097 ceph-mon[32853]: pgmap v421: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-26T02:05:04.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:04 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/2160154980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:05:04.564 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:05:04.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:04 smithi173 ceph-mon[32553]: pgmap v421: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-26T02:05:04.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:04 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/2160154980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:05:06.610 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:06 smithi001 ceph-mon[27822]: pgmap v422: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T02:05:06.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:06 smithi097 ceph-mon[32853]: pgmap v422: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T02:05:06.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.787+0000 7f79ae800700 1 -- 172.21.15.1:0/1583014542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79a8075700 msgr2=0x7f79a8113430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:06.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.787+0000 7f79ae800700 1 --2- 172.21.15.1:0/1583014542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79a8075700 0x7f79a8113430 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7994009990 tx=0x7f7994009ca0 comp rx=0 tx=0).stop 2024-04-26T02:05:06.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.787+0000 7f79ae800700 1 -- 172.21.15.1:0/1583014542 shutdown_connections 2024-04-26T02:05:06.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.787+0000 7f79ae800700 1 --2- 172.21.15.1:0/1583014542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79a8113970 0x7f79a8115d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:06.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.787+0000 7f79ae800700 1 --2- 172.21.15.1:0/1583014542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79a8074de0 0x7f79a80751c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:06.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.787+0000 7f79ae800700 1 --2- 172.21.15.1:0/1583014542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79a8075700 0x7f79a8113430 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:06.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.787+0000 7f79ae800700 1 -- 172.21.15.1:0/1583014542 >> 172.21.15.1:0/1583014542 conn(0x7f79a80fe450 msgr2=0x7f79a8100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:06.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.787+0000 7f79ae800700 1 -- 172.21.15.1:0/1583014542 shutdown_connections 2024-04-26T02:05:06.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.787+0000 7f79ae800700 1 -- 172.21.15.1:0/1583014542 wait complete. 2024-04-26T02:05:06.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.788+0000 7f79ae800700 1 Processor -- start 2024-04-26T02:05:06.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.789+0000 7f79ae800700 1 -- start start 2024-04-26T02:05:06.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.789+0000 7f79ae800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79a8074de0 0x7f79a819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:06.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.789+0000 7f79ae800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79a8075700 0x7f79a819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:06.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.789+0000 7f79ae800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79a8113970 0x7f79a81a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:06.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.789+0000 7f79ae800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f79a81191a0 con 0x7f79a8075700 2024-04-26T02:05:06.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.789+0000 7f79ae800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f79a8119020 con 0x7f79a8074de0 2024-04-26T02:05:06.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.789+0000 7f79ae800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f79a8119320 con 0x7f79a8113970 2024-04-26T02:05:06.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.790+0000 7f79aca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79a8075700 0x7f79a819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:06.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.790+0000 7f79ad400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79a8074de0 0x7f79a819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:06.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.790+0000 7f79aca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79a8075700 0x7f79a819fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39472/0 (socket says 172.21.15.1:39472) 2024-04-26T02:05:06.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.790+0000 7f79ad400700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79a8074de0 0x7f79a819f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.1:56370/0 (socket says 172.21.15.1:56370) 2024-04-26T02:05:06.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.790+0000 7f79aca00700 1 -- 172.21.15.1:0/274935679 learned_addr learned my addr 172.21.15.1:0/274935679 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:05:06.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.790+0000 7f79ade00700 1 --2- 172.21.15.1:0/274935679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79a8113970 0x7f79a81a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:06.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.790+0000 7f79ad400700 1 -- 172.21.15.1:0/274935679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79a8113970 msgr2=0x7f79a81a3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:06.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.790+0000 7f79ad400700 1 --2- 172.21.15.1:0/274935679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79a8113970 0x7f79a81a3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:06.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.790+0000 7f79ad400700 1 -- 172.21.15.1:0/274935679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79a8075700 msgr2=0x7f79a819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:06.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.790+0000 7f79ad400700 1 --2- 172.21.15.1:0/274935679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79a8075700 0x7f79a819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:06.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.790+0000 7f79ad400700 1 -- 172.21.15.1:0/274935679 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a81a46f0 con 0x7f79a8074de0 2024-04-26T02:05:06.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.791+0000 7f79ad400700 1 --2- 172.21.15.1:0/274935679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79a8074de0 0x7f79a819f730 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f799c00d950 tx=0x7f799c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:06.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.791+0000 7f79a6000700 1 -- 172.21.15.1:0/274935679 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c016070 con 0x7f79a8074de0 2024-04-26T02:05:06.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.791+0000 7f79a6000700 1 -- 172.21.15.1:0/274935679 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f799c00fd40 con 0x7f79a8074de0 2024-04-26T02:05:06.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.791+0000 7f79ae800700 1 -- 172.21.15.1:0/274935679 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a81a49e0 con 0x7f79a8074de0 2024-04-26T02:05:06.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.791+0000 7f79a6000700 1 -- 172.21.15.1:0/274935679 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c00e3b0 con 0x7f79a8074de0 2024-04-26T02:05:06.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.791+0000 7f79ae800700 1 -- 172.21.15.1:0/274935679 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a81ac260 con 0x7f79a8074de0 2024-04-26T02:05:06.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.792+0000 7f79ae800700 1 -- 172.21.15.1:0/274935679 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a8069730 con 0x7f79a8074de0 2024-04-26T02:05:06.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.794+0000 7f79a6000700 1 -- 172.21.15.1:0/274935679 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f799c019070 con 0x7f79a8074de0 2024-04-26T02:05:06.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.796+0000 7f79a6000700 1 --2- 172.21.15.1:0/274935679 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f79900a3db0 0x7f79900a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:06.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.796+0000 7f79a6000700 1 -- 172.21.15.1:0/274935679 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8571+0+0 (secure 0 0 0) 0x7f799c0d07c0 con 0x7f79a8074de0 2024-04-26T02:05:06.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.796+0000 7f79aca00700 1 --2- 172.21.15.1:0/274935679 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f79900a3db0 0x7f79900a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:06.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.797+0000 7f79aca00700 1 --2- 172.21.15.1:0/274935679 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f79900a3db0 0x7f79900a6270 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f79a8100bb0 tx=0x7f799400b560 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:06.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:06.798+0000 7f79a6000700 1 -- 172.21.15.1:0/274935679 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f799c09cdd0 con 0x7f79a8074de0 2024-04-26T02:05:06.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:06 smithi173 ceph-mon[32553]: pgmap v422: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T02:05:07.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:07.033+0000 7f79ae800700 1 -- 172.21.15.1:0/274935679 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79a81a0a40 con 0x7f79a8074de0 2024-04-26T02:05:07.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:07.035+0000 7f79a6000700 1 -- 172.21.15.1:0/274935679 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+23110 (secure 0 0 0) 0x7f799c09c6d0 con 0x7f79a8074de0 2024-04-26T02:05:07.092 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-04-26T02:05:07.092 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":109,"fsid":"77cfcc80-036d-11ef-bc93-c7b262605968","created":"2024-04-26T01:38:03.531182+0000","modified":"2024-04-26T02:04:19.299333+0000","last_up_change":"2024-04-26T01:46:16.151275+0000","last_in_change":"2024-04-26T01:45:56.514228+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:42:04.805488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:47:21.957421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:47:25.860260+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bb10ae12-09ee-4d6b-bc78-8c6016613b81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6803","nonce":3176633965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6805","nonce":3176633965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6809","nonce":3176633965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3176633965},{"type":"v1","addr":"172.21.15.1:6807","nonce":3176633965}]},"public_addr":"172.21.15.1:6803/3176633965","cluster_addr":"172.21.15.1:6805/3176633965","heartbeat_back_addr":"172.21.15.1:6809/3176633965","heartbeat_front_addr":"172.21.15.1:6807/3176633965","state":["exists","up"]},{"osd":1,"uuid":"acd12888-6ff6-40bc-99e7-e5c395e85b1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6801","nonce":3755022621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6803","nonce":3755022621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6807","nonce":3755022621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3755022621},{"type":"v1","addr":"172.21.15.97:6805","nonce":3755022621}]},"public_addr":"172.21.15.97:6801/3755022621","cluster_addr":"172.21.15.97:6803/3755022621","heartbeat_back_addr":"172.21.15.97:6807/3755022621","heartbeat_front_addr":"172.21.15.97:6805/3755022621","state":["exists","up"]},{"osd":2,"uuid":"79971508-e256-4145-8111-a75666e886fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6801","nonce":734861771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6803","nonce":734861771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6807","nonce":734861771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":734861771},{"type":"v1","addr":"172.21.15.173:6805","nonce":734861771}]},"public_addr":"172.21.15.173:6801/734861771","cluster_addr":"172.21.15.173:6803/734861771","heartbeat_back_addr":"172.21.15.173:6807/734861771","heartbeat_front_addr":"172.21.15.173:6805/734861771","state":["exists","up"]},{"osd":3,"uuid":"a8ccc4a9-0f31-446a-8bcb-4691532397c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6811","nonce":3349961961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6813","nonce":3349961961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6817","nonce":3349961961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":3349961961},{"type":"v1","addr":"172.21.15.1:6815","nonce":3349961961}]},"public_addr":"172.21.15.1:6811/3349961961","cluster_addr":"172.21.15.1:6813/3349961961","heartbeat_back_addr":"172.21.15.1:6817/3349961961","heartbeat_front_addr":"172.21.15.1:6815/3349961961","state":["exists","up"]},{"osd":4,"uuid":"5b724648-4df7-40fc-89d5-55f170cdd806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6809","nonce":2494639397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6811","nonce":2494639397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6815","nonce":2494639397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2494639397},{"type":"v1","addr":"172.21.15.97:6813","nonce":2494639397}]},"public_addr":"172.21.15.97:6809/2494639397","cluster_addr":"172.21.15.97:6811/2494639397","heartbeat_back_addr":"172.21.15.97:6815/2494639397","heartbeat_front_addr":"172.21.15.97:6813/2494639397","state":["exists","up"]},{"osd":5,"uuid":"a5a24a9c-3007-4b87-8562-40c98c41ef8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6809","nonce":1026507957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6811","nonce":1026507957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6815","nonce":1026507957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1026507957},{"type":"v1","addr":"172.21.15.173:6813","nonce":1026507957}]},"public_addr":"172.21.15.173:6809/1026507957","cluster_addr":"172.21.15.173:6811/1026507957","heartbeat_back_addr":"172.21.15.173:6815/1026507957","heartbeat_front_addr":"172.21.15.173:6813/1026507957","state":["exists","up"]},{"osd":6,"uuid":"97cd2e88-a56c-4b1a-8104-65b23a77986a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6819","nonce":444030587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6821","nonce":444030587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6825","nonce":444030587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":444030587},{"type":"v1","addr":"172.21.15.1:6823","nonce":444030587}]},"public_addr":"172.21.15.1:6819/444030587","cluster_addr":"172.21.15.1:6821/444030587","heartbeat_back_addr":"172.21.15.1:6825/444030587","heartbeat_front_addr":"172.21.15.1:6823/444030587","state":["exists","up"]},{"osd":7,"uuid":"afc0c2dd-e112-4358-b3aa-944366d5d383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6817","nonce":2600022531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6819","nonce":2600022531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6823","nonce":2600022531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2600022531},{"type":"v1","addr":"172.21.15.97:6821","nonce":2600022531}]},"public_addr":"172.21.15.97:6817/2600022531","cluster_addr":"172.21.15.97:6819/2600022531","heartbeat_back_addr":"172.21.15.97:6823/2600022531","heartbeat_front_addr":"172.21.15.97:6821/2600022531","state":["exists","up"]},{"osd":8,"uuid":"41fa612b-793e-44da-af8b-44d17a9de303","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6817","nonce":1334205527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6819","nonce":1334205527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6823","nonce":1334205527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1334205527},{"type":"v1","addr":"172.21.15.173:6821","nonce":1334205527}]},"public_addr":"172.21.15.173:6817/1334205527","cluster_addr":"172.21.15.173:6819/1334205527","heartbeat_back_addr":"172.21.15.173:6823/1334205527","heartbeat_front_addr":"172.21.15.173:6821/1334205527","state":["exists","up"]},{"osd":9,"uuid":"bbbb163b-f8c4-4be5-8f80-82445e8d407f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6827","nonce":2973486839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6829","nonce":2973486839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6833","nonce":2973486839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2973486839},{"type":"v1","addr":"172.21.15.1:6831","nonce":2973486839}]},"public_addr":"172.21.15.1:6827/2973486839","cluster_addr":"172.21.15.1:6829/2973486839","heartbeat_back_addr":"172.21.15.1:6833/2973486839","heartbeat_front_addr":"172.21.15.1:6831/2973486839","state":["exists","up"]},{"osd":10,"uuid":"1b447b34-0600-4727-826d-b3d68adfb18d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6825","nonce":1693425268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6827","nonce":1693425268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6831","nonce":1693425268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1693425268},{"type":"v1","addr":"172.21.15.97:6829","nonce":1693425268}]},"public_addr":"172.21.15.97:6825/1693425268","cluster_addr":"172.21.15.97:6827/1693425268","heartbeat_back_addr":"172.21.15.97:6831/1693425268","heartbeat_front_addr":"172.21.15.97:6829/1693425268","state":["exists","up"]},{"osd":11,"uuid":"f439aae1-f50a-4176-889d-36fa5131541a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6825","nonce":1141524960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6827","nonce":1141524960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6831","nonce":1141524960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1141524960},{"type":"v1","addr":"172.21.15.173:6829","nonce":1141524960}]},"public_addr":"172.21.15.173:6825/1141524960","cluster_addr":"172.21.15.173:6827/1141524960","heartbeat_back_addr":"172.21.15.173:6831/1141524960","heartbeat_front_addr":"172.21.15.173:6829/1141524960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:12.955643+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:41:38.846652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:02.556062+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:42:34.900933+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:01.170878+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:24.842573+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:43:58.304440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:24.868689+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:44:48.772003+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:22.611681+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:45:49.470779+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:46:14.163984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1607430337":"2024-04-27T01:51:38.660496+0000","172.21.15.1:0/25191068":"2024-04-27T01:51:38.660496+0000","172.21.15.1:6801/2140737121":"2024-04-27T01:51:38.660496+0000","172.21.15.1:0/304803917":"2024-04-27T01:51:38.660496+0000","172.21.15.1:6801/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6800/1870580982":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/1000222947":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6800/2140737121":"2024-04-27T01:51:38.660496+0000","172.21.15.1:0/457958597":"2024-04-27T01:39:20.645943+0000","172.21.15.1:6801/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:6800/810889357":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/941157221":"2024-04-27T01:51:38.660496+0000","172.21.15.1:0/2715798962":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/1923058728":"2024-04-27T01:39:20.645943+0000","172.21.15.1:0/2489038777":"2024-04-27T01:38:45.974031+0000","172.21.15.1:0/2869502448":"2024-04-27T01:38:45.974031+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T02:05:07.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:07.038+0000 7f79ae800700 1 -- 172.21.15.1:0/274935679 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f79900a3db0 msgr2=0x7f79900a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:07.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:07.038+0000 7f79ae800700 1 --2- 172.21.15.1:0/274935679 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f79900a3db0 0x7f79900a6270 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f79a8100bb0 tx=0x7f799400b560 comp rx=0 tx=0).stop 2024-04-26T02:05:07.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:07.038+0000 7f79ae800700 1 -- 172.21.15.1:0/274935679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79a8074de0 msgr2=0x7f79a819f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:07.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:07.038+0000 7f79ae800700 1 --2- 172.21.15.1:0/274935679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79a8074de0 0x7f79a819f730 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f799c00d950 tx=0x7f799c00dc60 comp rx=0 tx=0).stop 2024-04-26T02:05:07.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:07.038+0000 7f79ae800700 1 -- 172.21.15.1:0/274935679 shutdown_connections 2024-04-26T02:05:07.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:07.038+0000 7f79ae800700 1 --2- 172.21.15.1:0/274935679 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f79900a3db0 0x7f79900a6270 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:07.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:07.038+0000 7f79ae800700 1 --2- 172.21.15.1:0/274935679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79a8113970 0x7f79a81a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:07.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:07.038+0000 7f79ae800700 1 --2- 172.21.15.1:0/274935679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79a8074de0 0x7f79a819f730 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:07.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:07.038+0000 7f79ae800700 1 --2- 172.21.15.1:0/274935679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79a8075700 0x7f79a819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:07.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:07.038+0000 7f79ae800700 1 -- 172.21.15.1:0/274935679 >> 172.21.15.1:0/274935679 conn(0x7f79a80fe450 msgr2=0x7f79a80ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:07.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:07.038+0000 7f79ae800700 1 -- 172.21.15.1:0/274935679 shutdown_connections 2024-04-26T02:05:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:07.038+0000 7f79ae800700 1 -- 172.21.15.1:0/274935679 wait complete. 2024-04-26T02:05:07.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:07 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/274935679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T02:05:07.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:07 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/274935679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T02:05:07.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:07 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/274935679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T02:05:07.852 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs fail cephfs 2024-04-26T02:05:08.322 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:05:08.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:08 smithi001 ceph-mon[27822]: pgmap v423: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T02:05:08.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:08 smithi097 ceph-mon[32853]: pgmap v423: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T02:05:08.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:08 smithi173 ceph-mon[32553]: pgmap v423: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T02:05:09.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:09 smithi001 ceph-mon[27822]: pgmap v424: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T02:05:09.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:09 smithi097 ceph-mon[32853]: pgmap v424: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T02:05:09.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:09 smithi173 ceph-mon[32553]: pgmap v424: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T02:05:10.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.519+0000 7f2181600700 1 -- 172.21.15.1:0/3105113307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f217c105890 msgr2=0x7f217c1098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:10.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.519+0000 7f2181600700 1 --2- 172.21.15.1:0/3105113307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f217c105890 0x7f217c1098e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f21640099c0 tx=0x7f2164009cd0 comp rx=0 tx=0).stop 2024-04-26T02:05:10.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.519+0000 7f2181600700 1 -- 172.21.15.1:0/3105113307 shutdown_connections 2024-04-26T02:05:10.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.519+0000 7f2181600700 1 --2- 172.21.15.1:0/3105113307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c109fc0 0x7f217c111b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:10.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.519+0000 7f2181600700 1 --2- 172.21.15.1:0/3105113307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f217c104ee0 0x7f217c1052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:10.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.519+0000 7f2181600700 1 --2- 172.21.15.1:0/3105113307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f217c105890 0x7f217c1098e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:10.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.519+0000 7f2181600700 1 -- 172.21.15.1:0/3105113307 >> 172.21.15.1:0/3105113307 conn(0x7f217c100770 msgr2=0x7f217c102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:10.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.519+0000 7f2181600700 1 -- 172.21.15.1:0/3105113307 shutdown_connections 2024-04-26T02:05:10.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.520+0000 7f2181600700 1 -- 172.21.15.1:0/3105113307 wait complete. 2024-04-26T02:05:10.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.521+0000 7f2181600700 1 Processor -- start 2024-04-26T02:05:10.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.521+0000 7f2181600700 1 -- start start 2024-04-26T02:05:10.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.521+0000 7f2181600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f217c104ee0 0x7f217c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:10.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.521+0000 7f2181600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c105890 0x7f217c19b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:10.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.521+0000 7f2181600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f217c109fc0 0x7f217c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:10.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.521+0000 7f2181600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f217c114ea0 con 0x7f217c109fc0 2024-04-26T02:05:10.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.521+0000 7f2181600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f217c114d20 con 0x7f217c105890 2024-04-26T02:05:10.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.521+0000 7f2181600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f217c115020 con 0x7f217c104ee0 2024-04-26T02:05:10.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.522+0000 7f217aa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f217c104ee0 0x7f217c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:10.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.522+0000 7f217b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f217c109fc0 0x7f217c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:10.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.522+0000 7f2172000700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c105890 0x7f217c19b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:10.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.522+0000 7f217aa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f217c104ee0 0x7f217c19b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.1:33996/0 (socket says 172.21.15.1:33996) 2024-04-26T02:05:10.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.522+0000 7f217aa00700 1 -- 172.21.15.1:0/3266331568 learned_addr learned my addr 172.21.15.1:0/3266331568 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:05:10.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.522+0000 7f2172000700 1 -- 172.21.15.1:0/3266331568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f217c104ee0 msgr2=0x7f217c19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:10.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.522+0000 7f2172000700 1 --2- 172.21.15.1:0/3266331568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f217c104ee0 0x7f217c19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:10.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.522+0000 7f2172000700 1 -- 172.21.15.1:0/3266331568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f217c109fc0 msgr2=0x7f217c19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:10.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.522+0000 7f2172000700 1 --2- 172.21.15.1:0/3266331568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f217c109fc0 0x7f217c19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:10.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.522+0000 7f2172000700 1 -- 172.21.15.1:0/3266331568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217c1a03a0 con 0x7f217c105890 2024-04-26T02:05:10.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.523+0000 7f2172000700 1 --2- 172.21.15.1:0/3266331568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c105890 0x7f217c19b8d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f21640098e0 tx=0x7f2164004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:10.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.523+0000 7f2173e00700 1 -- 172.21.15.1:0/3266331568 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2164026070 con 0x7f217c105890 2024-04-26T02:05:10.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.523+0000 7f2181600700 1 -- 172.21.15.1:0/3266331568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f217c1a0630 con 0x7f217c105890 2024-04-26T02:05:10.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.523+0000 7f2173e00700 1 -- 172.21.15.1:0/3266331568 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2164003c70 con 0x7f217c105890 2024-04-26T02:05:10.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.523+0000 7f2173e00700 1 -- 172.21.15.1:0/3266331568 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216400f9c0 con 0x7f217c105890 2024-04-26T02:05:10.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.523+0000 7f2181600700 1 -- 172.21.15.1:0/3266331568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f217c1a7f10 con 0x7f217c105890 2024-04-26T02:05:10.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.524+0000 7f2181600700 1 -- 172.21.15.1:0/3266331568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f217c1063c0 con 0x7f217c105890 2024-04-26T02:05:10.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.526+0000 7f2173e00700 1 -- 172.21.15.1:0/3266331568 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f2164029070 con 0x7f217c105890 2024-04-26T02:05:10.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.528+0000 7f2173e00700 1 --2- 172.21.15.1:0/3266331568 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f21600a3db0 0x7f21600a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:10.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.528+0000 7f2173e00700 1 -- 172.21.15.1:0/3266331568 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8571+0+0 (secure 0 0 0) 0x7f21640d0ff0 con 0x7f217c105890 2024-04-26T02:05:10.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.528+0000 7f217aa00700 1 --2- 172.21.15.1:0/3266331568 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f21600a3db0 0x7f21600a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:10.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.529+0000 7f217aa00700 1 --2- 172.21.15.1:0/3266331568 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f21600a3db0 0x7f21600a6270 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f216c009c10 tx=0x7f216c00b040 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:10.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.530+0000 7f2173e00700 1 -- 172.21.15.1:0/3266331568 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2164013070 con 0x7f217c105890 2024-04-26T02:05:10.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.813+0000 7f2181600700 1 -- 172.21.15.1:0/3266331568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f217c02df80 con 0x7f217c105890 2024-04-26T02:05:10.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.825+0000 7f2173e00700 1 -- 172.21.15.1:0/3266331568 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) ==== 161+0+0 (secure 0 0 0) 0x7f216409d690 con 0x7f217c105890 2024-04-26T02:05:10.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.828+0000 7f2181600700 1 -- 172.21.15.1:0/3266331568 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f21600a3db0 msgr2=0x7f21600a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:10.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.828+0000 7f2181600700 1 --2- 172.21.15.1:0/3266331568 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f21600a3db0 0x7f21600a6270 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f216c009c10 tx=0x7f216c00b040 comp rx=0 tx=0).stop 2024-04-26T02:05:10.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.828+0000 7f2181600700 1 -- 172.21.15.1:0/3266331568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c105890 msgr2=0x7f217c19b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:10.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.828+0000 7f2181600700 1 --2- 172.21.15.1:0/3266331568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c105890 0x7f217c19b8d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f21640098e0 tx=0x7f2164004270 comp rx=0 tx=0).stop 2024-04-26T02:05:10.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.829+0000 7f2181600700 1 -- 172.21.15.1:0/3266331568 shutdown_connections 2024-04-26T02:05:10.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.829+0000 7f2181600700 1 --2- 172.21.15.1:0/3266331568 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f21600a3db0 0x7f21600a6270 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:10.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.829+0000 7f2181600700 1 --2- 172.21.15.1:0/3266331568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f217c109fc0 0x7f217c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:10.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.829+0000 7f2181600700 1 --2- 172.21.15.1:0/3266331568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f217c104ee0 0x7f217c19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:10.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.829+0000 7f2181600700 1 --2- 172.21.15.1:0/3266331568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c105890 0x7f217c19b8d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:10.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.829+0000 7f2181600700 1 -- 172.21.15.1:0/3266331568 >> 172.21.15.1:0/3266331568 conn(0x7f217c100770 msgr2=0x7f217c102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:10.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.829+0000 7f2181600700 1 -- 172.21.15.1:0/3266331568 shutdown_connections 2024-04-26T02:05:10.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:10.829+0000 7f2181600700 1 -- 172.21.15.1:0/3266331568 wait complete. 2024-04-26T02:05:10.829 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T02:05:11.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:10 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-a[27817]: 2024-04-26T02:05:10.816+0000 7f6820000700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T02:05:11.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:10 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3266331568' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T02:05:11.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:10 smithi001 ceph-mon[27822]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T02:05:11.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:10 smithi001 ceph-mon[27822]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T02:05:11.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:10 smithi001 ceph-mon[27822]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T02:05:11.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:10 smithi001 ceph-mon[27822]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T02:05:11.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:10 smithi001 ceph-mon[27822]: osdmap e110: 12 total, 12 up, 12 in 2024-04-26T02:05:11.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:10 smithi001 ceph-mon[27822]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-26T02:05:11.234 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 02:05:10 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-b[62484]: ignoring --setuser ceph since I am not root 2024-04-26T02:05:11.234 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 02:05:10 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-b[62484]: ignoring --setgroup ceph since I am not root 2024-04-26T02:05:11.234 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 02:05:10 smithi097 ceph-mds[62488]: -- 172.21.15.97:0/2097478401 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561ea69b1a40 con 0x561ea6984800 2024-04-26T02:05:11.234 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 02:05:10 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-b[62484]: starting mds.b at 2024-04-26T02:05:11.235 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 26 02:05:10 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-h[62178]: ignoring --setuser ceph since I am not root 2024-04-26T02:05:11.235 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 26 02:05:10 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-h[62178]: ignoring --setgroup ceph since I am not root 2024-04-26T02:05:11.235 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 26 02:05:10 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-h[62178]: starting mds.h at 2024-04-26T02:05:11.235 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 26 02:05:10 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-e[61866]: ignoring --setuser ceph since I am not root 2024-04-26T02:05:11.235 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 26 02:05:10 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-e[61866]: ignoring --setgroup ceph since I am not root 2024-04-26T02:05:11.235 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 26 02:05:10 smithi097 ceph-mds[61870]: -- 172.21.15.97:0/3110758672 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c697e4fa40 con 0x55c697e23000 2024-04-26T02:05:11.235 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 26 02:05:10 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-e[61866]: starting mds.e at 2024-04-26T02:05:11.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:10 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3266331568' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T02:05:11.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:10 smithi097 ceph-mon[32853]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T02:05:11.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:10 smithi097 ceph-mon[32853]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T02:05:11.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:10 smithi097 ceph-mon[32853]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T02:05:11.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:10 smithi097 ceph-mon[32853]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T02:05:11.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:10 smithi097 ceph-mon[32853]: osdmap e110: 12 total, 12 up, 12 in 2024-04-26T02:05:11.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:10 smithi097 ceph-mon[32853]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-26T02:05:11.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:10 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3266331568' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T02:05:11.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:10 smithi173 ceph-mon[32553]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T02:05:11.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:10 smithi173 ceph-mon[32553]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T02:05:11.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:10 smithi173 ceph-mon[32553]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T02:05:11.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:10 smithi173 ceph-mon[32553]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T02:05:11.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:10 smithi173 ceph-mon[32553]: osdmap e110: 12 total, 12 up, 12 in 2024-04-26T02:05:11.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:10 smithi173 ceph-mon[32553]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-26T02:05:11.318 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 02:05:10 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[67264]: ignoring --setuser ceph since I am not root 2024-04-26T02:05:11.318 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 02:05:10 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[67264]: ignoring --setgroup ceph since I am not root 2024-04-26T02:05:11.319 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 02:05:10 smithi173 ceph-mds[67268]: -- 172.21.15.173:0/3886548510 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e38ddada40 con 0x55e38dd81000 2024-04-26T02:05:11.319 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 02:05:10 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[67264]: starting mds.f at 2024-04-26T02:05:11.319 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 26 02:05:10 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-i[63434]: ignoring --setuser ceph since I am not root 2024-04-26T02:05:11.319 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 26 02:05:10 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-i[63434]: ignoring --setgroup ceph since I am not root 2024-04-26T02:05:11.319 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 26 02:05:10 smithi173 ceph-mds[63438]: -- 172.21.15.173:0/186781206 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555b1e443a40 con 0x555b1e416800 2024-04-26T02:05:11.319 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 26 02:05:10 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-i[63434]: starting mds.i at 2024-04-26T02:05:11.319 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 26 02:05:10 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-c[63123]: ignoring --setuser ceph since I am not root 2024-04-26T02:05:11.319 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 26 02:05:10 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-c[63123]: ignoring --setgroup ceph since I am not root 2024-04-26T02:05:11.319 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 26 02:05:10 smithi173 ceph-mds[63127]: -- 172.21.15.173:0/3424791705 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556ec63b7a40 con 0x556ec638a800 2024-04-26T02:05:11.319 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 26 02:05:10 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-c[63123]: starting mds.c at 2024-04-26T02:05:11.650 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T02:05:12.127 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:05:12.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:11 smithi001 ceph-mon[27822]: pgmap v426: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-04-26T02:05:12.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:11 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:05:12.220 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:11 smithi173 ceph-mon[32553]: pgmap v426: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-04-26T02:05:12.220 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:11 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:05:12.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:11 smithi097 ceph-mon[32853]: pgmap v426: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-04-26T02:05:12.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:11 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:05:13.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:12 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.173:6834/2992420229,v1:172.21.15.173:6835/2992420229] up:boot 2024-04-26T02:05:13.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:12 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.97:6834/1191783036,v1:172.21.15.97:6835/1191783036] up:boot 2024-04-26T02:05:13.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:12 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.173:6837/3894499776,v1:172.21.15.173:6839/3894499776] up:boot 2024-04-26T02:05:13.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:12 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.173:6836/1682668663,v1:172.21.15.173:6838/1682668663] up:boot 2024-04-26T02:05:13.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:12 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.97:6836/630230304,v1:172.21.15.97:6837/630230304] up:boot 2024-04-26T02:05:13.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:12 smithi097 ceph-mon[32853]: mds.? [v2:172.21.15.97:6838/1016871002,v1:172.21.15.97:6839/1016871002] up:boot 2024-04-26T02:05:13.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:12 smithi097 ceph-mon[32853]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-26T02:05:13.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:12 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T02:05:13.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:12 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T02:05:13.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:12 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T02:05:13.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:12 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T02:05:13.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:12 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T02:05:13.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:12 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T02:05:13.286 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:12 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.173:6834/2992420229,v1:172.21.15.173:6835/2992420229] up:boot 2024-04-26T02:05:13.286 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:12 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.97:6834/1191783036,v1:172.21.15.97:6835/1191783036] up:boot 2024-04-26T02:05:13.286 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:12 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.173:6837/3894499776,v1:172.21.15.173:6839/3894499776] up:boot 2024-04-26T02:05:13.286 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:12 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.173:6836/1682668663,v1:172.21.15.173:6838/1682668663] up:boot 2024-04-26T02:05:13.286 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:12 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.97:6836/630230304,v1:172.21.15.97:6837/630230304] up:boot 2024-04-26T02:05:13.286 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:12 smithi001 ceph-mon[27822]: mds.? [v2:172.21.15.97:6838/1016871002,v1:172.21.15.97:6839/1016871002] up:boot 2024-04-26T02:05:13.286 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:12 smithi001 ceph-mon[27822]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-26T02:05:13.286 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:12 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T02:05:13.286 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:12 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T02:05:13.286 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:12 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T02:05:13.287 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:12 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T02:05:13.287 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:12 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T02:05:13.287 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:12 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T02:05:13.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:12 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.173:6834/2992420229,v1:172.21.15.173:6835/2992420229] up:boot 2024-04-26T02:05:13.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:12 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.97:6834/1191783036,v1:172.21.15.97:6835/1191783036] up:boot 2024-04-26T02:05:13.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:12 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.173:6837/3894499776,v1:172.21.15.173:6839/3894499776] up:boot 2024-04-26T02:05:13.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:12 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.173:6836/1682668663,v1:172.21.15.173:6838/1682668663] up:boot 2024-04-26T02:05:13.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:12 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.97:6836/630230304,v1:172.21.15.97:6837/630230304] up:boot 2024-04-26T02:05:13.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:12 smithi173 ceph-mon[32553]: mds.? [v2:172.21.15.97:6838/1016871002,v1:172.21.15.97:6839/1016871002] up:boot 2024-04-26T02:05:13.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:12 smithi173 ceph-mon[32553]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-26T02:05:13.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:12 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T02:05:13.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:12 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T02:05:13.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:12 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T02:05:13.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:12 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T02:05:13.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:12 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T02:05:13.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:12 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T02:05:14.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.118+0000 7f05a2600700 1 -- 172.21.15.1:0/2781834533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f059c103320 msgr2=0x7f059c10f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:14.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.118+0000 7f05a2600700 1 --2- 172.21.15.1:0/2781834533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f059c103320 0x7f059c10f830 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f059400b0a0 tx=0x7f059400b3b0 comp rx=0 tx=0).stop 2024-04-26T02:05:14.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.118+0000 7f05a2600700 1 -- 172.21.15.1:0/2781834533 shutdown_connections 2024-04-26T02:05:14.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.118+0000 7f05a2600700 1 --2- 172.21.15.1:0/2781834533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f059c103320 0x7f059c10f830 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:14.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.118+0000 7f05a2600700 1 --2- 172.21.15.1:0/2781834533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f059c108960 0x7f059c108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:14.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.118+0000 7f05a2600700 1 --2- 172.21.15.1:0/2781834533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f059c102960 0x7f059c102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:14.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.118+0000 7f05a2600700 1 -- 172.21.15.1:0/2781834533 >> 172.21.15.1:0/2781834533 conn(0x7f059c0fe490 msgr2=0x7f059c1008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:14.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.118+0000 7f05a2600700 1 -- 172.21.15.1:0/2781834533 shutdown_connections 2024-04-26T02:05:14.122 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:13 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:05:14.122 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:13 smithi001 ceph-mon[27822]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:05:14.122 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:13 smithi001 ceph-mon[27822]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:05:14.122 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:13 smithi001 ceph-mon[27822]: pgmap v427: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-04-26T02:05:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.119+0000 7f05a2600700 1 -- 172.21.15.1:0/2781834533 wait complete. 2024-04-26T02:05:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.120+0000 7f05a2600700 1 Processor -- start 2024-04-26T02:05:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.120+0000 7f05a2600700 1 -- start start 2024-04-26T02:05:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.120+0000 7f05a2600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f059c102960 0x7f059c19b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.120+0000 7f05a2600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f059c103320 0x7f059c19b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.120+0000 7f05a2600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f059c108960 0x7f059c19fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.120+0000 7f05a2600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f059c112bf0 con 0x7f059c102960 2024-04-26T02:05:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.120+0000 7f05a2600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f059c112a70 con 0x7f059c103320 2024-04-26T02:05:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.120+0000 7f05a2600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f059c112d70 con 0x7f059c108960 2024-04-26T02:05:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.121+0000 7f05a1200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f059c102960 0x7f059c19b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.121+0000 7f0593e00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f059c103320 0x7f059c19b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.121+0000 7f05a1200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f059c102960 0x7f059c19b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49666/0 (socket says 172.21.15.1:49666) 2024-04-26T02:05:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.121+0000 7f05a1200700 1 -- 172.21.15.1:0/1483857608 learned_addr learned my addr 172.21.15.1:0/1483857608 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:05:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.121+0000 7f05a1c00700 1 --2- 172.21.15.1:0/1483857608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f059c108960 0x7f059c19fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.121+0000 7f05a1200700 1 -- 172.21.15.1:0/1483857608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f059c108960 msgr2=0x7f059c19fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.121+0000 7f05a1200700 1 --2- 172.21.15.1:0/1483857608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f059c108960 0x7f059c19fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.121+0000 7f05a1200700 1 -- 172.21.15.1:0/1483857608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f059c103320 msgr2=0x7f059c19b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.121+0000 7f05a1200700 1 --2- 172.21.15.1:0/1483857608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f059c103320 0x7f059c19b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.121+0000 7f05a1200700 1 -- 172.21.15.1:0/1483857608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059c1a0150 con 0x7f059c102960 2024-04-26T02:05:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.122+0000 7f05a1200700 1 --2- 172.21.15.1:0/1483857608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f059c102960 0x7f059c19b140 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f058000e9a0 tx=0x7f058000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.122+0000 7f059a000700 1 -- 172.21.15.1:0/1483857608 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058000ca10 con 0x7f059c102960 2024-04-26T02:05:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.122+0000 7f059a000700 1 -- 172.21.15.1:0/1483857608 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f058000cbb0 con 0x7f059c102960 2024-04-26T02:05:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.122+0000 7f05a2600700 1 -- 172.21.15.1:0/1483857608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f059c1a0440 con 0x7f059c102960 2024-04-26T02:05:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.122+0000 7f059a000700 1 -- 172.21.15.1:0/1483857608 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0580018680 con 0x7f059c102960 2024-04-26T02:05:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.123+0000 7f05a2600700 1 -- 172.21.15.1:0/1483857608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f059c1a7cc0 con 0x7f059c102960 2024-04-26T02:05:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.124+0000 7f05a2600700 1 -- 172.21.15.1:0/1483857608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f059c069730 con 0x7f059c102960 2024-04-26T02:05:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.126+0000 7f059a000700 1 -- 172.21.15.1:0/1483857608 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7f0580010430 con 0x7f059c102960 2024-04-26T02:05:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.127+0000 7f059a000700 1 --2- 172.21.15.1:0/1483857608 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f05780a3e30 0x7f05780a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.127+0000 7f059a000700 1 -- 172.21.15.1:0/1483857608 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8785+0+0 (secure 0 0 0) 0x7f05800d0a10 con 0x7f059c102960 2024-04-26T02:05:14.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.128+0000 7f0593e00700 1 --2- 172.21.15.1:0/1483857608 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f05780a3e30 0x7f05780a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:14.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.128+0000 7f0593e00700 1 --2- 172.21.15.1:0/1483857608 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f05780a3e30 0x7f05780a62f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f059c076ba0 tx=0x7f058800a380 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:14.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.132+0000 7f059a000700 1 -- 172.21.15.1:0/1483857608 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f058009cf40 con 0x7f059c102960 2024-04-26T02:05:14.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:13 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:05:14.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:13 smithi097 ceph-mon[32853]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:05:14.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:13 smithi097 ceph-mon[32853]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:05:14.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:13 smithi097 ceph-mon[32853]: pgmap v427: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-04-26T02:05:14.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:13 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:05:14.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:13 smithi173 ceph-mon[32553]: from='mgr.25087 172.21.15.1:0/228461785' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:05:14.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:13 smithi173 ceph-mon[32553]: from='mgr.25087 ' entity='mgr.x' 2024-04-26T02:05:14.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:13 smithi173 ceph-mon[32553]: pgmap v427: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-04-26T02:05:14.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.418+0000 7f05a2600700 1 -- 172.21.15.1:0/1483857608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f059c1a7ef0 con 0x7f059c102960 2024-04-26T02:05:14.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.913+0000 7f059a000700 1 -- 172.21.15.1:0/1483857608 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7f058009c840 con 0x7f059c102960 2024-04-26T02:05:14.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.917+0000 7f05a2600700 1 -- 172.21.15.1:0/1483857608 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f05780a3e30 msgr2=0x7f05780a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:14.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.917+0000 7f05a2600700 1 --2- 172.21.15.1:0/1483857608 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f05780a3e30 0x7f05780a62f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f059c076ba0 tx=0x7f058800a380 comp rx=0 tx=0).stop 2024-04-26T02:05:14.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.917+0000 7f05a2600700 1 -- 172.21.15.1:0/1483857608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f059c102960 msgr2=0x7f059c19b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:14.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.917+0000 7f05a2600700 1 --2- 172.21.15.1:0/1483857608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f059c102960 0x7f059c19b140 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f058000e9a0 tx=0x7f058000ecb0 comp rx=0 tx=0).stop 2024-04-26T02:05:14.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.917+0000 7f05a2600700 1 -- 172.21.15.1:0/1483857608 shutdown_connections 2024-04-26T02:05:14.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.917+0000 7f05a2600700 1 --2- 172.21.15.1:0/1483857608 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7f05780a3e30 0x7f05780a62f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:14.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.917+0000 7f05a2600700 1 --2- 172.21.15.1:0/1483857608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f059c108960 0x7f059c19fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:14.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.917+0000 7f05a2600700 1 --2- 172.21.15.1:0/1483857608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f059c102960 0x7f059c19b140 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:14.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.917+0000 7f05a2600700 1 --2- 172.21.15.1:0/1483857608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f059c103320 0x7f059c19b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:14.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.917+0000 7f05a2600700 1 -- 172.21.15.1:0/1483857608 >> 172.21.15.1:0/1483857608 conn(0x7f059c0fe490 msgr2=0x7f059c0750b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:14.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.918+0000 7f05a2600700 1 -- 172.21.15.1:0/1483857608 shutdown_connections 2024-04-26T02:05:14.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:14.918+0000 7f05a2600700 1 -- 172.21.15.1:0/1483857608 wait complete. 2024-04-26T02:05:15.162 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:14 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1483857608' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T02:05:15.308 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:14 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1483857608' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T02:05:15.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:14 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1483857608' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T02:05:15.739 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T02:05:16.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:15 smithi001 ceph-mon[27822]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T02:05:16.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:15 smithi001 ceph-mon[27822]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T02:05:16.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:15 smithi001 ceph-mon[27822]: Cluster is now healthy 2024-04-26T02:05:16.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:15 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1483857608' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T02:05:16.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:15 smithi001 ceph-mon[27822]: fsmap 12 up:standby 2024-04-26T02:05:16.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:15 smithi001 ceph-mon[27822]: pgmap v428: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-26T02:05:16.210 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:05:16.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:15 smithi097 ceph-mon[32853]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T02:05:16.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:15 smithi097 ceph-mon[32853]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T02:05:16.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:15 smithi097 ceph-mon[32853]: Cluster is now healthy 2024-04-26T02:05:16.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:15 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1483857608' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T02:05:16.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:15 smithi097 ceph-mon[32853]: fsmap 12 up:standby 2024-04-26T02:05:16.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:15 smithi097 ceph-mon[32853]: pgmap v428: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-26T02:05:16.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:15 smithi173 ceph-mon[32553]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T02:05:16.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:15 smithi173 ceph-mon[32553]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T02:05:16.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:15 smithi173 ceph-mon[32553]: Cluster is now healthy 2024-04-26T02:05:16.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:15 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1483857608' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T02:05:16.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:15 smithi173 ceph-mon[32553]: fsmap 12 up:standby 2024-04-26T02:05:16.318 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:15 smithi173 ceph-mon[32553]: pgmap v428: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-26T02:05:18.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.320+0000 7efc70600700 1 -- 172.21.15.1:0/3969892283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc68074c70 msgr2=0x7efc68075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:18.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.320+0000 7efc70600700 1 --2- 172.21.15.1:0/3969892283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc68074c70 0x7efc68075050 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efc5c009990 tx=0x7efc5c009ca0 comp rx=0 tx=0).stop 2024-04-26T02:05:18.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.320+0000 7efc70600700 1 -- 172.21.15.1:0/3969892283 shutdown_connections 2024-04-26T02:05:18.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.320+0000 7efc70600700 1 --2- 172.21.15.1:0/3969892283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc6810f4a0 0x7efc68111890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:18.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.320+0000 7efc70600700 1 --2- 172.21.15.1:0/3969892283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc68074c70 0x7efc68075050 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:18.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.320+0000 7efc70600700 1 --2- 172.21.15.1:0/3969892283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc68075590 0x7efc6810ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:18.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.320+0000 7efc70600700 1 -- 172.21.15.1:0/3969892283 >> 172.21.15.1:0/3969892283 conn(0x7efc68079ba0 msgr2=0x7efc68079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:18.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.320+0000 7efc70600700 1 -- 172.21.15.1:0/3969892283 shutdown_connections 2024-04-26T02:05:18.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.320+0000 7efc70600700 1 -- 172.21.15.1:0/3969892283 wait complete. 2024-04-26T02:05:18.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.321+0000 7efc70600700 1 Processor -- start 2024-04-26T02:05:18.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.321+0000 7efc70600700 1 -- start start 2024-04-26T02:05:18.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.322+0000 7efc70600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc68074c70 0x7efc68102bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:18.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.322+0000 7efc70600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc68075590 0x7efc68103100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:18.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.322+0000 7efc70600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc6810f4a0 0x7efc68103640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:18.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.322+0000 7efc70600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efc68114e90 con 0x7efc68074c70 2024-04-26T02:05:18.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.322+0000 7efc70600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efc68114d10 con 0x7efc68075590 2024-04-26T02:05:18.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.322+0000 7efc70600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efc68115010 con 0x7efc6810f4a0 2024-04-26T02:05:18.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.322+0000 7efc6dc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc68074c70 0x7efc68102bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:18.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.322+0000 7efc6dc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc68074c70 0x7efc68102bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49694/0 (socket says 172.21.15.1:49694) 2024-04-26T02:05:18.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.322+0000 7efc6dc00700 1 -- 172.21.15.1:0/3191388070 learned_addr learned my addr 172.21.15.1:0/3191388070 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:05:18.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.322+0000 7efc6d200700 1 --2- 172.21.15.1:0/3191388070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc68075590 0x7efc68103100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:18.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.322+0000 7efc6e600700 1 --2- 172.21.15.1:0/3191388070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc6810f4a0 0x7efc68103640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:18.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.322+0000 7efc6d200700 1 -- 172.21.15.1:0/3191388070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc6810f4a0 msgr2=0x7efc68103640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:18.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.322+0000 7efc6d200700 1 --2- 172.21.15.1:0/3191388070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc6810f4a0 0x7efc68103640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:18.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.322+0000 7efc6d200700 1 -- 172.21.15.1:0/3191388070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc68074c70 msgr2=0x7efc68102bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:18.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.322+0000 7efc6d200700 1 --2- 172.21.15.1:0/3191388070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc68074c70 0x7efc68102bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:18.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.322+0000 7efc6d200700 1 -- 172.21.15.1:0/3191388070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc681ab9b0 con 0x7efc68075590 2024-04-26T02:05:18.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.323+0000 7efc6dc00700 1 --2- 172.21.15.1:0/3191388070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc68074c70 0x7efc68102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:05:18.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.323+0000 7efc6e600700 1 --2- 172.21.15.1:0/3191388070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc6810f4a0 0x7efc68103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:05:18.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.323+0000 7efc6d200700 1 --2- 172.21.15.1:0/3191388070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc68075590 0x7efc68103100 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7efc5400ea20 tx=0x7efc5400ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:18.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.323+0000 7efc66000700 1 -- 172.21.15.1:0/3191388070 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54004220 con 0x7efc68075590 2024-04-26T02:05:18.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.323+0000 7efc66000700 1 -- 172.21.15.1:0/3191388070 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc5400ce20 con 0x7efc68075590 2024-04-26T02:05:18.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.323+0000 7efc66000700 1 -- 172.21.15.1:0/3191388070 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54003cf0 con 0x7efc68075590 2024-04-26T02:05:18.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.323+0000 7efc70600700 1 -- 172.21.15.1:0/3191388070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc681abca0 con 0x7efc68075590 2024-04-26T02:05:18.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.323+0000 7efc70600700 1 -- 172.21.15.1:0/3191388070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc681ac190 con 0x7efc68075590 2024-04-26T02:05:18.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.325+0000 7efc70600700 1 -- 172.21.15.1:0/3191388070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc68076330 con 0x7efc68075590 2024-04-26T02:05:18.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.326+0000 7efc66000700 1 -- 172.21.15.1:0/3191388070 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7efc5400c970 con 0x7efc68075590 2024-04-26T02:05:18.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.328+0000 7efc66000700 1 --2- 172.21.15.1:0/3191388070 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efc500a3db0 0x7efc500a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:18.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.328+0000 7efc66000700 1 -- 172.21.15.1:0/3191388070 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8785+0+0 (secure 0 0 0) 0x7efc540d0ad0 con 0x7efc68075590 2024-04-26T02:05:18.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.328+0000 7efc6dc00700 1 --2- 172.21.15.1:0/3191388070 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efc500a3db0 0x7efc500a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:18.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.329+0000 7efc6dc00700 1 --2- 172.21.15.1:0/3191388070 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efc500a3db0 0x7efc500a6270 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efc5c0098e0 tx=0x7efc5c00b2a0 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:18.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.332+0000 7efc66000700 1 -- 172.21.15.1:0/3191388070 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc5409d000 con 0x7efc68075590 2024-04-26T02:05:18.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:18 smithi001 ceph-mon[27822]: pgmap v429: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-26T02:05:18.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:18 smithi097 ceph-mon[32853]: pgmap v429: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-26T02:05:18.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:18 smithi173 ceph-mon[32553]: pgmap v429: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-26T02:05:18.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:18.568+0000 7efc70600700 1 -- 172.21.15.1:0/3191388070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7efc68069730 con 0x7efc68075590 2024-04-26T02:05:19.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:19.121+0000 7efc66000700 1 -- 172.21.15.1:0/3191388070 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v111) ==== 181+0+0 (secure 0 0 0) 0x7efc68069730 con 0x7efc68075590 2024-04-26T02:05:19.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:19.127+0000 7efc70600700 1 -- 172.21.15.1:0/3191388070 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efc500a3db0 msgr2=0x7efc500a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:19.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:19.127+0000 7efc70600700 1 --2- 172.21.15.1:0/3191388070 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efc500a3db0 0x7efc500a6270 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efc5c0098e0 tx=0x7efc5c00b2a0 comp rx=0 tx=0).stop 2024-04-26T02:05:19.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:19.127+0000 7efc70600700 1 -- 172.21.15.1:0/3191388070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc68075590 msgr2=0x7efc68103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:19.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:19.127+0000 7efc70600700 1 --2- 172.21.15.1:0/3191388070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc68075590 0x7efc68103100 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7efc5400ea20 tx=0x7efc5400ed30 comp rx=0 tx=0).stop 2024-04-26T02:05:19.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:19.127+0000 7efc70600700 1 -- 172.21.15.1:0/3191388070 shutdown_connections 2024-04-26T02:05:19.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:19.127+0000 7efc70600700 1 --2- 172.21.15.1:0/3191388070 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7efc500a3db0 0x7efc500a6270 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:19.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:19.127+0000 7efc70600700 1 --2- 172.21.15.1:0/3191388070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc6810f4a0 0x7efc68103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:19.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:19.127+0000 7efc70600700 1 --2- 172.21.15.1:0/3191388070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc68074c70 0x7efc68102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:19.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:19.127+0000 7efc70600700 1 --2- 172.21.15.1:0/3191388070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc68075590 0x7efc68103100 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:19.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:19.127+0000 7efc70600700 1 -- 172.21.15.1:0/3191388070 >> 172.21.15.1:0/3191388070 conn(0x7efc68079ba0 msgr2=0x7efc680ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:19.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:19.129+0000 7efc70600700 1 -- 172.21.15.1:0/3191388070 shutdown_connections 2024-04-26T02:05:19.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:19.129+0000 7efc70600700 1 -- 172.21.15.1:0/3191388070 wait complete. 2024-04-26T02:05:19.129 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' removed 2024-04-26T02:05:19.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:19 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/3191388070' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:05:19.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:19 smithi001 ceph-mon[27822]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:05:19.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:19 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/3191388070' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:05:19.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:19 smithi097 ceph-mon[32853]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:05:19.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:19 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/3191388070' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:05:19.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:19 smithi173 ceph-mon[32553]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:05:19.958 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T02:05:20.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:20 smithi001 ceph-mon[27822]: pgmap v430: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-26T02:05:20.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:20 smithi001 ceph-mon[27822]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T02:05:20.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:20 smithi001 ceph-mon[27822]: osdmap e111: 12 total, 12 up, 12 in 2024-04-26T02:05:20.428 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:05:20.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:20 smithi097 ceph-mon[32853]: pgmap v430: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-26T02:05:20.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:20 smithi097 ceph-mon[32853]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T02:05:20.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:20 smithi097 ceph-mon[32853]: osdmap e111: 12 total, 12 up, 12 in 2024-04-26T02:05:20.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:20 smithi173 ceph-mon[32553]: pgmap v430: 97 pgs: 97 active+clean; 159 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 204 B/s wr, 3 op/s 2024-04-26T02:05:20.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:20 smithi173 ceph-mon[32553]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T02:05:20.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:20 smithi173 ceph-mon[32553]: osdmap e111: 12 total, 12 up, 12 in 2024-04-26T02:05:22.369 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:22 smithi001 ceph-mon[27822]: pgmap v432: 65 pgs: 65 active+clean; 155 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T02:05:22.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:22 smithi097 ceph-mon[32853]: pgmap v432: 65 pgs: 65 active+clean; 155 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T02:05:22.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.548+0000 7faa93200700 1 -- 172.21.15.1:0/2917399746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa94075790 msgr2=0x7faa9410ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:22.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.548+0000 7faa93200700 1 --2- 172.21.15.1:0/2917399746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa94075790 0x7faa9410ec70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7faa80009990 tx=0x7faa80009ca0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.549+0000 7faa93200700 1 -- 172.21.15.1:0/2917399746 shutdown_connections 2024-04-26T02:05:22.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.549+0000 7faa93200700 1 --2- 172.21.15.1:0/2917399746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa9410f1b0 0x7faa94111620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.549+0000 7faa93200700 1 --2- 172.21.15.1:0/2917399746 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa94074e70 0x7faa94075250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.549+0000 7faa93200700 1 --2- 172.21.15.1:0/2917399746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa94075790 0x7faa9410ec70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.549+0000 7faa93200700 1 -- 172.21.15.1:0/2917399746 >> 172.21.15.1:0/2917399746 conn(0x7faa940fe4c0 msgr2=0x7faa941008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:22.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.549+0000 7faa93200700 1 -- 172.21.15.1:0/2917399746 shutdown_connections 2024-04-26T02:05:22.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.549+0000 7faa93200700 1 -- 172.21.15.1:0/2917399746 wait complete. 2024-04-26T02:05:22.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.550+0000 7faa93200700 1 Processor -- start 2024-04-26T02:05:22.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.550+0000 7faa93200700 1 -- start start 2024-04-26T02:05:22.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.550+0000 7faa93200700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa94074e70 0x7faa9419b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.550+0000 7faa93200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa94075790 0x7faa9419b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.550+0000 7faa93200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa9410f1b0 0x7faa9419fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.550+0000 7faa93200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faa94114d30 con 0x7faa94075790 2024-04-26T02:05:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.550+0000 7faa93200700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faa94114bb0 con 0x7faa94074e70 2024-04-26T02:05:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.550+0000 7faa93200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faa94114eb0 con 0x7faa9410f1b0 2024-04-26T02:05:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.551+0000 7faa91400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa94075790 0x7faa9419b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.551+0000 7faa91400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa94075790 0x7faa9419b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49710/0 (socket says 172.21.15.1:49710) 2024-04-26T02:05:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.551+0000 7faa91400700 1 -- 172.21.15.1:0/1581363060 learned_addr learned my addr 172.21.15.1:0/1581363060 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-04-26T02:05:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.551+0000 7faa91e00700 1 --2- 172.21.15.1:0/1581363060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa94074e70 0x7faa9419b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.551+0000 7faa92800700 1 --2- 172.21.15.1:0/1581363060 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa9410f1b0 0x7faa9419fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.551+0000 7faa91e00700 1 -- 172.21.15.1:0/1581363060 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa9410f1b0 msgr2=0x7faa9419fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.551+0000 7faa91e00700 1 --2- 172.21.15.1:0/1581363060 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa9410f1b0 0x7faa9419fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.551+0000 7faa91e00700 1 -- 172.21.15.1:0/1581363060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa94075790 msgr2=0x7faa9419b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.551+0000 7faa91e00700 1 --2- 172.21.15.1:0/1581363060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa94075790 0x7faa9419b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:22.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.551+0000 7faa91e00700 1 -- 172.21.15.1:0/1581363060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa941a02d0 con 0x7faa94074e70 2024-04-26T02:05:22.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.551+0000 7faa91400700 1 --2- 172.21.15.1:0/1581363060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa94075790 0x7faa9419b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:05:22.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.552+0000 7faa91e00700 1 --2- 172.21.15.1:0/1581363060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa94074e70 0x7faa9419b2c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7faa8400e9f0 tx=0x7faa8400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:22.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.552+0000 7faa8aa00700 1 -- 172.21.15.1:0/1581363060 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8400c9c0 con 0x7faa94074e70 2024-04-26T02:05:22.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.552+0000 7faa8aa00700 1 -- 172.21.15.1:0/1581363060 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa8400cb60 con 0x7faa94074e70 2024-04-26T02:05:22.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.552+0000 7faa93200700 1 -- 172.21.15.1:0/1581363060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa941a05c0 con 0x7faa94074e70 2024-04-26T02:05:22.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.552+0000 7faa8aa00700 1 -- 172.21.15.1:0/1581363060 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84018680 con 0x7faa94074e70 2024-04-26T02:05:22.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.552+0000 7faa93200700 1 -- 172.21.15.1:0/1581363060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa941a7e40 con 0x7faa94074e70 2024-04-26T02:05:22.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.553+0000 7faa93200700 1 -- 172.21.15.1:0/1581363060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa94069730 con 0x7faa94074e70 2024-04-26T02:05:22.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.555+0000 7faa8aa00700 1 -- 172.21.15.1:0/1581363060 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138776+0+0 (secure 0 0 0) 0x7faa84010430 con 0x7faa94074e70 2024-04-26T02:05:22.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.557+0000 7faa8aa00700 1 --2- 172.21.15.1:0/1581363060 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7faa780a3e30 0x7faa780a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:05:22.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.557+0000 7faa8aa00700 1 -- 172.21.15.1:0/1581363060 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8260+0+0 (secure 0 0 0) 0x7faa840d0840 con 0x7faa94074e70 2024-04-26T02:05:22.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.557+0000 7faa91400700 1 --2- 172.21.15.1:0/1581363060 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7faa780a3e30 0x7faa780a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:05:22.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.558+0000 7faa91400700 1 --2- 172.21.15.1:0/1581363060 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7faa780a3e30 0x7faa780a62f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faa94110600 tx=0x7faa80009510 comp rx=0 tx=0).ready entity=mgr.25087 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:05:22.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.561+0000 7faa8aa00700 1 -- 172.21.15.1:0/1581363060 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa8409cf80 con 0x7faa94074e70 2024-04-26T02:05:22.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:22 smithi173 ceph-mon[32553]: pgmap v432: 65 pgs: 65 active+clean; 155 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T02:05:22.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:22.797+0000 7faa93200700 1 -- 172.21.15.1:0/1581363060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7faa9402df80 con 0x7faa94074e70 2024-04-26T02:05:23.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:23.133+0000 7faa8aa00700 1 -- 172.21.15.1:0/1581363060 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v112) ==== 169+0+0 (secure 0 0 0) 0x7faa8409c880 con 0x7faa94074e70 2024-04-26T02:05:23.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:23.140+0000 7faa93200700 1 -- 172.21.15.1:0/1581363060 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7faa780a3e30 msgr2=0x7faa780a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:23.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:23.140+0000 7faa93200700 1 --2- 172.21.15.1:0/1581363060 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7faa780a3e30 0x7faa780a62f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faa94110600 tx=0x7faa80009510 comp rx=0 tx=0).stop 2024-04-26T02:05:23.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:23.140+0000 7faa93200700 1 -- 172.21.15.1:0/1581363060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa94074e70 msgr2=0x7faa9419b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:05:23.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:23.140+0000 7faa93200700 1 --2- 172.21.15.1:0/1581363060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa94074e70 0x7faa9419b2c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7faa8400e9f0 tx=0x7faa8400ed00 comp rx=0 tx=0).stop 2024-04-26T02:05:23.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:23.140+0000 7faa93200700 1 -- 172.21.15.1:0/1581363060 shutdown_connections 2024-04-26T02:05:23.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:23.140+0000 7faa93200700 1 --2- 172.21.15.1:0/1581363060 >> [v2:172.21.15.1:6800/275906565,v1:172.21.15.1:6801/275906565] conn(0x7faa780a3e30 0x7faa780a62f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:23.140+0000 7faa93200700 1 --2- 172.21.15.1:0/1581363060 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa9410f1b0 0x7faa9419fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:23.140+0000 7faa93200700 1 --2- 172.21.15.1:0/1581363060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa94074e70 0x7faa9419b2c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:23.140+0000 7faa93200700 1 --2- 172.21.15.1:0/1581363060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa94075790 0x7faa9419b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:05:23.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:23.140+0000 7faa93200700 1 -- 172.21.15.1:0/1581363060 >> 172.21.15.1:0/1581363060 conn(0x7faa940fe4c0 msgr2=0x7faa940ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:05:23.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:23.142+0000 7faa93200700 1 -- 172.21.15.1:0/1581363060 shutdown_connections 2024-04-26T02:05:23.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-04-26T02:05:23.142+0000 7faa93200700 1 -- 172.21.15.1:0/1581363060 wait complete. 2024-04-26T02:05:23.144 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' removed 2024-04-26T02:05:23.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:23 smithi173 ceph-mon[32553]: from='client.? 172.21.15.1:0/1581363060' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:05:23.317 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:23 smithi173 ceph-mon[32553]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:05:23.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:23 smithi097 ceph-mon[32853]: from='client.? 172.21.15.1:0/1581363060' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:05:23.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:23 smithi097 ceph-mon[32853]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:05:23.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:23 smithi001 ceph-mon[27822]: from='client.? 172.21.15.1:0/1581363060' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:05:23.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:23 smithi001 ceph-mon[27822]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:05:23.965 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T02:05:24.060 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T02:05:24.087 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T02:05:24.112 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-26T02:05:24.112 DEBUG:teuthology.orchestra.run.smithi001:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T02:05:24.176 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T02:05:24.190 DEBUG:teuthology.orchestra.run.smithi173:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T02:05:24.205 INFO:tasks.cephadm:Stopping all daemons... 2024-04-26T02:05:24.205 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-26T02:05:24.205 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mon.a 2024-04-26T02:05:24.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:24 smithi097 ceph-mon[32853]: pgmap v433: 65 pgs: 65 active+clean; 155 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T02:05:24.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:24 smithi097 ceph-mon[32853]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T02:05:24.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:24 smithi097 ceph-mon[32853]: osdmap e112: 12 total, 12 up, 12 in 2024-04-26T02:05:24.567 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:24 smithi173 ceph-mon[32553]: pgmap v433: 65 pgs: 65 active+clean; 155 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T02:05:24.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:24 smithi173 ceph-mon[32553]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T02:05:24.568 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:24 smithi173 ceph-mon[32553]: osdmap e112: 12 total, 12 up, 12 in 2024-04-26T02:05:24.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:24 smithi001 ceph-mon[27822]: pgmap v433: 65 pgs: 65 active+clean; 155 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T02:05:24.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:24 smithi001 ceph-mon[27822]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T02:05:24.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:24 smithi001 ceph-mon[27822]: osdmap e112: 12 total, 12 up, 12 in 2024-04-26T02:05:24.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:24 smithi001 systemd[1]: Stopping Ceph mon.a for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:05:25.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:24 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-a[27817]: 2024-04-26T02:05:24.810+0000 7f6826e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:05:25.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:24 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-a[27817]: 2024-04-26T02:05:24.810+0000 7f6826e00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-26T02:05:25.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:24 smithi001 podman[118712]: 2024-04-26 02:05:24.863061493 +0000 UTC m=+0.206567975 container died 42fcca49f2d3af9466a6e2432a77e3ed5c221f4cca3e86821263bd49aa4eefe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-26T02:05:25.535 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:25 smithi001 podman[118712]: 2024-04-26 02:05:25.276757467 +0000 UTC m=+0.620263903 container cleanup 42fcca49f2d3af9466a6e2432a77e3ed5c221f4cca3e86821263bd49aa4eefe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-26T02:05:25.535 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:25 smithi001 bash[118712]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-a 2024-04-26T02:05:25.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:Apr 26 02:05:25 smithi001 podman[118726]: 2024-04-26 02:05:25.536383409 +0000 UTC m=+0.664243967 container remove 42fcca49f2d3af9466a6e2432a77e3ed5c221f4cca3e86821263bd49aa4eefe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-26T02:05:26.054 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mon.a.service' 2024-04-26T02:05:26.208 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:05:26.209 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-26T02:05:26.209 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-26T02:05:26.209 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mon.b 2024-04-26T02:05:26.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:26 smithi097 systemd[1]: Stopping Ceph mon.b for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:05:26.807 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:26 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-b[32849]: 2024-04-26T02:05:26.552+0000 7f57df6f0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:05:26.807 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:26 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-b[32849]: 2024-04-26T02:05:26.552+0000 7f57df6f0700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-26T02:05:27.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:26 smithi097 podman[68965]: 2024-04-26 02:05:26.808478326 +0000 UTC m=+0.407563288 container died e232f88726108e2e1027d8409f615909f84d343e626049130fe45b6da6800de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-b, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T02:05:27.460 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:27 smithi097 podman[68965]: 2024-04-26 02:05:27.194756066 +0000 UTC m=+0.793841035 container cleanup e232f88726108e2e1027d8409f615909f84d343e626049130fe45b6da6800de2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-26T02:05:27.461 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 26 02:05:27 smithi097 bash[68965]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-b 2024-04-26T02:05:27.636 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mon.b.service' 2024-04-26T02:05:27.679 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:05:27.679 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-26T02:05:27.679 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-26T02:05:27.679 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mon.c 2024-04-26T02:05:27.992 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:27 smithi173 systemd[1]: Stopping Ceph mon.c for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:05:28.256 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:27 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-c[32527]: 2024-04-26T02:05:27.991+0000 7f279f826700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:05:28.256 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:27 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-c[32527]: 2024-04-26T02:05:27.991+0000 7f279f826700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-26T02:05:28.525 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:28 smithi173 podman[70529]: 2024-04-26 02:05:28.257853511 +0000 UTC m=+0.385720080 container died 6dd41fdc413dd010beb0b585f4b99c21bc76c99c8d331103ca1368740c07ddb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-c, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:05:28.817 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:28 smithi173 podman[70529]: 2024-04-26 02:05:28.634622771 +0000 UTC m=+0.762489349 container cleanup 6dd41fdc413dd010beb0b585f4b99c21bc76c99c8d331103ca1368740c07ddb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:05:28.818 INFO:journalctl@ceph.mon.c.smithi173.stdout:Apr 26 02:05:28 smithi173 bash[70529]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mon-c 2024-04-26T02:05:29.012 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mon.c.service' 2024-04-26T02:05:29.055 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:05:29.055 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-26T02:05:29.055 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-26T02:05:29.056 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mgr.x 2024-04-26T02:05:29.380 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 02:05:29 smithi001 systemd[1]: Stopping Ceph mgr.x for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:05:29.881 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 02:05:29 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T02:05:29.595+0000 7f8dafc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:05:29.881 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 02:05:29 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x[28059]: 2024-04-26T02:05:29.595+0000 7f8dafc00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-26T02:05:29.881 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 02:05:29 smithi001 podman[118842]: 2024-04-26 02:05:29.700129385 +0000 UTC m=+0.239592449 container died 8387a05c9c89fb71da1a5240a3872aee14cf844486d3c5d73b9ac5501ca5ba55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-26T02:05:30.380 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 02:05:30 smithi001 podman[118842]: 2024-04-26 02:05:30.215363676 +0000 UTC m=+0.754826652 container cleanup 8387a05c9c89fb71da1a5240a3872aee14cf844486d3c5d73b9ac5501ca5ba55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:05:30.381 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 02:05:30 smithi001 bash[118842]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x 2024-04-26T02:05:30.880 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Apr 26 02:05:30 smithi001 podman[118855]: 2024-04-26 02:05:30.433628924 +0000 UTC m=+0.715483565 container remove 8387a05c9c89fb71da1a5240a3872aee14cf844486d3c5d73b9ac5501ca5ba55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-x, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-26T02:05:30.934 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-26T02:05:31.051 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:05:31.051 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-26T02:05:31.051 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-26T02:05:31.051 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mgr.y 2024-04-26T02:05:31.468 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 02:05:31 smithi097 systemd[1]: Stopping Ceph mgr.y for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:05:31.734 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 02:05:31 smithi097 podman[69088]: 2024-04-26 02:05:31.470655147 +0000 UTC m=+0.226065945 container died b29fd37f9ca712deefc52882fb097e7f5fae919e5a4b74fcb79b060921304da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-26T02:05:32.098 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 02:05:31 smithi097 podman[69088]: 2024-04-26 02:05:31.857308814 +0000 UTC m=+0.612719602 container cleanup b29fd37f9ca712deefc52882fb097e7f5fae919e5a4b74fcb79b060921304da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-26T02:05:32.098 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 26 02:05:31 smithi097 bash[69088]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-y 2024-04-26T02:05:32.224 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-26T02:05:32.257 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:05:32.257 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-26T02:05:32.257 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-26T02:05:32.257 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mgr.z 2024-04-26T02:05:32.567 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 02:05:32 smithi173 systemd[1]: Stopping Ceph mgr.z for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:05:32.887 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 02:05:32 smithi173 podman[70651]: 2024-04-26 02:05:32.601059698 +0000 UTC m=+0.152234246 container died 28ffc01326f14f2bfdf9fc08524a9a9d031ab86fe640eadcac71c7421b7b084c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:05:33.164 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 02:05:32 smithi173 podman[70651]: 2024-04-26 02:05:32.964098448 +0000 UTC m=+0.515272986 container cleanup 28ffc01326f14f2bfdf9fc08524a9a9d031ab86fe640eadcac71c7421b7b084c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:05:33.164 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Apr 26 02:05:32 smithi173 bash[70651]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mgr-z 2024-04-26T02:05:33.386 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-26T02:05:33.418 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:05:33.418 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-26T02:05:33.418 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-26T02:05:33.418 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.0 2024-04-26T02:05:33.880 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 02:05:33 smithi001 systemd[1]: Stopping Ceph osd.0 for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:05:34.380 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 02:05:33 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-0[40041]: 2024-04-26T02:05:33.976+0000 7fa9ee200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:05:34.381 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 02:05:33 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-0[40041]: 2024-04-26T02:05:33.976+0000 7fa9ee200700 -1 osd.0 112 *** Got signal Terminated *** 2024-04-26T02:05:34.381 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 02:05:33 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-0[40041]: 2024-04-26T02:05:33.976+0000 7fa9ee200700 -1 osd.0 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:05:39.881 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 02:05:39 smithi001 podman[118961]: 2024-04-26 02:05:39.581545907 +0000 UTC m=+5.736852583 container died bdf5bb80655969e6427f32022f79d8f34c79c7d3311ec017f975808969d15827 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T02:05:40.237 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 02:05:40 smithi001 podman[118961]: 2024-04-26 02:05:40.044748762 +0000 UTC m=+6.200055474 container cleanup bdf5bb80655969e6427f32022f79d8f34c79c7d3311ec017f975808969d15827 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T02:05:40.237 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 02:05:40 smithi001 bash[118961]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-0 2024-04-26T02:05:40.630 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 02:05:40 smithi001 podman[118980]: 2024-04-26 02:05:40.237328515 +0000 UTC m=+0.645017873 container remove bdf5bb80655969e6427f32022f79d8f34c79c7d3311ec017f975808969d15827 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-26T02:05:41.380 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 02:05:41 smithi001 podman[119075]: 2024-04-26 02:05:41.0285497 +0000 UTC m=+0.036651030 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:05:41.381 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 02:05:41 smithi001 podman[119075]: 2024-04-26 02:05:41.163595083 +0000 UTC m=+0.171696457 container create aa081eb3d5d8003976b0c939075d17b3ba8af6d0871fbec9c23fd7770e215e79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:05:41.881 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 02:05:41 smithi001 podman[119075]: 2024-04-26 02:05:41.4343757 +0000 UTC m=+0.442477094 container init aa081eb3d5d8003976b0c939075d17b3ba8af6d0871fbec9c23fd7770e215e79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-26T02:05:41.881 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 02:05:41 smithi001 podman[119075]: 2024-04-26 02:05:41.446727988 +0000 UTC m=+0.454829369 container start aa081eb3d5d8003976b0c939075d17b3ba8af6d0871fbec9c23fd7770e215e79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T02:05:41.881 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 02:05:41 smithi001 podman[119075]: 2024-04-26 02:05:41.566222667 +0000 UTC m=+0.574324017 container attach aa081eb3d5d8003976b0c939075d17b3ba8af6d0871fbec9c23fd7770e215e79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:05:42.273 INFO:journalctl@ceph.osd.0.smithi001.stdout:Apr 26 02:05:41 smithi001 podman[119075]: 2024-04-26 02:05:41.918782088 +0000 UTC m=+0.926883445 container died aa081eb3d5d8003976b0c939075d17b3ba8af6d0871fbec9c23fd7770e215e79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T02:05:42.726 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.0.service' 2024-04-26T02:05:42.839 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:05:42.840 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-26T02:05:42.840 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-26T02:05:42.840 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.1 2024-04-26T02:05:43.152 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 02:05:42 smithi097 systemd[1]: Stopping Ceph osd.1 for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:05:43.484 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 02:05:43 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-1[37441]: 2024-04-26T02:05:43.152+0000 7fb236ac8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:05:43.484 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 02:05:43 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-1[37441]: 2024-04-26T02:05:43.152+0000 7fb236ac8700 -1 osd.1 112 *** Got signal Terminated *** 2024-04-26T02:05:43.484 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 02:05:43 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-1[37441]: 2024-04-26T02:05:43.152+0000 7fb236ac8700 -1 osd.1 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:05:48.984 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 02:05:48 smithi097 podman[69213]: 2024-04-26 02:05:48.702247529 +0000 UTC m=+5.672876336 container died f62b60ea36ac8635b624a5817f61df9e84bb3d9b9b60a811fee837da519e3b4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-1, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T02:05:49.344 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 02:05:49 smithi097 podman[69213]: 2024-04-26 02:05:49.103080308 +0000 UTC m=+6.073709123 container cleanup f62b60ea36ac8635b624a5817f61df9e84bb3d9b9b60a811fee837da519e3b4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-1, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-26T02:05:49.344 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 02:05:49 smithi097 bash[69213]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-1 2024-04-26T02:05:49.711 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 02:05:49 smithi097 podman[69230]: 2024-04-26 02:05:49.362721423 +0000 UTC m=+0.653764034 container remove f62b60ea36ac8635b624a5817f61df9e84bb3d9b9b60a811fee837da519e3b4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-1, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T02:05:49.984 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 02:05:49 smithi097 podman[69329]: 2024-04-26 02:05:49.713497601 +0000 UTC m=+0.108027416 container create dadf1c5a8d89b17080913994c5b0772a7cb3b3fa338ac6685549c34c53e48664 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:05:49.984 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 02:05:49 smithi097 podman[69329]: 2024-04-26 02:05:49.624104761 +0000 UTC m=+0.018634588 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:05:49.984 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 02:05:49 smithi097 podman[69329]: 2024-04-26 02:05:49.940716365 +0000 UTC m=+0.335246188 container init dadf1c5a8d89b17080913994c5b0772a7cb3b3fa338ac6685549c34c53e48664 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:05:49.984 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 02:05:49 smithi097 podman[69329]: 2024-04-26 02:05:49.94371043 +0000 UTC m=+0.338240247 container start dadf1c5a8d89b17080913994c5b0772a7cb3b3fa338ac6685549c34c53e48664 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:05:50.238 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 02:05:50 smithi097 podman[69329]: 2024-04-26 02:05:50.074768347 +0000 UTC m=+0.469298167 container attach dadf1c5a8d89b17080913994c5b0772a7cb3b3fa338ac6685549c34c53e48664 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:05:50.576 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 26 02:05:50 smithi097 podman[69329]: 2024-04-26 02:05:50.239373154 +0000 UTC m=+0.633902971 container died dadf1c5a8d89b17080913994c5b0772a7cb3b3fa338ac6685549c34c53e48664 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-26T02:05:50.863 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.1.service' 2024-04-26T02:05:50.904 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:05:50.904 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-26T02:05:50.904 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-26T02:05:50.904 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.2 2024-04-26T02:05:51.218 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 02:05:50 smithi173 systemd[1]: Stopping Ceph osd.2 for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:05:51.557 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 02:05:51 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-2[37698]: 2024-04-26T02:05:51.219+0000 7fd79bec5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:05:51.557 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 02:05:51 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-2[37698]: 2024-04-26T02:05:51.219+0000 7fd79bec5700 -1 osd.2 112 *** Got signal Terminated *** 2024-04-26T02:05:51.558 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 02:05:51 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-2[37698]: 2024-04-26T02:05:51.219+0000 7fd79bec5700 -1 osd.2 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:05:56.879 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 02:05:56 smithi173 podman[70775]: 2024-04-26 02:05:56.611846143 +0000 UTC m=+5.506573667 container died aef5d18593148fe11c0cfcc290214c5d7a799d14a82bd7d0ddd9935b1308c305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-26T02:05:57.228 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 02:05:56 smithi173 podman[70775]: 2024-04-26 02:05:56.970691861 +0000 UTC m=+5.865419382 container cleanup aef5d18593148fe11c0cfcc290214c5d7a799d14a82bd7d0ddd9935b1308c305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-26T02:05:57.228 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 02:05:56 smithi173 bash[70775]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-2 2024-04-26T02:05:57.567 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 02:05:57 smithi173 podman[70805]: 2024-04-26 02:05:57.230199129 +0000 UTC m=+0.612914607 container remove aef5d18593148fe11c0cfcc290214c5d7a799d14a82bd7d0ddd9935b1308c305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-26T02:05:57.874 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 02:05:57 smithi173 podman[70897]: 2024-04-26 02:05:57.492687813 +0000 UTC m=+0.018025468 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:05:57.874 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 02:05:57 smithi173 podman[70897]: 2024-04-26 02:05:57.61463237 +0000 UTC m=+0.139970004 container create 24f156204405d095986f7250f09147552d2d64417ebaa82660743bbcfa286630 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:05:57.874 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 02:05:57 smithi173 podman[70897]: 2024-04-26 02:05:57.800793954 +0000 UTC m=+0.326131589 container init 24f156204405d095986f7250f09147552d2d64417ebaa82660743bbcfa286630 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:05:57.874 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 02:05:57 smithi173 podman[70897]: 2024-04-26 02:05:57.804015957 +0000 UTC m=+0.329353596 container start 24f156204405d095986f7250f09147552d2d64417ebaa82660743bbcfa286630 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-26T02:05:57.874 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 02:05:57 smithi173 podman[70897]: 2024-04-26 02:05:57.875682546 +0000 UTC m=+0.401020198 container attach 24f156204405d095986f7250f09147552d2d64417ebaa82660743bbcfa286630 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T02:05:58.317 INFO:journalctl@ceph.osd.2.smithi173.stdout:Apr 26 02:05:58 smithi173 podman[70897]: 2024-04-26 02:05:58.095667879 +0000 UTC m=+0.621005517 container died 24f156204405d095986f7250f09147552d2d64417ebaa82660743bbcfa286630 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-26T02:05:58.714 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.2.service' 2024-04-26T02:05:58.753 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:05:58.753 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-26T02:05:58.753 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-26T02:05:58.753 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.3 2024-04-26T02:05:59.130 INFO:journalctl@ceph.osd.3.smithi001.stdout:Apr 26 02:05:58 smithi001 systemd[1]: Stopping Ceph osd.3 for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:05:59.630 INFO:journalctl@ceph.osd.3.smithi001.stdout:Apr 26 02:05:59 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-3[47007]: 2024-04-26T02:05:59.337+0000 7fd218600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:05:59.630 INFO:journalctl@ceph.osd.3.smithi001.stdout:Apr 26 02:05:59 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-3[47007]: 2024-04-26T02:05:59.337+0000 7fd218600700 -1 osd.3 112 *** Got signal Terminated *** 2024-04-26T02:05:59.630 INFO:journalctl@ceph.osd.3.smithi001.stdout:Apr 26 02:05:59 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-3[47007]: 2024-04-26T02:05:59.337+0000 7fd218600700 -1 osd.3 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:06:00.380 INFO:journalctl@ceph.osd.3.smithi001.stdout:Apr 26 02:06:00 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-3[47007]: 2024-04-26T02:06:00.110+0000 7fd213e00700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.535348+0000 front 2024-04-26T02:05:35.535878+0000 (oldest deadline 2024-04-26T02:05:59.635262+0000) 2024-04-26T02:06:00.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:00 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:00.139+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:01.130 INFO:journalctl@ceph.osd.3.smithi001.stdout:Apr 26 02:06:01 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-3[47007]: 2024-04-26T02:06:01.095+0000 7fd213e00700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.535348+0000 front 2024-04-26T02:05:35.535878+0000 (oldest deadline 2024-04-26T02:05:59.635262+0000) 2024-04-26T02:06:01.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:01 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:01.167+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:02.067 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:01 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:01.787+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.772086+0000 front 2024-04-26T02:05:35.772298+0000 (oldest deadline 2024-04-26T02:06:01.671962+0000) 2024-04-26T02:06:02.380 INFO:journalctl@ceph.osd.3.smithi001.stdout:Apr 26 02:06:02 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-3[47007]: 2024-04-26T02:06:02.122+0000 7fd213e00700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.535348+0000 front 2024-04-26T02:05:35.535878+0000 (oldest deadline 2024-04-26T02:05:59.635262+0000) 2024-04-26T02:06:02.380 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:02 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:02.291+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:02.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:02 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:02.187+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:03.067 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:02 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:02.779+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.772086+0000 front 2024-04-26T02:05:35.772298+0000 (oldest deadline 2024-04-26T02:06:01.671962+0000) 2024-04-26T02:06:03.074 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:02 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:02.809+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:03.380 INFO:journalctl@ceph.osd.3.smithi001.stdout:Apr 26 02:06:03 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-3[47007]: 2024-04-26T02:06:03.073+0000 7fd213e00700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.535348+0000 front 2024-04-26T02:05:35.535878+0000 (oldest deadline 2024-04-26T02:05:59.635262+0000) 2024-04-26T02:06:03.381 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:03 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:03.294+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:03.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:03 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:03.193+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:04.067 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:03 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:03.762+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.772086+0000 front 2024-04-26T02:05:35.772298+0000 (oldest deadline 2024-04-26T02:06:01.671962+0000) 2024-04-26T02:06:04.103 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:03 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:03.775+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:04.380 INFO:journalctl@ceph.osd.3.smithi001.stdout:Apr 26 02:06:04 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-3[47007]: 2024-04-26T02:06:04.104+0000 7fd213e00700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.535348+0000 front 2024-04-26T02:05:35.535878+0000 (oldest deadline 2024-04-26T02:05:59.635262+0000) 2024-04-26T02:06:04.380 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:04 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:04.273+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:04.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:04.172+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:05.067 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:04.802+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.772086+0000 front 2024-04-26T02:05:35.772298+0000 (oldest deadline 2024-04-26T02:06:01.671962+0000) 2024-04-26T02:06:05.076 INFO:journalctl@ceph.osd.3.smithi001.stdout:Apr 26 02:06:04 smithi001 podman[119181]: 2024-04-26 02:06:04.780538461 +0000 UTC m=+5.574640430 container died fca45df65be1c749762e78b0be5a5ab269700490a543f611af5ccfe39e635c91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-3, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-26T02:06:05.076 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:04 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:04.769+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:05.380 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:05 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:05.290+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:05.380 INFO:journalctl@ceph.osd.3.smithi001.stdout:Apr 26 02:06:05 smithi001 podman[119181]: 2024-04-26 02:06:05.288466746 +0000 UTC m=+6.082568705 container cleanup fca45df65be1c749762e78b0be5a5ab269700490a543f611af5ccfe39e635c91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:06:05.380 INFO:journalctl@ceph.osd.3.smithi001.stdout:Apr 26 02:06:05 smithi001 bash[119181]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-3 2024-04-26T02:06:05.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:05 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:05.155+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:05.767 INFO:journalctl@ceph.osd.3.smithi001.stdout:Apr 26 02:06:05 smithi001 podman[119211]: 2024-04-26 02:06:05.598277014 +0000 UTC m=+0.793928932 container remove fca45df65be1c749762e78b0be5a5ab269700490a543f611af5ccfe39e635c91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T02:06:06.067 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:05 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:05.766+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.772086+0000 front 2024-04-26T02:05:35.772298+0000 (oldest deadline 2024-04-26T02:06:01.671962+0000) 2024-04-26T02:06:06.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:05 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:05.767+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:06.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:06 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:06.185+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:06.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:06 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:06.314+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:06.630 INFO:journalctl@ceph.osd.3.smithi001.stdout:Apr 26 02:06:06 smithi001 podman[119297]: 2024-04-26 02:06:06.369247319 +0000 UTC m=+0.039475315 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:06:06.630 INFO:journalctl@ceph.osd.3.smithi001.stdout:Apr 26 02:06:06 smithi001 podman[119297]: 2024-04-26 02:06:06.491127059 +0000 UTC m=+0.161355079 container create cdd6b0016d14f85ac06e1228a58d6ef0eb29ec79e400bbebf86efe91e5aac45b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-26T02:06:07.067 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:06 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:06.791+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.772086+0000 front 2024-04-26T02:05:35.772298+0000 (oldest deadline 2024-04-26T02:06:01.671962+0000) 2024-04-26T02:06:07.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:06 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:06.783+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:07.130 INFO:journalctl@ceph.osd.3.smithi001.stdout:Apr 26 02:06:06 smithi001 podman[119297]: 2024-04-26 02:06:06.79519626 +0000 UTC m=+0.465424286 container init cdd6b0016d14f85ac06e1228a58d6ef0eb29ec79e400bbebf86efe91e5aac45b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-26T02:06:07.131 INFO:journalctl@ceph.osd.3.smithi001.stdout:Apr 26 02:06:06 smithi001 podman[119297]: 2024-04-26 02:06:06.806651409 +0000 UTC m=+0.476879435 container start cdd6b0016d14f85ac06e1228a58d6ef0eb29ec79e400bbebf86efe91e5aac45b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T02:06:07.131 INFO:journalctl@ceph.osd.3.smithi001.stdout:Apr 26 02:06:06 smithi001 podman[119297]: 2024-04-26 02:06:06.919017514 +0000 UTC m=+0.589245588 container attach cdd6b0016d14f85ac06e1228a58d6ef0eb29ec79e400bbebf86efe91e5aac45b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:06:07.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:07 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:07.227+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:07.630 INFO:journalctl@ceph.osd.3.smithi001.stdout:Apr 26 02:06:07 smithi001 podman[119297]: 2024-04-26 02:06:07.291698732 +0000 UTC m=+0.961926786 container died cdd6b0016d14f85ac06e1228a58d6ef0eb29ec79e400bbebf86efe91e5aac45b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T02:06:07.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:07 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:07.342+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:07.955 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:07 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:07.830+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:08.001 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.3.service' 2024-04-26T02:06:08.119 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:06:08.119 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-26T02:06:08.119 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-26T02:06:08.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.4 2024-04-26T02:06:08.155 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:07 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:07.824+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.772086+0000 front 2024-04-26T02:05:35.772298+0000 (oldest deadline 2024-04-26T02:06:01.671962+0000) 2024-04-26T02:06:08.429 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 26 02:06:08 smithi097 systemd[1]: Stopping Ceph osd.4 for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:06:08.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:08 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:08.203+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:08.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:08 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:08.309+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:08.734 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 26 02:06:08 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-4[43984]: 2024-04-26T02:06:08.430+0000 7fe935e35700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:06:08.734 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 26 02:06:08 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-4[43984]: 2024-04-26T02:06:08.430+0000 7fe935e35700 -1 osd.4 112 *** Got signal Terminated *** 2024-04-26T02:06:08.734 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 26 02:06:08 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-4[43984]: 2024-04-26T02:06:08.430+0000 7fe935e35700 -1 osd.4 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:06:09.067 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:08 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:08.820+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.772086+0000 front 2024-04-26T02:05:35.772298+0000 (oldest deadline 2024-04-26T02:06:01.671962+0000) 2024-04-26T02:06:09.068 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:08 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:08.769+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:09.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:08 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:08.808+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:09.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:09 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:09.186+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:09.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:09 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:09.352+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:09.984 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:09 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:09.587+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:10.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:09 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:09.754+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:10.067 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:09 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:09.820+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.772086+0000 front 2024-04-26T02:05:35.772298+0000 (oldest deadline 2024-04-26T02:06:01.671962+0000) 2024-04-26T02:06:10.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:09 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:09.763+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:10.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:10 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:10.118+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:10.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:10 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:10.206+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:10.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:10 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:10.331+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:10.984 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:10 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:10.546+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:11.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:10 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:10.756+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:11.068 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:10 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:10.827+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.772086+0000 front 2024-04-26T02:05:35.772298+0000 (oldest deadline 2024-04-26T02:06:01.671962+0000) 2024-04-26T02:06:11.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:10 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:10.740+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:11.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:11 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:11.147+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:11.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:11 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:11.256+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:11.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:11 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:11.256+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:11.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:11 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:11.339+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:11.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:11 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:11.339+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:44.664035+0000 front 2024-04-26T02:05:44.664095+0000 (oldest deadline 2024-04-26T02:06:10.465154+0000) 2024-04-26T02:06:11.984 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:11 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:11.557+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:12.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:11 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:11.737+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:12.068 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:11 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:11.819+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.772086+0000 front 2024-04-26T02:05:35.772298+0000 (oldest deadline 2024-04-26T02:06:01.671962+0000) 2024-04-26T02:06:12.068 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:11 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:11.819+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:48.073880+0000 front 2024-04-26T02:05:48.073930+0000 (oldest deadline 2024-04-26T02:06:11.573956+0000) 2024-04-26T02:06:12.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:11 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:11.771+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:12.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:11 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:11.771+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:12.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:12 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:12.170+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:12.569 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:12 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:12.234+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:12.570 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:12 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:12.234+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:12.770 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:12 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:12.387+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:12.770 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:12 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:12.387+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:44.664035+0000 front 2024-04-26T02:05:44.664095+0000 (oldest deadline 2024-04-26T02:06:10.465154+0000) 2024-04-26T02:06:12.984 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:12 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:12.526+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:13.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:12 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:12.693+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:13.068 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:12 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:12.843+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.772086+0000 front 2024-04-26T02:05:35.772298+0000 (oldest deadline 2024-04-26T02:06:01.671962+0000) 2024-04-26T02:06:13.068 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:12 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:12.843+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:48.073880+0000 front 2024-04-26T02:05:48.073930+0000 (oldest deadline 2024-04-26T02:06:11.573956+0000) 2024-04-26T02:06:13.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:12 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:12.771+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:13.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:12 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:12.771+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:13.484 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 26 02:06:13 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-4[43984]: 2024-04-26T02:06:13.003+0000 7fe932429700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.239612+0000 front 2024-04-26T02:05:47.239589+0000 (oldest deadline 2024-04-26T02:06:12.539786+0000) 2024-04-26T02:06:13.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:13 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:13.138+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:13.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:13 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:13.192+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:13.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:13 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:13.192+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:13.740 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:13 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:13.423+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:13.740 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:13 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:13.423+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:44.664035+0000 front 2024-04-26T02:05:44.664095+0000 (oldest deadline 2024-04-26T02:06:10.465154+0000) 2024-04-26T02:06:13.973 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:13 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:13.501+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:14.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:13 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:13.726+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:14.068 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:13 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:13.808+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.772086+0000 front 2024-04-26T02:05:35.772298+0000 (oldest deadline 2024-04-26T02:06:01.671962+0000) 2024-04-26T02:06:14.068 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:13 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:13.808+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:48.073880+0000 front 2024-04-26T02:05:48.073930+0000 (oldest deadline 2024-04-26T02:06:11.573956+0000) 2024-04-26T02:06:14.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:13 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:13.741+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:14.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:13 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:13.741+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:14.234 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:14 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:14.184+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:14.234 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 26 02:06:13 smithi097 podman[69433]: 2024-04-26 02:06:13.974723349 +0000 UTC m=+5.659217084 container died 2a40c1f1c7679213952912e8926df6c5fe83fa04048286e33271f6c5781fe265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-26T02:06:14.532 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 26 02:06:14 smithi097 podman[69433]: 2024-04-26 02:06:14.347145202 +0000 UTC m=+6.031638939 container cleanup 2a40c1f1c7679213952912e8926df6c5fe83fa04048286e33271f6c5781fe265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T02:06:14.532 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 26 02:06:14 smithi097 bash[69433]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-4 2024-04-26T02:06:14.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:14 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:14.236+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:14.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:14 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:14.236+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:14.751 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:14 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:14.395+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:14.751 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:14 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:14.395+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:44.664035+0000 front 2024-04-26T02:05:44.664095+0000 (oldest deadline 2024-04-26T02:06:10.465154+0000) 2024-04-26T02:06:14.900 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:14 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:14.533+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:14.900 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 26 02:06:14 smithi097 podman[69449]: 2024-04-26 02:06:14.589934342 +0000 UTC m=+0.611026468 container remove 2a40c1f1c7679213952912e8926df6c5fe83fa04048286e33271f6c5781fe265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:06:15.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:14 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:14.682+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:15.068 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:14 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:14.834+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.772086+0000 front 2024-04-26T02:05:35.772298+0000 (oldest deadline 2024-04-26T02:06:01.671962+0000) 2024-04-26T02:06:15.068 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:14 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:14.834+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:48.073880+0000 front 2024-04-26T02:05:48.073930+0000 (oldest deadline 2024-04-26T02:06:11.573956+0000) 2024-04-26T02:06:15.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:14 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:14.752+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:15.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:14 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:14.752+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:15.161 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 26 02:06:14 smithi097 podman[69542]: 2024-04-26 02:06:14.80554807 +0000 UTC m=+0.018394408 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:06:15.161 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 26 02:06:14 smithi097 podman[69542]: 2024-04-26 02:06:14.965900908 +0000 UTC m=+0.178747236 container create 53256d1f54dffd4eebb8ec7befb218f6449f2a30efa5762df6757e240607e5d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-26T02:06:15.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:15 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:15.162+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:15.485 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 26 02:06:15 smithi097 podman[69542]: 2024-04-26 02:06:15.20152438 +0000 UTC m=+0.414370709 container init 53256d1f54dffd4eebb8ec7befb218f6449f2a30efa5762df6757e240607e5d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-04-26T02:06:15.485 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 26 02:06:15 smithi097 podman[69542]: 2024-04-26 02:06:15.204578972 +0000 UTC m=+0.417425302 container start 53256d1f54dffd4eebb8ec7befb218f6449f2a30efa5762df6757e240607e5d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:06:15.485 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 26 02:06:15 smithi097 podman[69542]: 2024-04-26 02:06:15.28489366 +0000 UTC m=+0.497739998 container attach 53256d1f54dffd4eebb8ec7befb218f6449f2a30efa5762df6757e240607e5d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:06:15.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:15 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:15.237+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:15.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:15 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:15.237+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:15.780 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:15 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:15.399+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:15.780 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:15 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:15.399+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:44.664035+0000 front 2024-04-26T02:05:44.664095+0000 (oldest deadline 2024-04-26T02:06:10.465154+0000) 2024-04-26T02:06:15.786 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:15 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:15.553+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:15.786 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 26 02:06:15 smithi097 podman[69542]: 2024-04-26 02:06:15.496474721 +0000 UTC m=+0.709321051 container died 53256d1f54dffd4eebb8ec7befb218f6449f2a30efa5762df6757e240607e5d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-26T02:06:16.068 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:15 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:15.689+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:16.068 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:15 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:15.865+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.772086+0000 front 2024-04-26T02:05:35.772298+0000 (oldest deadline 2024-04-26T02:06:01.671962+0000) 2024-04-26T02:06:16.068 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:15 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:15.865+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:48.073880+0000 front 2024-04-26T02:05:48.073930+0000 (oldest deadline 2024-04-26T02:06:11.573956+0000) 2024-04-26T02:06:16.092 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.4.service' 2024-04-26T02:06:16.129 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:06:16.129 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-26T02:06:16.130 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-26T02:06:16.130 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.5 2024-04-26T02:06:16.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:15 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:15.781+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:16.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:15 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:15.781+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:16.440 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:16 smithi173 systemd[1]: Stopping Ceph osd.5 for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:06:16.440 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:16 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:16.203+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:16.440 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:16 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:16.203+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:16.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:16 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:16.164+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:16.765 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:16 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:16.394+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:16.765 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:16 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:16.394+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:44.664035+0000 front 2024-04-26T02:05:44.664095+0000 (oldest deadline 2024-04-26T02:06:10.465154+0000) 2024-04-26T02:06:16.817 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:16 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:16.641+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:16.817 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:16 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:16.440+0000 7ff3c6904700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:06:16.818 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:16 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:16.440+0000 7ff3c6904700 -1 osd.5 112 *** Got signal Terminated *** 2024-04-26T02:06:16.818 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:16 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:16.440+0000 7ff3c6904700 -1 osd.5 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:06:16.984 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:16 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:16.534+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:17.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:16 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:16.766+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:17.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:16 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:16.766+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:17.212 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:16 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:16.900+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.772086+0000 front 2024-04-26T02:05:35.772298+0000 (oldest deadline 2024-04-26T02:06:01.671962+0000) 2024-04-26T02:06:17.212 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:16 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:16.900+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:48.073880+0000 front 2024-04-26T02:05:48.073930+0000 (oldest deadline 2024-04-26T02:06:11.573956+0000) 2024-04-26T02:06:17.212 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:16 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:16.900+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.575153+0000 front 2024-04-26T02:05:55.575177+0000 (oldest deadline 2024-04-26T02:06:16.675367+0000) 2024-04-26T02:06:17.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:17 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:17.137+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:17.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:17 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:17.213+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:17.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:17 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:17.213+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:17.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:17 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:17.213+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:17.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:17 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:17.376+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:17.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:17 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:17.376+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:44.664035+0000 front 2024-04-26T02:05:44.664095+0000 (oldest deadline 2024-04-26T02:06:10.465154+0000) 2024-04-26T02:06:17.876 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:17 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:17.597+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:17.984 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:17 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:17.537+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:18.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:17 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:17.770+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:18.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:17 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:17.770+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:18.173 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:17 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:17.878+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.772086+0000 front 2024-04-26T02:05:35.772298+0000 (oldest deadline 2024-04-26T02:06:01.671962+0000) 2024-04-26T02:06:18.173 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:17 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:17.878+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:48.073880+0000 front 2024-04-26T02:05:48.073930+0000 (oldest deadline 2024-04-26T02:06:11.573956+0000) 2024-04-26T02:06:18.173 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:17 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:17.878+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.575153+0000 front 2024-04-26T02:05:55.575177+0000 (oldest deadline 2024-04-26T02:06:16.675367+0000) 2024-04-26T02:06:18.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:18 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:18.112+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:18.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:18 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:18.174+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:18.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:18 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:18.174+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:18.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:18 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:18.174+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:18.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:18 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:18.378+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:18.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:18 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:18.378+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:44.664035+0000 front 2024-04-26T02:05:44.664095+0000 (oldest deadline 2024-04-26T02:06:10.465154+0000) 2024-04-26T02:06:18.631 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:18 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:18.378+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.967195+0000 front 2024-04-26T02:05:53.967261+0000 (oldest deadline 2024-04-26T02:06:18.067436+0000) 2024-04-26T02:06:18.755 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:18 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:18.571+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:18.916 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:18 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:18.576+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:19.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:18 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:18.725+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:19.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:18 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:18.725+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:19.204 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:18 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:18.916+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.772086+0000 front 2024-04-26T02:05:35.772298+0000 (oldest deadline 2024-04-26T02:06:01.671962+0000) 2024-04-26T02:06:19.204 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:18 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:18.916+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:48.073880+0000 front 2024-04-26T02:05:48.073930+0000 (oldest deadline 2024-04-26T02:06:11.573956+0000) 2024-04-26T02:06:19.204 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:18 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:18.916+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.575153+0000 front 2024-04-26T02:05:55.575177+0000 (oldest deadline 2024-04-26T02:06:16.675367+0000) 2024-04-26T02:06:19.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:19 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:19.087+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:19.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:19 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:19.087+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:19.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:19 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:19.204+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:19.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:19 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:19.204+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:19.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:19 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:19.204+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:19.746 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:19 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:19.407+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:19.746 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:19 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:19.408+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:44.664035+0000 front 2024-04-26T02:05:44.664095+0000 (oldest deadline 2024-04-26T02:06:10.465154+0000) 2024-04-26T02:06:19.746 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:19 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:19.408+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.967195+0000 front 2024-04-26T02:05:53.967261+0000 (oldest deadline 2024-04-26T02:06:18.067436+0000) 2024-04-26T02:06:19.898 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:19 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:19.595+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:19.984 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:19 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:19.618+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:19.984 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:19 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:19.618+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.460304+0000 front 2024-04-26T02:05:53.460242+0000 (oldest deadline 2024-04-26T02:06:19.360240+0000) 2024-04-26T02:06:20.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:19 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:19.746+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:20.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:19 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:19.746+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:20.214 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:19 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:19.899+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.772086+0000 front 2024-04-26T02:05:35.772298+0000 (oldest deadline 2024-04-26T02:06:01.671962+0000) 2024-04-26T02:06:20.214 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:19 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:19.899+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:48.073880+0000 front 2024-04-26T02:05:48.073930+0000 (oldest deadline 2024-04-26T02:06:11.573956+0000) 2024-04-26T02:06:20.214 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:19 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:19.899+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.575153+0000 front 2024-04-26T02:05:55.575177+0000 (oldest deadline 2024-04-26T02:06:16.675367+0000) 2024-04-26T02:06:20.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:20 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:20.115+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:20.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:20 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:20.115+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:20.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:20 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:20.215+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:20.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:20 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:20.215+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:20.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:20 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:20.215+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:20.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:20 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:20.362+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:20.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:20 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:20.362+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:44.664035+0000 front 2024-04-26T02:05:44.664095+0000 (oldest deadline 2024-04-26T02:06:10.465154+0000) 2024-04-26T02:06:20.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:20 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:20.362+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.967195+0000 front 2024-04-26T02:05:53.967261+0000 (oldest deadline 2024-04-26T02:06:18.067436+0000) 2024-04-26T02:06:20.817 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:20 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:20.568+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:20.984 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:20 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:20.655+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:20.984 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:20 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:20.655+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.460304+0000 front 2024-04-26T02:05:53.460242+0000 (oldest deadline 2024-04-26T02:06:19.360240+0000) 2024-04-26T02:06:21.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:20 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:20.736+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:21.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:20 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:20.736+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:21.242 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:20 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:20.929+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.772086+0000 front 2024-04-26T02:05:35.772298+0000 (oldest deadline 2024-04-26T02:06:01.671962+0000) 2024-04-26T02:06:21.242 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:20 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:20.929+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:48.073880+0000 front 2024-04-26T02:05:48.073930+0000 (oldest deadline 2024-04-26T02:06:11.573956+0000) 2024-04-26T02:06:21.242 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:20 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5[44202]: 2024-04-26T02:06:20.929+0000 7ff3beef0700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.575153+0000 front 2024-04-26T02:05:55.575177+0000 (oldest deadline 2024-04-26T02:06:16.675367+0000) 2024-04-26T02:06:21.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:21 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:21.156+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:21.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:21 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:21.156+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:21.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:21 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:21.243+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:21.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:21 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:21.243+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:21.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:21 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:21.243+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:21.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:21 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:21.342+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:21.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:21 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:21.342+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:44.664035+0000 front 2024-04-26T02:05:44.664095+0000 (oldest deadline 2024-04-26T02:06:10.465154+0000) 2024-04-26T02:06:21.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:21 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:21.342+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.967195+0000 front 2024-04-26T02:05:53.967261+0000 (oldest deadline 2024-04-26T02:06:18.067436+0000) 2024-04-26T02:06:21.972 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:21 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:21.596+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:21.984 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:21 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:21.696+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:21.984 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:21 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:21.696+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.460304+0000 front 2024-04-26T02:05:53.460242+0000 (oldest deadline 2024-04-26T02:06:19.360240+0000) 2024-04-26T02:06:22.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:21 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:21.726+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:22.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:21 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:21.726+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:22.225 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:21 smithi173 podman[71004]: 2024-04-26 02:06:21.97417086 +0000 UTC m=+5.651927962 container died b45ebcdd3f7fcf5b5771a456d6f2182f2ce5a0631abf6b79395f0e2db5155e57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-26T02:06:22.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:22 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:22.127+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:22.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:22 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:22.127+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:22.567 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:22 smithi173 podman[71004]: 2024-04-26 02:06:22.325283496 +0000 UTC m=+6.003040585 container cleanup b45ebcdd3f7fcf5b5771a456d6f2182f2ce5a0631abf6b79395f0e2db5155e57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T02:06:22.567 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:22 smithi173 bash[71004]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5 2024-04-26T02:06:22.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:22 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:22.245+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:22.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:22 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:22.245+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:22.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:22 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:22.245+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:22.678 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:22 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:22.391+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:22.679 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:22 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:22.391+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:44.664035+0000 front 2024-04-26T02:05:44.664095+0000 (oldest deadline 2024-04-26T02:06:10.465154+0000) 2024-04-26T02:06:22.679 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:22 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:22.391+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.967195+0000 front 2024-04-26T02:05:53.967261+0000 (oldest deadline 2024-04-26T02:06:18.067436+0000) 2024-04-26T02:06:22.897 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:22 smithi173 podman[71021]: 2024-04-26 02:06:22.585426285 +0000 UTC m=+0.612796323 container remove b45ebcdd3f7fcf5b5771a456d6f2182f2ce5a0631abf6b79395f0e2db5155e57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-04-26T02:06:22.898 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:22 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:22.606+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:22.898 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:22 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:22.606+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:23.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:22 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:22.679+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:23.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:22 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:22.679+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:23.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:22 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:22.679+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:23.171 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:22 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:22.736+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:23.171 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:22 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:22.736+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.460304+0000 front 2024-04-26T02:05:53.460242+0000 (oldest deadline 2024-04-26T02:06:19.360240+0000) 2024-04-26T02:06:23.187 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:22 smithi173 podman[71112]: 2024-04-26 02:06:22.802167675 +0000 UTC m=+0.017760535 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:06:23.187 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:22 smithi173 podman[71112]: 2024-04-26 02:06:22.903067997 +0000 UTC m=+0.118660844 container create afa444ab4856ce3014ffdd3df425e46dd6b824349f8f2e16d8ad0d4855887346 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:06:23.187 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:23 smithi173 podman[71112]: 2024-04-26 02:06:23.105868245 +0000 UTC m=+0.321461092 container init afa444ab4856ce3014ffdd3df425e46dd6b824349f8f2e16d8ad0d4855887346 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:06:23.187 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:23 smithi173 podman[71112]: 2024-04-26 02:06:23.108832995 +0000 UTC m=+0.324425843 container start afa444ab4856ce3014ffdd3df425e46dd6b824349f8f2e16d8ad0d4855887346 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-26T02:06:23.187 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:23 smithi173 podman[71112]: 2024-04-26 02:06:23.189080949 +0000 UTC m=+0.404673824 container attach afa444ab4856ce3014ffdd3df425e46dd6b824349f8f2e16d8ad0d4855887346 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-04-26T02:06:23.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:23 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:23.173+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:23.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:23 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:23.173+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:23.567 INFO:journalctl@ceph.osd.5.smithi173.stdout:Apr 26 02:06:23 smithi173 podman[71112]: 2024-04-26 02:06:23.403675511 +0000 UTC m=+0.619268366 container died afa444ab4856ce3014ffdd3df425e46dd6b824349f8f2e16d8ad0d4855887346 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-26T02:06:23.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:23 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:23.237+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:23.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:23 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:23.237+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:23.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:23 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:23.237+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:23.692 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:23 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:23.404+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:23.692 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:23 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:23.404+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:44.664035+0000 front 2024-04-26T02:05:44.664095+0000 (oldest deadline 2024-04-26T02:06:10.465154+0000) 2024-04-26T02:06:23.692 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:23 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:23.404+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.967195+0000 front 2024-04-26T02:05:53.967261+0000 (oldest deadline 2024-04-26T02:06:18.067436+0000) 2024-04-26T02:06:23.949 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:23 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:23.587+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:23.950 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:23 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:23.587+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:23.968 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.5.service' 2024-04-26T02:06:24.004 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:06:24.004 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-26T02:06:24.004 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-26T02:06:24.005 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.6 2024-04-26T02:06:24.089 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:23 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:23.692+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:24.089 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:23 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:23.692+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:24.089 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:23 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:23.692+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:24.200 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:23 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:23.772+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:24.200 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:23 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:23.772+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.460304+0000 front 2024-04-26T02:05:53.460242+0000 (oldest deadline 2024-04-26T02:06:19.360240+0000) 2024-04-26T02:06:24.380 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:24 smithi001 systemd[1]: Stopping Ceph osd.6 for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:06:24.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:24 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:24.202+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:24.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:24 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:24.202+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:24.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:24 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:24.248+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:24.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:24 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:24.248+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:24.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:24 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:24.248+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:24.881 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:24 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:24.418+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:24.881 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:24 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:24.418+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:44.664035+0000 front 2024-04-26T02:05:44.664095+0000 (oldest deadline 2024-04-26T02:06:10.465154+0000) 2024-04-26T02:06:24.881 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:24 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:24.418+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.967195+0000 front 2024-04-26T02:05:53.967261+0000 (oldest deadline 2024-04-26T02:06:18.067436+0000) 2024-04-26T02:06:24.881 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:24 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:24.614+0000 7efed9200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:06:24.881 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:24 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:24.614+0000 7efed9200700 -1 osd.6 112 *** Got signal Terminated *** 2024-04-26T02:06:24.881 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:24 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:24.614+0000 7efed9200700 -1 osd.6 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:06:24.881 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:24 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:24.656+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:24.881 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:24 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:24.656+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:24.881 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:24 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:24.656+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:25.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:24 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:24.635+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:25.068 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:24 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:24.635+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:25.228 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:24 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:24.816+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:25.228 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:24 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:24.816+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.460304+0000 front 2024-04-26T02:05:53.460242+0000 (oldest deadline 2024-04-26T02:06:19.360240+0000) 2024-04-26T02:06:25.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:25 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:25.230+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:25.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:25 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:25.230+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:25.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:25 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:25.230+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:25.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:25 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:25.286+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:25.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:25 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:25.286+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:25.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:25 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:25.286+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:25.880 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:25 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:25.453+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:25.880 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:25 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:25.453+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:44.664035+0000 front 2024-04-26T02:05:44.664095+0000 (oldest deadline 2024-04-26T02:06:10.465154+0000) 2024-04-26T02:06:25.880 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:25 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:25.453+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.967195+0000 front 2024-04-26T02:05:53.967261+0000 (oldest deadline 2024-04-26T02:06:18.067436+0000) 2024-04-26T02:06:25.881 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:25 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:25.640+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:25.881 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:25 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:25.640+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:25.881 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:25 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:25.640+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:26.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:25 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:25.605+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:26.068 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:25 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:25.605+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:26.206 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:25 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:25.848+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:26.206 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:25 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:25.848+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.460304+0000 front 2024-04-26T02:05:53.460242+0000 (oldest deadline 2024-04-26T02:06:19.360240+0000) 2024-04-26T02:06:26.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:26 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:26.207+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:26.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:26 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:26.207+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:26.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:26 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:26.207+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:26.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:26 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:26.270+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:26.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:26 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:26.270+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:26.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:26 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:26.270+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:26.880 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:26 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:26.480+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:26.880 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:26 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:26.480+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:44.664035+0000 front 2024-04-26T02:05:44.664095+0000 (oldest deadline 2024-04-26T02:06:10.465154+0000) 2024-04-26T02:06:26.881 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:26 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:26.480+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.967195+0000 front 2024-04-26T02:05:53.967261+0000 (oldest deadline 2024-04-26T02:06:18.067436+0000) 2024-04-26T02:06:26.881 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:26 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:26.480+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:00.369764+0000 front 2024-04-26T02:06:00.369614+0000 (oldest deadline 2024-04-26T02:06:26.269784+0000) 2024-04-26T02:06:26.881 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:26 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:26.637+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:26.881 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:26 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:26.637+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:26.881 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:26 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:26.637+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:27.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:26 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:26.589+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:27.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:26 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:26.589+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:27.217 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:26 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:26.816+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:27.217 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:26 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:26.816+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.460304+0000 front 2024-04-26T02:05:53.460242+0000 (oldest deadline 2024-04-26T02:06:19.360240+0000) 2024-04-26T02:06:27.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:27 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:27.219+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:27.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:27 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:27.219+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:27.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:27 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:27.219+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:27.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:27 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:27.267+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:27.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:27 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:27.267+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:27.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:27 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:27.267+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:27.880 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:27 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:27.509+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:27.880 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:27 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:27.509+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:44.664035+0000 front 2024-04-26T02:05:44.664095+0000 (oldest deadline 2024-04-26T02:06:10.465154+0000) 2024-04-26T02:06:27.880 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:27 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:27.509+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.967195+0000 front 2024-04-26T02:05:53.967261+0000 (oldest deadline 2024-04-26T02:06:18.067436+0000) 2024-04-26T02:06:27.881 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:27 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:27.509+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:00.369764+0000 front 2024-04-26T02:06:00.369614+0000 (oldest deadline 2024-04-26T02:06:26.269784+0000) 2024-04-26T02:06:27.881 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:27 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:27.647+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:27.881 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:27 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:27.647+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:27.881 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:27 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:27.647+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:27.881 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:27 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:27.647+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:28.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:27 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:27.574+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:28.068 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:27 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:27.574+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:28.068 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:27 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:27.574+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:03.318369+0000 front 2024-04-26T02:06:03.318444+0000 (oldest deadline 2024-04-26T02:06:27.418372+0000) 2024-04-26T02:06:28.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:27 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:27.793+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:28.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:27 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:27.793+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.460304+0000 front 2024-04-26T02:05:53.460242+0000 (oldest deadline 2024-04-26T02:06:19.360240+0000) 2024-04-26T02:06:28.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:27 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:27.793+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:05:59.361120+0000 front 2024-04-26T02:05:59.360973+0000 (oldest deadline 2024-04-26T02:06:27.561366+0000) 2024-04-26T02:06:28.550 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:28 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:28.219+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:28.550 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:28 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:28.219+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:28.550 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:28 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:28.219+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:28.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:28 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:28.265+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:28.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:28 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:28.265+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:28.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:28 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:28.265+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:28.817 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:28 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:28.551+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:28.817 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:28 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:28.551+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:28.818 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:28 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:28.551+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:03.318369+0000 front 2024-04-26T02:06:03.318444+0000 (oldest deadline 2024-04-26T02:06:27.418372+0000) 2024-04-26T02:06:28.880 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:28 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:28.488+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:28.881 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:28 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:28.488+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:44.664035+0000 front 2024-04-26T02:05:44.664095+0000 (oldest deadline 2024-04-26T02:06:10.465154+0000) 2024-04-26T02:06:28.881 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:28 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:28.488+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.967195+0000 front 2024-04-26T02:05:53.967261+0000 (oldest deadline 2024-04-26T02:06:18.067436+0000) 2024-04-26T02:06:28.881 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:28 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:28.488+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:00.369764+0000 front 2024-04-26T02:06:00.369614+0000 (oldest deadline 2024-04-26T02:06:26.269784+0000) 2024-04-26T02:06:28.881 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:28 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:28.697+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:28.881 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:28 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:28.697+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:28.881 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:28 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:28.697+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:28.881 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:28 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:28.697+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:29.185 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:28 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:28.813+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:29.186 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:28 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:28.813+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.460304+0000 front 2024-04-26T02:05:53.460242+0000 (oldest deadline 2024-04-26T02:06:19.360240+0000) 2024-04-26T02:06:29.186 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:28 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:28.813+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:05:59.361120+0000 front 2024-04-26T02:05:59.360973+0000 (oldest deadline 2024-04-26T02:06:27.561366+0000) 2024-04-26T02:06:29.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:29 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:29.283+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:29.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:29 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:29.283+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:29.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:29 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:29.283+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:29.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:29 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:29.237+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:29.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:29 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:29.237+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:29.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:29 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:29.237+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:29.785 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:29 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:29.535+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.461393+0000 front 2024-04-26T02:05:36.461713+0000 (oldest deadline 2024-04-26T02:06:01.761623+0000) 2024-04-26T02:06:29.785 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:29 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:29.535+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:44.664035+0000 front 2024-04-26T02:05:44.664095+0000 (oldest deadline 2024-04-26T02:06:10.465154+0000) 2024-04-26T02:06:29.785 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:29 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:29.535+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.967195+0000 front 2024-04-26T02:05:53.967261+0000 (oldest deadline 2024-04-26T02:06:18.067436+0000) 2024-04-26T02:06:29.785 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:29 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6[53691]: 2024-04-26T02:06:29.535+0000 7efed4a00700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:00.369764+0000 front 2024-04-26T02:06:00.369614+0000 (oldest deadline 2024-04-26T02:06:26.269784+0000) 2024-04-26T02:06:29.785 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:29 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:29.682+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:29.786 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:29 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:29.682+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:29.786 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:29 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:29.682+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:29.786 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:29 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:29.682+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:29.817 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:29 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:29.568+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:29.817 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:29 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:29.568+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:29.817 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:29 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:29.568+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:03.318369+0000 front 2024-04-26T02:06:03.318444+0000 (oldest deadline 2024-04-26T02:06:27.418372+0000) 2024-04-26T02:06:30.130 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:30 smithi001 podman[119405]: 2024-04-26 02:06:30.027062372 +0000 UTC m=+5.551905427 container died b179dfef39c77822d80f66ef86b2db06f036692c158c1e4a1438e62288f7c9b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:06:30.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:29 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:29.807+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:30.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:29 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:29.807+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.460304+0000 front 2024-04-26T02:05:53.460242+0000 (oldest deadline 2024-04-26T02:06:19.360240+0000) 2024-04-26T02:06:30.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:29 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:29.807+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:05:59.361120+0000 front 2024-04-26T02:05:59.360973+0000 (oldest deadline 2024-04-26T02:06:27.561366+0000) 2024-04-26T02:06:30.547 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:30 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:30.236+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:30.548 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:30 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:30.236+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:30.548 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:30 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:30.236+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:30.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:30 smithi001 podman[119405]: 2024-04-26 02:06:30.516469981 +0000 UTC m=+6.041313073 container cleanup b179dfef39c77822d80f66ef86b2db06f036692c158c1e4a1438e62288f7c9b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:06:30.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:30 smithi001 bash[119405]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6 2024-04-26T02:06:30.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:30 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:30.269+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:30.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:30 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:30.269+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:30.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:30 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:30.269+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:30.817 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:30 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:30.549+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:30.818 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:30 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:30.549+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:30.818 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:30 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:30.549+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:03.318369+0000 front 2024-04-26T02:06:03.318444+0000 (oldest deadline 2024-04-26T02:06:27.418372+0000) 2024-04-26T02:06:31.130 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:30 smithi001 podman[119423]: 2024-04-26 02:06:30.859786898 +0000 UTC m=+0.823652521 container remove b179dfef39c77822d80f66ef86b2db06f036692c158c1e4a1438e62288f7c9b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:06:31.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:30 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:30.690+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:31.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:30 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:30.690+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:31.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:30 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:30.690+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:31.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:30 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:30.690+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:31.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:30 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:30.796+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:31.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:30 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:30.796+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.460304+0000 front 2024-04-26T02:05:53.460242+0000 (oldest deadline 2024-04-26T02:06:19.360240+0000) 2024-04-26T02:06:31.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:30 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:30.796+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:05:59.361120+0000 front 2024-04-26T02:05:59.360973+0000 (oldest deadline 2024-04-26T02:06:27.561366+0000) 2024-04-26T02:06:31.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:31 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:31.268+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:31.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:31 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:31.268+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:31.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:31 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:31.268+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:31.568 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:31 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:31.502+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:31.568 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:31 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:31.502+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:31.568 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:31 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:31.502+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:03.318369+0000 front 2024-04-26T02:06:03.318444+0000 (oldest deadline 2024-04-26T02:06:27.418372+0000) 2024-04-26T02:06:31.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:31 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:31.298+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:31.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:31 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:31.298+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:31.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:31 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:31.298+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:32.022 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:31 smithi001 podman[119513]: 2024-04-26 02:06:31.628907145 +0000 UTC m=+0.038027920 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:06:32.022 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:31 smithi001 podman[119513]: 2024-04-26 02:06:31.743901762 +0000 UTC m=+0.153022566 container create 1e71b218e6d67e4c8e785c93ae5dba3b094ff8f26a1105342f23efeae7c34d76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True) 2024-04-26T02:06:32.022 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:32 smithi001 podman[119513]: 2024-04-26 02:06:32.022833318 +0000 UTC m=+0.431954185 container init 1e71b218e6d67e4c8e785c93ae5dba3b094ff8f26a1105342f23efeae7c34d76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-04-26T02:06:32.023 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:31 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:31.707+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:32.023 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:31 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:31.707+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:32.023 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:31 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:31.707+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:32.023 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:31 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:31.707+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:32.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:31 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:31.822+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:32.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:31 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:31.822+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.460304+0000 front 2024-04-26T02:05:53.460242+0000 (oldest deadline 2024-04-26T02:06:19.360240+0000) 2024-04-26T02:06:32.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:31 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:31.822+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:05:59.361120+0000 front 2024-04-26T02:05:59.360973+0000 (oldest deadline 2024-04-26T02:06:27.561366+0000) 2024-04-26T02:06:32.380 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:32 smithi001 podman[119513]: 2024-04-26 02:06:32.034533045 +0000 UTC m=+0.443653882 container start 1e71b218e6d67e4c8e785c93ae5dba3b094ff8f26a1105342f23efeae7c34d76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-26T02:06:32.380 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:32 smithi001 podman[119513]: 2024-04-26 02:06:32.121129112 +0000 UTC m=+0.530249962 container attach 1e71b218e6d67e4c8e785c93ae5dba3b094ff8f26a1105342f23efeae7c34d76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-26T02:06:32.517 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:32 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:32.258+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:32.518 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:32 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:32.258+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:32.518 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:32 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:32.258+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:32.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:32 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:32.329+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:32.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:32 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:32.329+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:32.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:32 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:32.329+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:32.748 INFO:journalctl@ceph.osd.6.smithi001.stdout:Apr 26 02:06:32 smithi001 podman[119513]: 2024-04-26 02:06:32.498534281 +0000 UTC m=+0.907655052 container died 1e71b218e6d67e4c8e785c93ae5dba3b094ff8f26a1105342f23efeae7c34d76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-26T02:06:32.817 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:32 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:32.519+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:32.817 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:32 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:32.519+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:32.818 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:32 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:32.519+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:03.318369+0000 front 2024-04-26T02:06:03.318444+0000 (oldest deadline 2024-04-26T02:06:27.418372+0000) 2024-04-26T02:06:33.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:32 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:32.748+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:33.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:32 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:32.748+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:33.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:32 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:32.748+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:33.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:32 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:32.748+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:33.205 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.6.service' 2024-04-26T02:06:33.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:32 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:32.822+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:33.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:32 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:32.822+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.460304+0000 front 2024-04-26T02:05:53.460242+0000 (oldest deadline 2024-04-26T02:06:19.360240+0000) 2024-04-26T02:06:33.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:32 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:32.822+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:05:59.361120+0000 front 2024-04-26T02:05:59.360973+0000 (oldest deadline 2024-04-26T02:06:27.561366+0000) 2024-04-26T02:06:33.322 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:06:33.323 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-26T02:06:33.323 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-26T02:06:33.323 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.7 2024-04-26T02:06:33.547 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:33 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:33.256+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:33.547 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:33 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:33.256+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:33.547 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:33 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:33.256+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:33.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:33 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:33.311+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:33.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:33 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:33.311+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:33.673 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:33 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:33.311+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:33.673 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:33 smithi097 systemd[1]: Stopping Ceph osd.7 for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:06:33.673 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:33 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:33.673+0000 7f5f210ee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:06:33.673 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:33 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:33.673+0000 7f5f210ee700 -1 osd.7 112 *** Got signal Terminated *** 2024-04-26T02:06:33.673 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:33 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:33.673+0000 7f5f210ee700 -1 osd.7 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:06:33.817 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:33 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:33.548+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:33.817 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:33 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:33.548+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:33.818 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:33 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:33.548+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:03.318369+0000 front 2024-04-26T02:06:03.318444+0000 (oldest deadline 2024-04-26T02:06:27.418372+0000) 2024-04-26T02:06:33.984 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:33 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:33.856+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:33.984 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:33 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:33.856+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.460304+0000 front 2024-04-26T02:05:53.460242+0000 (oldest deadline 2024-04-26T02:06:19.360240+0000) 2024-04-26T02:06:33.984 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:33 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:33.856+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:05:59.361120+0000 front 2024-04-26T02:05:59.360973+0000 (oldest deadline 2024-04-26T02:06:27.561366+0000) 2024-04-26T02:06:34.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:33 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:33.786+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:34.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:33 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:33.786+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:34.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:33 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:33.786+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:34.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:33 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:33.786+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:34.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:34 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:34.217+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:34.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:34 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:34.217+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:34.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:34 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:34.217+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:34.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:34 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:34.326+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:34.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:34 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:34.326+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:34.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:34 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:34.326+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:35.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:34 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:34.590+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:35.068 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:34 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:34.590+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:35.068 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:34 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:34.590+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:03.318369+0000 front 2024-04-26T02:06:03.318444+0000 (oldest deadline 2024-04-26T02:06:27.418372+0000) 2024-04-26T02:06:35.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:34 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:34.817+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:35.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:34 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:34.817+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:35.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:34 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:34.817+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:35.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:34 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:34.817+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:35.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:34 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:34.877+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:35.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:34 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:34.877+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.460304+0000 front 2024-04-26T02:05:53.460242+0000 (oldest deadline 2024-04-26T02:06:19.360240+0000) 2024-04-26T02:06:35.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:34 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:34.877+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:05:59.361120+0000 front 2024-04-26T02:05:59.360973+0000 (oldest deadline 2024-04-26T02:06:27.561366+0000) 2024-04-26T02:06:35.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:35 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:35.198+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:35.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:35 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:35.198+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:35.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:35 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:35.198+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:35.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:35 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:35.320+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:35.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:35 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:35.320+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:35.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:35 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:35.320+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:35.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:35 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:35.320+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:36.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:35 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:35.583+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:36.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:35 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:35.583+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:36.068 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:35 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:35.583+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:03.318369+0000 front 2024-04-26T02:06:03.318444+0000 (oldest deadline 2024-04-26T02:06:27.418372+0000) 2024-04-26T02:06:36.068 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:35 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:35.583+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:12.119529+0000 front 2024-04-26T02:06:12.119471+0000 (oldest deadline 2024-04-26T02:06:34.920254+0000) 2024-04-26T02:06:36.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:35 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:35.794+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:36.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:35 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:35.794+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:36.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:35 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:35.794+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:36.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:35 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:35.794+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:36.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:35 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:35.880+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:36.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:35 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:35.880+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.460304+0000 front 2024-04-26T02:05:53.460242+0000 (oldest deadline 2024-04-26T02:06:19.360240+0000) 2024-04-26T02:06:36.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:35 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:35.880+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:05:59.361120+0000 front 2024-04-26T02:05:59.360973+0000 (oldest deadline 2024-04-26T02:06:27.561366+0000) 2024-04-26T02:06:36.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:36 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:36.236+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:36.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:36 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:36.236+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:36.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:36 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:36.236+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:36.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:36 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:36.351+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:36.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:36 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:36.351+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:36.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:36 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:36.351+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:36.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:36 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:36.351+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:37.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:36 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:36.585+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:37.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:36 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:36.585+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:37.068 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:36 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:36.585+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:03.318369+0000 front 2024-04-26T02:06:03.318444+0000 (oldest deadline 2024-04-26T02:06:27.418372+0000) 2024-04-26T02:06:37.068 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:36 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:36.585+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:12.119529+0000 front 2024-04-26T02:06:12.119471+0000 (oldest deadline 2024-04-26T02:06:34.920254+0000) 2024-04-26T02:06:37.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:36 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:36.812+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:37.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:36 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:36.812+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:37.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:36 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:36.812+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:37.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:36 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:36.812+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:37.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:36 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:36.812+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:07.641632+0000 front 2024-04-26T02:06:07.641823+0000 (oldest deadline 2024-04-26T02:06:36.443294+0000) 2024-04-26T02:06:37.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:36 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:36.843+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:37.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:36 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:36.843+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.460304+0000 front 2024-04-26T02:05:53.460242+0000 (oldest deadline 2024-04-26T02:06:19.360240+0000) 2024-04-26T02:06:37.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:36 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:36.843+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:05:59.361120+0000 front 2024-04-26T02:05:59.360973+0000 (oldest deadline 2024-04-26T02:06:27.561366+0000) 2024-04-26T02:06:37.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:36 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:36.843+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:11.062172+0000 front 2024-04-26T02:06:11.062119+0000 (oldest deadline 2024-04-26T02:06:36.362266+0000) 2024-04-26T02:06:37.542 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:37 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:37.192+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:37.542 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:37 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:37.192+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:37.542 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:37 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:37.192+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:37.542 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:37 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:37.192+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:37.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:37 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:37.309+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:37.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:37 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:37.309+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:37.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:37 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:37.309+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:37.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:37 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:37.309+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:37.817 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:37 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:37.543+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:37.817 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:37 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:37.543+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:37.817 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:37 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:37.543+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:03.318369+0000 front 2024-04-26T02:06:03.318444+0000 (oldest deadline 2024-04-26T02:06:27.418372+0000) 2024-04-26T02:06:37.817 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:37 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:37.543+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:12.119529+0000 front 2024-04-26T02:06:12.119471+0000 (oldest deadline 2024-04-26T02:06:34.920254+0000) 2024-04-26T02:06:38.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:37 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:37.817+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:38.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:37 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:37.817+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:38.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:37 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:37.817+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:38.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:37 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:37.817+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:38.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:37 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:37.817+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:07.641632+0000 front 2024-04-26T02:06:07.641823+0000 (oldest deadline 2024-04-26T02:06:36.443294+0000) 2024-04-26T02:06:38.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:37 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:37.839+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.358529+0000 front 2024-04-26T02:05:45.358559+0000 (oldest deadline 2024-04-26T02:06:09.458648+0000) 2024-04-26T02:06:38.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:37 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:37.839+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.460304+0000 front 2024-04-26T02:05:53.460242+0000 (oldest deadline 2024-04-26T02:06:19.360240+0000) 2024-04-26T02:06:38.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:37 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:37.839+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:05:59.361120+0000 front 2024-04-26T02:05:59.360973+0000 (oldest deadline 2024-04-26T02:06:27.561366+0000) 2024-04-26T02:06:38.234 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:37 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7[50696]: 2024-04-26T02:06:37.839+0000 7f5f1d6e2700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:11.062172+0000 front 2024-04-26T02:06:11.062119+0000 (oldest deadline 2024-04-26T02:06:36.362266+0000) 2024-04-26T02:06:38.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:38 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:38.182+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:38.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:38 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:38.182+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:38.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:38 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:38.182+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:38.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:38 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:38.182+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:38.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:38 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:38.346+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:38.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:38 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:38.346+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:38.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:38 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:38.346+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:38.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:38 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:38.346+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:39.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:38 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:38.587+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:39.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:38 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:38.587+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:39.068 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:38 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:38.587+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:03.318369+0000 front 2024-04-26T02:06:03.318444+0000 (oldest deadline 2024-04-26T02:06:27.418372+0000) 2024-04-26T02:06:39.068 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:38 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:38.587+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:12.119529+0000 front 2024-04-26T02:06:12.119471+0000 (oldest deadline 2024-04-26T02:06:34.920254+0000) 2024-04-26T02:06:39.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:38 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:38.850+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:39.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:38 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:38.850+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:39.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:38 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:38.850+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:39.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:38 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:38.850+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:39.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:38 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:38.850+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:07.641632+0000 front 2024-04-26T02:06:07.641823+0000 (oldest deadline 2024-04-26T02:06:36.443294+0000) 2024-04-26T02:06:39.473 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:39 smithi097 podman[69659]: 2024-04-26 02:06:39.212318578 +0000 UTC m=+5.691453183 container died 22aeccdef0a963a9920225acfe73c2d22eb8352a9d8c20d78a48b99682f11438 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-26T02:06:39.473 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:39 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:39.364+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:39.473 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:39 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:39.364+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:39.473 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:39 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:39.364+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:39.474 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:39 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:39.364+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:39.539 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:39 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:39.170+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:39.539 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:39 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:39.170+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:39.539 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:39 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:39.170+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:39.539 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:39 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:39.170+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:39.734 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:39 smithi097 podman[69659]: 2024-04-26 02:06:39.573628281 +0000 UTC m=+6.052762896 container cleanup 22aeccdef0a963a9920225acfe73c2d22eb8352a9d8c20d78a48b99682f11438 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:06:39.734 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:39 smithi097 bash[69659]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7 2024-04-26T02:06:39.817 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:39 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:39.540+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:39.818 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:39 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:39.540+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:39.818 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:39 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:39.540+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:03.318369+0000 front 2024-04-26T02:06:03.318444+0000 (oldest deadline 2024-04-26T02:06:27.418372+0000) 2024-04-26T02:06:39.818 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:39 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:39.540+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:12.119529+0000 front 2024-04-26T02:06:12.119471+0000 (oldest deadline 2024-04-26T02:06:34.920254+0000) 2024-04-26T02:06:40.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:39 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:39.814+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:40.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:39 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:39.814+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:40.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:39 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:39.814+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:40.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:39 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:39.814+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:40.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:39 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:39.814+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:07.641632+0000 front 2024-04-26T02:06:07.641823+0000 (oldest deadline 2024-04-26T02:06:36.443294+0000) 2024-04-26T02:06:40.196 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:39 smithi097 podman[69676]: 2024-04-26 02:06:39.849710545 +0000 UTC m=+0.632568454 container remove 22aeccdef0a963a9920225acfe73c2d22eb8352a9d8c20d78a48b99682f11438 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-26T02:06:40.476 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:40 smithi097 podman[69770]: 2024-04-26 02:06:40.101131439 +0000 UTC m=+0.017783480 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:06:40.476 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:40 smithi097 podman[69770]: 2024-04-26 02:06:40.217884041 +0000 UTC m=+0.134536095 container create ce182183991c941d86814b01a8f753ef8e3098b19e583416a55f9d91bddbf311 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-26T02:06:40.477 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:40 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:40.392+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:40.477 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:40 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:40.392+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:40.477 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:40 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:40.392+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:40.477 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:40 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:40.392+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:40.541 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:40 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:40.215+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:40.541 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:40 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:40.215+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:40.541 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:40 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:40.215+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:40.541 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:40 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:40.215+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:40.734 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:40 smithi097 podman[69770]: 2024-04-26 02:06:40.478291924 +0000 UTC m=+0.394943935 container init ce182183991c941d86814b01a8f753ef8e3098b19e583416a55f9d91bddbf311 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7-deactivate, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:06:40.734 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:40 smithi097 podman[69770]: 2024-04-26 02:06:40.481501034 +0000 UTC m=+0.398153047 container start ce182183991c941d86814b01a8f753ef8e3098b19e583416a55f9d91bddbf311 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:06:40.734 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:40 smithi097 podman[69770]: 2024-04-26 02:06:40.569944497 +0000 UTC m=+0.486596519 container attach ce182183991c941d86814b01a8f753ef8e3098b19e583416a55f9d91bddbf311 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-26T02:06:40.817 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:40 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:40.542+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:40.818 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:40 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:40.542+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:40.818 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:40 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:40.542+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:03.318369+0000 front 2024-04-26T02:06:03.318444+0000 (oldest deadline 2024-04-26T02:06:27.418372+0000) 2024-04-26T02:06:40.818 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:40 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:40.542+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:12.119529+0000 front 2024-04-26T02:06:12.119471+0000 (oldest deadline 2024-04-26T02:06:34.920254+0000) 2024-04-26T02:06:41.129 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 26 02:06:40 smithi097 podman[69770]: 2024-04-26 02:06:40.77295738 +0000 UTC m=+0.689609407 container died ce182183991c941d86814b01a8f753ef8e3098b19e583416a55f9d91bddbf311 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-26T02:06:41.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:40 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:40.863+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:41.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:40 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:40.863+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:41.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:40 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:40.863+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:41.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:40 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:40.863+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:41.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:40 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:40.863+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:07.641632+0000 front 2024-04-26T02:06:07.641823+0000 (oldest deadline 2024-04-26T02:06:36.443294+0000) 2024-04-26T02:06:41.482 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.7.service' 2024-04-26T02:06:41.522 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:06:41.522 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-26T02:06:41.523 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-26T02:06:41.523 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.8 2024-04-26T02:06:41.549 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:41 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:41.231+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:41.549 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:41 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:41.231+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:41.549 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:41 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:41.231+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:41.549 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:41 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:41.231+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:41.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:41 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:41.391+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:41.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:41 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:41.391+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:41.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:41 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:41.391+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:41.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:41 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:41.391+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:41.802 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:41 smithi173 systemd[1]: Stopping Ceph osd.8 for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:06:41.802 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:41 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:41.575+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:41.802 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:41 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:41.575+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:41.802 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:41 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:41.575+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:03.318369+0000 front 2024-04-26T02:06:03.318444+0000 (oldest deadline 2024-04-26T02:06:27.418372+0000) 2024-04-26T02:06:41.805 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:41 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:41.575+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:12.119529+0000 front 2024-04-26T02:06:12.119471+0000 (oldest deadline 2024-04-26T02:06:34.920254+0000) 2024-04-26T02:06:41.805 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:41 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:41.803+0000 7fc179d00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:06:41.805 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:41 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:41.803+0000 7fc179d00700 -1 osd.8 112 *** Got signal Terminated *** 2024-04-26T02:06:41.805 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:41 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:41.803+0000 7fc179d00700 -1 osd.8 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:06:42.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:41 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:41.861+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:42.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:41 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:41.861+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:42.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:41 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:41.861+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:42.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:41 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:41.861+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:42.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:41 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:41.861+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:07.641632+0000 front 2024-04-26T02:06:07.641823+0000 (oldest deadline 2024-04-26T02:06:36.443294+0000) 2024-04-26T02:06:42.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:42 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:42.187+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:42.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:42 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:42.187+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:42.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:42 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:42.187+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:42.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:42 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:42.187+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:42.648 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:42 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:42.369+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:42.648 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:42 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:42.369+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:42.648 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:42 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:42.369+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:42.648 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:42 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:42.369+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:42.648 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:42 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:42.369+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:06:43.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:42 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:42.583+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:43.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:42 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:42.583+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:43.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:42 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:42.583+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:03.318369+0000 front 2024-04-26T02:06:03.318444+0000 (oldest deadline 2024-04-26T02:06:27.418372+0000) 2024-04-26T02:06:43.068 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:42 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:42.583+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:12.119529+0000 front 2024-04-26T02:06:12.119471+0000 (oldest deadline 2024-04-26T02:06:34.920254+0000) 2024-04-26T02:06:43.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:42 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:42.819+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:43.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:42 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:42.819+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:43.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:42 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:42.819+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:43.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:42 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:42.819+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:43.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:42 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:42.819+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:07.641632+0000 front 2024-04-26T02:06:07.641823+0000 (oldest deadline 2024-04-26T02:06:36.443294+0000) 2024-04-26T02:06:43.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:43 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:43.143+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:43.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:43 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:43.143+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:43.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:43 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:43.143+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:43.569 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:43 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:43.143+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:43.569 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:43 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:43.143+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:06:43.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:43 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:43.321+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:43.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:43 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:43.321+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:43.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:43 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:43.321+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:43.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:43 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:43.321+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:43.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:43 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:43.321+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:06:44.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:43 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:43.606+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:44.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:43 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:43.606+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:44.068 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:43 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:43.606+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:03.318369+0000 front 2024-04-26T02:06:03.318444+0000 (oldest deadline 2024-04-26T02:06:27.418372+0000) 2024-04-26T02:06:44.068 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:43 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:43.606+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:12.119529+0000 front 2024-04-26T02:06:12.119471+0000 (oldest deadline 2024-04-26T02:06:34.920254+0000) 2024-04-26T02:06:44.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:43 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:43.866+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:44.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:43 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:43.866+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:44.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:43 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:43.866+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:44.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:43 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:43.866+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:44.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:43 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:43.866+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:07.641632+0000 front 2024-04-26T02:06:07.641823+0000 (oldest deadline 2024-04-26T02:06:36.443294+0000) 2024-04-26T02:06:44.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:44 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:44.140+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:44.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:44 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:44.140+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:44.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:44 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:44.140+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:44.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:44 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:44.140+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:44.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:44 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:44.140+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:06:44.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:44 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:44.296+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:44.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:44 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:44.296+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:44.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:44 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:44.296+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:44.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:44 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:44.296+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:44.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:44 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:44.296+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:06:45.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:44 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:44.635+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:45.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:44 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:44.635+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:45.068 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:44 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:44.635+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:03.318369+0000 front 2024-04-26T02:06:03.318444+0000 (oldest deadline 2024-04-26T02:06:27.418372+0000) 2024-04-26T02:06:45.068 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:44 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:44.635+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:12.119529+0000 front 2024-04-26T02:06:12.119471+0000 (oldest deadline 2024-04-26T02:06:34.920254+0000) 2024-04-26T02:06:45.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:44 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:44.890+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:45.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:44 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:44.890+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:45.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:44 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:44.890+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:45.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:44 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:44.890+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:45.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:44 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:44.890+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:07.641632+0000 front 2024-04-26T02:06:07.641823+0000 (oldest deadline 2024-04-26T02:06:36.443294+0000) 2024-04-26T02:06:45.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:45 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:45.104+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:45.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:45 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:45.104+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:45.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:45 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:45.104+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:45.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:45 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:45.104+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:45.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:45 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:45.104+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:06:45.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:45 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:45.306+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:45.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:45 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:45.306+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:45.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:45 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:45.306+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:45.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:45 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:45.306+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:45.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:45 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:45.306+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:06:46.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:45 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:45.661+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:46.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:45 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:45.661+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:46.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:45 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:45.661+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:03.318369+0000 front 2024-04-26T02:06:03.318444+0000 (oldest deadline 2024-04-26T02:06:27.418372+0000) 2024-04-26T02:06:46.068 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:45 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:45.661+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:12.119529+0000 front 2024-04-26T02:06:12.119471+0000 (oldest deadline 2024-04-26T02:06:34.920254+0000) 2024-04-26T02:06:46.068 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:45 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:45.661+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.121825+0000 front 2024-04-26T02:06:20.121811+0000 (oldest deadline 2024-04-26T02:06:44.821955+0000) 2024-04-26T02:06:46.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:45 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:45.860+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:46.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:45 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:45.860+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:46.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:45 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:45.860+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:46.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:45 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:45.860+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:46.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:45 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:45.860+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:07.641632+0000 front 2024-04-26T02:06:07.641823+0000 (oldest deadline 2024-04-26T02:06:36.443294+0000) 2024-04-26T02:06:46.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:45 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:45.860+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:21.145081+0000 front 2024-04-26T02:06:21.144867+0000 (oldest deadline 2024-04-26T02:06:45.844803+0000) 2024-04-26T02:06:46.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:46 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:46.070+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:46.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:46 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:46.070+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:46.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:46 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:46.070+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:46.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:46 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:46.070+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:46.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:46 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:46.070+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:06:46.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:46 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:46.317+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:46.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:46 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:46.317+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:46.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:46 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:46.317+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:46.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:46 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:46.317+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:46.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:46 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:46.317+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:06:47.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:46 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:46.639+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.414758+0000 front 2024-04-26T02:05:46.414827+0000 (oldest deadline 2024-04-26T02:06:08.714859+0000) 2024-04-26T02:06:47.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:46 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:46.639+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.416047+0000 front 2024-04-26T02:05:53.416028+0000 (oldest deadline 2024-04-26T02:06:22.216906+0000) 2024-04-26T02:06:47.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:46 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:46.639+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:03.318369+0000 front 2024-04-26T02:06:03.318444+0000 (oldest deadline 2024-04-26T02:06:27.418372+0000) 2024-04-26T02:06:47.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:46 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:46.639+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:12.119529+0000 front 2024-04-26T02:06:12.119471+0000 (oldest deadline 2024-04-26T02:06:34.920254+0000) 2024-04-26T02:06:47.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:46 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8[50783]: 2024-04-26T02:06:46.639+0000 7fc1722ec700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.121825+0000 front 2024-04-26T02:06:20.121811+0000 (oldest deadline 2024-04-26T02:06:44.821955+0000) 2024-04-26T02:06:47.317 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:47 smithi173 podman[71217]: 2024-04-26 02:06:47.182451384 +0000 UTC m=+5.469722369 container died a2694a52152e2205d84a36e6adfa106269984c4d50524d54fe7fe61e5a95dad3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:06:47.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:47 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:47.067+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:47.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:47 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:47.067+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:47.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:47 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:47.067+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:47.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:47 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:47.067+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:47.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:47 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:47.067+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:06:47.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:46 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:46.888+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:47.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:46 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:46.888+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:47.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:46 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:46.888+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:47.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:46 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:46.888+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:47.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:46 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:46.888+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:07.641632+0000 front 2024-04-26T02:06:07.641823+0000 (oldest deadline 2024-04-26T02:06:36.443294+0000) 2024-04-26T02:06:47.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:46 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:46.888+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:21.145081+0000 front 2024-04-26T02:06:21.144867+0000 (oldest deadline 2024-04-26T02:06:45.844803+0000) 2024-04-26T02:06:47.695 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:47 smithi173 podman[71217]: 2024-04-26 02:06:47.520980842 +0000 UTC m=+5.808251873 container cleanup a2694a52152e2205d84a36e6adfa106269984c4d50524d54fe7fe61e5a95dad3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T02:06:47.695 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:47 smithi173 bash[71217]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8 2024-04-26T02:06:47.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:47 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:47.276+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:47.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:47 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:47.276+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:47.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:47 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:47.276+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:47.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:47 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:47.276+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:47.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:47 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:47.276+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:06:48.067 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:47 smithi173 podman[71234]: 2024-04-26 02:06:47.697379962 +0000 UTC m=+0.517863366 container remove a2694a52152e2205d84a36e6adfa106269984c4d50524d54fe7fe61e5a95dad3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-26T02:06:48.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:47 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:47.872+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:48.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:47 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:47.872+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:48.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:47 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:47.872+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:48.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:47 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:47.872+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:48.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:47 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:47.872+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:07.641632+0000 front 2024-04-26T02:06:07.641823+0000 (oldest deadline 2024-04-26T02:06:36.443294+0000) 2024-04-26T02:06:48.131 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:47 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:47.872+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:21.145081+0000 front 2024-04-26T02:06:21.144867+0000 (oldest deadline 2024-04-26T02:06:45.844803+0000) 2024-04-26T02:06:48.366 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:48 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:48.094+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:48.366 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:48 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:48.094+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:48.366 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:48 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:48.094+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:48.366 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:48 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:48.094+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:48.366 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:48 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:48.094+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:06:48.366 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:48 smithi173 podman[71326]: 2024-04-26 02:06:48.043238581 +0000 UTC m=+0.017980810 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:06:48.366 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:48 smithi173 podman[71326]: 2024-04-26 02:06:48.156618422 +0000 UTC m=+0.131360635 container create ca08bae6889a7609328cd6b7e3e78e178757cd99dcf262e38f373a7ca5836942 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-04-26T02:06:48.367 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:48 smithi173 podman[71326]: 2024-04-26 02:06:48.367902863 +0000 UTC m=+0.342645076 container init ca08bae6889a7609328cd6b7e3e78e178757cd99dcf262e38f373a7ca5836942 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-26T02:06:48.635 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:48 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:48.325+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:48.635 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:48 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:48.325+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:48.635 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:48 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:48.325+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:48.635 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:48 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:48.325+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:48.636 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:48 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:48.325+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:06:48.662 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:48 smithi173 podman[71326]: 2024-04-26 02:06:48.371246546 +0000 UTC m=+0.345988760 container start ca08bae6889a7609328cd6b7e3e78e178757cd99dcf262e38f373a7ca5836942 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-26T02:06:48.662 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:48 smithi173 podman[71326]: 2024-04-26 02:06:48.469141736 +0000 UTC m=+0.443883952 container attach ca08bae6889a7609328cd6b7e3e78e178757cd99dcf262e38f373a7ca5836942 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-26T02:06:48.952 INFO:journalctl@ceph.osd.8.smithi173.stdout:Apr 26 02:06:48 smithi173 podman[71326]: 2024-04-26 02:06:48.664017664 +0000 UTC m=+0.638759879 container died ca08bae6889a7609328cd6b7e3e78e178757cd99dcf262e38f373a7ca5836942 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:06:49.229 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:49 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:49.095+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:49.229 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:49 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:49.095+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:49.229 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:49 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:49.095+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:49.229 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:49 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:49.095+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:49.229 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:49 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:49.095+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:06:49.254 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.8.service' 2024-04-26T02:06:49.331 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:06:49.332 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-26T02:06:49.332 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-26T02:06:49.332 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.9 2024-04-26T02:06:49.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:48 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:48.912+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:49.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:48 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:48.912+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:49.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:48 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:48.912+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:49.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:48 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:48.912+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:49.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:48 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:48.912+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:07.641632+0000 front 2024-04-26T02:06:07.641823+0000 (oldest deadline 2024-04-26T02:06:36.443294+0000) 2024-04-26T02:06:49.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:48 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:48.912+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:21.145081+0000 front 2024-04-26T02:06:21.144867+0000 (oldest deadline 2024-04-26T02:06:45.844803+0000) 2024-04-26T02:06:49.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:49 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:49.278+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:49.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:49 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:49.278+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:49.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:49 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:49.278+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:49.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:49 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:49.278+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:49.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:49 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:49.278+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:06:49.880 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:49 smithi001 systemd[1]: Stopping Ceph osd.9 for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:06:50.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:49 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:49.935+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:50.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:49 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:49.935+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:50.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:49 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:49.935+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:50.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:49 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:49.935+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:50.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:49 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:49.935+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:07.641632+0000 front 2024-04-26T02:06:07.641823+0000 (oldest deadline 2024-04-26T02:06:36.443294+0000) 2024-04-26T02:06:50.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:49 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:49.935+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:21.145081+0000 front 2024-04-26T02:06:21.144867+0000 (oldest deadline 2024-04-26T02:06:45.844803+0000) 2024-04-26T02:06:50.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:49 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:49.975+0000 7fad95400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:06:50.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:49 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:49.975+0000 7fad95400700 -1 osd.9 112 *** Got signal Terminated *** 2024-04-26T02:06:50.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:49 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:49.975+0000 7fad95400700 -1 osd.9 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:06:50.422 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:50 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:50.081+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:50.422 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:50 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:50.081+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:50.422 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:50 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:50.081+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:50.423 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:50 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:50.081+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:50.423 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:50 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:50.081+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:06:50.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:50 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:50.259+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:50.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:50 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:50.259+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:50.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:50 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:50.259+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:50.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:50 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:50.259+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:50.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:50 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:50.259+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:06:51.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:50 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:50.892+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:51.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:50 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:50.892+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:51.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:50 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:50.892+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:51.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:50 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:50.892+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:51.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:50 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:50.892+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:07.641632+0000 front 2024-04-26T02:06:07.641823+0000 (oldest deadline 2024-04-26T02:06:36.443294+0000) 2024-04-26T02:06:51.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:50 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:50.892+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:21.145081+0000 front 2024-04-26T02:06:21.144867+0000 (oldest deadline 2024-04-26T02:06:45.844803+0000) 2024-04-26T02:06:51.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:51 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:51.103+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:51.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:51 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:51.103+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:51.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:51 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:51.103+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:51.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:51 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:51.103+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:51.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:51 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:51.103+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:06:51.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:51 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:51.238+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:51.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:51 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:51.238+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:51.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:51 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:51.238+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:51.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:51 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:51.238+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:51.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:51 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:51.238+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:06:51.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:51 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:51.238+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.940203+0000 front 2024-04-26T02:06:27.940031+0000 (oldest deadline 2024-04-26T02:06:50.839859+0000) 2024-04-26T02:06:52.317 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:52 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:52.064+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:52.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:52 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:52.064+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:52.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:52 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:52.064+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:52.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:52 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:52.064+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:52.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:52 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:52.064+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:06:52.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:51.929+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:52.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:51.929+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:52.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:51.929+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:52.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:51.929+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:52.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:51.929+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:07.641632+0000 front 2024-04-26T02:06:07.641823+0000 (oldest deadline 2024-04-26T02:06:36.443294+0000) 2024-04-26T02:06:52.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:51 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:51.929+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:21.145081+0000 front 2024-04-26T02:06:21.144867+0000 (oldest deadline 2024-04-26T02:06:45.844803+0000) 2024-04-26T02:06:52.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:52 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:52.267+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:52.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:52 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:52.267+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:52.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:52 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:52.267+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:52.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:52 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:52.267+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:52.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:52 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:52.267+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:06:52.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:52 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:52.267+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.940203+0000 front 2024-04-26T02:06:27.940031+0000 (oldest deadline 2024-04-26T02:06:50.839859+0000) 2024-04-26T02:06:53.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:52 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:52.885+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:53.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:52 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:52.885+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:53.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:52 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:52.885+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:53.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:52 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:52.885+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:53.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:52 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:52.885+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:07.641632+0000 front 2024-04-26T02:06:07.641823+0000 (oldest deadline 2024-04-26T02:06:36.443294+0000) 2024-04-26T02:06:53.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:52 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:52.885+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:21.145081+0000 front 2024-04-26T02:06:21.144867+0000 (oldest deadline 2024-04-26T02:06:45.844803+0000) 2024-04-26T02:06:53.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:53 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:53.219+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:53.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:53 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:53.219+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:53.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:53 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:53.219+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:53.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:53 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:53.219+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:53.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:53 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:53.219+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:06:53.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:53 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:53.219+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.940203+0000 front 2024-04-26T02:06:27.940031+0000 (oldest deadline 2024-04-26T02:06:50.839859+0000) 2024-04-26T02:06:53.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:53 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:53.094+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:53.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:53 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:53.094+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:53.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:53 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:53.094+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:53.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:53 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:53.094+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:53.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:53 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:53.094+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:06:54.317 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:54 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:54.066+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:54.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:54 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:54.066+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:54.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:54 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:54.066+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:54.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:54 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:54.066+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:54.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:54 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:54.066+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:06:54.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:54 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:54.066+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.969164+0000 front 2024-04-26T02:06:27.968989+0000 (oldest deadline 2024-04-26T02:06:53.868764+0000) 2024-04-26T02:06:54.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:53 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:53.935+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:54.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:53 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:53.935+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:54.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:53 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:53.935+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:54.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:53 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:53.935+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:54.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:53 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:53.935+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:07.641632+0000 front 2024-04-26T02:06:07.641823+0000 (oldest deadline 2024-04-26T02:06:36.443294+0000) 2024-04-26T02:06:54.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:53 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:53.935+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:21.145081+0000 front 2024-04-26T02:06:21.144867+0000 (oldest deadline 2024-04-26T02:06:45.844803+0000) 2024-04-26T02:06:54.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:54 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:54.263+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:54.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:54 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:54.263+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:54.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:54 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:54.263+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:54.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:54 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:54.263+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:54.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:54 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:54.263+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:06:54.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:54 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:54.263+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.940203+0000 front 2024-04-26T02:06:27.940031+0000 (oldest deadline 2024-04-26T02:06:50.839859+0000) 2024-04-26T02:06:55.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:54 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:54.886+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:36.230446+0000 front 2024-04-26T02:05:36.230562+0000 (oldest deadline 2024-04-26T02:06:02.129945+0000) 2024-04-26T02:06:55.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:54 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:54.886+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:47.234537+0000 front 2024-04-26T02:05:47.234332+0000 (oldest deadline 2024-04-26T02:06:11.334169+0000) 2024-04-26T02:06:55.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:54 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:54.886+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:55.937741+0000 front 2024-04-26T02:05:55.937802+0000 (oldest deadline 2024-04-26T02:06:21.837879+0000) 2024-04-26T02:06:55.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:54 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:54.886+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:01.838724+0000 front 2024-04-26T02:06:01.838902+0000 (oldest deadline 2024-04-26T02:06:27.138457+0000) 2024-04-26T02:06:55.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:54 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:54.886+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:07.641632+0000 front 2024-04-26T02:06:07.641823+0000 (oldest deadline 2024-04-26T02:06:36.443294+0000) 2024-04-26T02:06:55.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:54 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:54.886+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:21.145081+0000 front 2024-04-26T02:06:21.144867+0000 (oldest deadline 2024-04-26T02:06:45.844803+0000) 2024-04-26T02:06:55.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:54 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9[60248]: 2024-04-26T02:06:54.886+0000 7fad90c00700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:28.648478+0000 front 2024-04-26T02:06:28.648142+0000 (oldest deadline 2024-04-26T02:06:54.547820+0000) 2024-04-26T02:06:55.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:55 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:55.108+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:55.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:55 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:55.108+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:55.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:55 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:55.108+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:55.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:55 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:55.108+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:55.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:55 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:55.108+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:06:55.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:55 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:55.108+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.969164+0000 front 2024-04-26T02:06:27.968989+0000 (oldest deadline 2024-04-26T02:06:53.868764+0000) 2024-04-26T02:06:55.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:55 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:55.277+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:55.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:55 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:55.277+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:55.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:55 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:55.277+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:55.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:55 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:55.277+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:55.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:55 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:55.277+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:06:55.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:55 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:55.277+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.940203+0000 front 2024-04-26T02:06:27.940031+0000 (oldest deadline 2024-04-26T02:06:50.839859+0000) 2024-04-26T02:06:55.757 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:55 smithi001 podman[119617]: 2024-04-26 02:06:55.461557797 +0000 UTC m=+5.634848195 container died bb4a71d9cbfd5a5c3673729b7edd25b9be18a370477310a66122f9dae78bfb04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-26T02:06:56.026 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:56 smithi001 podman[119617]: 2024-04-26 02:06:56.001044697 +0000 UTC m=+6.174335132 container cleanup bb4a71d9cbfd5a5c3673729b7edd25b9be18a370477310a66122f9dae78bfb04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:06:56.026 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:56 smithi001 bash[119617]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9 2024-04-26T02:06:56.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:56 smithi001 podman[119637]: 2024-04-26 02:06:56.269283105 +0000 UTC m=+0.794807243 container remove bb4a71d9cbfd5a5c3673729b7edd25b9be18a370477310a66122f9dae78bfb04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:06:56.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:56 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:56.232+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:56.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:56 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:56.232+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:56.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:56 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:56.232+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:56.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:56 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:56.232+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:56.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:56 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:56.232+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:06:56.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:56 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:56.232+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.940203+0000 front 2024-04-26T02:06:27.940031+0000 (oldest deadline 2024-04-26T02:06:50.839859+0000) 2024-04-26T02:06:56.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:56 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:56.105+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:56.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:56 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:56.105+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:56.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:56 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:56.105+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:56.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:56 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:56.105+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:56.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:56 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:56.105+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:06:56.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:56 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:56.105+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.969164+0000 front 2024-04-26T02:06:27.968989+0000 (oldest deadline 2024-04-26T02:06:53.868764+0000) 2024-04-26T02:06:57.365 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:57 smithi001 podman[119725]: 2024-04-26 02:06:57.006974784 +0000 UTC m=+0.040791564 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:06:57.365 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:57 smithi001 podman[119725]: 2024-04-26 02:06:57.119930945 +0000 UTC m=+0.153747687 container create 511c958b3f861980f91ba3c15080083827a1021eb3d110d7dccd7fb9bb2789f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-26T02:06:57.365 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:57 smithi001 podman[119725]: 2024-04-26 02:06:57.36576302 +0000 UTC m=+0.399579801 container init 511c958b3f861980f91ba3c15080083827a1021eb3d110d7dccd7fb9bb2789f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:06:57.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:57 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:57.234+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:57.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:57 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:57.234+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:57.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:57 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:57.234+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:57.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:57 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:57.234+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:57.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:57 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:57.234+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:06:57.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:57 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:57.234+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.940203+0000 front 2024-04-26T02:06:27.940031+0000 (oldest deadline 2024-04-26T02:06:50.839859+0000) 2024-04-26T02:06:57.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:57 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:57.153+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:57.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:57 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:57.153+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:57.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:57 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:57.153+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:57.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:57 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:57.153+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:57.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:57 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:57.153+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:06:57.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:57 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:57.153+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.969164+0000 front 2024-04-26T02:06:27.968989+0000 (oldest deadline 2024-04-26T02:06:53.868764+0000) 2024-04-26T02:06:57.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:57 smithi001 podman[119725]: 2024-04-26 02:06:57.378551942 +0000 UTC m=+0.412368712 container start 511c958b3f861980f91ba3c15080083827a1021eb3d110d7dccd7fb9bb2789f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-26T02:06:57.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:57 smithi001 podman[119725]: 2024-04-26 02:06:57.455951025 +0000 UTC m=+0.489767832 container attach 511c958b3f861980f91ba3c15080083827a1021eb3d110d7dccd7fb9bb2789f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-26T02:06:58.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:Apr 26 02:06:57 smithi001 podman[119725]: 2024-04-26 02:06:57.87392879 +0000 UTC m=+0.907745538 container died 511c958b3f861980f91ba3c15080083827a1021eb3d110d7dccd7fb9bb2789f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-26T02:06:58.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:58 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:58.123+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:58.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:58 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:58.123+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:58.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:58 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:58.123+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:58.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:58 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:58.123+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:58.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:58 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:58.123+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:06:58.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:58 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:58.123+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.969164+0000 front 2024-04-26T02:06:27.968989+0000 (oldest deadline 2024-04-26T02:06:53.868764+0000) 2024-04-26T02:06:58.672 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.9.service' 2024-04-26T02:06:58.687 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:58 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:58.246+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:58.688 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:58 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:58.246+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:58.688 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:58 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:58.246+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:58.688 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:58 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:58.246+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:58.688 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:58 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:58.246+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:06:58.688 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:58 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:58.246+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.940203+0000 front 2024-04-26T02:06:27.940031+0000 (oldest deadline 2024-04-26T02:06:50.839859+0000) 2024-04-26T02:06:58.819 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:06:58.819 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-26T02:06:58.819 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-26T02:06:58.819 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.10 2024-04-26T02:06:58.984 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:58 smithi097 systemd[1]: Stopping Ceph osd.10 for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:06:59.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:59 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:59.124+0000 7fdc4e6ef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:06:59.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:59 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:59.124+0000 7fdc4e6ef700 -1 osd.10 112 *** Got signal Terminated *** 2024-04-26T02:06:59.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:59 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:59.124+0000 7fdc4e6ef700 -1 osd.10 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:06:59.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:59 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:59.241+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:06:59.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:59 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:59.241+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:06:59.485 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:59 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:59.241+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:06:59.485 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:59 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:59.241+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:06:59.485 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:59 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:59.241+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:06:59.485 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:06:59 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:06:59.241+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.940203+0000 front 2024-04-26T02:06:27.940031+0000 (oldest deadline 2024-04-26T02:06:50.839859+0000) 2024-04-26T02:06:59.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:59 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:59.136+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:06:59.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:59 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:59.136+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:06:59.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:59 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:59.136+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:06:59.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:59 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:59.136+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:06:59.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:59 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:59.136+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:06:59.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:06:59 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:06:59.136+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.969164+0000 front 2024-04-26T02:06:27.968989+0000 (oldest deadline 2024-04-26T02:06:53.868764+0000) 2024-04-26T02:07:00.317 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:00 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:00.095+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:07:00.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:00 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:00.095+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:07:00.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:00 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:00.095+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:07:00.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:00 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:00.095+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:07:00.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:00 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:00.095+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:07:00.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:00 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:00.095+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.969164+0000 front 2024-04-26T02:06:27.968989+0000 (oldest deadline 2024-04-26T02:06:53.868764+0000) 2024-04-26T02:07:00.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:00 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:00.245+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:07:00.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:00 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:00.245+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:07:00.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:00 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:00.245+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:07:00.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:00 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:00.245+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:07:00.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:00 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:00.245+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:07:00.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:00 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:00.245+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.940203+0000 front 2024-04-26T02:06:27.940031+0000 (oldest deadline 2024-04-26T02:06:50.839859+0000) 2024-04-26T02:07:00.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:00 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:00.245+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-26T02:06:36.140901+0000 front 2024-04-26T02:06:36.140935+0000 (oldest deadline 2024-04-26T02:07:00.241042+0000) 2024-04-26T02:07:01.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:01 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:01.120+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:07:01.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:01 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:01.120+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:07:01.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:01 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:01.120+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:07:01.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:01 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:01.120+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:07:01.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:01 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:01.120+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:07:01.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:01 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:01.120+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.969164+0000 front 2024-04-26T02:06:27.968989+0000 (oldest deadline 2024-04-26T02:06:53.868764+0000) 2024-04-26T02:07:01.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:01 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:01.287+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:07:01.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:01 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:01.287+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:07:01.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:01 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:01.287+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:07:01.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:01 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:01.287+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:07:01.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:01 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:01.287+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:07:01.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:01 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:01.287+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.940203+0000 front 2024-04-26T02:06:27.940031+0000 (oldest deadline 2024-04-26T02:06:50.839859+0000) 2024-04-26T02:07:01.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:01 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:01.287+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-26T02:06:36.140901+0000 front 2024-04-26T02:06:36.140935+0000 (oldest deadline 2024-04-26T02:07:00.241042+0000) 2024-04-26T02:07:02.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:02 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:02.090+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:07:02.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:02 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:02.090+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:07:02.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:02 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:02.090+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:07:02.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:02 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:02.090+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:07:02.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:02 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:02.090+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:07:02.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:02 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:02.090+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.969164+0000 front 2024-04-26T02:06:27.968989+0000 (oldest deadline 2024-04-26T02:06:53.868764+0000) 2024-04-26T02:07:02.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:02 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:02.246+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:07:02.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:02 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:02.246+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:07:02.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:02 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:02.246+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:07:02.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:02 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:02.246+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:07:02.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:02 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:02.246+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:07:02.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:02 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:02.246+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.940203+0000 front 2024-04-26T02:06:27.940031+0000 (oldest deadline 2024-04-26T02:06:50.839859+0000) 2024-04-26T02:07:02.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:02 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:02.246+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-26T02:06:36.140901+0000 front 2024-04-26T02:06:36.140935+0000 (oldest deadline 2024-04-26T02:07:00.241042+0000) 2024-04-26T02:07:03.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:03 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:03.048+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:07:03.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:03 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:03.048+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:07:03.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:03 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:03.048+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:07:03.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:03 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:03.048+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:07:03.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:03 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:03.048+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:07:03.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:03 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:03.048+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.969164+0000 front 2024-04-26T02:06:27.968989+0000 (oldest deadline 2024-04-26T02:06:53.868764+0000) 2024-04-26T02:07:03.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:03 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:03.224+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:45.932206+0000 front 2024-04-26T02:05:45.932374+0000 (oldest deadline 2024-04-26T02:06:10.032276+0000) 2024-04-26T02:07:03.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:03 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:03.224+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:53.433971+0000 front 2024-04-26T02:05:53.433793+0000 (oldest deadline 2024-04-26T02:06:18.733826+0000) 2024-04-26T02:07:03.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:03 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:03.224+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-04-26T02:06:02.835393+0000 front 2024-04-26T02:06:02.835482+0000 (oldest deadline 2024-04-26T02:06:25.135400+0000) 2024-04-26T02:07:03.484 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:03 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:03.224+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.836774+0000 front 2024-04-26T02:06:09.836831+0000 (oldest deadline 2024-04-26T02:06:35.136899+0000) 2024-04-26T02:07:03.485 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:03 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:03.224+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:20.437998+0000 front 2024-04-26T02:06:20.437894+0000 (oldest deadline 2024-04-26T02:06:42.138079+0000) 2024-04-26T02:07:03.485 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:03 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:03.224+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.940203+0000 front 2024-04-26T02:06:27.940031+0000 (oldest deadline 2024-04-26T02:06:50.839859+0000) 2024-04-26T02:07:03.485 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:03 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10[57270]: 2024-04-26T02:07:03.224+0000 7fdc4ace3700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-26T02:06:36.140901+0000 front 2024-04-26T02:06:36.140935+0000 (oldest deadline 2024-04-26T02:07:00.241042+0000) 2024-04-26T02:07:04.317 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:04.062+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:07:04.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:04.062+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:07:04.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:04.062+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:07:04.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:04.062+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:07:04.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:04.062+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:07:04.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:04 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:04.062+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.969164+0000 front 2024-04-26T02:06:27.968989+0000 (oldest deadline 2024-04-26T02:06:53.868764+0000) 2024-04-26T02:07:04.933 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:04 smithi097 podman[69888]: 2024-04-26 02:07:04.661202859 +0000 UTC m=+5.657027263 container died 50f8e9572f89a7e466d0e8eefb391e86e76d3397636e5bbb4088990f6972a6fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:07:05.234 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:05 smithi097 podman[69888]: 2024-04-26 02:07:05.025296744 +0000 UTC m=+6.021121155 container cleanup 50f8e9572f89a7e466d0e8eefb391e86e76d3397636e5bbb4088990f6972a6fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-26T02:07:05.234 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:05 smithi097 bash[69888]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10 2024-04-26T02:07:05.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:05 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:05.025+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:07:05.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:05 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:05.025+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:07:05.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:05 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:05.025+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:07:05.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:05 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:05.025+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:07:05.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:05 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:05.025+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:07:05.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:05 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:05.025+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.969164+0000 front 2024-04-26T02:06:27.968989+0000 (oldest deadline 2024-04-26T02:06:53.868764+0000) 2024-04-26T02:07:05.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:05 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:05.025+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-26T02:06:38.570232+0000 front 2024-04-26T02:06:38.570334+0000 (oldest deadline 2024-04-26T02:07:04.970458+0000) 2024-04-26T02:07:05.589 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:05 smithi097 podman[69906]: 2024-04-26 02:07:05.276166643 +0000 UTC m=+0.618303067 container remove 50f8e9572f89a7e466d0e8eefb391e86e76d3397636e5bbb4088990f6972a6fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:07:05.910 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:05 smithi097 podman[69999]: 2024-04-26 02:07:05.494187445 +0000 UTC m=+0.017886149 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:07:05.910 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:05 smithi097 podman[69999]: 2024-04-26 02:07:05.601889839 +0000 UTC m=+0.125588572 container create bdf0ac669836e7196d5862cfe642c6be91a79737dba8ef12a7bc561bccdf5176 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-26T02:07:05.910 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:05 smithi097 podman[69999]: 2024-04-26 02:07:05.8293152 +0000 UTC m=+0.353013891 container init bdf0ac669836e7196d5862cfe642c6be91a79737dba8ef12a7bc561bccdf5176 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-26T02:07:05.911 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:05 smithi097 podman[69999]: 2024-04-26 02:07:05.83315078 +0000 UTC m=+0.356849479 container start bdf0ac669836e7196d5862cfe642c6be91a79737dba8ef12a7bc561bccdf5176 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-04-26T02:07:05.911 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:05 smithi097 podman[69999]: 2024-04-26 02:07:05.912304083 +0000 UTC m=+0.436002773 container attach bdf0ac669836e7196d5862cfe642c6be91a79737dba8ef12a7bc561bccdf5176 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-26T02:07:06.234 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 26 02:07:06 smithi097 podman[69999]: 2024-04-26 02:07:06.125109843 +0000 UTC m=+0.648808536 container died bdf0ac669836e7196d5862cfe642c6be91a79737dba8ef12a7bc561bccdf5176 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:07:06.317 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:05 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:05.988+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:07:06.317 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:05 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:05.988+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:07:06.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:05 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:05.988+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:07:06.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:05 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:05.988+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:07:06.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:05 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:05.988+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:07:06.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:05 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:05.988+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.969164+0000 front 2024-04-26T02:06:27.968989+0000 (oldest deadline 2024-04-26T02:06:53.868764+0000) 2024-04-26T02:07:06.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:05 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:05.988+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-26T02:06:38.570232+0000 front 2024-04-26T02:06:38.570334+0000 (oldest deadline 2024-04-26T02:07:04.970458+0000) 2024-04-26T02:07:06.746 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.10.service' 2024-04-26T02:07:06.784 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:07:06.784 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-26T02:07:06.784 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-26T02:07:06.784 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.11 2024-04-26T02:07:07.068 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:06 smithi173 systemd[1]: Stopping Ceph osd.11 for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:07:07.068 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:06 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:06.997+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:07:07.068 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:06 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:06.997+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:07:07.068 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:06 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:06.997+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:07:07.068 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:06 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:06.997+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:07:07.068 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:06 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:06.997+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:07:07.068 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:06 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:06.997+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.969164+0000 front 2024-04-26T02:06:27.968989+0000 (oldest deadline 2024-04-26T02:06:53.868764+0000) 2024-04-26T02:07:07.068 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:06 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:06.997+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-26T02:06:38.570232+0000 front 2024-04-26T02:06:38.570334+0000 (oldest deadline 2024-04-26T02:07:04.970458+0000) 2024-04-26T02:07:07.567 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:07 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:07.099+0000 7f741c33a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:07:07.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:07 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:07.099+0000 7f741c33a700 -1 osd.11 112 *** Got signal Terminated *** 2024-04-26T02:07:07.568 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:07 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:07.099+0000 7f741c33a700 -1 osd.11 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:07:08.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:07 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:07.987+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:07:08.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:07 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:07.987+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:07:08.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:07 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:07.987+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:07:08.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:07 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:07.987+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:07:08.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:07 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:07.987+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:07:08.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:07 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:07.987+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.969164+0000 front 2024-04-26T02:06:27.968989+0000 (oldest deadline 2024-04-26T02:06:53.868764+0000) 2024-04-26T02:07:08.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:07 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:07.987+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-26T02:06:38.570232+0000 front 2024-04-26T02:06:38.570334+0000 (oldest deadline 2024-04-26T02:07:04.970458+0000) 2024-04-26T02:07:09.317 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:08 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:08.957+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:07:09.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:08 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:08.957+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:07:09.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:08 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:08.957+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:07:09.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:08 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:08.957+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:07:09.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:08 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:08.957+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:07:09.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:08 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:08.957+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.969164+0000 front 2024-04-26T02:06:27.968989+0000 (oldest deadline 2024-04-26T02:06:53.868764+0000) 2024-04-26T02:07:09.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:08 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:08.957+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-26T02:06:38.570232+0000 front 2024-04-26T02:06:38.570334+0000 (oldest deadline 2024-04-26T02:07:04.970458+0000) 2024-04-26T02:07:10.317 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:09 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:09.923+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:07:10.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:09 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:09.923+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:07:10.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:09 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:09.923+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:07:10.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:09 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:09.923+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:07:10.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:09 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:09.923+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:07:10.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:09 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:09.923+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.969164+0000 front 2024-04-26T02:06:27.968989+0000 (oldest deadline 2024-04-26T02:06:53.868764+0000) 2024-04-26T02:07:10.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:09 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:09.923+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-26T02:06:38.570232+0000 front 2024-04-26T02:06:38.570334+0000 (oldest deadline 2024-04-26T02:07:04.970458+0000) 2024-04-26T02:07:11.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:10 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:10.944+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:07:11.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:10 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:10.944+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:07:11.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:10 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:10.944+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:07:11.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:10 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:10.944+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:07:11.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:10 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:10.944+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:07:11.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:10 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:10.944+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.969164+0000 front 2024-04-26T02:06:27.968989+0000 (oldest deadline 2024-04-26T02:06:53.868764+0000) 2024-04-26T02:07:11.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:10 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:10.944+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-26T02:06:38.570232+0000 front 2024-04-26T02:06:38.570334+0000 (oldest deadline 2024-04-26T02:07:04.970458+0000) 2024-04-26T02:07:11.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:10 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:10.944+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6820 osd.8 since back 2024-04-26T02:06:44.970935+0000 front 2024-04-26T02:06:44.970974+0000 (oldest deadline 2024-04-26T02:07:10.270954+0000) 2024-04-26T02:07:12.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:11 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:11.955+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-04-26T02:05:35.054572+0000 front 2024-04-26T02:05:35.054497+0000 (oldest deadline 2024-04-26T02:05:59.654950+0000) 2024-04-26T02:07:12.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:11 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:11.955+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-26T02:05:46.656988+0000 front 2024-04-26T02:05:46.657000+0000 (oldest deadline 2024-04-26T02:06:10.756973+0000) 2024-04-26T02:07:12.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:11 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:11.955+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2024-04-26T02:05:50.757402+0000 front 2024-04-26T02:05:50.757389+0000 (oldest deadline 2024-04-26T02:06:16.657558+0000) 2024-04-26T02:07:12.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:11 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:11.955+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-26T02:06:09.966112+0000 front 2024-04-26T02:06:09.966019+0000 (oldest deadline 2024-04-26T02:06:36.966531+0000) 2024-04-26T02:07:12.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:11 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:11.955+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2024-04-26T02:06:19.167740+0000 front 2024-04-26T02:06:19.167869+0000 (oldest deadline 2024-04-26T02:06:42.667824+0000) 2024-04-26T02:07:12.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:11 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:11.955+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-04-26T02:06:27.969164+0000 front 2024-04-26T02:06:27.968989+0000 (oldest deadline 2024-04-26T02:06:53.868764+0000) 2024-04-26T02:07:12.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:11 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:11.955+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-26T02:06:38.570232+0000 front 2024-04-26T02:06:38.570334+0000 (oldest deadline 2024-04-26T02:07:04.970458+0000) 2024-04-26T02:07:12.318 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:11 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11[57852]: 2024-04-26T02:07:11.956+0000 7f741892e700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.173:6820 osd.8 since back 2024-04-26T02:06:44.970935+0000 front 2024-04-26T02:06:44.970974+0000 (oldest deadline 2024-04-26T02:07:10.270954+0000) 2024-04-26T02:07:12.798 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:12 smithi173 podman[71444]: 2024-04-26 02:07:12.53935497 +0000 UTC m=+5.556849803 container died da2d77cd08809e171a8d7e42c4aabeaeb35e1cbe394fa1f218ff7642ab9be027 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-26T02:07:13.067 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:12 smithi173 podman[71444]: 2024-04-26 02:07:12.891143346 +0000 UTC m=+5.908638176 container cleanup da2d77cd08809e171a8d7e42c4aabeaeb35e1cbe394fa1f218ff7642ab9be027 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T02:07:13.068 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:12 smithi173 bash[71444]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11 2024-04-26T02:07:13.425 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:13 smithi173 podman[71461]: 2024-04-26 02:07:13.142539992 +0000 UTC m=+0.600322921 container remove da2d77cd08809e171a8d7e42c4aabeaeb35e1cbe394fa1f218ff7642ab9be027 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-26T02:07:13.694 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:13 smithi173 podman[71550]: 2024-04-26 02:07:13.427028738 +0000 UTC m=+0.087312141 container create a77a42b4bcb06e2c2a2e162096586b01e7dec0bf38e634b4fa25b20ae7a9dda8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:07:13.694 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:13 smithi173 podman[71550]: 2024-04-26 02:07:13.357550925 +0000 UTC m=+0.017834359 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:07:13.694 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:13 smithi173 podman[71550]: 2024-04-26 02:07:13.621001949 +0000 UTC m=+0.281285360 container init a77a42b4bcb06e2c2a2e162096586b01e7dec0bf38e634b4fa25b20ae7a9dda8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-26T02:07:13.694 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:13 smithi173 podman[71550]: 2024-04-26 02:07:13.624002634 +0000 UTC m=+0.284286037 container start a77a42b4bcb06e2c2a2e162096586b01e7dec0bf38e634b4fa25b20ae7a9dda8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-26T02:07:13.694 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:13 smithi173 podman[71550]: 2024-04-26 02:07:13.695880318 +0000 UTC m=+0.356163739 container attach a77a42b4bcb06e2c2a2e162096586b01e7dec0bf38e634b4fa25b20ae7a9dda8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:07:14.068 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:13 smithi173 podman[71550]: 2024-04-26 02:07:13.917726169 +0000 UTC m=+0.578009584 container died a77a42b4bcb06e2c2a2e162096586b01e7dec0bf38e634b4fa25b20ae7a9dda8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:07:14.446 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.11.service' 2024-04-26T02:07:14.467 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:14 smithi173 podman[71576]: 2024-04-26 02:07:14.424117802 +0000 UTC m=+0.497287073 container remove a77a42b4bcb06e2c2a2e162096586b01e7dec0bf38e634b4fa25b20ae7a9dda8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:07:14.468 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:14 smithi173 systemd[1]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.11.service: Deactivated successfully. 2024-04-26T02:07:14.468 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:14 smithi173 systemd[1]: Stopped Ceph osd.11 for 77cfcc80-036d-11ef-bc93-c7b262605968. 2024-04-26T02:07:14.468 INFO:journalctl@ceph.osd.11.smithi173.stdout:Apr 26 02:07:14 smithi173 systemd[1]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968@osd.11.service: Consumed 51.462s CPU time. 2024-04-26T02:07:14.482 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:07:14.482 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-26T02:07:14.482 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-26T02:07:14.483 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.a 2024-04-26T02:07:14.880 INFO:journalctl@ceph.mds.a.smithi001.stdout:Apr 26 02:07:14 smithi001 systemd[1]: Stopping Ceph mds.a for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:07:15.381 INFO:journalctl@ceph.mds.a.smithi001.stdout:Apr 26 02:07:15 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-a[67363]: 2024-04-26T02:07:15.027+0000 7f6bac600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:07:15.381 INFO:journalctl@ceph.mds.a.smithi001.stdout:Apr 26 02:07:15 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-a[67363]: 2024-04-26T02:07:15.027+0000 7f6bac600700 -1 mds.a *** got signal Terminated *** 2024-04-26T02:07:19.130 INFO:journalctl@ceph.mds.a.smithi001.stdout:Apr 26 02:07:18 smithi001 podman[119842]: 2024-04-26 02:07:18.827851986 +0000 UTC m=+3.938125521 container died 61178bdc637aac0b17dd7973c13f6356d89789bf9e2dbeb9b1d0664c4a284573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-04-26T02:07:19.586 INFO:journalctl@ceph.mds.a.smithi001.stdout:Apr 26 02:07:19 smithi001 podman[119842]: 2024-04-26 02:07:19.360800816 +0000 UTC m=+4.471074336 container cleanup 61178bdc637aac0b17dd7973c13f6356d89789bf9e2dbeb9b1d0664c4a284573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-26T02:07:19.586 INFO:journalctl@ceph.mds.a.smithi001.stdout:Apr 26 02:07:19 smithi001 bash[119842]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-a 2024-04-26T02:07:19.880 INFO:journalctl@ceph.mds.a.smithi001.stdout:Apr 26 02:07:19 smithi001 podman[119856]: 2024-04-26 02:07:19.587121936 +0000 UTC m=+0.737529559 container remove 61178bdc637aac0b17dd7973c13f6356d89789bf9e2dbeb9b1d0664c4a284573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-26T02:07:20.121 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.a.service' 2024-04-26T02:07:20.323 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:07:20.323 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-26T02:07:20.323 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-26T02:07:20.324 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.d 2024-04-26T02:07:20.880 INFO:journalctl@ceph.mds.d.smithi001.stdout:Apr 26 02:07:20 smithi001 systemd[1]: Stopping Ceph mds.d for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:07:21.380 INFO:journalctl@ceph.mds.d.smithi001.stdout:Apr 26 02:07:20 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-d[66139]: 2024-04-26T02:07:20.963+0000 7f0dee200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:07:21.380 INFO:journalctl@ceph.mds.d.smithi001.stdout:Apr 26 02:07:20 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-d[66139]: 2024-04-26T02:07:20.963+0000 7f0dee200700 -1 mds.d *** got signal Terminated *** 2024-04-26T02:07:23.101 INFO:journalctl@ceph.mds.d.smithi001.stdout:Apr 26 02:07:22 smithi001 podman[119963]: 2024-04-26 02:07:22.833558988 +0000 UTC m=+2.029481421 container died 49c38e895f49913864012419e786531b53e2f3fef010703f782157ba65c4c2b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-26T02:07:23.380 INFO:journalctl@ceph.mds.d.smithi001.stdout:Apr 26 02:07:23 smithi001 podman[119963]: 2024-04-26 02:07:23.223130166 +0000 UTC m=+2.419052618 container cleanup 49c38e895f49913864012419e786531b53e2f3fef010703f782157ba65c4c2b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:07:23.380 INFO:journalctl@ceph.mds.d.smithi001.stdout:Apr 26 02:07:23 smithi001 bash[119963]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-d 2024-04-26T02:07:23.880 INFO:journalctl@ceph.mds.d.smithi001.stdout:Apr 26 02:07:23 smithi001 podman[119975]: 2024-04-26 02:07:23.415585652 +0000 UTC m=+0.564887942 container remove 49c38e895f49913864012419e786531b53e2f3fef010703f782157ba65c4c2b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-d, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:07:23.984 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.d.service' 2024-04-26T02:07:24.095 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:07:24.095 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-26T02:07:24.096 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-26T02:07:24.096 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.g 2024-04-26T02:07:24.630 INFO:journalctl@ceph.mds.g.smithi001.stdout:Apr 26 02:07:24 smithi001 systemd[1]: Stopping Ceph mds.g for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:07:25.130 INFO:journalctl@ceph.mds.g.smithi001.stdout:Apr 26 02:07:24 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-g[66956]: 2024-04-26T02:07:24.743+0000 7f3733600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:07:25.131 INFO:journalctl@ceph.mds.g.smithi001.stdout:Apr 26 02:07:24 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-g[66956]: 2024-04-26T02:07:24.743+0000 7f3733600700 -1 mds.g *** got signal Terminated *** 2024-04-26T02:07:26.629 INFO:journalctl@ceph.mds.g.smithi001.stdout:Apr 26 02:07:26 smithi001 podman[120082]: 2024-04-26 02:07:26.374490443 +0000 UTC m=+1.759444145 container died 01bfdcf9c32e04137a64b82843e5ffdfab8b2f05f55a3312ed79c6acf7e83aa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-26T02:07:26.880 INFO:journalctl@ceph.mds.g.smithi001.stdout:Apr 26 02:07:26 smithi001 podman[120082]: 2024-04-26 02:07:26.743351151 +0000 UTC m=+2.128304871 container cleanup 01bfdcf9c32e04137a64b82843e5ffdfab8b2f05f55a3312ed79c6acf7e83aa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:07:26.880 INFO:journalctl@ceph.mds.g.smithi001.stdout:Apr 26 02:07:26 smithi001 bash[120082]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-g 2024-04-26T02:07:27.322 INFO:journalctl@ceph.mds.g.smithi001.stdout:Apr 26 02:07:26 smithi001 podman[120096]: 2024-04-26 02:07:26.89760183 +0000 UTC m=+0.502745558 container remove 01bfdcf9c32e04137a64b82843e5ffdfab8b2f05f55a3312ed79c6acf7e83aa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-g, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-26T02:07:27.356 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.g.service' 2024-04-26T02:07:27.470 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:07:27.470 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-26T02:07:27.470 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-26T02:07:27.470 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.j 2024-04-26T02:07:28.035 INFO:journalctl@ceph.mds.j.smithi001.stdout:Apr 26 02:07:27 smithi001 systemd[1]: Stopping Ceph mds.j for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:07:28.380 INFO:journalctl@ceph.mds.j.smithi001.stdout:Apr 26 02:07:28 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-j[65723]: 2024-04-26T02:07:28.035+0000 7fcef4200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:07:28.380 INFO:journalctl@ceph.mds.j.smithi001.stdout:Apr 26 02:07:28 smithi001 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-j[65723]: 2024-04-26T02:07:28.035+0000 7fcef4200700 -1 mds.j *** got signal Terminated *** 2024-04-26T02:07:30.261 INFO:journalctl@ceph.mds.j.smithi001.stdout:Apr 26 02:07:30 smithi001 podman[120202]: 2024-04-26 02:07:30.00856309 +0000 UTC m=+2.086804379 container died caccc4506c76456b2ec2cbd0a9968e42a1e3883bb0949749343dd355b715bf67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-j, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-26T02:07:30.574 INFO:journalctl@ceph.mds.j.smithi001.stdout:Apr 26 02:07:30 smithi001 podman[120202]: 2024-04-26 02:07:30.405193575 +0000 UTC m=+2.483434860 container cleanup caccc4506c76456b2ec2cbd0a9968e42a1e3883bb0949749343dd355b715bf67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-26T02:07:30.574 INFO:journalctl@ceph.mds.j.smithi001.stdout:Apr 26 02:07:30 smithi001 bash[120202]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-j 2024-04-26T02:07:30.880 INFO:journalctl@ceph.mds.j.smithi001.stdout:Apr 26 02:07:30 smithi001 podman[120225]: 2024-04-26 02:07:30.572699894 +0000 UTC m=+0.549303205 container remove caccc4506c76456b2ec2cbd0a9968e42a1e3883bb0949749343dd355b715bf67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-j, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:07:31.074 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.j.service' 2024-04-26T02:07:31.187 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:07:31.187 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-26T02:07:31.187 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-26T02:07:31.187 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.b 2024-04-26T02:07:31.484 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 02:07:31 smithi097 systemd[1]: Stopping Ceph mds.b for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:07:31.984 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 02:07:31 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-b[62484]: 2024-04-26T02:07:31.507+0000 7f07fba41700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:07:31.984 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 02:07:31 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-b[62484]: 2024-04-26T02:07:31.507+0000 7f07fba41700 -1 mds.b *** got signal Terminated *** 2024-04-26T02:07:35.182 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 02:07:34 smithi097 podman[70105]: 2024-04-26 02:07:34.911912788 +0000 UTC m=+3.519010411 container died 1ded51ec221f25975ae82448e9d2a8a49932017381671665f31e97f4780d2489 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:07:35.484 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 02:07:35 smithi097 podman[70105]: 2024-04-26 02:07:35.282885572 +0000 UTC m=+3.889983203 container cleanup 1ded51ec221f25975ae82448e9d2a8a49932017381671665f31e97f4780d2489 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-26T02:07:35.484 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 26 02:07:35 smithi097 bash[70105]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-b 2024-04-26T02:07:35.715 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.b.service' 2024-04-26T02:07:35.750 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:07:35.750 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-26T02:07:35.750 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-26T02:07:35.751 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.e 2024-04-26T02:07:36.234 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 26 02:07:35 smithi097 systemd[1]: Stopping Ceph mds.e for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:07:36.234 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 26 02:07:36 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-e[61866]: 2024-04-26T02:07:36.086+0000 7f8dedb5e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:07:36.234 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 26 02:07:36 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-e[61866]: 2024-04-26T02:07:36.086+0000 7f8dedb5e700 -1 mds.e *** got signal Terminated *** 2024-04-26T02:07:39.177 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 26 02:07:38 smithi097 podman[70236]: 2024-04-26 02:07:38.903921935 +0000 UTC m=+2.929220197 container died 1ae8bb4ee485c3913513c06db8e8f45b4a90065e03da8c35ba51878ef6f15b3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-e, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-26T02:07:39.484 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 26 02:07:39 smithi097 podman[70236]: 2024-04-26 02:07:39.261582033 +0000 UTC m=+3.286880291 container cleanup 1ae8bb4ee485c3913513c06db8e8f45b4a90065e03da8c35ba51878ef6f15b3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-26T02:07:39.484 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 26 02:07:39 smithi097 bash[70236]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-e 2024-04-26T02:07:39.686 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.e.service' 2024-04-26T02:07:39.717 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:07:39.717 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-26T02:07:39.718 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-26T02:07:39.718 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.h 2024-04-26T02:07:39.984 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 26 02:07:39 smithi097 systemd[1]: Stopping Ceph mds.h for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:07:40.484 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 26 02:07:40 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-h[62178]: 2024-04-26T02:07:40.039+0000 7f17cb72c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:07:40.484 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 26 02:07:40 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-h[62178]: 2024-04-26T02:07:40.039+0000 7f17cb72c700 -1 mds.h *** got signal Terminated *** 2024-04-26T02:07:43.156 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 26 02:07:42 smithi097 podman[70357]: 2024-04-26 02:07:42.896970558 +0000 UTC m=+2.968751497 container died 529af91b6a60ae00c159c33e66feb891349bb7e71cf025f0023208d0355aacb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-26T02:07:43.484 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 26 02:07:43 smithi097 podman[70357]: 2024-04-26 02:07:43.25686018 +0000 UTC m=+3.328641114 container cleanup 529af91b6a60ae00c159c33e66feb891349bb7e71cf025f0023208d0355aacb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-26T02:07:43.484 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 26 02:07:43 smithi097 bash[70357]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-h 2024-04-26T02:07:43.682 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.h.service' 2024-04-26T02:07:43.713 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:07:43.713 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-26T02:07:43.713 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-26T02:07:43.713 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.k 2024-04-26T02:07:43.984 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 26 02:07:43 smithi097 systemd[1]: Stopping Ceph mds.k for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:07:44.484 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 26 02:07:44 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-k[61548]: 2024-04-26T02:07:44.026+0000 7fcc08f5f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:07:44.484 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 26 02:07:44 smithi097 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-k[61548]: 2024-04-26T02:07:44.026+0000 7fcc08f5f700 -1 mds.k *** got signal Terminated *** 2024-04-26T02:07:48.326 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 26 02:07:48 smithi097 podman[70481]: 2024-04-26 02:07:48.034724253 +0000 UTC m=+4.110860275 container died 4fb93c802074b43b52277bd6a8b5ac10c84706ef22d1ffe6c209a81e0d83e797 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T02:07:48.651 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 26 02:07:48 smithi097 podman[70481]: 2024-04-26 02:07:48.410762621 +0000 UTC m=+4.486898642 container cleanup 4fb93c802074b43b52277bd6a8b5ac10c84706ef22d1ffe6c209a81e0d83e797 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:07:48.651 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 26 02:07:48 smithi097 bash[70481]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-k 2024-04-26T02:07:48.792 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.k.service' 2024-04-26T02:07:48.822 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:07:48.822 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-26T02:07:48.822 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-26T02:07:48.822 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.c 2024-04-26T02:07:49.142 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 26 02:07:48 smithi173 systemd[1]: Stopping Ceph mds.c for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:07:49.495 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 26 02:07:49 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-c[63123]: 2024-04-26T02:07:49.139+0000 7fcf9482f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:07:49.495 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 26 02:07:49 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-c[63123]: 2024-04-26T02:07:49.139+0000 7fcf9482f700 -1 mds.c *** got signal Terminated *** 2024-04-26T02:07:51.214 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 26 02:07:50 smithi173 podman[71655]: 2024-04-26 02:07:50.89394713 +0000 UTC m=+1.865440136 container died e9cd24ca2a36b5a3e11f2b9fc295fbcb2f294a06efc3a11c7418a9934b63f049 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-26T02:07:51.214 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 26 02:07:51 smithi173 podman[71655]: 2024-04-26 02:07:51.215621615 +0000 UTC m=+2.187114627 container cleanup e9cd24ca2a36b5a3e11f2b9fc295fbcb2f294a06efc3a11c7418a9934b63f049 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:07:51.567 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 26 02:07:51 smithi173 bash[71655]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-c 2024-04-26T02:07:51.567 INFO:journalctl@ceph.mds.c.smithi173.stdout:Apr 26 02:07:51 smithi173 podman[71680]: 2024-04-26 02:07:51.408517242 +0000 UTC m=+0.510701721 container remove e9cd24ca2a36b5a3e11f2b9fc295fbcb2f294a06efc3a11c7418a9934b63f049 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T02:07:51.638 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.c.service' 2024-04-26T02:07:51.672 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:07:51.672 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-26T02:07:51.673 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-26T02:07:51.673 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.f 2024-04-26T02:07:51.976 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 02:07:51 smithi173 systemd[1]: Stopping Ceph mds.f for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:07:52.317 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 02:07:51 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[67264]: 2024-04-26T02:07:51.977+0000 7f08eed1c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:07:52.317 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 02:07:51 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f[67264]: 2024-04-26T02:07:51.977+0000 7f08eed1c700 -1 mds.f *** got signal Terminated *** 2024-04-26T02:07:55.209 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 02:07:54 smithi173 podman[71786]: 2024-04-26 02:07:54.896804561 +0000 UTC m=+3.008532295 container died 9b8b0befaba88f43fddc553b78bf0af47e9db628f09c22f73cbec527f02e2ba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T02:07:55.209 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 02:07:55 smithi173 podman[71786]: 2024-04-26 02:07:55.211017384 +0000 UTC m=+3.322745119 container cleanup 9b8b0befaba88f43fddc553b78bf0af47e9db628f09c22f73cbec527f02e2ba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T02:07:55.554 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 02:07:55 smithi173 bash[71786]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f 2024-04-26T02:07:55.554 INFO:journalctl@ceph.mds.f.smithi173.stdout:Apr 26 02:07:55 smithi173 podman[71800]: 2024-04-26 02:07:55.431461591 +0000 UTC m=+0.528060300 container remove 9b8b0befaba88f43fddc553b78bf0af47e9db628f09c22f73cbec527f02e2ba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:07:55.568 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.f.service' 2024-04-26T02:07:55.601 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:07:55.601 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-26T02:07:55.601 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-26T02:07:55.601 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.i 2024-04-26T02:07:55.817 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 26 02:07:55 smithi173 systemd[1]: Stopping Ceph mds.i for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:07:56.318 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 26 02:07:55 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-i[63434]: 2024-04-26T02:07:55.916+0000 7fada2db0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:07:56.318 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 26 02:07:55 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-i[63434]: 2024-04-26T02:07:55.916+0000 7fada2db0700 -1 mds.i *** got signal Terminated *** 2024-04-26T02:07:59.165 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 26 02:07:58 smithi173 podman[71905]: 2024-04-26 02:07:58.899608534 +0000 UTC m=+3.082884952 container died 3fcc96c7635733abd0d7cf43945a7ac6328d4f4254664dc508050d9af513ac27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-26T02:07:59.508 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 26 02:07:59 smithi173 podman[71905]: 2024-04-26 02:07:59.250173721 +0000 UTC m=+3.433450145 container cleanup 3fcc96c7635733abd0d7cf43945a7ac6328d4f4254664dc508050d9af513ac27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:07:59.508 INFO:journalctl@ceph.mds.i.smithi173.stdout:Apr 26 02:07:59 smithi173 bash[71905]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-i 2024-04-26T02:07:59.683 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.i.service' 2024-04-26T02:07:59.714 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:07:59.714 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-26T02:07:59.714 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-26T02:07:59.714 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.l 2024-04-26T02:08:00.067 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 26 02:07:59 smithi173 systemd[1]: Stopping Ceph mds.l for 77cfcc80-036d-11ef-bc93-c7b262605968... 2024-04-26T02:08:00.068 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 26 02:08:00 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-l[62494]: 2024-04-26T02:08:00.020+0000 7ff628653700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:08:00.068 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 26 02:08:00 smithi173 ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-l[62494]: 2024-04-26T02:08:00.020+0000 7ff628653700 -1 mds.l *** got signal Terminated *** 2024-04-26T02:08:02.302 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 26 02:08:01 smithi173 podman[72026]: 2024-04-26 02:08:01.980063102 +0000 UTC m=+2.051180717 container died 4d65dc01c89eb0fb95c08fb97efdbdc23fac9402ff42a2492d0ee1333cec3d0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T02:08:02.302 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 26 02:08:02 smithi173 podman[72026]: 2024-04-26 02:08:02.303839113 +0000 UTC m=+2.374956737 container cleanup 4d65dc01c89eb0fb95c08fb97efdbdc23fac9402ff42a2492d0ee1333cec3d0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-26T02:08:02.567 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 26 02:08:02 smithi173 bash[72026]: ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-l 2024-04-26T02:08:02.567 INFO:journalctl@ceph.mds.l.smithi173.stdout:Apr 26 02:08:02 smithi173 podman[72040]: 2024-04-26 02:08:02.463239646 +0000 UTC m=+0.480735870 container remove 4d65dc01c89eb0fb95c08fb97efdbdc23fac9402ff42a2492d0ee1333cec3d0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-77cfcc80-036d-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-26T02:08:02.669 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-77cfcc80-036d-11ef-bc93-c7b262605968@mds.l.service' 2024-04-26T02:08:02.699 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:08:02.699 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-26T02:08:02.699 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T02:08:03.035 INFO:teuthology.orchestra.run.smithi001.stdout:Deleting cluster with fsid: 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T02:08:08.166 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T02:08:08.354 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T02:08:10.817 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T02:08:11.006 INFO:teuthology.orchestra.run.smithi173.stdout:Deleting cluster with fsid: 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T02:08:13.305 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T02:08:13.400 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T02:08:13.432 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T02:08:13.458 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-26T02:08:13.459 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674252/remote/smithi001/crash 2024-04-26T02:08:13.460 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/crash -- . 2024-04-26T02:08:13.564 INFO:teuthology.orchestra.run.smithi001.stderr:tar: /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T02:08:13.564 INFO:teuthology.orchestra.run.smithi001.stderr:tar: Error is not recoverable: exiting now 2024-04-26T02:08:13.568 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674252/remote/smithi097/crash 2024-04-26T02:08:13.569 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/crash -- . 2024-04-26T02:08:13.666 INFO:teuthology.orchestra.run.smithi097.stderr:tar: /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T02:08:13.666 INFO:teuthology.orchestra.run.smithi097.stderr:tar: Error is not recoverable: exiting now 2024-04-26T02:08:13.668 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674252/remote/smithi173/crash 2024-04-26T02:08:13.669 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/crash -- . 2024-04-26T02:08:13.707 INFO:teuthology.orchestra.run.smithi173.stderr:tar: /var/lib/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T02:08:13.707 INFO:teuthology.orchestra.run.smithi173.stderr:tar: Error is not recoverable: exiting now 2024-04-26T02:08:13.708 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-26T02:08:13.708 DEBUG:teuthology.orchestra.run.smithi001:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-26T02:08:13.828 INFO:tasks.cephadm:Compressing logs... 2024-04-26T02:08:13.828 DEBUG:teuthology.orchestra.run.smithi001:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T02:08:13.833 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T02:08:13.835 DEBUG:teuthology.orchestra.run.smithi173:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T02:08:13.860 INFO:teuthology.orchestra.run.smithi173.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T02:08:13.860 INFO:teuthology.orchestra.run.smithi173.stderr:: No such file or directory 2024-04-26T02:08:13.860 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.log 2024-04-26T02:08:13.860 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-26T02:08:13.871 INFO:teuthology.orchestra.run.smithi097.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T02:08:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:: No such file or directory 2024-04-26T02:08:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.log 2024-04-26T02:08:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T02:08:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-26T02:08:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-26T02:08:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-26T02:08:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.7.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T02:08:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-26T02:08:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-26T02:08:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.1.log: /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mgr.y.log: 90.3% 87.3% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.log.gz -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T02:08:13.872 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-04-26T02:08:13.872 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-26T02:08:13.873 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-26T02:08:13.873 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T02:08:13.873 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-26T02:08:13.873 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.6% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T02:08:13.873 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T02:08:13.875 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.e.log: 92.1% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-26T02:08:13.876 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-26T02:08:13.876 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T02:08:13.876 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-26T02:08:13.877 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.f.log: /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-26T02:08:13.877 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T02:08:13.877 INFO:teuthology.orchestra.run.smithi173.stderr: --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-26T02:08:13.877 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-26T02:08:13.877 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-26T02:08:13.877 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.3% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T02:08:13.877 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T02:08:13.877 INFO:teuthology.orchestra.run.smithi173.stderr: 90.4% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T02:08:13.877 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-26T02:08:13.877 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.6% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T02:08:13.877 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-26T02:08:13.877 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.11.log: 92.2% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-26T02:08:13.877 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T02:08:13.881 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-26T02:08:13.887 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-26T02:08:13.889 INFO:teuthology.orchestra.run.smithi001.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-26T02:08:13.891 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T02:08:13.892 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.log 2024-04-26T02:08:13.893 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T02:08:13.894 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-26T02:08:13.894 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-26T02:08:13.895 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T02:08:13.896 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-26T02:08:13.897 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.g.log: 87.3% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T02:08:13.897 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-26T02:08:13.899 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.j.log: 90.2% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T02:08:13.899 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-26T02:08:13.900 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-26T02:08:13.901 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-26T02:08:13.903 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-26T02:08:13.905 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mon.a.log: 93.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T02:08:13.905 INFO:teuthology.orchestra.run.smithi001.stderr: 94.4% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-26T02:08:13.905 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T02:08:13.905 INFO:teuthology.orchestra.run.smithi001.stderr: 94.4% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-26T02:08:13.906 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-26T02:08:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T02:08:13.908 INFO:teuthology.orchestra.run.smithi001.stderr: 84.0% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T02:08:13.911 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.a.log: /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-26T02:08:13.913 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-26T02:08:13.913 INFO:teuthology.orchestra.run.smithi001.stderr: 94.3% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-26T02:08:13.914 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T02:08:13.917 INFO:teuthology.orchestra.run.smithi173.stderr: 94.4% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-26T02:08:13.918 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.k.log: 94.4% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-26T02:08:13.921 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.d.log: /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.3% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-26T02:08:13.927 INFO:teuthology.orchestra.run.smithi001.stderr: 90.9% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T02:08:13.948 INFO:teuthology.orchestra.run.smithi097.stderr: 94.3% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-26T02:08:13.955 INFO:teuthology.orchestra.run.smithi097.stderr: 94.2% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-26T02:08:14.360 INFO:teuthology.orchestra.run.smithi173.stderr: 94.2% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-26T02:08:14.375 INFO:teuthology.orchestra.run.smithi173.stderr: 94.2% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-26T02:08:14.377 INFO:teuthology.orchestra.run.smithi001.stderr: 90.1% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-26T02:08:14.435 INFO:teuthology.orchestra.run.smithi173.stderr: 91.7% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-26T02:08:14.564 INFO:teuthology.orchestra.run.smithi097.stderr: 91.6% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-26T02:08:15.436 INFO:teuthology.orchestra.run.smithi001.stderr: 90.2% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-26T02:08:15.844 INFO:teuthology.orchestra.run.smithi001.stderr: 93.6% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-26T02:08:16.228 INFO:teuthology.orchestra.run.smithi173.stderr: 95.7% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-26T02:08:17.565 INFO:teuthology.orchestra.run.smithi097.stderr: 94.1% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-26T02:08:18.061 INFO:teuthology.orchestra.run.smithi001.stderr: 93.6% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-26T02:08:18.182 INFO:teuthology.orchestra.run.smithi097.stderr: 94.2% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-26T02:08:18.565 INFO:teuthology.orchestra.run.smithi097.stderr: 93.8% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-26T02:08:18.874 INFO:teuthology.orchestra.run.smithi001.stderr: 93.6% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-26T02:08:19.041 INFO:teuthology.orchestra.run.smithi173.stderr: 94.0% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-26T02:08:19.175 INFO:teuthology.orchestra.run.smithi173.stderr: 94.0% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-26T02:08:19.292 INFO:teuthology.orchestra.run.smithi173.stderr: 94.0% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-26T02:08:19.422 INFO:teuthology.orchestra.run.smithi097.stderr: 93.8% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-26T02:08:19.841 INFO:teuthology.orchestra.run.smithi173.stderr: 93.9% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-26T02:08:19.843 INFO:teuthology.orchestra.run.smithi173.stderr: 2024-04-26T02:08:19.843 INFO:teuthology.orchestra.run.smithi173.stderr:real 0m5.992s 2024-04-26T02:08:19.843 INFO:teuthology.orchestra.run.smithi173.stderr:user 0m24.936s 2024-04-26T02:08:19.843 INFO:teuthology.orchestra.run.smithi173.stderr:sys 0m0.890s 2024-04-26T02:08:21.547 INFO:teuthology.orchestra.run.smithi001.stderr: 93.9% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-26T02:08:21.550 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-04-26T02:08:21.551 INFO:teuthology.orchestra.run.smithi001.stderr:real 0m7.686s 2024-04-26T02:08:21.551 INFO:teuthology.orchestra.run.smithi001.stderr:user 0m19.226s 2024-04-26T02:08:21.551 INFO:teuthology.orchestra.run.smithi001.stderr:sys 0m1.637s 2024-04-26T02:08:51.797 INFO:teuthology.orchestra.run.smithi097.stderr: 94.0% -- replaced with /var/log/ceph/77cfcc80-036d-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-26T02:08:51.798 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-04-26T02:08:51.798 INFO:teuthology.orchestra.run.smithi097.stderr:real 0m37.948s 2024-04-26T02:08:51.798 INFO:teuthology.orchestra.run.smithi097.stderr:user 0m55.158s 2024-04-26T02:08:51.798 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m1.860s 2024-04-26T02:08:51.799 INFO:tasks.cephadm:Archiving logs... 2024-04-26T02:08:51.799 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674252/remote/smithi001/log 2024-04-26T02:08:51.800 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T02:08:54.645 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674252/remote/smithi097/log 2024-04-26T02:08:54.646 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T02:09:02.293 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674252/remote/smithi173/log 2024-04-26T02:09:02.294 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T02:09:05.624 INFO:tasks.cephadm:Removing cluster... 2024-04-26T02:09:05.625 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 --force 2024-04-26T02:09:05.958 INFO:teuthology.orchestra.run.smithi001.stdout:Deleting cluster with fsid: 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T02:09:06.597 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 --force 2024-04-26T02:09:06.791 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T02:09:07.104 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 77cfcc80-036d-11ef-bc93-c7b262605968 --force 2024-04-26T02:09:07.294 INFO:teuthology.orchestra.run.smithi173.stdout:Deleting cluster with fsid: 77cfcc80-036d-11ef-bc93-c7b262605968 2024-04-26T02:09:07.581 INFO:tasks.cephadm:Removing cephadm ... 2024-04-26T02:09:07.581 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T02:09:07.642 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T02:09:07.658 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T02:09:07.674 INFO:tasks.cephadm:Teardown complete 2024-04-26T02:09:07.674 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T02:09:07.693 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T02:09:07.693 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T02:09:07.701 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T02:09:07.703 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T02:09:07.862 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T02:09:07.876 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T02:09:07.876 DEBUG:teuthology.orchestra.run.smithi001:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T02:09:07.884 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T02:09:07.887 DEBUG:teuthology.orchestra.run.smithi173:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T02:09:07.901 INFO:teuthology.orchestra.run.smithi173.stderr:bash: line 1: ntpq: command not found 2024-04-26T02:09:07.901 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-04-26T02:09:07.905 INFO:teuthology.orchestra.run.smithi173.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T02:09:07.905 INFO:teuthology.orchestra.run.smithi173.stdout:=============================================================================== 2024-04-26T02:09:07.905 INFO:teuthology.orchestra.run.smithi173.stdout:^* hv01.front.sepia.ceph.com 3 6 377 33 +1672us[+1682us] +/- 49ms 2024-04-26T02:09:07.905 INFO:teuthology.orchestra.run.smithi173.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 33 -1972us[-1963us] +/- 70ms 2024-04-26T02:09:07.905 INFO:teuthology.orchestra.run.smithi173.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 -662us[ -662us] +/- 80ms 2024-04-26T02:09:07.905 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T02:09:07.907 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T02:09:07.907 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-04-26T02:09:07.907 INFO:teuthology.orchestra.run.smithi097.stdout:^* hv01.front.sepia.ceph.com 3 6 377 33 +1665us[+1673us] +/- 49ms 2024-04-26T02:09:07.907 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 35 -1975us[-1968us] +/- 70ms 2024-04-26T02:09:07.907 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 -667us[ -659us] +/- 80ms 2024-04-26T02:09:07.907 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T02:09:07.933 INFO:teuthology.orchestra.run.smithi001.stderr:bash: line 1: ntpq: command not found 2024-04-26T02:09:07.944 INFO:teuthology.orchestra.run.smithi001.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T02:09:07.944 INFO:teuthology.orchestra.run.smithi001.stdout:=============================================================================== 2024-04-26T02:09:07.944 INFO:teuthology.orchestra.run.smithi001.stdout:^* hv01.front.sepia.ceph.com 3 6 377 31 +1702us[+1772us] +/- 50ms 2024-04-26T02:09:07.944 INFO:teuthology.orchestra.run.smithi001.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 31 -1946us[-1876us] +/- 70ms 2024-04-26T02:09:07.944 INFO:teuthology.orchestra.run.smithi001.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 34 -616us[ -546us] +/- 80ms 2024-04-26T02:09:07.944 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T02:09:07.946 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T02:09:07.956 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T02:09:07.957 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T02:09:07.970 DEBUG:teuthology.orchestra.run.smithi001:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T02:09:08.199 DEBUG:teuthology.orchestra.run.smithi097:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T02:09:08.269 DEBUG:teuthology.orchestra.run.smithi173:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T02:09:08.334 DEBUG:teuthology.orchestra.run.smithi001:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T02:09:08.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:09:08.434 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T02:09:08.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:09:08.464 DEBUG:teuthology.orchestra.run.smithi173:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T02:09:08.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:09:08.491 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T02:09:08.503 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T02:09:08.540 INFO:teuthology.task.internal:Duration was 2361.679312 seconds 2024-04-26T02:09:08.541 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T02:09:08.552 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T02:09:08.553 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T02:09:08.560 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T02:09:08.563 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T02:09:08.598 INFO:teuthology.orchestra.run.smithi173.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T02:09:08.601 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T02:09:08.707 INFO:teuthology.orchestra.run.smithi001.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T02:09:09.223 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T02:09:09.223 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi001.front.sepia.ceph.com 2024-04-26T02:09:09.224 DEBUG:teuthology.orchestra.run.smithi001:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T02:09:09.285 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2024-04-26T02:09:09.285 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T02:09:09.314 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi173.front.sepia.ceph.com 2024-04-26T02:09:09.315 DEBUG:teuthology.orchestra.run.smithi173:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T02:09:09.339 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T02:09:09.339 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T02:09:09.343 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T02:09:09.357 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T02:09:09.405 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T02:09:09.405 DEBUG:teuthology.orchestra.run.smithi001:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T02:09:09.409 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T02:09:09.426 DEBUG:teuthology.orchestra.run.smithi173:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T02:09:10.285 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T02:09:10.297 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T02:09:10.298 DEBUG:teuthology.orchestra.run.smithi001:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T02:09:10.377 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T02:09:10.401 DEBUG:teuthology.orchestra.run.smithi173:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T02:09:10.427 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T02:09:10.439 DEBUG:teuthology.orchestra.run.smithi001:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:09:10.447 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:09:10.449 DEBUG:teuthology.orchestra.run.smithi173:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:09:10.470 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2024-04-26T02:09:10.491 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = core 2024-04-26T02:09:10.540 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = core 2024-04-26T02:09:10.583 DEBUG:teuthology.orchestra.run.smithi001:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:09:10.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:09:10.633 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:09:10.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:09:10.647 DEBUG:teuthology.orchestra.run.smithi173:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:09:10.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:09:10.663 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T02:09:10.676 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T02:09:10.677 DEBUG:teuthology.misc:Transferring archived files from smithi001:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674252/remote/smithi001 2024-04-26T02:09:10.677 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T02:09:10.795 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674252/remote/smithi097 2024-04-26T02:09:10.795 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T02:09:10.829 DEBUG:teuthology.misc:Transferring archived files from smithi173:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674252/remote/smithi173 2024-04-26T02:09:10.830 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T02:09:10.866 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T02:09:10.866 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T02:09:10.874 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T02:09:10.876 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T02:09:10.927 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T02:09:10.940 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T02:09:10.941 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T02:09:10.953 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T02:09:10.953 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T02:09:10.960 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T02:09:10.963 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T02:09:10.975 INFO:teuthology.orchestra.run.smithi097.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 02:09 /home/ubuntu/cephtest 2024-04-26T02:09:10.978 INFO:teuthology.orchestra.run.smithi173.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 02:09 /home/ubuntu/cephtest 2024-04-26T02:09:11.012 INFO:teuthology.orchestra.run.smithi001.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 02:09 /home/ubuntu/cephtest 2024-04-26T02:09:11.016 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T02:09:11.030 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T02:09:11.061 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} duration: 2361.6793122291565 flavor: default owner: scheduled_leonidus@teuthology success: true 2024-04-26T02:09:11.061 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T02:09:11.142 INFO:teuthology.run:pass